Bridging WhatsApp and PBX: Low-Latency SIP Trunking Internals

Connecting a WhatsApp call to a phone system sounds like it should be a solved problem. It isn't. On one side sits WhatsApp's end-to-end-encrypted voice: wideband Opus riding the app's own relay transport, with a bespoke signaling handshake and payload conventions that never appear in any RFC. On the other sits a SIP PBX — Asterisk or FreeSWITCH — that expects a plain RTP stream of 8 kHz G.711, negotiated over SDP, from a peer it can route like any trunk. Neither speaks the other's language, at either layer.

A bridge that joins them has to be two translators at once: a signaling translator that maps a SIP INVITE to a WhatsApp call and back, and a media translator that transcodes and re-frames audio in both directions, in real time, without ever letting the pipeline stall. This is a tour of how we built ours, and the decisions — several of them counter-intuitive — that a low-latency, production bridge actually turns on.

Two planes, two entirely separate problems

Every telephony bridge has to solve two problems that share almost nothing except a call ID. Conflating them is the first mistake.

The signaling plane establishes, maintains, and tears down the call. On the SIP side that's the classic dialog: INVITE → 100 Trying → 180 Ringing → 200 OK → ACK, and a BYE to hang up. On the WhatsApp side it's an offer/accept/terminate exchange over the app's own channel — a companion account drives it through the WebSocket protocol, a primary account through its raw-TCP noise transport. The bridge runs a SIP UAS (user-agent server) that answers the trunk, and a WhatsApp call client that places the outbound leg, and it has to keep the two state machines married so that a hangup on either side cleanly ends the other.

The media plane is where the audio actually flows, and it is completely independent of the signaling. WhatsApp media is SRTP-encrypted Opus, delivered through WhatsApp's relay rather than a direct peer-to-peer path you can negotiate with ICE. SIP media is unencrypted (or SDES/DTLS-SRTP) RTP carrying G.711. Bridging them is not "forwarding packets" — the codecs, sample rates, frame sizes, and encryption are all different. It is a full transcode.

Keeping these planes decoupled in the code is what let the same bridge orchestration drive two very different WhatsApp transports without change. The signaling logic reacts to exactly two events from the WhatsApp leg — peer accepted and call terminated — and everything else is media. Swap the transport underneath and the bridge doesn't notice.

The media pipeline, frame by frame

Here is where the interesting engineering lives. Consider the SIP → WhatsApp direction first, because it establishes the shape of the whole thing.

A softphone sends us 20 ms RTP packets: 160 bytes of µ-law (G.711 PCMU, payload type 0) at 8 kHz. To hand that to WhatsApp we have to reach wideband Opus at 16 kHz. Each stage matters:

1. Strip and decode. Remove the 12-byte RTP header, then expand each µ-law byte through a 256-entry lookup table into linear PCM. A subtle trap lives here: a naïve µ-law table yields 14-bit samples (±8031), but the rest of the pipeline wants 16-bit. Skip the << 2 shift and every call is silently 12 dB too quiet — audible as "the WhatsApp side can barely hear me," with nothing in the logs to explain it.
2. Resample 8 kHz → 16 kHz. Not by duplicating samples. We use a windowed-sinc FIR filter; an early pair-averaging shortcut ate the sibilants (−3 dB at 2 kHz) and made every voice sound muffled.
3. Buffer and re-frame. G.711 arrives in 20 ms frames; we encode Opus in 60 ms frames. A small drop-oldest ring absorbs the mismatch and hands the encoder exactly 960 samples at a time.
4. Encode Opus and push each packet onto a queue that the SRTP send loop drains on its own cadence, encrypts, and relays to the peer's device.

The WhatsApp → SIP direction is the mirror image, and every stage has to run in the opposite order: SRTP-decrypt the inbound Opus (AES-CTR keystream per RFC 3711), decode to 16 kHz PCM, resample down to 8 kHz through the same class of FIR filter, re-frame from Opus's variable, sometimes-multi-frame packets into steady 20 ms chunks, µ-law-encode, and write RTP to the softphone with a monotonic sequence number and a timestamp that advances by exactly 160 per packet.

None of this is exotic on its own. What makes it hard is that both directions run concurrently, per call, at scale, and any stall of more than a frame or two is instantly audible.

Why we pin Opus to SILK at 16 kbps

The single most expensive lesson in the whole project was a codec-configuration one, and it is the kind of thing no specification warns you about.

Opus is really two codecs in a trench coat: SILK for speech and CELT for music, with a hybrid mode in between, and libopus switches between them automatically based on bitrate and content. That automatic switch is a landmine here. When we let the encoder run at 24 kbps, it would spontaneously flip from SILK to CELT mid-call — you can see it in the packet's table-of-contents byte changing to 0xbb. Desktop WhatsApp coped. WhatsApp on iPhone stuttered every time the mode flipped.

The fix is to never let the switch happen. We pin the encoder to SILK, wideband, at 16 kbps with complexity 10, and DTX (discontinuous transmission) turned off. We reach that pin through the bitrate ceiling because the binding we use doesn't expose OPUS_SET_FORCE_MODE directly — a reminder that your codec configuration is only as expressive as the wrapper you call it through.

Two related traps in the same family:

• Payload types are a handshake, not a suggestion. WhatsApp carries Opus at its own dynamic payload types internally; a SIP peer negotiates Opus at whatever it advertised in SDP (commonly PT 111) and G.711 at the static PT 0. If the numbers the bridge emits don't exactly match what the far end negotiated, the stream doesn't error — it goes silently dead. Half the "the call connects but there's no audio" reports trace back to a payload-type mismatch, not a routing problem.
• DTX is a liability at a transcoding boundary. Discontinuous transmission saves bandwidth by not sending packets during silence, but a bridge that has to emit a steady RTP cadence to the PBX would then have to synthesize the gaps anyway. We turn it off and control silence ourselves — which leads directly to the next problem.

Fighting jitter without wrecking latency

The hardest audible defect we chased wasn't distortion or dropouts. It was a subtle, intermittent chopping on the WhatsApp-to-SIP leg that testers described as "bad quality" but couldn't pin down. It survived a full audit of the codec, the pacing, and the ring buffers — all clean.

The root cause was a jitter-buffer design flaw. WhatsApp delivers audio in bursts: wideband Opus packets that can each carry up to 200 ms of audio, arriving unevenly. Our paced sender drained its ring the instant it held a single frame, so the moment a burst arrived late, the sender starved and spliced flat comfort-silence into otherwise-clean speech. The audio was never corrupted; it was being interrupted.

The fix is a prebuffer. Before playout starts (or restarts after a dry spell), we hold a cushion — around 120 ms — so a late burst has somewhere to be absorbed instead of starving the sender. It costs ~120 ms of one-way latency, and it is worth every millisecond: comfort-silence injections during active speech dropped from dozens per two-second window to essentially zero.

That still left one rough edge. When WhatsApp's own DTX cut the stream at the end of an utterance, the sender fell to dead silence abruptly, which the ear hears as a hard click. We now decay the last frame's energy to zero over ~80 ms — a PLC-style fade-out — before letting silence take over. We tried filling gaps with LFSR-generated comfort noise first; it was audibly hissy and we reverted it. Fade-to-silence won.

There is a matching subtlety on the other side. When a call first connects, the bridge emits its own comfort silence to the SIP leg so the softphone hears an established, live channel before the first real WhatsApp packet arrives. The instant genuine voice shows up, a callback cancels the silence sender — otherwise two writers would fight over the same UDP socket. Getting that handoff right (distinct SSRCs, single cancellation, no double-write) is the difference between a clean call open and a burst of garbage at "hello."

The latency budget

"Low-latency" is a claim you have to be able to defend with numbers. Here is where the milliseconds actually go on the media path, steady-state:

Stage	Budget
SIP → WhatsApp transcode + encode	~40–42 ms avg (≤ 50 ms peak)
WhatsApp → SIP decode	~0–1 ms
WhatsApp → SIP paced send	≤ 20 ms by design
Jitter prebuffer (WA → SIP)	~120 ms

The transcode itself is cheap; decode is nearly free. The money is spent deliberately, in the jitter prebuffer, buying smoothness. That is the central trade of real-time voice: latency you add on purpose is the price of latency you can't predict, and a bridge that refuses to spend it produces a call that is technically faster and subjectively worse. We phase-align the SRTP send loop to the pump's cadence to claw back the tens of milliseconds that aren't buying us anything, and spend the rest where it counts.

Making Asterisk and FreeSWITCH cooperate

A transcoding bridge is only half the job; the PBX has to be told to treat it correctly. A few configuration realities repeatedly bite operators.

Media must not be renegotiated away from the bridge. On a PJSIP endpoint that means direct_media=no — otherwise Asterisk tries to shortcut the RTP path peer-to-peer and the transcode never runs. A representative endpoint:

[whatsapp-trunk]
type=endpoint
transport=transport-wss
context=whatsapp-inbound
disallow=all
allow=opus
allow=ulaw
direct_media=no
rtp_symmetric=yes
force_rport=yes
rewrite_contact=yes

The last three lines are the NAT survival kit, and they matter more than anything else on this list. A bridge behind NAT advertises a private address in its SDP (c=IN IP4 192.168.x.y); the softphone dutifully tries to send RTP there and the packets die at the first router. With rtp_symmetric=yes, Asterisk ignores the advertised address and learns the real source IP and port from the bridge's outbound RTP, replying there instead. This single flag turns a one-way call into a two-way one. (The belt-and-suspenders alternative is client-side STUN discovery so the bridge advertises its public address in the first place.)

Routing the right WhatsApp number per call is a dialplan trick rather than a code change. A pattern extension carries the destination in the dialed digits, and the bridge reads the request-URI user to pick its target at call time:

exten => _1WA#X.,1,NoOp(bridge ${CALLERID(num)} -> WhatsApp +${EXTEN:4})
 same => n,Dial(PJSIP/${EXTEN:4}@whatsapp-trunk,30,tT)
 same => n,Hangup()

Now one bridge instance serves every destination the account can reach, chosen dynamically, with no per-number configuration.

Failover, teardown, and the order of operations

The least glamorous code is the code that ends a call correctly, and it is where bridges most often leak.

Teardown order is load-bearing. When the SIP side hangs up, we stop the SRTP send loop and close the active media before we signal the WhatsApp hangup — reverse that order and you race a still-running encoder against a torn-down session, which leaves half-dead calls and orphaned goroutines. On the inbound side we run an RTP "drought" detector: if a call that has already carried media goes quiet for five seconds, the bridge proactively hangs up, which covers the case where a PBX dialplan (Asterisk's tT flags, say) leaves a zombie leg after the far party is gone. Critically, the drought detector is gated on having seen media at all, so it never kills a call that's still legitimately ringing.

At the fleet level, sessions are supervised so that when an egress worker rotates or a regional provider times out, calls are re-homed rather than dropped — the signaling and media state a call needs to survive a worker bounce is deliberately kept where another worker can pick it up. And because the whole orchestration layer is transport-agnostic, the same teardown and supervision logic covers both companion accounts (driven through the WebSocket library) and primary accounts (driven through the raw-TCP transport) without a second implementation.

What the bridge really is

Strip away the specifics and a WhatsApp-to-PBX bridge is a study in a single principle: real-time audio is a scheduling problem wearing a codec's clothes. The transcoding is textbook. The encryption is textbook. What separates a demo that works once from a trunk that carries production traffic is everything around the edges — the 12 dB you lose to a missing bit-shift, the stutter you inherit from an automatic codec switch, the chop you get from a jitter buffer with no cushion, the one-way call you get from an honest-but-useless private IP in your SDP, and the orphaned sessions you get from tearing down in the wrong order.

Get those right, spend your latency where it buys smoothness and nowhere else, and two systems that were never designed to talk to each other carry a clean, natural conversation — in both directions, at trunk scale.