Pion ReplaceTrack: why video never reaches the second SFU

Published 2026-10-09 · Pion WebRTC v4 · layered SFU debugging

A successful ReplaceTrack call proves that a local track was bound to an existing sender. It does not prove that video is allowed by the negotiated SDP, compatible with the selected codec, or arriving at the next SFU.

The misleading part of the log

In a Master SFU to Sub SFU topology, the Master can report all of the following while the Sub SFU still receives only audio:

Received remote audio track
Received remote video track
Successfully replaced sub SFU video track
Starting RTP forwarding TrackKind=video
Successfully replaced sub SFU audio track
Starting RTP forwarding TrackKind=audio

Those messages show that the application reached the expected branches. They do not show that the Sub SFU can receive the video m-line or that the payload can be decoded. I would separate the problem in this order:

1. Was the video direction negotiated correctly on both PeerConnections?
2. Did both legs negotiate a compatible video codec?
3. Are video RTP packets leaving the Master and arriving at the Sub SFU?
4. Is the Sub SFU receiving a decodable frame?

Changing the forwarding loop before answering those questions usually creates more moving parts without explaining why audio works and video does not.

First: inspect the negotiated video m-line

ReplaceTrack replaces the track behind an existing sender. Outside of a future negotiation, it does not change the SDP direction. The sender side needs that m-line to be sendonly or sendrecv. The receiving side needs recvonly or sendrecv.

m=video ...
a=mid:<same mid on both peers>
a=sendrecv
# or
a=sendonly

# On the receiving Sub SFU:
a=recvonly
# or
a=sendrecv

Setting Sendrecv only in the constructor is not the same as verifying the final answer. Compare the local and remote descriptions after the initial negotiation is stable. If the answer is recvonly or inactive, the receiver will never create an inbound video track regardless of how successful the replacement call was.

Second: force a codec both legs can use

TrackLocalStaticRTP rewrites the outgoing SSRC and payload type for each binding. It does not transcode the payload or repair codec-specific parameters.

Forwarding H.264 packets into a leg that negotiated a different H.264 profile, or writing VP8 packets into a sender that selected H.264, can produce a sender that exists but never yields a decodable frame. The mime type alone is not enough.

For a controlled Master to Sub SFU link, explicitly select one video codec on both sides and verify the final SDP. A small allowlist also prevents accidental codec drift:

videoCodecs := []webrtc.RTPCodecParameters{
    webrtc.RTPCodecParameters{
        RTPCodecCapability: webrtc.RTPCodecCapability{
            MimeType:  webrtc.MimeTypeVP8,
            ClockRate: 90000,
        },
        PayloadType: 96,
    },
}

if err := transceiver.SetCodecPreferences(videoCodecs); err != nil {
    return err
}

If the two sides must negotiate different codecs, transcode at the boundary. Do not pass the SSRC and payload type unchanged and expect the media pipeline to repair the mismatch.

Third: instrument the RTP path, not just the track creation

Logging that a local track was created does not tell you whether the first packet reached the Sub SFU. Add a counter at the write site and inspect the receiver on the other side.

var videoPackets atomic.Uint64

for {
    packet, _, err := remoteTrack.ReadRTP()
    if err != nil {
        log.Printf("upstream video read failed: %v", err)
        return
    }

    if err := localTrack.WriteRTP(packet); err != nil {
        log.Printf("sub video write failed: %v", err)
        return
    }

    videoPackets.Add(1)
    if n := videoPackets.Load(); n == 1 || n%300 == 0 {
        log.Printf("sub video packets written=%d ssrc=%d pt=%d",
            n, packet.SSRC, packet.PayloadType)
    }
}

On the Sub SFU, log the track that OnTrack receives and read the inbound video statistics:

peerConn.OnTrack(func(track *webrtc.TrackRemote, _ *webrtc.RTPReceiver) {
    log.Printf("sub track kind=%s id=%s stream=%s ssrc=%d",
        track.Kind(), track.ID(), track.StreamID(), track.SSRC())
})

for id, stat := range peerConn.GetStats() {
    if report, ok := stat.(webrtc.InboundRTPStreamStats); ok &&
        report.Kind == "video" {
        log.Printf("inbound video id=%s packets=%d bytes=%d frames=%d decoded=%d",
            id, report.PacketsReceived, report.BytesReceived,
            report.FramesReceived, report.FramesDecoded)
    }
}

Three outcomes point to different fixes:

• No inbound video stream: the problem is negotiation, codec selection or sending.
• Packets increase, frames stay at zero: RTP arrives, but the depacketizer or codec is wrong.
• Frames arrive but decoded stays at zero: request a keyframe or inspect the decoder path.

A more reliable no-renegotiation pattern

The cleanest layered design is to negotiate the video sender before the first upstream track exists. Create a placeholder TrackLocalStaticRTP, add it to the Sub SFU peer through a video transceiver, and negotiate the initial offer and answer. Once the session is stable, create the real local track from the Master stream and replace the placeholder.

The placeholder keeps the m-line present and lets you verify the final direction before media starts. It also makes the lifecycle explicit: negotiation first, media binding second.

For every Sub SFU, keep the local track and its binding separate. A single write can fan out to multiple bindings, but one failed binding should not be hidden by the success of another.

Fourth: distinguish transport from decodability

If the Sub SFU receives video packets but the viewer remains black, do not keep changing the sender. Send a picture-loss indication to the upstream publisher and confirm that the inbound statistics move from framesReceived to framesDecoded.

RTP flow and a decodable first frame are different milestones. A connection can carry packets for minutes while the decoder waits for a keyframe, especially after an encoder starts mid-GOP or a track is replaced.

The debugging order that saves the most time

1. Compare the final SDP direction on both legs.
2. Force one video codec and compare the negotiated payload type.
3. Count packets at the Master write and at the Sub SFU read.
4. Check frames received before changing the forwarding loop.
5. Request a keyframe when packets flow but frames do not decode.

The key point is simple: ReplaceTrack returning nil is a local operation. A working video path is a negotiated, codec-compatible, packet-producing path across two PeerConnections.