skip to content

With the Cloudflare Realtime SFU API, how does a backend publish a participant's camera and let another participant pull it?

level: middleimportance: must knowfreq 30%

answer

  1. one session per PeerConnection
  2. local versus remote track entries
  3. who sends the offer each way
  4. a second call to finish pulling

basics

~20 s

Publish by calling sessions/new, then tracks/new with the browser's SDP offer and a local entry naming the track. Pull by calling tracks/new on the viewer's own session with a remote entry, then submit the browser's answer to /renegotiate.

solid answer

~40 s

Every call goes from the backend to `https://rtc.live.cloudflare.com/v1/apps/{appId}` with `Authorization: Bearer <APP_SECRET>`. To **publish**, the browser adds a `sendonly` transceiver and sets its local offer; the backend calls `POST /sessions/new`, then `POST /sessions/{sessionId}/tracks/new` with that offer and an entry `{ location: "local", mid, trackName: "camera" }`, and the browser applies the returned answer. To **pull**, the viewer gets its own session and the backend calls `tracks/new` with `{ location: "remote", sessionId: <publisher's>, trackName: "camera" }`; the SFU returns an offer, the browser answers, and the backend sends that answer to `PUT /sessions/{sessionId}/renegotiate`. The SFU announces nothing: your app shares session IDs and track names with authorized viewers.

code

json · 14 lines
json
{
  "tracks": [
    {
      "location": "remote",
      "sessionId": "<CLINICIAN_PUBLISH_SESSION_ID>",
      "trackName": "camera"
    },
    {
      "location": "remote",
      "sessionId": "<CLINICIAN_PUBLISH_SESSION_ID>",
      "trackName": "microphone"
    }
  ]
}

go deeper

for a junior

Know the nouns: app, session as one PeerConnection, track named by trackName, and that the App Secret stays on the server.

for a middle

Walk through both directions and say who sends the offer each way, and why pulling needs the extra /renegotiate call.

for a senior

Bring up per-item errors, per-session serialization, separate publish and receive sessions, and the 30-second media inactivity rule.

for a principal

Point out that rooms, discovery and permissions are your code on the SFU, which is exactly what RealtimeKit would otherwise provide.

## The model: applications, sessions and tracks Cloudflare's **Realtime SFU** forwards WebRTC media between endpoints, but it has no rooms and no presence. You get three building blocks: - An **SFU application**, with an **App ID** and **App Secret**. Separate apps keep development and production apart. - A **session**, which is one endpoint's WebRTC connection to the SFU; in a browser, one `RTCPeerConnection` maps to one `sessionId`. - A **track**, which is a published audio or video stream named by a `trackName` inside its publishing session. A subscriber locates it by the publisher's `sessionId` plus that `trackName`. Each participant uploads its camera once and the SFU forwards a copy to every session that asks for it. The general case for an SFU over a mesh is covered in the WebRTC topology lessons. What is specific here is that nothing is forwarded until a session explicitly pulls a track. ## Who calls what | Layer | Owns | | --- | --- | | Browser | Captures media, owns the `RTCPeerConnection`, creates and applies SDP | | Your backend (for example a Worker) | Authenticates users, holds the App Secret, calls the SFU API | | Application state (for example one Durable Object per visit) | Membership, which tracks exist, who may pull them | | Realtime SFU | Establishes WebRTC sessions and forwards the selected tracks | Every SFU API call carries `Authorization: Bearer <APP_SECRET>`. The secret stays on the backend: a browser holding it could open, pull or close any session in the app. ## Publishing a camera 1. In the browser, call `getUserMedia()`, add each track with `pc.addTransceiver(track, { direction: "sendonly" })`, then `createOffer()` and `setLocalDescription()`. Once ICE candidates are gathered, read each transceiver's `mid`. 2. The backend calls `POST /apps/{appId}/sessions/new` with no body and stores the `sessionId`. Doing this only when the browser is ready matters, because an unconnected session can expire. 3. The backend calls `POST /sessions/{sessionId}/tracks/new` with `sessionDescription` set to the browser's offer and `tracks: [{ location: "local", mid, trackName: "camera" }]`. Camera and microphone can go in one request, each with its own `mid` and `trackName`. 4. The browser applies the returned **answer** with `setRemoteDescription()`. No `/renegotiate` call is needed in this direction. 5. When the connection reports `connected`, the app records the publisher's `sessionId` and track names in its visit state. ## Pulling a track 1. The viewer creates a new `RTCPeerConnection` with a `track` handler, and the backend opens a second SFU session for it. 2. The backend calls `tracks/new` on the **viewer's** session with `tracks: [{ location: "remote", sessionId: "<publisher session>", trackName: "camera" }]` and no offer. One request can pull several tracks, even from different publishers. 3. This time the SFU returns an **offer**, flagged `requiresImmediateRenegotiation: true`. The browser applies it, creates and sets an answer, and returns it. 4. The backend submits the answer to `PUT /sessions/{sessionId}/renegotiate`. The pull is finished only when that call succeeds. A browser reaching `signalingState: "stable"` does not prove the SFU has the answer. 5. In the `track` handler, `event.transceiver.mid` tells the viewer which publication arrived. The receiving `mid` can differ from the publisher's. ## What it looks like for a ten-person visit - Each participant opens a **publishing session** carrying two local tracks, `camera` and `microphone`, in one `tracks/new` request. - Each participant also opens a **receiving session** and pulls the other nine people's tracks: up to 18 remote entries, which fits in one request under the 64-track cap. - When someone new joins, every existing receiving session pulls two more tracks, and each of those pulls is its own offer/answer exchange. - When someone leaves, the app removes their tracks from discovery and closes the matching subscriptions on every receiving session. ## Details that bite - **Check every item.** A `200` response can still carry per-track `errorCode` values; record a track as ready only when its own result succeeded. - **One mutation at a time per session.** Track creation, closure and renegotiation on one session must be serialized; different sessions can run in parallel. - **Limits.** The API allows 50 requests per second per session and up to 64 tracks in one request. A track that receives no media packets for 30 seconds is garbage-collected. - **Two sessions per participant is fine.** Cloudflare's video-room example gives each browser separate publishing and receiving sessions so their negotiations never collide. - **Closing.** `PUT /sessions/{sessionId}/tracks/close` takes the `mid`s to stop. With `force: true` it stops forwarding without an SDP exchange. - **NAT traversal is still yours.** The SFU path does not hand the browser TURN servers, so a team building on the SFU directly adds Cloudflare TURN credentials to its `iceServers` itself.

  • Why might a team give each participant two SFU sessions instead of one?
    Each session has one PeerConnection and one offer/answer state machine. With separate publishing and receiving sessions, a new pull never has to wait for, or collide with, the participant's own publish negotiation. One bidirectional session also works if every mutation on it goes through the same queue.
  • What happens to a published track when the clinician's camera stops sending for a while?
    A track that receives no incoming media packets for 30 seconds is garbage-collected. The app must publish it again and rebuild the subscriptions that pointed at it. Session reuse after a connectivity loss is a separate 30-second window.
  • How does a viewer pick a lower video quality from a simulcast publisher?
    The remote track entry in `tracks/new` can carry a `simulcast` object with `preferredRid` and fallback policies, and `PUT .../tracks/update` changes `preferredRid` on an existing subscription by its receiving `mid`. Both `priorityOrdering` and `ridNotAvailable` default to `none`, so nothing falls back unless you set them.

  • The SFU automatically forwards every participant's tracks to everyone in the room.
  • The browser can call the SFU API itself with the App Secret embedded in the page.
  • Pulling a track is finished once the browser's signalingState is stable.
  • A subscriber locates a track by its mid alone.
  • An HTTP 200 from tracks/new means every requested track succeeded.