Check out Optimize, our new AI powered product!

API Update: Handling Multiple Media Stream Tracks with callstats.js
By: , ,
17 July 2015

WebRTC is progressing rapidly; the main trend seems to be transitioning from peer-to-peer to conferences with centralized servers. This is occurring mainly for conferences with more than 5 participants. In a full-mesh video, every participant sends media to all other participants, i.e., if there are M participants, an endpoint sends M-1 media streams and receives M-1 media streams (shown in Fig 1) 1. In this case, each peerconnection carries media stream tracks corresponding to the audio, camera, screen, and data between the pair of participants.

WebRTC conference with mesh topology

Figure 1. Mesh topology

Alternatively, when using a conferencing server a participant sends its media stream to the conference server and receives media streams from the conference server and not the individual participants. Hence, an endpoint sends 1 media stream and receives M-1 media streams (shown in Fig 2). Mapping this to WebRTC 1.0 terminology, an M participant conference will have M links 2. However, in this case, a single peer connection carries multiple incoming media streams corresponding to each participant.

WebRTC conference star topology

Figure 2. Star topology

In our API, when setting up a peer connection, the developer specifies the localUserID and remoteUserID for the peerconnection (See addNewFabric()). The remoteUserID in a mesh topology is the remote participant’s userID (see Figure 3a). However, when interacting with the conference server, the developer is most likely going to use the name or identifier of the conference bridge as the remoteUserID, as it cannot use a particular remote user ID as they may be several. A typical conference bridge (for example, Jitsi Videobridge) transmits multiple media stream tracks within a peer connection (see figure 3b).

WebRTC conference star topology

Figure 3. Connections viewed by a single participant.

Furthermore, correlating the received Synchronization Source Identifier (SSRC) with the SSRC generated by every participant for each media stream track may not work if the conferencing server rewrites the incoming SSRC. Therefore, making it difficult for the callstats.js to identify which incoming media stream belongs to which participant even though we might know the SSRCs generated by each endpoint.

Today, we are announcing a new API that solves the issue and will be available in callstats.js version 3.3.0 associateMstWithUserID() maps each media stream track received by an endpoint to a remote userID (see figure 4). The example below shows how the API can be integrated:

Associationg userID with MST

Figure 4. Associating userID with MST.

	// After O/A is complete, i.e., onAddStream is fired
	// localUserID = "Alice";
	var remoteUserID = "Bob";
	var conferenceID = "AliceAndBobAndCharlie";
	var mstLabel = "front-camera";
	// SSRC is passed from the signaling/origin server
	// mstLabel is a developer provided string that lets them identify
	// various tracks (e.g., front-camera, back-camera, without looking at the
	// configurations of the individual MSTs).
	callStats.associateMstWithUserID(pc, remoteUserID, conferenceID, ssrc, mstLabel);

At callstats.io, our vision to provide high quality of media experience to every interaction extends beyond serving endpoints and to middleboxes. There are several types of middleboxes in the wild: relay servers (for NAT/firewall traversals), conferencing server, and transcoders (H.264 to VP8, AMR to Opus). Getting metrics and statistics from the middlebox is important for scaling large deployments. Unlike endpoints which either send or receive a particular stream, middleboxes send and receive each stream, unless a local policy dictates not sending a particular subset of streams (e.g., send only active speakers). To facilitate middleboxes to report statistics on the media they received and they sent, we use the following structure:

{
    "localUserID": "conferenceServerID",
    "stats": {
        "streams": {
            "209773412": {
                "fromUserID": "user-aaaa",
                "inbound":   {        }
            },
            "674768567": {
                "fromUserID": "user-aaaa",
                "toUserID": "user-bbbb",
                "outbound": {        }
            },
           "342342222": {
                "fromUserID": "user-aaaa",
                "toUserID": "user-cccc",
                "outbound": {        }
            },
            "342342222": {
                "fromUserID": "user-aaaa",
                "toUserID": "user-dddd",
                "outbound": {        }
            }
        },
        "Transport": [{active-candidate-pair-1}, {active-candidate-pair-2}]
    }
}

Comments and feedback on evolving associateMstWithUserID() is appreciated. If you would like to get a free 30-day trial of callstats.io, please email us at support-at-callstats.io.


Addendum (31 July 2015): changed last paragraph to reflect stability of the API, it is not in testing any more, and is being used in production.



comments powered by Disqus