Exploring Stream Live: Cloudflare’s Live Streaming Service Behind the Scenes

Cloudflare introduced Stream Live in 2021, opening it up for open beta, and officially launching it in 2022. While we’ve discussed the user experience and benefits of the service, let’s take a look at how it was actually built. Stream Live is designed to utilize Cloudflare’s vast network in a distributed manner, rather than relying on centralized locations like many other live streaming services. The main objectives are to keep content ingestion close to broadcasters, content delivery close to viewers, and to be prepared for any unexpected usage scenarios.

Stream Live works by receiving audio and video content from broadcasters and making it accessible to viewers globally through the Cloudflare network, which spans over 330 cities in more than 120 countries. There are two main components to this process: ingesting data from broadcasters and delivering encoded content to viewers. Both aspects involve a combination of internal systems and Cloudflare products such as Cloudflare Workers, Durable Objects, Spectrum, and Cache.

Let’s dive into the ingestion side first. Broadcasters generate real-time content in the form of video and audio frames, which they need to transmit to Cloudflare over the Internet. They can choose from protocols like RTMPS, SRT, or WHIP to define how their content is packaged and sent. Regardless of the protocol chosen, the ingest process is similar. Broadcasters connect to an Anycast IP address using their custom ingest domain or the default live.cloudflare.com. This setup ensures minimal latency for broadcasters of all sizes to reach Cloudflare’s ingest points globally.

When the media content reaches the Cloudflare server, it is processed by a Spectrum application. Spectrum manages things like TLS termination for RTMPS, security against potential DDOS attacks, and supports protocols like SRT for clients that don’t fully comply with the standard. Spectrum efficiently forwards all connections to the ingest service running on the same server, streamlining the process.

The ingest service is responsible for receiving content from broadcasters, forwarding it to live outputs, and recording it for later viewing. This service is built in Go and relies on Live Config Durable Objects to store broadcaster configurations and ongoing broadcast states. Durable Objects are chosen over a centralized database for their ability to be geographically closer to broadcasters while maintaining high availability. The broadcast state is split into ‘volatile’ and ‘committed’ categories, allowing for seamless resumption of live broadcasts if a broadcaster disconnects and reconnects.

The ingest service operates as a relay, mapping incoming packets from one input stream to multiple output streams. It also includes Live Playback and Recording functionality, where incoming packets are sanitized, aligned, and stored as fragmented MP4s called ‘original segments.’ These original segments serve as the primary copy of uploaded content and are used for both live and on-demand viewing.

With Stream Live, Cloudflare has created a robust and efficient live streaming service that leverages its global network to deliver high-quality content to viewers worldwide.