When you come across a website that has a video player embedded in it, there’s quite a bit going on behind the scenes.
You have the player UI, with the pause/play button, subtitle controls, video speed, and other options.
Players will support different options around DRM, ad injection, thumbnail previews, etc.
Behind the scenes, modern video platforms will use adaptive bitrate streaming to stream the video from the server.
Adaptive bitrate streaming means that the server has several different versions of the video (known as renditions) and each version differs in display size (resolution) and file size (bitrate).
The video player will dynamically choose the best rendition based on the user’s screen size and bandwidth. It will choose the rendition that minimizes buffering and gives the best user experience.
HLS
HTTP Live Streaming (HLS) is a protocol designed by Apple for HTTP-based adaptive bitrate streaming. It’s the most popular streaming format on the internet.
The basic concept is that you take your video file and break it up into small segments, where each segment is 2-12 seconds long.
If you have a 2-hour long video, you could break it up into segments that are 10 seconds long and end up with 720 segments.
Each of the segments is a file that ends with a .ts extension. The files are numbered sequentially, so you get a directory that looks like this
segments/
00001.ts
00002.ts
00003.ts
00004.ts
00005.ts
The player will then download and play each segment as the user is streaming. It will also keep a buffer of segments in case the user loses the network connection.
Again, HLS is an adaptive bitrate streaming protocol, so the webserver will have several different renditions (versions) of the video that is being played.
All of the renditions will be broken into segments of the same length. So, going back to our example with the 2-hour long video, it could have 720 segment files at 1080p, 720 segment files at 720p, 720 segment files at 480p.
All the segment files are ordered and are each 10 seconds in length. The player will then look at the amount of bandwidth available and make the best guess as to which rendition segment file it should download next.
If your network connection slows down while you’re watching a video, the player can downgrade you to a lower quality rendition for the next segment files.
When your connection gets faster, the player can upgrade your rendition.
MP4 & WebM
An alternative is taking an HTML <video> element and adding an “src” attribute that points directly to an MP4 or WebM file.
This is called pseudo-streaming or progressive download, where the video file is downloaded to a physical drive on the user’s device.
Typically, the video is stored in the temporary directory of the web browser and the user can start watching while the file is being downloaded in the background.
The user can also jump to specific points in the video and the player will use byte-range requests to estimate which part of the file corresponds to the place in the video that the user is attempting to seek.
What makes MP4 and WebM playback inefficient is that they do not support adaptive bitrates.
Every user who wants to watch the file buffer-free must have an internet connection that is fast enough to download the file faster than the playback.
Therefore, when you are using these formats you have to make a tradeoff between serving a higher quality video file vs. decreasing the internet connection speed requirements.
Delivery
When delivering video to your user, there are two primary components
- the origin server
- the content delivery network
The origin server is the source of truth. It’s where the developer uploads the original video files.
The CDN will then pull files from the origin server and cache that file on a bunch of interconnected servers around the world (in locations that are close to your users).
That way, when users want to request the file, they can do so from a server in the CDN. This is way faster (and much more scalable) than your origin server sending the entire file to all your users.
Many enterprises will choose a Multi-CDN environment, where the load is distributed among multiple CDNs. This improves the user experience by giving them more servers to choose from and improving the availability of your website.
Happy Learning …
