Using YouTube to Transcode Videos to DASH on the Command Line

This is part of my redecentralisation efforts to liberate my videos from YouTube.

MPEG-DASH is a simple method of streaming videos which doesn't require any specialised server software. You convert a high resolution video into a series of smaller resolution videos. You chop each of the videos up into several chunks. As the video plays, your browser then decides which chunk of the video to load next depend on the bandwidth available to it. Easy!

Aim

From the command line:

• Upload a video to YouTube
• Download transcoded video in several formats suitable for MPEG DASH Streaming
• Generate DASH manifest
• Generate the HTML5 markup

Uploading

There are a couple of YouTube video uploaders. These are useful if you've got a video on a server, or don't want to waste a browser window.

• porjo/youtubeuploader has binaries available for Linux/Mac/Windows.
• tokland/shoogle a Python interface for all of Google's APIs.

I'm using Porjo's client. After following the authentication instructions, the output is something like:

./youtubeuploader_linux_amd64 -filename testing.mp4

Uploading file 'testing.mp4'...
Progress:     1.37 Mbps, 9852099 / 9852099 (100.000%) ETA	  0s
Upload successful! Video ID: 6NV5heuaVuc

Downloading

The only program worth using is youtube-dl.

We can see every video format that YouTube has created using:

youtube-dl -F 6NV5heuaVuc

That will return something like:

format code	extension	resolution	note
139	m4a	audio only DASH audio	49k , m4a_dash container, mp4a.40.5@ 48k (22050Hz)
140	m4a	audio only DASH audio	128k , m4a_dash container, mp4a.40.2@128k (44100Hz)
134	mp4	640x360	DASH video 104k , avc1.4d401e, 30fps, video only
160	mp4	256x144	DASH video 108k , avc1.4d400b, 30fps, video only
133	mp4	426x240	DASH video 242k , avc1.4d400c, 30fps, video only
135	mp4	854x480	DASH video 1155k , avc1.4d4014, 30fps, video only
137	mp4	1920x1080	DASH video 2306k , avc1.640028, 30fps, video only
136	mp4	1280x720	DASH video 2310k , avc1.4d4016, 30fps, video only
17	3gp	176x144	small , mp4v.20.3, mp4a.40.2@ 24k
36	3gp	320x180	small , mp4v.20.3, mp4a.40.2
43	webm	640x360	medium , vp8.0, vorbis@128k
18	mp4	640x360	medium , avc1.42001E, mp4a.40.2@ 96k
22	mp4	1280x720	hd720 , avc1.64001F, mp4a.40.2@192k (best)

It may take several minutes for YouTube to generate all the video formats - the length of time will depend on the length and quality of the video.

We want to download all the DASH videos and the DASH audios. We also want the subtitles - even if they are autogenerated. Finally, we want the thumbnail image for the HTML5 poster.

The youtube-dl command to download all the DASH audio is:

youtube-dl -f 'all[vcodec=none]' -o '%(title)s-%(format)s.%(ext)s' 6NV5heuaVuc

To get all the DASH videos:

youtube-dl -f 'all[acodec=none]' -o '%(title)s-%(format)s.%(ext)s' 6NV5heuaVuc

This will create a separate MP4 video for each resolution, and separate audio file for each bitrate.

To generate a single thumbnail, run:

youtube-dl --write-thumbnail --skip-download 6NV5heuaVuc

To download the subtitles:

youtube-dl --write-auto-sub --sub-format vtt --write-sub 6NV5heuaVuc

Convert to DASH

For this step, we need the open source Bento4 toolkit.

The video should already have "fragments" - the chunks which make up the stream - you can check this with:

./mp4info "Video-133 - 426x240 (DASH video).mp4"

In the output, you should see:

Movie:
  duration:   0 ms
  time scale: 90000
  fragments:  yes

The audio downloaded from YouTube will not have fragments, so you'll need to run this command on every audio file you've downloaded:

./mp4fragment "Video-140 - audio only (DASH audio).m4a" high.m4a

Combine and create manifest

Let's take those audio and video files and make ourselves a manifest!

./mp4dash -o output-video 133.mp4 135.mp4 136.mp4 137.mp4 160.mp4 high.m4a low.m4a

This creates the folder output-video and creates folders for the chunked audio and video.

It will also produce the MPD - a manifest file which references all of the chunks.

We're nearly ready to play back.

Play DASH Video in the browser

As far as I can tell, no browser natively supports MPEG-DASH. You will need to use the latest version of dash.js to make everything work smoothly.

Add this piece of JavaScript to your page:

<script src="https://cdn.dashjs.org/latest/dash.all.min.js"></script>

Then reference the MPD in a video element:

<video width="1080" data-dashjs-player src="stream.mpd" controls poster="example.jpg">
    <track kind="subtitles" src="en.vtt" srclang="en" label="English">
</video>

Demo

Here is an adaptive stream of one of my videos - hosted natively on this site, with no server-side streaming.

Problems?

Fallback video

Some browsers can't play back DASH, even with the JavaScript - so you can put a fallback video in the HTML.

<video width="1080" data-dashjs-player controls >
    <source src="stream.mpd"   type="application/dash+xml">
    <source src="fallback.mp4" type="video/mp4">
</video>

HLS

Because Apple are special, they don't support DASH on its phones. So you'll need to use their proprietary HLS streaming solution. You can use Bento4 to create HLS in exactly the same way as you create DASH.

You may also need to use hls.js JavaScript to get things to work smoothly.

Subtitles

You can use DASH native subtitles, or HTML5 subtitles.

I found that the HTML5 subtitles downloaded by youtube-dl contain some strange CSS which didn't work well with the DASH video. You may need to test to see how it works for you.

All in one command?

Hahahaha! Not yet! If you know of a good way to do this - let me know!