Using YouTube to Transcode Videos to DASH on the Command Line

by @edent | # # # # | 9 comments | Read ~1,483 times.

This is part of my redecentralisation efforts to liberate my videos from YouTube.

MPEG-DASH is a simple method of streaming videos which doesn't require any specialised server software. You convert a high resolution video into a series of smaller resolution videos. You chop each of the videos up into several chunks. As the video plays, your browser then decides which chunk of the video to load next depend on the bandwidth available to it. Easy!

Aim

From the command line:

  • Upload a video to YouTube
  • Download transcoded video in several formats suitable for MPEG DASH Streaming
  • Generate DASH manifest
  • Generate the HTML5 markup

Uploading

There are a couple of YouTube video uploaders. These are useful if you've got a video on a server, or don't want to waste a browser window.

I'm using Porjo's client. After following the authentication instructions, the output is something like:

./youtubeuploader_linux_amd64 -filename testing.mp4

Uploading file 'testing.mp4'...
Progress:     1.37 Mbps, 9852099 / 9852099 (100.000%) ETA          0s
Upload successful! Video ID: 6NV5heuaVuc

Downloading

The only program worth using is youtube-dl.

We can see every video format that YouTube has created using:

youtube-dl -F 6NV5heuaVuc

That will return something like:

format codeextensionresolutionnote
139m4aaudio only DASH audio49k , m4a_dash container, mp4a.40.5@ 48k (22050Hz)
140m4aaudio only DASH audio128k , m4a_dash container, mp4a.40.2@128k (44100Hz)
134mp4640x360DASH video 104k , avc1.4d401e, 30fps, video only
160mp4256x144DASH video 108k , avc1.4d400b, 30fps, video only
133mp4426x240DASH video 242k , avc1.4d400c, 30fps, video only
135mp4854x480DASH video 1155k , avc1.4d4014, 30fps, video only
137mp41920x1080DASH video 2306k , avc1.640028, 30fps, video only
136mp41280x720DASH video 2310k , avc1.4d4016, 30fps, video only
173gp176x144small , mp4v.20.3, mp4a.40.2@ 24k
363gp320x180small , mp4v.20.3, mp4a.40.2
43webm640x360medium , vp8.0, vorbis@128k
18mp4640x360medium , avc1.42001E, mp4a.40.2@ 96k
22mp41280x720hd720 , avc1.64001F, mp4a.40.2@192k (best)

It may take several minutes for YouTube to generate all the video formats - the length of time will depend on the length and quality of the video.

We want to download all the DASH videos and the DASH audios. We also want the subtitles - even if they are autogenerated. Finally, we want the thumbnail image for the HTML5 poster.

The youtube-dl command to download all the DASH audio is:

youtube-dl -f 'all[vcodec=none]' -o '%(title)s-%(format)s.%(ext)s' 6NV5heuaVuc

To get all the DASH videos:

youtube-dl -f 'all[acodec=none]' -o '%(title)s-%(format)s.%(ext)s' 6NV5heuaVuc

This will create a separate MP4 video for each resolution, and separate audio file for each bitrate.

To generate a single thumbnail, run:

youtube-dl --write-thumbnail --skip-download 6NV5heuaVuc

To download the subtitles:

youtube-dl --write-auto-sub --sub-format vtt --write-sub 6NV5heuaVuc

Convert to DASH

For this step, we need the open source Bento4 toolkit.

The video should already have "fragments" - the chunks which make up the stream - you can check this with:

./mp4info "Video-133 - 426x240 (DASH video).mp4"

In the output, you should see:

Movie:
  duration:   0 ms
  time scale: 90000
  fragments:  yes

The audio downloaded from YouTube will not have fragments, so you'll need to run this command on every audio file you've downloaded:

./mp4fragment "Video-140 - audio only (DASH audio).m4a" high.m4a

Combine and create manifest

Let's take those audio and video files and make ourselves a manifest!

./mp4dash -o output-video 133.mp4 135.mp4 136.mp4 137.mp4 160.mp4 high.m4a low.m4a

This creates the folder output-video and creates folders for the chunked audio and video.

It will also produce the MPD - a manifest file which references all of the chunks.

We're nearly ready to play back.

Play DASH Video in the browser

As far as I can tell, no browser natively supports MPEG-DASH. You will need to use the latest version of dash.js to make everything work smoothly.

Add this piece of JavaScript to your page:

<script src="https://cdn.dashjs.org/latest/dash.all.min.js"></script>

Then reference the MPD in a video element:

<video width="1080" data-dashjs-player src="stream.mpd" controls poster="example.jpg">
    <track kind="subtitles" src="en.vtt" srclang="en" label="English">
</video>

Demo

Here is an adaptive stream of one of my videos - hosted natively on this site, with no server-side streaming.

Problems?

Fallback video

Some browsers can't play back DASH, even with the JavaScript - so you can put a fallback video in the HTML.

<video width="1080" data-dashjs-player controls >
    <source src="stream.mpd"   type="application/dash+xml">
    <source src="fallback.mp4" type="video/mp4">
</video>

HLS

Because Apple are special, they don't support DASH on its phones. So you'll need to use their proprietary HLS streaming solution. You can use Bento4 to create HLS in exactly the same way as you create DASH.

You may also need to use hls.js JavaScript to get things to work smoothly.

Subtitles

You can use DASH native subtitles, or HTML5 subtitles.

I found that the HTML5 subtitles downloaded by youtube-dl contain some strange CSS which didn't work well with the DASH video. You may need to test to see how it works for you.

All in one command?

Hahahaha! Not yet! If you know of a good way to do this - let me know!

9 thoughts on “Using YouTube to Transcode Videos to DASH on the Command Line

  1. Raphael says:

    Great...but I don't understand why you want to use Youtube in the first place?
    You can convert any avi directly to MPREG-DASH yourself by using https://github.com/Cloudoki/mp4-to-mpegdash
    This is the all in one command you wanted 😉
    Cheers!

    1. Terence Eden says:

      Because I have a weak and feeble laptop which takes ages to convert videos. Using other people's computers is cheaper 🙂

  2. Whoa, this is amazing! I really like how the browser decides which subsequent chunk of the video to load next depending on the bandwidth available. The video played super smoothly on my super crappy internet connection (I live in a developing country), which is impressive.

    It's also interesting how in chrome "right-click > save video as..." is disabled, but I guess that's to be expecting considering how this works.

    Sucks how Safari on iOS doesn't support it though, although HLS seems easy enough to implement.

  3. wristy says:

    Hi, thanks for article, very interesting. I have installed youtube-dl and upload, then I uploaded a mp4 file which is about 30MB in size, the very famous big_buck_bunny.mp4. It's been more than 30 minutes since i have uploaded the file, and when i executed youtube-dl -F "filename.mp4" it says:

    [youtube] Yc74rOEtCFY: Downloading webpage
    [youtube] Yc74rOEtCFY: Downloading video info webpage
    ERROR: Yc74rOEtCFY: YouTube said: This video is unavailable.

    1. @edent says:

      That video doesn't exist - https://www.youtube.com/watch?v=Yc74rOEtCFY

      If you're having problems with youtube-dl you can get support at https://github.com/rg3/youtube-dl/issues/

      1. wristy says:

        That's interesting ! I clicked on the link you provided and i'm seeing my video. But when i execute the command line

        C:\Program Files\YoutubeDL>youtube-dl -F Yc74rOEtCFY
        [youtube] Yc74rOEtCFY: Downloading webpage
        [youtube] Yc74rOEtCFY: Downloading video info webpage
        ERROR: Yc74rOEtCFY: YouTube said: This video is unavailable.

        ...The vid is not available. Thanks for you help, I'll try contact YouTube.

        Regards

        1. @edent says:

          Have you set the video to private? Or not published it?

          1. wristy says:

            Hi, sorry i'm not uploading every day videos so I did not even notice is on Private. I change to Public but I think it will take a while to process. You should be able to view it shortly.

          2. wristy says:

            Quick update, I am able to see all the information about the video now, using the command line
            C:\Program Files\YoutubeDL>youtube-dl -F Yc74rOEtCFY

            It just took a whole lot of time to process the video. Thanks again.

            Regards,

Leave a Reply to rahuldottech Cancel reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.