Creating subtitles and audio descriptions with HTML5 video

At the a11y hackdays last week I was inspired by Bruce’s talk on HTML5 video and Web-VTT (web video text track) to make a W3C Widget for quickly creating text tracks.

The basic components of the widget are the HTML5 <video> element itself, and a button for pausing the video and creating an entry in the text track.

This is pretty simple to wire up; to control the video I just need a reference to the <video> element in the document model, and then in JavaScript I can call:

video.pause();

… to pause the vide at the current frame. I can then get the time in the video where it has been paused using:

video.getCurrentTime();

Next, the widget shows a caption input box and an OK and Cancel button.

If the user enters some text in the input box and clicks OK, the widget creates an entry in a WebVTT file using the time paused as the start. I can resume the video by calling:

video.play();

Users can keep doing this until the end of the video. To get the actual text track there is a “VTT” button that shows a textarea containing the Web-VTT output we’ve created:

So the widget has captured the following as the VTT output:

WEBVTT FILE

1
00:00:01.665 --> 00:00:03.915
The Sun rises over a sleeping Earth

The first line indicates this is a Web-VTT file. Then we have the sequence number of the text track element. After this is the time information for synchronising the text with the video, and finally the text to use. There are lots of other things we can do with Web-VTT like styling and positioning, but for this widget I just kept it simple.

I didn’t get chance to make the live preview actually work, but its quite easy to create a demo using Playr:

Of course this isn’t the only way to use these files. You could also create a really nice standalone transcript widget that put the text track alongside the video player, or feed the Web-VTT file through a reader application for adding audio descriptions to the audio track. Or use it to pull up related content alongside the video while its playing.

If you’d like to play with this widget, it is available in Apache Wookie. You can also get a standalone HTML version and one for use with Opera Widgets from my Web-VTT github project. Or, if you’re using Opera Unite there is also something that does roughly the same thing called SubVid (thanks Brucel).