Video Accessibility on the Web

Jason Ronallo
NCSU Libraries
Interim Head of Digital Library Initiatives
@ronallo

I Want to Trick You into Using an Accessibility Technology

Timed Text Tracks

Captions

Subtitles

"It's hot in here." translated to "Il fait chaud ici." "It's not the heat it's the humanity." translated to "La chaleur humaine!"

From Brigadoon (1954)

Chapters

• No browser currently supports chapter tracks natively.
• Some players do provide this functionality
  • MediaElement.js
  • Video.js
  • JWPlayer

Audio Description

Audio description is narration of the visual content of a video.

Audio description is an art. How do you fit the description into the pauses in the regular audio content?

Works now with Text to Speech with a Chrome browser plugin

Text Tracks in HTML

Minimal Video Markup With The Track Element

<video>
  <source src="video.webm">
  <track src="track.vtt">
</video>

WebVTT File Format

WebVTT is...

• A W3C Community Group Draft Report.
• A timed text file format specified for use as HTML tracks.
• A plain text file with some special formatting rules.
• Intended to fit the needs of every track kind.
• Inspired by SubRip but expanded and standardized.

WebVTT is a Plain Old Text File

WEBVTT

00:00.000 --> 00:03.000
This is the first cue.

00:03.000 --> 00:07.000
This is the second cue and
breaks over two lines.

00:07.000 --> 00:10.000
Line breaks are kept though 
additional line breaks 
may be added to make the text fit 
the available space for the cue.

WebVTT Features

• Comments/Notes
• Voices
• Cue Settings
• Regions
• CSS styling
• Metadata Header

This Is A Test

WebVTT and Text Track Tricks

WebVTT Caption Reuse

Improve and Enrich Access

• Site search--find the right video
• Search engine optimization

HTML5 Video JavaScript API

$('#no-controls-controls .play').on('click', 
  function(){
    $('video#no-controls')[0].play();
});

Transcript on the Page

Timeline Search

Access Before Discoverability for Archives

Automated Summarization Services

Time Rail Time Stamp

Preview Thumbnails For Archival Video

Boiling Water

Head Banging JFK

President John F. Kennedy Speech at conference on milk and nutrition

Process for HTML5 Video Preview Thumbnails

1. Take a thumbnail snapshot image of the video every N seconds.*
2. Stitch the thumbnails into 1 or more sprite images.
3. Create a WebVTT file to document which thumbnail in the sprite corresponds with which portion of the video.
4. Add the track to your video.
5. Add some JavaScript to read in the WebVTT file, load the sprite, and show the correct thumbnail.

Look in the links for the open source tools to do all of this this!

*Or use scene detection to to create the thumbnails.

JSON Metadata Payload

Use any text or data for the WebVTT cue payload.

WEBVTT

00:01.000 --> 00:03.000
{"title": "First image", 
"image": "http://example.com/images/first.png"}

00:05.000 --> 00:07.000
{"title": "2nd image", 
"image": "http://example.com/images/2nd.png"}

00:08.000 --> 00:10.000
{"title": "Image C", 
"image": "http://example.com/images/c.png"}

WebVTT for Video Preview Thumbnails

WEBVTT

00:00:00.000 --> 00:00:05.000
http://example.co/sprite.jpg#xywh=0,0,150,100

00:00:05.000 --> 00:00:10.000
http://example.co/sprite.jpg#xywh=150,0,150,100

00:00:10.000 --> 00:00:15.000
http://example.co/sprite.jpg#xywh=300,0,150,100

00:00:15.000 --> 00:00:20.000
http://example.co/sprite.jpg#xywh=450,0,150,100

Media Fragments

Spatial Dimension

Get part of an image.

http://example.co/sprite.jpg#xywh=450,0,150,100

Video Thumbnail Tools

MediaElement.js time rail thumbnails plugin. https://github.com/jronallo/mep-feature-time-rail-thumbnails

video-sprites https://github.com/jronallo/video-sprites

Karaoke

Karaoke

Tubthumping VTT

WEBVTT

00:00.000 --> 00:01.600 line:40%
<00:00.100><c>He</c> <00:00.200><c>drinks</c>
<00:00.400><c>a</c> <00:00.900><c>whiskey</c> 
<00:01.000><c>drink</c>

00:01.601 --> 00:02.699 line:40%
<00:01.602><c>He</c> <00:01.700><c>drinks</c>
<00:02.000><c>a</c> <00:02.100><c>vodka</c> 
<00:02.300><c>drink</c>

00:02.700 --> 00:03.770 line:40%
<00:02.800><c>He</c> <00:03.000><c>drinks</c>
<00:03.200><c>a</c> <00:03.300><c>lager</c> 
<00:03.600><c>drink</c>

Styling a Karaoke Track

Past & Future Selectors

video#tubthumping::cue { font-size: 3em; }

video#tubthumping::cue(:past)   { color: red }

video#tubthumping::cue(:future) { color: gray }

Live Captioning

Thanks

To Silvia Pfeiffer for being editor of the WebVTT specification and answering some of my questions.

To the Association of Moving Image Archivists for sponsoring the longer presentation this lightning talk is based on.

Project Links

• Longer version of this presentation with detailed notes and an audience handout http://webvtt-presentation.ronallo.com/
• Test WebVTT Cue Settings and Styling http://ronallo.com/demos/webvtt-cue-settings/
• Kinney A Rails Engine for oral history video clips projects. https://github.com/jronallo/kinney
• webvtt A WebVTT parser library written in Ruby. https://github.com/jronallo/webvtt
• MediaElement.js time rail thumbnails plugin. https://github.com/jronallo/mep-feature-time-rail-thumbnails
• video-sprites https://github.com/jronallo/video-sprites
• Blog http://ronallo.com/blog/categories/video/

Jason Ronallo

@ronallo

http://ronallo.com

Slides: http://ronallo.com/presentations/

License

This work by Jason Ronallo is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Permissions beyond the scope of this license may be available at http://ronallo.com.

CSS Styling of HTML5 Video

My Favorite Slide Ever

Accessibility

Alternative or augmented access to video for those with a range of disabilities.

• Alternative text for the deaf and hard of hearing.
• Alternative audio for the blind or those with decreased vision.
• Contrast controls for the color blind.
• Graphical interfaces that aid those with motion disabilities.

Accessibility Too: Improving Access

• Search engine optimization.
• Getting to the needed content.
• Make working with the content easier and more efficient.

version

v.0.1