Dynamic Captioning: Video Accessibility Enhancement for Hearing Impairment

author: Richang Hong, School of Computing, National University of Singapore
published: Feb. 1, 2011,   recorded: October 2010,   views: 6219


Related Open Educational Resources

Related content

Report a problem or upload files

If you have found a problem with this lecture or would like to send us extra material, articles, exercises, etc., please use our ticket system to describe your request and upload the data.
Enter your e-mail into the 'Cc' field, and we will keep you updated with your request's status.
Lecture popularity: You need to login to cast your vote.


There are more than 66 million people su®ering from hear- ing impairment and this disability brings them di±culty in video content understanding due to the loss of audio infor- mation. If the scripts are available, captioning technology can help them in a certain degree by synchronously illus- trating the scripts during the playing of videos. However, we show that the existing captioning techniques are far from satisfactory in assisting the hearing impaired audience to en- joy videos. In this paper, we introduce a scheme to enhance video accessibility using a Dynamic Captioning approach, which explores a rich set of technologies including face de- tection and recognition, visual saliency analysis, text-speech alignment, etc. Di®erent from the existing methods that are categorized as static captioning, dynamic captioning puts scripts at suitable positions to help hearing impaired audi- ence better recognize the speaking characters. In addition, it progressively highlights the scripts word-by-word via align- ing them with the speech signal and illustrates the variation of voice volume. In this way, the special audience can better track the scripts and perceive the moods that are conveyed by the variation of volume. We implemented the technol- ogy on 20 video clips and conducted an in-depth study with 60 real hearing impaired users. The results demonstrated the e®ectiveness and usefulness of the video accessibility en- hancement scheme.

See Also:

Download slides icon Download slides: acmmm2010_hong_dcv_01.pdf (6.9 MB)

Help icon Streaming Video Help

Link this page

Would you like to put a link to this lecture on your homepage?
Go ahead! Copy the HTML snippet !

Write your own review or comment:

make sure you have javascript enabled or clear this field: