Multimodal Interfaces
Description
Two important issues in multimodal system design, is the selection and mix of input and output modalities and the exploitation of the synergies between the modalities in order to maximize the usability of the system. In this talk, we propose two new objective metrics, relative modality efficiency and multimodal synergy, that can provide valuable information and identify usability problems during the evaluation of multimodal systems. Relative modality efficiency (when compared with modality usage) can identify suboptimal use of modalities due to poor interface design or information asymmetries. Multimodal synergy measures the added value from efficiently combining multiple input modalities, and can be used as a single measure of the quality of modality fusion and fission in a multimodal system. The proposed metrics are used to evaluate a multimodal system that combines pen and speech input, and are compared with traditional evaluation metrics. The results provide much insight into multimodal interface usability issues, and demonstrate how multimodal systems should adapt to maximize modalities synergy resulting in efficient, natural, and intelligent multimodal interfaces.
| Slides | |
| 0:00 | Multimodal Dialogue Interfaces |
| 0:09 | Research Goals |
| 1:07 | Speech: An interaction modality and more |
| 2:17 | Idiosyncrasies of the speech modality |
| 4:24 | Design principles for multimodal dialogue systems |
| 6:12 | Multimodal dialogue systems and synergies |
| 7:07 | Interaction modes evaluated |
| 8:39 | System demo (desktop version) |
| 10:01 | PDA environment: “Modality selection” example |
| 10:49 | Evaluation and mode statistics |
| 11:32 | Modality selection and unimodal efficiency (context statistics) |
| 12:30 | Evaluation of multimodal form filling systems |
| 13:25 | Relative modality efficiency |
| 14:12 | Multimodal synergy |
| 14:50 | Random multimodal synergy |
| 15:24 | Relative speech efficiency for the four contexts |
| 16:39 | Relative speech efficiency for the three modes |
| 17:01 | Relative speech efficiency for mode/context |
| 17:25 | Relative speech efficiency for the eight users |
| 17:43 | Relative speech efficiency for users & contexts |
| 18:28 | Results: Synergy and multimodal modes |
| 21:17 | Results: Synergy and contexts |
| 21:39 | Results: Synergy and users - 1 |
| 22:00 | Results: Synergy and users - 2 |
| 22:09 | - Questions |
Lecture rating
| People found this lecture: | ||
| Worth seeing | ||
| because it is: | ||
| Valuable and informative | ||
| Well presented | ||
| Easily understandable | ||
| Acceptably recorded | ||
| You need to login to cast your vote. | ||
Report a problem or upload files
If you have found a problem with this lecture or would like to send us extra material, articles, exercises, etc., please use our ticket system to describe your request and upload the data.Enter your e-mail into the 'Cc' field, and we will keep you updated with your request's status.
Related content
SEE ALSO:
Link this page
Would you like to put a link to this lecture on your homepage?Go ahead! Copy the HTML snippet !




