Learning language from its perceptual context

author: Raymond J. Mooney, Department of Computer Sciences, The University of Texas at Austin
published: Oct. 10, 2008,   recorded: September 2008,   views: 400
Categories
You might be experiencing some problems with Your Video player.

Slides

Slides
0:00 Learning Language from its Perceptual Context
0:25 Current State of Natural Language Learning
1:41 Semantic Parsing
2:59 GeoQuery: A Database Query Application
3:50 CLang: RoboCup Coach Language
5:11 Learning Semantic Parsers
6:01 Our Semantic-Parser Learners
6:58 WASP
8:04 A Unifying Framework for Parsing and Generation (1)
8:16 A Unifying Framework for Parsing and Generation (2)
8:34 A Unifying Framework for Parsing and Generation (3)
8:52 A Unifying Framework for Parsing and Generation (4)
9:00 A Unifying Framework for Parsing and Generation (5)
9:26 Synchronous Context-Free Grammars (SCFG)
9:51 Synchronous Context-Free GrammarProduction Rule
10:25 Synchronous Context-Free Grammar Derivation
11:54 Probabilistic Parsing Model (1)
12:10 Probabilistic Parsing Model (2)
12:31 Probabilistic Parsing Model (3)
13:57 Overview of WASP
15:40 Tactical Generation
16:13 Generation by Inverting WASP
16:53 Learning Language from Perceptual Context
18:58 Language Grounding
20:59 “Mary is on the phone”
21:44 Ambiguous Supervision for Learning Semantic Parsers
23:10 “Mary is on the phone” (1)
23:14 “Mary is on the phone” (2)
23:20 “Mary is on the phone” (3)
23:24 “Mary is on the phone” (4)
23:33 “Mary is on the phone” (5)
23:43 “Mary is on the phone” (6)
23:52 Next Ambiguous Training Example
24:09 Ambiguous Supervision for Learning Semantic Parsers (cont.)
25:03 Sample Ambiguous Corpus
25:45 KRISPER: KRISPwith EM-like Retraining
26:28 KRISPER’s Training Algorithm (1)
26:55 KRISPER’s Training Algorithm (2)
27:21 KRISPER’s Training Algorithm (3)
27:58 KRISPER’s Training Algorithm (4)
28:26 KRISPER’s Training Algorithm (5)
28:50 KRISPER’s Training Algorithm (6)
29:21 KRISPER’s Training Algorithm (7)
29:46 Results on Ambig-ChildWorld Corpus
30:40 New Challenge:Learning to Be a Sportscaster
32:37 Robocup Sportscaster Trace (1)
32:51 Robocup Sportscaster Trace (2)
33:08 Robocup Sportscaster Trace (3)
33:54 Robocup Sportscaster Trace (4)
34:24 Sportscasting Data
35:31 WASPER
36:07 KRISPER-WASP
37:23 WASPER-GEN
38:46 Strategic Generation
39:05 Example of Strategic Generation (1)
39:15 Example of Strategic Generation (2)
39:27 Learning for Strategic Generation
40:02 Iterative Generation Strategy Learning (IGSL)
40:21 Demo
41:35 Experimental Evaluation
42:30 Evaluating Matching Accuracy
42:52 Results on Matching
43:32 Evaluating Semantic Parsing
43:51 Results on Semantic Parsing
44:28 Evaluating Tactical Generation
44:54 Results on Tactical Generation
45:26 Evaluating Strategic Generation
45:43 Results on Strategic Generation
46:28 Human Evaluation(Quasi Turing Test)
47:17 Human Evaluation Metrics
47:39 Results on Human Evaluation
48:24 Immediate Future Directions
49:16 Machine Learning Research Direction
50:56 Longer Term Future Directions
52:15 Blatant Talk Advertisement
52:56 Conclusions

Related content

Report a problem or upload files

If you have found a problem with this lecture or would like to send us extra material, articles, exercises, etc., please use our ticket system to describe your request and upload the data.
Enter your e-mail into the 'Cc' field, and we will keep you updated with your request's status.
Lecture popularity: You need to login to cast your vote.
 
    Delicious Bibliography

Description

Current systems that learn to process natural language require laboriously constructed human-annotated training data. Ideally, a computer would be able to acquire language like a child by being exposed to linguistic input in the context of a relevant but ambiguous perceptual environment. As a step in this direction, we present a system that learns to sportscast simulated robot soccer games by example. The training data consists of textual human commentaries on Robocup simulation games. A set of possible alternative meanings for each comment is automatically constructed from game event traces. Our previously developed systems for learning to parse and generate natural language (KRISP and WASP) were augmented to learn from this data and then commentate novel games. The system is evaluated based on its ability to parse sentences into correct meanings and generate accurate descriptions of game events. Human evaluation was also conducted on the overall quality of the generated sportscasts and compared to human-generated commentaries.

Link this page

Would you like to put a link to this lecture on your homepage?
Go ahead! Copy the HTML snippet !

Write your own review or comment:

make sure you have javascript enabled or clear this field: