Learning to See

author: Antonio Torralba, Center for Future Civic Media, Massachusetts Institute of Technology, MIT
published: Aug. 23, 2016,   recorded: August 2016,   views: 9988


Related Open Educational Resources

Related content

Report a problem or upload files

If you have found a problem with this lecture or would like to send us extra material, articles, exercises, etc., please use our ticket system to describe your request and upload the data.
Enter your e-mail into the 'Cc' field, and we will keep you updated with your request's status.
Lecture popularity: You need to login to cast your vote.


It is an exciting time for computer vision. With the success of new computational architectures for visual processing, such as deep neural networks (e.g., convNets) and access to image databases with millions of labeled examples (e.g., ImageNet, Places), the state of the art in computer vision is advancing rapidly. Computer vision is now present among many commercial products, such as digital cameras, web applications, security applications, etc.

The performances achieved by convNets are remarkable and constitute the state of the art on many recognition tasks. But why it works so well? what is the nature of the internal representation learned by the network? I will show that the internal representation can be interpretable. In particular, object detectors emerge in a scene classification task. Then, I will show that an ambient audio signal can be used as a supervisory signal for learning visual representations. We do this by taking advantage of the fact that vision and hearing often tell us about similar structures in the world, such as when we see an object and simultaneously hear it make a sound. We train a convNet to predict ambient sound from video frames, and we show that, through this process, the model learns a visual representation that conveys significant information about objects and scenes.

See Also:

Download slides icon Download slides: deeplearning2016_torralba_learning_see_01.pdf (13.7┬áMB)

Help icon Streaming Video Help

Link this page

Would you like to put a link to this lecture on your homepage?
Go ahead! Copy the HTML snippet !

Reviews and comments:

Comment1 Krishna, December 12, 2016 at 10:39 a.m.:

First half was really amazing....

Comment2 Devoted CG, December 4, 2022 at 1:32 p.m.:

Great lecture. I think you'd be also interested in visiting https://devotedcg.com/ to get custom game 2D and 3D game assets there.

Write your own review or comment:

make sure you have javascript enabled or clear this field: