Mask-based Light Field Capture and Display

author: Douglas Lanman, MIT Media Lab, School of Architecture + Planning, Massachusetts Institute of Technology, MIT
published: Jan. 12, 2011,   recorded: December 2010,   views: 691
Categories
You might be experiencing some problems with Your Video player.

Slides

Slides
0:00 Mask-based Light Field Capture and Display
0:23 Problem Statement
1:13 Challenges
2:06 Representing Light Transport
3:23 Representing Occlusion - 1
3:31 Representing Occlusion - 2
4:26 Conventional vs. Light Field Cameras
7:01 Conventional vs. Light Field Displays
8:41 Previous Approaches
10:55 Proposed Approach
13:14 Outline - 1
13:22 Light Field Photography
14:17 Light Field Analysis of Barrier Cameras
15:14 Modeling Sensors in the Frequency Domain
17:05 Heterodyne Light Field Cameras - 1
17:27 Heterodyne Light Field Cameras - 2
17:28 Which Heterodyne Mask to Use? - 1
18:26 Which Heterodyne Mask to Use? - 2
18:27 Which Heterodyne Mask to Use? - 3
18:34 General Tiled-broadband Patterns
19:20 Specific Choice: Tiled-MURA
19:45 Benefits of Heterodyne Masks
20:07 Implementation
20:18 Results: Mask-based Light Field Capture - 1
20:41 Results: Mask-based Light Field Capture - 2
20:56 Results: Sub-Aperture Images
21:07 Results: Digital Image Refocusing
21:30 Outline - 3
21:44 Limitations of Traditional Visual Hull
22:25 Shadowgram-based Visual Hull
23:52 Simultaneous Projections?
24:13 Naive Solution: Pinhole Analysis
24:27 Implementation - 1
25:09 Pinhole Array Results
25:13 Tiled-MURA Results: Sensor Image
25:34 Tiled-MURA Results: Shadowgrams
25:43 Pinhole Array Results: Visual Hull
26:04 Outline - 4
26:17 How to Capture Touch and Gestures?
27:23 Optical Touch and Gesture Recognitions
28:15 Designing a Thin, Depth-Sensing LCD - 1
29:56 Designing a Thin, Depth-Sensing LCD - 2
30:11 Implementation - 2
30:36 Model Viewer Application
31:58 Light Field Transfer Application
32:41 Outline - 5
33:21 Parallax Barrier Light Field Displays
33:25 Target 4D Light Field - 1
33:32 Parallax Barrier Light Field Displays
33:51 Target 4D Light Field - 1
34:20 Target 4D Light Field - 2
34:37 Parallax Barrier Front Mask
34:50 Parallax Barrier Rear Mask
35:22 Parallax Barrier Front Mask
35:33 Parallax Barrier Rear Mask
35:35 Light Field Analysis of Barriers
37:32 Time-Multiplexing using Shifted Pinholes
38:48 Content-Adaptive Parallax Barriers - 1
39:21 Content-Adaptive Parallax Barriers - 2
39:30 Content-Adaptive Parallax Barriers - 3
39:32 Content-Adaptive Parallax Barriers - 2
39:43 Content-Adaptive Parallax Barriers - 3
39:59 Content-Adaptive Parallax Barriers - 4
40:02 Content-Adaptive Parallax Barriers - 3
40:29 Content-Adaptive Parallax Barriers - 4
40:53 Content-Adaptive Parallax Barriers - 5
40:55 Target 4D Light Field
40:57 Content-Adaptive Parallax Barriers - 5
41:12 Target 4D Light Field
41:14 Optimization: Iteration 1
41:26 Optimization: Iteration 10
41:27 Optimization: Iteration 20
41:28 Optimization: Iteration 30
41:29 Optimization: Iteration 40
41:30 Optimization: Iteration 50
41:30 Optimization: Iteration 60
41:31 Optimization: Iteration 70
41:31 Optimization: Iteration 80
41:32 Optimization: Iteration 90
41:39 Content-Adaptive Front Mask (1 of 9)
41:52 Content-Adaptive Rear Mask (1 of 9)
41:57 Emitted 4D Light Field
42:03 Content-Adaptive Rear Mask (1 of 9)
42:04 Emitted 4D Light Field
42:20 Benefits of Content-Adaptation
42:32 Implementation
42:37 Motion Parallax
43:16 Increasing Brightness and Refresh Rate - 1
43:41 Increasing Brightness and Refresh Rate - 2
43:42 Outline - 6
43:43 Contributions

Related content

Report a problem or upload files

If you have found a problem with this lecture or would like to send us extra material, articles, exercises, etc., please use our ticket system to describe your request and upload the data.
Enter your e-mail into the 'Cc' field, and we will keep you updated with your request's status.
Lecture popularity: You need to login to cast your vote.
 
    Delicious Bibliography

Description

This talk describes light-efficient methods for capturing and displaying 3D images using thin, optically-attenuating masks. Light transport is modeled, under geometrical optics, as a 4D function: the light field; this function records the amount of light traveling through any point along any direction. Conventional photographs only record a 2D projection of the incident light field. Each image point is produced by integrating over the full hemisphere of incidence angles. Similarly, conventional displays only approximate a diffuse surface, where the amount of light leaving any point is constant over the full hemisphere of viewing angles. Thus, conventional cameras and displays only support 2D images, for which the perception of scene depth is lost. 3D images can be captured and displayed by including masks in conventional camera and display architectures. Parallax barriers are one example; a mask containing a uniform array of slits is placed slightly in front of a conventional display. This mask only allows certain disjoint display regions to be visible from each viewpoint. 3D image capture is achieved by placing a similar mask close to a sensor. In both cases, 3D images come at the cost of decreased resolution and brightness. This talk will present a first-principles analysis of dual-layer camera and display architectures, wherein the first layer is a conventional sensor or display and the second layer is a mask. Novel masks are developed that facilitate 3D image capture and display, outperforming conventional parallax barriers in terms of total light transmission and light field resolution. For 3D capture, a family of static, periodic, non-adaptive masks is derived from a frequencydomain analysis. For 3D display, a linear algebraic analysis reveals a set of time-multiplexed, aperiodic, adaptive masks. Four motivating applications are presented: digital photography, single-shot visual hull reconstruction, depth-sensing LCDs, and 3D display using dual-stacked LCDs.

Link this page

Would you like to put a link to this lecture on your homepage?
Go ahead! Copy the HTML snippet !

Write your own review or comment:

make sure you have javascript enabled or clear this field: