14th International Conference on Artificial Intelligence and Statistics (AISTATS), Ft. Lauderdale 2011

Lightweight Implementations of Probabilistic Programming Languages Via Transformational Compilation

author: David Wingate, Computational Cognitive Science Group, Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, MIT
published: May 6, 2011, recorded: April 2011, views: 251

Slides

Slides
0:00	Lightweight Implementations of Probabilistic Programming Languages via Transformational Compilation
0:14	Implementing Probabilistic Programming LanguagesWithout the Agonizing Pain
0:20	Summary
1:17	Outline
1:34	Probabilistic Programming
1:42	The Big Idea
2:21	Distributions over Traces
3:34	Example: LDA
4:06	Nonparametrics
5:20	Example: ((H)DP)MM
7:03	Meta-Modeling
7:38	Lightweight PPL Implementations
7:43	Inference
8:09	Observation: Execution Trace (1)
9:18	Observation: Execution Trace (2)
9:38	Transformational Compilation
10:09	MCMC over Execution Traces
11:21	But What Name? (1)
12:31	But What Name? (2)
12:53	But What Name? (3)
13:44	Generating Names
15:19	Example: Geometric
15:58	Minimal Interpretative Overhead
17:25	New Inference Options
17:39	Different Inference Options
19:08	Dynamic Dependency Analysis (1)
20:11	Dynamic Dependency Analysis (2)
20:31	Dynamic Dependency Analysis (3)
21:18	Example: mesh inference
22:41	Summary
23:31	Thank you!

Report a problem or upload files

If you have found a problem with this lecture or would like to send us extra material, articles, exercises, etc., please use our ticket system to describe your request and upload the data.
Enter your e-mail into the 'Cc' field, and we will keep you updated with your request's status.

Lecture popularity: You need to login to cast your vote.

Description

We describe a general method of transforming arbitrary programming languages into probabilistic programming languages with straightforward MCMC inference engines. Random choices in the program are “named” with information about their position in an execution trace; these names are used in conjunction with a database holding values of random variables to implement MCMC inference in the space of execution traces. We encode naming information using lightweight source-to-source compilers. Our method enables us to reuse existing infrastructure (compilers, profilers, etc.) with minimal additional code, implying fast models with low development overhead. We illustrate the technique on two languages, one functional and one imperative: Bher, a compiled version of the Church language which eliminates interpretive overhead of the originalMIT-Church implementation, and Stochastic Matlab, a new open-source language.