A Stochastic Memoizer for Sequence Data

author: Frank Wood, Gatsby Computational Neuroscience Unit, University College London
published: Aug. 26, 2009,   recorded: June 2009,   views: 6475


Related Open Educational Resources

Related content

Report a problem or upload files

If you have found a problem with this lecture or would like to send us extra material, articles, exercises, etc., please use our ticket system to describe your request and upload the data.
Enter your e-mail into the 'Cc' field, and we will keep you updated with your request's status.
Lecture popularity: You need to login to cast your vote.


We propose an unbounded-depth, hierarchical, Bayesian nonparametric model for discrete sequence data. This model can be estimated from a single training sequence, yet shares statistical strength between subsequent symbol predictive distributions in such a way that predictive performance generalizes well. The model builds on a specific parameterization of an unbounded-depth hierarchical Pitman-Yor process. We introduce analytic marginalization steps (using coagulation operators) to reduce this model to one that can be represented in time and space linear in the length of the training sequence. We show how to perform inference in such a model without truncation approximation and introduce fragmentation operators necessary to do predictive inference. We demonstrate the sequence memoizer by using it as a language model, achieving state-of-the-art results.

See Also:

Download slides icon Download slides: icml09_wood_sms_01.pdf (983.5 KB)

Download slides icon Download slides: icml09_wood_sms_01.ppt (1.9 MB)

Help icon Streaming Video Help

Link this page

Would you like to put a link to this lecture on your homepage?
Go ahead! Copy the HTML snippet !

Reviews and comments:

Comment1 Memming, November 1, 2012 at 7:32 p.m.:

The talk only covers a part of the paper; the paper has much more information.

The slides on the website are based on the ppt file which has corrupted fonts. See the pdf version.

Write your own review or comment:

make sure you have javascript enabled or clear this field: