event thumbnail image
NIPS '07 Workshop on Music, Brain and Cognition
Pascal

What/When Causal Expectation Modelling in Monophonic Pitched and Percussive Audio

author: Amaury Hazan, Music Technology Group, Universitat Pompeu Fabra

Description

A causal system for representing a musical stream and generating further expected events is presented. Starting from an auditory front-end which extracts low-level (e.g. spectral shape, MFCC, pitch) and mid-level features such as onsets and beats, an unsupervised clustering process builds and maintains a set of symbols aimed at representing musical stream events using both timbre and time descriptions. The time events are represented using inter-onset intervals relative to the beats. These symbols are then processed by an expectation module based on Predictive Partial Match, a multiscale technique based on N-grams. To characterise the system capacity to generate an expectation that matches its transcription, we use a weighted average F-measure, that takes into account the uncertainty associated with the unsupervised encoding of the musical sequence. The potential of the system is demonstrated in the case of processing audio streams which contain drum loops or monophonic singing voice. In preliminary experiments, we show that the induced representation is useful for generating expectation patterns in a causal way. During exposure, we observe a globally decreasing prediction entropy combined with structure-specific variations.

You might be experiencing some problems with Your Video player.
Slides
0:00 What/when causal expectation modeling in monophonc pitched melodies and percussive audio
0:32 Outline
0:54 Goals
1:28 Background
2:12 System Design
3:06 Feature Extraction Module
3:54 Feature Extraction Module: Output
4:08 Dimensionality Redaction Module
4:51 Bootstrap GMM+EM Grid
6:02 GMM Grid: Example (Drums)
6:21 Running State: Online K-Means
6:48 Dimensionality Reduction Module: Output (Drums)
7:28 Dimensionality Reduction Module: Output (Sung Melody) - 1
7:51 Dimensionality Reduction Module: Output (Sung Melody) - 2
8:01 Dimensionality Reduction Module: Output (Sung Melody) - 3
8:06 Prediction by Partial Match
9:31 Next Event Prediction Module: Output
10:14 Evaluation
10:36 Next Event Prediction Module: Output
11:04 Evaluation
11:27 - Questions
14:04 Expectation Entropy - 1
14:57 Expectation Entropy - 2
15:10 Concatenative Synteshis
15:18 Discussion
16:05 Summary
17:19 Thanks

Lecture rating

People found this lecture:
Worth seeing
because it is:
 Valuable and informative
Well presented
Easily understandable
Acceptably recorded
You need to login to cast your vote.

Report a problem or upload files

If you have found a problem with this lecture or would like to send us extra material, articles, exercises, etc., please use our ticket system to describe your request and upload the data.
Enter your e-mail into the 'Cc' field, and we will keep you updated with your request's status.

Link this page

Would you like to put a link to this lecture on your homepage?
Go ahead! Copy the HTML snippet !

Write your own review or comment: