event thumbnail image
Probabilistic Modelling of Networks and Pathways

A comparison of hypothesis testing methods for ODE models of biochemical systems

author: Vladislav Vyshemirsky, University of Glasgow

Description

In this talk we present a comparison of different methods of testing alternative hypotheses expressed using ODE models of biochemical systems. We investigated applicability, limitations and stability of a range of hypotheses testing methods including maximum likelihood based information criteria, local deterministic approximations around maximum a posteriori estimates (Laplace approximations) for computing marginal likelihoods, importance sampling based marginal likelihood estimators, and a path sampling estimator built upon the principles of thermodynamic integration. We demonstrate that in the cases where models are linear in the parameter space, Laplace approximations provide a fast and stable estimate of the marginal likelihoods required for computing Bayes factors. This estimate, however, fails when the models have non-trivial parameter posteriors. We reject common importance sampling estimators as they produce very unstable estimates in practical cases (relative errors of the estimates vary from 40% to 600% depending on the particular example used). We demonstrate that the annealed importance sampling estimator of the marginal likelihoods and path sampling methods produce very good estimates even in non-trivial cases (relative error within 1%-8%). Maximum likelihood information criteria often produce the correct ordering of the hypotheses. These methods, however, do not produce a quantitative measure of model preference (odds) and sometimes even fail, preferring a more complex model over the true one, and there is no general method to detect such a failure. The study is performed over realistically sized ODE models of biochemical systems using simulated data sets.

You might be experiencing some problems with Your Video player.
Slides
0:00 A Comparison of Hypotheses Testing Methods for ODE Models of Biochemical Systems
0:12 Uncertainty - part 1
0:27 Uncertainty - part 2
0:29 Uncertainty - part 3
0:33 Noisy Data - part 1
0:56 Noisy Data - part 2
1:10 Noisy Data - part 3
1:21 Noisy Data - part 4
1:40 Problems With the “Best Fit” Approach
2:19 Bayesian Hypotheses Testing - part 1
2:40 Bayesian Hypotheses Testing - part 2
2:56 Bayesian Hypotheses Testing - part 3
3:06 Marginal Likelihood Estimators - part 1
3:12 Marginal Likelihood Estimators - part 2
3:25 Marginal Likelihood Estimators - part 3
3:26 Marginal Likelihood Estimators - part 4
3:56 Marginal Likelihood Estimators - part 5
3:57 Marginal Likelihood Estimators - part 6
4:11 Marginal Likelihood Estimators - part 7
4:16 Marginal Likelihood Estimators - part 8
4:28 Marginal Likelihood Estimators - part 9
4:36 Models - part 1
4:52 Models - part 2
5:15 Models - part 3
5:29 Models - part 4
5:51 Models - part 5
6:06 Data
6:47 Prior Arithmetic Mean - part 1
7:08 Prior Arithmetic Mean - part 2
8:05 Posterior Harmonic Mean - part 1
8:47 Posterior Harmonic Mean - part 2
9:08 Posterior Harmonic Mean - part 3
9:10 Posterior Harmonic Mean - part 4
9:19 Posterior Harmonic Mean - part 5
9:22 Posterior Harmonic Mean - part 6
9:30 Posterior Harmonic Mean - part 7
9:59 Annealed Importance Sampling - part 1
11:02 Annealed Importance Sampling - part 2
11:04 Annealed Importance Sampling - part 3
11:04 Annealed Importance Sampling - part 4
11:05 Annealed Importance Sampling - part 5
11:06 Annealed Importance Sampling - part 6
11:11 Annealing-Melting Integration - part 1
11:35 Annealing-Melting Integration - part 2
11:54 Annealing-Melting Integration - part 3
11:57 Annealing-Melting Integration - part 4
11:57 Annealing-Melting Integration - part 5
11:58 Annealing-Melting Integration - part 6
11:59 Annealing-Melting Integration - part 7
12:00 Standard Deviation of the Estimates
15:30 Model Ranking - part 1
16:04 Model Ranking - part 2
16:08 Model Ranking - part 3
16:10 Model Ranking - part 4
16:12 Model Ranking - part 5
16:19 Conclusions - part 1
17:22 Conclusions - part 2
17:45 Future Developments
18:09 Acknowledgements

Lecture rating

People found this lecture:
Worth seeing
because it is:
 Valuable and informative
Well presented
Easily understandable
Acceptably recorded
You need to login to cast your vote.

Report a problem or upload files

If you have found a problem with this lecture or would like to send us extra material, articles, exercises, etc., please use our ticket system to describe your request and upload the data.
Enter your e-mail into the 'Cc' field, and we will keep you updated with your request's status.

Link this page

Would you like to put a link to this lecture on your homepage?
Go ahead! Copy the HTML snippet !

Write your own review or comment: