A comparison of hypothesis testing methods for ODE models of biochemical systems
Description
In this talk we present a comparison of different methods of testing alternative
hypotheses expressed using ODE models of biochemical systems.
We investigated applicability, limitations and stability of a range of hypotheses
testing methods including maximum likelihood based information criteria,
local deterministic approximations around maximum a posteriori estimates
(Laplace approximations) for computing marginal likelihoods, importance sampling
based marginal likelihood estimators, and a path sampling estimator built
upon the principles of thermodynamic integration.
We demonstrate that in the cases where models are linear in the parameter
space, Laplace approximations provide a fast and stable estimate of the
marginal likelihoods required for computing Bayes factors. This estimate, however,
fails when the models have non-trivial parameter posteriors. We reject
common importance sampling estimators as they produce very unstable estimates
in practical cases (relative errors of the estimates vary from 40% to 600%
depending on the particular example used). We demonstrate that the annealed
importance sampling estimator of the marginal likelihoods and path sampling
methods produce very good estimates even in non-trivial cases (relative error
within 1%-8%).
Maximum likelihood information criteria often produce the correct ordering
of the hypotheses. These methods, however, do not produce a quantitative
measure of model preference (odds) and sometimes even fail, preferring a more
complex model over the true one, and there is no general method to detect such
a failure.
The study is performed over realistically sized ODE models of biochemical
systems using simulated data sets.
| Slides | |
| 0:00 | A Comparison of Hypotheses Testing Methods for ODE Models of Biochemical Systems |
| 0:12 | Uncertainty - part 1 |
| 0:27 | Uncertainty - part 2 |
| 0:29 | Uncertainty - part 3 |
| 0:33 | Noisy Data - part 1 |
| 0:56 | Noisy Data - part 2 |
| 1:10 | Noisy Data - part 3 |
| 1:21 | Noisy Data - part 4 |
| 1:40 | Problems With the “Best Fit” Approach |
| 2:19 | Bayesian Hypotheses Testing - part 1 |
| 2:40 | Bayesian Hypotheses Testing - part 2 |
| 2:56 | Bayesian Hypotheses Testing - part 3 |
| 3:06 | Marginal Likelihood Estimators - part 1 |
| 3:12 | Marginal Likelihood Estimators - part 2 |
| 3:25 | Marginal Likelihood Estimators - part 3 |
| 3:26 | Marginal Likelihood Estimators - part 4 |
| 3:56 | Marginal Likelihood Estimators - part 5 |
| 3:57 | Marginal Likelihood Estimators - part 6 |
| 4:11 | Marginal Likelihood Estimators - part 7 |
| 4:16 | Marginal Likelihood Estimators - part 8 |
| 4:28 | Marginal Likelihood Estimators - part 9 |
| 4:36 | Models - part 1 |
| 4:52 | Models - part 2 |
| 5:15 | Models - part 3 |
| 5:29 | Models - part 4 |
| 5:51 | Models - part 5 |
| 6:06 | Data |
| 6:47 | Prior Arithmetic Mean - part 1 |
| 7:08 | Prior Arithmetic Mean - part 2 |
| 8:05 | Posterior Harmonic Mean - part 1 |
| 8:47 | Posterior Harmonic Mean - part 2 |
| 9:08 | Posterior Harmonic Mean - part 3 |
| 9:10 | Posterior Harmonic Mean - part 4 |
| 9:19 | Posterior Harmonic Mean - part 5 |
| 9:22 | Posterior Harmonic Mean - part 6 |
| 9:30 | Posterior Harmonic Mean - part 7 |
| 9:59 | Annealed Importance Sampling - part 1 |
| 11:02 | Annealed Importance Sampling - part 2 |
| 11:04 | Annealed Importance Sampling - part 3 |
| 11:04 | Annealed Importance Sampling - part 4 |
| 11:05 | Annealed Importance Sampling - part 5 |
| 11:06 | Annealed Importance Sampling - part 6 |
| 11:11 | Annealing-Melting Integration - part 1 |
| 11:35 | Annealing-Melting Integration - part 2 |
| 11:54 | Annealing-Melting Integration - part 3 |
| 11:57 | Annealing-Melting Integration - part 4 |
| 11:57 | Annealing-Melting Integration - part 5 |
| 11:58 | Annealing-Melting Integration - part 6 |
| 11:59 | Annealing-Melting Integration - part 7 |
| 12:00 | Standard Deviation of the Estimates |
| 15:30 | Model Ranking - part 1 |
| 16:04 | Model Ranking - part 2 |
| 16:08 | Model Ranking - part 3 |
| 16:10 | Model Ranking - part 4 |
| 16:12 | Model Ranking - part 5 |
| 16:19 | Conclusions - part 1 |
| 17:22 | Conclusions - part 2 |
| 17:45 | Future Developments |
| 18:09 | Acknowledgements |
Lecture rating
| People found this lecture: | ||
| Worth seeing | ||
| because it is: | ||
| Valuable and informative | ||
| Well presented | ||
| Easily understandable | ||
| Acceptably recorded | ||
| You need to login to cast your vote. | ||
Report a problem or upload files
If you have found a problem with this lecture or would like to send us extra material, articles, exercises, etc., please use our ticket system to describe your request and upload the data.Enter your e-mail into the 'Cc' field, and we will keep you updated with your request's status.
Related content
Link this page
Would you like to put a link to this lecture on your homepage?Go ahead! Copy the HTML snippet !




