Machine Learning Summer School on Theory and Practice of Computational Learning

PAC-Bayes Analysis: Background and Applications

author: John Shawe-Taylor, Centre for Computational Statistics and Machine Learning, University College London

You might be experiencing some problems with Your Video player.

*Slides*
0:00	PAC-Bayes Analysis: Background and Applications
1:02	Aims
1:18	Outline
1:23	General perspectives (1)
1:59	General perspectives (2)
2:14	General perspectives (3)
2:20	General perspectives (4)
2:28	General perspectives (5)
2:42	Historical notes: Frequentist approach (1)
2:58	Historical notes: Frequentist approach (2)
3:17	Historical notes: Frequentist approach (3)
3:44	Historical notes: Frequentist approach (4)
4:19	Historical notes: Frequentist approach (5)
4:31	Historical notes: Bayesian approach (1)
5:57	Historical notes: Bayesian approach (2)
6:35	Historical notes: Bayesian approach (3)
7:37	Historical notes: Bayesian approach (4)
7:51	Version space: evidence
8:38	Evidence and generalisation (1)
8:47	Evidence and generalisation (2)
9:00	Version space: evidence
9:09	Evidence and generalisation (1)
9:10	Evidence and generalisation (2)
9:11	Evidence and generalisation (3)
9:20	Evidence and generalisation (4)
9:40	PAC-Bayes Theorem (1)
10:14	PAC-Bayes Theorem (2)
10:24	PAC-Bayes Theorem (3)
10:34	PAC-Bayes Theorem (4)
10:39	Definitions for main result - Prior and posterior distributions (1)
10:56	Definitions for main result - Prior and posterior distributions (2)
11:19	Definitions for main result - Prior and posterior distributions (3)
12:11	Definitions for main result - Error measures (1)
12:26	Definitions for main result - Error measures (2)
12:32	Definitions for main result - Error measures (3)
12:45	Definitions for main result - Error measures (4)
13:35	Definitions for main result - Assessing the posterior (1)
14:03	Definitions for main result - Assessing the posterior (2)
14:27	Definitions for main result - Generalisation error
15:17	Definitions for main result - Assessing the posterior (2)
15:31	Definitions for main result - Generalisation error
16:04	PAC-Bayes Theorem
18:54	Finite Classes (1)
19:14	PAC-Bayes Theorem
19:30	Finite Classes (1)
19:44	Finite Classes (2)
20:11	Linear classifiers and SVMs (1)
20:17	Linear classifiers and SVMs (2)
20:19	Linear classifiers and SVMs (3)
20:21	Linear classifiers and SVMs (4)
20:40	Linear classifiers (1)
21:16	Linear classifiers (2)
21:18	Linear classifiers (3)
21:46	PAC-Bayes Bound for SVM (1/2) (1)
21:48	PAC-Bayes Bound for SVM (1/2) (2)
21:52	PAC-Bayes Bound for SVM (1/2) (3)
21:54	PAC-Bayes Bound for SVM (1/2) (4)
21:58	PAC-Bayes Bound for SVM (2/2) (1)
22:08	PAC-Bayes Bound for SVM (2/2) (2)
22:12	PAC-Bayes Bound for SVM (2/2) (3)
23:23	PAC-Bayes Bound for SVM (2/2) (4)
23:36	PAC-Bayes Bound for SVM (2/2) (5)
23:42	PAC-Bayes Bound for SVM (2/2) (6)
23:51	PAC-Bayes Bound for SVM (2/2) (7)
23:59	PAC-Bayes Bound for SVM (2/2) (8)
24:09	PAC-Bayes Bound for SVM (2/2) (9)
24:35	PAC-Bayes Bound for SVM (2/2) (10)
24:38	PAC-Bayes Bound for SVM (2/2) (11)
24:41	PAC-Bayes Bound for SVM (2/2) (12)
24:45	PAC-Bayes Bound for SVM (2/2) (13)
24:49	PAC-Bayes Bound for SVM (2/2) (14)
24:50	PAC-Bayes Bound for SVM (2/2) (15)
24:53	PAC-Bayes Bound for SVM (2/2) (16)
24:56	Form of the SVM bound (1)
24:58	Form of the SVM bound (2)
25:37	Gives SVM Optimisation
25:44	Slack variable conversion
25:57	Learning the prior (1/3) (1)
26:11	Learning the prior (1/3) (2)
26:15	Learning the prior (1/3) (3)
26:18	Learning the prior (1/3) (4)
26:21	Learning the prior (1/3) (5)
26:35	Tightness of the new bound
27:02	Model Selection with the new bound: results
27:14	Model selection with p-SVM
27:15	Tightness of the new bound
27:34	Model Selection with the new bound: results
28:05	Model selection with p-SVM
28:42	Tightness of the bound with p-SVM
29:02	Maximum entropy learning (1)
29:32	Maximum entropy learning (2)
30:07	Maximum entropy learning (3)
31:38	Posterior distribution Q(w) (1)
32:05	Posterior distribution Q(w) (2)
33:14	Error expression
33:41	Error expression proof
33:59	Generalisation error
34:25	Error expression proof
34:26	Error expression
34:31	Error expression proof
34:33	Generalisation error
34:43	Base result (1)
34:51	Base result (2)
35:01	Base result (3)
35:46	Interpretation (1)
35:49	Interpretation (2)
36:03	Interpretation (3)
36:08	Boosting the bound (1)
36:42	Boosting the bound (2)
36:50	Full result (1)
37:00	Full result (2)
37:25	Algorithmics (1)
37:48	Algorithmics (2)
37:53	Dual optimisation (1)
38:21	Dual optimisation (2)
38:25	Dual optimisation (3)
38:27	Dual optimisation (4)
38:34	Results: effect of varying T
38:45	Results
39:08	Gaussian Process Regression (1)
39:46	Gaussian Process Regression (2)
40:27	Gaussian Process Regression (3)
40:52	Gaussian Process Regression (4)
41:22	Applying PAC-Bayes theorem (1)
41:36	Applying PAC-Bayes theorem (2)
42:18	Applying PAC-Bayes theorem (3)
42:49	GP Result
45:20	GP Experimental Results (1)
46:29	GP Experimental Results (2)
47:17	GP Experimental Results (3)
47:36	GP Experimental Results (4)
47:46	Stochastic Differential Equation Models (1)
49:03	Stochastic Differential Equation Models (2)
49:10	Variational approximation (1)
49:13	Variational approximation (2)
50:02	Girsanov change of measure (1)
50:04	Girsanov change of measure (2)
50:26	KL divergence
50:47	Variational approximation (1)
50:52	Variational approximation (2)
50:55	Algorithmics (1)
50:59	Algorithmics (2)
51:00	Error estimation (1)
51:36	Error estimation (2)
51:39	Error estimation (3)
51:52	Error estimation (4)
51:52	Generalisation analysis (1)
52:02	Generalisation analysis (2)
52:06	Generalisation analysis (3)
52:31	Error estimates (1)
52:45	Error estimates (2)
52:59	Error estimates (3)
53:01	Refining the distributions (1)
53:09	Refining the distributions (2)
53:15	Final result
53:37	Small scale experiment (1)
53:42	Small scale experiment (2)
53:44	Small scale experiment (3)
54:10	Conclusions (1)
54:12	Conclusions (2)
54:14	Conclusions (3)
54:21	Conclusions (4)
54:29	Conclusions (5)
54:31	Conclusions (6)
54:38	Conclusions (7)
54:41	Conclusions (8)
54:47	Conclusions (9)
57:45	- Questions

Lecture rating

People found this lecture:
Worth seeing
because it is:
Valuable and informative
Well presented
Easily understandable
Acceptably recorded
You need to login to cast your vote.

Report a problem or upload files

If you have found a problem with this lecture or would like to send us extra material, articles, exercises, etc., please use our ticket system to describe your request and upload the data.
Enter your e-mail into the 'Cc' field, and we will keep you updated with your request's status.