PAC-Bayes Analysis of Classification

author: John Shawe-Taylor, University College London

Description

The lecture will introduce the PAC Bayes approach to the statistical analysis of learning. After some historical introduction, the key theorems will be covered. We will then consider some applications including for Support Vector Machines and novelty detection. A discussion of the status of the prior in the approach will lead to an investigation of how learning the prior can be used in practical applications. Discussions of further extensions of the approach will conclude the presentation.

Categories

Top: Computer Science: Machine Learning

You might be experiencing some problems with Your Video player.

*Slides*
0:00	PAC - Bayes Analysis of Classification
1:34	Aims - 1
1:35	Aims - 2
1:39	Aims - 3
1:40	Aims - 4
2:07	Aims - 5
2:09	Aims - 6
2:42	Outline
3:48	General perspectives - 1
5:06	General perspectives - 2
5:19	General perspectives - 3
5:48	General perspectives - 4
6:07	General perspectives - 5
6:29	Historical notes: Frequentist approach - 1
6:52	Historical notes: Frequentist approach - 2
7:04	Historical notes: Frequentist approach - 3
8:30	Historical notes: Frequentist approach - 4
10:01	Historical notes: Frequentist approach - 5
10:08	Historical notes: Bayesian approach - 1
11:41	Historical notes: Bayesian approach - 2
13:04	- Questions
15:28	Historical notes: Bayesian approach - 4
16:27	- Questions
19:01	Evidence and generalisation - 1
19:59	Evidence and generalisation - 2
20:34	Evidence and generalisation - 3
21:08	Evidence and generalisation - 4
21:40	PAC-Bayes theorem - 1
22:10	PAC-Bayes theorem - 2
22:31	PAC-Bayes theorem - 3
22:56	PAC-Bayes theorem - 4
23:11	Definitions for main result: Prior and posterior distributions - 1
24:19	Definitions for main result: Prior and posterior distributions - 2
25:14	Definitions for main result: Prior and posterior distributions - 3
27:20	Definitions for main result: Error measures - 1
28:15	Definitions for main result: Error measures - 2
28:24	Definitions for main result: Error measures - 3
28:56	Definitions for main result: Error measures - 4
29:23	- Questions
34:02	Definitions for main result: Assessing the posterior - 2
34:45	Definitions for main result: Generalisation error
36:10	- Questions
39:59	- Questions
41:40	Finite classes - 1
43:00	Definitions for main result: Assessing the posterior - 2
43:32	- Questions
46:02	Ingredients of proof (1/3) - 1
46:05	PAC-Bayes theorem - 5
46:21	Ingredients of proof (1/3) - 1
47:20	Ingredients of proof (1/3) - 2
49:15	Ingredients of proof (2/3) - 1
49:40	Ingredients of proof (2/3) - 2
51:00	Ingredients of proof (3/3) - 1
51:29	Ingredients of proof (2/3) - 3
51:47	Ingredients of proof (3/3) - 1
51:50	Ingredients of proof (2/3) - 3
52:09	Ingredients of proof (3/3) - 2
52:16	Ingredients of proof (2/3) - 3
52:26	Ingredients of proof (3/3) - 2
52:54	Finite classes - 1
53:37	Finite classes - 2
53:58	Other extensions/applications - 1
54:13	Other extensions/applications - 2
54:50	Other extensions/applications - 3
55:05	Other extensions/applications - 4
55:16	Linear classifiers and SVMs - 1
55:24	Linear classifiers and SVMs - 2
55:28	Linear classifiers and SVMs - 3
55:33	Linear classifiers and SVMs - 4
56:26	Linear classifiers - 1
56:38	Linear classifiers - 2
56:41	Linear classifiers - 3
57:03	PAC-Bayes bound for SVM (1/2) - 1
57:09	PAC-Bayes bound for SVM (1/2) - 2
57:23	PAC-Bayes bound for SVM (1/2) - 3
57:26	PAC-Bayes bound for SVM (1/2) - 4
57:35	PAC-Bayes bound for SVM (2/2) - 1
57:45	PAC-Bayes bound for SVM (2/2) - 2
57:56	PAC-Bayes bound for SVM (2/2) - 3
59:28	PAC-Bayes bound for SVM (2/2) - 4
59:58	PAC-Bayes bound for SVM (2/2) - 5
60:10	PAC-Bayes bound for SVM (2/2) - 6
61:41	PAC-Bayes bound for SVM (2/2) - 7
62:10	PAC-Bayes bound for SVM (2/2) - 8
62:11	PAC-Bayes bound for SVM (2/2) - 9
62:12	PAC-Bayes bound for SVM (2/2) - 10
62:41	PAC-Bayes bound for SVM (2/2) - 6
62:52	PAC-Bayes bound for SVM (2/2) - 10
62:55	PAC-Bayes bound for SVM (2/2) - 11
62:59	PAC-Bayes bound for SVM (2/2) - 12
63:01	PAC-Bayes bound for SVM (2/2) - 13
63:04	Learning the prior (1/3) - 1
63:23	Learning the prior (1/3) - 2
63:44	Learning the prior (1/3) - 3
64:28	Learning the prior (1/3) - 4
64:33	Learning the prior (1/3) - 5
64:34	New prior for the SVM (3/3) - 1
64:43	New prior for the SVM (3/3) - 2
64:53	New prior for the SVM (3/3) - 3
65:10	New prior for the SVM (3/3) - 4
65:32	New bound for the SVM (2/3) - 1
65:39	New bound for the SVM (2/3) - 2
65:41	New bound for the SVM (2/3) - 3
65:43	New bound for the SVM (2/3) - 4
65:59	New bound for the SVM (2/3) - 5
66:08	New bound for the SVM (2/3) - 6
66:13	New bound for the SVM (2/3) - 7
67:01	New bound for the SVM (2/3) - 8
67:03	Model selection with the new bound: Setup - 1
67:29	Model selection with the new bound: Setup - 2
67:30	Model selection with the new bound: Setup - 3
67:50	Model selection with the new bound: Setup - 4
67:59	Model selection with the new bound: Setup - 5
68:22	Model selection with the new bound: Results
69:09	Tightness of the new bound
69:33	Model selection with the new bound: Results
69:39	Tightness of the new bound
69:50	Prior-SVM - 1
70:24	Prior-SVM - 2
70:28	Prior-SVM - 3
70:50	Prior-SVM - 4
70:54	Bound for p-SVM - 1
71:02	Bound for p-SVM - 2
71:05	Bound for p-SVM - 3
71:11	Bound for p-SVM - 4
71:16	Prior-SVM - 5
71:28	Prior-SVM - 4
71:37	Prior-SVM - 5
71:52	Prior-SVM - 6
71:53	Prior-SVM - 7
71:59	Prior-SVM - 8
72:22	Bound for prior-SVM - 1
72:23	Bound for prior-SVM - 2
72:24	Bound for prior-SVM - 3
73:50	Model selection with p-SVM
74:23	Tightness of the bound with p-SVM
75:04	- Questions
76:52	Concluding remarks - 1
77:21	Concluding remarks - 2
77:37	Concluding remarks - 3
77:41	Concluding remarks - 4
78:05	Concluding remarks - 5
78:11	Concluding remarks - 6
78:20	- Questions

Lecture rating

People found this lecture:
Worth seeing
because it is:
Valuable and informative
Well presented
Easily understandable
Acceptably recorded
You need to login to cast your vote.

Report a problem or upload files

If you have found a problem with this lecture or would like to send us extra material, articles, exercises, etc., please use our ticket system to describe your request and upload the data.
Enter your e-mail into the 'Cc' field, and we will keep you updated with your request's status.