0.25
0.5
0.75
1.25
1.5
1.75
2
Online Learning and Bregman Divergences
Published on Feb 25, 200711740 Views
L 1: Introduction to Online Learning (Predicting as good as the best expert, Predicting as good as the best linear combination of experts, Additive versus multiplicative family of updates)\\ L 2: Breg
Related categories
Chapter list
Online Learning and Bregman Divergences00:06
Bregman Divergences [Br,CL,Cs]01:01
Exponential Family of Distributions01:42
Expectation parameter04:10
Exponential Family of Distributions04:48
Expectation parameter04:53
Primal & Dual Parameters05:55
Gaussian (unit variance)06:50
Primal & Dual Parameters08:00
Gaussian (unit variance)08:15
Bernoulli08:39
Poisson10:40
Manfred1_Page_2512:07
Exponential Family of Distributions15:55
Expectation parameter16:05
Bregman Div. as Rel. Ent. between Distributions16:26
Area unchanged When Slide Flipped17:23
Area unchanged When Slide Flipped18:55
Area unchanged When Slide Flipped19:25
Area unchanged When Slide Flipped19:34
Dual divergence for Bernoulli19:56
Area unchanged When Slide Flipped20:55
Dual divergence for Bernoulli20:58
Area unchanged When Slide Flipped21:30
Dual divergence for Bernoulli21:34
Dual divergence for Poisson22:35
Dual matching loss for sigmoid transfer func.24:02
Example: Gaussian density estimation27:15
Derivation of Updates29:12
Example: Gaussian density estimation29:32
Derivation of Updates29:53
On-line Algorithm [AW]31:52
Alternate Motivation of Same On-Line Update32:27
Alternate Motivation of Same On-Line Update34:14
Shrinkage Towards Initial34:46
Shrinkage Towards Initial34:48
Key Lemma [AW]37:29
Main Theorem37:55
Bounds for the Forward Algorithm38:15
Shrinkage Towards Initial39:07
Bounds for the Forward Algorithm39:43
Why Bregman divergences?42:17
General setup of on-line learning43:15
Minimax Algorithm for T Trials43:52
Gaussian45:51
Last-step Minimax47:35
Minimax Algorithm for T Trials47:39
Last-step Minimax48:05
Last-step Minimax: Bernoulli49:23
Minimax Algorithm for T Trials51:26
Synopsis of methods52:12
Minimax Algorithm for T Trials52:59
Gaussian53:39
Synopsis of methods54:45
Content of this tutorial56:47
Simple conversions56:54
Expected loss bounds [HW]57:00
Expected loss bounds [HW]01:00:24
Expected loss bounds [HW]01:01:39
Expected loss bounds [HW]01:02:20
Tail bound [CCG]01:03:20
Application: Adaptive Channel Equalization01:04:22
Application: Caching [GBW]01:07:02
Caching Policies01:07:58
Which Policy to Choose?01:09:02
Characteristics Vary with Time01:09:44
Randomly Permuted Request Stream01:11:12
Characteristics Vary with Time01:11:40
Randomly Permuted Request Stream01:11:48
Want “Adaptive” Policy01:12:27