## Some Mathematical Tools for Machine Learning

published: Feb. 25, 2007, recorded: August 2003, views: 7977

# Slides

# Related content

# Report a problem or upload files

If you have found a problem with this lecture or would like to send us extra material, articles, exercises, etc., please use our**to describe your request and upload the data.**

__ticket system__*Enter your e-mail into the 'Cc' field, and we will keep you updated with your request's status.*

# Description

These are lectures on some fundamental mathematics underlying many approaches and algorithms in machine learning. They are not about particular learning algorithms; they are about the basic concepts and tools upon which such algorithms are built. Often students feel intimidated by such material: there is a vast amount of "classical mathematics", and it can be hard to find the wood for the trees. The main topics of these lectures are Lagrange multipliers, functional analysis, some notes on matrix analysis, and convex optimization. I've concentrated on things that are often not dwelt on in typical CS coursework. Lots of examples are given; if it's green, it's a puzzle for the student to think about. These lectures are far from complete: perhaps the most significant omissions are probability theory, statistics for learning, information theory, and graph theory. I hope eventually to turn all this into a series of short tutorials. Please let me know of any errors, etc. ; :*from Chris Burges homepage : http://research.microsoft.com/~cburges Lecture contains: Lagrange multipliers: * Lagrange the Mathematician * Lagrange multipliers: an indirect approach can be easier * Multiple Equality Constraints * Multiple Inequality Constraints * Two points on a d-sphere * The Largest Parallelogram * Resource allocation * A convex combination of numbers is maximized by choosing the largest * The Isoperimetric problem * For fixed mean and variance, which univariate distribution has maximum entropy? * An exact solution for an SVM living on a simplex Notes on some Basic Statistics * Probabilities can be Counter-Intuitive (Simpson's paradox; the Monty Hall puzzle) * IID-ness: Measurement Error decreases as 1/sqrt{n} * Correlation versus Independence * The Ubiquitous Gaussian: Product of Gaussians is Gaussian Convolution of two Gaussians is a Gaussian Projection of a Gaussian is a Gaussian Sum of Gaussian random variables is a Gaussian random variables Uncorrelated Gaussian variables are also independent Maximum Likelihood Estimates for mean and covariance (prove required matrix identities) Aside: For 1-dim Laplacian, max. likelihood gives the median * Using cumulative distributions to derive densities Principal Component Analysis and Generalizations * Ordering by Variance * Does Grouping Change Things? * PCA Decorrelates the Samples * PCA gives Reconstruction with Minimal Mean Squared Error * PCA preserves Mutual Information on Gaussian data * PCA directions lie in the span of the data * PCA: second order moments only * The Generalized Rayleigh Quotient Non-orthogonal principal directions OPCA Fisher Linear Discriminant Multiple Discriminant Analysis Elements of Functional Analysis * High Dimensional Spaces * Is Winning Transitive? * Most of the Volume is Near the Surface: Cubes * Spheres in n-dimensions * Banach Spaces, Hilbert Spaces, Compactness * Norms * Useful Inequalities (Minkowski and Holder) * Vector Norms * Matrix Norms * The Hamming Norm * L1, L2, L_infty norms - is L0 a norm? * Example: Using a Norm as a Constraint in Kernel Algorithms*

# Link this page

Would you like to put a link to this lecture on your homepage?

Go ahead! Copy the HTML snippet !

## Reviews and comments:

John Smith, October 4, 2007 at 4:10 a.m.:Horrible audio, poor video makes this an unwatchable presentation.

victor, November 6, 2007 at 4:31 p.m.:I can't know what he says.

mrb, June 27, 2008 at 11:59 a.m.:Terrible, completely useless...

john, September 24, 2008 at 7:52 p.m.:very very bad video.... poor audio and video

Abhinav, April 13, 2009 at 2:58 a.m.:I don't have rtsp or rtmp protocols suuported =, how can I download on http? why isn't available on http itself?

Abhinav, April 13, 2009 at 2:59 a.m.:I don't have rtsp or rtmp protocols suuported , how can I download on http? why isn't it available on http itself?

## Write your own review or comment: