Probabilistic Interpretation of Quasi-Newton Methods

author: Philipp Hennig, Max Planck Institute for Intelligent Systems, Max Planck Institute
published: Jan. 15, 2013,   recorded: December 2012,   views: 3363


Related Open Educational Resources

Related content

Report a problem or upload files

If you have found a problem with this lecture or would like to send us extra material, articles, exercises, etc., please use our ticket system to describe your request and upload the data.
Enter your e-mail into the 'Cc' field, and we will keep you updated with your request's status.
Lecture popularity: You need to login to cast your vote.


This talk is a case-study about the utility of probabilistic formulations for numerical mathematics. I present a recent result showing that quasi-Newton methods can be interpreted as performing Gaussian (least-squares) regression on the Hessian of the objective function, using a particular noise process to keep uncertainty constant, and a non-obvious structured prior which ignores the duality between vectors and co-vectors. This insight connects these numerical methods to important areas of machine learning (regression) and control (Kalman filters). It allows cross-fertilization: Better numerical algorithms can be built using existing knowledge from machine learning, and machine learning can benefit from a new structured prior model allowing linear-cost inference on matrix-valued operators. Arguing for more and closer interaction between the fields of learning and numerical mathematics, I also point out some challenges arising from cultural differences between these communities.

Link this page

Would you like to put a link to this lecture on your homepage?
Go ahead! Copy the HTML snippet !

Write your own review or comment:

make sure you have javascript enabled or clear this field: