Tuning Optimizers for Time-Constrained Problems using Reinforcement Learning.

author: Paul Ruvolo, University of San Diego
published: Dec. 20, 2008,   recorded: December 2008,   views: 3144

Related Open Educational Resources

Related content

Report a problem or upload files

If you have found a problem with this lecture or would like to send us extra material, articles, exercises, etc., please use our ticket system to describe your request and upload the data.
Enter your e-mail into the 'Cc' field, and we will keep you updated with your request's status.
Lecture popularity: You need to login to cast your vote.


Many popular optimization algorithms, like the Levenberg-Marquardt algorithm (LMA), use heuristic-based “controllers” that modulate the behavior of the op- timizer during the optimization process. For example, in the LMA a damping parameter λ is dynamically modified based on a set of rules that were developed using various heuristic arguments. Here we show that a modern reinforcement learning technique utilizing a very simple state space can dramatically improve the performance of general purpose optimizers, like the LMA, on problems where the number of function evaluations allowed is constrained by a budget. Results are given on both classical non-linear optimization problems as well as a difficult computer vision task. Interestingly the controllers learned for a particular opti- mization domain work well on other optimization domains. Thus, the controller appeared to have extracted optimization rules that were not just domain specific but generalized across a range of optimization domains.

Link this page

Would you like to put a link to this lecture on your homepage?
Go ahead! Copy the HTML snippet !

Write your own review or comment:

make sure you have javascript enabled or clear this field: