Annealed Sparsity via Adaptive and Dynamic Shrinking
published: Sept. 25, 2016, recorded: August 2016, views: 1248
Report a problem or upload filesIf you have found a problem with this lecture or would like to send us extra material, articles, exercises, etc., please use our ticket system to describe your request and upload the data.
Enter your e-mail into the 'Cc' field, and we will keep you updated with your request's status.
Sparse learning has received tremendous amount of interest in high-dimensional data analysis due to its model interpretability and the low-computational cost. Among the various techniques, adaptive l1-regularization is an eﬀective framework to improve the convergence behaviour of the LASSO, by using varying strength of regularization across diﬀerent features. In the meantime, the adaptive structure makes it very powerful in modelling grouped sparsity pat-terns as well, being particularly useful in high-dimensional multi-task problems. However, choosing an appropriate, global regularization weight is still an open problem. In this paper, inspired by the annealing technique in matrial science, we propose to achieve “annealed sparsity” by designing a dynamic shrinking scheme that simultaneously optimizes the regularization weights and model coeﬃcients in sparse (multi-task) learning. The dynamic structures of our algorithm are twofold. Feature-wise (“spatially”), the regularization weights are updated interactively with model coeﬃcients, allowing us to improve the global regularization structure. Iteration-wise (“temporally”), such interaction is coupled with gradually boosted l1-regularization by adjusting an equality norm-constraint, achieving an “annealing” eﬀect to further improve model selection. This renders interesting shrinking behaviour in the whole solution path. Our method competes favorably with state-of-the-art methods in sparse (multi-task) learning. We also apply it in expression quantitative trait loci analysis (eQTL), which gives useful biological insights in human cancer (melanoma) study.
Link this pageWould you like to put a link to this lecture on your homepage?
Go ahead! Copy the HTML snippet !