We describe a manifold learning framework that naturally accommodates supervised learning manifold learning, partially supervised learning and unsupervised clustering as particular cases. Our method chooses a function by minimizing loss subject to a manifold regularization penalty. This augmented cost is minimized using a greedy stagewise functional minimization procedure, as in Gradientboost. Each stage of boosting is fast and efficient. We demonstrate our approach using both radial basis function approximations and classification trees. The performance of our method is at the state of the art on standard problems.