We propose a thresholded ensemble model for ordinal regression problems. The model consists of a weighted ensemble of confidence functions and an ordered vector of thresholds. Using such a model, we could theoretically and algorithmically reduce ordinal regression problems to binary classification problems in the area of ensemble learning. Based on the reduction, we derive novel large-margin bounds of common error functions, such as the classification error and the absolute error. In addition, we also design two novel boosting approaches for constructing thresholded ensembles. Both our approaches have comparable performance to the state-of-the-art algorithms, but enjoy the benefit of faster training. Experimental results on benchmark datasets demonstrate the usefulness of our boosting approaches.