Ranking objects is a challenging task for machines. The main difficulty is that some characteristics of interest lack objective criteria. As the Internet becomes more widely used, it is possible to integrate the human capability of evaluating unmeasurable properties with the computational power of machines. A good example is the Internet voting for photos, foods and many others. In this talk, we propose a paired comparison framework, in which users are asked to show preferences in a pair of objects. Experiments on a photo ranking task show that the paired method outperforms the commonly used scoring method.