Building personalized systems for fashion recommendation presents several challenges due to the complicated semantics of people's preferences and styles. One challenge is simply the need to deal with sparse, long-tailed datasets, where new content is constantly introduced and recommendation is inherently a cold-start problem. Another challenge is the need to model visual signals, where the semantics of what makes items "attractive" are incredibly subtle. Finally, there is the need to model temporal dynamics that account for how fashion continually (and rapidly) evolves. In this talk we'll see how traditional recommendation approaches can be extended to explicitly account for the visual appearance of the items being recommended, in order to overcome these challenges and make visually- and stylistically-aware recommendations.