In this tutorial I will discuss how reinforcement learning (RL) can be combined with deep learning (DL). There are several ways to combine DL and RL together, including value-based, policy-based, and model-based approaches with planning. Several of these approaches have well-known divergence issues, and I will present simple methods for addressing these instabilities. The talk will include a case study of recent successes in the Atari 2600 domain, where a single agent can learn to play many different games directly from raw pixel input.