RL-Glue is a protocol and software implementation for evaluating reinforcement learning algorithms. Our system facilitates the comparison of alternative algorithms and can greatly accelerate research progress as the UCI database has accelerated progress in supervised machine learning. Creating a comparable bench- marking resource for reinforcement learning is challenging because of the temporal nature of reinforcement learning. Reinforcement learning agents interact with a dynamic process (the environment) which gener- ates observations and rewards. The observations and rewards received by the learning agent depend on the actions; training data cannot simply be stored in a file as they are in supervised learning. Instead, the rein- forcement learning agent and environment must be interacting programs. RL-Glue agents and environments can be written in Java, C/C++, Matlab, Python, and Lisp and can all run on one machine, or can connect across the Internet. In this seminar, we will introduce the design principles that helped shape RL-Glue and demonstrate some of the interesting extensions that have been created by the reinforcement learning community.