Peeking at A/B Tests: Why it matters, and what to do about it

This paper reports on novel statistical methodology, which has been deployed by the commercial A/B testing platform Optimizely to communicate experimental results to their customers. Our methodology a