I'll present some thoughts and research directions in Bayesian machine learning. I'll contrast black-box approaches to machine learning with model-based Bayesian statistics. Can we meaningfully create Bayesian black-boxes? If so what should the prior be? Is non-parametrics the only way to go? Since we often can't control the effect of using approximate inference, are coherence arguments meaningless? How can we convert the pagan majority of ML researchers to Bayesianism? If the audience gets bored of these philosophical musings, I will switch to talking about our latest technical work on Indian buffet processes.