Linear programming relaxations are one promising approach to solving the MAP estimation problem in Markov random fields; in particular, a body of past work has focused on the first-order tree-based LP relaxation for the MAP problem. Although a variety of algorithms with interesting connections to this LP have been proposed, to date none is guaranteed to always solve the LP for any problem. In this paper, we develop a family of provably convergent LP solvers based on proximal minimization schemes using Bregman divergences that exploit the underlying graphical structure, and so scale well to large problems. All of our algorithms have a double-loop character, with the outer loop corresponding to the proximal sequence, and an inner loop of cyclic Bregman divergences used to compute each proximal update. The inner loop updates are distributed and respect the graph structure, and thus can be cast as message-passing algorithms. We establish various convergence guarantees for our algorithms, illustrate their performance on medium to large-scale problems, and also present a tree-based rounding scheme with provable optimality guarantees.