We address the problem of learning decision functions from training data in which some attribute values are unobserved. This problem can arise for instance, when training data is aggregated from multiple sources, and some sources record only a subset of attributes. We derive a joint optimization problem for the final classifier in which the distribution governing the missing values is a free parameter. We show that the optimal solution concentrates the density mass on finitely many atoms, and provide a corresponding algorithm for learning from incomplete data. We report on empirical results on benchmark data, and on the email spam application that motivates the problem setting