We consider a class of stochastic control problems where the state process is a probability measure-valued process satisfying an additional martingale condition on its dynamics, called measure-valued martingales (MVMs). We establish the “classical” results of stochastic control for these problems: specifically, we prove that the value function for the problem can be characterised as the unique solution to the Hamilton–Jacobi–Bellman equation in the sense of viscosity solutions. In order to prove this result, we exploit structural properties of the MVM processes. Our results also include an appropriate version of Itô’s formula for controlled MVMs. We also show how problems of this type arise in a number of applications, including model-independent derivatives pricing, the optimal Skorokhod embedding problem, and two player games with asymmetric information.