We propose to learn a data-adaptive convex regularizer, which is parameterized using an input-convex neural network (ICNN), for variational image reconstruction. The regularizer parameters are learned adversarially by telling apart clean images from the artifact-ridden ones in a training dataset. Convexity of the regularizer is theoretically and practically important since (i) one can establish well-posedness guarantees for the corresponding variational reconstruction problem and (ii) devise provably convergent optimization algorithms for reconstruction. In particular, the resulting method is shown to be convergent in the sense of regularization and can be solved provably using a gradient-based solver. To demonstrate the performance of our approach for solving inverse problems, we consider deblurring natural images and reconstruction in X-ray computed tomography (CT) and show that the proposed convex regularizer is on par with and sometimes superior to state-of-the-art classical and data-driven techniques for inverse problems, especially with severely ill-posed forward operators (such as in limited-angle tomography).