The primal-dual active set method is observed to be the limit of a sequence of penalty formulations. Using this perspective, we propose a penalty method that adaptively becomes the active set method as the residual of the iterate decreases. The adaptive penalty method (APM) therewith combines the main advantages of both methods, namely the ease of implementation of penalty methods and the exact imposition of inequality constraints inherent to the active set method. The scheme can be considered a quasi-Newton method in which the Jacobian is approximated using a penalty parameter. This spatially varying parameter is chosen at each iteration by solving an auxiliary problem.