In this paper, we study the problem of poisoning the learning of an optimal controller by means of an actuation attack. We specifically consider a user who is gathering data from a linear system in the form of input and state measurements, and who uses these data to learn an optimal controller. Nevertheless, these measurements are corrupted by an attacker who has access to the system's actuators, and who is using them to launch an actuation attack during the learning process. We design this actuation attack so that it optimally corrupts the data used by the user: it forces the user to learn as closely as possible a gain that the attacker has selected, and which is unrelated to the actual optimal control gain. We prove that this poisoning actuation attack design boils down to the solution of certain coupled matrix equations, which we solve using the block successive over-relaxation (SOR) iterative procedure. Simulations on an aircraft model demonstrate theoretical findings, showing how the poisoning attack is effective in misleading the user towards learning an incorrect gain for the system.