We use direct numerical simulations to study convection in rotating Rayleigh-B & eacute;nard convection in horizontally confined geometries of a given aspect ratio, with the walls held at fixed temperatures. We show that this arrangement is unconditionally unstable to flow that takes the form of wall-adjacent convection rolls. For wall temperatures close to the temperatures of the upper or lower boundaries, we show that the base state undergoes a Hopf bifurcation to a state comprised of spatiotemporal oscillations - 'wall modes' - precessing in a retrograde direction. We study the saturated nonlinear state of these modes, and show that the velocity boundary conditions at the upper and lower boundaries are crucial to the formation and propagation of the wall modes: asymmetric velocity boundary conditions at the upper and lower boundaries can lead to prograde wall modes, while stress-free boundary conditions at both walls can lead to wall modes that have no preferred direction of propagation.