Recently, building owners are investing in rooftop photovoltaic (PV) installations and batteries in order to meet the (facility) load in their buildings. As a consequence, several commercial and research solutions have emerged for battery energy management in such buildings. Most of these solutions rely on sufficiently accurate system models and are tailor-made for those systems. This work proposes the use of transfer learning in model-free reinforcement learning (RL) to control the operation of batteries in buildings. This enables knowledge from the control of a battery in one building to be used by a RL algorithm to control a battery in another building with similar characteristics. In this paper, the K-shape clustering algorithm is used to group buildings with similar characteristics - based on their energy consumption patterns. To plan the operation of the batteries, we use fitted Q-iteration, a RL algorithm. Simulation results using real-world data show that by including forecast information on energy consumption and PV generation in the feature space of the control algorithm, RL competes with mixed integer linear programming - which assumes perfect knowledge of the system. We also investigate through simulation, the effect of transferring a policy learned with data from one building to another building - all buildings belonging to the same cluster. Simulation results show a faster convergence - convergence achieved with fewer training samples required - to a near optimal policy.