Reinforcement Learning for EV charging optimization has gained significant academic attention in recent years, due to its ability to handle uncertainty, non-linearity, and real-time problem-solving. While the number of articles published on the matter has surged, the number of open-source environments for EV charging optimization remains small, and a research gap still exists when it comes to customizable frameworks for commercial vehicle fleets. To bridge the gap between research and real-world deployment of RL-based charging optimization, this paper introduces FleetRL as the first customizable RL environment for fleet charging optimization. Researchers and fleet operators can easily adapt the framework to fit their use-cases, and assess the impact of RL-based charging on economic feasibility, battery degradation, and operations.