The ongoing development of telecommunication systems like 5G has led to an increase in demand of well calibrated base transceiver station (BTS) components. A pivotal component of every BTS is cavity filters, which provide a sharp frequency characteristic to select a particular band of interest and reject the rest. Unfortunately, their characteristics in combination with manufacturing tolerances make them difficult for mass production and often lead to costly manual post-production fine tuning. To address this, numerous approaches have been proposed to automate the tuning process. One particularly promising one, that has emerged in the past few years, is to use model free reinforcement learning (MFRL); however, the agents are not sample efficient. This poses a serious bottleneck, as utilising complex simulators or training with real filters is prohibitively time demanding. This work advocates for the usage of model based reinforcement learning (MBRL) and showcases how its utilisation can significantly decrease sample complexity, while maintaining similar levels of success rate. More specifically, we propose an improvement over a state-of-the-art (SoTA) MBRL algorithm, namely the Dreamer algorithm. This improvement can serve as a template for applications in other similar, high-dimensional non-image data problems. We carry experiments on two complex filter types, and show that our novel modification on the Dreamer architecture reduces sample complexity by a factor of 4 and 10, respectively. Our findings pioneer the usage of MBRL which paves the way for utilising more precise and accurate simulators which was previously prohibitively time demanding.