Providing relevant information is crucial in public transport systems. With the rise of digital passenger information systems (PIS), personalization has emerged as a means to meet passengers’ information needs better. To better understand how personalization can be implemented in PIS, five levels of personalization have been identified in the literature, highlighting varying degrees of system autonomy and passenger involvement. While these levels have advanced the understanding of personalization, their practical application remains limited. This paper builds upon an existing framework of personalization levels. It introduces an evaluation tool composed of distinct performance measures to help PIS developers assess their system’s current personalization level and identify opportunities for technological advancement. The tool enables a comparison of the personalization functionalities against the best practices defined by the personalization levels. The paper further outlines the creation of the tool through functional benchmarking, introduces system behaviors across levels, and evaluates commercial PIS through case studies, offering actionable insights for advancing PIS personalization.