Recently, multi-task traffic prediction has received increasing attention, as it enables knowledge sharing between heterogeneous variables or regions, thereby improving prediction accuracy while satisfying the prediction requirements of multi-source data in Intelligent Transportation Systems (ITS). However, current studies present two significant challenges. First, they often tend to construct specialized models for a limited set of predictive parameters, which results in a lack of generality. Second, modeling the graph-based multi-task interaction and message passing processes remains difficult due to the heterogeneity of graph structures arising from multi-source traffic data. To address these challenges, this paper proposes a Multi-Graph Interaction Learning Network (MuGIL), characterized by three key innovations: 1) A flexible end-to-end multi-task prediction framework that is generalizable for varied variables or scenarios; 2) A multi-source graph representation module that aligns heterogeneous information through semantic graphs; 3) A novel message passing mechanism for multi-task graph neural networks, which enables effective knowledge among tasks. The model is validated using data from California by comparing it with the state-of-the-art prediction models. The results show that the MuGIL model achieves better prediction performance than these baselines. Ablation experiments further highlight the critical role of the designed multi-source graph representation module and message passing mechanism in the model's success. The MuGIL model we have proposed is now open-sourced at the following link: https://github.com/trafficpre/MuGIL.