Prognostics and Health Management (PHM) services play a critical role in maintaining system performance by monitoring and managing the status of industrial Cyber-Physical Systems (CPS). These services typically involve analyzing multi-dimensional data from various sensors to extract spatial and temporal features. The advancement of data-driven methods, such as Machine Learning (ML) and Deep Learning (DL), provides an end-to-end solution for straightforwardly manipulating sensory data and learning patterns. In addition to these methods, the rise of Large Language Models (LLM) also facilitates the analysis and comprehension of multi-dimensional data. Compared to ML/DL models, LLM demonstrate powerful capabilities due to their extensive training data and cross-domain knowledge. However, continuous challenges remain in seamlessly detecting anomalies by integrating domain knowledge with these methods to effectively and efficiently monitor system status for health management (e.g., fault propagation via causality analysis). To alleviate this issue, this paper presents a functional framework to support the health management in CPS by leveraging the integration of LLM and domain knowledge. As case studies, this work uses the Tennessee Eastman (TE) process and an Ultra-Processed (UP) food manufacturing plant to evaluate the performance of integrating LLM with domain knowledge by identifying and analyzing the causality of collected data across sensors. Compared to baseline methods that rely solely on the LLM, the results show an increase in F1-score from 0.15 to 0.62 for the TE process, and from 0.05 to 0.64 for the UP process.