In an increasingly competitive business environment characterized by rapid technological advances, evolving customer behaviors, and dynamic markets, organizations face growing challenges to maintain their competitive edge. This study explores the potential of Large Language Model (LLM)s in enhancing Competitive Intelligence (CI) processes, specifically focusing on automating competitor data analysis. The primary objective is to develop and evaluate a use case demonstrating the potential of LLMs in improving CI practices. The research employs a mixed-methods approach, combining the evaluation of an LLM-powered prototype with qualitative interviews and quantitative performance assessments. The prototype leverages customer success stories as a new data source for CI, automating the collection, analysis and visualization of competitor information. Qualitative evaluation through interviews comparing the previous manual CI process with the LLM-powered approach reveal that the prototype can address the limitations of manual CI practices. The prototype demonstrates the ability to analyze larger volumes of data and provide more frequent insights compared to traditional methods. Performance evaluations show that larger LLM models outperform smaller ones in complex domain-specific classification tasks, such as classifying competitors’ areas of expertise, while in-context learning enhances classification accuracy. This research contributes to the growing body of knowledge on LLM applications in the field of CI, offering practical insights for organizations seeking to integrate LLMs into their CI workflows. The study suggests that AI is transforming the field of CI, allowing organizations to gain valuable insights through LLM-based processes. It also highlights the importance of carefully considering what data to publish and how to gain a competitive advantage by incorporating LLMs into CI practices.