kth.sePublications KTH
Change search
Link to record
Permanent link

Direct link
Publications (6 of 6) Show all publications
Hammerfald, K., Schmidt, F., Vlassov, V., Haaland Jahren, H. & Solbakken, O. A. (2025). Leveraging large language models to identify microcounseling skills in psychotherapy transcripts. Psychotherapy Research, 1-19
Open this publication in new window or tab >>Leveraging large language models to identify microcounseling skills in psychotherapy transcripts
Show others...
2025 (English)In: Psychotherapy Research, ISSN 1050-3307, E-ISSN 1468-4381, p. 1-19Article in journal (Refereed) Published
Abstract [en]

Objective: Microcounseling skills are fundamental to effective psychotherapy, yet manual coding is time- and resource-intensive. This study explores the potential of large language models (LLMs) to automate the identification of these skills in therapy sessions. Method: We fine-tuned GPT-4.1 on a set of psychotherapy transcripts annotated by human coders. The model was trained to classify therapist utterances, generate explanations for its decisions, and propose alternative responses. The pipeline included transcript preprocessing, dialogue segmentation, and supervised fine-tuning. Results: The model achieved solid performance (Accuracy: 0.78; Precision: 0.79; Recall: 0.78; F1: 0.78; Specificity: 0.77; Cohen's κ: 0.69). It reliably detected common and structurally distinct skills but struggled with more nuanced skills that rely on understanding implicit relational dynamics. Conclusion: Despite limitations, fine-tuned LLMs have potential for enhancing psychotherapy research and clinical practice by providing scalable, automated coding of therapist skills.

Place, publisher, year, edition, pages
Informa UK Limited, 2025
Keywords
artificial intelligence, counseling skills, large language models, machine learning, natural language processing
National Category
Applied Psychology Natural Language Processing
Identifiers
urn:nbn:se:kth:diva-369938 (URN)10.1080/10503307.2025.2539405 (DOI)001550802700001 ()40817802 (PubMedID)2-s2.0-105013461117 (Scopus ID)
Note

QC 20250918

Available from: 2025-09-18 Created: 2025-09-18 Last updated: 2025-09-18Bibliographically approved
Schmidt, F., Kurzawski, M. G., Hammerfald, K., Jahren, H. H., Solbakken, O. A. & Vlassov, V. (2024). A Scalable System Architecture for Composition and Deployment of Machine Learning Models in Cognitive Behavioral Therapy. In: 2024 IEEE International Conference on Digital Health (ICDH): . Paper presented at 2024 IEEE International Conference on Digital Health (ICDH), Shenzhen, China, 07-13 July 2024 (pp. 79-86). Institute of Electrical and Electronics Engineers (IEEE)
Open this publication in new window or tab >>A Scalable System Architecture for Composition and Deployment of Machine Learning Models in Cognitive Behavioral Therapy
Show others...
2024 (English)In: 2024 IEEE International Conference on Digital Health (ICDH), Institute of Electrical and Electronics Engineers (IEEE), 2024, p. 79-86Conference paper, Published paper (Refereed)
Abstract [en]

Machine learning (ML) models are a valuable tool for decision support in internet-delivered cognitive behavioral therapy (iCBT). However, while the literature extensively covers model development, a gap exists in the practical deployment of these models. This work proposes a novel system architecture to efficiently compose and deploy an ensemble of ML models tailored for iCBT in the cloud. We first establish system requirements and evaluation metrics based on the iCBT workflow and derive the system architecture based on these. We develop and implement a prototype of the system architecture for the composition and deployment of ML models in iCBT and validate the prototype with representative data through unit and integration tests. The results of the conceptual validation show that the prototype successfully facilitates the deployment of the models per the desired system requirements. Finally, we outline a path to a scalable deployment.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE), 2024
Keywords
Measurement, Systems architecture, Prototypes, Medical treatment, Machine learning, Data models, Electronic healthcare, scalable machine learning, internet-delivered cognitive behavioral therapy, iCBT, clinical decision support system, CDSS
National Category
Computer Sciences
Research subject
Computer Science
Identifiers
urn:nbn:se:kth:diva-352649 (URN)10.1109/ICDH62654.2024.00024 (DOI)001308534900012 ()2-s2.0-85203816332 (Scopus ID)
Conference
2024 IEEE International Conference on Digital Health (ICDH), Shenzhen, China, 07-13 July 2024
Projects
ALEC-2
Note

Part of ISBN 979-8-3503-6857-4

QC 20240906

Available from: 2024-09-04 Created: 2024-09-04 Last updated: 2024-11-05Bibliographically approved
Krylova, S., Schmidt, F. & Vlassov, V. (2024). Leveraging Machine Learning Models to Predict the Outcome of Digital Medical Triage Interviews. In: Proceedings - 2024 International Conference on Machine Learning and Applications, ICMLA 2024: . Paper presented at 23rd IEEE International Conference on Machine Learning and Applications, ICMLA 2024, Miami, United States of America, December 18-20, 2024 (pp. 160-167). Institute of Electrical and Electronics Engineers (IEEE)
Open this publication in new window or tab >>Leveraging Machine Learning Models to Predict the Outcome of Digital Medical Triage Interviews
2024 (English)In: Proceedings - 2024 International Conference on Machine Learning and Applications, ICMLA 2024, Institute of Electrical and Electronics Engineers (IEEE) , 2024, p. 160-167Conference paper, Published paper (Refereed)
Abstract [en]

One of the key advances in digital healthcare is the implementation of digital triage, which, using online tools, web, and mobile apps, allows for efficient assessment of patient needs, prioritizing cases, and directing them to appropriate healthcare services. Many existing digital triage systems are questionnaire-based, guiding patients to appropriate care levels based on infor-mation (e.g., symptoms, medical history, and urgency) provided by the patients answering questionnaires. Such a system often uses a deterministic model with predefined rules to determine care levels. It faces challenges with incomplete triage interviews since it can only assist patients who finish the process. In this study, we explore the use of machine learning (ML) to predict outcomes of unfinished interviews, aiming to enhance patient care and service quality. Predicting triage outcomes from incomplete data is crucial for patient safety and healthcare efficiency. Our findings show that decision-tree models, particularly LGBMClassifier and CatBoostClassifier, achieve over 80% ac-curacy in predicting outcomes from complete interviews while having a linear correlation between the prediction accuracy and interview completeness degree. For example, LGBMClassifier achieves 88,2 % prediction accuracy for interviews with 100 % completeness, 79,6% accuracy for interviews with 80% complete-ness, 58,9 % accuracy for 60 % completeness, and 45,7% accuracy for 40% completeness. The Tab Transformer model demonstrated exceptional accuracy of over 80 % for all degrees of completeness but required extensive training time, indicating a need for more powerful computational resources. The study highlights the linear correlation between interview completeness and predictive power of the decision-tree models.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE), 2024
Keywords
digital health, digital triage, machine learning classification
National Category
Computer Systems
Identifiers
urn:nbn:se:kth:diva-361974 (URN)10.1109/ICMLA61862.2024.00028 (DOI)001468515500022 ()2-s2.0-105001043732 (Scopus ID)
Conference
23rd IEEE International Conference on Machine Learning and Applications, ICMLA 2024, Miami, United States of America, December 18-20, 2024
Note

Part of ISBN 9798350374889

QC 20250404

Available from: 2025-04-03 Created: 2025-04-03 Last updated: 2025-12-05Bibliographically approved
Insalata, B., Schmidt, F. & Vlassov, V. (2024). Multimodal survival prediction using TabTransformer and BioClinicalBERT on MIMIC-III. In: Proceedings - 2024 IEEE International Conference on Big Data, BigData 2024: . Paper presented at 2024 IEEE International Conference on Big Data, BigData 2024, Washington, United States of America, December 15-18, 2024 (pp. 1986-1992). Institute of Electrical and Electronics Engineers (IEEE)
Open this publication in new window or tab >>Multimodal survival prediction using TabTransformer and BioClinicalBERT on MIMIC-III
2024 (English)In: Proceedings - 2024 IEEE International Conference on Big Data, BigData 2024, Institute of Electrical and Electronics Engineers (IEEE) , 2024, p. 1986-1992Conference paper, Published paper (Refereed)
Abstract [en]

This paper explores the development and evaluation of a multimodal system for survival prediction in clinical settings, leveraging both structured electronic health records and unstructured clinical notes. The core objective is to enhance the accuracy and reliability of survival predictions in Intensive Care Units by integrating diverse data types through advanced machine learning models. The system combines the novel architecture of Tabular Transformers, adapted to process structured data such as patient demographics, medical history, and diagnoses, with Multi-Layer Perceptrons for text embeddings obtained from BioClinicalBERT, a specialized model for clinical narratives. These models' integration aims to capture the complex and multifaceted nature of patient profiles, thereby improving prediction performance. The finalized system, a Logistic Regression instance that aggregates the obtained predictions, demonstrates superior performance on evaluation metrics, highlighting the system's ability to identify high-risk patients. Comprehensive benchmarking against decision trees, standalone MLPs, and various configurations underscored the robustness of the proposed system. This research highlights the transformative potential of multimodal data integration in medical predictive modeling. Thus, medical professionals can guide their efforts and prioritization of patient care, enabling more efficient and targeted allocation of resources during triage.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE), 2024
Keywords
Clinical Notes, Electronic Health Records, Multimodality, Survival Prediction, Tabular Transformers
National Category
Computer Sciences
Identifiers
urn:nbn:se:kth:diva-360559 (URN)10.1109/BigData62323.2024.10826011 (DOI)2-s2.0-85218041072 (Scopus ID)
Conference
2024 IEEE International Conference on Big Data, BigData 2024, Washington, United States of America, December 15-18, 2024
Note

Part of ISBN 9798350362480

QC 20250226

Available from: 2025-02-26 Created: 2025-02-26 Last updated: 2025-02-26Bibliographically approved
Schmidt, F., Hammerfald, K., Jahren, H. H., Payberah, A. H. & Vlassov, V. (2024). Single-pass Hierarchical Text Classification with Large Language Models. In: Proceedings - 2024 IEEE International Conference on Big Data, BigData 2024: . Paper presented at 2024 IEEE International Conference on Big Data, BigData 2024, Washington, United States of America, Dec 15 2024 - Dec 18 2024 (pp. 5412-5421). Institute of Electrical and Electronics Engineers (IEEE)
Open this publication in new window or tab >>Single-pass Hierarchical Text Classification with Large Language Models
Show others...
2024 (English)In: Proceedings - 2024 IEEE International Conference on Big Data, BigData 2024, Institute of Electrical and Electronics Engineers (IEEE) , 2024, p. 5412-5421Conference paper, Published paper (Refereed)
Abstract [en]

Numerous text classification tasks inherently possess hierarchical structures among classes, often overlooked in traditional classification paradigms. This study introduces novel approaches for hierarchical text classification using Large Language Models (LLMs), exploiting taxonomies to improve accuracy and traceability in a zero-shot setting. We propose two hierarchical classification methods, namely (i) single-path and (ii) path-traversal, which all leverage the hierarchical class structures inherent in the target classes (e.g., a bird is a type of animal that belongs to a species) and improve naïve hierarchical text classification from literature. We implement them as prompts for generative models such as OpenAI GPTs and benchmark them against discriminative language models (BERT and RoBERTa). We measure the classification performance (precision, recall, and F1-score) vs. computational efficiency (time and cost). Throughout the evaluations of the classification methods on two diverse datasets, namely ComFaSyn, containing mental health patients' diary entries, and DBpedia, containing structured information extracted from Wikipedia, we observed that our methods, without any form of fine-tuning and few-shot examples, achieve comparable results to flat classification and existing methods from literature with minimal increases in the prompts and processing time.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE), 2024
Keywords
Hierarchical text classification, Large Language Models (LLMs), zero-shot classification
National Category
Natural Language Processing
Identifiers
urn:nbn:se:kth:diva-360563 (URN)10.1109/BigData62323.2024.10825412 (DOI)2-s2.0-85218008858 (Scopus ID)
Conference
2024 IEEE International Conference on Big Data, BigData 2024, Washington, United States of America, Dec 15 2024 - Dec 18 2024
Note

Part of ISBN 9798350362480

QC 20250226

Available from: 2025-02-26 Created: 2025-02-26 Last updated: 2025-02-26Bibliographically approved
Schmidt, F., Hammerfald, K., Jahren, H. H., Solbakken, O. A., Payberah, A. H. & Vlassov, V. (2023). Using Machine Learning to Recommend Personalized Modular Treatments for Common Mental Health Disorders. In: Proceedings - 2023 IEEE International Conference on Digital Health, ICDH 2023: . Paper presented at 2023 IEEE International Conference on Digital Health, ICDH 2023, Hybrid, Chicago, United States of America, Jul 8 2023 - Jul 2 2023 (pp. 150-157). Institute of Electrical and Electronics Engineers (IEEE)
Open this publication in new window or tab >>Using Machine Learning to Recommend Personalized Modular Treatments for Common Mental Health Disorders
Show others...
2023 (English)In: Proceedings - 2023 IEEE International Conference on Digital Health, ICDH 2023, Institute of Electrical and Electronics Engineers (IEEE) , 2023, p. 150-157Conference paper, Published paper (Refereed)
Abstract [en]

So far, initial treatment recommendations for internet-based cognitive behavioral therapy (iCBT) decision support were mostly high-level or static. Personalized treatment recommendations could pave the way toward better treatment outcomes and adaptive treatments by leveraging information from past patients. We explore the disadvantages of multi-class recommendation and propose a modular approach using multilabel classification for treatment recommendations. Our machine learning-based treatment recommender composes treatment programs from a set of modules. It achieves a 79.02% F1-score on historically successful treatments, significantly outperforming the existing system by around 4% while offering other advantages such as interpretability and robustness. Using our recommendation as an initial starting point, clinicians can adjust the modular treatments to provide a more personalized treatment.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE), 2023
Keywords
Common mental health disorders, Internet-based cognitive behavioral therapy, Machine learning, Modular treatments, Personalized treatment, Treatment recommendation
National Category
Computer Sciences
Identifiers
urn:nbn:se:kth:diva-337993 (URN)10.1109/ICDH60066.2023.00030 (DOI)001062475200020 ()2-s2.0-85172318259 (Scopus ID)
Conference
2023 IEEE International Conference on Digital Health, ICDH 2023, Hybrid, Chicago, United States of America, Jul 8 2023 - Jul 2 2023
Note

Part of ISBN 9798350341034

QC 20231012

Available from: 2023-10-12 Created: 2023-10-12 Last updated: 2023-10-16Bibliographically approved
Organisations
Identifiers
ORCID iD: ORCID iD iconorcid.org/0000-0002-4310-0867

Search in DiVA

Show all publications