kth.sePublications KTH
System disruptions
There are recurring service disruptions on some of the search portals due to high traffic. We are working to resolve the issue, you may temporarily encounter an error message.
Change search
Link to record
Permanent link

Direct link
Alternative names
Publications (10 of 57) Show all publications
Layegh, A., Payberah, A. H. & Matskin, M. (2026). From Code to Execution: Multi-Agent GraphRAG for Automated Artifact Generation. In: EuroMLSys 2026 - Proceedings of the 2026 the 6th European Workshop on Machine Learning and Systems: . Paper presented at 6th Workshop on Machine Learning and Systems, EuroMLSys 2026, Edinburgh, United Kingdom of Great Britain and Northern Ireland, April 27-30, 2026 (pp. 147-154). Association for Computing Machinery (ACM)
Open this publication in new window or tab >>From Code to Execution: Multi-Agent GraphRAG for Automated Artifact Generation
2026 (English)In: EuroMLSys 2026 - Proceedings of the 2026 the 6th European Workshop on Machine Learning and Systems, Association for Computing Machinery (ACM) , 2026, p. 147-154Conference paper, Published paper (Refereed)
Abstract [en]

Modern data and Machine Learning (ML) pipelines are developed as code repositories but must be manually transformed into deployable artifacts such as Docker images and Kubernetes workflows, a process prone to dependency and configuration errors. While Large Language Models (LLMs) can assist with code understanding, they often fail on multifile repositories due to missing structural context and incomplete dependency resolution. We propose a multi-agent graph-based Retrieval-Augmented Generation (GraphRAG) framework that automates the generation of deployment artifacts from code repositories. The framework constructs an Abstract Syntax Tree (AST)-grounded knowledge graph, performs structure-aware retrieval, and coordinates specialized agents to synthesize containerization and workflow specifications. The system further improves reliability through iterative self-healing driven by execution logs. Evaluated on 17 GitHub repositories spanning ETL, ML training, and LLM-based evaluation pipelines, our approach achieves 65% end-to-end execution success on Kubernetes, outperforming a vector-only RAG baseline (24%) and a single-agent GraphRAG variant (47%). These results indicate that combining graph-centric retrieval with agent decomposition improves execution reliability.

Place, publisher, year, edition, pages
Association for Computing Machinery (ACM), 2026
Keywords
Deployment automation, GraphRAG, Knowledge graphs, Multi-agent systems
National Category
Computer Sciences Software Engineering
Identifiers
urn:nbn:se:kth:diva-382911 (URN)10.1145/3805621.3807637 (DOI)2-s2.0-105038623807 (Scopus ID)
Conference
6th Workshop on Machine Learning and Systems, EuroMLSys 2026, Edinburgh, United Kingdom of Great Britain and Northern Ireland, April 27-30, 2026
Note

Part of ISBN 9798400726057

QC 20260605

Available from: 2026-06-05 Created: 2026-06-05 Last updated: 2026-06-05Bibliographically approved
Tahmasebi, S., Hamad, M., Payberah, A. H. & Matskin, M. (2026). PureBiasoMeter: Decoupling Popularity Bias from User Fairness in LLM-Based Recommender Systems. In: Recommender Systems for Sustainability and Social Good - Second International Workshop, RecSoGood 2025, Proceedings: . Paper presented at 2nd International Workshop on Recommender Systems for Sustainability and Social Good, RecSoGood 2025, Prague, Czechia, September 26, 2025 (pp. 155-168). Springer Science and Business Media Deutschland GmbH
Open this publication in new window or tab >>PureBiasoMeter: Decoupling Popularity Bias from User Fairness in LLM-Based Recommender Systems
2026 (English)In: Recommender Systems for Sustainability and Social Good - Second International Workshop, RecSoGood 2025, Proceedings, Springer Science and Business Media Deutschland GmbH , 2026, p. 155-168Conference paper, Published paper (Refereed)
Abstract [en]

Large Language Models (LLMs) have transformed Recommendation Systems (RecSys) by enabling more user-aware interactions. However, this shift raises new challenges in evaluating fairness, particularly due to confounding systemic biases such as popularity bias. Conventional fairness assessments often confuse disparities in user treatment with systemic biases toward popular items, resulting in misleading conclusions. In this paper, we introduce PureBiasoMeter, a diagnostic framework that decouples popularity bias from user-level fairness in LLM-based RecSys. Our approach is based on the hypothesis that accurate fairness evaluation requires first mitigating popularity bias and then remeasuring fairness metrics. We generate a comprehensive set of 72 prompts using different user profiling strategies, demographic variants, and bias mitigation instructions. By applying these prompts in a black-box LLM setting, we evaluate fairness sensitivity, recommendation quality, and bias levels across multiple dimensions. Our results demonstrate that removing popularity bias substantially alters fairness measurements and reveals underlying disparities that were previously unknown. PureBiasoMeter thus provides a more reliable basis for fairness analysis and contributes a practical tool for disentangling intertwined sources of bias in modern RecSys.

Place, publisher, year, edition, pages
Springer Science and Business Media Deutschland GmbH, 2026
Keywords
Large Language Models, Popularity Bias, Recommendation Systems, User Fairness
National Category
Computer Sciences
Identifiers
urn:nbn:se:kth:diva-376718 (URN)10.1007/978-3-032-13342-7_14 (DOI)2-s2.0-105028087664 (Scopus ID)
Conference
2nd International Workshop on Recommender Systems for Sustainability and Social Good, RecSoGood 2025, Prague, Czechia, September 26, 2025
Note

Part of ISBN 9783032133410

QC 20260218

Available from: 2026-02-18 Created: 2026-02-18 Last updated: 2026-02-18Bibliographically approved
Tahmasebi, S., Nikzad, N., Payberah, A. H., Asgari-Chenaghlu, M. & Matskin, M. (2026). StarDTox: Is Fairness in Language Models Just a Few Prompts Away?. In: Proceedings - 2026 IEEE 50th Annual Computers, Software, and Applications Conference, COMPSAC 2026: . Paper presented at 50th IEEE Annual Computers, Software, and Applications Conference, COMPSAC 2026, Madrid, Spain, Jul 07 2026 - Jul 10 2026 (pp. 2909-2916). Institute of Electrical and Electronics Engineers (IEEE)
Open this publication in new window or tab >>StarDTox: Is Fairness in Language Models Just a Few Prompts Away?
Show others...
2026 (English)In: Proceedings - 2026 IEEE 50th Annual Computers, Software, and Applications Conference, COMPSAC 2026, Institute of Electrical and Electronics Engineers (IEEE) , 2026, p. 2909-2916Conference paper, Published paper (Refereed)
Abstract [en]

Large Language Models (LLMs) often produce outputs that reflect social biases, toxicity, or unfair treatment of demographic groups, undermining trust and fairness. While prior mitigation strategies frequently rely on complex architectures, access to model internals, or costly fine-tuning, we argue that simplicity can be a strength. We introduce StarDTox, a lightweight, critique-and-revise multi-agent framework that leverages the LLM's own internal knowledge, via a small number of coordinated prompts, to self-correct harmful outputs. Dedicated agents independently assess bias and overall output quality, and their feedback is integrated to guide prompt-based revision. Without modifying model weights or requiring any extra finetuning, StarDTox offers strong bias mitigation and high-quality outputs across both open-ended text generation and structured tasks, outperforming other baselines. For the text generation task, on the RealToxicityPrompt dataset, it reduces toxicity by over 50% compared to other baselines, while maintaining over 90% fluency. In addition, in structured tasks, on the BBQ benchmark, it achieves the lowest bias scores across both ambiguous and disambiguated examples, without sacrificing accuracy.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE), 2026
National Category
Natural Language Processing Software Engineering
Identifiers
urn:nbn:se:kth:diva-388182 (URN)10.1109/COMPSAC69091.2026.00438 (DOI)2-s2.0-105048902776 (Scopus ID)
Conference
50th IEEE Annual Computers, Software, and Applications Conference, COMPSAC 2026, Madrid, Spain, Jul 07 2026 - Jul 10 2026
Note

Part of ISBN 9798331544973

QC 20260915

Available from: 2026-09-15 Created: 2026-09-15 Last updated: 2026-09-15Bibliographically approved
Tahmasebi, S., Nikzad, N., Payberah, A. H., Asgari-chenaghlu, M. & Matskin, M. (2025). Fact vs. Fiction: Are the Reportedly “Magical” LLM-Based Recommenders Reproducible?. In: Advances in Information Retrieval - 47th European Conference on Information Retrieval, ECIR 2025, Proceedings: . Paper presented at 47th European Conference on Information Retrieval, ECIR 2025, Lucca, Italy, Apr 06 2025 - Apr 10 2025 (pp. 79-94). Springer Nature, 15575 LNCS
Open this publication in new window or tab >>Fact vs. Fiction: Are the Reportedly “Magical” LLM-Based Recommenders Reproducible?
Show others...
2025 (English)In: Advances in Information Retrieval - 47th European Conference on Information Retrieval, ECIR 2025, Proceedings, Springer Nature , 2025, Vol. 15575 LNCS, p. 79-94Conference paper, Published paper (Refereed)
Abstract [en]

Reproducibility is a cornerstone of scientific progress, allowing researchers to validate findings and build upon previous work. While reproducibility has been an important issue in traditional recommender systems, the rise of Large Language Model (LLM)-based recommendation systems introduces new challenges, particularly in top-N recommendation tasks. In this study, we investigate the reproducibility of state-of-the-art LLM-based recommendation systems. We categorize key factors affecting the reproducibility of recommendation performance into four groups: code, data, methodological details, and evaluation. Our findings highlight significant performance fluctuations based on these factors, emphasizing the need for these factors to be clearly documented and considered during evaluations. To enhance reproducibility, we propose LLMReClarify, a comprehensive set of guidelines adapted from the NeurIPS reproducibility checklist, tailored specifically for LLM-based recommendation systems.

Place, publisher, year, edition, pages
Springer Nature, 2025
Keywords
Large Language Models, Recommendation Systems, Reproducibility, Top-N Recommendation
National Category
Computer and Information Sciences
Identifiers
urn:nbn:se:kth:diva-385647 (URN)10.1007/978-3-031-88717-8_7 (DOI)2-s2.0-105006495365 (Scopus ID)
Conference
47th European Conference on Information Retrieval, ECIR 2025, Lucca, Italy, Apr 06 2025 - Apr 10 2025
Note

Part of ISBN 9783031887161

QC 20260717

Available from: 2026-07-17 Created: 2026-07-17 Last updated: 2026-07-17Bibliographically approved
Layegh, A., Payberah, A. H. & Matskin, M. (2025). IntersectionRE: Mitigating Intersectional Bias in Relation Extraction Through Coverage-Driven Augmentation. In: IAAI-ALA 2025 - Joint Proceedings of the Workshops Identity-aware AI and Awareness in Learning Agents, co-located with 28th European Conference on Artificial Intelligence, ECAI 2025: . Paper presented at 2025 Workshops Identity-aware AI and Awareness in Learning Agents, IAAI-ALA 2025, Bologna, Italy, October 28, 2025. CEUR-WS
Open this publication in new window or tab >>IntersectionRE: Mitigating Intersectional Bias in Relation Extraction Through Coverage-Driven Augmentation
2025 (English)In: IAAI-ALA 2025 - Joint Proceedings of the Workshops Identity-aware AI and Awareness in Learning Agents, co-located with 28th European Conference on Artificial Intelligence, ECAI 2025, CEUR-WS , 2025Conference paper, Published paper (Refereed)
Abstract [en]

Relation Extraction (RE) models are crucial to many Natural Language Processing (NLP) applications, but often inherit and deepen biases in their training data. The underrepresentation of certain demographic groups can lead to performance disparities, particularly when considering intersectional fairness, where biases intersect across attributes such as gender and ancestry. To address this issue, we present IntersectionRE, a framework to improve the representation of underrepresented groups by generating synthetic training data. IntersectionRE identifies gaps in demographic coverage and optimizes data generation, ensuring the quality of augmented data through Large Language Models (LLMs), perplexity scoring, and factual consistency validation. Experimental results on the NYT-10, and Wiki-ZSL datasets demonstrate that our approach effectively reduces intersectional representation and model performance disparities, particularly for historically underrepresented groups.

Place, publisher, year, edition, pages
CEUR-WS, 2025
Keywords
Relation Extraction, Representation Bias, Synthetic Data Generation
National Category
Natural Language Processing
Identifiers
urn:nbn:se:kth:diva-382914 (URN)2-s2.0-105039060904 (Scopus ID)
Conference
2025 Workshops Identity-aware AI and Awareness in Learning Agents, IAAI-ALA 2025, Bologna, Italy, October 28, 2025
Note

QC 20260605

Available from: 2026-06-05 Created: 2026-06-05 Last updated: 2026-06-05Bibliographically approved
Yoneki, E. & Payberah, A. H. (2025). Message from EuroMLSys 25 Chairs. In: EuroMLSys 2025 - Proceedings of the 2025 5th Workshop on Machine Learning and Systems: . Paper presented at 5th Workshop on Machine Learning and Systems, EuroMLSys 2025, held in conjunction with ACM EuroSys 2025 and ASPLOS 2025, Rotterdam, Netherlands, Mar 30 2025 - Apr 03 2025 (pp. 1). Association for Computing Machinery, Inc
Open this publication in new window or tab >>Message from EuroMLSys 25 Chairs
2025 (English)In: EuroMLSys 2025 - Proceedings of the 2025 5th Workshop on Machine Learning and Systems, Association for Computing Machinery, Inc , 2025, p. 1-Conference paper, Published paper (Refereed)
Place, publisher, year, edition, pages
Association for Computing Machinery, Inc, 2025
National Category
Computer Sciences
Identifiers
urn:nbn:se:kth:diva-383715 (URN)2-s2.0-105003688905 (Scopus ID)
Conference
5th Workshop on Machine Learning and Systems, EuroMLSys 2025, held in conjunction with ACM EuroSys 2025 and ASPLOS 2025, Rotterdam, Netherlands, Mar 30 2025 - Apr 03 2025
Note

Part of ISBN 9798400715389

QC 20260618

Available from: 2026-06-18 Created: 2026-06-18 Last updated: 2026-06-18Bibliographically approved
Sheikholeslami, S., Ghasemirahni, H., Payberah, A. H., Wang, T., Dowling, J. & Vlassov, V. (2025). Utilizing Large Language Models for Ablation Studies in Machine Learning and Deep Learning. In: : . Paper presented at The 5th Workshop on Machine Learning and Systems (EuroMLSys), co-located with the 20th European Conference on Computer Systems (EuroSys). ACM Digital Library
Open this publication in new window or tab >>Utilizing Large Language Models for Ablation Studies in Machine Learning and Deep Learning
Show others...
2025 (English)Conference paper, Published paper (Refereed)
Abstract [en]

In Machine Learning (ML) and Deep Learning (DL) research, ablation studies are typically performed to provide insights into the individual contribution of different building blocks and components of an ML/DL system (e.g., a deep neural network), as well as to justify that certain additions or modifications to an existing ML/DL system can result in the proposed improved performance. Although dedicated frameworks for performing ablation studies have been introduced in recent years, conducting such experiments is still associated with requiring tedious, redundant work, typically involving maintaining redundant and nearly identical versions of code that correspond to different ablation trials. Inspired by the recent promising performance of Large Language Models (LLMs) in the generation and analysis of ML/DL code, in this paper we discuss the potential of LLMs as facilitators of ablation study experiments for scientific research projects that involve or deal with ML and DL models. We first discuss the different ways in which LLMs can be utilized for ablation studies and then present the prototype of a tool called AblationMage, that leverages LLMs to semi-automate the overall process of conducting ablation study experiments. We showcase the usability of AblationMage as a tool through three experiments, including one in which we reproduce the ablation studies from a recently published applied DL paper.

Place, publisher, year, edition, pages
ACM Digital Library, 2025
Keywords
Ablation Studies, Deep Learning, Feature Ablation, Model Ablation, Large Language Models
National Category
Computer Sciences
Research subject
Computer Science; Computer Science
Identifiers
urn:nbn:se:kth:diva-360719 (URN)10.1145/3721146.3721957 (DOI)001477868300025 ()2-s2.0-105003634645 (Scopus ID)
Conference
The 5th Workshop on Machine Learning and Systems (EuroMLSys), co-located with the 20th European Conference on Computer Systems (EuroSys)
Funder
Vinnova, 2016–05193
Note

QC 20250303

Available from: 2025-02-28 Created: 2025-02-28 Last updated: 2025-07-01
Puhach, D., Payberah, A. H. & Székely, É. (2025). Who Gets the Mic? Investigating Gender Bias in the Speaker Assignment of a Speech-LLM. In: Interspeech 2025: . Paper presented at 26th Interspeech Conference 2025, Rotterdam, Netherlands, Kingdom of the, August 17-21, 2025 (pp. 2058-2062). International Speech Communication Association
Open this publication in new window or tab >>Who Gets the Mic? Investigating Gender Bias in the Speaker Assignment of a Speech-LLM
2025 (English)In: Interspeech 2025, International Speech Communication Association , 2025, p. 2058-2062Conference paper, Published paper (Refereed)
Abstract [en]

Similar to text-based Large Language Models (LLMs), Speech-LLMs exhibit emergent abilities and context awareness. However, whether these similarities extend to gender bias remains an open question. This study proposes a methodology leveraging speaker assignment as an analytic tool for bias investigation. Unlike text-based models, which encode gendered associations implicitly, Speech-LLMs must produce a gendered voice, making speaker selection an explicit bias cue. We evaluate Bark, a Text-to-Speech (TTS) model, analyzing its default speaker assignments for textual prompts. If Bark's speaker selection systematically aligns with gendered associations, it may reveal patterns in its training data or model design. To test this, we construct two datasets: (i) Professions, containing gender-stereotyped occupations, and (ii) Gender-Colored Words, featuring gendered connotations. While Bark does not exhibit systematic bias, it demonstrates gender awareness and has some gender inclinations.

Place, publisher, year, edition, pages
International Speech Communication Association, 2025
Keywords
gender bias, speech synthesis, speech-LLM, TTS
National Category
Natural Language Processing Gender Studies
Identifiers
urn:nbn:se:kth:diva-372806 (URN)10.21437/Interspeech.2025-1402 (DOI)001585350500416 ()2-s2.0-105020092426 (Scopus ID)
Conference
26th Interspeech Conference 2025, Rotterdam, Netherlands, Kingdom of the, August 17-21, 2025
Note

QC 20251113

Available from: 2025-11-13 Created: 2025-11-13 Last updated: 2026-05-29Bibliographically approved
Karimi, S., Asadi, S. & Payberah, A. H. (2024). BaziGooshi: A Hybrid Model of Reinforcement Learning for Generalization in Gameplay. IEEE Transactions on Games, 16(3), 722-734
Open this publication in new window or tab >>BaziGooshi: A Hybrid Model of Reinforcement Learning for Generalization in Gameplay
2024 (English)In: IEEE Transactions on Games, ISSN 2475-1502, E-ISSN 2475-1510, Vol. 16, no 3, p. 722-734Article in journal (Refereed) Published
Abstract [en]

While reinforcement learning (RL) is gaining popularity in gameplay, creating a generalized RL model is still challenging. This study presents BaziGooshi, a generalized RL solution for games, focusing on two different types of games: 1) a puzzle game Candy Crush Friends Saga and 2) a platform game Sonic the Hedgehog Genesis. BaziGooshi rewards RL agents for mastering a set of intrinsic basic skills as well as achieving the game objectives. The solution includes a hybrid model that takes advantage of a combination of several agents pretrained using intrinsic or extrinsic rewards to determine the actions. We propose an RL-based method for assigning weights to the pretrained agents. Through experiments, we show that the RL-based approach improves generalization to unseen levels, and BaziGooshi surpasses the performance of most of the defined baselines in both games. Also, we perform additional experiments to investigate further the impacts of using intrinsic rewards and the effects of using different combinations in the proposed hybrid models.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE), 2024
Keywords
Games, Color, Training, Green products, Encoding, Shape, Reinforcement learning, Deep reinforcement learning
National Category
Computer Sciences
Identifiers
urn:nbn:se:kth:diva-355137 (URN)10.1109/TG.2024.3355172 (DOI)001319570900012 ()2-s2.0-85184322584 (Scopus ID)
Note

QC 20241023

Available from: 2024-10-23 Created: 2024-10-23 Last updated: 2025-08-28Bibliographically approved
Sheikholeslami, S., Wang, T., Payberah, A. H., Dowling, J. & Vlassov, V. (2024). Deep Neural Network Weight Initialization from Hyperparameter Tuning Trials. In: Neural Information Processing: . Paper presented at ICONIP: International Conference on Neural Information Processing, December 2-6, Auckland, New Zeeland. Springer Nature
Open this publication in new window or tab >>Deep Neural Network Weight Initialization from Hyperparameter Tuning Trials
Show others...
2024 (English)In: Neural Information Processing, Springer Nature , 2024Conference paper, Published paper (Refereed)
Abstract [en]

Training of deep neural networks from scratch requires initialization of the neural network weights as a first step. Over the years, many policies and techniques for weight initialization have been proposed and widely used, including Kaiming initialization and different variants of random initialization. On the other hand, another requirement for starting the training stage is to choose and set suitable hyperparameter values, which are usually obtained by performing several hyperparameter tuning trials. In this paper, we study the suitability of weight initialization using weights obtained from different epochs of hyperparameter tuning trials and compare it to Kaiming uniform (random) weight initialization for image classification tasks. Based on an experimental evaluation using ResNet-18, ResNet-152, and InceptionV3 models, and CIFAR-10, CIFAR-100, Tiny ImageNet, and Food-101 datasets, we show that weight initialization from hyperparameter tuning trials can speed up the training of deep neural networks by up to 2x while maintaining or improving the best test accuracy of the trained models, when compared to random initialization.

Place, publisher, year, edition, pages
Springer Nature, 2024
Keywords
weight initialization, deep neural network training, hyperparameter tuning, model training, deep learning
National Category
Computer Sciences Artificial Intelligence
Research subject
Computer Science
Identifiers
urn:nbn:se:kth:diva-358848 (URN)10.1007/978-981-96-6954-7_5 (DOI)2-s2.0-105009958310 (Scopus ID)
Conference
ICONIP: International Conference on Neural Information Processing, December 2-6, Auckland, New Zeeland
Note

QC 20250303

Available from: 2025-02-28 Created: 2025-02-28 Last updated: 2026-06-22Bibliographically approved
Organisations
Identifiers
ORCID iD: ORCID iD iconorcid.org/0000-0002-2748-8929

Search in DiVA

Show all publications