Change search
Link to record
Permanent link

Direct link
Publications (10 of 16) Show all publications
Bayram, F., Ahmed, B. S. & Hallin, E. (2026). End-to-end data quality-driven framework for machine learning in production environment. Heliyon, 12(1), Article ID e44416.
Open this publication in new window or tab >>End-to-end data quality-driven framework for machine learning in production environment
2026 (English)In: Heliyon, E-ISSN 2405-8440, Vol. 12, no 1, article id e44416Article in journal (Refereed) Published
Abstract [en]

This paper introduces a novel end-to-end framework that efficiently integrates data quality assessment with machine learning (ML) model operations in real-time production environments. While existing approaches treat data quality assessment and ML systems as isolated processes, our framework addresses the critical gap between theoretical methods and practical implementation by combining dynamic drift detection, adaptive data quality metrics, and MLOps into a cohesive, lightweight system. The key innovation lies in its operational efficiency, enabling real-time, quality-driven ML decision-making with minimal computational overhead. We validate the framework in a steel manufacturing company’s Electroslag Remelting (ESR) vacuum pumping process, demonstrating a 12 % improvement in model performance (R2 = 94 %) and a fourfold reduction in prediction latency. By exploring the impact of data quality acceptability thresholds, we provide actionable insights into balancing data quality standards and predictive performance in industrial applications. This framework represents a significant advancement in MLOps, offering a robust solution for time-sensitive, data-driven decision-making in dynamic industrial environments. 

Place, publisher, year, edition, pages
Elsevier, 2026
Keywords
Data quality, Data-driven AI, Drift detection, Machine learning, MLOps
National Category
Computer and Information Sciences
Research subject
Computer Science
Identifiers
urn:nbn:se:kau:diva-108182 (URN)10.1016/j.heliyon.2025.e44416 (DOI)001772336900001 ()2-s2.0-105025701238 (Scopus ID)
Available from: 2026-01-13 Created: 2026-01-13 Last updated: 2026-06-08Bibliographically approved
Jagstedt, S., Magnusson, P., Ahmed, B. S. & Bayram, F. (2025). Exploring AI as a lever for personalised tourism services. In: : . Paper presented at QUIS19 - 19th International Research Symposium on Service Excellence in Management.
Open this publication in new window or tab >>Exploring AI as a lever for personalised tourism services
2025 (English)Conference paper, Oral presentation with published abstract (Refereed)
National Category
Industrial engineering and management
Research subject
Business Administration; Computer Science
Identifiers
urn:nbn:se:kau:diva-106945 (URN)
Conference
QUIS19 - 19th International Research Symposium on Service Excellence in Management
Available from: 2025-09-19 Created: 2025-09-19 Last updated: 2026-02-12Bibliographically approved
Bayram, F. (2025). Machine Learning in Motion: Engineering Self-Adaptive Systems. (Doctoral dissertation). Karlstad: Karlstads universitet
Open this publication in new window or tab >>Machine Learning in Motion: Engineering Self-Adaptive Systems
2025 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

In the rapidly evolving landscape of artificial intelligence (AI) and machine learning (ML), transitioning from theoretical models to robust, adaptive production systems remains a significant challenge in the digital age. As data-driven methodologies revolutionize problem solving across industries, a critical gap exists between research achievements and reliable real-world deployment. Key obstacles include concept drift and data quality issues that arise from unpredictable changes in data distributions and operational complexities that systematically affect model performance, reliability, and efficiency. This thesis addresses these challenges by introducing an overarching framework for adaptive ML systems that operate reliably in dynamic, real-world environments, incorporating innovative methodologies for dynamic drift detection and real-time data quality assessment bounded by robust Machine Learning Operations (MLOps) strategies. These integrated components enable the creation of production-grade ML systems that can efficiently adapt to shifts in data distributions and assess data quality in real-time, ensuring stable and reliable performance in dynamic environments. The proposed approaches are validated through real-world use cases, demonstrating significant improvements in predictive accuracy and operational efficiency. By deploying these adaptive systems in industrial contexts, the thesis highlights their potential to deliver reliable, high-performance ML solutions tailored to the demands of complex time-sensitive applications. This work offers concrete solutions for translating theoretical advances into practical applications, contributing to developing robust and scalable ML systems for real-world deployment.

Abstract [en]

In the era of AI-driven innovation, ensuring the reliability and adaptability of machine learning (ML) systems in dynamic environments remains a fundamental challenge. This thesis addresses key obstacles such as concept drift and data quality, which affect model performance in real-world applications. By integrating novel drift detection mechanisms, real-time data quality assessment, and robust MLOps strategies, this work proposes a comprehensive framework for self-adaptive ML systems. These solutions enable industrial ML deployments to maintain predictive accuracy, efficiency, and resilience against evolving data conditions. Through real-world case studies, this research demonstrates significant improvements in operational stability and decision-making, bridging the gap between theoretical advancements and practical implementation. By tackling these challenges, this thesis contributes to the development of scalable, trustworthy, and high-performance ML solutions tailored for industrial and mission-critical applications.

Place, publisher, year, edition, pages
Karlstad: Karlstads universitet, 2025. p. 30
Series
Karlstad University Studies, ISSN 1403-8099 ; 2025:14
Keywords
machine learning, concept drift, data quality, adaptive learning, MLOps, data-driven development, performance robustness
National Category
Artificial Intelligence Computer Sciences
Research subject
Computer Science
Identifiers
urn:nbn:se:kau:diva-103579 (URN)10.59217/rjrp8643 (DOI)978-91-7867-560-9 (ISBN)978-91-7867-561-6 (ISBN)
Public defence
2025-04-25, Eva Eriksson lecture hall, 21A342, Karlstad, 09:00 (English)
Opponent
Supervisors
Available from: 2025-04-07 Created: 2025-03-17 Last updated: 2026-02-12Bibliographically approved
Ahmed, B. S., Baader, O. L., Bayram, F., Jagstedt, S. & Magnusson, P. (2025). Quality Assurance for LLM-RAG Systems: Empirical Insights from Tourism Application Testing. In: : . Paper presented at ITEQS conference 2025, The 9th International Workshop on Testing Extra-Functional Properties and Quality Characteristics of Software Systems.
Open this publication in new window or tab >>Quality Assurance for LLM-RAG Systems: Empirical Insights from Tourism Application Testing
Show others...
2025 (English)Conference paper, Oral presentation with published abstract (Refereed)
Abstract [en]

This paper presents a comprehensive framework for testing and evaluating quality characteristics of Large Language Model (LLM) systems enhanced with Retrieval-Augmented Generation (RAG) in tourism applications. Through systematic empirical evaluation of three different LLM variants across multiple parameter configurations, we demonstrate the effectiveness of our testing methodology in assessing both functional correctness and extra-functional properties. Our framework implements 17 distinct metrics that encompass syntactic analysis, semantic evaluation, and behavioral evaluation through LLM judges. The study reveals significant information about how different architectural choices and parameter configurations affect system performance, particularly highlighting the impact of temperature and top-p parameters on response quality. The tests were carried out on a tourism recommendation system for the Värmland region, utilizing standard and RAG-enhanced configurations. The results indicate that the newer LLM versions show modest improvements in performance metrics, though the differences are more pronounced in response length and complexity rather than in semantic quality. The research contributes practical insights for implementing robust testing practices in LLM-RAG systems, providing valuable guidance to organizations deploying these architectures in production environments.

National Category
Computer Systems
Research subject
Computer Science; Business Administration
Identifiers
urn:nbn:se:kau:diva-104250 (URN)
Conference
ITEQS conference 2025, The 9th International Workshop on Testing Extra-Functional Properties and Quality Characteristics of Software Systems
Available from: 2025-05-07 Created: 2025-05-07 Last updated: 2026-02-12Bibliographically approved
Ahmed, B. S., Baader, L. O., Bayram, F., Jagstedt, S. & Magnusson, P. (2025). Quality Assurance for LLM-RAG Systems: Empirical Insights from Tourism Application Testing. In: Fasolino A.R., Panichella S., Aleti A., Mesbah A. (Ed.), Proceedings of International Conference on Software Testing, Verification and Validation Workshops: . Paper presented at International Conference on Software Testing, Verification and Validation Workshops, ICSTW, Naples, Italy, March 31-April 4, 2025. (pp. 200-207). Institute of Electrical and Electronics Engineers (IEEE)
Open this publication in new window or tab >>Quality Assurance for LLM-RAG Systems: Empirical Insights from Tourism Application Testing
Show others...
2025 (English)In: Proceedings of International Conference on Software Testing, Verification and Validation Workshops / [ed] Fasolino A.R., Panichella S., Aleti A., Mesbah A., Institute of Electrical and Electronics Engineers (IEEE), 2025, p. 200-207Conference paper, Published paper (Refereed)
Abstract [en]

This paper presents a comprehensive framework for testing and evaluating quality characteristics of Large Language Model (LLM) systems enhanced with Retrieval-Augmented Generation (RAG) in tourism applications. Through systematic empirical evaluation of three different LLM variants across multiple parameter configurations, we demonstrate the effectiveness of our testing methodology in assessing both functional correctness and extra-functional properties. Our framework implements 17 distinct metrics that encompass syntactic analysis, semantic evaluation, and behavioral evaluation through LLM judges. The study reveals significant information about how different architectural choices and parameter configurations affect system performance, particularly highlighting the impact of temperature and top-p parameters on response quality. The tests were carried out on a tourism recommendation system for the Varmland region in Sweden, utilizing standard and RAG-enhanced configurations. The results indicate that the newer LLM versions show modest improvements in performance metrics, though the differences are more pronounced in response length and complexity rather than in semantic quality. The research contributes practical insights for implementing robust testing practices in LLM-RAG systems, providing valuable guidance to organizations deploying these architectures in production environments. 

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE), 2025
Keywords
Application programs, Recommender systems, Redundancy, Reliability analysis, Reliability theory, Software testing, AI quality assurance, Extra-functional properties, Language model, Large language model, ML system testing, Quality testing, Retrieval-augmented generation, Software Quality, Software quality testing, System testing, Software quality
National Category
Computer Systems Computer Sciences
Research subject
Business Administration; Computer Science
Identifiers
urn:nbn:se:kau:diva-104820 (URN)10.1109/ICSTW64639.2025.10962487 (DOI)001483187700031 ()2-s2.0-105004730418 (Scopus ID)979-8-3315-3467-7 (ISBN)979-8-3315-3468-4 (ISBN)
Conference
International Conference on Software Testing, Verification and Validation Workshops, ICSTW, Naples, Italy, March 31-April 4, 2025.
Funder
Region VärmlandKarlstad University
Available from: 2025-06-06 Created: 2025-06-06 Last updated: 2026-02-12Bibliographically approved
Bayram, F. & Ahmed, B. S. (2025). Towards Trustworthy Machine Learning in Production: An Overview of the Robustness in MLOps Approach. ACM Computing Surveys, 57(5), Article ID 121.
Open this publication in new window or tab >>Towards Trustworthy Machine Learning in Production: An Overview of the Robustness in MLOps Approach
2025 (English)In: ACM Computing Surveys, ISSN 0360-0300, E-ISSN 1557-7341, Vol. 57, no 5, article id 121Article in journal (Refereed) Published
Abstract [en]

Artificial intelligence (AI), and especially its sub-field of Machine Learning (ML), are impacting the daily lives of everyone with their ubiquitous applications. In recent years, AI researchers and practitioners have introduced principles and guidelines to build systems that make reliable and trustworthy decisions. From a practical perspective, conventional ML systems process historical data to extract the features that are consequently used to train ML models that perform the desired task. However, in practice, a fundamental challenge arises when the system needs to be operationalized and deployed to evolve and operate in real-life environments continuously. To address this challenge, Machine Learning Operations (MLOps) have emerged as a potential recipe for standardizing ML solutions in deployment. Although MLOps demonstrated great success in streamlining ML processes, thoroughly defining the specifications of robust MLOps approaches remains of great interest to researchers and practitioners. In this paper, we provide a comprehensive overview of the trustworthiness property of MLOps systems. Specifically, we highlight technical practices to achieve robust MLOps systems. In addition, we survey the existing research approaches that address the robustness aspects of ML systems in production. We also review the tools and software available to build MLOps systems and summarize their support to handle the robustness aspects. Finally, we present the open challenges and propose possible future directions and opportunities within this emerging field. The aim of this paper is to provide researchers and practitioners working on practical AI applications with a comprehensive view to adopt robust ML solutions in production environments.

Place, publisher, year, edition, pages
Association for Computing Machinery (ACM), 2025
Keywords
Contrastive Learning, Dataops, Machine learning operation system, Machine learning systems, Machine-learning, Modeling performance, Modelop, Operation system, Robustness, Trustworthy artificial intelligence, Adversarial machine learning
National Category
Computer Sciences Software Engineering
Research subject
Computer Science; Computer Science
Identifiers
urn:nbn:se:kau:diva-103395 (URN)10.1145/3708497 (DOI)001416336700014 ()2-s2.0-85217023025 (Scopus ID)
Funder
Knowledge Foundation, 20200067
Available from: 2025-02-25 Created: 2025-02-25 Last updated: 2026-02-12Bibliographically approved
Bayram, F., Ahmed, B. S. & Hallin, E. (2024). Adaptive data quality scoring operations framework using drift-aware mechanism for industrial applications. Journal of Systems and Software, 217, Article ID 112184.
Open this publication in new window or tab >>Adaptive data quality scoring operations framework using drift-aware mechanism for industrial applications
2024 (English)In: Journal of Systems and Software, ISSN 0164-1212, E-ISSN 1873-1228, Vol. 217, article id 112184Article in journal (Refereed) Published
Abstract [en]

Within data-driven artificial intelligence (AI) systems for industrial applications, ensuring the reliability of theincoming data streams is an integral part of trustworthy decision-making. An approach to assess data validityis data quality scoring, which assigns a score to each data point or stream based on various quality dimensions.However, certain dimensions exhibit dynamic qualities, which require adaptation on the basis of the system’scurrent conditions. Existing methods often overlook this aspect, making them inefficient in dynamic productionenvironments. In this paper, we introduce the Adaptive Data Quality Scoring Operations Framework, a novelframework developed to address the challenges posed by dynamic quality dimensions in industrial data streams.The framework introduces an innovative approach by integrating a dynamic change detector mechanism thatactively monitors and adapts to changes in data quality, ensuring the relevance of quality scores. We evaluatethe proposed framework performance in a real-world industrial use case. The experimental results reveal highpredictive performance and efficient processing time, highlighting its effectiveness in practical quality-drivenAI applications.

Place, publisher, year, edition, pages
Elsevier, 2024
Keywords
Industrial application, Adaptive data quality, Data quality scoring, Data validation, Data-driven AI, Drift detection
National Category
Artificial Intelligence
Research subject
Computer Science
Identifiers
urn:nbn:se:kau:diva-103575 (URN)10.1016/j.jss.2024.112184 (DOI)001298108100001 ()2-s2.0-85201595080 (Scopus ID)
Available from: 2025-03-17 Created: 2025-03-17 Last updated: 2026-02-12Bibliographically approved
Bayram, F. & Ahmed, B. S. (2023). A domain-region based evaluation of ML performance robustness to covariate shift. Neural Computing & Applications, 35(24), 17555-17577
Open this publication in new window or tab >>A domain-region based evaluation of ML performance robustness to covariate shift
2023 (English)In: Neural Computing & Applications, ISSN 0941-0643, E-ISSN 1433-3058, Vol. 35, no 24, p. 17555-17577Article in journal (Refereed) Published
Abstract [en]

Most machine learning methods assume that the input data distribution is the same in the training and testing phases.However, in practice, this stationarity is usually not met and the distribution of inputs differs, leading to unexpectedperformance of the learned model in deployment. The issue in which the training and test data inputs follow differentprobability distributions while the input–output relationship remains unchanged is referred to as covariate shift. In thispaper, the performance of conventional machine learning models was experimentally evaluated in the presence of covariateshift. Furthermore, a region-based evaluation was performed by decomposing the domain of probability density function ofthe input data to assess the classifier’s performance per domain region. Distributional changes were simulated in a twodimensional classification problem. Subsequently, a higher four-dimensional experiments were conducted. Based on theexperimental analysis, the Random Forests algorithm is the most robust classifier in the two-dimensional case, showing thelowest degradation rate for accuracy and F1-score metrics, with a range between 0.1% and 2.08%. Moreover, the resultsreveal that in higher-dimensional experiments, the performance of the models is predominantly influenced by the complexity of the classification function, leading to degradation rates exceeding 25% in most cases. It is also concluded that themodels exhibit high bias toward the region with high density in the input space domain of the training samples.

Place, publisher, year, edition, pages
Springer, 2023
Keywords
Degradation, Function evaluation, Input output programs, Machine learning, Probability distributions, Classifier evaluation, Concept drifts, Covariate shifts, Input datas, Machine-learning, Model degradations, Performance, Region-based, Robust machine learning, Two-dimensional, Probability density function
National Category
Computer Sciences
Research subject
Computer Science
Identifiers
urn:nbn:se:kau:diva-94905 (URN)10.1007/s00521-023-08622-w (DOI)000988193400003 ()2-s2.0-85159332578 (Scopus ID)
Funder
Knowledge Foundation, 20200067Karlstad University
Available from: 2023-05-29 Created: 2023-05-29 Last updated: 2026-02-12Bibliographically approved
Chahed, H., Usman, M., Chatterjee, A., Bayram, F., Chaudhary, R., Brunstrom, A., . . . Kassler, A. (2023). AIDA—Aholistic AI-driven networking and processing framework for industrial IoT applications. Internet of Things: Engineering Cyber Physical Human Systems, 22, Article ID 100805.
Open this publication in new window or tab >>AIDA—Aholistic AI-driven networking and processing framework for industrial IoT applications
Show others...
2023 (English)In: Internet of Things: Engineering Cyber Physical Human Systems, E-ISSN 2542-6605, Vol. 22, article id 100805Article in journal (Refereed) Published
Abstract [en]

Industry 4.0 is characterized by digitalized production facilities, where a large volume of sensors collect a vast amount of data that is used to increase the sustainability of the production by e.g. optimizing process parameters, reducing machine downtime and material waste, and the like. However, making intelligent data-driven decisions under timeliness constraints requires the integration of time-sensitive networks with reliable data ingestion and processing infrastructure with plug-in support of Machine Learning (ML) pipelines. However, such integration is difficult due to the lack of frameworks that flexibly integrate and program the networking and computing infrastructures, while allowing ML pipelines to ingest the collected data and make trustworthy decisions in real time. In this paper, we present AIDA - a novel holistic AI-driven network and processing framework for reliable data-driven real-time industrial IoT applications. AIDA manages and configures Time-Sensitive networks (TSN) to enable real-time data ingestion into an observable AI-powered edge/cloud continuum. Pluggable and trustworthy ML components that make timely decisions for various industrial IoT applications and the infrastructure itself are an intrinsic part of AIDA. We introduce the AIDA architecture, demonstrate the building blocks of our framework and illustrate it with two use cases. 

Place, publisher, year, edition, pages
Elsevier, 2023
Keywords
Edge/cloud computing, Internet of Things (IoT), Machine Learning, Time-Sensitive Networks (TSN)
National Category
Computer Engineering Computer Sciences
Research subject
Computer Science
Identifiers
urn:nbn:se:kau:diva-94900 (URN)10.1016/j.iot.2023.100805 (DOI)001053228900001 ()2-s2.0-85159450974 (Scopus ID)
Funder
Knowledge Foundation, 20200067
Available from: 2023-05-29 Created: 2023-05-29 Last updated: 2026-02-12Bibliographically approved
Bayram, F., Aupke, P., Ahmed, B. S., Kassler, A., Theocharis, A. & Forsman, J. (2023). DA-LSTM: A dynamic drift-adaptive learning framework for interval load forecasting with LSTM networks. Engineering applications of artificial intelligence, 123, Article ID 106480.
Open this publication in new window or tab >>DA-LSTM: A dynamic drift-adaptive learning framework for interval load forecasting with LSTM networks
Show others...
2023 (English)In: Engineering applications of artificial intelligence, ISSN 0952-1976, E-ISSN 1873-6769, Vol. 123, article id 106480Article in journal (Refereed) Published
Abstract [en]

Load forecasting is a crucial topic in energy management systems (EMS) due to its vital role in optimizing energy scheduling and enabling more flexible and intelligent power grid systems. As a result, these systems allow power utility companies to respond promptly to demands in the electricity market. Deep learning (DL) models have been commonly employed in load forecasting problems supported by adaptation mechanisms to cope with the changing pattern of consumption by customers, known as concept drift. A drift magnitude threshold should be defined to design change detection methods to identify drifts. While the drift magnitude in load forecasting problems can vary significantly over time, existing literature often assumes a fixed drift magnitude threshold, which should be dynamically adjusted rather than fixed during system evolution. To address this gap, in this paper, we propose a dynamic drift-adaptive Long Short-Term Memory (DA-LSTM) framework that can improve the performance of load forecasting models without requiring a drift threshold setting. We integrate several strategies into the framework based on active and passive adaptation approaches. To evaluate DA-LSTM in real-life settings, we thoroughly analyze the proposed framework and deploy it in a real-world problem through a cloud-based environment. Efficiency is evaluated in terms of the prediction performance of each approach and computational cost. The experiments show performance improvements on multiple evaluation metrics achieved by our framework compared to baseline methods from the literature. Finally, we present a trade-off analysis between prediction performance and computational costs.

Place, publisher, year, edition, pages
Elsevier, 2023
Keywords
Concept drift Change-point detection Dynamic drift adaptation Adaptive LSTM Interval load forecasting
National Category
Computer Sciences
Research subject
Computer Science
Identifiers
urn:nbn:se:kau:diva-95022 (URN)10.1016/j.engappai.2023.106480 (DOI)001013639800001 ()2-s2.0-85160615665 (Scopus ID)
Funder
Knowledge Foundation, 20200067Swedish Energy Agency, 50246-1SOLVE, 52693-1
Available from: 2023-06-02 Created: 2023-06-02 Last updated: 2026-02-12Bibliographically approved
Organisations
Identifiers
ORCID iD: ORCID iD iconorcid.org/0000-0003-0683-2783

Search in DiVA

Show all publications