Change search
Refine search result
12 1 - 50 of 59
CiteExportLink to result list
Permanent link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • apa.csl
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Rows per page
  • 5
  • 10
  • 20
  • 50
  • 100
  • 250
Sort
  • Standard (Relevance)
  • Author A-Ö
  • Author Ö-A
  • Title A-Ö
  • Title Ö-A
  • Publication type A-Ö
  • Publication type Ö-A
  • Issued (Oldest first)
  • Issued (Newest first)
  • Created (Oldest first)
  • Created (Newest first)
  • Last updated (Oldest first)
  • Last updated (Newest first)
  • Disputation date (earliest first)
  • Disputation date (latest first)
  • Standard (Relevance)
  • Author A-Ö
  • Author Ö-A
  • Title A-Ö
  • Title Ö-A
  • Publication type A-Ö
  • Publication type Ö-A
  • Issued (Oldest first)
  • Issued (Newest first)
  • Created (Oldest first)
  • Created (Newest first)
  • Last updated (Oldest first)
  • Last updated (Newest first)
  • Disputation date (earliest first)
  • Disputation date (latest first)
Select
The maximal number of hits you can export is 250. When you want to export more records please use the Create feeds function.
  • 1.
    Ahmed, Bestoun S.
    Karlstad University, Faculty of Health, Science and Technology (starting 2013), Department of Mathematics and Computer Science (from 2013).
    Open-source Defect Injection Benchmark Testbed for the Evaluation of Testing2020In: IEEE 13th International Conference on Software Testing, Validation and Verification (ICST), IEEE Computer Society, 2020, p. 442-447Conference paper (Refereed)
    Abstract [en]

    A natural method to evaluate the effectiveness of a testing technique is to measure the defect detection rate when applying the created test cases. Here, real or artificial software defects can be injected into the source code of software. For a more extensive evaluation, injection of artificial defects is usually needed and can be performed via mutation testing using code mutation operators. However, to simulate complex defects arising from a misunderstanding of design specifications, mutation testing might reach its limit in some cases. In this paper, we present an open-source benchmark testbed application that employs a complement method of artificial defect injection. The application is compiled after artificial defects are injected into its source code from predefined building blocks. The majority of the functions and user interface elements are covered by creating front-end-based automated test cases that can be used in experiments.

    Download full text (pdf)
    fulltext
  • 2.
    Ahmed, Bestoun S.
    USI SUPSI, Ist Dalle Molle Studi Intelligenza Artificiale ID, Lugano, Switzerland.;Salahaddin Univ Hawler, Software Engn Dept, Erbil, Iraq. IRQ.
    Test case minimization approach using fault detection and combinatorial optimization techniques for configuration-aware structural testing2016In: ENGINEERING SCIENCE AND TECHNOLOGY-AN INTERNATIONAL JOURNAL-JESTECH, ISSN 2215-0986, Vol. 19, no 2, p. 737-753Article in journal (Refereed)
    Abstract [en]

    This paper presents a technique to minimize the number of test cases in configuration-aware structural testing. Combinatorial optimization is used first to generate an optimized test suite by sampling the input configuration. Second, for further optimization, the generated test suite is filtered based on an adaptive mechanism by using a mutation testing technique. The initialized test suite is optimized using cuckoo search (CS) along with combinatorial approach, and mutation testing is used to seed different faults to the software-under-test, as well as to filter the test cases based on the detected faults. To measure the effectiveness of the technique, an empirical study is conducted on a software system. The technique proves its effectiveness through the conducted case study. The paper also shows the application of combinatorial optimization and CS to the software testing. (C) 2016, Karabuk University. Publishing services by Elsevier B.V.

  • 3. Ahmed, Bestoun S.
    et al.
    Bures, M.
    Czech Technical University, CZE.
    Testing of Smart TV applications: Key ingredients, challenges and proposed solutions2019In: Proceedings of the Future Technologies Conference / [ed] Bhatia R., Arai K., Kapoor S, Springer, 2019, Vol. 880, p. 241-256Conference paper (Refereed)
    Abstract [en]

    Smart TV applications are software applications that have been designed to run on smart TVs which are televisions with integrated Internet features. Nowadays, the smart TVs are going to dominate the television market, and the number of connected TVs is growing exponentially. This growth is accompanied by the increase of consumers and the use of smart TV applications that drive these devices. Due to the increasing demand for smart TV applications especially with the rise of the Internet of Things (IoT) services, it is essential to building an application with a certain level of quality. Despite the analogy between the smart TV and mobile apps, testing smart TV applications is different in many aspects due to the different nature of user interaction and development environment. To develop the field and formulate the concepts of smart TV application testing, this paper aims to provide the essential ingredients, solutions, answers to the most critical questions, and open problems. In addition, we offer initial results and proof of concepts for a creeper algorithm to detect essential views of the applications. This paper serves as an effort to report the key ingredients and challenges of the smart TV application testing systematically to the research community. © Springer Nature Switzerland AG 2019.

  • 4.
    Ahmed, Bestoun S.
    et al.
    Karlstad University, Faculty of Health, Science and Technology (starting 2013), Department of Mathematics and Computer Science (from 2013).
    Bures, Miroslav
    Czech Technical University, Czech Republic.
    EvoCreeper: Automated Black-Box Model Generation for Smart TV Applications2019In: IEEE transactions on consumer electronics, ISSN 0098-3063, E-ISSN 1558-4127, Vol. 65, no 2, p. 160-169Article in journal (Refereed)
    Abstract [en]

    Abstract—Smart TVs are coming to dominate the televisionmarket. This accompanied by an increase in the use of the smartTV applications (apps). Due to the increasing demand, developersneed modeling techniques to analyze these apps and assess theircomprehensiveness, completeness, and quality. In this paper, wepresent an automated strategy for generating models of smartTV apps based on a black-box reverse engineering. The strategycan be used to cumulatively construct a model for a given app byexploring the user interface in a manner consistent with the use ofa remote control device and extracting the runtime information.The strategy is based on capturing the states of the user interfaceto create a model during runtime without any knowledge ofthe internal structure of the app. We have implemented ourstrategy in a tool called EvoCreeper. The evaluation results showthat our strategy can automatically generate unique states anda comprehensive model that represents the real user interactionswith an app using a remote control device. The models thusgenerated can be used to assess the quality and completeness ofsmart TV apps in various contexts, such as the control of otherconsumer electronics in smart houses.

    Download full text (pdf)
    fulltext
  • 5.
    Ahmed, Bestoun S.
    et al.
    Karlstad University, Faculty of Health, Science and Technology (starting 2013), Department of Mathematics and Computer Science (from 2013).
    Bures, Miroslav
    Czech Technical University in Prague, Prague.
    Frajtak, Karel
    Czech Technical University in Prague, Prague.
    Cerny, Tomas
    Baylor University, Waco.
    Aspects of Quality in Internet of Things (IoT) Solutions: A Systematic Mapping Study2019In: IEEE Access, E-ISSN 2169-3536, Vol. 7, p. 13758-13780Article in journal (Refereed)
    Abstract [en]

    Internet of Things (IoT) is an emerging technology that has the promising power to change our future. Due to the market pressure, IoT systems may be released without sufficient testing. However, it is no longer acceptable to release IoT systems to the market without assuring the quality. As in the case of new technologies, the quality assurance process is a challenging task. This paper shows the results of the first comprehensive and systematic mapping study to structure and categories the research evidence in the literature starting in 2009 when the early publication of IoT papers for IoT quality assurance appeared. The conducted research is based on the most recent guidelines on how to perform systematic mapping studies. A set of research questions is defined carefully regarding the quality aspects of the IoT. Based on these questions, a large number of evidence and research papers is considered in the study (478 papers). We have extracted and analyzed different levels of information from those considered papers. Also, we have classified the topics addressed in those papers into categories based on the quality aspects. The study results carry out different areas that require more work and investigation in the context of IoT quality assurance. The results of the study can help in a further understanding of the research gaps. Moreover, the results show a roadmap for future research directions.

  • 6.
    Ahmed, Bestoun S.
    et al.
    Karlstad University, Faculty of Health, Science and Technology (starting 2013), Department of Mathematics and Computer Science (from 2013).
    Eduard, Enoiu
    Mälardalen University, Västerås, SWE.
    Wasif, Afzal
    Mälardalen University, Västerås, SWE.
    Kamal Z, Zamli
    University Malaysia Pahang, Pekan, MYS.
    An evaluation of Monte Carlo-based hyper-heuristic for interaction testing of industrial embedded software applications2020In: Soft Computing - A Fusion of Foundations, Methodologies and Applications, ISSN 1432-7643, E-ISSN 1433-7479, Vol. 24, no 18, p. 13929-13954Article in journal (Refereed)
    Abstract [en]

    Hyper-heuristic is a new methodology for the adaptive hybridization of meta-heuristic algorithms to derive a general algorithm for solving optimization problems. This work focuses on the selection type of hyper-heuristic, called the exponential Monte Carlo with counter (EMCQ). Current implementations rely on the memory-less selection that can be counterproductive as the selected search operator may not (historically) be the best performing operator for the current search instance. Addressing this issue, we propose to integrate the memory into EMCQ for combinatorial t-wise test suite generation using reinforcement learning based on the Q-learning mechanism, called Q-EMCQ. The limited application of combinatorial test generation on industrial programs can impact the use of such techniques as Q-EMCQ. Thus, there is a need to evaluate this kind of approach against relevant industrial software, with a purpose to show the degree of interaction required to cover the code as well as finding faults. We applied Q-EMCQ on 37 real-world industrial programs written in Function Block Diagram (FBD) language, which is used for developing a train control management system at Bombardier Transportation Sweden AB. The results show that Q-EMCQ is an efficient technique for test case generation. Addition- ally, unlike the t-wise test suite generation, which deals with the minimization problem, we have also subjected Q-EMCQ to a maximization problem involving the general module clustering to demonstrate the effectiveness of our approach. The results show the Q-EMCQ is also capable of outperforming the original EMCQ as well as several recent meta/hyper-heuristic including modified choice function, Tabu high-level hyper-heuristic, teaching learning-based optimization, sine cosine algorithm, and symbiotic optimization search in clustering quality within comparable execution time.

    Download full text (pdf)
    fulltext
  • 7. Ahmed, Bestoun S.
    et al.
    Gambardella, L. M.
    Istituto Dalle Molle di Studi SullIntelligenza Artificíale (IDSIA), CHE.
    Zamli, K. Z.
    University Malaysia Pahang, MYS.
    A new approach to speed up combinatorial search strategies using stack and hash table2016In: Proceedings of 2016 SAI Computing Conference, SAI 2016, Institute of Electrical and Electronics Engineers (IEEE), 2016, p. 1217-1222Conference paper (Refereed)
    Abstract [en]

    Owing to the significance of combinatorial search strategies both for academia and industry, the introduction of new techniques is a fast growing research field these days. These strategies have really taken different forms ranging from simple to complex strategies in order to solve all forms of combinatorial problems. Nonetheless, despite the kind of problem these approaches solve, they are prone to heavy computation with the number of combinations and growing search space dimensions. This paper presents a new approach to speed up the generation and search processes using a combination of stack and hash table data structures. This approach could be put to practice for the combinatorial approaches to speed up the generation of combinations and search process in the search space. Furthermore, this new approach proved its performance in diverse stages better than other known strategies. © 2016 IEEE.

  • 8.
    Ahmed, Bestoun S.
    et al.
    Ist Dalle Molle Studi sullIntelligenza Artificial, CH-6928 Manno Lugano, Switzerland.;Czech Tech Univ, Fac Elect Engn, Dept Comp Sci, Karlovo Nam 13, Prague 12135 2, Czech Republic. CZE.
    Gambardella, Luca M.
    Ist Dalle Molle Studi sullIntelligenza Artificial, CH-6928 Manno Lugano, Switzerland. CHE.
    Afzal, Wasif
    Mlardalen Univ, Sch Innovat Design & Engn, Vasteras, Sweden..
    Zamli, Kamal Z.
    Univ Malaysia Pahang, Fac Comp Syst & Software Engn, Gambang, Malaysia. MYS.
    Handling constraints in combinatorial interaction testing in the presence of multi objective particle swarm and multithreading2017In: Information and Software Technology, ISSN 0950-5849, E-ISSN 1873-6025, Vol. 86, p. 20-36Article in journal (Refereed)
    Abstract [en]

    Context: Combinatorial testing strategies have lately received a lot of attention as a result of their diverse applications. In its simple form, a combinatorial strategy can reduce several input parameters (configurations) of a system into a small set based on their interaction (or combination). In practice, the input configurations of software systems are subjected to constraints, especially in case of highly configurable systems. To implement this feature within a strategy, many difficulties arise for construction. While there are many combinatorial interaction testing strategies nowadays, few of them support constraints. Objective: This paper presents a new strategy, to construct combinatorial interaction test suites in the presence of constraints. Method: The design and algorithms are provided in detail. To overcome the multi-judgement criteria for an optimal solution, the multi-objective particle swarm optimisation and multithreading are used. The strategy and its associated algorithms are evaluated extensively using different benchmarks and comparisons. Results: Our results are promising as the evaluation results showed the efficiency and performance of each algorithm in the strategy. The benchmarking results also showed that the strategy can generate constrained test suites efficiently as compared to state-of-the-art strategies. Conclusion: The proposed strategy can form a new way for constructing of constrained combinatorial interaction test suites. The strategy can form a new and effective base for future implementations. (C) 2017 Elsevier B.V. All rights reserved.

  • 9.
    Ahmed, Bestoun S.
    et al.
    Karlstad University, Faculty of Health, Science and Technology (starting 2013), Department of Mathematics and Computer Science (from 2013).
    Gargantin, Angelo
    University of Bergamo, ITA.
    Bures, Miroslav
    Czech Technical University, CZE.
    An Automated Testing Framework For Smart TVapps Based on Model Separation2020In: IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW), IEEE Computer Society, 2020, p. 62-73Conference paper (Refereed)
    Abstract [en]

    Smart TV application (app) is a new technological software app that can deal with smart TV devices to add more functionality and features. Despite its importance nowadays, far too little attention has been paid to present a systematic approach to test this kind of app so far. In this paper, we present a systematic model-based testing approach for smart TV app. We used our new notion of model separation to use sub-models based on the user preference instead of the exhaustive testing to generate the test cases. Based on the constructed model, we generated a set of test cases to assess the selected paths to the chosen destination in the app. We also defined new mutation operators for smart TV app to assess our testing approach. The evaluation results showed that our approach can generate more comprehensive models of smart TV apps with less time as compared to manual exploratory testing. The results also showed that our approach can generate effective test cases in term of fault detection.

    Download full text (pdf)
    fulltext
  • 10.
    Ahmed, Bestoun S.
    et al.
    Karlstad University, Faculty of Health, Science and Technology (starting 2013), Department of Mathematics and Computer Science (from 2013). Czech Technical University, Prague, Czech Republic.
    Gargantini, Angelo
    University of Bergamo, Italy.
    Zamli, Kamal Z.
    University Malaysia Pahang, Pahang, Malaysia.
    Yilmaz, Cemal
    Sabanci University, Istanbul, Turkey.
    Bures, Miroslav
    Czech Technical University, Prague, Czech Republic.
    Szeles, Marek
    Czech Technical University, Prague, Czech Republic.
    Code-Aware Combinatorial Interaction Testing2019In: IET Software, ISSN 1751-8806, E-ISSN 1751-8814, Vol. 13, no 6, p. 600-609Article in journal (Refereed)
    Abstract [en]

    Combinatorial interaction testing (CIT) is a useful testing technique to address the interaction of input parameters in software systems. In many applications, the technique has been used as a systematic sampling technique to sample the enormous possibilities of test cases. In the last decade, most of the research activities focused on the generation of CIT test suites as it is a computationally complex problem. Although promising, less effort has been paid for the application of CIT. In general, to apply the CIT, practitioners must identify the input parameters for the Software-under-test (SUT), feed these parameters to the CIT tool to generate the test suite, and then run those tests on the application with some pass and fail criteria for verification. Using this approach, CIT is used as a black-box testing technique without knowing the effect of the internal code. Although useful, practically, not all the parameters having the same impact on the SUT. This paper introduces a different approach to use the CIT as a gray-box testing technique by considering the internal code structure of the SUT to know the impact of each input parameter and thus use this impact in the test generation stage. We applied our approach to five reliable case studies. The results showed that this approach would help to detect new faults as compared to the equal impact parameter approach.

    Download full text (pdf)
    fulltext
  • 11.
    Ahmed, Bestoun S.
    et al.
    Karlstad University, Faculty of Health, Science and Technology (starting 2013), Department of Mathematics and Computer Science (from 2013).
    Pahim, Amador
    Red Hat Czech s.r.o., Brno, Czech Republic .
    Rosa Junior, Cleber R
    Red Hat, Inc., Westford, USA .
    Kuhn, D. Richard
    Natl Inst of Standards and Technology, Gaithersburg, MD, USA .
    Bures, Miroslav
    Dept of Computer Science, Czech Technical Univ, Prague, Czech Republic .
    Towards an Automated Unified Framework to Run Applications for Combinatorial Interaction Testing2019In: EASE '19 Proceedings of the Evaluation and Assessment on Software Engineering, NY, USA: Association for Computing Machinery (ACM), 2019, p. 252-258Conference paper (Refereed)
    Abstract [en]

    Combinatorial interaction testing (CIT) is a well-known technique,but the industrial experience is needed to determine its effectivenessin different application domains. We present a case study introducinga unified framework for generating, executing and verifyingCIT test suites, based on the open-source Avocado test framework.In addition, we present a new industrial case study to demonstratethe effectiveness of the framework. This evaluation showed thatthe new framework can generate, execute, and verify effective combinatorialinteraction test suites for detecting configuration failures(invalid configurations) in a virtualization system.

  • 12.
    Ahmed, Bestoun S.
    et al.
    Ist Dalle Molle Studi SullIntelligenza Artificial, CH-6928 Manno Lugano, Switzerland.;Salahaddin Univ, Engn Coll, Software & Informat Engn Dept, Erbil, Kurdistan Regio, Iraq. IRQ.
    Sahib, Mouayad A.
    Salahaddin Univ, Engn Coll, Software & Informat Engn Dept, Erbil, Kurdistan Regio, Iraq. IRQ.
    Gambardella, Luca M.
    Ist Dalle Molle Studi SullIntelligenza Artificial, CH-6928 Manno Lugano, Switzerland. CHE.
    Afzal, Wasif
    Malardalen Univ, Sch Innovat Design & Engn, Vasteras, Sweden..
    Zamli, Kamal Z.
    Univ Malaysia Pahang Lebuhraya Tun Rezak, Fac Comp Syst & Software Engn, IBM Ctr Excellence, Kuantan 26300, Pahang Darul Ma, Malaysia. MYS.
    Optimum Design of (PID mu)-D-lambda controller for an automatic voltage regulator system using combinatorial test design2016In: PLOS ONE, E-ISSN 1932-6203, Vol. 11, no 11, article id e0166150Article in journal (Refereed)
    Abstract [en]

    Combinatorial test design is a plan of test that aims to reduce the amount of test cases systematically by choosing a subset of the test cases based on the combination of input variables. The subset covers all possible combinations of a given strength and hence tries to match the effectiveness of the exhaustive set. This mechanism of reduction has been used successfully in software testing research with t-way testing (where t indicates the interaction strength of combinations). Potentially, other systems may exhibit many similarities with this approach. Hence, it could form an emerging application in different areas of research due to its usefulness. To this end, more recently it has been applied in a few research areas successfully. In this paper, we explore the applicability of combinatorial test design technique for Fractional Order (FO), Proportional-Integral-Derivative (PID) parameter design controller, named as FOPID, for an automatic voltage regulator (AVR) system. Throughout the paper, we justify this new application theoretically and practically through simulations. In addition, we report on first experiments indicating its practical use in this field. We design different algorithms and adapted other strategies to cover all the combinations with an optimum and effective test set. Our findings indicate that combinatorial test design can find the combinations that lead to optimum design. Besides this, we also found that by increasing the strength of combination, we can approach to the optimum design in a way that with only 4-way combinatorial set, we can get the effectiveness of an exhaustive test set. This significantly reduced the number of tests needed and thus leads to an approach that optimizes design of parameters quickly.

  • 13.
    Ahmed, Bestoun S.
    et al.
    Czech Tech Univ, Fac Elect Engn, Dept Comp Sci, Software Testing Intelligent Lab, Prague 12135, Czech Republic. CZE.
    Zamli, Kamal Z.
    Univ Malaysia Pahang, Fac Comp Syst & Software Engn, Gambang 26300, Malaysia. MYS.
    Afzal, Wasif
    Malardalen Univ, Sch Innovat Design & Engn, S-72123 Vasteras, Sweden..
    Bures, Miroslav
    Czech Tech Univ, Fac Elect Engn, Dept Comp Sci, Software Testing Intelligent Lab, Prague 12135, Czech Republic. CZE.
    Constrained interaction testing: A systematic literature study2017In: IEEE Access, E-ISSN 2169-3536, Vol. 5, p. 25706-25730Article in journal (Refereed)
    Abstract [en]

    Interaction testing can be used to effectively detect faults that are otherwise difficult to find by other testing techniques. However, in practice, the input configurations of software systems are subjected to constraints, especially in the case of highly configurable systems. Handling constraints effectively and efficiently in combinatorial interaction testing is a challenging problem. Nevertheless, researchers have attacked this challenge through different techniques, and much progress has been achieved in the past decade. Thus, it is useful to reflect on the current achievements and shortcomings and to identify potential areas of improvements. This paper presents the first comprehensive and systematic literature study to structure and categorize the research contributions for constrained interaction testing. Following the guidelines of conducting a literature study, the relevant data are extracted from a set of 103 research papers belonging to constrained interaction testing. The topics addressed in constrained interaction testing research are classified into four categories of constraint test generation, application, generation and application, and model validation studies. The papers within each of these categories are extensively reviewed. Apart from answering several other research questions, this paper also discusses the applications of constrained interaction testing in several domains, such as software product lines, fault detection and characterization, test selection, security, and graphical user interface testing. This paper ends with a discussion of limitations, challenges, and future work in the area.

  • 14.
    Alsarhan, Qusay
    et al.
    University of Duhok, IRQ.
    Ahmed, Bestoun S.
    Ceske Vysoke Uceni Technicke v Praze, CZE.
    Bures, Miroslav
    Ceske Vysoke Uceni Technicke v Praze, CZE.
    Zamli, Kamal
    Universiti Malaysia Pahang, MYS.
    Software module clustering: an in-depth literature analysis2022In: IEEE Transactions on Software Engineering, ISSN 0098-5589, E-ISSN 1939-3520, Vol. 48, no 6, p. 1905-1928Article in journal (Refereed)
    Abstract [en]

    Software module clustering is an unsupervised learning method used to cluster software entities (e.g., classes, modules, or files) of similar features. The obtained clusters may be used to study, analyze, and understand the structure and behavior of the software entities. Implementing software module clustering with optimal results is challenging. Accordingly, researchers have addressed many aspects of software module clustering in the last decade. Thus, it is essential to present research evidence that has been published in this area. In this study, 143 research papers that examined software module clustering from well-known literature databases were extensively reviewed to extract useful data. The obtained data were then used to answer several research questions regarding state-of-the-art clustering approaches, applications of clustering in software engineering, clustering process, clustering algorithms, and evaluation methods. Several research gaps and challenges in software module clustering are discussed in this paper to provide a useful reference for researchers in this field.

  • 15.
    Bayram, Firas
    et al.
    Karlstad University, Faculty of Health, Science and Technology (starting 2013), Department of Mathematics and Computer Science (from 2013).
    Ahmed, Bestoun S.
    Karlstad University, Faculty of Health, Science and Technology (starting 2013), Department of Mathematics and Computer Science (from 2013). Czech Technical University in Prague, Czech Republic.
    A domain-region based evaluation of ML performance robustness to covariate shift2023In: Neural Computing & Applications, ISSN 0941-0643, E-ISSN 1433-3058, Vol. 35, no 24, p. 17555-17577Article in journal (Refereed)
    Abstract [en]

    Most machine learning methods assume that the input data distribution is the same in the training and testing phases.However, in practice, this stationarity is usually not met and the distribution of inputs differs, leading to unexpectedperformance of the learned model in deployment. The issue in which the training and test data inputs follow differentprobability distributions while the input–output relationship remains unchanged is referred to as covariate shift. In thispaper, the performance of conventional machine learning models was experimentally evaluated in the presence of covariateshift. Furthermore, a region-based evaluation was performed by decomposing the domain of probability density function ofthe input data to assess the classifier’s performance per domain region. Distributional changes were simulated in a twodimensional classification problem. Subsequently, a higher four-dimensional experiments were conducted. Based on theexperimental analysis, the Random Forests algorithm is the most robust classifier in the two-dimensional case, showing thelowest degradation rate for accuracy and F1-score metrics, with a range between 0.1% and 2.08%. Moreover, the resultsreveal that in higher-dimensional experiments, the performance of the models is predominantly influenced by the complexity of the classification function, leading to degradation rates exceeding 25% in most cases. It is also concluded that themodels exhibit high bias toward the region with high density in the input space domain of the training samples.

    Download full text (pdf)
    fulltext
  • 16.
    Bayram, Firas
    et al.
    Karlstad University, Faculty of Health, Science and Technology (starting 2013), Department of Mathematics and Computer Science (from 2013).
    Ahmed, Bestoun S.
    Karlstad University, Faculty of Health, Science and Technology (starting 2013), Department of Mathematics and Computer Science (from 2013).
    Hallin, ERIK
    Uddeholms AB, Värmlands län.
    Engman, Anton
    Uddeholms AB, Värmlands län.
    A Drift Handling Approach for Self-Adaptive ML Software in Scalable Industrial Processes2022In: Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering / [ed] Mario Aehnelt and Thomas Kirste, Association for Computing Machinery (ACM), 2022, p. 1-5, article id 129Conference paper (Refereed)
    Abstract [en]

    Most industrial processes in real-world manufacturing applications are characterized by the scalability property, which requires an automated strategy to self-adapt machine learning (ML) software systems to the new conditions. In this paper, we investigate an Electroslag Remelting (ESR) use case process from the Uddeholms AB steel company. The use case involves predicting the minimum pressure value for a vacuum pumping event. Taking into account the long time required to collect new records and efficiently integrate the new machines with the built ML software system. Additionally, to accommodate the changes and satisfy the non-functional requirement of the software system, namely adaptability, we propose an automated and adaptive approach based on a drift handling technique called importance weighting. The aim is to address the problem of adding a new furnace to production and enable the adaptability attribute of the ML software. The overall results demonstrate the improvements in ML software performance achieved by implementing the proposed approach over the classical non-adaptive approach. 

    Download full text (pdf)
    fulltext
  • 17.
    Bayram, Firas
    et al.
    Karlstad University, Faculty of Health, Science and Technology (starting 2013), Department of Mathematics and Computer Science (from 2013).
    Ahmed, Bestoun S.
    Karlstad University, Faculty of Health, Science and Technology (starting 2013), Department of Mathematics and Computer Science (from 2013).
    Hallin, Erik
    Uddeholms AB, Sweden.
    Engman, Anton
    Uddeholms AB, Sweden.
    DQSOps: Data Quality Scoring Operations Framework for Data-Driven Applications2023In: EASE '23: Proceedings of the 27th International Conference on Evaluation and Assessment in Software Engineering, Association for Computing Machinery (ACM), 2023, p. 32-41Conference paper (Refereed)
    Abstract [en]

    Data quality assessment has become a prominent component in the successful execution of complex data-driven artificial intelligence (AI) software systems. In practice, real-world applications generate huge volumes of data at speeds. These data streams require analysis and preprocessing before being permanently stored or used in a learning task. Therefore, significant attention has been paid to the systematic management and construction of high-quality datasets. Nevertheless, managing voluminous and high-velocity data streams is usually performed manually (i.e. offline), making it an impractical strategy in production environments. To address this challenge, DataOps has emerged to achieve life-cycle automation of data processes using DevOps principles. However, determining the data quality based on a fitness scale constitutes a complex task within the framework of DataOps. This paper presents a novel Data Quality Scoring Operations (DQSOps) framework that yields a quality score for production data in DataOps workflows. The framework incorporates two scoring approaches, an ML prediction-based approach that predicts the data quality score and a standard-based approach that periodically produces the ground-truth scores based on assessing several data quality dimensions. We deploy the DQSOps framework in a real-world industrial use case. The results show that DQSOps achieves significant computational speedup rates compared to the conventional approach of data quality scoring while maintaining high prediction performance.

    Download full text (pdf)
    fulltext
  • 18.
    Bayram, Firas
    et al.
    Karlstad University, Faculty of Health, Science and Technology (starting 2013), Department of Mathematics and Computer Science (from 2013).
    Ahmed, Bestoun S.
    Karlstad University, Faculty of Health, Science and Technology (starting 2013), Department of Mathematics and Computer Science (from 2013).
    Kassler, Andreas
    Karlstad University, Faculty of Health, Science and Technology (starting 2013), Department of Mathematics and Computer Science (from 2013).
    From concept drift to model degradation: An overview on performance-aware drift detectors2022In: Knowledge-Based Systems, ISSN 0950-7051, E-ISSN 1872-7409, Vol. 245, article id 108632Article, review/survey (Refereed)
    Abstract [en]

    The dynamicity of real-world systems poses a significant challenge to deployed predictive machine learning (ML) models. Changes in the system on which the ML model has been trained may lead to performance degradation during the system’s life cycle. Recent advances that study non-stationary environments have mainly focused on identifying and addressing such changes caused by a phenomenon called concept drift. Different terms have been used in the literature to refer to the same type of concept drift and the same term for various types. This lack of unified terminology is set out to create confusion on distinguishing between different concept drift variants. In this paper, we start by grouping concept drift types by their mathematical definitions and survey the different terms used in the literature to build a consolidated taxonomy of the field. We also review and classify performance-based concept drift detection methods proposed in the last decade. These methods utilize the predictive model’s performance degradation to signal substantial changes in the systems. The classification is outlined in a hierarchical diagram to provide an orderly navigation between the methods. We present a comprehensive analysis of the main attributes and strategies for tracking and evaluating the model’s performance in the predictive system. The paper concludes by discussing open research challenges and possible research directions.

    Download full text (pdf)
    fulltext
  • 19.
    Bayram, Firas
    et al.
    Karlstad University, Faculty of Health, Science and Technology (starting 2013), Department of Mathematics and Computer Science (from 2013).
    Aupke, Phil
    Karlstad University, Faculty of Health, Science and Technology (starting 2013), Department of Mathematics and Computer Science (from 2013).
    Ahmed, Bestoun S.
    Karlstad University, Faculty of Health, Science and Technology (starting 2013), Department of Mathematics and Computer Science (from 2013).
    Kassler, Andreas
    Karlstad University, Faculty of Health, Science and Technology (starting 2013), Department of Mathematics and Computer Science (from 2013).
    Theocharis, Andreas
    Karlstad University, Faculty of Health, Science and Technology (starting 2013), Department of Engineering and Physics (from 2013).
    Forsman, Jonas
    CGI, Karlstad, Sweden.
    DA-LSTM: A dynamic drift-adaptive learning framework for interval load forecasting with LSTM networks2023In: Engineering applications of artificial intelligence, ISSN 0952-1976, E-ISSN 1873-6769, Vol. 123, article id 106480Article in journal (Refereed)
    Abstract [en]

    Load forecasting is a crucial topic in energy management systems (EMS) due to its vital role in optimizing energy scheduling and enabling more flexible and intelligent power grid systems. As a result, these systems allow power utility companies to respond promptly to demands in the electricity market. Deep learning (DL) models have been commonly employed in load forecasting problems supported by adaptation mechanisms to cope with the changing pattern of consumption by customers, known as concept drift. A drift magnitude threshold should be defined to design change detection methods to identify drifts. While the drift magnitude in load forecasting problems can vary significantly over time, existing literature often assumes a fixed drift magnitude threshold, which should be dynamically adjusted rather than fixed during system evolution. To address this gap, in this paper, we propose a dynamic drift-adaptive Long Short-Term Memory (DA-LSTM) framework that can improve the performance of load forecasting models without requiring a drift threshold setting. We integrate several strategies into the framework based on active and passive adaptation approaches. To evaluate DA-LSTM in real-life settings, we thoroughly analyze the proposed framework and deploy it in a real-world problem through a cloud-based environment. Efficiency is evaluated in terms of the prediction performance of each approach and computational cost. The experiments show performance improvements on multiple evaluation metrics achieved by our framework compared to baseline methods from the literature. Finally, we present a trade-off analysis between prediction performance and computational costs.

    Download full text (pdf)
    fulltext
  • 20.
    Bures, M.
    et al.
    Czech Technical University, CZE.
    Ahmed, Bestoun S.
    Czech Technical University, CZE.
    On the Effectiveness of Combinatorial Interaction Testing: A Case Study2017In: Proceedings - 2017 IEEE International Conference on Software Quality, Reliability and Security Companion, QRS-C 2017, 2017, p. 69-76Conference paper (Refereed)
    Abstract [en]

    Combinatorial interaction testing (CIT) stands as one of the efficient testing techniques that have been used in different applications recently. The technique is useful when there is a need to take the interaction of input parameters into consideration for testing a system. The key insight the technique is that not every single parameter may contribute to the failure of the system and there could be interactions among these parameters. Hence, there must be combinations of these input parameters based on the interaction strength. This technique has been used in many applications to assess its effectiveness. In this paper, we are addressing the effectiveness of CIT for a real-world case study using model-based mutation testing experiments. The contribution of the paper is threefold: First we introduce an effective testing application for CIT; Second, we address the effectiveness of increasing the interaction strength beyond the pairwise (i.e., interaction of more than two parameters); Third, model-based mutation testing is used to mutate the input model of the program in contrast to the traditional code-based mutation testing process. Experimental results showed that CIT is an effective testing technique for this kind of application. In addition, the results also showed the usefulness of model-based mutation testing to assess CIT applications. For the subject of this case study, the results also indicate that 3-way test suite (i.e., interaction of three parameters) could detect new faults that can not be detected by pairwise. © 2017 IEEE.

  • 21.
    Bures, M.
    et al.
    FEE, Czech Technical University, CZE.
    Cerny, T.
    Baylor University, USA.
    Ahmed, Bestoun S.
    Karlstad University, Faculty of Health, Science and Technology (starting 2013), Department of Mathematics and Computer Science (from 2013). FEE, Czech Technical University, CZE.
    Internet of things: Current challenges in the quality assurance and testing methods2019In: Information Science and Applications 2018. ICISA / [ed] K Kim ; N Baek N, 2019, p. 625-634Conference paper (Refereed)
    Abstract [en]

    Contemporary development of the Internet of Things (IoT) technology brings a number of challenges in the Quality Assurance area. Current issues related to security, user’s privacy, the reliability of the service, interoperability, and integration are discussed. All these create a demand for specific Quality Assurance methodology for the IoT solutions. In the paper, we present the state of the art of this domain and we discuss particular areas of system testing discipline, which is not covered by related work sufficiently so far. This analysis is supported by results of a recent survey we performed among ten IoT solutions providers, covering various areas of IoT applications. © 2019, Springer Nature Singapore Pte Ltd.

  • 22.
    Bures, M.
    et al.
    Czech Technical University in Prague, CZE.
    Klima, M.
    Czech Technical University in Prague, CZE.
    Rechtberger, V.
    Czech Technical University in Prague, CZE.
    Bellekens, X.
    University of Strathclyde, GBR.
    Tachtatzis, C.
    University of Strathclyde, GBR.
    Atkinson, R.
    University of Strathclyde, GBR.
    Ahmed, Bestoun S.
    Karlstad University, Faculty of Health, Science and Technology (starting 2013), Department of Mathematics and Computer Science (from 2013). Czech Technical University in Prague, CZE.
    Interoperability and Integration Testing Methods for IoT Systems: A Systematic Mapping Study2020In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Springer Science+Business Media B.V., 2020, p. 93-112Conference paper (Refereed)
    Abstract [en]

    The recent active development of Internet of Things (IoT) solutions in various domains has led to an increased demand for security, safety, and reliability of these systems. Security and data privacy are currently the most frequently discussed topics; however, other reliability aspects also need to be focused on to maintain smooth and safe operation of IoT systems. Until now, there has been no systematic mapping study dedicated to the topic of interoperability and integration testing of IoT systems specifically; therefore, we present such an overview in this study. We analyze 803 papers from four major primary databases and perform detailed assessment and quality check to find 115 relevant papers. In addition, recently published testing techniques and approaches are analyzed and classified; the challenges and limitations in the field are also identified and discussed. Research trends related to publication time, active researchers, and publication media are presented in this study. The results suggest that studies mainly focus only on general testing methods, which can be applied to integration and interoperability testing of IoT systems; thus, there are research opportunities to develop additional testing methods focused specifically on IoT systems, so that they are more effective in the IoT context.

  • 23.
    Bures, Miroslav
    et al.
    Czech Technical University, Czech Republic.
    Ahmed, Bestoun S.
    Karlstad University, Faculty of Health, Science and Technology (starting 2013), Department of Mathematics and Computer Science (from 2013). Czech Technical University, Czech Republic.
    Employment of multiple algorithms for optimal path-based test selection strategy2019In: Information and Software Technology, ISSN 0950-5849, E-ISSN 1873-6025, Vol. 114, p. 21-36Article in journal (Refereed)
    Abstract [en]

    Context

    Executing various sequences of system functions in a system under test represents one of the primary techniques in software testing. The natural method for creating effective, consistent and efficient test sequences is to model the system under test and employ an algorithm to generate tests that satisfy a defined test coverage criterion. Several criteria for preferred test set properties can be defined. In addition, to optimize the test set from an economic viewpoint, the priorities of the various parts of the system model under test must be defined.

    Objective

    Using this prioritization, the test cases exercise the high-priority parts of the system under test by more path combinations than those with low priority (this prioritization can be combined with the test coverage criterion that determines how many path combinations of the individual parts of the system are tested). Evidence from the literature and our observations confirm that finding a universal algorithm that produces a test set with preferred properties for all test coverage criteria is a challenging task. Moreover, for different individual problem instances, different algorithms provide results with the best value of a preferred property. In this paper, we present a portfolio-based strategy to perform the best test selection.

    Method

    The proposed strategy first employs a set of current algorithms to generate test sets; then, a preferred property of each test set is assessed in terms of the selected criterion, and finally, the test set with the best value of a preferred property is chosen.

    Results

    The experimental results confirm the validity and usefulness of this strategy. For individual instances of 50 system under test models, different algorithms provided results having the best preferred property value; these results varied by the required test coverage level, the size of the priority parts of the model, and the selected test set preferred property criteria.

    Conclusion

    In addition to the used algorithms, the proposed strategy can be used to assess the optimality of different path-based testing algorithms and choose a suitable algorithm for the testing.

  • 24.
    Bures, Miroslav
    et al.
    Czech Tech Univ, Dept Comp Sci, FEE, Prague, Czech Republic.
    Ahmed, Bestoun. S
    Karlstad University, Faculty of Health, Science and Technology (starting 2013), Department of Mathematics and Computer Science (from 2013).
    Rechtberger, Vaclav
    Czech Tech Univ, Dept Comp Sci, FEE, Prague, Czech Republic.
    Klima, Matej
    Czech Tech Univ, Dept Comp Sci, FEE, Prague, Czech Republic.
    Trnka, Michal
    Czech Tech Univ, Dept Comp Sci, FEE, Prague, Czech Republic.
    Jaros, Miroslav
    Red Hat Czech Sro, Brno, Czech Republic.
    Bellekens, Xavier
    Univ Strathclyde, Dept Elect & Elect Engn, Glasgow, Lanark, Scotland.
    Almog, Dani
    Shamoon Coll Engn, Software Engn Dept, Beer Sheva, Israel.
    Herout, Pavel
    Univ West Bohemia, Dept CS & Engn, FAS, Plzen, Czech Republic.
    PatrIoT: IoT Automated Interoperability and Integration Testing Framework2021In: IEEE International Conference on Software Testing Verification and Validation, IEEE, 2021, p. 454-459, article id 9438582Conference paper (Refereed)
    Abstract [en]

    With the rapid growth of the contemporary Internet of Things (IoT) market, the established systems raise a number of concerns regarding the reliability and the potential presence of critical integration defects. In this paper, we present a PatrIoT framework that aims to provide flexible support to construct an effective IoT system testbed to implement automated interoperability and integration testing. The framework allows scaling from a pure physical testbed to a simulated environment using a number of predefined modules and elements to simulate an IoT device or part of the tested infrastructure. PatrIoT also contains a set of reference example testbeds and several sets of example automated tests for a smart street use case. 

  • 25.
    Bures, Miroslav
    et al.
    Czech Tech Univ, Software Testing Intelligent Lab STILL, Dept Comp Sci, Fac Elect Engn, Karlovo Nam 13, Prague 12135 2, Czech Republic. CZE.
    Ahmed, Bestoun S.
    Czech Tech Univ, Software Testing Intelligent Lab STILL, Dept Comp Sci, Fac Elect Engn, Karlovo Nam 13, Prague 12135 2, Czech Republic. CZE.
    Zamli, Kamal Z.
    Univ Malaysia Pahang, IBM Ctr Excellence, Fac Comp Syst & Software Engn, Gambang, Malaysia. MYS.
    Prioritized Process Test: An Alternative to Current Process Testing Strategies2019In: International journal of software engineering and knowledge engineering, ISSN 0218-1940, Vol. 29, no 7, p. 997-1028Article in journal (Refereed)
    Abstract [en]

    Testing processes and workflows in information and Internet of Things systems is a major part of the typical software testing effort. Consistent and efficient path-based test cases are desired to support these tests. Because certain parts of software system workflows have a higher business priority than others, this fact has to be involved in the generation of test cases. In this paper, we propose a Prioritized Process Test (PPT), which is a model-based test case generation algorithm that represents an alternative to currently established algorithms that use directed graphs and test requirements to model the system under test. The PPT accepts a directed multigraph as a model to express priorities, and edge weights are used instead of test requirements. To determine the test-coverage level of test cases, a test-depth-level concept is used. We compared the presented PPT with five alternatives (i.e. the Process Cycle Test (PCT), a naive reduction of test set created by the PCT, Brute Force algorithm, Set-covering-Based Solution and Matching-based Prefix Graph Solution) for edge coverage and edge-pair coverage. To assess the optimality of the path-based test cases produced by these strategies, we used 14 metrics based on the properties of these test cases and 59 models that were created for three real-world systems. For all edge coverage, the PPT produced more optimal test cases than the alternatives in terms of the majority of the metrics. For edge-pair coverage, the PPT strategy yielded similar results to those of the alternatives. Thus, the PPT strategy is an applicable alternative as it reflects both the required test coverage level and the business priority in parallel.

  • 26.
    Bures, Miroslav
    et al.
    The Czech Technical University in Prague, CZE.
    Cerny, Tomas
    ECS Baylor University, USA..
    Frajtak, Karel
    The Czech Technical University in Prague, CZE.
    Ahmed, Bestoun S.
    The Czech Technical University in Prague, CZE ;Salahaddin University, IRQ.
    Testing the consistency of business data objects using extended static testing of CRUD matrices2019In: Cluster Computing, ISSN 1386-7857, E-ISSN 1573-7543, Vol. 22, p. 963-976Article in journal (Refereed)
    Abstract [en]

    Static testing is used to detect software defects in the earlier phases of the software development lifecycle, which makes the total costs caused by defects lower and the software development project less risky. Different types of static testing have been introduced and are used in software projects. In this paper, we focus on static testing related to data consistency in a software system. In particular, we propose extensions to contemporary static testing techniques based on CRUD matrices, employing cross-verifications between various types of CRUD matrices made by different parties at various stages of the software project. Based on performed experiments, the proposed static testing technique significantly improves the consistency of Data Cycle Test cases. Together with this trend, we observe growing potential of test cases to detect data consistency defects in the system under test, when utilizing the proposed technique.

  • 27.
    Bures, Miroslav
    et al.
    Czech Tech Univ, Fac Elect Engn, Dept Comp Sci, Software Testing Intelligent Lab, Prague 12135 2, Czech Republic. CZE.
    Frajtak, Karel
    Czech Tech Univ, Fac Elect Engn, Dept Comp Sci, Software Testing Intelligent Lab, Prague 12135 2, Czech Republic. CZE.
    Ahmed, Bestoun S.
    Czech Tech Univ, Fac Elect Engn, Dept Comp Sci, Software Testing Intelligent Lab, Prague 12135 2, Czech Republic. CZE.
    Tapir: Automation Support of Exploratory Testing Using Model Reconstruction of the System Under Test2018In: IEEE Transactions on Reliability, ISSN 0018-9529, E-ISSN 1558-1721, Vol. 67, no 2, p. 557-580Article in journal (Refereed)
    Abstract [en]

    For a considerable number of software projects, the creation of effective test cases is hindered by design documentation that is either lacking, incomplete, or obsolete. The exploratory testing approach can serve as a sound method in such situations. However, the efficiency of this testing approach strongly depends on the method, the documentation of explored parts of a system, the organization and distribution of work among individual testers on a team, and the minimization of potential (very probable) duplicities in performed tests. In this paper, we present a framework for replacing and automating a portion of these tasks. A screen-flow-based model of the tested system is incrementally reconstructed during the exploratory testing process by tracking testers' activities. With additional metadata, the model serves for an automated navigation process for a tester. Compared with the exploratory testing approach, which is manually performed in two case studies, the proposed framework allows the testers to explore a greater extent of the tested system and enables greater detection of the defects present in the system. The results show that the time efficiency of the testing process improved with the framework support. This efficiency can be increased by team-based navigational strategies that are implemented within the proposed framework, which is documented by another case study presented in this paper.

  • 28.
    Bures, Miroslav
    et al.
    Czech Technical University in Prague, Czechia.
    Klima, Matej
    Czech Technical University in Prague, Czechia.
    Rechtberger, Vaclav
    Czech Technical University in Prague, Czechia.
    Ahmed, Bestoun. S
    Karlstad University, Faculty of Health, Science and Technology (starting 2013), Department of Mathematics and Computer Science (from 2013).
    Hindy, Hanan
    Abertay University, UK.
    Bellekens, Xavier
    University of Strathclyde, UK.
    Review of Specific Features and Challenges in the Current Internet of Things Systems Impacting Their Security and Reliability2021In: Trends and Applications in Information Systems and Technologies, Springer, 2021, Vol. 1367Conference paper (Refereed)
    Abstract [en]

    The current development of the Internet of Things (IoT) technology poses significant challenges to researchers and industry practitioners. Among these challenges, security and reliability particularly deserve attention. In this paper, we provide a consolidated analysis of the root causes of these challenges, their relations, and their possible impacts on IoT systems’ general quality characteristics. Further understanding of these challenges is useful for IoT quality engineers when defining testing strategies for their systems and researchers to consider when discussing possible research directions. In this study, twenty specific features of current IoT systems are discussed, divided into five main categories: (1) Economic, managerial and organisational aspects, (2) Infrastructural challenges, (3) Security and privacy challenges, (4) Complexity challenges and (5) Interoperability problems. 

  • 29.
    Bures, Miroslav
    et al.
    Czech Technical University, Czechia.
    Macik, Miroslav
    Czech Technical University, Czechia.
    Ahmed, Bestoun S.
    Karlstad University, Faculty of Health, Science and Technology (starting 2013), Department of Mathematics and Computer Science (from 2013). Czech Technical University, Czechia.
    Rechtberger, Vaclav
    Czech Technical University, Czechia.
    Slavik, Pavel
    Czech Technical University, Czechia.
    Testing the Usability and Accessibility of Smart TV Applications Using an Automated Model-based Approach2020In: IEEE transactions on consumer electronics, ISSN 0098-3063, E-ISSN 1558-4127, Vol. 66, no 2, p. 134-143Article in journal (Refereed)
    Abstract [en]

    As the popularity of Smart Televisions (TVs) and interactive Smart TV applications (apps) has recently grown, the usability of these apps has become an important quality characteristic. Previous studies examined Smart TV apps from a usability perspective. However, these methods are mainly manual, and the potential of automated model-based testing methods for usability testing purposes has not yet been fully explored. In this paper, we propose an approach to test the usability of Smart TV apps based on the automated generation of a Smart TV user interaction model from an existing app by a specialized automated crawler. By means of this model, defined user tasks in the Smart TV app can be evaluated automatically in terms of their feasibility and estimated user effort, which reflects the usability of the analyzed app. This analysis can be applied in the context of regular users and users with various specific needs. The findings from this model-based automated analysis approach can be used to optimize the user interface of a Smart TV app to increase its usability, accessibility, and quality.

    Download full text (pdf)
    fulltext
  • 30.
    Chahed, Hamza
    et al.
    Karlstad University, Faculty of Health, Science and Technology (starting 2013), Department of Mathematics and Computer Science (from 2013).
    Usman, Muhammad
    Karlstad University, Faculty of Health, Science and Technology (starting 2013), Department of Mathematics and Computer Science (from 2013).
    Chatterjee, Ayan
    Karlstad University, Faculty of Health, Science and Technology (starting 2013), Department of Mathematics and Computer Science (from 2013).
    Bayram, Firas
    Karlstad University, Faculty of Health, Science and Technology (starting 2013), Department of Mathematics and Computer Science (from 2013).
    Chaudhary, Rajat
    Karlstad University, Faculty of Health, Science and Technology (starting 2013), Department of Mathematics and Computer Science (from 2013).
    Brunstrom, Anna
    Karlstad University, Faculty of Health, Science and Technology (starting 2013), Department of Mathematics and Computer Science (from 2013).
    Taheri, Javid
    Karlstad University, Faculty of Health, Science and Technology (starting 2013), Department of Mathematics and Computer Science (from 2013).
    Ahmed, Bestoun S.
    Karlstad University, Faculty of Health, Science and Technology (starting 2013), Department of Mathematics and Computer Science (from 2013). Czech Technical University in Prague, Czech Republic.
    Kassler, Andreas
    Karlstad University, Faculty of Health, Science and Technology (starting 2013), Department of Mathematics and Computer Science (from 2013). Deggendorf Institute of Technology, Germany.
    AIDA—Aholistic AI-driven networking and processing framework for industrial IoT applications2023In: Internet of Things: Engineering Cyber Physical Human Systems, E-ISSN 2542-6605, Vol. 22, article id 100805Article in journal (Refereed)
    Abstract [en]

    Industry 4.0 is characterized by digitalized production facilities, where a large volume of sensors collect a vast amount of data that is used to increase the sustainability of the production by e.g. optimizing process parameters, reducing machine downtime and material waste, and the like. However, making intelligent data-driven decisions under timeliness constraints requires the integration of time-sensitive networks with reliable data ingestion and processing infrastructure with plug-in support of Machine Learning (ML) pipelines. However, such integration is difficult due to the lack of frameworks that flexibly integrate and program the networking and computing infrastructures, while allowing ML pipelines to ingest the collected data and make trustworthy decisions in real time. In this paper, we present AIDA - a novel holistic AI-driven network and processing framework for reliable data-driven real-time industrial IoT applications. AIDA manages and configures Time-Sensitive networks (TSN) to enable real-time data ingestion into an observable AI-powered edge/cloud continuum. Pluggable and trustworthy ML components that make timely decisions for various industrial IoT applications and the infrastructure itself are an intrinsic part of AIDA. We introduce the AIDA architecture, demonstrate the building blocks of our framework and illustrate it with two use cases. 

    Download full text (pdf)
    fulltext
  • 31.
    Chatterjee, Ayan
    et al.
    Karlstad University, Faculty of Health, Science and Technology (starting 2013), Department of Mathematics and Computer Science (from 2013).
    Ahmed, Bestoun S.
    Karlstad University, Faculty of Health, Science and Technology (starting 2013), Department of Mathematics and Computer Science (from 2013). Czech Technical University in Prague, Czech Republic.
    IoT anomaly detection methods and applications: A survey2022In: Internet of Things: Engineering Cyber Physical Human Systems, E-ISSN 2542-6605, Vol. 19, article id 100568Article, review/survey (Refereed)
    Abstract [en]

    Ongoing research on anomaly detection for the Internet of Things (IoT) is a rapidly expanding field. This growth necessitates an examination of application trends and current gaps. The vast majority of those publications are in areas such as network and infrastructure security, sensor monitoring, smart home, and smart city applications and are extending into even more sectors. Recent advancements in the field have increased the necessity to study the many IoT anomaly detection applications. This paper begins with a summary of the detection methods and applications, accompanied by a discussion of the categorization of IoT anomaly detection algorithms. We then discuss the current publications to identify distinct application domains, examining papers chosen based on our search criteria. The survey considers 64 papers among recent publications published between January 2019 and July 2021. In recent publications, we observed a shortage of IoT anomaly detection methodologies, for example, when dealing with the integration of systems with various sensors, data and concept drifts, and data augmentation where there is a shortage of Ground Truth data. Finally, we discuss the present such challenges and offer new perspectives where further research is required.

    Download full text (pdf)
    fulltext
  • 32.
    Chatterjee, Ayan
    et al.
    Karlstad University, Faculty of Health, Science and Technology (starting 2013), Department of Mathematics and Computer Science (from 2013).
    Ahmed, Bestoun S.
    Karlstad University, Faculty of Health, Science and Technology (starting 2013), Department of Mathematics and Computer Science (from 2013).
    Hallin, Erik
    Uddeholms AB.
    Engman, Anton
    Uddeholms AB.
    Testing of machine learning models with limited samples: an industrial vacuum pumping application2022In: ESEC/FSE ’22-Proceedings of the 30th ACM Joint Meeting European Software Engineering Conference and Symposium on the Foundations of Software Engineering / [ed] Abhik Roychoudhury, Cristian Cadar, and Miryung Kim, Association for Computing Machinery (ACM), 2022, p. 1280-1290Conference paper (Refereed)
    Abstract [en]

    There is often a scarcity of training data for machine learning (ML) classification and regression models in industrial production, especially for time-consuming or sparsely run manufacturing processes. Traditionally, a majority of the limited ground-truth data is used for training, while a handful of samples are left for testing. In that case, the number of test samples is inadequate to properly evaluate the robustness of the ML models under test (i.e., the system under test) for classification and regression. Furthermore, the output of these ML models may be inaccurate or even fail if the input data differ from the expected. This is the case for ML models used in the Electroslag Remelting (ESR) process in the refined steel industry to predict the pressure in a vacuum chamber. A vacuum pumping event that occurs once a workday generates a few hundred samples in a year of pumping for training and testing. In the absence of adequate training and test samples, this paper first presents a method to generate a fresh set of augmented samples based on vacuum pumping principles. Based on the generated augmented samples, three test scenarios and one test oracle are presented to assess the robustness of an ML model used for production on an industrial scale. Experiments are conducted with real industrial production data obtained from Uddeholms AB steel company. The evaluations indicate that Ensemble and Neural Network are the most robust when trained on augmented data using the proposed testing strategy. The evaluation also demonstrates the proposed method's effectiveness in checking and improving ML algorithms' robustness in such situations. The work improves software testing's state-of-the-art robustness testing in similar settings. Finally, the paper presents an MLOps implementation of the proposed approach for real-time ML model prediction and action on the edge node and automated continuous delivery of ML software from the cloud. 

    Download full text (pdf)
    fulltext
  • 33.
    Hasan, Imad H.
    et al.
    Salahaddin University-Erbil (SUE), Iraq.
    Ahmed, Bestoun S.
    Karlstad University, Faculty of Health, Science and Technology (starting 2013), Department of Mathematics and Computer Science (from 2013). Czech Technical University in Prague, Czech Republic.
    Potrus, Moyad Y.
    Salahaddin University-Erbil (SUE), Iraq.
    Zamli, Kamal Z.
    University Malaysia Pahang, Malaysia.
    Generation and Application of Constrained Interaction Test Suites Using Base Forbidden Tuples with a Mixed Neighborhood Tabu Search2020In: International journal of software engineering and knowledge engineering, ISSN 0218-1940, Vol. 30, no 3, p. 363-398Article in journal (Refereed)
    Abstract [en]

    To ensure the quality of current highly configurable software systems, intensive testing is needed to test all the configuration combinations and detect all the possible faults. This task becomes more challenging for most modern software systems when constraints are given for the configurations. Here, intensive testing is almost impossible, especially considering the additional computation required to resolve the constraints during the test generation process. In addition, this testing process is exhaustive and time-consuming. Combinatorial interaction strategies can systematically reduce the number of test cases to construct a minimal test suite without affecting the effectiveness of the tests. This paper presents a new efficient search-based strategy to generate constrained interaction test suites to cover all possible combinations. The paper also shows a new application of constrained interaction testing in software fault searches. The proposed strategy initially generates the set of all possible t-tuple combinations; then, it filters out the set by removing the forbidden t-tuples using the Base Forbidden Tuple (BFT) approach. The strategy also utilizes a mixed neighborhood tabu search (TS) to construct optimal or near-optimal constrained test suites. The efficiency of the proposed method is evaluated through a comparison against two well-known state-of-the-art tools. The evaluation consists of three sets of experiments for 35 standard benchmarks. Additionally, the effectiveness and quality of the results are assessed using a real-world case study. Experimental results show that the proposed strategy outperforms one of the competitive strategies, ACTS, for approximately 83% of the benchmarks and achieves similar results to CASA for 65% of the benchmarks when the interaction strength is 2. For an interaction strength of 3, the proposed method outperforms other competitive strategies for approximately 60% and 42% of the benchmarks. The proposed strategy can also generate constrained interaction test suites for an interaction strength of 4, which is not possible for many strategies. The real-world case study shows that the generated test suites can effectively detect injected faults using mutation testing.

     

  • 34.
    Kader, Md. Abdul
    et al.
    University Malaysia Pahang, MYS.
    Zamli, Kamal Z.
    University Malaysia Pahang, MYS.
    Ahmed, Bestoun S.
    Karlstad University, Faculty of Health, Science and Technology (starting 2013), Department of Mathematics and Computer Science (from 2013). ;Czech Technical University, CZE.
    A systematic review on emperor penguin optimizer2021In: Neural Computing & Applications, ISSN 0941-0643, E-ISSN 1433-3058, Vol. 33, no 23, p. 15933-15953Article in journal (Refereed)
    Abstract [en]

    Emperor Penguin Optimizer (EPO) is a recently developed metaheuristic algorithm to solve general optimization problems. The main strength of EPO is twofold. Firstly, EPO has low learning curve (i.e., based on the simple analogy of huddling behavior of emperor penguins in nature (i.e., surviving strategy during Antarctic winter). Secondly, EPO offers straightforward implementation. In the EPO, the emperor penguins represent the candidate solution, huddle denotes the search space that comprises a two-dimensional L-shape polygon plane, and randomly positioned of the emperor penguins represents the feasible solution. Among all the emperor penguins, the focus is to locate an effective mover representing the global optimal solution. To-date, EPO has slowly gaining considerable momentum owing to its successful adoption in many broad range of optimization problems, that is, from medical data classification, economic load dispatch problem, engineering design problems, face recognition, multilevel thresholding for color image segmentation, high-dimensional biomedical data analysis for microarray cancer classification, automatic feature selection, event recognition and summarization, smart grid system, and traffic management system to name a few. Reflecting on recent progress, this paper thoroughly presents an in-depth study related to the current EPO's adoption in the scientific literature. In addition to highlighting new potential areas for improvements (and omission), the finding of this study can serve as guidelines for researchers and practitioners to improve the current state-of-the-arts and state-of-practices on general adoption of EPO while highlighting its new emerging areas of applications.

  • 35.
    Klima, Matej
    et al.
    Czech Technical University in Prague, CZE.
    Bures, Miroslav
    Czech Technical University in Prague, CZE.
    Ahmed, Bestoun S.
    Karlstad University, Faculty of Health, Science and Technology (starting 2013), Department of Mathematics and Computer Science (from 2013). Czech Technical University in Prague, CZE.
    Bellekens, Xavier
    Lupovis.io, GBR.
    Atkinson, Robert
    University of Strathclyde, GBR.
    Tachtatzis, Christos
    University of Strathclyde, GBR.
    Herout, Pavel
    University of West Bohemia, CZE.
    Specialized path-based technique to test Internet of Things system functionality under limited network connectivity2023In: Internet of Things: Engineering Cyber Physical Human Systems, E-ISSN 2542-6605, Vol. 22, article id 100706Article in journal (Refereed)
    Abstract [en]

    Contemporary Internet-of-Things (IoT) systems are hindered by several reliability-related issues, especially, the dynamic behavior of IoT systems caused by limited and often unstable network connectivity. Several intuitive ad-hoc approaches can be employed to test this behavior; however, the effectiveness of these approaches in detecting defects and their overall testing costs remain questionable. Therefore, we present a new specialized path-based technique to test the processes of an IoT system in scenarios wherein parts of these processes are influenced by limited or disrupted network connectivity. The proposed technique can be scaled using two levels of test coverage criteria to determine the strengths of the test cases. For this purpose, we propose two algorithms for generating test cases to implement the technique: an ant colony optimization-based search and a graph-traversal-based test case composition. We compared the efficiency of the proposed approach with possible solutions obtained using a standard path-based testing approach based on prime paths computed by a set-covering algorithm. We consider the total number of test case steps as the main proxy for test effort in experiments employing 150 problem models. For the less intensive of the two used test-coverage criteria, EachBorderOnce, an ant colony optimization-based algorithm, produced test sets with the same averaged number of steps as the graph traversal-based test-case composition; however, this algorithm performed with averaged number of steps 10% lower than a prime paths-based algorithm. For the more intensive test coverage criterion, AllBorderCombinations, these differences favoring the ant colony optimization-based algorithm were 18% and 25%, respectively. For these two types of defined test coverage criteria, the ant colony optimization-based search, graph-traversal-based algorithm, and standard path-based testing approach based on prime paths achieved the best results for 93 and 78, 14 and 24, and 13 and 17 models for AllBorderCombinations and EachBorderOnce criterion, respectively. Therefore, to guarantee the best test set, all compared algorithms are combined in a portfolio strategy that yields the best results based on the potential of the produced test sets to detect simulated defects caused by limited network connectivity. Additionally, this portfolio strategy also yields test sets, implying the lowest test effort for experimental problem instances. 

    Download full text (pdf)
    fulltext
  • 36.
    Klima, Matej
    et al.
    Czech Technical University in Prague, Czech Republic.
    Bures, Miroslav
    Czech Technical University in Prague, Czech Republic.
    Frajtak, Karel
    Czech Technical University in Prague, Czech Republic.
    Rechtberger, Vaclav
    Czech Technical University in Prague Czech Republic.
    Trnka, Michal
    Czech Technical University in Prague, Czech Republic.
    Bellekens, Xavier
    Lupovis.io, Glasgow, U.K..
    Cerny, Tomas
    Baylor University, USA.
    Ahmed, Bestoun S.
    Karlstad University, Faculty of Health, Science and Technology (starting 2013), Department of Mathematics and Computer Science (from 2013).
    Selected Code-Quality Characteristics and Metrics for Internet of Things Systems2022In: IEEE Access, E-ISSN 2169-3536, Vol. 10, p. 46144-46161Article in journal (Refereed)
    Abstract [en]

    Software code is present on multiple levels within current Internet of Things (IoT) systems. The quality of this code impacts system reliability, safety, maintainability, and other quality aspects. In this paper, we provide a comprehensive overview of code quality-related metrics, specifically revised for the context of IoT systems. These metrics are divided into main code quality categories: Size, redundancy, complexity, coupling, unit test coverage and effectiveness, cohesion, code readability, security, and code heterogeneity. The metrics are then linked to selected general quality characteristics from the ISO/IEC 25010:2011 standard by their possible impact on the quality and reliability of an IoT system, the principal layer of the system, the code levels and the main phases of the project to which they are relevant. This analysis is followed by a discussion of code smells and their relation to the presented metrics. The overview presented in the paper is the result of a thorough analysis and discussion of the author’s team with the involvement of external subject-matter experts in which a defined decision algorithm was followed. The primary result of the paper is an overview of the metrics accompanied by applicability notes related to the quality characteristics, the system layer, the level of the code, and the phase of the IoT project.

    Download full text (pdf)
    fulltext
  • 37.
    Klima, Matej
    et al.
    Czech Technical University in Prague, CZE.
    Rechtberger, Vaclav
    Czech Technical University in Prague, CZE.
    Bures, Miroslav
    Czech Technical University in Prague, CZE.
    Bellekens, Xavier
    University of Strathclyde, GBR.
    Hindy, Hanan
    Abertay University, GBR.
    Ahmed, Bestoun S.
    Karlstad University, Faculty of Health, Science and Technology (starting 2013), Department of Mathematics and Computer Science (from 2013). Czech Technical University in Prague, CZE.
    Quality and Reliability Metrics for IoT Systems: A Consolidated View2020In: Science and Technologies for Smart Cities / [ed] S Paiva; SI Lopes ; R Zitouni ; N Gupta ; SF Lopes ; T Yonezawa, Springer, 2020, p. 635-650Conference paper (Refereed)
    Abstract [en]

    Quality and reliability metrics play an important role in the evaluation of the state of a system during the development and testing phases, and serve as tools to optimize the testing process or to define the exit or acceptance criteria of the system. This study provides a consolidated view on the available quality and reliability metrics applicable to Internet of Things (IoT) systems, as no comprehensive study has provided such a view specific to these systems. The quality and reliability metrics categorized and discussed in this paper are divided into three categories: metrics assessing the quality of an IoT system or service, metrics for assessing the effectiveness of the testing process, and metrics that can be universally applied in both cases. In the discussion, recommendations of proper usage of discussed metrics in a testing process are then given. 

  • 38.
    Ma, Yunpeng
    et al.
    Karlstad University, Faculty of Health, Science and Technology (starting 2013), Department of Mathematics and Computer Science (from 2013).
    Kassler, Andreas
    Karlstad University, Faculty of Health, Science and Technology (starting 2013), Department of Mathematics and Computer Science (from 2013).
    Ahmed, Bestoun S.
    Karlstad University, Faculty of Health, Science and Technology (starting 2013), Department of Mathematics and Computer Science (from 2013).
    Krakhmalev, Pavel
    Karlstad University, Faculty of Health, Science and Technology (starting 2013), Department of Engineering and Physics (from 2013).
    Thore, Andreas
    Smart Industrial Automation, RISE Research Institutes of Sweden, Västerås, Sweden.
    Toyser, Arash
    Viking Analytics AB, Göteborg, Sweden.
    Lindbäck, Hans
    Bharat Forge Kilsta AB, Karlskoga, Sweden.
    Using Deep Reinforcement Learning for Zero Defect Smart Forging2022In: Advances in Transdisciplinary Engineering / [ed] Ng A.H.C., Syberfeldt A., Hogberg D., Holm M., IOS Press, 2022, Vol. 21, p. 701-712Conference paper (Refereed)
    Abstract [en]

    Defects during production may lead to material waste, which is a significant challenge for many companies as it reduces revenue and negatively impacts sustainability and the environment. An essential reason for material waste is a low degree of automation, especially in industries that currently have a low degree of digitalization, such as steel forging. Those industries typically rely on heavy and old machinery such as large induction ovens that are mostly controlled manually or using well-known recipes created by experts. However, standard recipes may fail when unforeseen events happen, such as an unplanned stop in production, which may lead to overheating and thus material degradation during the forging process. In this paper, we develop a digital twin-based optimization strategy for the heating process for a forging line to automate the development of an optimal control policy that adjusts the power for the heating coils in an induction oven based on temperature data observed from pyrometers. We design a digital twin-based deep reinforcement learning (DTRL) framework and train two different deep reinforcement learning (DRL) models for the heating phase using a digital twin of the forging line. The twin is based on a simulator that contains a heating transfer and movement model, which is used as an environment for the DRL training. Our evaluation shows that both models significantly reduce the temperature unevenness and can help to automate the traditional heating process.

    Download full text (pdf)
    fulltext
  • 39.
    Ma, Yunpeng
    et al.
    Karlstad University, Faculty of Health, Science and Technology (starting 2013), Department of Mathematics and Computer Science (from 2013).
    Younis, Khalil
    Ahmed, Bestoun S.
    Karlstad University, Faculty of Health, Science and Technology (starting 2013), Department of Mathematics and Computer Science (from 2013).
    Kassler, Andreas
    Karlstad University, Faculty of Health, Science and Technology (starting 2013), Department of Mathematics and Computer Science (from 2013).
    Krakhmalev, Pavel
    Karlstad University, Faculty of Health, Science and Technology (starting 2013), Department of Engineering and Physics (from 2013).
    Thore, Andreas
    RISE Research Institutes of Sweden, Västerås, Sweden.
    Lindback, Hans
    Bharat Forge Kilsta AB, Sweden.
    Automated and Systematic Digital Twins Testing for Industrial Processes2023In: Proceedings - 2023 IEEE 16th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2023, Institute of Electrical and Electronics Engineers (IEEE), 2023, p. 149-158Conference paper (Refereed)
    Abstract [en]

    Digital twins (DT) of industrial processes have become increasingly important. They aim to digitally represent the physical world to help evaluate, optimize, and predict physical processes and behaviors. Therefore, DT is a vital tool to improve production automation through digitalization and becomes more sophisticated due to rapidly evolving simulation and modeling capabilities, integration of IoT sensors with DT, and high-capacity cloud/edge computing infrastructure. However, the fidelity and reliability of DT software are essential to represent the physical world. This paper shows an automated and systematic test architecture for DT that correlates DT states with real-time sensor data from a production line in the forging industry. Our evaluation shows that the architecture can significantly accelerate the automatic DT testing process and improve its reliability. A systematic online DT testing method can significantly detect the performance shift and continuously improve the DT’s fidelity. The snapshot creation methodology and testing agent architecture can be an inspiration and can be generally applicable to other industrial processes that use DT to generalize their automated testing. 

  • 40.
    Mahmoud, Thair Shakir
    et al.
    Edith Cowan University, Australia.
    Ahmed, Bestoun S.
    Karlstad University, Faculty of Health, Science and Technology (starting 2013), Department of Mathematics and Computer Science (from 2013). Czech Technical University, Czech Republic.
    Hassan, Mohammed Y.
    University of Technology, Iraq.
    The role of intelligent generation control algorithms in optimizing battery energy storage systems size in microgrids: A case study from Western Australia2019In: Energy Conversion and Management, ISSN 0196-8904, E-ISSN 1879-2227, Vol. 196, p. 1335-1352Article in journal (Refereed)
    Abstract [en]

    Battery energy storage systems can play a substantial role in maintaining low-cost operation in microgrids, and therefore finding their optimal size is a key element of microgrids’ planning and design. This paper explores the optimal sizing options for batteries in microgrids that include wind turbines, solar photovoltaics, synchronous machines and a grid connection supply under various types of retail tariff schemes. The optimal size of batteries is hypothesized to be significantly related to the intelligent control rules applied to dispatch the microgrid sources. This problem can be formulated as a mixed linear integer problem and can be solved using linear/non-linear solvers depending on the complexity of the generation control plan. The main objective of this work is to apply online intelligent adaptation mechanism to tune the economic generation control (dispatch) rules of the microgrid. This tuning objectives are maintaining secure operation, maximizing profitable utilization of batteries and managing their charging life-cycles. While sizing options exploration has been formulated as a linear programming based optimization problem, Fuzzy-Logic is proposed to control the charging/discharging time and quantity for batteries. For the sake of performance comparison, various optimization techniques, i.e., Particle Swarm Optimization, Genetic Algorithm and Flower Pollination Algorithm are applied to perform the economic dispatch calculation. As a case study, a commercial type load connected to the 22 kV distribution network in south Western Australia was used in the testing and validation if the results of the proposed sizing method. The operation condition data was obtained from Western Power the distribution and transmission company in south Western Australia, the Australian Bureau Of Meteorology (BOM) and the Australian Energy Market Operator (AEMO). The results showed that employing intelligent batteries in operation can reduce the annual generation cost of microgrids. However, the decision on selecting the size of batteries depends heavily on the amount of upfront investment cost. The simulation results showed that the intelligence added to batteries’ control could achieve 6.5%, 7.6% and 11.5% of the annual generation cost in the Islanded, Grid-connected with no-export and Grid-connected with export operating modes respectively. Also, intelligent batteries operation control was proven to minimize their payback time to 2.8, 2.7 and 2.7 years in the Islanded, Grid-connected with no-export and Grid-connected with export operating modes respectively.

  • 41.
    Min, Hong
    et al.
    Hoseo University, KOR.
    Kim, Taesik
    Hongik University, KOR.
    Heo, Junyoung
    Hansung University, KOR.
    Cerny, Tomas
    Baylor University, USA..
    Sankaran, Sriram
    Amrita Vishwa Vidyapeetham, IND.
    Ahmed, Bestoun S.
    Czech Technical University, CZE.
    Jung, Jinman
    Hannam University, KOR.
    Pattern Matching Based Sensor Identification Layer for an Android Platform2018In: Wireless Communications & Mobile Computing, ISSN 1530-8669, E-ISSN 1530-8677, article id 4734527Article in journal (Refereed)
    Abstract [en]

    As sensor-related technologies have been developed, smartphones obtain more information from internal and external sensors. This interaction accelerates the development of applications in the Internet of Things environment. Due to many attributes that may vary the quality of the IoT system, sensor manufacturers provide their own data format and application even if there is a well-defined standard, such as ISO/IEEE 11073 for personal health devices. In this paper, we propose a client-server-based sensor adaptation layer for an Android platform to improve interoperability among nonstandard sensors. Interoperability is an important quality aspect for the IoT that may have a strong impact on the system especially when the sensors are coming from different sources. Here, the server compares profiles that have clues to identify the sensor device with a data packet stream based on a modified Boyer-Moore-Horspool algorithm. Our matching model considers features of the sensor data packet. To verify the operability, we have implemented a prototype of this proposed system. The evaluation results show that the start and end pattern of the data packet are more efficient when the length of the data packet is longer.

  • 42.
    Nasser, A. B.
    et al.
    Universiti Malaysia Pahang, MYS.
    Zamli, K. Z.
    Universiti Malaysia Pahang, MYS.
    Ahmed, Bestoun S.
    Czech Technical University, CZE.
    Dynamic solution probability acceptance within the flower pollination algorithm for combinatorial t-way test suite generation2019In: Intelligent and Interactive Computing / [ed] V Piuri ; V Balas; S Borah ; S Syed Ahmad, Springer , 2019, p. 3-11Conference paper (Refereed)
    Abstract [en]

    In this paper, the enhanced Flower Pollination Algorithm (FPA) algorithm, called imFPA, has been proposed. Within imFPA, the static selection probability is replaced by the dynamic solution selection probability in order to enhance the intensification and diversification of the overall search process. Experimental adoptions on combinatorial t-way test suite generation problem (where t indicates the interaction strength) show that imFPA produces very competitive results as compared to existing strategies. © Springer Nature Singapore Pte Ltd. 2019.

  • 43.
    Nasser, Abdullah B.
    et al.
    University Malaysia Pahang, MYS.
    Zamli, Kamal Z.
    University Malaysia Pahang, MYS.
    Alsewari, AbdulRahman A.
    University Malaysia Pahang, MYS.
    Ahmed, Bestoun S.
    Czech Technical University, CZE.
    An elitist-flower pollination-based strategy for constructing sequence and sequence-less t-way test suite2018In: International Journal of Bio-Inspired Computation (IJBIC), ISSN 1758-0366, E-ISSN 1758-0374, Vol. 12, no 2, p. 115-127Article in journal (Refereed)
    Abstract [en]

    In line with the upcoming of a new field called search-based software engineering (SBSE), many newly developed t-way strategies adopting meta-heuristic algorithms can be seen in the literature for constructing interaction test suite (such as simulated annealing (SA), genetic algorithm (GA), ant colony optimisation algorithm (ACO), particle swarm optimisation (PSO), harmony search (HS) and cuckoo search (CS). Although useful, most of the aforementioned t-way strategies have assumed sequence-less interactions amongst input parameters. In the case of reactive system, such an assumption is invalid as some parameter operations (or events) occur in sequence and hence, creating a possibility of bugs triggered by the order (or sequence) of input parameters. If t-way strategies are to be adopted in such a system, there is also a need to support test data generation based on sequence of interactions. In line with such a need, this paper presents a unified strategy based on the new meta-heuristic algorithm, called the elitist flower pollination algorithm (eFPA), for sequence and sequence-less coverage. Experimental results demonstrate the proposed strategy gives sufficiently competitive results as compared with existing works.

  • 44.
    Nasser, Abdullah B.
    et al.
    University Malaysia Pahang, MYS.
    Zamli, Kamal Z.
    University Malaysia Pahang, MYS.
    Alsewari, AbdulRahman A.
    University Malaysia Pahang, MYS.
    Ahmed, Bestoun S.
    Czech Technical University, CZE.
    Hybrid flower pollination algorithm strategies for t-way test suite generation2018In: PLOS ONE, E-ISSN 1932-6203, Vol. 13, no 5, article id e0195187Article in journal (Refereed)
    Abstract [en]

    The application of meta-heuristic algorithms for t-way testing has recently become prevalent. Consequently, many useful meta-heuristic algorithms have been developed on the basis of the implementation of t-way strategies (where t indicates the interaction strength). Mixed results have been reported in the literature to highlight the fact that no single strategy appears to be superior compared with other configurations. The hybridization of two or more algorithms can enhance the overall search capabilities, that is, by compensating the limitation of one algorithm with the strength of others. Thus, hybrid variants of the flower pollination algorithm (FPA) are proposed in the current work. Four hybrid variants of FPA are considered by combining FPA with other algorithmic components. The experimental results demonstrate that FPA hybrids overcome the problems of slow convergence in the original FPA and offers statistically superior performance compared with existing t-way strategies in terms of test suite size.

  • 45.
    Rahal, Manal
    et al.
    Karlstad University, Faculty of Health, Science and Technology (starting 2013), Department of Mathematics and Computer Science (from 2013).
    Ahmed, Bestoun S.
    Karlstad University, Faculty of Health, Science and Technology (starting 2013), Department of Mathematics and Computer Science (from 2013). Czech Technical University in Prague, Czech Republic.
    Samuelsson, Jörgen
    Karlstad University, Faculty of Health, Science and Technology (starting 2013), Department of Engineering and Chemical Sciences (from 2013).
    Machine Learning Data Suitability and Performance Testing Using Fault Injection Testing Framework2024In: ECBS 2023: Engineering of Computer-Based Systems / [ed] Jan Kofroň, Tiziana Margaria, Cristina Seceleanu, Springer, 2024, Vol. 14390 LNCS, p. 42-59Conference paper (Refereed)
    Abstract [en]

    Creating resilient machine learning (ML) systems has become necessary to ensure production-ready ML systems that acquire user confidence seamlessly. The quality of the input data and the model highly influence the successful end-to-end testing in data-sensitive systems. However, the testing approaches of input data are not as systematic and are few compared to model testing. To address this gap, this paper presents the Fault Injection for Undesirable Learning in input Data (FIUL-Data) testing framework that tests the resilience of ML models to multiple intentionally-triggered data faults. Data mutators explore vulnerabilities of ML systems against the effects of different fault injections. The proposed framework is designed based on three main ideas: The mutators are not random; one data mutator is applied at an instance of time, and the selected ML models are optimized beforehand. This paper evaluates the FIUL-Data framework using data from analytical chemistry, comprising retention time measurements of anti-sense oligonucleotide. Empirical evaluation is carried out in a two-step process in which the responses of selected ML models to data mutation are analyzed individually and then compared with each other. The results show that the FIUL-Data framework allows the evaluation of the resilience of ML models. In most experiments cases, ML models show higher resilience at larger training datasets, where gradient boost performed better than support vector regression in smaller training sets. Overall, the mean squared error metric is useful in evaluating the resilience of models due to its higher sensitivity to data mutation. 

  • 46.
    Rechtberger, V.
    et al.
    Czech Technical University in Prague, CZE.
    Bures, M.
    Czech Technical University in Prague, CZE.
    Ahmed, Bestoun S.
    Karlstad University, Faculty of Health, Science and Technology (starting 2013), Department of Mathematics and Computer Science (from 2013).
    Alternative Effort-optimal Model-based Strategy for State Machine Testing of IoT Systems2020In: ACM International Conference Proceeding Series, Association for Computing Machinery (ACM), 2020, p. 141-145Conference paper (Refereed)
    Abstract [en]

    To effectively test parts of the Internet of Things (IoT) systems having a character of a state machine, Model-based Testing (MBT) approach can be taken. In MBT, a model of a system is created, and test cases generated automatically from the model, and a number of current strategies exist. In this paper we propose a novel alternative strategy, that concurrently allows to flexibly adjust the preferred length of the generated test cases, as well as to mark the states, in which the test case can start and end. Compared with an intuitive N-switch coverage-based strategy that aims at the same goals, our proposal generates a lower number of shorter test cases with less test step duplications.

  • 47.
    Rechtberger, Vaclav
    et al.
    Czech Technical University in Prague, Czech Republic.
    Bures, Miroslav
    Czech Technical University in Prague, Czech Republic.
    Ahmed, Bestoun S.
    Karlstad University, Faculty of Health, Science and Technology (starting 2013), Department of Mathematics and Computer Science (from 2013). Czech Technical University in Prague, Czech Republic.
    Overview of Test Coverage Criteria for Test Case Generation from Finite State Machines Modelled as Directed Graphs2022In: Proceedings - 2022 IEEE International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2022, Institute of Electrical and Electronics Engineers (IEEE), 2022, p. 207-214Conference paper (Refereed)
    Abstract [en]

    Test Coverage criteria are an essential concept for test engineers when generating the test cases from a System Under Test model. They are routinely used in test case generation for user interfaces, middleware, and back-end system parts for software, electronics, or Internet of Things (IoT) systems. Test Coverage criteria define the number of actions or combinations by which a system is tested, informally determining a potential "strength"of a test set. As no previous study summarized all commonly used test coverage criteria for Finite State Machines and comprehensively discussed them regarding their subsumption, equivalence, or non-comparability, this paper provides this overview. In this study, 14 most common test coverage criteria and seven of their synonyms for Finite State Machines defined via a directed graph are summarized and compared. The results give researchers and industry testing engineers a helpful overview when setting a software-based or IoT system test strategy.

  • 48.
    Rechtberger, Vaclav
    et al.
    Czech Technical University in Prague, Czech Republic.
    Bures, Miroslav
    Czech Technical University in Prague, Czech Republic.
    Ahmed, Bestoun S.
    Karlstad University, Faculty of Health, Science and Technology (starting 2013), Department of Mathematics and Computer Science (from 2013). Czech Technical University in Prague, Czech Republic.
    Belkhier, Youcef
    Czech Technical University in Prague, Czech Republic.
    Nema, Jiri
    University of Defence, Czech Republic.
    Schvach, Hynek
    University of Defence, Czech Republic.
    Prioritized Variable-length Test Cases Generation for Finite State Machines2022In: Proceedings - 2022 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW 2022), Institute of Electrical and Electronics Engineers (IEEE), 2022, p. 11-20Conference paper (Refereed)
    Abstract [en]

    Model-based Testing (MBT) is an effective approach for testing when parts of a system-under-test have the characteristics of a finite state machine (FSM). Despite various strategies in the literature on this topic, little work exists to handle special testing situations. More specifically, when concurrently: (1) the test paths can start and end only in defined states of the FSM, (2) a prioritization mechanism that requires only defined states and transitions of the FSM to be visited by test cases is required, and (3) the test paths must be in a given length range, not necessarily of explicit uniform length. This paper presents a test generation strategy that satisfies all these requirements. A concurrent combination of these requirements is highly practical for real industrial testing. Six variants of possible algorithms to implement this strategy are described. Using a mixture of 180 problem instances from real automotive and defense projects and artificially generated FSMs, all variants are compared with a baseline strategy based on an established N-switch coverage concept modification. Various properties of the generated test paths and their potential to activate fictional defects defined in FSMs are evaluated. The presented strategy outperforms the baseline in most problem configurations. Out of the six analyzed variants, three give the best results even though a universal best performer is hard to identify. Depending on the application of the FSM, the strategy and evaluation presented in this paper are applicable both in testing functional and non-functional software requirements. 

  • 49.
    Rechtberger, Vaclav
    et al.
    Czech Technical University in Prague, CZE.
    Bures, Miroslav
    Czech Technical University in Prague, CZE.
    Ahmed, Bestoun S.
    Karlstad University, Faculty of Health, Science and Technology (starting 2013), Department of Mathematics and Computer Science (from 2013). Czech Technical University, CZE.
    Schvach, Hynek
    University of Defence, CZE.
    Novel Strategy Generating Variable-Length State Machine Test Paths2022In: International journal of software engineering and knowledge engineering, ISSN 0218-1940, Vol. 32, no 08, p. 1247-1278Article in journal (Refereed)
    Abstract [en]

    Finite State Machine is a popular modeling notation for various systems, especially software and electronic. Test paths (TPs) can be automatically generated from the system model to test such systems using a suitable algorithm. This paper presents a strategy that generates TPs and allows to start and end TPs only in defined states of the finite state machine. The strategy also simultaneously supports generating TPs only of length in a given range. For this purpose, alternative system models, test coverage criteria, and a set of algorithms are developed. The strategy is compared with the best alternative based on the reduction of the test set generated by the established N-switch coverage approach on a mix of 171 industrial and artificially generated problem instances. The proposed strategy outperforms the compared variant in a smaller number of TP steps. The extent varies with the used test coverage criterion and preferred TP length range from none to two and half fold difference. Moreover, the proposed technique detected up to 30% more simple artificial defects inserted into experimental SUT models per one test step than the compared alternative technique. The proposed strategy is well applicable in situations where a possible TP starts and ends in a state machine needs to be reflected and, concurrently, the length of the TPs has to be in a defined range.

  • 50.
    Richter, Jan
    et al.
    Czech Technical University, CZE.
    Ahmed, Bestoun S.
    Karlstad University, Faculty of Health, Science and Technology (starting 2013), Department of Mathematics and Computer Science (from 2013).
    Bures, Miroslav
    Karlstad University.
    Junior, Cleber R. Rosa
    Red Hat Inc, USA.
    Avocado: Open-Source Flexible ConstrainedInteraction Testing for Practical Application2020In: IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW), IEEE Computer Society, 2020, p. 185-190Conference paper (Refereed)
    Abstract [en]

    This paper presents the outcome of a research collaboration between academia and industry to implement and utilize the capabilities of constrained interaction testing for an open-source tool for industrial-scale application. The project helps promote flexibility in generating constrained interaction test suites, executing them, and setting up a test oracle to report them–all within the same tool called Avocado. Avocado employs a constraint solver with computational algorithms to generate constrained interaction test suites. The environment of the application under test can be set up to execute the generated test suite with minimum effort. A test oracle can be set up by the tool to report the status and the results of the executed test cases. Avocado represents a comprehensive and flexible solution for conducting combinatorial interaction testing (CIT) and constrained CIT on an industrial application. In this paper, we present the structure of the tool and our method of implementing the algorithms in detail.

    Download full text (pdf)
    fulltext
12 1 - 50 of 59
CiteExportLink to result list
Permanent link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • apa.csl
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf