Change search
Link to record
Permanent link

Direct link
Publications (10 of 61) Show all publications
Maqbool, B. & Herold, S. (2024). Potential effectiveness and efficiency issues in usability evaluation within digital health: A systematic literature review. Journal of Systems and Software, 208, Article ID 111881.
Open this publication in new window or tab >>Potential effectiveness and efficiency issues in usability evaluation within digital health: A systematic literature review
2024 (English)In: Journal of Systems and Software, ISSN 0164-1212, E-ISSN 1873-1228, Vol. 208, article id 111881Article in journal (Refereed) Published
Abstract [en]

Context: Digital Health (DH) is widely considered essential for sustainable future healthcare systems. Software quality, particularly usability, is crucial for the success and adoption of most DH products. However, concerns about the effectiveness and efficiency of usability evaluation of DH products have been raised. Objective: This article aims to analyse the prevalence and application contexts of usability evaluation methods in DH and to highlight potential issues related to their effectiveness and efficiency. Method: A systematic literature review of usability evaluation studies, published by (academic) practitioners between 2016 and April 2023, was conducted. 610 primary articles were identified and analysed, utilising five major scientific databases. Results: Our findings show a preference for inquiry (85%) and testing (63%) methods, with inspection used less frequently (17%). The published studies employed methods like questionnaires (75%); notably the SUS (49%), semi-structured interviews (25%), and heuristic evaluations (73%), with percentages based on their group. Data collection mainly involved the use of participant feedback (45%), audio/video recordings (44%), and system logs (20%), with both qualitative and quantitative data analyses prevalent in studies. However, several usability characteristics such as accessibility, memorability, and operability were found to be largely overlooked, and automation tools or platforms were not widely used. Among the systems evaluated were mHealth applications (70%), telehealth platforms (36%), health information technology (HIT) solutions (29%), personalized medicine (Per. Med.) (17%), wearable devices (12%), and digital therapeutics (DTx) interventions (6%), with the participation of general users, patients, healthcare providers, and informal caregivers varying based on the health condition studied. Furthermore, insights and experiences gathered from 24 articles underscored the importance of a mixed-method approach in usability evaluations, the limitations of traditional methods, the necessity for sector-specific customisation, and the potential benefits of remote usability studies. Moreover, while eye-tracking emerged as a promising evaluation technique, careful execution and interpretation are crucial to avoid data misinterpretation. Conclusion: The study’s findings showed that employing a combination of inquiry and testing-based methods is prevalent for evaluating DH platforms. Despite an array of DH systems, method distribution remained consistent across platforms and targeted user groups. The study also underlines the importance of involving target user groups in the process. Potentially affected cognitive abilities of participants and potential user groups of interest have to be taken into account when choosing evaluation methods, and methods might therefore need to be tailored. Complementary inspection methods might be particularly useful when recruiting representative participants is difficult. Several potential paths for future research are outlined, such as exploring novel technologies like artificial intelligence, for improved automation tool support in the usability evaluation process. 

Place, publisher, year, edition, pages
Elsevier, 2024
Keywords
Computer software selection and evaluation, Efficiency, Heuristic methods, mHealth, Quality control, Usability engineering, Digital healthcare, Effectiveness and efficiencies, Ehealth, Healthcare systems, Software Quality, Systematic literature review, Usability evaluation, Usability evaluation methods, User groups, Eye tracking
National Category
Human Computer Interaction Software Engineering
Research subject
Computer Science
Identifiers
urn:nbn:se:kau:diva-97573 (URN)10.1016/j.jss.2023.111881 (DOI)001111515100001 ()2-s2.0-85176240774 (Scopus ID)
Funder
Vinnova, 2018-03025Region Värmland, RUN/220266
Available from: 2023-11-29 Created: 2023-11-29 Last updated: 2024-02-27Bibliographically approved
Sinkala, Z. T. & Herold, S. (2023). An Integrated Approach to Package and Class Code-to-Architecture Mapping Using InMap. In: Proceedings - IEEE 20th International Conference on Software Architecture, ICSA 2023: . Paper presented at 20th IEEE International Conference on Software Architecture, ICSA 2023,L'Aquila, Italy, March 13-17, 2023. (pp. 164-174). Institute of Electrical and Electronics Engineers (IEEE)
Open this publication in new window or tab >>An Integrated Approach to Package and Class Code-to-Architecture Mapping Using InMap
2023 (English)In: Proceedings - IEEE 20th International Conference on Software Architecture, ICSA 2023, Institute of Electrical and Electronics Engineers (IEEE), 2023, p. 164-174Conference paper, Published paper (Refereed)
Abstract [en]

Reflexion Modelling is a successful method used in industry for Software Architectural Consistency Checking (SACC). However, it includes a mapping step that is manual and tedious, especially for large complex systems. Various studies have shown how to successfully automate the interactive mapping of the basic units of the software’s codebase, i.e. its classes, to its architecture modules. However, their inherent drawback is that the effort required by an architect to review the mapping recommendations produced, whether during the mapping occurs or at the end of mapping, can be considerable. Subsequent studies have attempted to reduce this effort by use of a hierarchical mapping approach. These studies have demonstrated a reduction in the review effort required by a software architect; however, the gain in effort reduction occurred at the price of a lower recall and precision than similar non-hierarchical mapping approaches. In this study, we present an integrated approach of automated code-to-architecture mapping that draws from hierarchical (package mapping) and non-hierarchical (class mapping) techniques to keep effort minimal for an architect with marginal loss in recall and precision. Using the harmonic mean of f1-scores and effort reduction, our results show that with our integrated approach, we could achieve 0.90 on average, compared to 0.87 for the other two methods. 

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE), 2023
Keywords
automated source code mapping, software architecture conformance, software architecture consistency, software maintenance
National Category
Software Engineering
Identifiers
urn:nbn:se:kau:diva-94906 (URN)10.1109/ICSA56044.2023.00023 (DOI)2-s2.0-85159157207 (Scopus ID)
Conference
20th IEEE International Conference on Software Architecture, ICSA 2023,L'Aquila, Italy, March 13-17, 2023.
Available from: 2023-05-29 Created: 2023-05-29 Last updated: 2023-05-29Bibliographically approved
Sinkala, Z. T. & Herold, S. (2023). Investigating the Effect of Partial and Real-Time Feedback in INMAP Code-to-Architecture Mapping. In: Proceedings of the 18th Conference on Computer Science and Intelligence Systems, FedCSIS 2023: . Paper presented at 18th Conference on Computer Science and Intelligence Systems, FedCSIS, Warsaw, Poland, September 20, 2023. (pp. 749-758). Institute of Electrical and Electronics Engineers (IEEE)
Open this publication in new window or tab >>Investigating the Effect of Partial and Real-Time Feedback in INMAP Code-to-Architecture Mapping
2023 (English)In: Proceedings of the 18th Conference on Computer Science and Intelligence Systems, FedCSIS 2023, Institute of Electrical and Electronics Engineers (IEEE), 2023, p. 749-758Conference paper, Published paper (Refereed)
Abstract [en]

InMap is an interactive and iterative information retrieval-based automated mapping algorithm that produces code-to-architecture mapping recommendations. In its original form, InMap requires an architect to provide feedback for each code-to-architecture mapping recommendation in a given set produced (complete feedback). However, architects may delay/defer deciding on some of the mapping recommendations provided. This leads us to ask, how would InMap perform if only a subset of the recommendations provided (partial feedback) or only a single recommendation (real-time feedback) is reviewed by the architect Through carefully designed mapping experiments, we show that an architect giving partial or real-time feedback does not harm the recall and precision of the recommendations produced by InMap. On the contrary, we observed from the results of the systems tested a net increase of 2-5% (depending on the approach). This shows that in addition to InMap’s original complete feedback approach, the two new approaches of collecting feedback presented in this paper, i.e. partial and real-time, create flexibility in how software architecture consistency checking tool developers may choose to collect mapping feedback and how architects may opt-to provide feedback, with no harm to the recall and precision of the results. 

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE), 2023
Keywords
Computer software maintenance, Conformal mapping, Information systems, Information use, Iterative methods, Automated mapping, Automated source code mapping, Feedback approach, Mapping algorithms, Partial feedback, Real-time feedback, Recall and precision, Software architecture conformance, Software architecture consistency, Source codes, Software architecture
National Category
Software Engineering
Research subject
Computer Science
Identifiers
urn:nbn:se:kau:diva-97897 (URN)10.15439/2023F5070 (DOI)2-s2.0-85179179532 (Scopus ID)
Conference
18th Conference on Computer Science and Intelligence Systems, FedCSIS, Warsaw, Poland, September 20, 2023.
Available from: 2024-01-03 Created: 2024-01-03 Last updated: 2024-01-03Bibliographically approved
Herold, S. & Sinkala, Z. T. (2023). Using Automatically Recommended Seed Mappings for Machine Learning-Based Code-to-Architecture Mappers. In: Proceedings of the 38th ACM/SIGAPP Symposium on Applied Computing: . Paper presented at 38th Annual ACM Symposium on Applied Computing, Tallinn, Estonia, March 27-31, 2023. (pp. 1432-1439). Association for Computing Machinery (ACM)
Open this publication in new window or tab >>Using Automatically Recommended Seed Mappings for Machine Learning-Based Code-to-Architecture Mappers
2023 (English)In: Proceedings of the 38th ACM/SIGAPP Symposium on Applied Computing, Association for Computing Machinery (ACM), 2023, p. 1432-1439Conference paper, Published paper (Refereed)
Abstract [en]

Software architecture consistency checking (SACC) is a popular method to detect architecture degradation. Most SACC techniques require software engineers to manually map a subset of entities of a system’s implementation onto elements of its intended software architecture. Manually creating such a "seed mapping"for complex systems is a time-consuming activity.The objective of this paper is to investigate if creating seed mappings semi-automatically based on mapping recommendations for training automatic, machine learning-based mappers can reduce the effort for this task.To this end, we applied InMap, a highly accurate, interactive code-to-architecture mapping approach, to create seed mappings for five open source system with known architectures and mappings. Three different machine learning-based mappers were trained with these seed mappings and analysed regarding their predictive performance. We then compared the manual effort involved in using the combination of InMap and the most accurate automatic mapper and the manual effort of mapping the systems solely with InMap.The results suggest that InMap, with a minor adaption, can be used to seed an accurate mapper based on Naive Bayes. A full mapping with only InMap though turns out to involve slightly less manual effort on average; this is, however, not consistent across all systems. These results give reason to assume that more advanced ways of combining automatic mappers with InMap may further reduce that effort. 

Place, publisher, year, edition, pages
Association for Computing Machinery (ACM), 2023
Keywords
Mapping, Open source software, Open systems, Software architecture, Automatic machines, Code-to-architecture mapping, Consistency checking, Highly accurate, Machine-learning, Open source system, Predictive performance, Software architecture consistency, Software architecture degradation, Systems implementation, Machine learning
National Category
Software Engineering Computer Sciences
Research subject
Computer Science
Identifiers
urn:nbn:se:kau:diva-96057 (URN)10.1145/3555776.3577628 (DOI)2-s2.0-85162854475 (Scopus ID)978-1-4503-9517-5 (ISBN)
Conference
38th Annual ACM Symposium on Applied Computing, Tallinn, Estonia, March 27-31, 2023.
Funder
Region Värmland
Available from: 2023-07-07 Created: 2023-07-07 Last updated: 2023-07-07Bibliographically approved
Sinkala, Z. T. & Herold, S. (2022). Hierarchical Code-to-Architecture Mapping. In: Patrizia Scandurra; Matthias Galster; Raffaela Mirandola; Danny Weyns (Ed.), Software Architecture: 15th European Conference, ECSA 2021 Tracks and Workshops; Växjö, Sweden, September 13–17, 2021, Revised Selected Papers. Paper presented at 15th European Conference on Software Architecture, ECSA 2021, 13-17 September, 2021. Växjö, Sweden. (pp. 86-104). Springer
Open this publication in new window or tab >>Hierarchical Code-to-Architecture Mapping
2022 (English)In: Software Architecture: 15th European Conference, ECSA 2021 Tracks and Workshops; Växjö, Sweden, September 13–17, 2021, Revised Selected Papers / [ed] Patrizia Scandurra; Matthias Galster; Raffaela Mirandola; Danny Weyns, Springer, 2022, p. 86-104Conference paper, Published paper (Refereed)
Abstract [en]

Automating the mapping of a system’s code to its architecture is important in improving the adoption of successful Software Architecture Consistency Checking (SACC) methods like Reflexion Modelling. InMap is an interactive and iterative code-to-architecture mapping recommendation approach that achieves a rather decent recall and precision of 0.97 and 0.82 respectively, using minimal architecture documentation to apply natural language techniques to a software’s codebase. Nevertheless, InMap like most other automated recommendations techniques maps to architectural modules, low-level source code units like classes. For large complex systems, this can still hinder adoption due to the review effort required by a software architect when accepting or rejecting the recommendations. In this paper, we present a hierarchical package mapping technique that provides recommendations for higher-level source code units, i.e. packages. It utilizes InMap’s information retrieval capabilities to recommend mappings between the software’s packages and its architectural modules. We show that using our proposed technique we are able to reduce the recommendation review effort required by an architect, by 95% on average, for the six systems tested, and still achieve a code coverage of 75%. 

Place, publisher, year, edition, pages
Springer, 2022
Series
Lecture Notes in Computer Science, ISSN 0302-9743, E-ISSN 1611-3349 ; 13365
Keywords
Computer programming languages, Computer software maintenance, Iterative methods, Software architecture, Automated source code mapping, Consistency checking, ITS architecture, Large complex systems, Natural language techniques, Recall and precision, Recommendation techniques, Software architecture conformance, Software architecture consistency, Source codes, Mapping
National Category
Computer and Information Sciences
Research subject
Computer Science
Identifiers
urn:nbn:se:kau:diva-91875 (URN)10.1007/978-3-031-15116-3_5 (DOI)000874750000005 ()2-s2.0-85136984508 (Scopus ID)978-3-031-15115-6 (ISBN)978-3-031-15116-3 (ISBN)
Conference
15th European Conference on Software Architecture, ECSA 2021, 13-17 September, 2021. Växjö, Sweden.
Available from: 2022-09-13 Created: 2022-09-13 Last updated: 2022-11-11Bibliographically approved
Florean, A., Jalal, L., Sinkala, Z. T. & Herold, S. (2021). A Comparison of Machine Learning-Based Text Classifiers for Mapping Source Code to Architectural Modules. In: Companion Proceedings of the 15th European Conference on Software Architecture: . Paper presented at 15th European Conference on Software Architecture. CEUR-WS, 2978
Open this publication in new window or tab >>A Comparison of Machine Learning-Based Text Classifiers for Mapping Source Code to Architectural Modules
2021 (English)In: Companion Proceedings of the 15th European Conference on Software Architecture, CEUR-WS , 2021, Vol. 2978Conference paper, Published paper (Refereed)
Abstract [en]

A mapping between a system's implementation and its software architecture is mandatory in many architecture consistency checking techniques. Creating such a mapping manually is a non-trivial task for most complex software systems. Machine learning-based text classification may be an highly effective tool for automating this task. How to make use of this tool most effectively has not been thoroughly investigated yet.

This article presents a comparative analysis of three classifiers applied to map the implementations of five open-source systems to their architectures. The performance of the classifiers is evaluated for different extraction and preprocessing settings as well as different training set sizes.

The results suggest that Logical Regression and Support Vector Machines both outperform Naive Bayes unless information about coarse-grained implementation structures cannot be exploited. Moreover, initial manual mappings of more than 15% of all source code files, or 10 files per module, do not seem to lead to a significantly better classification.

Place, publisher, year, edition, pages
CEUR-WS, 2021
Series
CEUR Workshop Proceedings, ISSN 1613-0073
Keywords
software architecture consistency, code-to-architecture mapping, text classification, machine learning
National Category
Software Engineering
Research subject
Computer Science
Identifiers
urn:nbn:se:kau:diva-86313 (URN)2-s2.0-85117782941 (Scopus ID)
Conference
15th European Conference on Software Architecture
Available from: 2021-10-26 Created: 2021-10-26 Last updated: 2022-05-12Bibliographically approved
Maqbool, B. & Herold, S. (2021). Challenges in Developing Software for the Swedish Healthcare Sector. In: Proceedings of the 14th International Joint Conference on Biomedical Engineering Systems and Technologies - Volume 5 HEALTHINF, Austria, February 11-13, 2021: . Paper presented at 14th International Conference on Health Informatics - HEALTHINF21 (pp. 175-187). Portugal: SciTePress, 5
Open this publication in new window or tab >>Challenges in Developing Software for the Swedish Healthcare Sector
2021 (English)In: Proceedings of the 14th International Joint Conference on Biomedical Engineering Systems and Technologies - Volume 5 HEALTHINF, Austria, February 11-13, 2021, Portugal: SciTePress, 2021, Vol. 5, p. 175-187Conference paper, Published paper (Refereed)
Abstract [en]

Context: High-quality software is essential to the progressing digitalisation of the Swedish healthcare sector. Developing software with the desired high quality is far from trivial due to the sophisticated requirements of the domain.

Problem: Studies on healthcare digitalisation challenges in Sweden and other countries, however, largely focus on the perceptions of healthcare professionals and patients and less on opinions of IT professionals.

Method: In this exploratory study, we conducted semi-structured interviews with nine IT professionals about observed challenges in developing software for the Swedish healthcare sector. A qualitative analysis was performed to identify common themes.

Results: We identified the prevalent challenges to be related to data integrity, privacy and security, rules and regulations, engineering usability, and software testing.

Conclusion: The results suggest that further research is required regarding agile methods, efficient requirement engineering, and testing in eHealth as well as in privacy and usability engineering. 

Place, publisher, year, edition, pages
Portugal: SciTePress, 2021
Keywords
eHealth, Software Development, Exploratory Study, Empirical Study, Interview Study.
National Category
Software Engineering Information Systems Computer Sciences
Research subject
Computer Science
Identifiers
urn:nbn:se:kau:diva-83370 (URN)10.5220/0010248901750187 (DOI)000664063000017 ()2-s2.0-85103840449 (Scopus ID)978-989-758-490-9 (ISBN)
Conference
14th International Conference on Health Informatics - HEALTHINF21
Note

This work was funded by Region Varmland via the DigitalWell Arena project (Dnr RV2018-678)

Available from: 2021-03-08 Created: 2021-03-08 Last updated: 2021-08-05Bibliographically approved
Sinkala, Z. T. & Herold, S. (2021). InMap: Automated Interactive Code-to-Architecture Mapping. In: Proceedings of the 36th Annual ACM Symposium on Applied Computing: . Paper presented at 36th ACM/SIGAPP Symposium on Applied Computing (SAC 2021) (pp. 1439-1442). Association for Computing Machinery (ACM)
Open this publication in new window or tab >>InMap: Automated Interactive Code-to-Architecture Mapping
2021 (English)In: Proceedings of the 36th Annual ACM Symposium on Applied Computing, Association for Computing Machinery (ACM), 2021, p. 1439-1442Conference paper, Poster (with or without abstract) (Refereed)
Abstract [en]

There exist techniques that attempt to automate the mapping step in Reflexion Modelling, a popular software architecture consistency checking method. However, most of these techniques require an initial set of pre-mapped entities in order to execute automated mapping. This study proposes an interactive technique that is able to automate mapping without the need for pre-mapped entities. Using some minimal architecture documentation, the technique utilises information retrieval concepts applied to the software’s codebase to recommend mappings between the software’s source code entities and its architectural modules. The technique achieves higher f1-scores than existing techniques that require initial pre-mapping before automation.

Place, publisher, year, edition, pages
Association for Computing Machinery (ACM), 2021
Series
SAC ’21
Keywords
source code mapping, reflexion modelling, software maintenance, software architecture consistency checking, information retrieval
National Category
Software Engineering
Research subject
Computer Science
Identifiers
urn:nbn:se:kau:diva-83887 (URN)10.1145/3412841.3442124 (DOI)000675593000017 ()2-s2.0-85105020065 (Scopus ID)
Conference
36th ACM/SIGAPP Symposium on Applied Computing (SAC 2021)
Available from: 2021-05-05 Created: 2021-05-05 Last updated: 2021-09-07Bibliographically approved
Sinkala, Z. T. & Herold, S. (2021). InMap: Automated Interactive Code-to-Architecture Mapping Recommendations. In: 2021 IEEE 18th International Conference on Software Architecture (ICSA): . Paper presented at 2021 IEEE 18th International Conference on Software Architecture (ICSA) 22-26 March 2021. IEEE
Open this publication in new window or tab >>InMap: Automated Interactive Code-to-Architecture Mapping Recommendations
2021 (English)In: 2021 IEEE 18th International Conference on Software Architecture (ICSA), IEEE, 2021Conference paper, Published paper (Refereed)
Abstract [en]

Reflexion Modelling is a popular method used in industry for Software Architectural Consistency Checking (SACC). However, it involves a mapping step that is manual and tedious. There exist techniques and tools that attempt to automate mapping, yet they are either limited in their approach or they require an initial set of manually pre-mapped entities. This study proposes a novel technique, InMap, that improves the mapping process in reflexion modelling by both providing versatility and eliminating the constraint of needing a set of manually pre-mapped entities in order to automate mapping. Using a software's architecture descriptions, InMap applies information retrieval concepts to the software's source code to interactively provide mapping recommendations to an architect. For the six systems InMap was evaluated on, the recommendations it provided achieved an average recall of 0.97, and an average precision of 0.82. InMap also achieved higher, f 1 -scores in comparison to existing techniques that require premapping. This provides a basis for improving industry tools that use reflexion modelling or similar SACC methods.

Place, publisher, year, edition, pages
IEEE, 2021
Keywords
automating architecture conformance, software architecture descriptions, source code mapping, reflexion modelling, software architecture consistency checking, software maintenance, information retrieval
National Category
Software Engineering
Research subject
Computer Science
Identifiers
urn:nbn:se:kau:diva-87460 (URN)10.1109/ICSA51549.2021.00024 (DOI)978-1-7281-6260-7 (ISBN)978-1-7281-6261-4 (ISBN)
Conference
2021 IEEE 18th International Conference on Software Architecture (ICSA) 22-26 March 2021
Available from: 2021-12-03 Created: 2021-12-03 Last updated: 2022-02-10Bibliographically approved
Herold, S., Buckley, J. & van der Werf, J. M. (2021). Preface of the 8th Workshop on Software Architecture Erosion and Architectural Consistency (SAEroCon). In: CEUR Workshop Proceedings: . Paper presented at 15th European Conference on Software Architecture - Companion, ECSA-C 2021, Online, September 13-17, 2021. CEUR-WS
Open this publication in new window or tab >>Preface of the 8th Workshop on Software Architecture Erosion and Architectural Consistency (SAEroCon)
2021 (English)In: CEUR Workshop Proceedings, CEUR-WS , 2021Conference paper (Refereed)
Abstract [en]

The more complex a software system is and the longer a software system evolves, the more difficult it is to avoid effects like software architecture erosion or architectural drifts. These effects can lead to systems where the realization of the system diverges from the intended architecture with resultant negative impacts on quality attributes associated with the intended architecture. As studies have shown, untreated divergence can lead to systems which are impossible to maintain in the long run. SAEroCon is dedicated to research related to these phenomena.

Place, publisher, year, edition, pages
CEUR-WS, 2021
Series
CEUR Workshop Proceedings, ISSN 1613-0073
Keywords
Erosion, Software architecture, Architecture erosion, Quality attributes, Software-systems, Computer software
National Category
Electrical Engineering, Electronic Engineering, Information Engineering
Research subject
Computer Science
Identifiers
urn:nbn:se:kau:diva-88835 (URN)2-s2.0-85117845724 (Scopus ID)
Conference
15th European Conference on Software Architecture - Companion, ECSA-C 2021, Online, September 13-17, 2021
Available from: 2022-02-28 Created: 2022-02-28 Last updated: 2022-10-24Bibliographically approved
Organisations
Identifiers
ORCID iD: ORCID iD iconorcid.org/0000-0002-3180-9182

Search in DiVA

Show all publications