This document presents an experimental evaluation carried out during the summer of 2003 in an academic environment. The goal of the experiment was to compare how different methods of documenting semantic information affect software reuse. More specifically, we wanted to measure if there were any differences between the methods with regard to the time needed to implement changes to existing software. Four methods of documentation were used; executable contracts, non-executable contracts, Javadoc-style documentation and sequence diagrams. The experimental setup used was a factorial design with documentation method as the treatment, and with test persons and assignments as blocking variables. The results indicate that executable contracts demanded more time than the other three methods and that sequence diagrams and Javadoc demanded the least time. ANOVA-tests on the data indicates that there were statistically significant differences between the methods
This paper deals with semantic-based programming and presents two experiments, one executed and one planned where the aim is to see what benefits a semantic-based program development method would bring. The first experiment was designed to compare a contract-based programming method with a standard programming method based on exceptions. The purpose was to evaluate if using contracts would shorten development time, if contracts would improve work satisfaction and if contracts would increase the quality of the resulting software program. The experiment was performed in a project work course for computer science students who were to solve an assignment in groups of four. Half of the groups used the contract-based method and the other half used the exception-based method. The results indicate that there is a gain in the time spent on implementation of the assignment if the contract-based method is used, but no significant difference in total time consumption was found. The results also indicate that the work satisfaction was slightly higher in the groups using the contract-based method. The resulting program was analyzed for a range of quality metrics, but no significant differences were found. The main problems of this experiment were too few data points, diversity in competence and ethical considerations. These problems all contribute to the statistically weak results. The last part of this paper presents a planned experiment where the problems in the first experiment hopefully will be remedied. This part is the main new contribution of the paper and is thought of as a starting point for discussions and not as a finished product or paper. The first part of the paper has been published before in a longer version, but is included for background and reference
Handling semantic aspects of a software system is one important issue in software development along with developing code and managing the organization and process. Code related issues has been thoroughly researched throughout the years as has process/organization related issues in more recent years. The semantic level inbetween has also had its share of contributions, but to a lesser extent than the two other levels. The paper presents some insight into how semantic issues are handled in both academia and industry. It presents a taxonomy based on literature studies of research papers for the academic part, and a case study that gives some insight into the work on semantic issues in industry
This article discusses the question if Scrum and XP is well suited for development of Computational Science and Engineering (CSE) software. The reason for choosing Scrum and XP is based on two facts. Firstly, CSE developers are not used to formal processes for software development and Scrum and XP are informal in nature, hence easier to adopt. Secondly CSE projects are often once-off and aimed at producing scientific results rather than commercial software, so the overhead of a process or methodology needs to be kept at a minimum. Scrum and XP are focused on producing software, thus keeping everything else, i.e. the overhead to a minimum, a fact that benefits CSE development. In this article, the characteristics of Scrum and XP in particular and agile development in general are evaluated against the needs of CSE developers. The results show that almost all the key points in both methodologies are well suited for CSE development and that agile development in general is a good match with CSE
This thesis presents research on how to raise the quality of software by helping developers focus on semantic aspects during the development process. Our research has so far been concentrated on software design, impolementation, testing and maintenance, but we believe that the ideas are applicable to other phases as well. The need for methods that better support software developers when working with semantic aspects was initially found in some of our industrial surveys. In these, we tried to evaluate the documentation for real software artefacts and found that semantic aspects were not handled as well as other aspects. To remedy this, we have developed an industrially applicable method that helps the developer focus on issues related to semantics. The method has been tested in a controlled experiment in a university course to evaluate its effects on the development process and the resulting product. The initial experiences of using the method were positive and the results indicate that it is possible to introduce a focus on semantics without any large overhead and still profit from higher quality software. In summary, we believe that improving the semantic aspects will improve the quality of the development process and of the resulting software
This paper reports our initial findings regarding the state of testing of software in the Swedish public health information system. At present, the system is only available through a black-box interface, i.e. through the GUI. This and other issues related to politics, management and organization indicate that much work is needed in order for the software to have the quality level expected by a safety-critical system. The proposed solution by the public health organization for raising the quality is to use an independent test database. Based on our initial understanding of the problem, we argue that there might be other solutions that would perhaps be more cost-effective and have a stronger impact on the quality of the system. Our main contribution lies in the data analysis, where we have collected the problems and suggested alternative cost-saving solutions.
In empirical software engineering research there is an increased use of questionnaires and surveys to collect information from practitioners. Typically, such data is then analyzed based on overall, descriptive statistics. Even though this can capture the general trends there is a risk that the opinions of different (minority) sub-groups are lost. Here we propose the use of clustering to segment the respondents so that a more detailed analysis can be achieved. Our findings suggest that it can give a better insight about the survey population and the participants' opinions. This partitioning approach can show more precisely the extent of opinion differences between different groups. This approach also gives an opportunity for the minorities to be heard. Through the process significant new findings may also be obtained. In our example study regarding the state of testing and requirement activities in industry, we found several significant groups that showed significant opinion differences from the overall conclusion.
This paper presents an experimental evaluation carried out in an academic environment. The goal of the experiment was to compare how different methods of documenting semantic information affect software reuse. More specifically, the goal was to measure if there were any differences between the methods with regard to the time needed to implement changes to existing software. Four methods of documentation were used; executable contracts, non-executable contracts, Javadoc-style documentation and sequence diagrams. The results indicate that executable contracts demanded more time than the other three methods and that sequence diagrams and Javadoc demanded the least time
This paper presents an experiment designed to compare a contract-based programming method with a standard programming method based on exceptions. The purpose was to evaluate if using contracts would shorten development time, if contracts would improve work satisfaction and if contracts would increase the quality of the resulting software program. The experiment was performed in a project work course for students in the computer science program at Karlstad University. The students were to solve an assignment in groups of four within a period of ten weeks. Half of the groups used the contract-based method and the other half used the exception-based method. We gathered data on time consumption and work satisfaction on daily forms for statistical analysis. The results indicate that there is a gain in the time spent on implementation of the assignment if the contract-based method is used, but no significant difference in total time consumption was found. The results also indicate that the work satisfaction was slightly higher in the groups using the contract-based method. The resulting program was analyzed for a range of quality metrics, but no significant differences were found
This paper presents a case study performed in an industrial project where we introduced and evaluated a method for supporting contract-based program development. The project constructed a large software system and involved around 30 people for 8 months. This paper reports on the designers' and programmers' attitudes towards programming by contract and how they perceived the effect the method had on their work. The results show that the designers and programmers were generally positive to contract-based program development and believed that the quality of their products would improve as an effect of the method
Contract-based Development and Test-Driven Development are two techniques for handling and avoiding software errors. They are generally not combined and we have studied the processes associated with the techniques tosee how this combination could be done. We have found possible ways to move to and from both processes and some benefits for both ways. Moving from contracts to test cases is rather straight forward due to the completeness of the contracts, whereas moving from test cases to contracts demand some more work since the test cases are only examples of usage.
The concept of software testability has been researched in several different dimensions, however the relation of this important concept with other quality attributes is a grey area where existing evidence is scattered. The objective of this study is to present a state-of-the-art with respect to issues of importance concerning software testability and an important quality attribute: software robustness. The objective is achieved by conducting a systematic literature review (SLR) on the topic. Our results show that a variety of testability issues are in focus with observability and controllability issues being most researched. Fault tolerance, exception handling and handling external influence are prominent robustness issues in focus.
In most of the research on software testability, functionalcorrectness of the software has been the focus while the ev-idence regarding testability and non-functional propertiessuch as performance is sporadic. The objective of this studyis to present the current state-of-the-art related to issuesof importance, types and domains of software under test,types of research, contribution types and design evaluationmethods concerning testability and software performance.We nd that observability, controllability and testing e ortare the main testability issues while timeliness and responsetime (i.e., time constraints) are the main performance issuesin focus. The primary studies in the area use diverse types ofsoftware under test within di erent domains, with real-timesystems as being a dominant domain. The researchers haveproposed many di erent methods in the area, however thesemethods lack implementation in practice as suggested byour gures for research type, contribution type and designevaluation methods.
Software architecture degradation is a phenomenon that frequently occurs during software evolution. Source code anomalies are one of the several aspects that potentially contribute to software architecture degradation. Many techniques for automating the detection of such anomalies are based on source code metrics. It is, however, unclear how accurate these techniques are in identifying the architecturally relevant anomalies in a system. The objective of this paper is to shed light on the extent to which source code metrics on their own can be used to characterize classes contributing to software architecture degradation. We performed a multi-case study on three open-source systems for each of which we gathered the intended architecture and data for 49 different source code metrics taken from seven different code quality tools. This data was analyzed to explore the links between architectural inconsistencies, as detected by applying reflexion modeling, and metric values indicating potential design problems at the implementation level. The results show that there does not seem to be a direct correlation between metrics and architectural inconsistencies. For many metrics, however, classes more problematic as indicated by their metric value seem significantly more likely to contribute to inconsistencies than less problematic classes. In particular, the fan-in, a classes’ public API, and method counts seem to be suitable indicators. The fan-in metric seems to be a particularly interesting indicator, as class size does not seem to have a confounding effect on this metric. This finding may be useful for focusing code restructuring efforts on architecturally relevant metrics in case the intended architecture is not explicitly specified and to further improve architecture recovery and consistency checking tool support.
Context: Several studies suggest that there is a relation between code smells and architecture degradation. They claim that classes, which have degraded architecture-wise, can be detected on the basis of code smells, at least if these are manually identiÿed in the source code.
Objective: To evaluate the suitability of contemporary code smell detection tools by combining different smell categories for ÿnding classes that show symptoms of architecture degradation.
Method: A case study is performed in which architectural in-consistencies in an open source system are detected via reflexion modeling and code smell metrics are collected through several tools. Using data mining techniques, we investigate if it is possible to auto-matically and accurately classify classes connected to architectural inconsistencies based on the gathered code smell data.
Results: Results suggest that existing code smell detection techniques, as implemented in contemporary tools, are not sufficiently accurate for classifying whether a class contains architectural in-consistencies, even when combining categories of code smells.
Conclusion: It seems that current automated code smell detection techniques require ÿne-tuning for a speciÿc system if they are to be used for ÿnding classes with architectural inconsistencies. More research on architecture violation causes is needed to build more accurate detection techniques that work out-of-the-box.
Due to the distributed nature of the growing field of Component Based Software Engineering (CBSE) and the increasing complexity of modern systems, the semantic aspects of software components must be well specified. In this literature survey, we have investigated how semantic integrity aspects are promoted and managed for components. The importance of semantic aspects and of preserving the semantic integrity is well known and well understood in academia for traditional object-oriented (OO) programming. Although component based programming has many similarities with OO programming, there are certain differences which may cause the semantic integrity to be managed differently. In component-based development, when components are downloaded to be used in systems that may or may not behave well, the responsibilities for the functionality and robustness are not as clear as in traditional programming
When designing a software module or system, a systems engineer must consider and differentiate between how the system responds to external and internal errors. External errors cannot be eliminated and must be tolerated by the system, while the number of internal errors should be minimized and the faults they result in should be detected and removed. This paper presents a development strategy based on design contracts and a case study of an industrial project in which the strategy was successfully applied. The goal of the strategy is to minimize the number of internal errors during the development of a software system while accommodating external errors. A distinction is made between weak and strong contracts. These two types of contracts are applicable to external and internal errors respectively. According to the strategy, strong contracts should be applied initially to promote the correctness of the system. Before release, the contracts governing external interfaces should be weakened and error management of external errors enabled. This transformation of a strong contract to a weak one is harmless to client modules
Migrating a family of software systems from ad-hoc development approaches such as `clone-and-own' towards software product lines (SPL) is a challenging task. Software architecture recovery techniques can play a crucial role in such a migration. However, it is to date still unclear how these techniques, which have been mostly developed for single system architecture recovery in mind, can be utilized in an SPL context most effectively. In this paper, we present a mapping study examining 35 research articles with the purpose of discussing the current state of the art in applying software architecture recovery techniques for SPL and identifying potential research gaps in this area. The results provide evidence that currently used approaches do not seem to consider the potential architectural degradation that might exist in the family of systems to be migrated. Moreover, it is hard to generalize across empirical studies as currently it seems difficult to compare and benchmark the approaches applied for software product line architecture (SPLA) extraction/reconstruction.