Towards a Unified Definition of Open Source Quality

Submitted by msquire on Thu, 2011-12-01 09:40

Title	Towards a Unified Definition of Open Source Quality
Publication Type	Conference Proceedings
Year of Publication	2011
Authors	Ruiz, C, Robinson, W
Secondary Title	Open Source Systems: Grounding Research (OSS 2011)
Pagination	17-33
Date Published	10/2011
Publisher	Springer
Keywords	literature review, measurement, open source, quality, Software
Abstract	Software quality needs to be specified and evaluated in order to determine the success of a development project, but this is a challenge with Free/Libre Open Source Software (FLOSS) because of its permanently emergent state. This has not deterred the growth of the assumption that FLOSS is higher quality than traditionally developed software, despite of mixed research results. With this literature review, we found the reason for these mixed results is that that quality is being defined, measured, and evaluated differently. We report the most popular definitions, such as software structure measures, process measures, such as defect fixing, and maturity assessment models. The way researchers have built their samples has also contributed to the mixed results with different project properties being considered and ignored. Because FLOSS projects are evolving, their quality is too, and it must be measured using metrics that take into account its community’s commitment to quality rather than just its software structure. Challenges exist in defining what constitutes a defect or bug, and the role of modularity in affecting FLOSS quality.
Notes	"In order to answer the research question, how is quality defined in the FLOSS literature, we performed a literature review." "we searched Google Scholar for journal articles and conference papers containing the terms “open source” and “quality”" "This process left us with 24 papers, to which we then added 16 from the quality and defect-fixing categories in [34] that met the above stated criteria. This left us with 40 papers that defined quality and performed some form of empirical validation of that definition." "there is little consensus in the FLOSS literature when it comes to defining quality." defect resolution versus modularity: "Defect resolution rates (amount of defects resolved, speed of resolution) are the best way to measure a community’s commitment to quality, because they recognize that FLOSS is not a static product, but ever evolving. These rates should be calculated per release, and not cumulatively, because the cycle of FLOSS evolution is the release. Researchers should be careful to only include defects and not new feature requests, duplicates, or poorly reported bugs into their calculations. Modularity is being touted as the main driver of FLOSS quality success, but it needs to be further defined and studied in order to understand how it works."
Full Text