Measuring the performance of open source development communities: The QualOSS approach

Free and Open Source Software (F/OSS) has an increasing importance for the software industry. Similar to traditional (closed) software acquisition, F/OSS acquisition requires an assessment of whether its quality is sufficient for the intended purpose, and of whether the chances of being maintained and supported in the future, as well as of keeping certain quality standards over time, are sufficiently high. The first one of these aspects is strictly product related, and can be assessed using techniques that are not specific to F/OSS. The last two aspects, however, are related to the community developing the software, and require novel approaches to be evaluated properly.
In this paper, we present an approach toward a comprehensive measurement framework for F/OSS projects, developed in the EU project QualOSS. Although this approach takes into account product quality as well as process maturity and sustainability of the underlying F/OSS community, we concentrate here on its community-related aspects. After describing our quality model and assessment techniques in some detail, we close with a description of our ongoing evaluation effort and a discussion of lessons learned.