Open source software peer review practices: a case study of the apache server

TitleOpen source software peer review practices: a case study of the apache server
Publication TypeConference Paper
Year of Publication2008
AuthorsRigby, Peter C., German Daniel M., and Storey Margaret-Anne
Secondary TitleProceedings of the 30th International Conference on Software Engineering (ICSE 2008)
Place PublishedNew York, NY, USA
ISBN Number978-1-60558-079-1
Keywordsapache, cvs, email, inspection, mining software repositories (email), open source software, peer review, version control

Peer review is seen as an important quality assurance mechanism in both industrial development and the open source software (OSS) community. The techniques for performing inspections have been well studied in industry; in OSS development, peer reviews are less well understood. We examine the two peer review techniques used by the successful, mature Apache server project: review-then-commit and commit-then-review. Using archival records of email discussion and version control repositories, we construct a series of metrics that produces measures similar to those used in traditional inspection experiments. Specifically, we measure the frequency of review, the level of participation in reviews, the size of the artifact under review, the calendar time to perform a review, and the number of reviews that find defects. We provide a comparison of the two Apache review techniques as well as a comparison of Apache review to inspection in an industrial project. We conclude that Apache reviews can be described as (1) early, frequent reviews (2) of small, independent, complete contributions (3) conducted asynchronously by a potentially large, but actually small, group of self-selected experts (4) leading to an efficient and effective peer review technique.


p541-rigby.pdf306.98 KB