A preliminary examination of code review processes in open source projects

In this paper, we provide preliminary answers to the following questions regarding OSS peer review or inspection. What is the patch process and review process used by the projects? What types of review does a the project use? Why are patches rejected? What percentage of patches are rejected? Who performs the review? Are the top developers also the top reviewers? When are reviews performed? What is the frequency of review? How long do reviews take to perform? How does the patch size affect the review? How does merit-based trust among actors affect the review? Are more trusted individuals reviewed less often? How much feedback is provided in the review? What kinds of non-source code patches are reviewed? How does the kind of patch affect the review? What affect does reviewing have on other elements of the patch process? What is the relationship between reviewing and testing? The first two questions are answered in a qualitative manner for GCC, Linux, Mozilla, and Apache. The remaining questions are answered for the Apache project. The most striking similarities among projects is there use of a pre-commit review and requests for small, complete, independent patches. The Apache project also uses a post-commit review of trusted members. Reviews in the Apache project occur very frequently and usually have a review interval of hours. A small core group of reviewers conduct over 80% of reviews for Apache; however, the number of and actual individuals fluctuates over the 9 years of data we examine.