Cliff Walls: An Analysis of Monolithic Commits Using Latent Dirichlet Allocation

TitleCliff Walls: An Analysis of Monolithic Commits Using Latent Dirichlet Allocation
Publication TypeConference Proceedings
Year of Publication2011
AuthorsPratt, Landon J., MacLean Alexander C., Knutson Charles D., and Ringger Eric K.
Secondary TitleOpen Source Systems: Grounding Research (OSS 2011)
Date Published10/2011
Keywordsartifacts, commit, cvs, LDA, lines of code, log files, scm, sloc, sourceforge, version control

Artifact-based research provides a mechanism whereby researchers may study the creation of software yet avoid many of the difficulties of direct observation and experimentation. However, there are still many challenges that can affect the quality of artifact-based studies, especially those studies examining software evolution. Large commits, which we refer to as “Cliff Walls,” are one significant threat to studies of software evolution because they do not appear to represent incremental development. We used Latent Dirichlet Allocation to extract topics from over 2 million commit log messages, taken from 10,000 SourceForge projects. The topics generated through this method were then analyzed to determine the causes of over 9,000 of the largest commits. We found that branch merges, code imports, and auto-generated documentation were significant causes of large commits. We also found that corrective maintenance tasks, such as bug fixes, did not play a significant role in the creation of large commits.


"Our data set consists of the version control logs of almost 10,000 projects from SourceForge, acquired in late 2006"