A Large-Scale Study on the Usage of Testing Patterns that Address Maintainability Attributes

TitleA Large-Scale Study on the Usage of Testing Patterns that Address Maintainability Attributes
Publication TypeConference Proceedings
Year of Publication2017
AuthorsGonzalez, Danielle, Santos Joanna C. S., Popovich Andrew, Mirakhorli Mehdi, and Nagappan Mei
Secondary Title2017 IEEE/ACM 14th International Conference on Mining Software Repositories (MSR)
Date Published05/2017
Keywordsmaintenance, mining software repositories, msr, Unit Test Frameworks, Unit Test Patterns, Unit Testing

Test case maintainability is an important concern,
especially in open source and distributed development environments
where projects typically have high contributor turnover
with varying backgrounds and experience, and where code
ownership changes often. Similar to design patterns, patterns
for unit testing promote maintainability quality attributes such
as ease of diagnoses, modifiability, and comprehension. In this
paper, we report the results of a large-scale study on the usage
of four xUnit testing patterns which can be used to satisfy these
maintainability attributes. This is a first-of-its-kind study which
developed automated techniques to investigate these issues across
82,447 open source projects, and the findings provide more insight
into testing practices in open source projects. Our results indicate
that only 17% of projects had test cases, and from the 251
testing frameworks we studied, 93 of them were being used.
We found 24% of projects with test files implemented patterns
that could help with maintainability, while the remaining did
not use these patterns. Multiple qualitative analyses indicate that
usage of patterns was an ad-hoc decision by individual developers,
rather than motivated by the characteristics of the project, and
that developers sometimes used alternative techniques to address
maintainability concerns.


"we conducted a large-scale empirical study to measure
the application of software testing in the open source community"

Our novel approach
includes a data set of 82,447 open source projects written in
48 languages

The data used in this study include 82,447 open source
projects, 251 unit testing frameworks, and 4 unit testing

data url: https://goo.gl/Mc7tHk