Evaluating the performance of open source software projects using data envelopment analysis

TitleEvaluating the performance of open source software projects using data envelopment analysis
Publication TypeJournal Article
Year of Publication2008
AuthorsWray, B, Mathieu, R, Teets, J
Secondary TitleInformation Management & Computer Security
Pagination449 - 462
Date Published2008
ISSN Number0968-5227
Keywordsdea, efficiency, Project performance, sourceforge

Purpose – The purpose of this paper is to develop and test a model of the relative performance of open source software (OSS) projects.

Design/methodology/approach – This paper evaluates the relative performance of OSS projects by evaluating multiple project inputs and multiple project outputs by using a data envelopment analysis (DEA) model. The DEA model produces an efficiency score for each project based on project inputs and outputs. The method of producing an efficiency score is based on the convex envelopment technology structure. The efficiency measure quantifies a “distance” to an efficient frontier.

Findings – The DEA model produced an index of corresponding intensities linking an inefficient project to its benchmark efficient project(s). The inefficiency measures produced an ordering of inefficient projects. Eight projects were found to be “efficient” and used as benchmarking projects.

Research limitations/implications – This research is limited to only security-based OSS projects. Future research on other areas of OSS projects is warranted.

Practical implications – The result of this research is a practical model that can be used by OSS project developers to evaluate the relative performance of their projects and make resource decisions.

Originality/value – This research extends the work of previous studies that have examined the relative performance of software development projects in a traditional development environment. As a result of this research, OSS projects can now be adequately benchmarked and evaluated according to project performance. An OSS project manger can effectively use these results to critically evaluate resources for their project and judge the relative efficiency of the resources.


"The data set used for this research consists of 34 OSS software projects listed on sourceforge.net." "only highly ranked projects in the security domain were considered."
"Ultimately data were collected on the 34 highest ranked security-based OSS software projects on Sourcefore.net."[sic]
fields: bug (?), developers, rank, downloads, Kperdownload (?)

Short TitleInformation Management & Computer Security
Full Text