Validity of network analyses in Open Source Projects

TitleValidity of network analyses in Open Source Projects
Publication TypeConference Paper
Year of Publication2010
AuthorsNia, R, Bird, C, Devanbu, P, Filkov, V
Secondary Title2010 7th IEEE Working Conference on Mining Software Repositories (MSR 2010)2010 7th IEEE Working Conference on Mining Software Repositories (MSR 2010)
Pagination201 - 209
Place PublishedCape Town, South Africa
ISBN Number978-1-4244-6802-7
Keywordsapache, email archives, mailing lists, missing data, mysql, perl, social networks

Social network methods are frequently used to analyze networks derived from Open Source Project communication and collaboration data. Such studies typically discover patterns in the information flow between contributors or contributions in these projects. Social network metrics have also been used to predict defect occurrence. However, such studies often ignore or side-step the issue of whether (and in what way) the metrics and networks of study are influenced by inadequate or missing data. In previous studies email archives of OSS projects have provided a useful trace of the communication and co-ordination activities of the participants. These traces have been used to construct social networks that are then subject to various types of analysis. However, during the construction of these networks, some assumptions are made, that may not always hold; this leads to incomplete, and sometimes incorrect networks. The question then becomes, do these errors affect the validity of the ensuing analysis? In this paper we specifically examine the stability of network metrics in the presence of inadequate and missing data. The issues that we study are: 1) the effect of paths with broken information flow (i.e. consecutive edges which are out of temporal order) on measures of centrality of nodes in the network, and 2) the effect of missing links on such measures. We demonstrate on three different OSS projects that while these issues do change network topology, the metrics used in the analysis are stable with respect to such changes.


"We have mined archival records of developer mailing lists to generate reply-to social networks for the three OSS projects: Apache, MySQL, and Perl."

"For each of these projects, we construct an information flow network based on messages that are sent as replies to previous messages....We use this methodology on all mined data to create a network of mailing list participants. "

Full Text
PDF icon 201NetworkAnalysis.pdf318.9 KB