Developing Architectural Documentation for the Hadoop Distributed File System

Submitted by msquire on Thu, 2011-12-01 10:51
TitleDeveloping Architectural Documentation for the Hadoop Distributed File System
Publication TypeConference Proceedings
Year of Publication2011
AuthorsBass, Len, Kazman Rick, and Ozkaya Ipek
Secondary TitleOpen Source Systems: Grounding Research (OSS 2011)
Pagination50-61
Date Published10/2011
PublisherSpringer
Abstract

Many open source projects are lacking architectural documentation that describes the major pieces of the system, how they are structured, and how they interact. We have produced architectural documentation for the Hadoop Distributed File System (HDFS), a major open source project. This paper describes our process and experiences in developing this documentation. We illustrate the documentation we have produced and how it differs from existing documentation by describing the redundancy mechanisms used in HDFS for reliability.