A Distributed and Cooperative NameNode Cluster for a Highly-Available Hadoop Distributed File System
-
- KIM Yonghwan
- Graduate School of Information Science and Technology, Osaka University
-
- ARARAGI Tadashi
- Proassist, Ltd.
-
- NAKAMURA Junya
- Information and Media Center, Toyohashi University of Technology
-
- MASUZAWA Toshimitsu
- Graduate School of Information Science and Technology, Osaka University
Abstract
Recently, Hadoop has attracted much attention from engineers and researchers as an emerging and effective framework for Big Data. HDFS (Hadoop Distributed File System) can manage a huge amount of data with high performance and reliability using only commodity hardware. However, HDFS requires a single master node, called a NameNode, to manage the entire namespace (or all the i-nodes) of a file system. This causes the SPOF (Single Point Of Failure) problem because the file system becomes inaccessible when the NameNode fails. This also causes a bottleneck of efficiency since all the access requests to the file system have to contact the NameNode. Hadoop 2.0 resolves the SPOF problem by introducing manual failover based on two NameNodes, Active and Standby. However, it still has the efficiency bottleneck problem since all the access requests have to contact the Active in ordinary executions. It may also lose the advantage of using commodity hardware since the two NameNodes have to share a highly reliable sophisticated storage. In this paper, we propose a new HDFS architecture to resolve all the problems mentioned above.
Journal
-
- IEICE Transactions on Information and Systems
-
IEICE Transactions on Information and Systems E98.D (4), 835-851, 2015
The Institute of Electronics, Information and Communication Engineers
- Tweet
Details 詳細情報について
-
- CRID
- 1390282679354319360
-
- NII Article ID
- 130005061847
-
- ISSN
- 17451361
- 09168532
-
- Text Lang
- en
-
- Data Source
-
- JaLC
- Crossref
- CiNii Articles
- KAKEN
-
- Abstract License Flag
- Disallowed