A Multi-Layer Hadoop Platform

LI Zhao-xing; MA Zi-tang

Chinese

您当前的位置：

首页 >

文章列表页 >

A Multi-Layer Hadoop Platform

更新时间：2026-09-04

- A Multi-Layer Hadoop Platform
- Vol. 33, Issue 2, Pages: 27-30(2016)
- 作者机构：
  
  解放军信息工程大学密码工程学院,河南,郑州,450000
- 作者简介：
- 基金信息：
- DOI：
  CLC：
- Published：2016
- 稿件说明：
移动端阅览
李兆兴, 马自堂. 一种多层次Hadoop平台设计[J]. 微电子学与计算机, 2016, 33(2): 27-30.

LI Zhao-xing, MA Zi-tang. A Multi-Layer Hadoop Platform[J]. 2016, 33(2): 27-30.
李兆兴, 马自堂. 一种多层次Hadoop平台设计[J]. 微电子学与计算机, 2016, 33(2): 27-30. DOI：

LI Zhao-xing, MA Zi-tang. A Multi-Layer Hadoop Platform[J]. 2016, 33(2): 27-30. DOI：

摘要

大数据和大数据中心规模的进一步扩大为Hadoop平台的可扩展性提出了新的要求.在系统地分析了制约Hadoop平台性能和可扩展性因素的基础上

提出了一种多层次Hadoop平台.通过将Hadoop平台划分为若干区域

形成全局-区域-节点的多层次结构.由Master节点负责系统级的元数据管理和任务分发

区域管理节点负责区域级的元数据管理和任务分发

从而提高Hadoop平台可扩展性和容错性.实验表明

该平台可有效提高Hadoop的可扩展性

具有一定的可行性.

Abstract

With the further expansion of Big Data and Big Data Centre

new extensibility requirements are proposed on the Hadoop Platform. In the base of systematically analysis constraints on performance and extensibility of Hadoop Platform

this paper presents a multi-level Hadoop Platform which has a structure of Global-Region-Node after being divided. In this platform

Master Node is responsible for metadata management and task development on the global level

and regional management node takes charge of metadata management and task development on the regional level

which improve both scalability and fault tolerance of Hadoop Platform. Experiments show that this platform can effectively promote the scalability of Hadoop and has certain feasibility.

关键词

Keywords

references

Borthakur D, Sarma J. Apache hadoop goes realtime at facebook[C]// Proceedings of the 2011 ACM SIGMOD International Conference on Management of data. New York, ACM, 2011: 1071.

Shvachko K, Kuang H, Radia S, et al. The hadoop distributed file system[C]// 2010. Mass Storage Systems and Technologies (MSST), 2010 IEEE 26th Symposium on. Nevada, Indine Village, IEEE, 2010: 1-10.

龙柏. 并行计算平台上的数据索引技术研究[D]. 合肥: 中国科学技术大学, 2011: 25-33.

O'Reilly Tom White T. Hadoop: the definitive guide[M]. O'Reilly: Yahoo Press, 2011: 35-36. DOI: 35

王凯. MapReduce集群多用户作业调度方法的研究与实现[D]. 长沙: 国防科技大学, 2010: 15-24.

覃雄派. 大数据分析: RDBMS与MapReduce的竞争与共生[J]. 软件学报, 2012, 23(1): 23. DOI: 23

邓鹏, 李枚毅. 何诚. Namenode单点故障解决方案研究[J]. 计算机工程, 2012, 38(21): 40-44. DOI: 40

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Algorithm for Mining Frequent Patterns in Big Data Based on Hadoop

Research on Efficient Data Filtering Based on FPGA

Hadoop Architecture Based-on Virtual Technology of KVM

A feature-oriented context understanding and slice analysis network for tuberculosis detection

Lightweight countermeasures against Spectre attacks on the branch prediction unit

Related Author

LI Xiao-lin

DU Tuo

XIE Yong

CHEN Bao-yuan

ZHANG Xiu-zhi

LIANG Zhuang

XIA Shang-yun

WANG Min-chao

Related Institution

Chongqing University of Posts and Telecommunications, New Technology Application Research Center

Chongqing Information Technology Designing CO. LTD

The Higher Education Key Laboratory for Measuring & Control Technology and Instrumentations of Heilongjiang Province, Harbin University of Science & Technology

School of Computer Engineering and Science, Shanghai University

High Performance Computing Center, Shanghai University

AI问答

Postal code：100079
Tel：（010）53879206 Email：tmw@bjxintong.com.cn
Technical support is provided by Beijing Founder electronics co., LTD 京ICP备09082226号-64 京公网安备11010602201714号
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰