A Calculation Method of Hadoop Cluster Average Life Based on Poisson Process

LI Jia-xuan; ZHENG Xiao-wei; ZHANG Yong

Chinese

您当前的位置：

首页 >

文章列表页 >

A Calculation Method of Hadoop Cluster Average Life Based on Poisson Process

更新时间：2026-09-04

- A Calculation Method of Hadoop Cluster Average Life Based on Poisson Process
- Vol. 33, Issue 7, Pages: 150-153(2016)
- 作者机构：
  
  辽宁师范大学机构计算机与信息技术学院,辽宁,大连,116081
- 作者简介：
- 基金信息：
- DOI：
  CLC：
- Published：2016
- 稿件说明：
移动端阅览
李佳宣, 郑晓薇, 张永. 基于Poisson过程的Hadoop集群平均寿命计算方法[J]. 微电子学与计算机, 2016, 33(7): 150-153, 158.

LI Jia-xuan, ZHENG Xiao-wei, ZHANG Yong. A Calculation Method of Hadoop Cluster Average Life Based on Poisson Process[J]. 2016, 33(7): 150-153.
李佳宣, 郑晓薇, 张永. 基于Poisson过程的Hadoop集群平均寿命计算方法[J]. 微电子学与计算机, 2016, 33(7): 150-153, 158. DOI：

LI Jia-xuan, ZHENG Xiao-wei, ZHANG Yong. A Calculation Method of Hadoop Cluster Average Life Based on Poisson Process[J]. 2016, 33(7): 150-153. DOI：

摘要

Hadoop集群在连续长时间执行任务的过程中

集群故障会对集群造成不同程度的冲击与损害

是集群性能下降、系统故障频发的主要原因

更会导致集群平均寿命的大大缩短.为解决此问题

提出了一种基于Poisson过程的Hadoop集群平均寿命预测方法.通过预测计算

及时对集群进行调整优化

使系统性能保持在正常状态.同时在三种不同特征类型的Hadoop集群的实验中观察集群在任务执行过程中平均寿命的变化.实验结果表明

该方法能有效地判断Hadoop集群故障发生的频率

预测出集群的平均寿命

对集群性能优劣进行评价并及时对集群性能进行优化

Abstract

After a long task execution on Hadoop cluster

as cluster faults would hurt Hadoop cluster on different strength

which became the main reason of cluster performance degradation and more frequent cluster faults

more serious would cause the decrease of cluster average life. To solve the problem

this paper proposes a method of Hadoop cluster average life prediction based on Poisson process. By prediction calculation

adjust and optimize the cluster timely to keep the cluster performance in a normal state. Then make several experiments on three different feature types of Hadoop cluster to observe the frequncy of cluster average life. According to the experiment results

this method is the significant to Hadoop cluster performance evaluation and optimization.

关键词

Keywords

references

Jian Sun, Hongyan Zhu, Zongben Xu, et al. Poisson image fusion based on Markov random field fusion model[J]. Information Fusion, 2013, 26(14) : 241-254. DOI: 241

张波, 商豪. 应用随机过程(第三版)[M]. 北京: 中国人民大学出版社, 2014.

P Szul, T Bednarz. Productivity frameworks in big data image processing computations - creating photographic mosaics with Hadoop and Scalding[C]// Proceeding of the 14th International Conference on Compatation Science. Elsevier: IEEE, 2014: 2306-2314.

Xiaowei Zheng, Jiaxuan Li, Yong Zhang, et al. An optimization model of Hadoop cluster performance prediction based on Markov process[J]. International Journal of Computer Systems Science % Engineering, 2015, 24(6): 724-734. DOI: 724

郑晓薇, 项明, 张大为, 等. 基于节点能力的Hadoop集群任务级调度自适应分配方法[J]. 计算机研究与发展, 2014, 51(3): 618-626. DOI: 618

王淑芬, 高军礼, 邹普, 等. 基于Hadoop的广域网分布式主题爬虫系统框架[J]. 计算机工程与科学, 2015, 37(4): 670-675. DOI: 670

Lorenzo Rossi, Jacob Chakareski, Pascal Frossard, Stefania Colonnese. A Poisson Hidden Markov Model for Multiview Video Traffic[J]. IEEE/ACM Transactions On Net Working, 2014, 23(2): 1063-6692 DOI: 1063

应毅, 任凯, 刘正涛. 基于云计算技术的数据挖掘[J]. 微电子学与计算机, 2013, 30(2): 161-164. DOI: 161

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

A high-efficiency charge pump for ultra-low voltage boost systems

Microservices anomaly detection based on the correlation of monitoring metrics

Simulation on the DC characteristics of double heterojunction p-GaN HEMT with AlN/GaN hybrid buffer layer

Design of ultra-low power real-time clock circuits

Aerial small object detection algorithm based on improved RT-DETR

Related Author

No data

Related Institution

No data

AI问答

Postal code：100079
Tel：（010）53879206 Email：tmw@bjxintong.com.cn
Technical support is provided by Beijing Founder electronics co., LTD 京ICP备09082226号-64 京公网安备11010602201714号
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰