MapReduce环境下处理多类别不平衡数据的改进随机森林算法

姚立; 张曦煌

Chinese

您当前的位置：

首页 >

文章列表页 >

MapReduce环境下处理多类别不平衡数据的改进随机森林算法

更新时间：2026-09-04

- MapReduce环境下处理多类别不平衡数据的改进随机森林算法
- An Improved Random Forest Algorithm for Multi-Class Imbalanced Data Problem Under Map Reduce
- 微电子学与计算机 2018年35卷第11期页码：139-144
- 作者机构：
  
  江南大学物联网工程学院,江苏,无锡,214122
- 作者简介：
- 基金信息：
  
  江苏省产学研合作项目 BY2015019-30江苏省自然科学基金 BK20140166
- DOI：
  中图分类号：
- 纸质出版：2018
- 稿件说明：
移动端阅览
姚立, 张曦煌. MapReduce环境下处理多类别不平衡数据的改进随机森林算法[J]. 微电子学与计算机, 2018, 35(11): 139-144.

YAO Li, ZHANG Xi-huang. An Improved Random Forest Algorithm for Multi-Class Imbalanced Data Problem Under Map Reduce[J]. 2018, 35(11): 139-144.
姚立, 张曦煌. MapReduce环境下处理多类别不平衡数据的改进随机森林算法[J]. 微电子学与计算机, 2018, 35(11): 139-144. DOI：

YAO Li, ZHANG Xi-huang. An Improved Random Forest Algorithm for Multi-Class Imbalanced Data Problem Under Map Reduce[J]. 2018, 35(11): 139-144. DOI：

摘要

针对传统MapReduce环境下的随机森林算法在处理多类别不平衡数据问题时仍然以全局最优点作为划分点

忽略了少数类对分类准确率的影响

文本提出了一种MapReduce环境下处理多类别不平衡数据的改进随机森林算法(MR-RF-SHDSE).该算法利用分层采样方法在各个类别中进行样本抽样

并以HDDT决策树作为基学习器以弱化数据偏置给分类准确率带来的影响

最后计算决策树的GMean值和不合度值

利用调和平均值作为衡量标准对决策树进行选择集成.通过实验证明

相比其他算法

MR-RF-SHDSE能够有效提高了对多类别不平衡数据集的分类准确率.

Abstract

Because the traditional random forest algorithm under the MapReduce still takes the global optimal point as the dividing point when dealing with the multi-class imbalance data problem

ignoring the influence of the minority class on the classification accuracy rate

this paper presents an improved random forest algorithm (MR-RF-SHDSE) for dealing with multi-class imbalance data under MapReduce. This algorithm uses the stratified sampling method to sample the samples in each category

and uses the HDDT decision tree as the learner to weaken the impact of data bias on the classification accuracy. Finally

the GMean value and the disagreement measure value of the decision tree are calculated

we use harmonic mean as a metric to select decision trees. Experiments show that compared with other algorithms

MR-RF-SHDSE can effectively improve the classification accuracy of multi-class imbalanced data sets.

关键词

Keywords

references

Zhou Z H, Wu J, Tang W. Ensembling neural networks: many could be better than all[J]. Artificial intelligence, 2002, 137(1-2): 239-263. DOI: 239

Espíndola R P, Ebecken N F F. On extending f-measure and g-mean metrics to multi-class problems[J]. Data Mining VI Data Mining Text Mining % Their business applications, 2014.

Liu T, Wang F, Agrawal G. Stratified Sampling for Data Mining on the Deep Web[J]. Frontiers of Computer Science, 2012, 6(2):179-196. DOI: 179

周志华.机器学习[M].北京:清华大学出版社, 2016.

Hernández-Lobato D, Martínez-Mu?oz G, Suárez A. Out of Bootstrap Estimation of Generalization Error Curves in Bagging Ensembles[M]// Intelligent Data Engineering and Automated Learning - IDEAL 2007. Springer Berlin Heidelberg, 2007: 47-56.

Dean J, Ghemawat S. MapReduce: simplified data processing on large clusters[J]. Communications of the ACM, 2008, 51(1): 107-113. DOI: 107

Herrera F. On the use of MapReduce for imbalanced big data using Random Forest[J]. Information Sciences An International Journal, 2014, 285(C):112-137. DOI: 112

Fernández-Delgado M, Cernadas E, Barro S, et al. Do we need hundreds of classifiers to solve real world classification problems[J]. Journal of Machine Learning Research, 2014, 15(1): 3133-3181. DOI: 3133

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于随机森林的网表级时序预测模型

面向处理器分支预测单元的Spectre攻击轻量化防御方法

V型沟槽SiC中子探测器的蒙特卡洛分析

基于MSS-YOLO-MPHands融合架构的装配手部动作识别研究

基于多尺度融合与小目标增强的航拍车辆检测算法