A Algorithm for Unbalanced Big Sata Using Paralleled Random Forest

QIN Jing; QIAN Xue-zhong; WANG Wei-tao; XIE Guo-wei; SONG Wei

Chinese

您当前的位置：

首页 >

文章列表页 >

A Algorithm for Unbalanced Big Sata Using Paralleled Random Forest

更新时间：2026-09-04

- A Algorithm for Unbalanced Big Sata Using Paralleled Random Forest
- Vol. 34, Issue 4, Pages: 22-27(2017)
- 作者机构：
  
  江南大学物联网技术应用教育部工程研究中心机构,江苏,无锡,214122
- 作者简介：
- 基金信息：
- DOI：
  CLC：
- Published：2017
- 稿件说明：
移动端阅览
秦静, 钱雪忠, 王卫涛, 谢国伟, 宋威. 一种处理不平衡大数据的并行随机森林算法[J]. 微电子学与计算机, 2017, 34(4): 22-27.

QIN Jing, QIAN Xue-zhong, WANG Wei-tao, et al. A Algorithm for Unbalanced Big Sata Using Paralleled Random Forest[J]. 2017, 34(4): 22-27.
秦静, 钱雪忠, 王卫涛, 谢国伟, 宋威. 一种处理不平衡大数据的并行随机森林算法[J]. 微电子学与计算机, 2017, 34(4): 22-27. DOI：

QIN Jing, QIAN Xue-zhong, WANG Wei-tao, et al. A Algorithm for Unbalanced Big Sata Using Paralleled Random Forest[J]. 2017, 34(4): 22-27. DOI：

摘要

基于MapReduce机制的并行随机森林算法MR_RF在处理不平衡大数据时

由于数据本身正类密度低且算法又以全局最优标准选择分割点

而导致正类有误分为负类的趋势

降低了分类效率.本文提出了一种改进的并行随机森林算法(SBWMR_RF)

该算法利用分层自助抽样方法增大对少数类的抽样数量

同时考虑正负类不同的误分代价

动态计算每个分区的代价敏感矩阵

将其引入到构建基分类器的关键步骤

弱化数据偏置的影响.实验证明SBWMR_RF算法提高了对不平衡大数据的分类准确率

没有出现过拟合现象

在极不平衡环境下优势明显.

Abstract

Paralleled random forest algorithm based on MapReduce(MR_RF) which constructing trees on partitions is a classic ensemble algorithm for big data classification.However

when encountering imbalanced big data

its performance will decrease with the tendency of positive samples misclassified because of the low density of positive samples themselves and the algorithms global optimal criteria for choosing split points.In this paper

An improved paralleled random forest called SBWMR_RF is proposed.It adopts stratified bootstrap to increase the minority during sampling.At the same time

cost-sensitive thought is applied through the key steps of tree construction

modify the distribution of the minority.The experiments prove that SBWMR_RF can effectively classify unbalanced big data especially in extremely unbalanced data scenario without overfitting but high speedup.

关键词

Keywords

references

Kamienski C, Kelner J, Mariz D, et al. A stratified traffic sampling methodology for seeing the big picture[J]. Computer Networks the International Journal of Computer % Telecommunications Networking, 2008, 52(14): 2677-2689. DOI: 2677

Chawla N V, Cieslak D A, Hall L O, et al. Automatically countering imbalance and its empirical relationship to cost[J]. Data Mining % Knowledge Discovery, 2008, 17(2): 225-252. DOI: 225

Wang S, Yao X. Diversity analysis on imbalanced data sets by using ensemble models[C]//IEEE Symposium on Computational Intelligence and Data Mining. Nashville, TN, USA, 2009: 535-548.

Breiman L. Random forest[J]. Machine Learning, 2001, 45(1): 5-32. DOI: 5

López V, Fernández A, García S, et al. An insight into classification with imbalanced data: Empirical results and current trends on using data intrinsic characteristics[J]. Information Sciences, 2013, 250(11): 113-141. DOI: 113

Genuer R, Poggi J M, Tuleau Malot C, et al. Random forests for big data[J]. arXiv preprint arXiv: 1511.08327, 2015.

Chen C, Breiman L. Using random forest to learn imbalanced data[J]. Telkomnika Indonesian Journal of Electrical Engineering, 2004, 45(1): 5-32. DOI: 5

谷琼, 袁磊, 熊启军, 等. 基于非均衡数据集的代价敏感学习算法比较研究[J]. 微电子学与计算机, 2011, 28(8): 146-149. DOI: 146

李建江, 崔健, 王聃, 等. MapReduce并行编程模型研究综述[J]. 电子学报, 2011, 39(11): 2635-2642. DOI: 2635

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Big data classification method of neighborhood search for online feature selection

Frequent Itemsets Mining Algorithm Based on MapReduce and Matrix

Task Scheduling Algorithm Used for Multiple MapReduce Tasks

High gain stability high temperature operational amplifier based on reverse body bias

Monte carlo analysis of V-layout trench SiC neutron detector

Related Author

LI Yueying

ZHOU Guo-jun

GONG Yu-tong

LIU Tao

WU Shu-hong

QIANG Yan

Related Institution

Xinyang Agriculture and Forestry University

School of Mathematics and Information Science, Yulin Normal University

School of Computer Science and Technology, Taiyuan University of Technology

AI问答

Postal code：100079
Tel：（010）53879206 Email：tmw@bjxintong.com.cn
Technical support is provided by Beijing Founder electronics co., LTD 京ICP备09082226号-64 京公网安备11010602201714号
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰