Big Data Classification Algorithm Based on MapReduce to Improve K-NN

JIANG Hua; HAN Fei; WANG Xin; WANG Hui-jiao

Chinese

您当前的位置：

首页 >

文章列表页 >

Big Data Classification Algorithm Based on MapReduce to Improve K-NN

更新时间：2026-09-04

- Big Data Classification Algorithm Based on MapReduce to Improve K-NN
- Vol. 35, Issue 10, Pages: 36-40(2018)
- 作者机构：
  
  桂林电子科技大学机构计算机与信息安全学院,广西,桂林,541000
- 作者简介：
- 基金信息：
- DOI：
  CLC：
- Published：2018
- 稿件说明：
移动端阅览
蒋华, 韩飞, 王鑫, 王慧娇. 基于MapReduce改进K-NN的大数据分类算法研究[J]. 微电子学与计算机, 2018, 35(10): 36-40, 45.

JIANG Hua, HAN Fei, WANG Xin, et al. Big Data Classification Algorithm Based on MapReduce to Improve K-NN[J]. 2018, 35(10): 36-40.
蒋华, 韩飞, 王鑫, 王慧娇. 基于MapReduce改进K-NN的大数据分类算法研究[J]. 微电子学与计算机, 2018, 35(10): 36-40, 45. DOI：

JIANG Hua, HAN Fei, WANG Xin, et al. Big Data Classification Algorithm Based on MapReduce to Improve K-NN[J]. 2018, 35(10): 36-40. DOI：

摘要

针对传统k-nearest neighbor algorithm(K-NN)分类算法计算量大、高维度海量数据集处理效率低的缺点

本文基于Hadoop平台依托MapReduce分布式编程模型改写Map和Reduce函数

并针对传统K-NN提出数据集主成分分析和临界区域数据预测时距离加权的方法.首先

对高维度数据进行主成分分析达到降维的目的

从而提高运行效率; 其次

在预测分类阶段加入完全区域和临界区域的概念

临界区域对个值种类别进行距离加权

提高准确率; 最后

在Hadoop集群环境下的算法运行

针对海量数据进一步提高其运行效率.实验结果表明:该算法在处理海量数据时极大地提高了计算效率和准确率.

Abstract

Aiming at the shortcomings of traditional k-nearest neighbor algorithm (K-NN) classification algorithm

such as large amount of calculation and high-dimension massive data set processing efficiency

this paper revises the Map and Reduce functions based on Hadoop platform by using MapReduce distributed programming model. Principal component analysis and critical region data when the distance weighted method. First

the principal component analysis of high-dimensional data to achieve the purpose of reducing dimension

so as to improve operational efficiency; secondly

in the classification stage of prediction

adding the concept of complete region and critical region

the critical region of values of species distance weighted

Finally

the algorithm running under the Hadoop cluster environment can further improve its operation efficiency against massive data. The experimental results show that this algorithm greatly improves the computational efficiency and accuracy when dealing with massive data

关键词

Keywords

references

Wang S, Wang H J, Qin X P, et al. Architecting big data: challenges, studies and forecasts[J]. Chinese Journal of Computers, 2011, 34(10):1741-1752. DOI: 1741

Dean J, Ghemawat S. MapReduce: a flexible data processing tool[J]. Communications of the Acm, 2010, 53(1):72-77. DOI: 72

苏毅娟, 邓振云, 程德波, 等.大数据下的快速KNN分类算法[J].计算机应用研究, 2016, 33(4):1003-1006. DOI: 1003

Zhang M L, Zhou Z H. ML-KNN: A lazy learning approach to multi-label learning[J]. Pattern Recognition, 2007, 40(7):2038-2048. DOI: 2038

周英, 旧金武, 卞月青.大数据挖掘:系统方法与实例分析[M].北京:机械工业出版社, 2016.

He B, Fang W, Luo Q, et al. Mars: a map reduce framework on graphics processors[C]// International Conference on Parallel Architectures and Compilation Techniques. Minneapolis, USA, IEEE, 2017: 260-269.

杨秀荣.并行数据库中异常数据优化分类挖掘方法研究[J].微电子学与计算机, 2015, 33(10):125-128. DOI: 125

戴健, 丁治明.基于MapReduce快速kNN Join方法[J].计算机学报, 2015, 38(1):99-108. DOI: 99

Li N, Kong H, Ma Y, et al. Human performance modeling for manufacturing based on an improved KNN algorithm[J]. International Journal of Advanced Manufacturing Technology, 2016, 84(1/4):473-483. DOI: 473

蒋华, 胡莹.基于MapReduce技术的Argo浮标剖面信息融合算法[J].计算机应用, 2015, 35(12):3403-3407. DOI: 3403

Jiang L, Cai Z, Wang D, et al. Bayesian Citation-KNN with distance weighting[J]. International Journal of Machine Learning % Cybernetics, 2014, 5(2):193-199. DOI: 193

Schmitt E, Vakili K. The FastHCS algorithm for robust PCA[J]. Statistics % Computing, 2015, 26(6):1-14. DOI: 1

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

A review of large model development: technological evolution from BERT to Deepseek

Lightweight countermeasures against Spectre attacks on the branch prediction unit

Research on assembly action recognition based on MSS-YOLO-MPHands fusion architecture

Aerial vehicle detection algorithm based on multi-scale fusion and small-object enhancement

A high-efficiency charge pump for ultra-low voltage boost systems

Related Author

No data

Related Institution

No data

AI问答

Postal code：100079
Tel：（010）53879206 Email：tmw@bjxintong.com.cn
Technical support is provided by Beijing Founder electronics co., LTD 京ICP备09082226号-64 京公网安备11010602201714号
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰