Improved CTK Weighting Clustering Algorithm Based on MapReduce

WANG Wan-liang; HU Yu

Chinese

您当前的位置：

首页 >

文章列表页 >

Improved CTK Weighting Clustering Algorithm Based on MapReduce

更新时间：2026-09-04

- Improved CTK Weighting Clustering Algorithm Based on MapReduce
- Vol. 35, Issue 12, Pages: 105-109(2018)
- 作者机构：
  
  浙江工业大学机构计算机科学与技术学院,浙江,杭州,310014
- 作者简介：
- 基金信息：
- DOI：
  CLC：
- Published：2018
- 稿件说明：
移动端阅览
王万良, 胡禹. 基于MapReduce的CTK加权聚类改进算法[J]. 微电子学与计算机, 2018, 35(12): 105-109, 114.

WANG Wan-liang, HU Yu. Improved CTK Weighting Clustering Algorithm Based on MapReduce[J]. 2018, 35(12): 105-109.
王万良, 胡禹. 基于MapReduce的CTK加权聚类改进算法[J]. 微电子学与计算机, 2018, 35(12): 105-109, 114. DOI：

WANG Wan-liang, HU Yu. Improved CTK Weighting Clustering Algorithm Based on MapReduce[J]. 2018, 35(12): 105-109. DOI：

摘要

本文提出了一种基于MapReduce的分布式聚类改进算法

该算法将聚类分两阶段进行

首先改进了Canopy算法

引入梯度变化来确定初始中心点以及最佳簇数

解决了传统算法对初始值的依赖性以及对聚类个数的不确定性.设置了区域半径并动态改变

避免了聚类过程中的局部最优

并采用了信息熵加权

解决了相似度计算的特征权重的问题.最终结合MapReduce分布式计算模型

设计了算法的并行策略与方案.试验结果表明该算法在准确性、加速比、扩展性上具有良好的性能。

Abstract

This paper introduces an improved algorithm of Distributed Clustering Based on MapReduce

the process of clustering will be divided into two stages

firstly

introduce Canopy algorithm

find out the suitable K of clustering algorithm by the change of Gradient value. That reduce the number of iterations and avoid the uncertainty of initial center point results in. Then dynamically change the radius of the region to solve the problem of similarity of high-dimensional data sets and solve the problem of characteristic weight of similarity calculation with the weighting of information entropy. Finally

the parallel strategy and scheme of the algorithm are designed according to the MapReduce distributed computing model. Experimental results show that the proposed algorithm has good performance in accuracy

speedup and scalability.

关键词

Keywords

references

Hadian A, Shahrivari S. High performance parallel k-means clustering for disk-resident datasets on multi-core CPUs[J]. The Journal of Supercomputing, 2014, 69(2):845-863. DOI: 845

Frey B J, Dueck D. Clustering by passing messages between data points[J]. science, 2007, 315(5814): 972-976. DOI: 972

Lee G, Tolia N, Ranganathan P. Computing cluster performance simulation using a genetic algorithm solution: U.S. Patent 9, 785, 472[P]. 2017-10-10.

Elhoseny M, Yuan X, Yu Z, et al. Balancing energyconsumption in heterogeneous wireless sensor networks.using genetic algorithm[J]. IEEE Communications Letters, 2015, 19(12): 2194-2197. DOI: 2194

Liu J, Han L, Hou L. Cluster Analysis Based on Particle Swarm Optimization Algorithm [J]. Systems Engineering-theory % Practice, 2005(6): 54-58. DOI: 54

Niknam T, Amiri B. An efficient hybrid approach based on PSO, ACO and k-means for cluster analysis[J]. Applied Soft Computing, 2010, 10(1): 183-197. DOI: 183

Shang T, Zhao Z, Guan Z, et al. A DP Canopy K-Means Algorithm for Privacy Preservation of Hadoop Platform[C]//International Symposium on Cyberspace Safety and Security. Springer, Cham, 2017: 189-198.

Wu G, Lin H, Fu E, et al. An Improved K-means Algorithm for Document Clustering[J]. Journal of Shandong University, 2008, 43(1):60-64. DOI: 60

Dean J, Ghemawat S. MapReduce: simplified data processing on large clusters[J]. Communications of the ACM, 2008, 51(1): 107-113. DOI: 107

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Microservices anomaly detection based on the correlation of monitoring metrics

Monte carlo analysis of V-layout trench SiC neutron detector

Aerial small object detection algorithm based on improved RT-DETR

A review of large model development: technological evolution from BERT to Deepseek

A high-efficiency charge pump for ultra-low voltage boost systems

Related Author

No data

Related Institution

No data

AI问答

Postal code：100079
Tel：（010）53879206 Email：tmw@bjxintong.com.cn
Technical support is provided by Beijing Founder electronics co., LTD 京ICP备09082226号-64 京公网安备11010602201714号
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰