基于MapReduce的CTK加权聚类改进算法

王万良; 胡禹

Chinese

您当前的位置：

首页 >

文章列表页 >

基于MapReduce的CTK加权聚类改进算法

更新时间：2026-09-04

- 基于MapReduce的CTK加权聚类改进算法
- Improved CTK Weighting Clustering Algorithm Based on MapReduce
- 微电子学与计算机 2018年35卷第12期页码：105-109
- 作者机构：
  
  浙江工业大学机构计算机科学与技术学院,浙江,杭州,310014
- 作者简介：
- 基金信息：
  
  国家自然科学基金项目 61572438国家自然科学基金项目 C119017327
- DOI：
  中图分类号：
- 纸质出版：2018
- 稿件说明：
移动端阅览
王万良, 胡禹. 基于MapReduce的CTK加权聚类改进算法[J]. 微电子学与计算机, 2018, 35(12): 105-109, 114.

WANG Wan-liang, HU Yu. Improved CTK Weighting Clustering Algorithm Based on MapReduce[J]. 2018, 35(12): 105-109.
王万良, 胡禹. 基于MapReduce的CTK加权聚类改进算法[J]. 微电子学与计算机, 2018, 35(12): 105-109, 114. DOI：

WANG Wan-liang, HU Yu. Improved CTK Weighting Clustering Algorithm Based on MapReduce[J]. 2018, 35(12): 105-109. DOI：

摘要

本文提出了一种基于MapReduce的分布式聚类改进算法

该算法将聚类分两阶段进行

首先改进了Canopy算法

引入梯度变化来确定初始中心点以及最佳簇数

解决了传统算法对初始值的依赖性以及对聚类个数的不确定性.设置了区域半径并动态改变

避免了聚类过程中的局部最优

并采用了信息熵加权

解决了相似度计算的特征权重的问题.最终结合MapReduce分布式计算模型

设计了算法的并行策略与方案.试验结果表明该算法在准确性、加速比、扩展性上具有良好的性能。

Abstract

This paper introduces an improved algorithm of Distributed Clustering Based on MapReduce

the process of clustering will be divided into two stages

firstly

introduce Canopy algorithm

find out the suitable K of clustering algorithm by the change of Gradient value. That reduce the number of iterations and avoid the uncertainty of initial center point results in. Then dynamically change the radius of the region to solve the problem of similarity of high-dimensional data sets and solve the problem of characteristic weight of similarity calculation with the weighting of information entropy. Finally

the parallel strategy and scheme of the algorithm are designed according to the MapReduce distributed computing model. Experimental results show that the proposed algorithm has good performance in accuracy

speedup and scalability.

关键词

Keywords

references

Hadian A, Shahrivari S. High performance parallel k-means clustering for disk-resident datasets on multi-core CPUs[J]. The Journal of Supercomputing, 2014, 69(2):845-863. DOI: 845

Frey B J, Dueck D. Clustering by passing messages between data points[J]. science, 2007, 315(5814): 972-976. DOI: 972

Lee G, Tolia N, Ranganathan P. Computing cluster performance simulation using a genetic algorithm solution: U.S. Patent 9, 785, 472[P]. 2017-10-10.

Elhoseny M, Yuan X, Yu Z, et al. Balancing energyconsumption in heterogeneous wireless sensor networks.using genetic algorithm[J]. IEEE Communications Letters, 2015, 19(12): 2194-2197. DOI: 2194

Liu J, Han L, Hou L. Cluster Analysis Based on Particle Swarm Optimization Algorithm [J]. Systems Engineering-theory % Practice, 2005(6): 54-58. DOI: 54

Niknam T, Amiri B. An efficient hybrid approach based on PSO, ACO and k-means for cluster analysis[J]. Applied Soft Computing, 2010, 10(1): 183-197. DOI: 183

Shang T, Zhao Z, Guan Z, et al. A DP Canopy K-Means Algorithm for Privacy Preservation of Hadoop Platform[C]//International Symposium on Cyberspace Safety and Security. Springer, Cham, 2017: 189-198.

Wu G, Lin H, Fu E, et al. An Improved K-means Algorithm for Document Clustering[J]. Journal of Shandong University, 2008, 43(1):60-64. DOI: 60

Dean J, Ghemawat S. MapReduce: simplified data processing on large clusters[J]. Communications of the ACM, 2008, 51(1): 107-113. DOI: 107

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

V型沟槽SiC中子探测器的蒙特卡洛分析

超低功耗实时时钟电路的设计

大模型发展综述：从BERT到Deepseek的技术演进

AlN/GaN混合缓冲层双异质结p-GaN HEMT直流特性仿真

基于改进RT-DETR的航拍小目标检测算法