Frequent Itemsets Mining Algorithm Based on MapReduce and Matrix

ZHOU Guo-jun; GONG Yu-tong

Chinese

您当前的位置：

首页 >

文章列表页 >

Frequent Itemsets Mining Algorithm Based on MapReduce and Matrix

更新时间：2026-09-04

- Frequent Itemsets Mining Algorithm Based on MapReduce and Matrix
- Vol. 33, Issue 5, Pages: 119-123(2016)
- 作者机构：
  
  玉林师范学院机构数学与信息科学学院,广西,玉林,537000
- 作者简介：
- 基金信息：
- DOI：
  CLC：
- Published：2016
- 稿件说明：
移动端阅览
周国军, 龚榆桐. 基于MapReduce和矩阵的频繁项集挖掘算法[J]. 微电子学与计算机, 2016, 33(5): 119-123.

ZHOU Guo-jun, GONG Yu-tong. Frequent Itemsets Mining Algorithm Based on MapReduce and Matrix[J]. 2016, 33(5): 119-123.
周国军, 龚榆桐. 基于MapReduce和矩阵的频繁项集挖掘算法[J]. 微电子学与计算机, 2016, 33(5): 119-123. DOI：

ZHOU Guo-jun, GONG Yu-tong. Frequent Itemsets Mining Algorithm Based on MapReduce and Matrix[J]. 2016, 33(5): 119-123. DOI：

摘要

为了能高效地从大数据集中挖掘所有频繁项集

提出了一种基于MapReduce和矩阵的频繁项集挖掘算法.该算法将事务数据库转化为矩阵

按照垂直划分方法将矩阵分解成多个规模相同的子矩阵

根据MapReduce模型将子矩阵分配给计算机集群的节点

各节点并行对子矩阵计算候选项集的支持度.该算法在执行过程中产生的通信量较少

实现了节点计算任务的负载平衡.在Hadoop平台上测试了算法的性能

实验结果表明该算法具有较好的加速比和可扩展性

适合对大数据集挖掘频繁项集.

Abstract

To efficiently find all frequent itemsets from large data sets

a frequent itemsets mining algorithm based on MapReduce and matrix is proposed. The algorithm includes the following points: Transaction database is transformed into matrix

then the matrix is divided into multiple submatrices by vertical partitioning method. According to MapReduce model

submatrices are assigned to the nodes of cluster

and the support of candidate itemsets is parallelly computed by the nodes. The amount of communication generated during the execution of the algorithm is small. Load balance among nodes is achieved. The performance of the algorithm is tested on Hadoop platform. Experimental results show that the algorithm has good speedup and scalability

which is suitable for mining frequent itemsets from large data sets.

关键词

Keywords

references

Dean J, Ghemawat S. MapReduce: simplified data processing on large clusters[J]. Communications of the ACM, 2008, 51(1): 107-113. DOI: 107

Agrawal R, Srikant R. Fast algorithms for mining association rules[C]//Proceedings of the 20th Interna- tional Conference on Very Large Data Bases. Santiago, Chile: IEEE, 1994: 487-499.

付沙, 宋丹. 基于矩阵的Apriori改进算法研究[J]. 微电子学与计算机, 2012, 29(5): 156-160. DOI: 156

吕婉琪, 钟诚, 唐印浒, 等. Hadoop分布式架构下大数据集的并行挖掘[J]. 计算机技术与发展, 2014, 24(1): 22-25. DOI: 22

施亮, 钱雪忠. 基于Hadoop的并行FP-Growth算法的研究与实现[J]. 微电子学与计算机, 2015, 32(4): 150-154. DOI: 150

易小华, 刘杰, 叶丹. 面向MapReduce的数据处理流程开发方法[J]. 计算机科学与探索, 2011, 5(2): 161-168. DOI: 161

王与力, 杨晓东. 一种更有效的并行系统可扩展性模型[J]. 计算机学报, 2001, 24(1): 84-90. DOI: 84

李成华, 张新访, 金海, 等. MapReduce: 新型的分布式并行计算编程模型[J]. 计算机工程与科学, 2011, 33(3): 129-135. DOI: 129

陈兴蜀, 张帅, 童浩, 等. 基于布尔矩阵和MapReduce的FP-Growth算法[J]. 华南理工大学学报(自然科学版), 2014, 42(1): 135-141. DOI: 135

张敏, 姚良威, 侯宇. 基于向量和矩阵的频繁项集挖掘算法研究[J]. 计算机工程与设计, 2013, 34(3): 939-943. DOI: 939

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Big data classification method of neighborhood search for online feature selection

A Algorithm for Unbalanced Big Sata Using Paralleled Random Forest

Task Scheduling Algorithm Used for Multiple MapReduce Tasks

Multi-predicate Constrained Frequent Itemsets Mining Based on Hash Table

The Improved Apriori Algorithm Based on Compression Matrix Approach

Related Author

LI Yueying

QIN Jing

QIAN Xue-zhong

WANG Wei-tao

XIE Guo-wei

SONG Wei

LIU Tao

WU Shu-hong

Related Institution

Xinyang Agriculture and Forestry University

Engineering Research Center of Internet of Things Technology Applications, Ministry of Education, Jiangnan University

School of Computer Science and Technology, Taiyuan University of Technology

1 School of IT, Jinling Institute of Technology

2 Jiangsu Information Analysis Engineering Laboratory

AI问答

Postal code：100079
Tel：（010）53879206 Email：tmw@bjxintong.com.cn
Technical support is provided by Beijing Founder electronics co., LTD 京ICP备09082226号-64 京公网安备11010602201714号
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰