Data Mining Based on Improved Map/Reduce and Pattern Space Division

LIU Qian; CHEN Ming

Chinese

您当前的位置：

首页 >

文章列表页 >

Data Mining Based on Improved Map/Reduce and Pattern Space Division

更新时间：2026-09-04

- Data Mining Based on Improved Map/Reduce and Pattern Space Division
- Vol. 28, Issue 8, Pages: 140-142(2011)
- 作者机构：
  
  中国石油大学计算机科学与技术系,北京,102249
- 作者简介：
- 基金信息：
- DOI：
  CLC：
- Published：2011
- 稿件说明：
移动端阅览
刘骞, 陈明. 基于改进的Map/Reduce及模式空间划分的数据挖掘[J]. 微电子学与计算机, 2011, 28(8): 140-142.

LIU Qian, CHEN Ming. Data Mining Based on Improved Map/Reduce and Pattern Space Division[J]. 2011, 28(8): 140-142.
刘骞, 陈明. 基于改进的Map/Reduce及模式空间划分的数据挖掘[J]. 微电子学与计算机, 2011, 28(8): 140-142. DOI：

LIU Qian, CHEN Ming. Data Mining Based on Improved Map/Reduce and Pattern Space Division[J]. 2011, 28(8): 140-142. DOI：

摘要

为通过Map/Reduce基于键/值对以处理数据集与模式集的多对多的对应关系的方式实现数据挖掘

解决在较复杂的类型的模式的挖掘中存在的由组合爆炸导致的模式集过大的问题

提出了通过模式空间划分实现将处理数据集与模式集的对应关系的问题转化为处理数据集与子模式集的集合的对应关系的问题的方法

并对Map/Reduce集群的调度机制及组织、处理键/值对的方式进行了改进

增强了Map/Reduce执行模式挖掘任务的能力.在Map/Reduce集群上采用该思路实现某些较复杂类型的模式的挖掘算法时的并行度高于Map/Reduce化的传统算法.

Abstract

In order to realize data mining through map/reduce based on key/value pairs

the way of processing the many-to-many corresponding relationship between the data set and the pattern set is adopted.For some of the more complex types of pattern

the pattern set is so large because of combinatorial explosion that the corresponding relationship cannot always be processed directly by the map/reduce.Therefore a way of pattern space is proposed to convert the problem of processing the many-to-many corresponding relationship between the date set and the pattern set to the problem of processing the many-to-many corresponding relationship between the data set and the set of the sub-pattern sets.At the same time

the scheduling mechanisms of the map/reduce cluster and the way of organizing the key/value pairs is improved to enhance the ability of map/reduce to execute pattern mining tasks.The results show that higher parallelism is achieved on map/reduce clusters by using this idea than the map/reduce of the traditional algorithm in mining some of the more complex types of pattern.

关键词

Keywords

references

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Fuzzy C-means Clustering Based on the Multivariable Weighted Attributes

Research on Massive Data Mining Method Based on Fuzzy Association Rules

An N-list based Parallel Algorithm for Mining Frequent Itemsets

Research on Fire Fighting Command Data Mining

Support Vector Machine Based on Boundary Vectors Adjustable Entropy Function

Related Author

LI Shi-jin

LI Qian

XU Gui-qiong

MAO Xiao-ju

CHEN Qi

ZHANG Xi-huang

SHI Wei-rong

LIN Wei

Related Institution

School of Management,Shanghai University

School of Computer Engineering, Shangqiu University

School of Internet Things Engineering Jiangnan University

Baoshan District Public Security Fire Brigade Headquarters

College of Science, Harbin Engineering University

AI问答

Postal code：100079
Tel：（010）53879206 Email：tmw@bjxintong.com.cn
Technical support is provided by Beijing Founder electronics co., LTD 京ICP备09082226号-64 京公网安备11010602201714号
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰