A Scheme for Massive Unstructured Iot Data Processing Based on MongoDB and Hadoop

YANG Peng; LIN Jun-hui

Chinese

您当前的位置：

首页 >

文章列表页 >

A Scheme for Massive Unstructured Iot Data Processing Based on MongoDB and Hadoop

更新时间：2026-09-04

- A Scheme for Massive Unstructured Iot Data Processing Based on MongoDB and Hadoop
- Vol. 35, Issue 4, Pages: 68-72(2018)
- 作者机构：
  
  1. 重庆邮电大学机构通信与信息工程学院,重庆,400065
  2. 中国信息通信研究院机构,北京,100191
  3. 重庆高校市级光通信与网络重点实验室,重庆,400065
- 作者简介：
- 基金信息：
- DOI：
  CLC：
- Published：2018
- 稿件说明：
移动端阅览
杨鹏, 林俊晖. 一种基于MongoDB和Hadoop的海量非结构化物联网数据处理方案[J]. 微电子学与计算机, 2018, 35(4): 68-72, 78.

YANG Peng, LIN Jun-hui. A Scheme for Massive Unstructured Iot Data Processing Based on MongoDB and Hadoop[J]. 2018, 35(4): 68-72.
杨鹏, 林俊晖. 一种基于MongoDB和Hadoop的海量非结构化物联网数据处理方案[J]. 微电子学与计算机, 2018, 35(4): 68-72, 78. DOI：

YANG Peng, LIN Jun-hui. A Scheme for Massive Unstructured Iot Data Processing Based on MongoDB and Hadoop[J]. 2018, 35(4): 68-72. DOI：

摘要

随着物联网数据种类的增多和数据规模的增大

对物联网数据的存储和计算提出了新的挑战

为了应对海量非结构化物联网数据的存储和计算要求

提出了一种NoSQL数据库技术与MapReduce编程框架相结合的方案.使用典型的NoSQL数据库MongoDB作为主数据库来存储海量非结构化的物联网数据

使用Hadoop MapReduce作为对物联网数据分析处理的计算框架.通过对MongoDB集群和Hadoop集群的重叠部署

降低了计算时数据传输的开销

构建了一套高可用、高性能的物联网大数据处理平台.通过使用该方案对海量非结构化物联网数据的处理分析实验验证了该方案的高可用性及高效性.

Abstract

As the IoT data type and data volume increase

a new challenge is proposed to the storage and calculation of IoT data.To meet the storage and calculation demands of the massive unstructured IoT data

a scheme that NoSQL database technology combined with MapReduce programming framework has been presented in this paper. Using the MongoDB

a typical database of NoSQL

as the main database to store massive unstructured IoT data. At the same time

the Hadoop MapReduce was adopted as the calculation framework for IoT data analysis and processing. Through the overlapping deployment of MongoDB cluster and Hadoop cluster

the cost of data transmission is reduced

and a set of high availability and high performance large data processing platform is established. The analysis of the experiment that using this scheme to deal with massive IoT data has verified the high efficiency and high efficiency of this method.

关键词

Keywords

references

苏婵. 非关系型数据库及在文档库管理平台中的研究与应用[D]. 南昌: 南昌大学, 2014.

李秋虹. 基于MapReduce的大规模数据挖掘技术研究[D]. 上海: 复旦大学, 2013.

安永辉. 基于MapReduce的文本处理算法的研究与应用[D]. 桂林: 桂林电子科技大学, 2013.

辛晓越. 文档型数据库的存储模型设计和研究[D]. 广州: 中山大学, 2015.

曾强, 缪力, 秦拯. 面向大数据处理的Hadoop与MongoDB整合技术研究[J]. 计算机应用与软件, 2016, 33(2): 21-24. DOI: 21

李伟. 大数据下的NoSQL数据库技术分析[J]. 信息通信, 2014, 28(7): 72-73. DOI: 72

何倩, 陈亦婷, 董庆贺, 等. 基于MongoDB的物联网接入云服务平台[J]. 桂林电子科技大学学报, 2017, 37(1): 1-7. DOI: 1

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

High gain stability high temperature operational amplifier based on reverse body bias

Implementation and verification of a dual-chain power gating strategy for low power design

A feature-oriented context understanding and slice analysis network for tuberculosis detection

Design of ultra-low power real-time clock circuits

Steganography model guided by attention mechanism and improved dense convolution

Related Author

No data

Related Institution

No data

AI问答

Postal code：100079
Tel：（010）53879206 Email：tmw@bjxintong.com.cn
Technical support is provided by Beijing Founder electronics co., LTD 京ICP备09082226号-64 京公网安备11010602201714号
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰