Research on Efficient Data Filtering Based on FPGA

CHEN Bao-yuan; ZHANG Xiu-zhi; LIANG Zhuang

Chinese

您当前的位置：

首页 >

文章列表页 >

Research on Efficient Data Filtering Based on FPGA

更新时间：2026-09-04

- Research on Efficient Data Filtering Based on FPGA
- Vol. 34, Issue 12, Pages: 130-133(2017)
- 作者机构：
  
  哈尔滨理工大学测控技术与仪器黑龙江省高校重点实验室机构,黑龙江,哈尔滨,150080
- 作者简介：
- 基金信息：
- DOI：
  CLC：
- Published：2017
- 稿件说明：
移动端阅览
陈宝远, 张秀芝, 梁状. 基于FPGA的高效数据过滤技术研究[J]. 微电子学与计算机, 2017, 34(12): 130-133, 137.

CHEN Bao-yuan, ZHANG Xiu-zhi, LIANG Zhuang. Research on Efficient Data Filtering Based on FPGA[J]. 2017, 34(12): 130-133.
陈宝远, 张秀芝, 梁状. 基于FPGA的高效数据过滤技术研究[J]. 微电子学与计算机, 2017, 34(12): 130-133, 137. DOI：

CHEN Bao-yuan, ZHANG Xiu-zhi, LIANG Zhuang. Research on Efficient Data Filtering Based on FPGA[J]. 2017, 34(12): 130-133. DOI：

摘要

随着数据量的爆炸式增长

为了挖掘海量数据中蕴含的巨大价值

如何在有效的时间内对海量的数据进行处理变得越来越重要.近年来出现的Hadoop凭借其高可靠性、高扩展性以及MapReduce编程模型获得了产业界和学术界的巨大关注.但是Hadoop设计之初是用来处理大规模聚集任务的

这类任务往往需要扫描全部的数据

这种基于扫描的数据处理方式效率比较低.为了提高Hadoop的数据处理效率

在Hadoop集群中加入FPGA对Hadoop分布式文件系统中存储的数据进行过滤

这样MapReduce程序只需要对过滤后的数据进行自定义的操作

从而避免了扫描全部数据.实验表明处理海量的数据时

FPGA能够显著地过滤掉无用数据

从而提高数据处理效率.

Abstract

With the explosive growth of data volume

how to process the massive data within a reasonable time is becoming more and more important. Recently

Hadoop has got a great attention in academia and industry with high reliability

high expansibility and MapReduce programing model. While Hadoop was designed to handle large scale gathering mission

and this kind of mission need to scan the whole data-sets.In order to increase the efficiency of data processing

FPGA was injected into Hadoop cluster. FPGA will filter the data stored in HDFS(Hadoop Distributed File System). In this way MapRedeuce just need to process the filtered data. A series experiment shows that FPGA could filter useless data and increase the efficiency of data processing.

关键词

Keywords

references

Becher A, Bauer F, Ziener D. Energy-aware SQL query acceleration through FPGA-Based synamic partial reconfiguration[C]// International Conference on Field Programmable Logic % Applications. IEEE, 2014: 1-8.

Belean B, Borda M, Bot A. FPGA based jardware architectures for interatIve algorithms implementations. [C]// International Conference on Telecommunications % Signal Processiong. IEEE, 2013: 751-754.

Mueller R, Teubner J, Alonso G. Glacier: A query-to-hardware compiler[C]//Proc. of the 2010 ACM SIGMOD Int'1 conf. ACM, 2010: 1159-1162.

Zarifi T, Malek M. FPGA Implementation of image processing technique for blood samples characterization[J]. Computers and Electrical Engineering, 2014, 40(5): 1750-1757. DOI: 1750

Sukhwani, M Hong, M Thoennes. Database analytics acceleration using FPGAs[C]// International Conference on Parallel Architectures % Complation Techniques. IEEE, 2012: 411-420.

Garcia H, Ludu A. The Google file system[C]//ACM sigops Operating System Review. ACM, 2003: 29-43.

YM Choi, KH So. Map-Reduce processing of K-means algorithm with FPGA-acceleratedcomputer cluster[C]// IEEE International Conference on Application-specific System, Architectures % Processors. IEEE, 2014: 9-16.

Becher A, Bauer F, Ziener D, et al. Energy-Aware SQL query acceleration through FPGA-based dynamic partial reconfiguration[C]// FPL. IEEE, 2014: 1-8.

White T. Hadoop: The definitive guide, [M]. O'reilly Media, 2015: 10-19.

Sukhwani B, Hong M, Thoennes M. Database analytics acceleration using FPGAs[C]//International Conference on Parallel Architectures %Complilation Techniques. IEEE, 2012: 411-420.

Chenini H, JP D, Chapuis R. Parallel embedded processor architecture for FPGA-based image processing using parallel software skeletons[J]. Journal on Advances in Signal Processing, 2013(1): 1-23. DOI: 1

马友忠, 孟小峰. 云数据管理索引技术研究[J]. 软件学报, 2015, 26(1): 145-166. DOI: 145

江务学, 张璟, 王志明. MapReduce并行编程架构模型研究[J]. 微电子学与计算机, 2011, 28(6): 168-170. DOI: 168

严宵凤, 张德馨. 大数据研究[J]. 计算机技术与发展, 2013, 23(4): 168-172. DOI: 168

钱松, 周钦, 俞军. AES算法的一种高效FPGA实现方法[J]. 微电子学与计算机, 2005, 22(7): 89-91. DOI: 89

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Design of a shared-memory-based spaceborne data switching architecture

The linear leastsquares positioning algorithm based on FPGA

Algorithm optimization and hardware acceleration for YOLO post processing

Optimization and implementation of FPGA-based H.264 variable block motion estimation algorithm

A secure encryption authentication system based on integrated structure of TRNG and PUF

Related Author

WANG Runze

TANG Lei

KUANG Nailiang

LIANG Yong

FENG Zhouwu

CUI Mohan

CHEN Tao

ZHU Yifei

Related Institution

Xi'an Microelectronics Technology Institute

School of Electronics and Communication Engineering, East China Normal University

School of Microelectronics, University of Science and Technology of China

Xi'an Institute of Microelectronics Technology

School of Physics and Electronic Engineering, Shanxi University

AI问答

Postal code：100079
Tel：（010）53879206 Email：tmw@bjxintong.com.cn
Technical support is provided by Beijing Founder electronics co., LTD 京ICP备09082226号-64 京公网安备11010602201714号
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰