LI De-you, ZHAO Li-bo, XIE Chen-guang. The Research of a Big Data Storage System Constructed with Hadoop[J]. 2015, 20(4): 60-065.
DOI:
LI De-you, ZHAO Li-bo, XIE Chen-guang. The Research of a Big Data Storage System Constructed with Hadoop[J]. 2015, 20(4): 60-065.DOI: 10.15938/j.jhust.2015.04.012.
The Research of a Big Data Storage System Constructed with Hadoop
Nowadays,the OLAP(Online Analytical Processing)technology of civil banks is not developedproperly according to the requirement of massive data processing. In view of this situation,a big data processingand store system base on Hadoop platform and relational database is proposed. The article introduced the implemen-tation of core technologies,such as the structure of the system,the design of file store layers,methods of datatransferring,algorithm of task scheduling etc. It is demonstrated by experiments that the system combines the meritsof Hadoop with the advantages of relational database,such as extendibility,effectively,high fault tolerance,andgood ability of transaction processing,Therefore,reliability and service ability of data have been enhanced,not on1y satisfies the real time business data processing,but also benefits data processing even the further requirements todata mining.Nowadays,the OLAP(Online Analytical Processing)technology of civil banks is not developedproperly according to the requirement of massive data processing. In view of this situation,a big data processingand store system base on Hadoop platform and relational database is proposed. The article introduced the implemen-tation of core technologies,such as the structure of the system,the design of file store layers,methods of datatransferring,algorithm of task scheduling etc. It is demonstrated by experiments that the system combines the meritsof Hadoop with the advantages of relational database,such as extendibility,effectively,high fault tolerance,andgood ability of transaction processing,Therefore,reliability and service ability of data have been enhanced,not on1y satisfies the real time business data processing,but also benefits data processing even the further requirements todata mining.