王洪彬, 王纯. Design and implementation of monitoring system for the ETL workflows in heterogeneous scheduling environment[J]. 2016, 29(12): 61-65. DOI: 10.13992/j.cnki.tetas.2016.12.015.
Design and implementation of monitoring system for the ETL workflows in heterogeneous scheduling environment
摘要
本文提出一种分布式ETL在异构调度环境下的工作流统一监控方法。通过对异构调度系统元数据的提取和抽象
采用增量抽取和跟踪监控对象的方式
通过网络消息传送将监控对象加载到监控数据中心并统一管理
实现了分布式ETL工作流在异构调度环境下的的全局监控
提高了数据恢复能力和仓库的数据质量。
Abstract
This paper provided a unified monitoring method of the distributed ETL workflows in heterogeneous environment. By extracting and abstracting the metadata of each workflow and tracing the monitored objects incrementally
all the ETL jobs could be monitored and managed by the monitor data center
meanwhile improving the quality of the data warehouse.