Building cloud storage platform based on open source software
摘要
为了解决在使用开源软件构建云存储平台的过程中
可能会面临的各种实际问题
比如存储集群的高可用、超大规模节点集群部署和维护、跨多数据中心解决方案等等。我们采用了诸多技术手段
如增加前端数据缓存、预备份数据来提高处理效率;通过增强数据冗余检查、数据的一致性检测保障数据安全
加强了健康检查和告警模块、自动化部署、优化通知和轮询机制
增强了网络监测技术来实现便捷的运维管理。我们通过将所有存储组件都多活热备
各种相同功能的组件高可用分布到不同机架
来达到提高产品可用性的目的。
Abstract
For a long time
the core technology of storage equipment
such as SAN and NAS
belongs to some large enterprises abroad
in order to break this situation
and get involved core storage technology
our team from China Mobile conducted in-depth research and extensive practice. Use open source software
such as Ceph
as a technical basis
engineers can be more convenient and quick to cut into the storage areas
which technology is believed not easy to master. This article is to solve the practical problems that may be faced in the process of building a cloud storage platform using open source software
for example
high availability of storage clusters
deployment and maintenance of ultra-large-scale nodes
cross-data center solutions
and so on. We have adopted a number of technical means
such as adding front-end data cache
pre-backup data to improve processing efficiency; by enhancing data redundancy checks
data consistency detection to protect our data
enhancing health check and alarm module
automated deployment
optimizing the notification and polling mechanism
and enhancing the network monitoring technology to achieve convenient operation and maintenance management. We are able to achieve the goal of improving product availability by distributing all the storage components in a variety of racks of the same function.