Recently spent a lot of time on distributed storage, do not want to spend a lot of time on this, so use this article to make a final summary.
In the face of distributed storage, it is divided into two angles, one is the customer side, the other is the operation and maintenance side, and the customer is God. Therefore, ignoring the operation of God, it focuses on the system construction on the operation and maintenance side.
In fact, all system constructions should be divided into two latitudes. One is the customer latitude, which focuses on the customer experience and carries out various customized output. One is the operation and maintenance latitude, focusing on the underlying operation and maintenance, various monitoring data, and various Operation, using a white screen operation, rather than command line operation every day, using the platform level, can prevent misuse, the system has lost most of the responsibility, but also can make the operation and maintenance do not have to remember those dumb commands every day, stupid parameters, reduce the low The level of operation, so that the brain has more room to think of other things. . . For example, look at the blue sky at dusk. . . .
The distributed storage system is to solve the bottleneck of capacity, performance, and availability, scalability, etc. of single-machine storage, and to store large amounts of data for large-scale storage applications by storing data on multiple storage devices. High-performance, highly available, and scalable storage services. This series of articles introduces the design and implementation of a typical distributed storage system. The system has already served a large number of services, reached a storage capacity of several hundred tons, and has undergone the test of massive services.
Overall structureThe overall architecture of the system is shown in the figure below, where the logical layer is the consumer of the storage service. The system consists of two parts. One part is the module contained in the data warehouse of the figure. It is the core part of providing data storage services directly. It is composed of the access layer, data layer, and configuration operation and maintenance center; the other part is an auxiliary system and is mainly responsible for the system. The monitoring, operation and maintenance and operation backup system, monitoring system, operation and maintenance management system, and user operation system.
A data warehouse is a storage cluster. Multiple services can share the resources of a data warehouse. We can deploy multiple data warehouses according to requirements. The auxiliary system is shared by all data warehouses.
The following briefly describes the main functions of each module.
Access layerThe access layer mainly provides two functions. One is to load balance the access layer of the logic layer; the other is to implement data fragmentation, that is, the data layer device where the data access request is forwarded.
Data layerThe data layer is storage data. The storage medium can support memory or SSD. The read-write service handles the read and write requests of users; the synchronization between the master and the backup of multiple copies of the synchronization module; the operation and maintenance tools are used to perform operations such as active/standby switchover, crash recovery, and capacity expansion.
Configure the operation and maintenance centerThe configuration operation and maintenance center consists of three parts. The configuration center is responsible for the configuration, maintenance, and delivery of the entire warehouse. The quota center is responsible for the management of quotas for resources, traffic, and CPU resources at various service levels. The operation and maintenance center is used to automatically or manually issue maintenance and maintenance commands.
Backup systemThe backup system is responsible for backing up, restoring, and restoring data for all services in the entire system. The stream center records all the write operations; the task center manages and schedules the execution of all data backup, restore, and restore tasks.
surveillance systemThe monitoring system reports and analyzes key information and operating conditions of the system, and monitors and alerts abnormal conditions. RBI reporting is to count or report critical path and abnormal points of the system; multidimensional reporting is a supplement to RBI reporting, and more dimensional information is reported.
Operation and maintenance management systemThe users of the O&M management system are the system operation and maintenance personnel, which can conveniently perform service management and operation and maintenance operations, such as common operations such as configuration management, fault management, and service expansion. The system operation status and service operation data can also be viewed.
User Operation SystemThe user of the user management system is a user who uses a storage service. Through the system, the user can grasp the operational data of the accessed service and perform user-level service management and operation and maintenance operations, such as renewals, capacity expansion, data clearing, and data backup. Data recovery and so on.
Plant Protection Machine Battery
Langrui Energy (Shenzhen) Co.,Ltd , https://www.langruibattery.com