1. Persistence
2. Copy
3. Blockage situation
4. Memory management
V. Redis Cluster
5.1. Data distribution theory
5.2. Redis data partition
5.3. Communication process
5.3.1. Gossip message
5.3.2. Node selection
5.4. Request a routing
5.4.1. Calculation slot
5.4.2. Slot node search
5.5. Smart Client
5.5.1. Smart client principle
5.5.2. Smart client—JedisCluster
5.6. ASK redirection
5.6.1. Client ASK redirection process
5.7. Fault detection
5.7.1. Subjectively offline
5.7.2. Objectively offline
5.8. Summary
1. Persistence
1） RedisTwo persistence methods are provided: RDB and AOF.
2) RDB is generated at one timeMemoryIn the snapshot method, the resulting file compact compression ratio is higher, so the RDB recovery speed is faster. Since RDB generation is expensive every time, real-time persistence cannot be achieved, it is generally used for data cold backup and copy transmission.
3) The save command will block the main thread. It is not recommended to use it. The bgsave command creates a child process through fork operations to generate RDB to avoid blocking.
4) AOF achieves persistence by appending write commands to the file, and real-time/second persistence can be controlled through the appendfsync parameter. Because the write commands are required to be added continuously, the AOF file size gradually becomes larger, and it is necessary to perform rewrite operations regularly to reduce the file size.
5) AOF rewrite can be passedautoThe -aof-rewrite-min-size and auto-aof-rewritepercentage parameters control automatic triggering, and can also be triggered manually using the bgrewriteaof command.
6) During the execution of the child process, use the copy-on-write mechanism to share memory with the parent process to avoid double memory consumption. During AOF rewriting, it is also necessary to maintain the rewrite buffer to save new write commands to avoid data loss.
7) The persistent blocking main thread scenarios include: fork blocking and AOF append blocking. Fork blocking time is related to the amount of memory and system. AOF additional blocking indicates that hard disk resources are tight.
8) When deploying multiple instances on a stand-alone machine, in order to prevent multiple subprocesses from performing rewrite operations, it is recommended to perform isolation control to avoid competition between CPU and IO resources.
2. Copy
1) Redis realizes multiple replicas of the master node through replication function. The slave node can flexibly establish or disconnect the replication process through the slaveof command.
2) Replication supports a tree structure, and the slave node can replicate another slave node to realize layer by layer downward replication flow. The replication process after Redis2.8 is divided into: full replication and partial replication. Full replication requires synchronization of all master nodesDataset, consumes a lot of machine and network resources. Partial replication effectively reduces unnecessary full replication caused by network abnormalities and other reasons. Try to avoid full replication by configuring a reasonable replication backlog buffer.
3) Maintain a heartbeat and offset check mechanism between master and slave nodes to ensure that the communication between master and slave nodes is normal and the data is consistent.
4) Redis to ensure highperformanceThe replication process is asynchronous. After the write command is processed, it is returned directly to the client without waiting for the replication of the slave node to complete. Therefore, there will be delays in slave node datasets.
5) When using slave nodes for read and write separation, there will be problems such as data delay, expired data, slave node availability, etc., and it is necessary to avoid it in advance based on your own business.
6) During the operation and maintenance process, if there are multiple slave nodes or a large number of master nodes are deployed on one machine, there is a risk of a replication storm.
3. Blockage situation
1) The client first perceives the Redis timeout behavior such as blocking. Adding log monitoring and alarm tools can quickly locate blocking problems, and at the same time, it is necessary to conduct comprehensive monitoring of the Redis process and machine.
2) The inherent reasons for blocking: Confirm whether there is blockage in the main thread, check slow query and other information, and find unreasonable use of API or data structures, such as keys, sort, hgetall, etc. Pay attention to CPU usage to prevent single core from running full. When the hard disk IO resources are tight, AOF appends will also block the main thread.
3) External reasons for blocking: Start with CPU competition, memory switching, network problems, etc. to check whether blocking is caused by system-level problems.
Four,Memory management
1) Redis's actual memory consumption mainly includes: key-value objects, buffer memory, and memory fragmentation.
2) Control the maximum available memory of Redis by adjusting maxmemory. When memory usage is exceeded, the memory recycling policy is controlled according to maxmemory-policy.
3) Memory is a relatively valuable resource. Through reasonable optimization, the memory usage can be effectively reduced.Memory optimizationThe ideas include:
- Simplify key-value pair sizes, literally streamlined key-value literals, using efficient binary serialization tools.
- Optimize small integer objects using object sharing pool.
- Data uses integers first, which saves more space than string types.
- Optimize string usage to avoid memory waste caused by pre-allocation.
- Use ziplist compression coding to optimize structures such as hash and list, focusing on efficiency and space balance.
- Optimize sets of integers using intset encoding.
- Use the hash structure encoded by ziplist to reduce the size of small object chains.