Table of contents
- Preface
- What is it
- What are the storage structures
- 3. Why use redis and redis so fast
- 4. Cache avalanche, cache penetration, cache breakdown
- The lasting mechanism
- Expiration strategy
- Elimination strategy
- How to set up high availability or cluster
- Implement distributed locking
- 10. Characteristics of distributed locks
- Application scenarios
Preface
Let’s not talk about the many, because it’s an interview question, just memorize everything, and it’s OK to ensure that you’re happy for two and a half years.
The latest java interview questions (java basics, collections, multithreading, jvm, locks, algorithms, CAS, Redis, databases, mybatis, spring, springMVC, springBoot, microservices)
What is it
It is a high-performance non-relational database developed by the C language.
What are the storage structures
String:Key-value pairs. Application scenarios: cache, SMS verification code, etc.
set key value //Set the value
get key //Get the value
Hash:Key-value pairs can store multiple key-value pairs, suitable for storing objects. Application scenarios: cache, etc., which saves more space than String.
hset key field value //Set the value
hget key field //Get the value
List:Equivalent to linked list. Application scenarios: message queue, etc.
LPUSH key element1 element2 element3 ... //Insert element on the left, insert the header method
LPOP key //Remove and return the first element on the left side of the list, if not, return nil
Set:HashSet is similar, disordered and cannot be repeated. Application scenario: common friends, etc.
SADD key member1 member2 member3 ... //Add one or more elements to set
SREM key member1 //Remove the specified element in set
Zset:Similar to TreeSet, it is not repeatable, and there is a weight parameter, sorted by this. Application scenario: ranking.
ZADD key score1 member1 score2 member2 score3 member3 //Add one or more elements to sorted set, and update its score value if it already exists
ZREM key member1 //Delete a specified element in the sorted set
3. Why use redis and redis so fast
① Memory operation, fast speed. Read 110,000 times per second and write 80,000 times.
② Support transactions and persistence.
③Single threading avoids thread switching. redis 6 introduces multi-threading.
④ IO multiplexing is adopted, and a single thread handles multiple connection requests.
4. Cache avalanche, cache penetration, cache breakdown
Cache avalanche:A large amount of data in the cache expires, and then a large number of requests are requested to access the database, resulting in excessive database pressure or down machine. Or redis is down.
solve:
① Make the cache expiration time distribution relatively evenly.
② Set up a high-availability cluster.
Cache penetration:Neither the cache nor the database needs the data, and then it will keep requesting, putting pressure on the database.
solve:
① Set a null value or default value to the cache.
② Use a Bloom filter. First, judge whether the data exists through the filter, and then continue to check down.
Cache breakdown:A certain key expires and happens to have a large number of requests to access the key, which will cause excessive pressure on the database.
solve:
① Use a mutex lock, only one request is accessed, then bring the data to the cache, and then the remaining requests are accessed.
②Set cache expires.
The lasting mechanism
RDB (default):Within a certain period of time, the memory data is saved to the hard disk in the form of a snapshot (recording data at a certain moment, which is equivalent to taking a photo). Write the data to a temporary file at a certain point in time and replace the last persisted file.
# After 10 seconds, if at least 1 key changes, Redis will automatically create a snapshot
save 10 1
advantage:Recovering large datasets is more efficient than AOF.
shortcoming:Insecure, data loss.
AOF:The commands written each time will be recorded to the log file (the same log file will not be replaced), and the redis restart will restore the persisted log file. If both persistences are enabled, priority will be given to restore AOF.
# Turn on AOF persistence
appendonly yes
# Every time a data modification occurs, AOF file is written, which is slow but the safest
appendfsync always
# Synchronize once every second, synchronize multiple write commands to the hard disk displayed. AOF is used by default
appendfsync everysec
# Let the operating system decide when to synchronize, the fastest speed
appendfsync no
advantage:Safe, almost no data loss.
shortcoming:AOF files are larger than RDB files and are slow to recover.
Expiration strategy
Timely deletion:Each key needs to create a timer, and the key will be cleared at the time, so it is very memory friendly, but it will occupy a large amount of CPU resources to handle expired.
Lazy deletion:When using the key, you can determine whether it has expired or not. Clear it if it expires. This can save CPU resources, but it is not memory friendly. There may be a large number of expired keys that are not cleared.
Regularly deleted:A combination of timed deletion and lazy deletion, and every period of time, set the expired key detection whether it expires (default is to perform 10 clearings in 1s, and 5 detections are extracted each time), and clear it if it expires.
# 10 checks in 1 second
hz 10
# The number of draws at a time is 5 by default
maxmemory-samples 5
Elimination strategy
Redis memory is full, memory elimination is performed and some keys are deleted. Redis 4.0 added lfu policy.
# The default maximum memory setting is 1GB
maxmemory 1GB
①volatile-lru:For keys whose expiration time are set, the lru algorithm (the least used key is based on time) is used for elimination.
②allkeys-lru:Use the lru algorithm for all keys to be eliminated.
③volatile-lfu:For keys whose expiration time are set, the lfu algorithm (most recently used: according to counter) is used for elimination.
④allkeys-lfu:Use the lfu algorithm for all keys to be eliminated.
⑤volatile-random:Use the random elimination mechanism to eliminate from all keys with expiration time set.
⑥allkeys-random:Use a random elimination mechanism for all keys to be eliminated.
⑦volatile-ttl:Delete the earliest expiration time (the shortest remaining survival time).
⑧no-eviction (default):If the key is not deleted, an error will be reported.
How to set up high availability or cluster
①Master-slave copy:A master, one or more slaves, and slave nodes copy data at the master node. The master node is responsible for writing, and the slave node is responsible for reading. It can better share the pressure of the master node, but if the master node goes down, some data will be out of sync.
② Sentinel mode (key point):It is also a master-slave mode. Sentinels regularly query the host. If the host does not correspond for too long, multiple sentries will vote for a new host. Improved availability, but still fails to work during the election of a new master node.
③Cluster cluster mode:Multi-master and multiple slaves (usually three masters and three slaves) are used to shard according to the rules. The data stored by each redis node is different, which solves the problem of stand-alone storage. Replication and failover are also provided. Configuration is more troublesome.
Implement distributed locking
Distributed lock:It is an implementation of a lock that controls different processes of distributed systems to access shared resources. Distributed locks are required for business scenarios such as instant orders and red envelope grabbing. Redis can be used as a distributed lock.
①Write the command setnx + expire separately.
setnx key value //Add a new key, the key will return 1, the key will not be added, the key will return 0 if it does not exist,
expire key 100 //The key expires after 100 seconds. Unit is secondsProblem: The atomicity cannot be guaranteed. After the setnx is locked, then the problem occurs. The expiration time is not set, so the lock is permanent.(Other threads will never get the lock)。
②setnx + value is the expiration time.
setnx key expiration time//Put the expiration time to value to solve the problem of not setting the expiration time. By comparing the expiration time with the current time, you can know whether it has expired.Problem: Take the current time, and the distributed method requires time synchronization.
③Use Lua scripts (including two instructions for SETNX + EXPIRE: Ensure atomicity).
④ Set extension command (SET key value[EX seconds][PX millionseconds][NX|XX]).
EX seconds:Set the expiration time of the key to seconds.PX milliseconds:Set the expiration time of the key to milliseconds milliseconds.NX:The key is set only when the key does not exist.SET key value NXEquivalent toSETNX key value
XX:Only set the key when it already exists
set key value ex10 nx //The key does not exist before it can be added, and the expiration time is set to 10s.question:(1)The lock has expired and the service has not been completed. aAcquisition lock, execution time exceeds10Seconds, then the lock is acquired by the b thread, resulting in inconsistent code execution order.(2)The lock was deleted by mistake because(1),a has not finished executing the lock, and is acquired by b, then a has finished executing it, deletes the lock, and then b may not have finished executing it.
⑤set ex px nx + check the unique random value and delete it.
Set the value to a random number, compare the random number when deleting, and then delete it consistently.
Problem: There is still a lock released if the service has not been completed.
⑥Open source framework ~Redisson.
There is still a problem with the problem of service not being completed and lock release is not released.Redisson: To the thread that obtains the lock, open a timed daemon thread, check whether the lock still exists every once in a while. If it exists, the expiration time of the lock will be extended to prevent the lock from being released in advance.
⑦ Distributed lock Redlock implemented by multiple machines. Suppose there are 5 master nodes and run on five servers.
In order5A master node requests lock
Determine whether the master node should be skipped based on the set timeout time. The lock expiration time is10ms, the timeout is20ms, then skip.
If greater than or equal to3One node(N/2+1)If the lock is successfully added and the time spent is less than the validity period of the lock, it is confirmed that the lock is successfully added.
If the lock is acquired, all master nodes will be unlocked, and if the lock is not acquired, it will be unlocked to prevent nine years of compulsory education from missing the net.
Detailed explanation link
10. Characteristics of distributed locks
Mutual Exclusion:At any time, only one client can hold the lock
Lock timeout release:Holding the lock timeout can be released to prevent unnecessary waste of resources and also to prevent deadlocks.
Reentrability:If a thread acquires the lock, it can request a lock again.
High performance and high availability:Locking and unlocking requires as low overhead as possible, and it also ensures high availability to avoid distributed lock failure
Security:The lock can only be deleted by the held client and cannot be deleted by other clients.
Application scenarios
①Cache
② Database
③ Ranking list
④ Counter
⑤ Message Queue
⑥Distributed lock
⑦Share Session