Redisson & RedLock distributed lock

Posted by millwardt on Thu, 20 Jan 2022 20:37:22 +0100

preface

This article aims to share experience and welcome your comments

Application scenario:

1. Second kill
2. Grab coupons
3. Interface idempotency check

Summary: grab resources

1. Single lock -- > to distributed lock

synchronized () is no longer described here

1.1. Based on opsforvalue() Setifabsent() implementation

Format: setnx key value
Set the value of the key to value if and only if the key does not exist
If the given key already exists, setnx will not do anything
setnx is the abbreviation of [set if not exists] (if it does not exist, set)
Available version > = 1.0.0

String lockKey = "lockKey";
Boolean result = redisTemplate. opsForValue().setIfAbsent(lockKey, "hello");
if(!result){
return "something exception";
}
// A large number of business codes
...
redisTemplate. delete(lockKey);

return "program End ...";

If the business code throws an exception, can it still delete?
If you can't delete it, it will deadlock

1.1.1 improvement: add try... finally In any case, it will eventually delete

try{
    String lockKey = "lockKey";
    Boolean result = redisTemplate. opsForValue().setIfAbsent(lockKey, "hello");
    if(!result){
    return "something exception";
    }
    // A large number of business codes
    ...
    }
finally{ 
    redisTemplate. delete(lockKey);
}
return "program End ...";

Continue to look at the existing problems?
The problem of releasing the lock is solved, but what if the execution of the business code goes down? finally, it's useless, and there will be deadlock again. Because the key in redis always exists.

1.1.2 improvement: set the expiration time of the key

try{
    String lockKey = "lockKey";
    Boolean result = redisTemplate. opsForValue().setIfAbsent(lockKey, "hello");
    // Add timeout
    redisTemplate.expire(lockKey,10,TimeUnit.SECONDS);
    if(!result){
    return "something exception";
    }
    // A large number of business codes
    ...
    }
finally{ 
    redisTemplate. delete(lockKey);
}
return "program End ...";

But if we write this, is it baa?

Boolean result = redisTemplate. opsForValue().setIfAbsent(lockKey, "hello");
    // Problems that will arise
redisTemplate.expire(lockKey,10,TimeUnit.SECONDS);

What if the key has just executed setIfAbsent() and the program hangs up before it reaches expire()?? These two lines of code have atomic problems

1.1.3 improve and solve the problem of atomicity

Don't mention that these two are integrated in redis. At the bottom, redis will help us ensure atomicity

// Boolean result = redisTemplate. opsForValue().setIfAbsent(lockKey, "hello");
// redisTemplate.expire(lockKey,10,TimeUnit.SECONDS);
redisTemplate. opsForValue().setIfAbsent(lockKey, "hello",10,TimeUnit.SECONDS);

What is the problem with this writing??

If you put it in the ultra-high concurrency scenario, it will be terrible. It's not a small problem.
Suppose that a request is executed for 15 seconds, and the lock is cleared and adjusted in 10 seconds, and new requests come in again and again

What are the consequences of such a vicious circle?? High concurrency, requests come in constantly What problems will this cause? Maybe this lock will fail forever!

1.1.4 how to solve the problem of high concurrency??

The essence is that my own lock will be deleted by other threads, right.

The next thing we can think of is to add a unique sign to the lock. Add the id to the lock.

String uniqueId = UUID.randomUUID.toString() + threadId;

When we want to unlock, judge whether the id of the lock is added by me

String uniqueId = UUID.randomUUID.toString() + threadId;
try{
    String lockKey = "lockKey";
    Boolean result = redisTemplate.opsForValue().setIfAbsent(lockKey,uniqueId, "hello",10,TimeUnit.SECONDS);
    
    if(!result){
    return "something exception";
    }
    // A large number of business codes
    ...
    }
finally{ 
    if(uniqueId.equals(redisTemplate.opsForValue().get(lockKey))){
    	redisTemplate. delete(lockKey);
    }
}
return "program End ...";

When the execution is here, there is still a problem. If the program is executed for 15 seconds, but the lock is released for 10 seconds. There will still be concurrency problems.

Time is not the solution to the problem. Is there any other idea?
There is indeed an implementation method for redis processing method: in this current situation, the timer is used to scan, and the threads are scanned every 10 seconds to judge whether the main thread holds the lock and whether the key still exists. If so, set the timeout longer, which is similar to the token renewal period.

2,Redisson

Now there are many open source frameworks on the market to help us realize these problems. And they've sealed it.

<dependency>
    <groupId>org.redisson</groupId>
    <artifactId>redisson</artifactId>
    <version>3.6.5</version>
</dependency>

// Initialize client
@SpringBootApplication
public class Application {

    public static void main(String[] args) {
        SpringApplication.run(Application.class, args);
    }
    
    // Inject into Spring
    @Bean
    public Redisson redisson(){
        // This is stand-alone mode
        Config  config = new Config();
        // Of course, there are many modes to choose from, such as master-slave, cluster, replication, sentry and so on
        config.useSingleServer().setAddress("redis://localhost:6379").setDatabase(0);
        return (Redisson) Redisson.create(config);    
    }
}

// Business end
@Autowire
private Redisson redisson;

RLock redissonLock = redisson.getLock(lockkey);
try{
    // Perform locking operation
    redissonLock.lock()  // This line of code can be regarded as setIfAbsent(lockKey,uniqueId, "hello",10,TimeUnit.SECONDS);
    // Business code
    ... ...
    }
finally{
    redissonLock.unlock();
}

The following is the underlying locking logic of the source code: lua script implementation

Will there be no problem with Redisson??

If redis is a master-slave and sentinel, there is still a problem

The master node writes the key asynchronously, and then synchronizes the data to the slave. As long as the master node writes the key successfully, it immediately returns to the client to tell the client that the lock is successfully added, the key is successfully set, and the client thread 1 starts to do business logic. At this time, the redis master node will synchronize with the slave node.

That is to say: after synchronization

Get the lock on the master node of Redis;
However, the locked key has not been synchronized to the slave node;
When the master fails, failover occurs, and the slave node is upgraded to the master node;
The lock is lost.

Suppose that he is just about to synchronize to the slave node, and then the master node hangs up. What should I do? Suppose that the slave node to be synchronized is elected as the master, and a new thread (thread 3) comes in in in the high concurrency scenario. When he requests, he finds that he has no key for a new master. How to solve this problem?

3. Redis master-slave architecture lock failure?

In fact, there are ways to solve the relative problems with redis. However, we have reached the extreme of redis. At this time, we can change our thinking. For example, we can use zk. It is also a key value structure, but it is a tree. CP is satisfied in the CAP. He will not immediately tell you success (C: indicates consistency) and sacrifice time for synchronization. So there is no such problem.

How to select redis or zookeeper? Do you want to select redis (a single machine can support 10W QPS) for high concurrency requirements? If the lock fails, you can only compensate the data manually, and the probability of error is relatively small.

4. RedLock implementation

redis officially recommends using the underlying implementation method, which is very similar to zk

4.1 introduction to RedLock

When different processes need to access shared resources mutually exclusive, distributed locking is a very useful technical means. There are three attributes to consider when implementing efficient distributed locks:

Security attribute: mutually exclusive. At any time, only one client holds the lock
Efficiency attribute A: no deadlock
Efficiency attribute B: fault tolerance. As long as most redis nodes can work normally, the client can obtain and release locks.

Redlock is an algorithm officially proposed by redis to implement distributed lock manager. This algorithm will be more secure and reliable than ordinary methods. The discussion of this algorithm can be seen below Official documents.

More than half of redis nodes (independent nodes without any dependencies) are locked successfully

4.2. What are the disadvantages of RedLock?

Performance. Originally, as long as the master node is written successfully, many machines need to be locked successfully.
Can not 100% solve the problem of lock failure

For detailed implementation, please refer to Fang Zhipeng's good article: Implementation of RedLock

Topics: Java Redis Distributed lock redisson

Programmer Think