scene
The company has a piece of business that needs to run the timer business to process data. Background uses springBoot to develop, springBoot integration timer task is very simple, use Scheduled is configured to use cron, and a business is executed 45 minutes per hour, because the background is cluster deployed, and the front end is responsible for balancing with Nginx, which involves a problem. At the same time, the background timer task can only be triggered with one trigger, the redis that was initially usedThere are too many articles about distributed locks based on redis on the Internet. This is not discussed here.
The initial code to implement a distributed lock based on Jedis is as follows
package com.juyi.camera.cache; import lombok.extern.slf4j.Slf4j; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.data.redis.core.RedisCallback; import org.springframework.data.redis.core.RedisTemplate; import org.springframework.stereotype.Component; import redis.clients.jedis.Jedis; import java.util.Collections; /** * Created by IntelliJ IDEA. * User: xuzhou * Date: 2019/2/26 * Time: 15:34 */ @Component @Slf4j public class RedisLock { @Autowired private RedisTemplate redisTemplate; private static final Long RELEASE_SUCCESS = 1L; private static final String LOCK_SUCCESS = "OK"; private static final String SET_IF_NOT_EXIST = "NX"; private static final String SET_WITH_EXPIRE_TIME = "EX"; private static final String RELEASE_LOCK_SCRIPT = "if redis.call('get', KEYS[1]) == ARGV[1] then return redis.call('del', KEYS[1]) else return 0 end"; /** * This locking method achieves distributed locking only for single-instance Redis * Not available for Redis clusters * <p> * Duplicate support, thread-safe * * @param lockKey Lock key * @param clientId Locking Client Unique Identification * @param seconds Lock expiration time * @return */ public Boolean tryLock(String lockKey, String clientId, long seconds) { return (Boolean) redisTemplate.execute((RedisCallback<Boolean>) redisConnection -> { Jedis jedis = (Jedis) redisConnection.getNativeConnection(); String result = jedis.set(lockKey, clientId, SET_IF_NOT_EXIST, SET_WITH_EXPIRE_TIME, seconds); redisConnection.close(); if (LOCK_SUCCESS.equals(result)) { return Boolean.TRUE; } return Boolean.FALSE; }); } /** * Corresponds to tryLock and is used as a release lock * * @param lockKey * @param clientId * @return */ public Boolean releaseLock(String lockKey, String clientId) { return (Boolean) redisTemplate.execute((RedisCallback<Boolean>) redisConnection -> { Jedis jedis = (Jedis) redisConnection.getNativeConnection(); Object result = jedis.eval(RELEASE_LOCK_SCRIPT, Collections.singletonList(lockKey), Collections.singletonList(clientId)); redisConnection.close(); if (RELEASE_SUCCESS.equals(result)) { return Boolean.TRUE; } return Boolean.FALSE; }); } }
The lock tool class in the code, which implements the atomicity of locking by itself using a lua script, is not covered much here
The specific business logic code is roughly as follows
@Scheduled(cron = "0 45 * ? * *") public void deleteCloudStorage() { //Adding redis locks to avoid duplicate cluster execution String CLOUD_STORAGE_EXPIRED_DELETE_LOACK_KEY = "storage_expired_delete_loack_"; String clientId = String.valueOf(IdWorker.nextId()); if (redisLock.tryLock(CLOUD_STORAGE_EXPIRED_DELETE_LOACK_KEY, clientId, 60)) { //Processing specific amount business logic } }
Reform
Later, when github was roaming around, he found that java also had a powerful redis client, called redisson. Overall, you can see that the document is completely written here [ https://github.com/redisson/redisson/wiki/Redisson%E9%A1%B9%E7%9B%AE%E4%BB%8B%E7%BB%8D] redisson uses the NIO-based Netty Framework with its own distributed lock implementation, allowing users to write more elegant code without having to worry about the atomicity of locking and unlocking.
A detailed description of distributed locking can be found here [ https://github.com/redisson/redisson/wiki/8.-%E5%88%86%E5%B8%83%E5%BC%8F%E9%94%81%E5%92%8C%E5%90%8C%E6%AD%A5%E5%99%A8] Based on this, bold attempts have been made to transform The code is roughly as follows, with the spring configuration, managed bean s
package com.juyi.camera.config; import com.fasterxml.jackson.annotation.JsonAutoDetect; import com.fasterxml.jackson.annotation.PropertyAccessor; import com.fasterxml.jackson.databind.ObjectMapper; import org.redisson.Redisson; import org.redisson.api.RedissonClient; import org.redisson.config.Config; import org.redisson.spring.data.connection.RedissonConnectionFactory; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; import org.springframework.core.io.ClassPathResource; import org.springframework.data.redis.connection.RedisConnectionFactory; import org.springframework.data.redis.core.RedisTemplate; import org.springframework.data.redis.core.StringRedisTemplate; import org.springframework.data.redis.serializer.Jackson2JsonRedisSerializer; import java.io.IOException; /** * Created by IntelliJ IDEA. * User: xuzhou * Date: 2020/5/14 * Time: 18:03 */ @Configuration public class RedissonConfig { @Bean public RedissonClient redisson() throws IOException { Config config = Config.fromYAML(new ClassPathResource("sentinelServersConfig.yml").getInputStream()); RedissonClient redisson = Redisson.create(config); return redisson; } @Bean public RedissonConnectionFactory redissonConnectionFactory(RedissonClient redisson) { return new RedissonConnectionFactory(redisson); } @Bean public RedisTemplate<String, String> redisTemplate(RedisConnectionFactory factory) { StringRedisTemplate template = new StringRedisTemplate(factory); setSerializer(template);//Setting up serialization tools template.afterPropertiesSet(); return template; } private void setSerializer(StringRedisTemplate template) { Jackson2JsonRedisSerializer jackson2JsonRedisSerializer = new Jackson2JsonRedisSerializer(Object.class); ObjectMapper om = new ObjectMapper(); om.setVisibility(PropertyAccessor.ALL, JsonAutoDetect.Visibility.ANY); om.enableDefaultTyping(ObjectMapper.DefaultTyping.NON_FINAL); jackson2JsonRedisSerializer.setObjectMapper(om); template.setValueSerializer(jackson2JsonRedisSerializer); } }
The configuration file is as follows
sentinelServersConfig: # Connection idle timeout in milliseconds default 10000 idleConnectionTimeout: 10000 pingTimeout: 1000 # Wait timeout when establishing connection with any node.Time unit is the default 10000 milliseconds connectTimeout: 10000 # The time to wait for the node to reply to the command.This time starts when the command is successfully sent.Default 3000 timeout: 3000 # Command Failure Retries retryAttempts: 3 # Command retry send interval in milliseconds retryInterval: 1500 # Reconnection interval in milliseconds reconnectionTimeout: 3000 # Maximum number of execution failures failedAttempts: 3 # Maximum number of subscriptions per connection subscriptionsPerConnection: 5 # mymaster masterName: mymaster # Selection of load Balancer load balancing algorithm class loadBalancer: !<org.redisson.connection.balancer.RoundRobinLoadBalancer> {} #Minimum number of idle connections to publish and subscribe from a node slaveSubscriptionConnectionMinimumIdleSize: 1 #Publish and subscribe connection pool size from node default value 50 slaveSubscriptionConnectionPoolSize: 50 # Minimum number of idle connections from node default 32 slaveConnectionMinimumIdleSize: 32 # Connection pool size from node default 64 slaveConnectionPoolSize: 64 # Primary Node Minimum Idle Connections Default 32 masterConnectionMinimumIdleSize: 32 # Primary Node Connection Pool Size Default 64 masterConnectionPoolSize: 64 # Load Balancing Mode for Subscription Operations subscriptionMode: SLAVE # Read only from server readMode: SLAVE # Burger address sentinelAddresses: - "redis://192.168.188.129:7700" - "redis://192.168.188.129:7800" - "redis://192.168.188.129:7900" # Time interval for scanning Redis cluster node status.The unit is in milliseconds.Default 1000 scanInterval: 1000 #This thread pool is shared by all RTopic object listeners, RRemoteService callers, and RExecutorService tasks.Default 2 threads: 0 #The number of threadpools is the number of threads stored in a Redisson instance, in all distributed data types and services it creates, and in the thread pool shared with the underlying clients.Default 2 nettyThreads: 0 # Encoding DefaultOrg.redisson.codec.JsonJacksonCodec codec: !<org.redisson.codec.JsonJacksonCodec> {} #transmission mode transportMode: NIO # Distributed lock automatic expiration time to prevent deadlocks, default 30000 lockWatchdogTimeout: 30000 # This parameter modifies whether messages come out in the order they are received by subscription publishing messages. If you choose whether messages will be processed in parallel, this parameter only applies to subscription publishing messages. Default true keepPubSubOrder: true
Then run down the code based on the unit test to validate the redisson distributed lock, the test code is rough, and the cluster timer task is simulated based on multi-threading
package com.juyi.camera; import lombok.extern.slf4j.Slf4j; import org.junit.Test; import org.junit.runner.RunWith; import org.redisson.api.RLock; import org.redisson.api.RedissonClient; import org.springframework.boot.test.context.SpringBootTest; import org.springframework.test.context.junit4.SpringRunner; import javax.annotation.Resource; import java.util.concurrent.ExecutorService; import java.util.concurrent.TimeUnit; /** * Created by IntelliJ IDEA. * User: xuzhou * Date: 2020/5/14 * Time: 18:06 */ @RunWith(SpringRunner.class) @SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT, classes = CameraApplication.class) @Slf4j public class RedissonTests { @Resource RedissonClient redissonClient; @Resource ExecutorService executorService; @Test public void testLock() { try { //Simulating cluster timer tasks with multiple threads for (int i = 1; i <= 5; i++) { int id = i; executorService.execute(new Runnable() { @Override public void run() { RLock rLock = redissonClient.getLock("testLock"); try { log.info(id + " Request Lock"); //Attempt to lock, wait up to 5 seconds, unlock automatically 10 seconds after lock boolean lockRes = rLock.tryLock(5, 10, TimeUnit.SECONDS); if (lockRes) { try { log.info(id + " Application Successful,Processing business logic"); Thread.sleep(10000); } finally { //Is it locked and whether the current thread has acquired a lock if (rLock.isLocked() && rLock.isHeldByCurrentThread()) { rLock.unlock(); log.info(id + " Release lock"); } } } else { log.info(id + " Failed to apply for lock=" + lockRes); } } catch (InterruptedException e) { e.printStackTrace(); } } }); } Thread.sleep(500000); } catch (InterruptedException e) { e.printStackTrace(); } } }
The final results are as follows
You can see that at the same time, only one thread gets the lock, avoiding competition and running as expected.
Postnote
There are still many functions of redisson. The transformation of business based on redisson is also over. Next, to replace Jedis redisson's own distributed lock as a whole is only one of the small functions. In your work, you often think and try to find different scenes.