shield-ratelimiter
Redis based distributed current limiting Toolkit

In the distributed field, we will inevitably encounter a sudden increase in concurrency, which will cause high pressure on back-end services, and even lead to system downtime. To avoid this problem, we usually add current limiting, degradation, fusing and other capabilities to the interface, so as to make the interface more robust. Common open source components in the Java field include Netflix's hystrix, Ali's open source sentinel, etc., which are quite good current limiting and fusing frameworks.

Today, we will implement a distributed current limiting component based on the characteristics of Redis components, and the name is shield ratelimiter.

principle
First, explain why Redis is used as the core of the current limiting component.

Generally speaking, assuming that a user (judged by IP) cannot access a service interface more than 10 times per second, we can create a key in Redis and set the expiration time of the key to 60 seconds.

When a user initiates an access to this service interface, the key value is increased by 1. When the key value increases to 10 in unit time (1s here), access to the service interface is prohibited. PS: it is necessary to add an access interval in a certain scenario. We do not consider the interval time this time, but only focus on the number of visits per unit time.

demand
The principle has been talked about. Let's talk about the requirements.

Development of incr and expiration mechanism based on Redis
Easy to call, declarative
Spring support
Based on the above requirements, we decided to develop the core functions based on annotation and Spring boot starter as the basic environment, so as to be well adapted to the Spring environment.

In addition, in this development, we do not simply call Redis's java class library API to realize the incr operation on Redis.

The reason is that we need to ensure that the whole current limiting operation is atomic. If we use Java code to operate and judge, there will be concurrency problems. Here I decided to use Lua script to define the core logic.

Why Lua
Before formal development, I will briefly introduce why Lua script is recommended in Redis operation.

Reduce network overhead: without using Lua's code, you need to send multiple requests to Redis, while the script only needs one time to reduce network transmission;
Atomic operation: Redis executes the entire script as an atom without worrying about concurrency and transactions;
Reuse: the script will be permanently saved in Redis, and other clients can continue to use it
Redis adds support for Lua, which can well meet the atomicity and transactional support, so that we can avoid a lot of exception logic processing. Lua's grammar is not the main content of this article. Those interested can find the information by themselves.

Formal development
Here, we officially start the process of handwritten current limiting component.

1. Project definition
The project is built on maven and mainly relies on spring boot starter. We mainly develop on springboot. Therefore, the customized development package can directly rely on the following coordinates to facilitate package management. The version number selects the stable version by itself.

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter</artifactId>
    <version>1.4.2.RELEASE</version>
</dependency>

1. Redis integration
   As we conduct current limiting operation based on Redis, we need to integrate Redis class library. As mentioned above, we develop based on Springboot, so we can directly integrate RedisTemplate here.

2.1 coordinate introduction
Here we introduce the dependency of spring boot starter redis.

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-redis</artifactId>
    <version>1.4.2.RELEASE</version>
</dependency>

2.2 inject CacheManager and RedisTemplate
Create a new Redis configuration class, named RedisCacheConfig, and inject CacheManager and RedisTemplate in the form of javaconfig. For ease of operation, we use Jackson for serialization. The code is as follows

@Configuration
@EnableCaching
public class RedisCacheConfig {

    private static final Logger LOGGER = LoggerFactory.getLogger(RedisCacheConfig.class);

    @Bean
    public CacheManager cacheManager(RedisTemplate<?, ?> redisTemplate) {
        CacheManager cacheManager = new RedisCacheManager(redisTemplate);
        if (LOGGER.isDebugEnabled()) {
            LOGGER.debug("Springboot Redis cacheManager Load complete");
        }
        return cacheManager;
    }

    @Bean
    public RedisTemplate<String, Object> redisTemplate(RedisConnectionFactory factory) {
        RedisTemplate<String, Object> template = new RedisTemplate<>();
        template.setConnectionFactory(factory);

        //Use Jackson2JsonRedisSerializer to serialize and deserialize the value of redis (the JDK serialization method is used by default)
        Jackson2JsonRedisSerializer serializer = new Jackson2JsonRedisSerializer(Object.class);

        ObjectMapper mapper = new ObjectMapper();
        mapper.setVisibility(PropertyAccessor.ALL, JsonAutoDetect.Visibility.ANY);
        mapper.enableDefaultTyping(ObjectMapper.DefaultTyping.NON_FINAL);
        serializer.setObjectMapper(mapper);

        template.setValueSerializer(serializer);
        //Use StringRedisSerializer to serialize and deserialize the key value of redis
        template.setKeySerializer(new StringRedisSerializer());
        template.afterPropertiesSet();
        LOGGER.info("Springboot RedisTemplate Load complete");
        return template;
    }
}

Note that @ Configuration should be used to label this Class as a Configuration Class. Of course, you can use @ component, but it is not recommended. The reason is that although @ Component annotation can also be used as a Configuration Class, it will not generate CGLIB proxy Class for it. Instead, using * * @ Configuration * *, CGLIB will generate proxy Class for it to improve performance.

2.3 caller application Redis configuration needs to be added to the property
After the development of our package, the caller's application.properties needs to be configured as follows:

#Stand alone redis
spring.redis.host=127.0.0.1
spring.redis.port=6379
spring.redis.pool.maxActive=8
spring.redis.pool.maxWait=-1
spring.redis.pool.maxIdle=8
spring.redis.pool.minIdle=0
spring.redis.timeout=10000
spring.redis.password=

If you have a password, configure it.

This is a stand-alone configuration. If you need to support sentinel cluster, the configuration is as follows. The Java code does not need to be changed, but only the configuration needs to be changed. Note that the two configurations cannot coexist!

#Sentinel cluster mode
# database name
spring.redis.database=0
# server password password. If it is not set, it can not be matched
spring.redis.password=
# pool settings ... Pool configuration
spring.redis.pool.max-idle=8
spring.redis.pool.min-idle=0
spring.redis.pool.max-active=8
spring.redis.pool.max-wait=-1
# name of Redis server the sentinel listens to
spring.redis.sentinel.master=mymaster
# Comma separated list of host: configuration list of port pairs sentry
spring.redis.sentinel.nodes=127.0.0.1:26379,127.0.0.1:26479,127.0.0.1:26579

3. Definition notes
For the convenience of calling, we define an annotation called RateLimiter, which is as follows

    /**
    * @author snowalker
    * @version 1.0
    * @date 2018/10/27 1:25
    * @className RateLimiter
    * @desc Current limiting notes
    */
    @Target(ElementType.METHOD)
    @Retention(RetentionPolicy.RUNTIME)
    @Documented
    public @interface RateLimiter {

        /**
        * Current limiting key
        * @return
        */
        String key() default "rate:limiter";
        /**
        * Limit the number of pass requests per unit time
        * @return
        */
        long limit() default 10;

        /**
        * Expiration time in seconds
        * @return
        */
        long expire() default 1;
    }

This annotation is explicitly used only for methods and has three main attributes.

key – indicates the name of the current limiting module. Specify this value to distinguish different applications and scenarios. The recommended format is: Application Name: module name: ip: interface name: method name
limit – indicates the number of requests allowed to pass per unit time
expire – the expiration time of the incr value, which represents the unit time of current limit in the business.
4. Analytical notes
After defining the annotation, we need to develop the aspect used by the annotation. Here, we directly use aspectj to develop the aspect. Look at the code first

@Aspect
@Component
public class RateLimterHandler {

    private static final Logger LOGGER = LoggerFactory.getLogger(RateLimterHandler.class);

    @Autowired
    RedisTemplate redisTemplate;

    private DefaultRedisScript<Long> getRedisScript;

    @PostConstruct
    public void init() {
        getRedisScript = new DefaultRedisScript<>();
        getRedisScript.setResultType(Long.class);
        getRedisScript.setScriptSource(new ResourceScriptSource(new ClassPathResource("rateLimter.lua")));
        LOGGER.info("RateLimterHandler[Distributed current limiting processor]Script loading completed");
    }

Here, RedisTemplate is injected and its API is used to call Lua script.

The init() method initializes the DefaultRedisScript when the application starts, and loads the Lua script for easy calling.

PS: Lua script is placed under classpath and loaded through ClassPathResource.

    @Pointcut("@annotation(com.snowalker.shield.ratelimiter.core.annotation.RateLimiter)")
    public void rateLimiter() {}

Here, we define a cut-off point, which means that the current limiting operation can be triggered as long as the @ RateLimiter method is annotated.

    @Around("@annotation(rateLimiter)")
    public Object around(ProceedingJoinPoint proceedingJoinPoint, RateLimiter rateLimiter) throws Throwable {
        if (LOGGER.isDebugEnabled()) {
            LOGGER.debug("RateLimterHandler[Distributed current limiting processor]Start current limiting operation");
        }
        Signature signature = proceedingJoinPoint.getSignature();
        if (!(signature instanceof MethodSignature)) {
            throw new IllegalArgumentException("the Annotation @RateLimter must used on method!");
        }
        /**
        * Get annotation parameters
        */
        // Current limiting module key
        String limitKey = rateLimiter.key();
        Preconditions.checkNotNull(limitKey);
        // Current limiting threshold
        long limitTimes = rateLimiter.limit();
        // Current limiting timeout
        long expireTime = rateLimiter.expire();
        if (LOGGER.isDebugEnabled()) {
            LOGGER.debug("RateLimterHandler[Distributed current limiting processor]The parameter value is-limitTimes={},limitTimeout={}", limitTimes, expireTime);
        }
        /**
        * Execute Lua script
        */
        List<String> keyList = new ArrayList();
        // Set the key value to the value in the annotation
        keyList.add(limitKey);
        /**
        * Call script and execute
        */
        Long result = (Long) redisTemplate.execute(getRedisScript, keyList, expireTime, limitTimes);
        if (result == 0) {
            String msg = "Due to exceeding unit time=" + expireTime + "-Number of requests allowed=" + limitTimes + "[Trigger current limiting]";
            LOGGER.debug(msg);
            return "false";
        }
        if (LOGGER.isDebugEnabled()) {
            LOGGER.debug("RateLimterHandler[Distributed current limiting processor]Current limiting execution results-result={},request[normal]response", result);
        }
        return proceedingJoinPoint.proceed();
    }
}

The logic of this code is to obtain the attributes configured by the @ ratelimit annotation: key, limit and expire, and use redistemplate The execute (redisscript, script, list keys, object... args) method is passed to Lua script for current limiting related operations. The logic is very clear.

Here, we define that if the return status of the script is 0, the current limit is triggered, and 1 indicates a normal request.

5. Lua script
Here is the core of our whole current limiting operation, which is performed by executing a Lua script. The script is as follows

--obtain KEY
local key1 = KEYS[1]

local val = redis.call('incr', key1)
local ttl = redis.call('ttl', key1)

--obtain ARGV Parameters in and print
local expire = ARGV[1]
local times = ARGV[2]

redis.log(redis.LOG_DEBUG,tostring(times))
redis.log(redis.LOG_DEBUG,tostring(expire))

redis.log(redis.LOG_NOTICE, "incr "..key1.." "..val);
if val == 1 then
    redis.call('expire', key1, tonumber(expire))
else
    if ttl == -1 then
        redis.call('expire', key1, tonumber(expire))
    end
end

if val > tonumber(times) then
    return 0
end

return 1

Logic is very popular. Let me briefly introduce it.

First, the script obtains the key of the module to be limited from the Java code. The key values of different modules must not be the same, otherwise they will be overwritten!
redis.call('incr ', key1) incr the incoming key. If the key is generated for the first time, set the timeout ARGV[1]; (initial value is 1)
ttl is a judgment to prevent some key s from being protected when the timeout time is not set and has existed for a long time;
The + 1 operation will be performed for each request. When the current limit value val is greater than the threshold value of our annotation, 0 will be returned, indicating that the request limit has been exceeded and the current limit will be triggered. Otherwise, it is a normal request.
After the expiration, a new round of cycle occurs. The whole process is an atomic operation, which can ensure that the unit time will not exceed our preset request threshold.

Here we can test in the project.

test

demo address:https://github.com/TaXueWWL/shield-ratelimter/tree/master/shleld-ratelimter-demo

Here I'll post the core code. We define an interface and annotate @ ratelimit (key = "ratedemo:1.0.0", limit = 5, expire = 100) to indicate that the module ratedemo:sendPayment:1.0.0 allows five requests within 100s. The parameter settings here are for the convenience of viewing the results. In practice, we usually set the number of passes allowed within 1s.

@Controller
public class TestController {

    private static final Logger LOGGER = LoggerFactory.getLogger(TestController.class);

    @ResponseBody
    @RequestMapping("ratelimiter")
    @RateLimiter(key = "ratedemo:1.0.0", limit = 5, expire = 100)
    public String sendPayment(HttpServletRequest request) throws Exception {

        return "Normal request";
    }

}

Through the RestClient request interface, the log returns as follows:

2018-10-28 00:00:00.602 DEBUG 17364 --- [nio-8888-exec-1] c.s.s.r.core.handler.RateLimterHandler   :
 RateLimterHandler[Distributed current limiting processor]Start current limiting operation
2018-10-28 00:00:00.688 DEBUG 17364 --- [nio-8888-exec-1] c.s.s.r.core.handler.RateLimterHandler   :
 RateLimterHandler[Distributed current limiting processor]Current limiting execution results-result=1,request[normal]response

2018-10-28 00:00:00.860 DEBUG 17364 --- [nio-8888-exec-3] c.s.s.r.core.handler.RateLimterHandler   :
 RateLimterHandler[Distributed current limiting processor]Start current limiting operation
2018-10-28 00:00:01.183 DEBUG 17364 --- [nio-8888-exec-4] c.s.s.r.core.handler.RateLimterHandler   :
 RateLimterHandler[Distributed current limiting processor]Start current limiting operation
2018-10-28 00:00:01.520 DEBUG 17364 --- [nio-8888-exec-3] c.s.s.r.core.handler.RateLimterHandler   :
 RateLimterHandler[Distributed current limiting processor]Current limiting execution results-result=1,request[normal]response
2018-10-28 00:00:01.521 DEBUG 17364 --- [nio-8888-exec-4] c.s.s.r.core.handler.RateLimterHandler   :
 RateLimterHandler[Distributed current limiting processor]Current limiting execution results-result=1,request[normal]response

2018-10-28 00:00:01.557 DEBUG 17364 --- [nio-8888-exec-5] c.s.s.r.core.handler.RateLimterHandler   :
 RateLimterHandler[Distributed current limiting processor]Start current limiting operation
2018-10-28 00:00:01.558 DEBUG 17364 --- [nio-8888-exec-5] c.s.s.r.core.handler.RateLimterHandler   :
 RateLimterHandler[Distributed current limiting processor]Current limiting execution results-result=1,request[normal]response

2018-10-28 00:00:01.774 DEBUG 17364 --- [nio-8888-exec-7] c.s.s.r.core.handler.RateLimterHandler   :
 RateLimterHandler[Distributed current limiting processor]Start current limiting operation
2018-10-28 00:00:02.111 DEBUG 17364 --- [nio-8888-exec-8] c.s.s.r.core.handler.RateLimterHandler   :
 RateLimterHandler[Distributed current limiting processor]start
2018-10-28 00:00:02.169 DEBUG 17364 --- [nio-8888-exec-7] c.s.s.r.core.handler.RateLimterHandler   :
 RateLimterHandler[Distributed current limiting processor]Current limiting execution results-result=1,request[normal]response

2018-10-28 00:00:02.169 DEBUG 17364 --- [nio-8888-exec-8] c.s.s.r.core.handler.RateLimterHandler   :
 Due to exceeding unit time=100-Number of requests allowed=5[Trigger current limiting]
2018-10-28 00:00:02.276 DEBUG 17364 --- [io-8888-exec-10] c.s.s.r.core.handler.RateLimterHandler   :
 RateLimterHandler[Distributed current limiting processor]Start current limiting operation
2018-10-28 00:00:02.276 DEBUG 17364 --- [io-8888-exec-10] c.s.s.r.core.handler.RateLimterHandler   :
 RateLimterHandler[Distributed current limiting processor]The parameter value is-limitTimes=5,limitTimeout=100
2018-10-28 00:00:02.278 DEBUG 17364 --- [io-8888-exec-10] c.s.s.r.core.handler.RateLimterHandler   :
 Due to exceeding unit time=100-Number of requests allowed=5[Trigger current limiting]
2018-10-28 00:00:02.445 DEBUG 17364 --- [nio-8888-exec-2] c.s.s.r.core.handler.RateLimterHandler   :
 RateLimterHandler[Distributed current limiting processor]Start current limiting operation
2018-10-28 00:00:02.445 DEBUG 17364 --- [nio-8888-exec-2] c.s.s.r.core.handler.RateLimterHandler   :
 RateLimterHandler[Distributed current limiting processor]The parameter value is-limitTimes=5,limitTimeout=100
2018-10-28 00:00:02.446 DEBUG 17364 --- [nio-8888-exec-2] c.s.s.r.core.handler.RateLimterHandler   :
 Due to exceeding unit time=100-Number of requests allowed=5[Trigger current limiting]
2018-10-28 00:00:02.628 DEBUG 17364 --- [nio-8888-exec-4] c.s.s.r.core.handler.RateLimterHandler   :
 RateLimterHandler[Distributed current limiting processor]Start current limiting operation
2018-10-28 00:00:02.628 DEBUG 17364 --- [nio-8888-exec-4] c.s.s.r.core.handler.RateLimterHandler   :
 RateLimterHandler[Distributed current limiting processor]The parameter value is-limitTimes=5,limitTimeout=100
2018-10-28 00:00:02.629 DEBUG 17364 --- [nio-8888-exec-4] c.s.s.r.core.handler.RateLimterHandler   :
 Due to exceeding unit time=100-Number of requests allowed=5[Trigger current limiting]

According to the log, we can see that after five normal requests, the current limit trigger is returned, indicating that our logic is effective. For the front end, we can also see the false flag, indicating that our Lua script current limit logic is correct. The specific flag returned here needs to be clearly defined by the caller.

summary

Through the incr and expire features of Redis, we have developed and defined a set of annotation based distributed current limiting operations, and the core logic is based on Lua to ensure atomicity. The purpose of current limiting has been achieved. In production, you can customize your own current limiting components based on this feature. Of course, you can refer to the code in this article. I believe what you write must be better than my demo!