Thousands of duplicate submissions in an instant, and I'm holding them with SpringBoot+Redis

Posted by Sheen on Sat, 29 Feb 2020 05:01:13 +0100

In actual development projects, an exposed interface often faces a large number of requests submitted in an instant. If you want to filter out duplicate requests and cause harm to your business, you need to achieve power!

Let's explain the concept of idempotency:

Any multiple executions will have the same impact as a single execution.In this sense, the ultimate meaning is that the impact on the database can only be one-time and cannot be repeated.

How to guarantee its idempotency is usually done by the following means:

1. The database is indexed uniquely to ensure that only one data 2, token mechanism is inserted into the database eventually. Each interface request is preceded by a token, and then the next request is made by adding the token to the header body of the request. Background validation is performed. If the validation is done by deleting the token, the next request is made to judge token3, pessimistic lock or optimistic lock again, pessimistic lockIt is guaranteed that no other sql will be able to update data each time for update (when the database engine is innodb, the select condition must be a unique index,Prevent locking the whole table) 4. Query first and then judge whether there is data in the database. If there is proof that the request has already been requested, reject the request directly. If there is no proof that it is the first time to enter and let it go directly.

The redis schematic diagram for automatic idempotency:

image.png

1. Build service Api for redis

1. First, set up the redis server.

2. Introduce the stater of redis in springboot, or jedis wrapped in Spring. The api used later is its set method and exists method. Here we use the wrapped redisTemplate of springboot

/**
 * redis Tool class
 */
@Component 
publicclassRedisService{
 @Autowired
 privateRedisTemplate redisTemplate;
 /**
     * Write Cache
     * @param key
     * @param value
     * @return
     */
 publicbooleanset(finalString key, Object value) {
 boolean result = false;
 try{
 ValueOperations<Serializable, Object> operations = redisTemplate.opsForValue();
            operations.set(key, value);
            result = true;
        } catch(Exception e) {
            e.printStackTrace();
        }
 return result;
    }
 /**
     * Write Cache Set Aging Time
     * @param key
     * @param value
     * @return
     */
 publicboolean setEx(finalString key, Object value, Long expireTime) {
 boolean result = false;
 try{
 ValueOperations<Serializable, Object> operations = redisTemplate.opsForValue();
            operations.set(key, value);
            redisTemplate.expire(key, expireTime, TimeUnit.SECONDS);
            result = true;
        } catch(Exception e) {
            e.printStackTrace();
        }
 return result;
    }
 /**
     * Determine if there is a corresponding value in the cache
     * @param key
     * @return
     */
 publicboolean exists(finalString key) {
 return redisTemplate.hasKey(key);
    }
 /**
     * Read Cache
     * @param key
     * @return
     */
 publicObjectget(finalString key) {
 Object result = null;
 ValueOperations<Serializable, Object> operations = redisTemplate.opsForValue();
        result = operations.get(key);
 return result;
    }
 /**
     * Delete the corresponding value
     * @param key
     */
 publicboolean remove(finalString key) {
 if(exists(key)) {
 Booleandelete= redisTemplate.delete(key);
 returndelete;
        }
 returnfalse;
    }
} 

2. Custom Annotation AutoIdempotent

Customize a comment whose main purpose is to add it to methods that need to be idempotent. Any method that annotates it will be automatically idempotent.Background utilizing reflection If this annotation is scanned, this method is processed to achieve automatic idempotency, using the meta-annotation ElementType.METHOD to indicate that it can only be placed on a method, and etentionPolicy.RUNTIME to indicate that it is running.

@Target({ElementType.METHOD})
@Retention(RetentionPolicy.RUNTIME)
public@interfaceAutoIdempotent{
} 

3. token creation and verification

1. token service interface

We create a new interface to create a token service. There are two main methods, one to create a token and one to validate a token.Creating a token mainly produces a string, and checking token conveys the request object. Why pass the request object?The main purpose is to get the token inside the header, then test it, and return the specific error information to the front end by throwing an Exception.

publicinterfaceTokenService{
 /**
     * Create token
     * @return
     */
 public String createToken();
 /**
     * Verify token
     * @param request
     * @return
     */
 publicboolean checkToken(HttpServletRequest request) throwsException;
} 

2. token's service implementation class

Token refers to the redis service, creates a token, generates a random uuid string using a random algorithm tool class, and puts it into redis (to prevent redundant data retention, set an expiration time of 10,000 seconds, depending on the business), and returns this token value if it is placed successfully.The checkToken method is to get the token from the header to the value (if not, from the paramter) and throw an exception if it does not exist.This exception information can be caught by the interceptor and returned to the front end.

@Service
publicclassTokenServiceImplimplementsTokenService{
 @Autowired
 privateRedisService redisService;
 /**
     * Create token
     *
     * @return
     */
 @Override
 publicString createToken() {
 String str = RandomUtil.randomUUID();
 StrBuilder token = newStrBuilder();
 try{
            token.append(Constant.Redis.TOKEN_PREFIX).append(str);
            redisService.setEx(token.toString(), token.toString(),10000L);
 boolean notEmpty = StrUtil.isNotEmpty(token.toString());
 if(notEmpty) {
 return token.toString();
            }
        }catch(Exception ex){
            ex.printStackTrace();
        }
 returnnull;
    }
 /**
     * Verify token
     *
     * @param request
     * @return
     */
 @Override
 publicboolean checkToken(HttpServletRequest request) throwsException{
 String token = request.getHeader(Constant.TOKEN_NAME);
 if(StrUtil.isBlank(token)) {// token does not exist in header
            token = request.getParameter(Constant.TOKEN_NAME);
 if(StrUtil.isBlank(token)) {// There is no token in the parameter either
 thrownewServiceException(Constant.ResponseCode.ILLEGAL_ARGUMENT, 100);
            }
        }
 if(!redisService.exists(token)) {
 thrownewServiceException(Constant.ResponseCode.REPETITIVE_OPERATION, 200);
        }
 boolean remove = redisService.remove(token);
 if(!remove) {
 thrownewServiceException(Constant.ResponseCode.REPETITIVE_OPERATION, 200);
        }
 returntrue;
    }
} 
4. Configuration of interceptors

1. web Configuration Class, which implements WebMvcConfigurerAdapter. The main function is to add autoIdempotentInterceptor to the configuration class so that we can take effect to the interceptor. Note that the @Configuration annotation is used so that it can be added to the context when the container is started

```cpp
@Configuration
publicclassWebConfigurationextendsWebMvcConfigurerAdapter{
 @Resource
 privateAutoIdempotentInterceptor autoIdempotentInterceptor;
 /**
     *Add Interceptors
     * @param registry
     */
 @Override
 publicvoid addInterceptors(InterceptorRegistry registry) {
        registry.addInterceptor(autoIdempotentInterceptor);
 super.addInterceptors(registry);
    } 
} 

2. Intercept Processor: The main function is to intercept the scan to AutoIdempotent to annotate to method, then call tokenService's checkToken() method to verify that token is correct, and render the exception information as json to the front end if it is caught

/**
 * Interceptor
 */
@Component
publicclassAutoIdempotentInterceptorimplementsHandlerInterceptor{
 @Autowired
 privateTokenService tokenService;
 /**
     * Preprocessing
     *
     * @param request
     * @param response
     * @param handler
     * @return
     * @throws Exception
     */
 @Override
 publicboolean preHandle(HttpServletRequest request, HttpServletResponse response, Object handler) throwsException{
 if(!(handler instanceofHandlerMethod)) {
 returntrue;
        }
 HandlerMethod handlerMethod = (HandlerMethod) handler;
 Method method = handlerMethod.getMethod();
 //Scanning marked by ApiIdempotment
 AutoIdempotent methodAnnotation = method.getAnnotation(AutoIdempotent.class);
 if(methodAnnotation != null) {
 try{
 return tokenService.checkToken(request);// Idempotency checks, checks pass, checks fail, exceptions are thrown, and friendly hints are returned through uniform exception handling
            }catch(Exception ex){
 ResultVo failedResult = ResultVo.getFailedResult(101, ex.getMessage());
                writeReturnJson(response, JSONUtil.toJsonStr(failedResult));
 throw ex;
            }
        }
 //Must return true, otherwise all requests will be blocked
 returntrue;
    }
 @Override
 publicvoid postHandle(HttpServletRequest request, HttpServletResponse response, Object handler, ModelAndView modelAndView) throwsException{
    }
 @Override
 publicvoid afterCompletion(HttpServletRequest request, HttpServletResponse response, Object handler, Exception ex) throwsException{
    }
 /**
     * json value returned
     * @param response
     * @param json
     * @throws Exception
     */
 privatevoid writeReturnJson(HttpServletResponse response, String json) throwsException{
 PrintWriter writer = null;
        response.setCharacterEncoding("UTF-8");
        response.setContentType("text/html; charset=utf-8");
 try{
            writer = response.getWriter();
            writer.print(json);
        } catch(IOException e) {
        } finally{
 if(writer != null)
                writer.close();
        }
    }
} 

5. Test Cases

1. Simulate Business Request Class

First we need to get a specific token through the getToken() method via the /get/token path, and then we call the testIdempotence method, which annotates @AutoIdempotent, and the interceptor will intercept all requests. When it is determined that the processing method has this annotation, the checkToken() method in TokenService will be called. If an exception is caught, it will be different.The caller is often thrown, so let's simulate the request below:

@RestController
publicclassBusinessController{
 @Resource
 privateTokenService tokenService;
 @Resource
 privateTestService testService;
 @PostMapping("/get/token")
 publicString  getToken(){
 String token = tokenService.createToken();
 if(StrUtil.isNotEmpty(token)) {
 ResultVo resultVo = newResultVo();
            resultVo.setCode(Constant.code_success);
            resultVo.setMessage(Constant.SUCCESS);
            resultVo.setData(token);
 returnJSONUtil.toJsonStr(resultVo);
        }
 returnStrUtil.EMPTY;
    }
 @AutoIdempotent
 @PostMapping("/test/Idempotence")
 publicString testIdempotence() {
 String businessResult = testService.testIdempotence();
 if(StrUtil.isNotEmpty(businessResult)) {
 ResultVo successResult = ResultVo.getSuccessResult(businessResult);
 returnJSONUtil.toJsonStr(successResult);
        }
 returnStrUtil.EMPTY;
    }
} 

2. Use postman requests

First access the get/token path to get specific to token:

image.png

By taking the token and putting it in the specific request header, you can see that the first request succeeds, and then we request the second time:

image.png

The second request, returned to the repetitive operation, visible repetitive validation passed, and multiple requests at that time we only let it succeed for the first time and fail for the second time:

image.png

6. Summary

This article describes how to elegantly implement interface power using springboot and interceptor, redis, etc. It is very important for power in the actual development process, because an interface may be called by countless clients, how to ensure that it does not affect the background business processing, how to ensure that it only affects the data once is very important, it can prevent dirty data or chaotic numberIt is a very useful thing to reduce the concurrency.The traditional practice is to judge the data every time, which is not intelligent and automated, and is more cumbersome.Today's automated processing can also improve program scalability.

 

Topics: Redis Database JSON SpringBoot