redis breakdown, avalanche and penetration solutions

Posted by foreverhex on Sat, 05 Mar 2022 15:46:02 +0100

Redis breakdown

Redis cache breakdown refers to the sudden failure of a very hot key (i.e. more keywords searched on the client). At this time, a large number of requests sent from the client cannot find the key in redis, they will find it in the data, and eventually lead to excessive pressure and collapse of the database.

Solution:
1. Set the time limit of value to never expire
This method is very simple and rough, but it is safe and reliable. But it takes up a lot of space and consumes a lot of memory. This method is not recommended by individuals and should be operated according to specific business logic.
2. Use Timetask to do a scheduled task
Use Timetask to query the database of some hot key s at regular intervals, and update the query results to redis. The precondition is not to put too much pressure on the database.
2. Through synchronized + double check mechanism
When reids penetration occurs, a large number of requests are sent to the database. At this time, our solution is to let only one thread query the hot key, while other threads remain blocked (they can sleep for a few seconds). When the thread entering the database finds the value corresponding to the key, we synchronize it to the redis cache. After other threads wake up, they go back to redis to request data.
example:

 private static volaite Object obj = new Object();
   public String getValue(String key){
     String value=redis.get(key,String.class);
     if(value==null||StringUtils.isBlank(value){
         synchronized(obj){
         		//After entering synchronized, check redis again to prevent the last thread that grabbed the lock from having been updated.
                value=redis.get(key,String.class);
                 if(value==null||StringUtils.isBlank(value){
                     value=db.query(key);
                      redis.set(key,value,1000); 
          }
       }
     }    
      return value;
   }

Disadvantages: there is a risk of deadlock and thread blocking.

Redis avalanche

This means that when a large number of requests query multiple key s, the redis cache fails or cannot be found, and then a large number of requests go to db for query, resulting in a sudden surge in db pressure and collapse.

Causes: 1 key fails at the same time, 2 redis itself crashes

Solution:
1. When setting the cache, initialize its expiration time randomly
If the key of redis fails at the same time, this method can be adopted. The specific failure time depends on the business situation
2. Place different hotspot key s on different nodes
Because redis is generally deployed in clusters, placing different hotspot key s evenly on different nodes can also effectively avoid avalanche.
3. Set the time limit of value to never expire
4. Use Timetask to do a scheduled task and re brush the redis cache before it expires

Redis penetration

Because bad users maliciously and frequently query will cause great problems to the system: the key cache and the database does not exist, so each query will query the database, resulting in database crash.
(for example, the primary key of the data we store in the database is self increasing and has no negative number. Some hackers take advantage of this and continue to use the parameter with the primary key id of - 1 to initiate massive query requests. As a result, these requests can not find the corresponding data in redis and can only be queried in the database, resulting in the collapse of the database.)

Solution:
1. When a similar request is sent, no matter what result is found, it will be put into the redis cache
In this way, when he initiates a request with the same parameter next time, he will directly enter redis instead of the database.
2. Pull Black ip
3. Check the validity of the requested parameters and return them directly on the premise that they are judged to be illegal
4. Use bloom filter
The bloom filter can be understood as a white list or blacklist. Its function is to judge whether an element exists in the filter.
White list:
The filter contains all the valid parameter keys in the database. The request passes through the bloom filter. The bloom filter determines whether the requested key is in the filter, releases the request to redis, and return s empty data directly.

public static void main(String[] args){
	Config config = new Config();
	config.useSingleServer().setAddress("redis://127.0.0.1:6379");
	config.useSingleServer().setPassword("1234");
	//Construct Redsson
	RedissonClient redisson = Redisson.create(config);
	RBloomFilter<String> bloomFilter = redisson.getBloomFilter("phoneList");//Give us your name, bron
	//Initializing the bloom filter setting, the expected element is 100000000L, and the error rate is 3%
	bloomFilter.tryInit(100000000L,0.03);
	//Insert 10086 into the bloom filter
	bloomFilter.add("10086");
	//Determine whether the following numbers exist in the bloom filter
	//false
	System.out.println("123456");
	//true
	System.out.println("10086");
}

Disadvantages:
Bloom filter may cause misjudgment and penetrate redis into DB, but the probability of misjudgment is very small.

Topics: Database MySQL