Use jedis2 There may be a problem with the small partners of the connection pool of 9.1. After the program runs with high concurrency for a period of time, the error message of Could not get a resource from the pool will appear, and the idle connection idelPool in the redistemplate connection factory viewed after stopping the interface call is always 0. This part of the connection will not be released again. It is preliminarily suspected that the jedis connection pool is leaking, After consulting the data, it is found that jedis2 There is a problem of connection pool leakage in versions below 9.2. You can see the bug repair instructions in 2.9.2 officially (after my comprehensive test, I found that this bug was modified in 2.9.1, and there is no problem in version 2.9.0. It's really a pit)
If the version cannot be upgraded, you can turn on the jedis connection pool patrol function: jedis connection pool patrol regularly releases unavailable resources
@Configuration
@EnableCaching
public class RedisConfig extends CachingConfigurerSupport {
@Value("${spring.redis.host}")
private String host;
@Value("${spring.redis.port}")
private Integer port;
@Value("${spring.redis.password}")
private String password;
@Value("${spring.redis.database}")
private Integer dataBase;
@Value("${spring.redis.jedis.pool.max-wait}")
private Integer MAX_WAIT_MILLIS;
@Value("${spring.redis.jedis.pool.max-active}")
private Integer MAX_TOTAL;
@Value("${spring.redis.jedis.pool.max-idle}")
private Integer MAX_IDLE;
// @Bean(name = "redisTemplate")
@Bean
RedisTemplate<String, Object> redisTemplate() {
RedisTemplate<String, Object> template = new RedisTemplate<>();
template.setConnectionFactory(connectionFactory(host, port, password, MAX_IDLE, MAX_TOTAL, MAX_WAIT_MILLIS, dataBase));
Jackson2JsonRedisSerializer jacksonSeial = new Jackson2JsonRedisSerializer(Object.class);
ObjectMapper om = new ObjectMapper();
om.setVisibility(PropertyAccessor.ALL, JsonAutoDetect.Visibility.ANY);
om.enableDefaultTyping(ObjectMapper.DefaultTyping.NON_FINAL);
jacksonSeial.setObjectMapper(om);
template.setKeySerializer(new StringRedisSerializer());
template.setValueSerializer(jacksonSeial);
template.setHashKeySerializer(new StringRedisSerializer());
template.setHashValueSerializer(jacksonSeial);
template.afterPropertiesSet();
return template;
}
//Configuration factory
public RedisConnectionFactory connectionFactory(String host, int port, String password, int maxIdle,
int maxTotal, long maxWaitMillis, int index) {
JedisConnectionFactory jedisConnectionFactory = new JedisConnectionFactory();
jedisConnectionFactory.setHostName(host);
jedisConnectionFactory.setPort(port);
if (!StringUtils.isEmpty(password)) {
jedisConnectionFactory.setPassword(password);
}
if (index != 0) {
jedisConnectionFactory.setDatabase(index);
}
jedisConnectionFactory.setPoolConfig(poolConfig(maxIdle, maxTotal, maxWaitMillis));
jedisConnectionFactory.afterPropertiesSet();
return jedisConnectionFactory;
}
//Connection pool configuration
public JedisPoolConfig poolConfig(int maxIdle, int maxTotal, long maxWaitMillis) {
JedisPoolConfig poolConfig = new JedisPoolConfig();
poolConfig.setMaxIdle(maxIdle);
poolConfig.setMaxTotal(maxTotal);
poolConfig.setMaxWaitMillis(maxWaitMillis);
// poolConfig.setTestOnBorrow(true);
// poolConfig.setTestOnReturn(true);
// //Connect scan when Idle
// poolConfig.setTestWhileIdle(true);
// //Indicates the number of milliseconds to sleep between two scans of idle object monitor
// poolConfig.setTimeBetweenEvictionRunsMillis(30000);
// //Indicates the maximum number of objects scanned each time by idle object monitor
// poolConfig.setNumTestsPerEvictionRun(10);
// //Indicates the shortest time an object stays in the idle state at least before it can be scanned and expelled by the idle object monitor; This item is only meaningful if timebetween evictionrunsmillis is greater than 0
// poolConfig.setMinEvictableIdleTimeMillis(60000);
return poolConfig;
}
}
However, the team size of jedis is also relatively small. Lettuce is the default and recommended by spring framework. Lettuce has a great advantage that connections are shared between threads. Even without connection pool, the performance is very excellent. Lettuce community is relatively active, so lettuce can also be used to solve the problem of connection pool leakage
Lettuce principle: the connection of lettuce is based on Netty. The connection instance (StatefulRedisConnection) can be accessed concurrently among multiple threads. It should be thread safe. Therefore, one connection instance (StatefulRedisConnection) can meet the concurrent access in multi-threaded environment. Of course, this is also a scalable design, If one connection instance is insufficient, you can also add connection instances as needed. The configuration of lettuce connection pool is as follows:
spring: redis: host: 192.168.1.1 # server address port: 6379 # Service port number database: 0 # Database index password: 123 # The server connection password is blank by default lettuce: pool: max-active: 1000 # Maximum connections in connection pool max-wait: 3000 # Maximum blocking waiting time of connection pool max-idle: 500 # The largest free connection in the connection pool min-idle: 100 # The smallest free connection in the connection pool time-between-eviction-runs: 60000 #This configuration must be. The minimum idle connection of lettuce will take effect according to whether it is configured or not. The default is - 1 timeout: 200 # Connection timeout
However, lettuce has the problems of cluster detection and occasional timeout
lettuce will not automatically scan whether the cluster nodes are normal. When there is a single point of redis failure in the cluster mode, there will always be different calls and error reports. There are two ways to solve this problem:
1. Using springboot2 Version after 3. X and topology refresh configuration
spring: redis: xxx... lettuce: cluster: refresh: adaptive: true #Automatic refresh once every 10 seconds period: 10
This method will detect whether the node is available every 10 seconds, but there is still a 10 second gap period when the node really has a problem. During this period, requests will continue to fail, which obviously does not meet expectations. If the scanning interval is shortened, it will increase the burden of the system and will not fundamentally solve the problem of the gap period, Therefore, it is also necessary to add the configuration of the redirection times (max redirects) of the cluster. When the node is not connectable, eliminate the node and connect to the available node. The topology refresh also runs at the same time. When the node recovers, the node rejoins the connection of the client
spring: redis: cluster: nodes: 192.168.1.1:8001,192.168.1.1:8002,192.168.1.1:8003 max-redirects: 2 lettuce: cluster: refresh: adaptive: true #Automatic refresh once every 10 seconds period: 10
Of course, the topology refresh needs to be upgraded by springboot, which has a great impact on the current network. In order to ensure the stability of the system, it may not be a big fight. Therefore, it may be better to adopt the jedis connection pool scheme at this time. Jedis will automatically refresh the topology without any configuration, but it also needs to add the configuration of the redirection times of the cluster (max redirects)
spring: redis: cluster: nodes: 192.168.1.1:8001,192.168.1.1:8002,192.168.1.1:8003 max-redirects: 2 jedis: pool: max-active: 100 # Maximum connections in connection pool max-wait: 3000 # Maximum blocking waiting time of connection pool max-idle: 50 # The largest free connection in the connection pool min-idle: 10 # The smallest free connection in the connection pool #time-between-eviction-runs: 60000 timeout: 500 # Connection timeout