Redis

1. Know Redis

1.1 what is Redis

Redis is stored in the form of key value, which is different from the traditional relational database It does not necessarily follow some basic requirements of traditional databases (non relational, distributed, open source, scalable)
advantage:
Highly concurrent data reading and writing (directly in memory)
Efficient storage and access of massive data
Expandability and high availability of data
Single threaded operation, each operation is an atomic operation, and there are no concurrency related problems (redis 6)

Disadvantages:
redis(ACID processing is very simple)
Cannot make too complex relational database model

Redis is stored in a key value store
Keys can include: (string) string, hash, (list) linked list, (set) set, (zset) ordered set These data sets point to push/pop,add/remove, intersection and union, and richer operations redis supports sorting in different ways. In order to ensure efficiency, data is cached in memory It can periodically write updated data to disk or write modification operations to additional files

redis positioning is to cache, improve data reading and writing speed, and reduce the pressure on database storage and access

1.2 redis advantages

Extremely high performance – Redis can support more than 10W reads and writes per second. Rich data types – Redis supports string, lists, hashes, sets and Ordered Sets data type operations of binary cases.
Atomic – all operations of Redis are atomic. At the same time, Redis also supports the atomic execution of several combined operations.
Rich features – Redis also supports publish / subscribe, notification, key expiration and other features.

The following is the official benchmark data:

• The test completed 50 concurrent execution of 100000 requests.
• The value set and obtained is a 256 byte string.
• Linux box is running Linux 2 6. This is x3320xeon2 5ghz.
• Text execution uses the loopback interface (127.0.0.1).
  Results: the reading speed is 110000 times / s and the writing speed is 81000 times / s.

2. Data type

2.1 String type

String type contains many special types and is binary safe For example, serialized objects are stored, for example, a picture is stored in binary, For example, a simple

Map<String, String> map

map.put("name", Json.toJsonString(user));

Type command key parameter data

set key value  =>  Save key value pair
get key  =>  Take out the value according to the key
incr key  =>  Increment the value by 1
decr key  =>  Decrement the value by 1
del key  =>  Delete key value pairs according to key
setex key timeout value  =>  Save key value pair,timeout Indicates the time of failure,Company s
ttl key => You can query the current key How much time is left to expire
setnx key value  =>  If key Already exists,No operation, If key non-existent,Add directly

incrby key num => Offset value
mset k1 v1 k2 v2 ... => Batch storage of key value pairs
mget k1 k2 ... => Batch fetch key value
append key 'value' => Splicing new content after original value
setnx key value => Save key value,Do not save when key exists
setex key timeout value => Save key value pair,timeout Indicates the time of failure,Company s
ttl =>You can query the current key How much time is left to expire
setrange key index value => Value corresponding to modification key,index Indicates the starting index position

Application scenario:****

Counter: many applications will use redis as the basic tool of counting. It can realize the functions of fast counting, query and cache, and the data can be landed to other data sources asynchronously.
For example, the video playback count system uses redis as the basic component of video playback count.

​ incr viewnum 1

Shared session: in consideration of load balancing, distributed services will balance the access of user information to different servers. Users may need to log in again to refresh an access. To avoid this problem, redis can be used to centrally manage user sessions,
In this mode, as long as the high availability and scalability of redis are guaranteed, each time the user update or query login information is obtained, it is directly obtained from redis.

2.2 hash type

Hash type is the mapping table of field and value of String type Or a String collection It is especially suitable for storing objects. Comparatively speaking, an object stored in hash type occupies less memory space than that stored in String type, and it is convenient to store the whole object

Mp<string, Map<string, ?>> map

hset key hashkey hashvalue => Deposit one hash object
hget key hashkey => according to hash Object key to get value
hexists key hashkey => judge hash An object contains a key
hdel key hashkey => according to hashkey delete hash Object key value pair


hincrby key hashkey Incremental value => Increasing hashkey Corresponding value
hlen key => obtain hash Number of object keys
hkeys key => obtain hash All keys of the object
hvals key => obtain hash All values of the object
hgetall key => obtain hash All data of the object

hsetnx is also available, and its function is the same as that of setnx

Application scenario:

Hash structure is more intuitive than string serialization cache information, and it is more convenient in update operation.
Therefore, it is often used for user information management, but the hash type is different from the relational database. The hash type is sparse,
The relational database is completely structured. The relational database can do complex relational queries, while redis simulates complex relational queries

Difficult development and high maintenance cost

Shared session:
key: user_token

value:

class User{
private String userame;
private String password;
private int age;
}
user("dafei", "666", 18)

Scheme 1: convert the user object into json format string and save it in redis [focus on query, it's very troublesome to change]

key value

user_token : "{name:dafei, age:19, password:666}"

Scheme 2: convert the user object into a hash object and save it in redis [focus on modification, query is relatively troublesome]

key value

user_token : {
name: ddafei
age 19
password: 666
}

2.3 list type

The List in Redis is similar to the queue in Java and can also be used as a List
List type is a collection of linked list structure. Its main functions include push,pop, obtaining elements, etc In more detail, the list type is a double ended linked list structure. We can add or delete elements at the head or tail of the set through relevant operations. The design of the list is very simple and exquisite, which can be used as both stack and queue Meet most needs

Map<String, List>

rpush key value => Add data to the right of the list
lrange key start end => Range display list data,Set 0 for full display -1 
lpush key value => Add data to the left of the list
lpop key => The leftmost data in the pop-up list
rpop key => The rightmost data in the pop-up list
llen key => Get list length

linsert key before/after refVal newVal => Before reference value/Insert data after
lset key index value => Modify data according to index
lrem key count value => Delete data by number in the list
ltrim key start end => Range interception list
lindex key index => Get the data in the list according to the index

Application scenario:

1. User favorite article list:

xxxx_user_articles:uid [aid1, aid2, aid3...]

2.4 set type

Set is an unordered set of string type. Set is implemented through hashtable. For sets, we can take intersection, union and difference sets

sadd key value => to set Add element to collection
smembers key => list set Elements in the collection
srem key value => delete set Elements in the collection
spop key count => Randomly pop up elements in the collection


sdiff key1 key2 => return key1 Unique elements in(Difference set)
sdiffstore var key1 key2 => return key1 Store unique elements in another set aggregate
sinter key1 key2 => Return two set Intersection of sets
sinterstore var key1 key2 => Return two set The intersection of sets is stored in another set aggregate
sunion  key1 key2 => Return two set Union of sets
sunionstore var key1 key2 => Return two set The union of sets is stored in another set aggregate
smove key1 key2 value => hold key1 Move an element in key2 in
scard key => return set Number of elements in the collection
sismember key value => Determine whether the set contains a value
srandmember key count => Random acquisition set Elements in collection

Application scenario:

1. Weight removal;

2. Lucky draw;
1. Prepare a lottery pool: Sadd lucky draw 1 2 3 4 5 6 7 8 9 10 11 12 13
2. Draw three third prizes: spop lucky draw 3
3. Draw two second prizes: spop lucky draw 2
4. Draw a second prize: spop lucky draw 1

2.4 sorted_set type

zadd key score column => Deposit score and name
zincrby key score column => Score corresponding to offset name
zrange key start end => Output names in ascending order of scores
zrevrange key start end => Output names in descending order of scores


zrank key name => Return ranking in ascending order
zrevrank key name => Return ranking in descending order
zcard key => Returns the number of elements
zrangebyscore key min max [withscores] => Output names in ascending order of score range
zrevrangebyscore key max min [withscores] => Output names in descending order of score range
zrem key name => Delete name and score
zremrangebyscore key min max [withscores] => Delete elements based on score range
zremrangebyrank key start end => Delete elements according to ranking
zcount key min max => Count the number according to the score range

Summary:

If you decide to use redis, you need to consider which data type to use [partial redis native data structure]

• 1> If you want to sort, select zset
• 2> If the data is multiple and the list is allowed to be selected repeatedly
• 3> If there are multiple data and it is not allowed to select set repeatedly
• 4> The rest use string

hash - > Convert to json format

{
  key1:value1
  key2:value2        ------> "{key1:value1, key2:value2}"
}

JSON.toJsonString(map)

Some companies agree that all redis key s and value s use strings (excluding zset scenarios) [partial redis String type json structure]

• 1> If you want to sort, select zset

• 2> The rest use string

Advantages: in java operations, if you use various types: list set, and other operations redis, you need to specify generics explicitly, which is troublesome
Therefore, some companies standardize and use strings uniformly to reduce generic operations

List<String>list = ...
Set<String> set = ....
Map<String, Object> map = ....


List<Object>  list = redis object.getList
Set<Object> set =redis object.getSet   
Map<Object, Object> map  =  redis object.getMap

How to design key and value values
key design

  1: Uniqueness
  2: Readability
    Design keyvalue Cache user favorite article list
    key                                    value
      article_favor_list:uid1         [1,2,3,4]
      article_favor:uid2         [1,2,3,4]
  3: flexibility
  4: Timeliness

Value value
According to demand

3. Advanced redis

3.1 redis advanced command

Return all the keys * * keys * * * (fuzzy query is allowed)

Existkey specifies whether existkeys exist

expire sets the expiration time of a key Use ttl to view the remaining time

persist cancel expiration time

Flush DB empties the current database and flush hall empties all databases

Select select a database from 0 to 15 (a total of 16 databases). The default entry is 0 database

move [key] [Database subscript] means that the key in the current data is transferred to other databases

Random key randomly returns a key in the database

Rename rename key

echo print name

dbsize view the number of key s in the database

info get database information

config get real-time transmission and storage of received requests (return relevant configuration information)

**config get * * * return all configurations

3.2 redis security

Because redis is very fast, under a good server, an external user can make 15w password attempts in one second, which means you need to set a very powerful password to crack it violently
vi edit redis.com Conf file, find the following to save and modify

requirepass [password]
Restart the server pkill redis-server
 Enter 127 again.0.01:6379>keys *
(error)NOAUTH Authentication required.
You will find that you do not have permission to query auth [password]
Enter the password to enter successfully
 Every time you enter, you need to enter password free,There is another simple way:

redis-cli -a [password]

3.3 redis transaction

Redis transaction is very simple and can be used as follows:
First, use the multi method to open the transaction, and then set it. At this time, the set data will be put into the queue for saving Finally, use exec to execute Store the data in redis in turn Use the discard method to cancel the transaction

3.4 redis persistence mechanism

Redis is an in memory database that supports persistence, which means that redis needs to synchronize the data in memory to the hard disk frequently to ensure persistence
There are two ways of Redis persistence:

3.4.1 RDB mode

Snapshot is the default mode Write in memory to binary file in the form of snapshot The default is dump rdb. You can configure and set the automatic snapshot persistence mode We can configure redis to automatically take a snapshot if there are more than m key s in n seconds

Snapshotting set up:
save 900 1  #If more than one Key is modified within 900 seconds, initiate snapshot saving
save 300 10 #If more than 10 key s are modified within 300 seconds, initiate snapshot saving
save 60 10000

3.4.2 AOF mode

The method of append only file (abbreviated as aof), because the snapshot method is to take a snapshot at a certain time interval, the unexpected down of reids may occur, and all modified data after the last snapshot will be lost aof has better persistence than snapshot because when using aof, redis will append every write command received to the command through the write function. When redis restarts, it will re execute the write command saved in the file to rebuild the contents of the database in memory This file is in the bin directory:

appendonly.aof

aof Not immediately written to hard disk,You can modify the configuration file to force writing to the hard disk.
aof set up:

appendonly yes //There are three ways to start aof persistence
#appendfsync always / / write to disk immediately after receiving the command, which is the slowest But it can ensure complete persistence
#appendfsync everysec / / write to disk once per second, which makes a good compromise between performance and persistence
#appendfsync no / / the best performance is to rely entirely on the os. Persistence is not guaranteed

3.5 Redis memory elimination mechanism and expired Key processing

https://www.cnblogs.com/maguanyue/p/12090414.html

Redis memory elimination mechanism and expired Key processing
"For a long time and sometimes, this hatred will last forever." Good poetry! What a poem! Even if it lasts forever, there will always be an end. Will Redis's memory run out sometimes? The answer is yes. So, when Redis's memory is full, what should we do to make new requests? At this time, you should understand Redis's memory elimination strategy. After understanding the relevant knowledge points, you can understand what will happen after "Redis memory sometimes runs out".

Redis memory elimination mechanism

Redis memory elimination mechanism means that when the memory usage reaches the upper limit (it can be configured through maxmemory, 0 is unlimited, i.e. the upper limit of server memory), it determines which data to eliminate according to a certain algorithm to ensure the storage of new data.

Common memory elimination mechanisms are divided into four categories:

1. LRU * *: * * LRU is Least recently used, which means to delete the least recently accessed data from the database. The algorithm believes that if you don't use the data for a long time, the probability of being accessed again is very small. The eliminated data is not used for the longest time, which is only related to time.

2. **LFU: * * LFU means Least Frequently Used. A simple understanding is to eliminate the data used the least times in a period of time, which is related to frequency and time.

3. **TTL: * * in Redis, some data has an expiration time set, and this part of data with an expiration time set is the object to be solved by the algorithm. If you're about to expire, I'm sorry. I don't have enough memory now. Anyway, you're going to retire. I'll give you a ride in advance and kill you.

4. Random elimination: life and death, wealth and honor are in heaven. Whether they are killed or not depends on God's will.

Specific elimination mechanisms can be configured through maxmemroy policy. After reading many articles on the Internet, it is said that there are only 6 kinds. In fact, there are 8 kinds. You can see redis5 0's configuration file with instructions:

1. volatile-lru => Find out the data set that has set the expiration time and kill the data that has been used least recently (accessed).
2. volatile-ttl => Find out the data set that has set the expiration time and kill the data that is about to expire.
3. volatile-random => Find out the data set that has set the expiration time, carry out indifference attack and kill the data randomly.
4. volatile-lfu => Find out the data set that has set the expiration time, and kill the data that has been used the least times in a period of time.
5. allkeys-lru =>Similar to the first one, the data set changes from setting the expiration time data to all data.
6. allkeys-lfu => Similar to the fourth, the data set changes from setting the expiration time data to all data.
7. allkeys-random => Similar to the third, the data set changes from setting the expiration time data to all data.
8. no-enviction => Do nothing, report an error, and tell you that there is insufficient memory. This advantage is to ensure that data is not lost. This is also the default elimination strategy of the system.

Redis expired Key clearing policy

In Redis, we will set the expiration time for the stored data. If these data expire, how does Redis eliminate them? Let's discuss it together. Here are three purge strategies:

**Lazy deletion: * * when accessing the key, you can judge whether it has expired. If it has expired, you can kill it directly. This method is CPU friendly, but it will waste memory for a long time.

**Scheduled deletion: * * while setting the expiration time of the Key, create a timer. When the expiration time point is reached, delete the Key immediately. This method is the most unfriendly.

**Delete regularly: * * check the data once every other period of time to delete the expired keys. As for how many expired keys to delete and how much data to check, it is up to the algorithm. Take an example to facilitate your understanding: Redis randomly takes 100 data per second for expiration inspection, and deletes all expired keys in the inspection data. If the expired keys account for more than 25% of the total, that is, more than 25, repeat the above inspection operation.

Redis server actually uses two strategies: lazy deletion and periodic deletion. By using these two deletion strategies together, we can strike a good balance between rational use of CPU and avoiding waste of memory.

OK, after the introduction of relevant knowledge, I hope this article can be helpful to you!

4. Practical application

4.1 basic use of jedis

Import Jedis related dependencies:

	<parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.4.3</version>
        <relativePath/>
    </parent>

   <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter</artifactId>
        </dependency>
    <dependency>
        <groupId>redis.clients</groupId>
        <artifactId>jedis</artifactId>
    </dependency>

   <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
   </dependency>

4.1.1 general API

@Test
public void testJedisPool() {
    // 1: Create Jedis connection pool
    JedisPool pool = new JedisPool("localhost", 6379);
    // 2: Get Jedis object from connection pool
    Jedis jedis = pool.getResource();
    /* Set password
	jedis.auth(Password); */
    // 3:TODO
    System.out.println(jedis);
    // 4: Close resource
    jedis.close();
    pool.destroy();
}
---------------------------------------------
	@Test
    public void testTool(){
        GenericObjectPoolConfig config = new GenericObjectPoolConfig();
        maximum connection, Default 8
        config.setMaxTotal(100);
        Maximum number of free connections, Default 8
        config.setMaxIdle(20);
        //Gets the maximum number of milliseconds to wait for a connection (if it is set to blockwhenexhausted when blocking). If it times out, throw an exception. It is less than zero: the blocking time is uncertain. The default is - 1
        config.setMaxWaitMillis(-1);
        //Check the validity when obtaining the connection. The default is false
        config.setTestOnBorrow(true);
        JedisPool pool = new JedisPool(config,"192.168.122.128",6379,5000,"wolfcode");
        Jedis j = pool.getResource();
        String name = j.get("name");
        System.out.println(name);
        j.close();
        pool.close();
        pool.destroy();
    }

4.2 integrated SpringBoot

Import dependency:

    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-data-redis</artifactId>
    </dependency>

   <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
   </dependency>

Configuration: application properties

spring.redis.host=127.0.0.1
spring.redis.port=6379
spring.redis.password=admin

General API:

@Autowired
private StringRedisTemplate redisTemplate;

@Test
public void testRedisTemplate() {
    // Operation string
    redisTemplate.opsForValue().xx();
    // Operation hash
    redisTemplate.opsForHash().xx();
    // Operation list
    redisTemplate.opsForList().xx();
    // Operation set
    redisTemplate.opsForSet().xx();
    // Operation zset
    redisTemplate.opsForZSet().xx();
}
-----------------------------------------
redisTemplate.opsForValue();//Operation string
redisTemplate.opsForHash();//Operation hash
redisTemplate.opsForList();//Operation list
redisTemplate.opsForSet();//Operation set
redisTemplate.opsForZSet();//Operation order set