Five basic data types of Redis

For redis, all key s are strings.

The basic data structure of Redis we usually talk about is the data type of stored value.

The five common data types in Redis are: String, List, Set, Zset and Hash.

structure type	Value stored by structure	Structure reading and writing ability
String string	Can be string, floating point number, integer	Operate on the whole string or part of the string; Self increment or self decrement of integers or floating-point numbers
List list	A linked list. Each node on the linked list contains a string	push and pop the two ends of the linked list, read single or multiple elements, and find or delete elements according to their values
Set set	An unordered collection containing strings	The collection of strings, including the basic methods: Yes, see whether there are add, obtain and delete; It also includes the functions of calculating intersection, union and difference sets
Hash hash table	Unordered hash table containing key value pairs	The methods include adding, obtaining and deleting a single element
Zset ordered set	Like hash tables, they are ordered and used to store key value pairs	Orderly mapping between string members and floating point number fractions; The arrangement order of elements is determined by the size of scores; It includes adding, getting, deleting individual elements and getting elements according to score range or members

String string

The String type in Redis is binary safe.

It can contain any data, such as numbers, strings, pictures, or serialized objects.

A string is composed of multiple bytes, and each byte is composed of 8 bits. In this way, a string can be regarded as a combination of many bits, which is the * * bitmap * * data structure.

The String type in Redis is a dynamic String and can be modified. The internal structure is actually similar to the arraylist of java. The method of pre allocating redundant space is adopted to reduce the frequent allocation of memory, as shown in the following figure:

• When the string length is less than 1M, the expansion is to double the existing space
• If it exceeds 1M, the expansion will only expand 1M more space at a time.

Note: the maximum length of the string is 512M.

The structure of string is widely used, and the most common use is to cache user information. We serialize the structure of user information into strings using JSON, and then insert the serialized strings into Redis for caching. Similarly, when retrieving information, the data will be deserialized once.

use

• Basic command

command	sketch
GET key_name	Gets the value corresponding to the specified key
SET key_name	Sets the value corresponding to the specified key
DEL key_name	Delete the value corresponding to the specified key
INCR key_name	Stores the value of the specified key + 1
DECR key_name	Stores the specified key with a value of - 1
INCRBY key_name amount	Adds an integer to the value stored in the specified key
DECRBY key_name amount	Subtracts an integer from the value stored in the specified key

• Key value pair

> set name codehole 
OK 
> get name 
"codehole"
> exists name 
(integer) 1 
> del name 
(integer) 1 
> get name 
(nil) 

• Batch key value pair

It can read and write multiple strings in batch, saving network time and overhead.

> set name1 codehole 
OK 
> set name2 holycoder 
OK 
> mget name1 name2 name3 # Returns a list
1) "codehole" 
2) "holycoder" 
3) (nil) 
> mset name1 boy name2 girl name3 unknown 
> mget name1 name2 name3 
1) "boy" 
2) "girl" 
3) "unknown"

• count

If our value is an integer at this time, we can also increase and decrease it automatically, but it has a range. The range is the maximum and minimum value of signed long. If it exceeds this value, redis will report an error.

> set age 30 
OK 
> incr age 
(integer) 31 
> incrby age 5 
(integer) 36 
> incrby age -5 
(integer) 31 
> set codehole 9223372036854775807 # Long.Max 
OK 
> incr codehole
(error) ERR increment or decrement would overflow # Error reporting memory overflow

There are also some set command extensions, which can set the expiration time of the key and automatically delete it at the point. I won't do it here. It's too troublesome.

Actual combat scene

Ownership of copyright https://pdai.tech All. Link: https://www.pdai.tech/md/db/nosql-redis/db-redis-data-types.html

• Cache:
  • In the classic usage scenario, common information, strings, pictures or videos are put into redis. Redis is used as the cache layer and mysql is used as the persistence layer to reduce the reading and writing pressure of mysql.
• Counter:
  • redis is a single thread model. The next command will be executed only after one command is executed. At the same time, the data can be landed to other data sources step by step.
• session:
  • Common solutions spring session + redis realize session sharing.

List list

List in Redis is a two-way linked list, which is equivalent to linkedlist in java.

When the last element pops up in the List, the data structure is automatically deleted and the memory is recycled. i

Redis's list structure is often used for asynchronous queues.

Serialize the task structure that needs to be processed later into a string and insert it into Redis's List. Another thread polls the data from this List for processing.

Using the List structure, we can relatively easily implement the latest message queuing function (such as TimeLine of sina Weibo). Therefore, another application of List is message queue. You can use the push operation of List to store tasks in the List, and then the working thread takes out the tasks for execution with pop operation.

use

• Basic command

command	sketch
LPUSH key value	Add the given value to the head of the linked list (left end)
RPUSH key value	Add the given value to the tail (right end) of the linked list
LPOP key	Pop up a value from the left end of the linked list and return the popped value
RPOP key	Pop up a value from the right end of the linked list and return the popped value
LRANGE key 0-x	Gets all the values of the list in the given range
LINDEX key index	Get the elements in the list by index. You can also use negative subscripts, with - 1 for the last element of the list, - 2 for the penultimate element of the list, and so on

• Right in left out: queue

> rpush books python java golang 
(integer) 3 
> llen books 
(integer) 3 
> lpop books 
"python" 
> lpop books 
"java" 
> lpop books 
"golang" 
> lpop books 
(nil)

• Right in right out: stack

> rpush books python java golang 
(integer) 3 
> rpop books 
"golang" 
> rpop books 
"java" 
> rpop books 
"python" 
> rpop books
(nil)

• Quick list

It is found that the underlying storage of Redis is not a simple linked list, but a quick linked list called quicklist.

In theory, when there are few list elements, a continuous memory will be used for storage. This structure is "compressed" into a ziplost structure in Redis, which can also be literally translated into a compressed list. It stores all elements next to each other and allocates a continuous memory. When there is a large amount of data, it will be changed into a quicklist structure.

Because the additional pointer space required by ordinary linked lists is too large, it will waste space and aggravate the fragmentation of memory. For example, only int type data is stored in this list, and two additional pointers prev and next are required in the structure. So Redis combines = = linked list and ziplist to form quicklist = =. That is to string multiple ziplists using bidirectional pointers.

This not only meets the fast insertion and deletion performance, but also does not appear too much spatial redundancy.

Actual combat scene

Ownership of copyright https://pdai.tech All. Link: https://www.pdai.tech/md/db/nosql-redis/db-redis-data-types.html

• Microblogging TimeLine:
  • Someone posted a microblog and added lpush to the timeline to display new list messages.
• Message queue

Set set

The collection in Redis is realized through hash table, which is equivalent to hashset in java. Therefore, the time complexity of operations such as adding, deleting and searching is O(1).

Its internal implementation is equivalent to a special dictionary. All value s in the dictionary are NULL.

When the last element in the collection is removed, the data structure is automatically deleted and the memory is recycled.

Because the set has the function of de duplication, the set structure can be envisaged to store the ID of the winning user in the lottery, which can ensure uniqueness, so that there will be no case that one person wins the lottery twice.

use

• Basic command

command	sketch
SADD key value	Add one or more members to the collection
SCARD key	Gets the number of members of the collection
SMEMBER key member	Returns all members of the collection
SISMEMBER key member	Judge whether the member element is a member of the set key

• Basic use

> sadd books python 
(integer) 1 
> sadd bookspython # repeat
(integer) 0 
> sadd books java golang 
(integer) 2 
> smembers books # Note that the order is not consistent with the inserted one, because the set is unordered
1) "java" 
2) "python" 
3) "golang" 
> sismember books java # Querying whether a value exists is equivalent to contains(o)
(integer) 1 
> sismember books rust 
(integer) 0 
> scard books # Get the length equivalent to count()
(integer) 3 
> spop books # Pop up a
"java" 

Actual combat scene

Ownership of copyright https://pdai.tech All. Link: https://www.pdai.tech/md/db/nosql-redis/db-redis-data-types.html

• Tags: add tags to users, or users add tags to messages, so that those with the same tag or similar tags can recommend things or people to pay attention to.
• Like, step, collect, etc. can be put into set.

Hash hash (Dictionary)

Redis's dictionary is equivalent to HashMap in Java language. It is an unordered dictionary.

The internal implementation structure is also consistent with the HashMap of Java. The same array + linked list two-dimensional structure. When the hash elements of the array are concatenated, the collision position of the first dimension of the array will be used.

The value of Redis's dictionary can only be a string. In addition, they rehash in different ways, because when the hashmap in java has a large dictionary, rehash is a time-consuming operation and needs to rehash all at once. In order to achieve high performance, Redis cannot block the service, so it adopts the progressive rehash strategy.

When the hash removes the last element, the data structure is automatically deleted and the memory is recycled.

Hash structure can also be used to store user information. Unlike the string, which needs to serialize the whole object at one time, hash can store each field in the user structure separately. In this way, when we need to obtain user information, we can obtain part of it. If you save user information in the form of an entire string, you can only read it all at once, which will waste network traffic.

Hash also has disadvantages. The storage consumption of hash structure is higher than that of a single string. Whether to use hash or string needs specific analysis.

use

• Basic command

command	sketch
HSET hash-key sub-key1 value1	Add key value pair
HGET hash-key key1	Gets the value of the specified hash key
HGETALL hash-key	Gets all key value pairs contained in the hash
HDEL hash-key sub-key1	If the given key exists in the hash, the key is removed

• Basic use

> hset books java "this is java" # If the string on the command line contains spaces, it should be enclosed in quotation marks
(integer) 1 
> hset books golang "this is go" 
(integer) 1 
> hset books python "this is python" 
(integer) 1 
> hgetall books # entries(), key and value intervals appear
1) "java" 
2) "this is java" 
3) "golang" 
4) "this is go" 
5) "python" 
6) "this is python" 
> hlen books 
(integer) 3 
> hget books java 
"this is java" 
> hset books golang "learning go programming" # 0 is returned because it is an update operation
(integer) 0 
> hget books golang "learning go programming" 
> hmset books java "effective java" python "learning python" golang "modern golang programming" # Batch set 
OK 

• count

A single key in the hash structure can also be counted.

> hincrby user age 1 
(integer) 30 

Actual combat scene

Ownership of copyright https://pdai.tech All. Link: https://www.pdai.tech/md/db/nosql-redis/db-redis-data-types.html

• Cache: it can be intuitive, save more space than string, and maintain cache information, such as user information, video information, etc.

Zset ordered set

Zset may be the most characteristic data structure provided by Redis. It is orderly based on Zset.

It is similar to the combination of Java SortedSet and HashMap. On the one hand, it is a set to ensure the uniqueness of the internal value. On the other hand, it can give each value a score to represent the sorting weight of the value.

Its internal implementation uses a data structure called = = "jump list" = =.

After the last value in zset is removed, the data structure is automatically deleted and the memory is recycled.

use

• Basic command

command	sketch
ZADD zset-key xxx member	Adds a member with a given score to an ordered set
ZRANGE zset-key 0-x withccores	According to the position of the element in the ordered set, multiple elements are obtained from the ordered set
ZREM zset-key member	If a given element member exists in an ordered collection, the element is removed

• Basic use

> zadd books 9.0 "this is java-3" 
(integer) 1 
> zadd books 8.9 "this is java-2" 
(integer) 1 
> zadd books 8.6 "this is java-1" 
(integer) 1 
> zrange books 0 -1 # Sort and list by score, and the parameter range is the ranking range
1) "this is java-1" 
2) "this is java-2" 
3) "this is java-3" 
> zrevrange books 0 -1 # It is listed in reverse order by score, and the parameter range is the ranking range
1) "this is java-3" 
2) "this is java-2" 
3) "this is java-1" 
> zcard books # Equivalent to count()
(integer) 3 
> zscore books "this is java-2" # Gets the score of the specified value
"8.9000000000000004" # The internal score is stored in double type, so there is a decimal point accuracy problem
> zrank books "this is java-2" # ranking
(integer) 1 
> zrangebyscore books 0 8.91 # Traverse zset according to the score interval
1) "this is java-1" 
2) "this is java-2" 
> zrangebyscore books -inf 8.91 withscores # Traverse zset according to the score interval (- ∞, 8.91] and return the score. inf stands for infinite.
1) "this is java-1" 
2) "8.5999999999999996" 
3) "this is java-2" 
4) "8.9000000000000004" 
> zrem books "this is java-2" # Delete value
(integer) 1 
> zrange books 0 -1 
1) "this is java-1" 
2) "this is java-3"

Actual combat scene

Ownership of copyright https://pdai.tech All. Link: https://www.pdai.tech/md/db/nosql-redis/db-redis-data-types.html

Redis deep Adventure: core principles and application practice - Qian wenpin

• Fan list: value is the user ID of fans, and score is the time of interest. We can sort the fan list according to the time of interest.
• Score: value is the student's ID and score is his test score. We can sort his grades by score to get his ranking.
• Leaderboard: an orderly collection of classic usage scenarios. For example, websites such as novel videos need to make a ranking list of novel videos uploaded by users. The list can be scored and ranked according to the number of users' attention, update time, number of words, etc.

Special types in Redis 3

Hyperlogs (cardinality Statistics)

What is it?

It allows fault tolerance and can accept certain errors

IF A = {1,2,3,4,5} B = {3,5,6,7,8} SO cardinality (non repeating elements) = {1,2,4,6,7,8}

What problems can be solved?

Under the condition of saving memory, he can count various counts, such as the number of registered IP, the number of daily access IP, page real-time UV, the number of online users, the number of common friends, etc.

What are the advantages?

For a large website, for example, there are 1 million IPS per day. Roughly calculate that an IP consumes 15 bytes, then 1 million IPS is 15M. The content occupied by each key of HyperLogLog in Redis is 12K, and the theoretical storage is close to 2 ^ 64 values. No matter what the stored content is, it is an algorithm based on cardinality estimation, which can only accurately estimate the cardinality, and can use a small amount of fixed memory to store and identify the only element in the set. Moreover, the cardinality of this estimation is not necessarily accurate. It is an approximate value with 0.81% standard error (it can be ignored for business scenarios that can accept certain fault tolerance, such as IP statistics, UV, etc.).

Ownership of copyright https://pdai.tech All. Link: https://www.pdai.tech/md/db/nosql-redis/db-redis-data-type-special.html?

Basic command usage

127.0.0.1:6379> pfadd key1 a b c d e f g h i	# Create the first set of elements
(integer) 1
127.0.0.1:6379> pfcount key1	# Count the cardinality number of elements
(integer) 9
127.0.0.1:6379> pfadd key2 a b c d e f g h	# Create a second set of elements
(integer) 1
127.0.0.1:6379> pfcount key2
(integer) 8
127.0.0.1:6379> pfmerge key3 key1 key2	# Merge two groups: key1, key2 - > Key3 Union
OK
127.0.0.1:6379> pfcount key3 # Count the number of union cardinality of elements
(integer) 8

Bitmap (bitmap)

During our normal development process, there will be some Boolean data that need to be accessed, such as attendance check-in record, which is 1 if signed, 0 if not signed, and 365 days to be recorded. If you use an ordinary key/value, each user needs to record 365. When users reach hundreds of millions, the storage space required is amazing. In order to solve this problem, Redis provides a bitmap data structure, so that the daily check-in record only occupies one bit, 365 days is 365 bits, and 46 bytes of face space is occupied, that is, a slightly longer string can be fully accommodated, which greatly saves storage space.

It is not a special data structure. Its content is an ordinary string, much like a string in java. It is a byte array.

basic operation

The digit group of Redis is automatically extended. If it exceeds the range, the bitmap array will be automatically zero extended.

127.0.0.1:6379> setbit sign 0 1 
(integer) 0
127.0.0.1:6379> setbit sign 1 1
(integer) 0
127.0.0.1:6379> setbit sign 2 0
(integer) 0
127.0.0.1:6379> setbit sign 3 1
(integer) 0
127.0.0.1:6379> setbit sign 4 0
(integer) 0
127.0.0.1:6379> setbit sign 5 0 # Here's the clock out for the whole week.
(integer) 0
127.0.0.1:6379> getbit sign 3 # Check whether the specified days are clocked in!
(integer) 1
127.0.0.1:6379> getbit sign 5
(integer) 0
127.0.0.1:6379> bitcount sign # By counting the clock in records of this week, you can see whether there is full attendance!
(integer) 3

127.0.0.1:6379> set w hello
OK
127.0.0.1:6379> bitfield w get u4 0 # Take 4 bits from the first bit and the result is an unsigned number (u)
(integer) 6
127.0.0.1:6379> bitfield w get u3 2 # Take 3 bits from the third bit and the result is an unsigned number (u)
(integer) 5
127.0.0.1:6379> bitfield w get i4 0 # Take 4 bits from the first bit, and the result is the signed number (i)
1) (integer) 6
127.0.0.1:6379> bitfield w get i3 2 # Take 3 bits from the third bit, and the result is the signed number (i)
1) (integer) -3

Code cloud warehouse synchronization notes can be taken by yourself. Welcome to star for correction: https://gitee.com/noblegasesgoo/notes

If something goes wrong, I hope the leaders in the comment area can discuss and correct each other, maintain the health of the community, and everyone can contribute together. We can't have the knowledge of tolerating mistakes.
										----------- Love you noblegasesgoo