1, Five basic data structures

1. Introduction to redis

“Redis is an open source (BSD licensed), in-memory data structure store, used as a database, cache and message broker.” —— Redis is an open source (BSD licensed) in memory data structure storage, which is used as database, cache and message agent. (from the official website)

Redis is an open source, advanced key value storage and an applicable solution for building high-performance and scalable Web applications. Redis is also jokingly called data structure server by the author, which means that users can access a set of variable data structures through some commands based on the simple server client protocol with TCP socket. (key value pairs are used in redis, but the corresponding data structures are different)

1) Advantages of redis

Here are some advantages of Redis:

• Unusually fast - Redis is very fast. It can perform about 110000 set operations per second and about 81000 read / get operations per second.
• Support rich data types - Redis supports most data types commonly used by developers, such as list, set, sorting set, hash, etc. This makes Redis easy to be used to solve various problems, because we know which problems can be better solved by using which data types.
• Operations are atomic - all Redis operations are atomic, which ensures that if two clients access concurrently, the Redis server can receive updated values.
• Multi utility - Redis is a multi utility that can be used for a variety of use cases, such as caching, message queuing (Redis supports publish / subscribe locally), any short-term data in applications, such as sessions in web applications, web page hit counts, etc.

2)Redis installation

This step is relatively simple. You can find many satisfactory tutorials on the Internet, which will not be repeated here.

The installation tutorial for a rookie tutorial is used as a reference: https://www.runoob.com/redis/redis-install.html

3) Test local Redis performance

After the installation is completed, you can first run Redis server to start Redis, and then run the command redis- benchmark -n 100000 -q to detect the performance when 100000 requests are executed locally at the same time:

Of course, there will be performance gaps between different computers for various reasons. You can take this test as a kind of "fun".

2. Five basic data structures of redis

Redis has five basic data structures: string, list, hash, set and Zset. These five are the most basic and important parts of redis related knowledge. Let's explain them to you in combination with the source code and some practices.

be careful:

Each data structure has its own underlying internal coding implementation, and there are multiple implementations, so Redis will choose the appropriate internal coding in the appropriate scenario.
You can see that each data structure has more than two internal coding implementations. For example, the string data structure includes three internal codes: raw, int and embstr.
At the same time, some internal codes can be used as internal implementations of various external data structures. For example, ziplist is the internal code shared by hash, list and zset.

1) String string

The string in Redis is a dynamic string, which means that users can modify it. Its underlying implementation is a bit similar to ArrayList in Java. It has a character array from SDS of the source code In the H / sdshdr file, you can see the definition SDS of the string at the bottom of Redis, that is, the Simple Dynamic String structure:

/* Note: sdshdr5 is never used, we just access the flags byte directly. 
	* However is here to document the layout of type 5 SDS strings. */ 
struct __attribute__ ((__packed__)) sdshdr5 { 
	unsigned char flags; /* 3 lsb of type, and 5 msb of string length */ 
	char buf[]; 
};
struct __attribute__ ((__packed__)) sdshdr8 { 
	uint8_t len; /* used */ 
	uint8_t alloc; /* excluding the header and null terminator */ 
	unsigned char flags; /* 3 lsb of type, 5 unused bits */ 
	char buf[]; 
};
struct __attribute__ ((__packed__)) sdshdr16 { 
	uint16_t len; /* used */ 
	uint16_t alloc; /* excluding the header and null terminator */ 
	unsigned char flags; /* 3 lsb of type, 5 unused bits */ 
	char buf[]; 
};
struct __attribute__ ((__packed__)) sdshdr32 { 
	uint32_t len; /* used */ 
	uint32_t alloc; /* excluding the header and null terminator */ 
	unsigned char flags; /* 3 lsb of type, 5 unused bits */ 
	char buf[]; 
};
struct __attribute__ ((__packed__)) sdshdr64 { 
	uint64_t len; /* used */ 
	uint64_t alloc; /* excluding the header and null terminator */ 
	unsigned char flags; /* 3 lsb of type, 5 unused bits */ 
	char buf[]; 
};

You will find that Redis has used generics to define the same set of structures many times. Why not use int type directly?

Because when the string is relatively short, len and alloc can be represented by byte and short. In order to optimize the memory, Redis uses different structures for strings of different lengths.

① The difference between SDS and C string

Why not consider using C language strings directly? Because the simple string representation of C language does not meet Redis's requirements for string security, efficiency and function. As we know, C language uses a character array with a length of N+1 to represent a string with a length of N, and the last element of the character array is always' \ 0 '. (the following figure shows a character array with the value of "Redis" in C language)

Such a simple data structure may cause the following problems:

• Obtain the operation with the string length of O(N) → because C does not save the length of the array, it needs to traverse the whole array every time;
• The problem of buffer overflow / memory leakage cannot be eliminated well → the reason for the above problem is the same. If the string is spliced or shortened, the above problem can be easily caused if the operation is improper;
• C string can only save text data → because the string in C language must comply with some coding (such as ASCII), for example, '\ 0' appearing in the middle may be judged as a string ending in advance and cannot be recognized;

We take the operation of appending strings as an example. Redis source code is as follows:

/* Append the specified binary-safe string pointed by 't' of 'len' bytes to the 
	* end of the specified sds string 's'. 
	*
	* After the call, the passed sds string is no longer valid and all the 
	* references must be substituted with the new pointer returned by the call. */ 
sds sdscatlen(sds s, const void *t, size_t len) { 
	// Gets the length of the original string 
	size_t curlen = sdslen(s); 
	
	// Adjust the space as needed. If the capacity is not enough to accommodate the additional contents, the byte array will be reallocated and the contents of the original string will be copied to the new array 
	s = sdsMakeRoomFor(s,len); 
	if (s == NULL) return NULL; // insufficient memory 
	memcpy(s+curlen, t, len); // Append target string to byte array 
	sdssetlen(s, curlen+len); // Set the length after appending 
	s[curlen+len] = '\0'; // End the string with \ 0 for debugging and printing 
	return s; 
}

• Note: Redis stipulates that the length of the string shall not exceed 512 MB.

② Basic operation of string

After installing Redis, we can use Redis cli to operate Redis on the command line. Of course, Redis official also provides an online debugger. You can also type in the command to operate: http://try.redis.io/#run

③ , set and get key value pairs

> SET key value 
OK
> GET key 
"value"

As you can see, we usually use SET and GET to SET and GET string values.

The value can be any kind of string (including binary data). For example, you can save one under a key jpeg images, just pay attention not to exceed the maximum limit of 512 MB.

When the key exists, the SET command will overwrite the last value you SET:

> SET key newValue 
OK
> GET key 
"newValue"

In addition, you can also use EXISTS and DEL keywords to query whether key value pairs exist and delete them:

> EXISTS key 
(integer) 1 
> DEL key 
(integer) 1 
> GET key 
(nil)

④ . batch setting key value pairs

> SET key1 value1 
OK
> SET key2 value2 
OK
> MGET key1 key2 key3 # Returns a list 
1) "value1" 
2) "value2" 
3) (nil) 
> MSET key1 value1 key2 value2 
> MGET key1 key2 
1) "value1" 
2) "value2"

⑤ , expiration, and SET command extensions

You can set the expiration time for the key, and it will be automatically deleted when it arrives. This function is often used to control the expiration time of the cache. (expiration can be any data structure)

> SET key value1 
> GET key 
"value1" 
> EXPIRE name 5 # Expires after 5s 
... # Wait for 5s 
> GET key 
(nil)

SETEX command equivalent to set + Express:

> SETEX key 5 value1 
... # Wait for 5s to get 
> GET key 
(nil) 

> SETNX key value1 		# If the key does not exist, SET succeeds 
(integer) 1 
> SETNX key value1 		# If the key exists, SET fails 
(integer) 0 
> GET key 
> "value" 				# No change

⑥ , counting

If value is an integer, you can also use the INCR command to perform atomic auto increment operation on it, which means that if multiple customers operate on the same key in time, it will never lead to competition:

> SET counter 100 
> INCR counter 
(integer) 101 
> INCRBY counter 50 
(integer) 151

⑦ . GETSET command that returns the original value

For strings, there is another interesting GETSET. Its function is the same as its name: set a value for key and return the original value:

> SET key value 
> GETSET key value1 
"value"

This can easily set and view some keys that need to be counted at intervals. For example, whenever the system is entered by the user, you use the INCR command to operate a key. When statistics are needed, you use the GETSET command to re assign this key to 0, so as to achieve the purpose of Statistics.

2) List list

Redis's list is equivalent to LinkedList in Java language. Note that it is a linked list rather than an array. This means that the insertion and deletion operations of the list are very fast, and the time complexity is O(1), but the index positioning is very slow, and the time complexity is O(n).

We can start from the source adlist H / listnode to see its definition:

/* 
Node, List, and Iterator are the only data structures used currently. */

typedef struct listNode { 
	struct listNode *prev; 
	struct listNode *next; 
	void *value; 
} listNode; 

typedef struct listIter { 
	listNode *next; 
	int direction; 
} listIter;

typedef struct list { 
	listNode *head; 
	listNode *tail; 
	void *(*dup)(void *ptr); 
	void (*free)(void *ptr); 
	int (*match)(void *ptr, void *key); 
	unsigned long len; 
} list;

It can be seen that multiple LISTNODES can form a two-way linked list through prev and next pointers:

Although only multiple listNode structures can be used to form a linked list, adlist If the linked list is held by H / list structure, the operation will be more convenient:

① Basic operation of linked list

• LPUSH and RPUSH can add a new element to the left (head) and right (tail) of the list respectively;
• The LRANGE command can extract a certain range of elements from the list;
• The LINDEX command can take out the elements of the specified table from the list, which is equivalent to the get(int index) operation in the Java linked list operation;

Demonstration:

> rpush mylist A 
(integer) 1 
> rpush mylist B 
(integer) 2 
> lpush mylist first 
(integer) 3 
> lrange mylist 0 -1 # -1 represents the penultimate element. Here, it represents all elements from the first element to the last element
1) "first" 
2) "A" 
3) "B"

② list implementation queue

Queue is a first in first out data structure, which is commonly used in message queuing and asynchronous logical processing. It will ensure the access order of elements:

> RPUSH books python java golang 
(integer) 3 
> LPOP books 
"python" 
> LPOP books 
"java" 
> LPOP books 
"golang" 
> LPOP books 
(nil)

③ list implementation stack

Stack is a first in and last out data structure, just opposite to queue:

> RPUSH books python java golang 
> RPOP books 
"golang" 
> RPOP books 
"java" 
> RPOP books 
"python" 
> RPOP books 
(nil)

3) Dictionary hash

The dictionary in Redis is equivalent to the HashMap in Java, and its internal implementation is similar. It solves some hash conflicts through the chain address method of "array + linked list". At the same time, this structure also absorbs the advantages of two different data structures. The source code is defined as dict.h/dictht:

typedef struct dictht { 
	// Hash table array 
	dictEntry **table; 
	// Hash table size 
	unsigned long size; 
	// Hash table size mask, used to calculate index value, always equal to size - 1 
	unsigned long sizemask; 
	// The number of existing nodes in the hash table 
	unsigned long used;
} dictht;

typedef struct dict { 
	dictType *type; 
	void *privdata; 
	// There are two dictht structures inside 
	dictht ht[2]; 
	long rehashidx; /* rehashing not in progress if rehashidx == -1 */ 
	unsigned long iterators; /* number of iterators currently running */ 
} dict;

The table attribute is an array. Each element in the array is a pointer to the dict.h/dictEntry structure, and each dictEntry structure holds a key value pair:

typedef struct dictEntry { 
	// key 
	void *key; 
	// value 
	union { 
		void *val; 
		uint64_t u64; 
		int64_t s64; 
		double d; 
	} v; 
	// Point to the next hash table node to form a linked list 
	struct dictEntry *next; 
} dictEntry;

As can be seen from the above source code, in fact, the dictionary structure contains two hashtables. Generally, only one hashtable is valuable. However, when the dictionary is expanded or shrunk, it is necessary to allocate a new hashtable and then move it gradually (the reasons are described below).

① Progressive rehash

The expansion of the large dictionary is time-consuming. It is necessary to re apply for a new array, and then re attach all the elements in the linked list of the old dictionary to the new array. This is an O(n) level operation. As a single thread, Redis is difficult to bear such a time-consuming process. Therefore, Redis uses progressive rehash for small-step relocation:

Progressive rehash will retain the old and new hash structures while rehash. As shown in the above figure, two hash structures will be queried at the same time during query, and then the contents of the old dictionary will be gradually migrated to the new dictionary in subsequent scheduled tasks and hash operation instructions. When the relocation is completed, a new hash structure will be used instead.

② Conditions for expansion and contraction

Under normal circumstances, when the number of elements in the hash table is equal to the length of the first dimensional array, it will start to expand. The expanded new array is twice the size of the original array. However, if Redis is doing bgsave (persistence command), Redis will try not to expand the capacity in order to reduce excessive separation of memory. However, if the hash table is very full and reaches five times the length of the first dimensional array, it will be forced to expand the capacity at this time.

When the hash table becomes more and more sparse because the elements are gradually deleted, Redis will shrink the hash table to reduce the space occupation of the first dimensional array of the hash table. The condition used is that the number of elements is less than 10% of the length of the array. Whether Redis is doing bgsave will not be considered for volume reduction.

③ Basic operation of dictionary

Hash also has disadvantages. The storage consumption of hash structure is higher than that of a single string. Therefore, whether to use hash or string needs to be weighed again and again according to the actual situation:

> HSET books java "think in java" 		# If the string on the command line contains spaces, it needs to be wrapped in quotation marks 
(integer) 1 
> HSET books python "python cookbook" 
(integer) 1 
> HGETALL books # The key and value intervals appear 
1) "java" 
2) "think in java" 
3) "python" 
4) "python cookbook" 
> HGET books java 
"think in java" 
> HSET books java "head first java" 
(integer) 0 						# 0 is returned because it is an update operation 
> HMSET books java "effetive java" python "learning python" 		# Batch operation OK

4) Set set

Redis's collection is equivalent to HashSet in Java language. Its internal key value pairs are unordered and unique. Its internal implementation is equivalent to a special dictionary. All values in the dictionary are NULL.

① Basic use of set

Since the structure is relatively simple, let's take a look at how it is used:

> SADD books java 
(integer) 1 
> SADD books java			 # repeat 
(integer) 0 
> SADD books python golang 
(integer) 2 
> SMEMBERS books 			# Note the order. set is out of order 
1) "java" 
2) "python" 
3) "golang" 
> SISMEMBER books java 		# Query whether a value exists, which is equivalent to contains 
(integer) 1 
> SCARD books 				# Get length 
(integer) 3 
> SPOP books 				# Pop up a 
"java"

5) zset with sequence table

This may be the most distinctive data structure of Redis. It is similar to the combination of SortedSet and HashMap in Java. On the one hand, it is a set to ensure the uniqueness of internal value. On the other hand, it can give each value a score value to represent the weight of sorting.

Its internal implementation uses a data structure called "jump table". Due to its complexity, it is good to simply mention the principle here:

Imagine you are the boss of a start-up company. At the beginning, there are only a few people, and everyone is on an equal footing. Later, with the development of the company, there were more and more people, and the team communication cost gradually increased. The team leader system was gradually introduced to divide the team, so some people were both employees and team leaders.

Later, the scale of the company further expanded, and the company needs to enter another level: Department. Therefore, each department will elect a minister from the group leader.

The jump table is similar to such a mechanism. All the elements in the lowest layer will be concatenated. They are all employees. Then a representative will be selected every few elements, and then these representatives will be concatenated with another level of pointer. Then select secondary representatives from these representatives and string them together. Finally formed a pyramid structure.

Think about your current geographical location: Asia > China > a province > a city >... That's such a structure!

① . basic operation of zset with sequence table

> ZADD books 9.0 "think in java" 
> ZADD books 8.9 "java concurrency" 
> ZADD books 8.6 "java cookbook" 

> ZRANGE books 0 -1 # Sort and list by score, and the parameter range is the ranking range 
1) "java cookbook" 
2) "java concurrency" 
3) "think in java" 

> ZREVRANGE books 0 -1 # It is listed in reverse order by score, and the parameter range is the ranking range 
1) "think in java" 
2) "java concurrency" 
3) "java cookbook" 

> ZCARD books # Equivalent to count() 
(integer) 3 

> ZSCORE books "java concurrency" # Gets the score of the specified value 
"8.9000000000000004" # The internal score is stored in double type, so there is a decimal point accuracy problem

> ZRANK books "java concurrency" # ranking 
(integer) 1 

> ZRANGEBYSCORE books 0 8.91 # Traverse zset according to the score interval 
1) "java cookbook" 
2) "java concurrency" 

> ZRANGEBYSCORE books -inf 8.91 withscores # Traverse zset according to the score interval (- ∞, 8.91] and return the score. inf stands for infinite. 
1) "java cookbook" 
2) "8.5999999999999996" 
3) "java concurrency" 
4) "8.9000000000000004" 

> ZREM books "java concurrency" # Delete value 
(integer) 1 
> ZRANGE books 0 -1 
1) "java cookbook" 
2) "think in java"

2, Jump table

1. Introduction to jump table

Skip list is a randomized data structure, which was proposed by William Pugh in his paper "Skip lists: a probabilistic alternative to balanced trees". It is a hierarchical linked list structure comparable to the balanced tree - operations such as searching, deleting and adding can be completed in logarithmic expected time. The following is a typical example of a skip list:

In the previous article, we mentioned that among the five basic structures of Redis, there is a data structure called zset with sequence table, which is similar to the combination of SortedSet and HashMap in Java. On the one hand, it is a set that ensures the uniqueness of internal values, on the other hand, it can give each value a sorting weight value score to achieve the purpose of sorting.

Its internal implementation relies on a data structure called * * jump list * *.

1) Why use jump tables

First of all, because zset needs to support random insertion and deletion, it is not suitable to use arrays to implement it. As for sorting, we can easily think of tree structures such as red black tree / balanced tree. Why doesn't Redis use such structures?

1. Performance considerations: in the case of high concurrency, the tree structure needs to perform some operations similar to rebalance, which may involve the whole tree. Relatively speaking, the change of the jump table only involves parts (described in detail below);
2. Implementation considerations: when the complexity is the same as that of red black tree, the implementation of jump table is simpler and looks more intuitive;

Based on the above considerations, Redis adopts the structure of jump table after making some improvements based on William Pugh's paper.

2) The essence is to solve the problem

Let's first look at a common linked list structure:

We need to sort the linked list according to the score value, which means that when we need to add new elements, we need to locate to the insertion point, so that we can continue to ensure that the linked list is orderly. Usually, we use the binary search method, but the binary search has an ordinal group, and the linked list cannot locate the position, There seems to be no better way for us except to traverse the whole and find the first node larger than the given data (time complexity O(n)).

However, if we add a pointer between every two adjacent nodes to point to the next node, as shown in the following figure:

In this way, all new pointers are connected into a new linked list, but it contains only half of the original data (3 and 11 in the figure).

Now suppose that when we want to find data, we can find it according to this new linked list. If we encounter a node larger than the data to be found, we can go back to the original linked list for search. For example, we want to find 7, and the search path is along the direction pointed by the red pointer marked in the figure below:

This is a slightly extreme example, but we can still see that through the newly added pointer search, we no longer need to compare each node on the linked list one by one. In this way, the number of nodes to be compared after improvement is about half of the original.

In the same way, we can continue to add a pointer to every two adjacent nodes on the newly generated linked list, so as to generate the third layer linked list:

In this new three-tier linked list structure, we try to find 13. The first comparison along the top-level linked list is 11. It is found that 11 is smaller than 13, so we know that we only need to continue to find after 11, so we skip all the nodes in front of 11 at once.

It is conceivable that when the linked list is long enough, such a multi-layer linked list structure can help us skip many lower nodes, so as to speed up the efficiency of search.

3) Further jump table

The skip list is inspired by this multi-layer linked list structure. According to the way of generating the linked list above, the number of nodes in each layer of the linked list above is half of the number of nodes in the lower layer. In this way, the search process is very similar to a binary search, so that the time complexity of the search can be reduced to O(logn).

However, this method has great problems in inserting data. After a new node is inserted, it will disrupt the strict 2:1 correspondence of the number of nodes in the upper and lower adjacent linked lists. If you want to maintain this correspondence, you must readjust all the nodes behind the newly inserted nodes (including the newly inserted nodes), which will degenerate the time complexity into O(n). Deleting data has the same problem.

In order to avoid this problem, skiplist does not require strict correspondence between the number of nodes in the upper and lower adjacent linked lists, but randomly selects a level for each node. For example, if the random number of layers of a node is 3, then link it to the three-layer linked list from layer 1 to layer 3. For clarity, the following figure shows how to form a skiplist through step-by-step insertion:

From the above creation and insertion process, we can see that the levels of each node are randomly selected, and the new insertion of a node will not affect the levels of other nodes. Therefore, the insertion operation only needs to modify the pointers before and after the node, rather than adjusting multiple nodes, which reduces the complexity of the insertion operation.

Now let's assume that we find the nonexistent number 23 from the structure we just created, and the search path will be as follows:

2. Implementation of jump table

The jump table in Redis is managed by server H / zskiplistNode and server H / zskiplist are two structure definitions. The former is a jump table node, and the latter saves the relevant information of the jump node. Similar to the previous set list structure, in fact, only zskiplistNode can be implemented, but the latter is introduced for more convenient operation:

/* ZSETs use a specialized version of Skiplists */ 
typedef struct zskiplistNode { 
	// value 
	sds ele; 
	// Score 
	double score; 
	// Backward pointer 
	struct zskiplistNode *backward; 
	// layer 
	struct zskiplistLevel { 
		// Forward pointer 
		struct zskiplistNode *forward; 
		// span 
		unsigned long span; 
	} level[]; 
} zskiplistNode; 

typedef struct zskiplist { 
	// Skip pointer 
	struct zskiplistNode *header, *tail; 
	// Number of nodes in the table 
	unsigned long length; 
	// The number of layers of the node with the largest number of layers in the table 
	int level; 
} zskiplist;

Just like the standard jump table drawn at the beginning of the article.

1) Random layers

For each newly inserted node, you need to call a random algorithm to assign a reasonable number of layers to it. The source code is in t_ zset. Defined in C / zslrandomlevel (void):

int zslRandomLevel(void) { 
	int level = 1; 
	while ((random()&0xFFFF) < (ZSKIPLIST_P * 0xFFFF)) 
		level += 1; 
	return (level<ZSKIPLIST_MAXLEVEL) ? level : ZSKIPLIST_MAXLEVEL; 
}

Intuitively, the expected goal is that 50% probability is assigned to Level 1, 25% probability is assigned to Level 2, 12.5% probability is assigned to Level 3, and so on... 2-63 probability is assigned to the top layer, because the promotion rate of each layer here is 50%.

By default, the maximum number of layers allowed for Redis jump table is 32, which is zskiplist in the source code_ MAXLEVEL defines that level 32 can only be reached when there are 264 elements in Level[0], so the definition of 32 is completely sufficient.

2) Create jump table

This process is relatively simple. In the source code, t_ zset. Defined in C / zslcreate:

zskiplist *zslCreate(void) { 
	int j; 
	zskiplist *zsl; 
	
	// Request memory space 
	zsl = zmalloc(sizeof(*zsl)); 
	// The number of initialization layers is 1 
	zsl->level = 1; 
	// Initialization length is 0 
	zsl->length = 0; 
	// Create a skip header node with 32 levels, 0 score and no value 
	zsl->header = zslCreateNode(ZSKIPLIST_MAXLEVEL,0,NULL); 
	
	// Jump header node initialization 
	for (j = 0; j < ZSKIPLIST_MAXLEVEL; j++) { 
		// Set all forward pointers of the skip header node to NULL 
		zsl->header->level[j].forward = NULL; 
		// Set all span s of the skip header node to 0 
		zsl->header->level[j].span = 0; 
	}
	// The backward pointer of the jump header node is set to NULL 
	zsl->header->backward = NULL; 
	// The pointer from the header to the jump footer node is set to NULL 
	zsl->tail = NULL; 
	return zsl; 
}

That is, after execution, an initialization jump table with the following structure is created:

3) Insert node implementation

This is almost the most important piece of code, but the overall idea is also relatively clear and simple. If you understand the principle of the jump list mentioned above, it is easy to sort out several actions that occur when inserting nodes (almost similar to the linked list):

1. Find the current position I need to insert (including the processing of the same score);
2. Create a new node, adjust the pointer before and after to complete the insertion;

In order to facilitate reading, I put the source code t_ zset. The insertion function defined by C / zslinsert is divided into several parts

Part I: declare the variables to be stored

// Store search path 
zskiplistNode *update[ZSKIPLIST_MAXLEVEL], *x; 
// Storage node span 
unsigned int rank[ZSKIPLIST_MAXLEVEL]; 
int i, level;

Part II: search the insertion position of the current node

serverAssert(!isnan(score)); 
x = zsl->header; 
// Step down to find the target node and get the "search path" 
for (i = zsl->level-1; i >= 0; i--) { 
	/* store rank that is crossed to reach the insert position */ 
	rank[i] = i == (zsl->level-1) ? 0 : rank[i+1]; 
	// If the score s are equal, you also need to compare the value value 
	while (x->level[i].forward && 
			(x->level[i].forward->score < score || 
				(x->level[i].forward->score == score && 
					sdscmp(x->level[i].forward->ele,ele) < 0))) 
	{ 
		rank[i] += x->level[i].span; 
		x = x->level[i].forward; 
	}
	// Record "search path" 
	update[i] = x; 
}

Discussion: in an extreme case, all score values in the jump table are the same. Will the lookup performance of zset degrade to O(n)?

From the above source code, we can find that the sorting elements of zset not only look at the score value, but also compare the value value (string comparison)

Part 3: generate insert node

/* we assume the element is not already inside, since we allow duplicated 
	* scores, reinserting the same element should never happen since the 
	* caller of zslInsert() should test in the hash table if the element is 
	* already inside or not. */ 
level = zslRandomLevel(); 
// If the randomly generated level exceeds the current maximum level, you need to update the information of the jump table 
if (level > zsl->level) { 
	for (i = zsl->level; i < level; i++) { 
		rank[i] = 0; 
		update[i] = zsl->header; 
		update[i]->level[i].span = zsl->length; 
	}
	zsl->level = level; 
}
// Create a new node 
x = zslCreateNode(level,score,ele);

Part IV: rearrange the forward pointer

for (i = 0; i < level; i++) { 
	x->level[i].forward = update[i]->level[i].forward; 
	update[i]->level[i].forward = x; 
	
	/* update span covered by update[i] as x is inserted here */ 
	x->level[i].span = update[i]->level[i].span - (rank[0] - rank[i]); 
	update[i]->level[i].span = (rank[0] - rank[i]) + 1; 
}

/* increment span for untouched levels */ 
for (i = level; i < zsl->level; i++) { 
	update[i]->level[i].span++; 
}

Part V: rearranging backward pointers and returning

x->backward = (update[0] == zsl->header) ? NULL : update[0]; 
if (x->level[0].forward) 
	x->level[0].forward->backward = x; 
else
	zsl->tail = x; 
zsl->length++; 
return x;

4) Node deletion implementation

The deletion process is controlled by T in the source code_ zset. The definition of C / zsldeletenode is similar to the insertion process. You need to find out the "search path" first, and then rearrange the forward and backward pointers for the relevant nodes of each layer. At the same time, you should also pay attention to updating the maxLevel of the highest layer and directly put the source code (it is easy to understand if you understand the insertion):

/* Internal function used by zslDelete, zslDeleteByScore and zslDeleteByRank */ 
void zslDeleteNode(zskiplist *zsl, zskiplistNode *x, zskiplistNode **update) { 
	int i; 
	for (i = 0; i < zsl->level; i++) {
		if (update[i]->level[i].forward == x) { 
			update[i]->level[i].span += x->level[i].span - 1; 
			update[i]->level[i].forward = x->level[i].forward; 
		} else { 
			update[i]->level[i].span -= 1; 
		} 
	}
	if (x->level[0].forward) { 
		x->level[0].forward->backward = x->backward; 
	} else { 
		zsl->tail = x->backward; 
	}
	while(zsl->level > 1 && zsl->header->level[zsl->level-1].forward == NULL) zsl->level--; 
	zsl->length--; 
}

/* Delete an element with matching score/element from the skiplist. 
	* The function returns 1 if the node was found and deleted, otherwise 
	* 0 is returned. 
	*
	* If 'node' is NULL the deleted node is freed by zslFreeNode(), otherwise 
	* it is not freed (but just unlinked) and *node is set to the node pointer, 
	* so that it is possible for the caller to reuse the node (including the 
	* referenced SDS string at node->ele). */

int zslDelete(zskiplist *zsl, double score, sds ele, zskiplistNode **node) { 
	zskiplistNode *update[ZSKIPLIST_MAXLEVEL], *x; 
	int i;
	
	x = zsl->header; 
	for (i = zsl->level-1; i >= 0; i--) { 
		while (x->level[i].forward && 
				(x->level[i].forward->score < score || 
					(x->level[i].forward->score == score && 
						sdscmp(x->level[i].forward->ele,ele) < 0))) 
		{ 
			x = x->level[i].forward; 
		}
		update[i] = x; 
	}
	/* We may have multiple elements with the same score, what we need 		
	 * is to find the element with both the right score and object. */ 
	x = x->level[0].forward; 
	if (x && score == x->score && sdscmp(x->ele,ele) == 0) { 
		zslDeleteNode(zsl, x, update); 
		if (!node) 
			zslFreeNode(x); 
		else
			*node = x; 
		return 1; 
	}
	return 0; /* not found */ 
}

5) Node update implementation

When we call the ZADD method, if the corresponding value does not exist, it is the insertion process. If the value already exists, just adjust the value of score, then we need to go through an update process.

Assuming that the new score value does not bring about any change in sorting, you do not need to adjust the position. You can directly modify the score value of the element. However, if the sorting position changes, you need to adjust the position. How to adjust it?

From source t_ zset. The C / zsetadd function can be seen in about 1350 lines. Redis adopts a very simple strategy:

/* Remove and re-insert when score changed. */ 
if (score != curscore) { 
	zobj->ptr = zzlDelete(zobj->ptr,eptr); 
	zobj->ptr = zzlInsert(zobj->ptr,ele,score); 
	*flags |= ZADD_UPDATED; 
}

Deleting and inserting this element requires two path searches. From this point of view, Redis's ZADD code seems to have room for further optimization.

6) Implementation of element ranking

The jump table itself is orderly. Redis optimizes the forward pointer of skiplist and adds a span attribute to each forward pointer to indicate how many nodes will be skipped when the previous node jumps from the forward pointer of the current layer to the current node. In the above source code, we can also see that redis will carefully update the size of span value during insertion and deletion.

Therefore, the final rank value of the current element can be calculated by accumulating the span values of all passing nodes along the "search path":

/* Find the rank for an element by both score and key. 
 * Returns 0 when the element cannot be found, rank otherwise. 
 * Note that the rank is 1-based due to the span of zsl->header to the 
 * first element. */ 
unsigned long zslGetRank(zskiplist *zsl, double score, sds ele) { 
	zskiplistNode *x; 
	unsigned long rank = 0; 
	int i; 
	
	x = zsl->header; 
	for (i = zsl->level-1; i >= 0; i--) { 
		while (x->level[i].forward && 
			(x->level[i].forward->score < score || 
				(x->level[i].forward->score == score && 
					sdscmp(x->level[i].forward->ele,ele) <= 0))) { 
				// span accumulation 
				rank += x->level[i].span; 
				x = x->level[i].forward; 
			}
			
			/* x might be equal to zsl->header, so test if obj is non-NULL */ 
			if (x->ele && sdscmp(x->ele,ele) == 0) { 
				return rank; 
			} 
		}
		return 0; 
	}
# **Summary**

The interview advice is,**Be confident and dare to express**，During the interview, it is sometimes difficult for us to master all the knowledge. We can add points by saying our ideas instead of directly telling the interviewer that we don't understand.

The above is the four sides of ant technology and HR Interview questions,**The latest summary below is the most complete**，The scope is the most complete MySQL,Spring,Redis,JVM The most comprehensive questions and answers are for reference only

![A hot ant gold mask (already taken) Offer)Interview process: 4 rounds of technical aspects+1 round HR](https://img-blog.csdnimg.cn/img_convert/8ca53c9ab48d4730691b12b0df0ef144.png)

 = 0; 
	int i; 
	
	x = zsl->header; 
	for (i = zsl->level-1; i >= 0; i--) { 
		while (x->level[i].forward && 
			(x->level[i].forward->score < score || 
				(x->level[i].forward->score == score && 
					sdscmp(x->level[i].forward->ele,ele) <= 0))) { 
				// span accumulation 
				rank += x->level[i].span; 
				x = x->level[i].forward; 
			}
			
			/* x might be equal to zsl->header, so test if obj is non-NULL */ 
			if (x->ele && sdscmp(x->ele,ele) == 0) { 
				return rank; 
			} 
		}
		return 0; 
	}
# **Summary**

The interview advice is,**Be confident and dare to express**，During the interview, it is sometimes difficult for us to master all the knowledge. We can add points by saying our ideas instead of directly telling the interviewer that we don't understand.

The above is the four sides of ant technology and HR Interview questions,**The latest summary below is the most complete**，The scope is the most complete MySQL,Spring,Redis,JVM The most comprehensive questions and answers are for reference only

[External chain picture transfer...(img-pE7jVFY8-1623564842326)]

**[How to obtain interview reference materials? You can get it here for free](https://docs.qq.com/doc/DSmxTbFJ1cmN1R2dB)**