Catch all Redis Lua script concurrent atomic combination operations
1. Preface
Redis is a high-performance KV in memory database. In addition to its basic role as a caching middleware, it also has many uses, such as what Pangge shared before Redis GEO geographic location information calculation . Redis provides a wealth of commands for us to use to implement some calculations. A single redis command is atomic. Sometimes we want to combine multiple redis commands and make the combination atomic and even reusable. Redis developers realized that this scenario is still very common, so they introduced a feature in version 2.6 to solve this problem, that is, redis executes Lua script.
2. Lua
Lua is also an ancient language. Players playing world of Warcraft should be familiar with it. The plug-in of WOW is written with Lua script. Lua is widely used in highly concurrent online games.
Lua is widely used as embedded scripts in other languages, especially C/C + +. Its syntax is simple and compact. The source code is only more than 200 K, which may also be the reason why Redis chose it officially.
Another star software Nginx also supports Lua, which can also realize many useful functions.
3. Lua is not difficult
The official Redis guide also points out that you should not write too complex logic in Lua scripts.
In order to achieve a function, we need to learn a language, which seems to make people feel like backing down. In fact, Lua is not difficult to learn, and as the scenario of this article, we don't need to learn the complete features of lua. We should use Lua language in Redis in a lightweight way. This is not difficult for you who have mastered the heavyweight language Java. Here, fat brother will only talk about the basic grammar involved in Redis.
Lua's simple syntax
Lua, in the Redis script, I personally suggest using only the following types:
-
nil = empty
-
boolean Boolean
-
Number = number
-
String = string
-
table
Declaration type
Declaring a type is very simple and does not carry a type.
--- global variable name = 'felord.cn' --- local variable local age = 18
Redis scripts should not use global variables in practice. Local variables are more efficient.
table type
The first four are very easy to understand. The fifth table needs to be briefly described. It is both an array and similar to the HashMap (Dictionary) in Java. It is the only data structure in Lua.
The array is not divided into specific types. The demonstration is as follows
Lua 5.1.5 Copyright (C) 1994-2012 Lua.org, PUC-Rio
> arr_table = {'felord.cn','Felordcn',1}
> print(arr_table[1])
felord.cn
> print(arr_table[3])
1
> print(#arr_table)
3
As a dictionary:
Lua 5.1.5 Copyright (C) 1994-2012 Lua.org, PUC-Rio
> arr_table = {name = 'felord.cn', age = 18}
> print(arr_table['name'])
felord.cn
> print(arr_table.name)
felord.cn
> print(arr_table[1])
nil
> print(arr_table['age'])
18
> print(#arr_table)
0
Mixed mode:
Lua 5.1.5 Copyright (C) 1994-2012 Lua.org, PUC-Rio
> arr_table = {'felord.cn','Felordcn',1,age = 18,nil}
> print(arr_table[1])
felord.cn
> print(arr_table[4])
nil
> print(arr_table['age'])
18
> print(#arr_table)
3
❗ # the length of the table may not be accurate, so use it with caution. At the same time, avoid using mixed mode table in Redis script, and the element should avoid containing null value nil. In the case of uncertain elements, loops should be used to calculate the true length.
judge
The judgment is very simple. The format is:
local a = 10
if a < 10 then
print('a Less than 10')
elseif a < 20 then
print('a Less than 20, greater than or equal to 10')
else
print('a Greater than or equal to 20')
end
Array loop
local arr = {1,2,name='felord.cn'}
for i, v in ipairs(arr) do
print('i = '..i)
print('v = '.. v)
end
print('-------------------')
for i, v in pairs(arr) do
print('p i = '..i)
print('p v = '.. v)
end
Print results:
i = 1 v = 1 i = 2 v = 2 ----------------------- p i = 1 p v = 1 p i = 2 p v = 2 p i = name p v = felord.cn
Return value
Like Python, Lua can return multiple return values. However, it is not recommended to use this feature in the Lua script of Redis. If so, please package it as an array structure. The return value rules supporting scripts in Spring Data Redis can be analyzed here:
public static ReturnType fromJavaType(@Nullable Class<?> javaType) {
if (javaType == null) {
return ReturnType.STATUS;
}
if (javaType.isAssignableFrom(List.class)) {
return ReturnType.MULTI;
}
if (javaType.isAssignableFrom(Boolean.class)) {
return ReturnType.BOOLEAN;
}
if (javaType.isAssignableFrom(Long.class)) {
return ReturnType.INTEGER;
}
return ReturnType.VALUE;
}
In practice, fat man will use three types: List, Boolean and Long to avoid single moth.
So far, the knowledge required by Redis Lua script is over. Other functions, coroutines and other features should not appear in Redis Lua script. If you use built-in functions, just search and query.
When you come into contact with a new technology, you should first use it in a regular way. If you want to play flower work, it means higher learning costs.
4. Lua in redis
Next is the actual operation of Redis Lua script.
EVAL command
The EVAL command is used in Redis to directly execute the specified Lua script.
EVAL luascript numkeys key [key ...] arg [arg ...]
-
Keyword of EVAL command.
-
luascript Lua script.
-
The number of keys that the Lua script specified by numkeys needs to process is actually the length of the Lua key array.
-
key: pass zero to multiple keys to Lua script, separated by spaces. In Lua script, obtain the corresponding value through "KEYS[INDEX], where 1 < = index < = numkeys.
-
arg is zero to multiple additional parameters passed to the script, separated by spaces. In Lua script, get the corresponding value through ARGV[INDEX], where 1 < = index < = numkeys.
Next, I'll demonstrate a simple script to get the key hello:
127.0.0.1:6379> set hello world
OK
127.0.0.1:6379> get hello
"world"
127.0.0.1:6379> EVAL "return redis.call('GET',KEYS[1])" 1 hello
"world"
127.0.0.1:6379> EVAL "return redis.call('GET','hello')"
(error) ERR wrong number of arguments for 'eval' command
127.0.0.1:6379> EVAL "return redis.call('GET','hello')" 0
"world"
It is found from the above demonstration code that KEYS[1] can be directly replaced by hello, but the official Redis document points out that this is not recommended. The purpose is to analyze the command before the command is executed to ensure that Redis Cluster can forward the command to the appropriate cluster node.
numkeys is a required command parameter in any case.
call function and pcall function
In the above example, we use redis Call() to execute a SET command. In fact, we can also replace it with redis pcall(). The only difference between them is the way to deal with errors. When the former executes a command error, it will directly return an error to the caller; The latter will package the error as a table we mentioned above:
127.0.0.1:6379> EVAL "return redis.call('no_command')" 0
(error) ERR Error running script (call to f_1e6efd00ab50dd564a9f13e5775e27b966c2141e): @user_script:1: @user_script: 1: Unknown Redis command called from Lua script
127.0.0.1:6379> EVAL "return redis.pcall('no_command')" 0
(error) @user_script: 1: Unknown Redis command called from Lua script
This is like when Java encounters an exception. The former will throw an exception directly; The latter will handle the exception as JSON return.
Value conversion
Due to the existence of two different operating environments in Redis, Redis and Lua, corresponding conversion operations must occur when Redis and Lua transfer data to each other. This conversion operation cannot be ignored in practice. For example, if Lua script returns decimal to Redis, decimal precision will be lost; It is safe if converted to a string.
127.0.0.1:6379> EVAL "return 3.14" 0 (integer) 3 127.0.0.1:6379> EVAL "return tostring(3.14)" 0 "3.14"
According to fat brother's experience, it is safe to pass strings and integers. For others, you need to carefully check the official documents and carry out actual verification.
Atomic execution
Lua script is executed atomically in Redis. When the Redis server executes EVAL command, only all logic contained in the Lua script specified by the current command will be executed before the command is executed and the result is returned to the caller. Commands sent by other clients will be blocked until the EVAL command is executed. Therefore, Lua scripts should not be written with too complex logic. The efficiency of lua scripts must be guaranteed as much as possible, otherwise other clients will be affected.
Script management
SCRIPT LOAD
Load scripts into the cache to achieve reuse and avoid wasting bandwidth by loading multiple times. Each script will return a unique string ID through SHA verification. You need to cooperate with the EVALSHA command to execute the cached script.
127.0.0.1:6379> SCRIPT LOAD "return 'hello'" "1b936e3fe509bcbc9cd0664897bbe8fd0cac101b" 127.0.0.1:6379> EVALSHA 1b936e3fe509bcbc9cd0664897bbe8fd0cac101b 0 "hello"
SCRIPT FLUSH
Since there is a cache, you can clear the cache. Unfortunately, the script cache is not deleted according to SHA, but all script caches are cleared. Therefore, this command is generally not used in production.
SCRIPT EXISTS
Check whether one or more caches exist with SHA ID as parameter.
127.0.0.1:6379> SCRIPT EXISTS 1b936e3fe509bcbc9cd0664897bbe8fd0cac101b 1b936e3fe509bcbc9cd0664897bbe8fd0cac1012 1) (integer) 1 2) (integer) 0
SCRIPT KILL
Terminate the executing script. However, for the sake of data integrity, this command does not guarantee successful termination. This command does not work if a script needs to be terminated because it executes part of the written logic. You need to execute SHUTDOWN nosave to terminate the server without data persistence to complete the termination script.
Some other points
Knowing the above knowledge can basically meet the needs of developing some simple Lua scripts. However, there are still some key points in the actual development.
-
The Lua script must be fully tested to ensure the robustness of its logic. When the Lua script encounters an exception, the executed logic will not be rolled back.
-
Try not to use random functions provided by Lua. See relevant official documents.
-
Do not write function function in Lua script. The whole script is the function body of a function.
-
All variables declared in scripting use the local keyword.
-
When using Lua script in the cluster, Redis Hash Tag technology can be used to ensure that all key s in the logic are assigned to the same machine, that is, in the same slot.
-
Again, Lua scripts must not contain too time-consuming and complex logic.
5. Summary
This paper explains and demonstrates the scenarios of Redis Lua script and the Lua programming syntax required to write Redis Lua script in detail, and also shares some key points that should be paid attention to in the actual development of Redis Lua script. Hope to help you master this technology. That's all for today's sharing. Next time I'll share how to use Lua script in actual Redis development, so this article must be mastered.
Original address: https://mp.weixin.qq.com/s?__biz=MzUzMzQ2MDIyMA%3D%3D&idx=1&mid=2247487159&scene=21&sn=9de21145e96eea4455c9587506dd34ef#wechat_redirect