1 Introduction to multi-level cache architecture

In any project, we have some frequent queries, and the frequent query data are basically the same. For example, in the project, we view user personal information, shopping Carnival query activity information. These function points are used frequently, and we generally need to deal with them in a special way.
Let's take the carnival information as an example. During the double 11, many people will check what today's preferential activities are. We can use multiple methods for these activity information
Level cache architecture to resist high concurrency.

2 Java common cache design

The implementation process of cache architecture based on Java version is as follows:

1. User request via Nginx
2. Nginx checks whether there is a cache. If nginx has a cache, it directly responds to user data
3. If Nginx has no cache, it routes the request to the back-end Java service
4. The Java service queries the Redis cache. If there is data, it will directly respond to nginx and store the data in the cache. Nginx will respond to the data to the user
5. If Redis does not have a cache, use the Java program to query MySQL, store the data in IDS, and then store the data in Nginx

advantage:

1. Nginx cache is adopted to reduce the path of data loading, so as to improve the efficiency of site data loading
2. Multi level cache effectively prevents cache breakdown and cache penetration

Disadvantages:

The low concurrency of Tomcat leads to the imbalance of cache synchronization concurrency, the low efficiency of cache first loading, and the high resource consumption of Tomcat large-scale clusters

3 Lua version multi-level cache architecture improvement

advantage:
1. Nginx cache is adopted to reduce the path of data loading, so as to improve the efficiency of site data loading
2. Multi level cache effectively prevents cache breakdown and cache penetration
3. Using Nginx+Lua integration, the efficiency is high no matter what cache load
4. Nginx has high concurrency, and the integration of Nginx+Lua greatly improves the concurrency capability

4. Actual combat of nginx + Lua multi-level cache

Lua preliminary knowledge

Taking the double 11 activity information loading as an example, we load the double 11 activity information through the multi-level cache architecture. The double 11 activity information table structure is as follows:

CREATE TABLE `activitt_info` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`name` varchar(100) DEFAULT NULL COMMENT 'Activity name',
`desc` varchar(3000) DEFAULT NULL COMMENT 'Activity introduction',
`starttime` datetime DEFAULT NULL,
`endtime` datetime DEFAULT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=3 DEFAULT CHARSET=utf8;

4.1 link MySQL encapsulation

Create a lua script mysql lua, which is used to perform query operations according to SQL statements. The code is as follows:

--Import dependency Library mysql
local mysql = require "resty.mysql"
--Configure database link information
local props = {
host = "192.168.211.141",
port = 3306,
database = "redpackage",
user = "root",
password = "123456"
}
--Create an object
local mysqlobj = {}
--according to SQL Statement query
function mysqlobj.query(sql)
--Create link, set timeout, encoding format
local db = mysql:new()
db:set_timeout(10000)
db:connect(props)
db:query("SET NAMES utf8")
--implement SQL sentence
local result = db:query(sql)
--Close link
db:close()
--Return result set
return result
end
return mysqlobj

4.2 link Redis cluster encapsulation

We need to install Lua rest Redis cluster and use the dependency library to realize the operation of Redis cluster. The github address is provided here. You can download it and press the operation
Configure it. Download address:< https://github.com/cuiweixie/lua-resty-redis-cluster >, download the configuration file
Redis cluster operation.
Next, we implement Redis cluster operation and create a script Redis Lua, the script implements the query and addition of data in Redis. The code is as follows:
Below:

--redis Connection configuration
local config = {
name = "test",
serv_list = {
{ip="192.168.211.141", port = 7001},
{ip="192.168.211.141", port = 7002},
{ip="192.168.211.141", port = 7003},
{ip="192.168.211.141", port = 7004},
{ip="192.168.211.141", port = 7005},
{ip="192.168.211.141", port = 7006},
},
idle_timeout = 1000,
pool_size = 10000,
}
--introduce redis Cluster configuration
local redis_cluster = require "resty.rediscluster"
--Define an object
local lredis = {}
--according to key query
function lredis.get(key)
--create link
local red = redis_cluster:new(config)
red:init_pipeline()
--according to key get data
red:get(key)
local rresult = red:commit_pipeline()
--Close link
red:close()
return rresult
end
--Add data with expiration
function lredis.set(key,value)
--create link
local red = redis_cluster:new(config)
red:init_pipeline()
--add to key，Also set the expiration time
red:set(key,value)
local rresult = red:commit_pipeline()
end
return lredis

4.3 multi level cache operation

Configure and create Nginx cache space
lua_shared_dict act_cache 128m;
Create multi-level cache operation script activity Lua, code as follows:

--Multi level cache process operation
--1)Lua Script query Nginx cache
--2)Nginx If there is no cache
--2.1)Lua Script query Redis
--2.1.1)Redis If there is data, store the data in Nginx Cache and respond to users
--2.1.2)Redis No data, Lua Script query MySQL
-- MySQL If there is data, store the data in Redis,Nginx cache[Additional definition required]，Response user
--3)Nginx If there is a cache, the cached response is directly sent to the user
--The response data is JSON type
ngx.header.content_type="application/json;charset=utf8"
--Import dependency Library
--cjson: Object rotation JSON perhaps JSON Transfer object
local cjson = require("cjson")
local mysql = require("mysql")
local lrredis = require("redis")
--Get request parameters ID http://192.168.211.141/act?id=1
local id = ngx.req.get_uri_args()["id"];
--Load local cache
local cache_ngx = ngx.shared.act_cache;
--Assemble locally cached key,And get nginx Local cache
local ngx_key = 'ngx_act_cache_'..id
local actCache = cache_ngx:get(ngx_key)
--If nginx If there is no cache in, query Redis Cluster cache
if actCache == "" or actCache == nil then
--from Redis Load data in cluster
local redis_key = 'redis_act_'..id
local result = lrredis.get(redis_key)
--Redis The data in is empty. Query the database
if result[1]==nil or result[1]==ngx.null then
--assemble SQL sentence
local sql = "select * from activity_info where id ="..id
--Execute query
result = mysql.query(sql)
--If the data is not empty, add to Redis in
if result[1]==nil or result[1]==ngx.null then
ngx.say("no data")
else
--Add data to Nginx Cache and Redis cache
lrredis.set(redis_key,cjson.encode(result))
cache_ngx:set(ngx_key, cjson.encode(result), 2*60);
ngx.say(cjson.encode(result))
end
else
--Add data to Nginx In cache
cache_ngx:set(ngx_key, result, 2*60);
--Direct output
ngx.say(result)
end
else
--Output cache data
ngx.say(actCache)
end

4.4 Nginx configuration

#Activity query

location /act {
content_by_lua_file /usr/local/openresty/nginx/lua/activity.lua;
}