1, Key value design

1. key name design

• (1) [suggestion]: readability and manageability

Prefix the business name (or database name) (to prevent key conflicts) and separate them with colons, such as business name: Table Name: id

ugc:video:1

• (2) [suggestion]: simplicity

On the premise of ensuring semantics, control the length of keys. When there are many keys, the memory occupation can not be ignored, for example:

user:{uid}:friends:messages:{mid}Simplified as u:{uid}:fr:m:{mid}. 

• (3) [mandatory]: do not include special characters

Counterexamples: include spaces, line breaks, single and double quotes, and other escape characters

Detailed analysis

2. Design

• (1) [force]: refuse bigkey (prevent network card traffic and slow query)

The string type shall be controlled within 10KB, and the number of hash, list, set and zset elements shall not exceed 5000.

Counterexample: a list containing 2 million elements.

For bigkeys that are not strings, do not use del to delete them. Use hscan, sscan and zscan to delete them gradually. At the same time, pay attention to prevent the automatic deletion of the expiration time of bigkeys (for example, if a 2 million zset is set to expire for 1 hour, the Del operation will be triggered and blocked, and the operation will not appear in the slow query (latency can be checked)), Search method and Delete method

Detailed analysis

• (2) [recommended]: select the appropriate data type.

For example: entity type (it is necessary to reasonably control and use data structure memory coding to optimize configuration, such as ziplost, but also pay attention to the balance between memory saving and performance)

Counterexample:

set user:1:name tom
set user:1:age 19
set user:1:favor football

Positive example:

hmset user:1 name tom age 19 favor football

3. [recommended]: control the life cycle of key s. redis is not a trash can.

It is recommended to use expire to set the expiration time (if conditions permit, you can break up the expiration time to prevent centralized expiration). For non expired data, focus on idletime.

2, Command use

1. [recommended] the O(N) command focuses on the number of N

For example, hgetall, lrange, SMEs, zrange, sinter, etc. are not unavailable, but the value of N needs to be specified. For traversal requirements, hscan, sscan and zscan can can be used instead.

2. [recommended]: disable the command

It is forbidden to use keys, flush, flush dB, etc. online. It is forbidden to use the rename mechanism of redis to disable commands, or to use scan for progressive processing.

3. [recommended] rational use of select

redis's multiple databases are weak, which are distinguished by numbers. Many clients have poor support. At the same time, multiple services are actually processed by single thread, which will cause interference.

4. [recommended] use batch operation to improve efficiency

Native commands: for example mget,mset. 
Non native commands: you can use pipeline increase of efficiency.

However, pay attention to control the number of elements in a batch operation (for example, within 500, it is actually related to the number of element bytes).

Note the difference:

1. Primitive is atomic operation, pipeline Is a non atomic operation.
2. pipeline You can package different commands, but native can't
3. pipeline Both client and server support is required.

5. [suggestion] Redis transaction function is weak, so it is not recommended to use it too much

Redis's transaction function is weak (rollback is not supported), and the cluster Version (self-developed and official) requires that the key of a transaction operation must be on a slot (which can be solved by using hashtag function)

6. [suggestion] the Redis cluster version has special requirements for using Lua:

• 1. All KEYS should be passed by the KEYS array, redis The location of the key of the redis command called in call / pcall must be KEYS array, otherwise error is returned directly, "- ERR bad lua script for redis cluster, all the keys that the script uses should be passed using the KEYS array"
• 2. All keys must be in one slot, otherwise error will be returned directly, "- err Eval / evalsha command keys must in the same slot"

7. [suggestion] when using the monitor command if necessary, be careful not to use it for a long time.

3, Client use

1. [recommended]

Avoid multiple applications using one Redis instance

Positive example: irrelevant business splitting and public data service.

2. [recommended]

Using a database with a connection pool can effectively control connections and improve efficiency. Standard usage:

Execute the following command:
Jedis jedis = null;
try {
    jedis = jedisPool.getResource();
    //Specific commands
    jedis.executeCommand()
} catch (Exception e) {
    logger.error("op key {} error: " + e.getMessage(), key, e);
} finally {
    //Note that the connection is not closed here. In JedisPool mode, Jedis will be returned to the resource pool.
    if (jedis != null) 
        jedis.close();
}

The following is an article on JedisPool optimization methods:

• Jedis common exception summary
• JedisPool resource pool optimization

3. [suggestions]

High and send suggestions to the client to add fuse function (such as netflix hystrix)

4. [recommended]

Set a reasonable password. If necessary, use SSL encryption to access (supported by Alibaba cloud Redis)

5. [suggestions]

Select maxmemory policy according to your business type and set the expiration time.

The default policy is volatile lru, that is, after the maximum memory is exceeded, lru algorithm is used to eliminate the keys in the expired keys to ensure that the expired data will not be deleted, but OOM problems may occur.

Other strategies are as follows:

• Allkeys LRU: delete keys according to the LRU algorithm, regardless of whether the timeout attribute is set for the data until enough space is made.
• All keys random: delete all keys randomly until enough space is made.
• Volatile random: randomly delete expired keys until enough space is made.
• Volatile ttl: delete the recently expired data according to the ttl attribute of the key value object. If not, go back to the noeviction policy.
• noeviction: it will not reject any data, reject all write operations, and return the client error message "(error) OOM command not allowed when used memory". At this time, Redis only responds to read operations.

4, Related tools

1. [recommended]: data synchronization

Redis port can be used for data synchronization between redis

2. [recommended]: big key search

redis big key search tool

3. [recommended]: hot key search (monitor is used in the internal implementation, so it is recommended to use it for a short time)

Redis Faina of facebook

Alibaba cloud Redis Hot spots have been resolved at the kernel level key Questions, welcome to use.

Appendix V: deleting bigkey

1. The following operations can be used pipeline Speed up.
2. redis 4.0 Already supported key Asynchronous deletion, welcome.

1. Hash delete: hscan + hdel

public void delBigHash(String host, int port, String password, String bigHashKey) {
    Jedis jedis = new Jedis(host, port);
    if (password != null && !"".equals(password)) {
        jedis.auth(password);
    }
    ScanParams scanParams = new ScanParams().count(100);
    String cursor = "0";
    do {
        ScanResult<Entry<String, String>> scanResult = jedis.hscan(bigHashKey, cursor, scanParams);
        List<Entry<String, String>> entryList = scanResult.getResult();
        if (entryList != null && !entryList.isEmpty()) {
            for (Entry<String, String> entry : entryList) {
                jedis.hdel(bigHashKey, entry.getKey());
            }
        }
        cursor = scanResult.getStringCursor();
    } while (!"0".equals(cursor));
    
    //Delete bigkey
    jedis.del(bigHashKey);
}

2. Delete list: ltrim

public void delBigList(String host, int port, String password, String bigListKey) {
    Jedis jedis = new Jedis(host, port);
    if (password != null && !"".equals(password)) {
        jedis.auth(password);
    }
    long llen = jedis.llen(bigListKey);
    int counter = 0;
    int left = 100;
    while (counter < llen) {
        //Cut 100 from the left at a time
        jedis.ltrim(bigListKey, left, llen);
        counter += left;
    }
    //Finally delete key
    jedis.del(bigListKey);
}

3. Set delete: sscan + srem

public void delBigSet(String host, int port, String password, String bigSetKey) {
    Jedis jedis = new Jedis(host, port);
    if (password != null && !"".equals(password)) {
        jedis.auth(password);
    }
    ScanParams scanParams = new ScanParams().count(100);
    String cursor = "0";
    do {
        ScanResult<String> scanResult = jedis.sscan(bigSetKey, cursor, scanParams);
        List<String> memberList = scanResult.getResult();
        if (memberList != null && !memberList.isEmpty()) {
            for (String member : memberList) {
                jedis.srem(bigSetKey, member);
            }
        }
        cursor = scanResult.getStringCursor();
    } while (!"0".equals(cursor));
    
    //Delete bigkey
    jedis.del(bigSetKey);
}

4. SortedSet delete: zscan + zrem

public void delBigZset(String host, int port, String password, String bigZsetKey) {
    Jedis jedis = new Jedis(host, port);
    if (password != null && !"".equals(password)) {
        jedis.auth(password);
    }
    ScanParams scanParams = new ScanParams().count(100);
    String cursor = "0";
    do {
        ScanResult<Tuple> scanResult = jedis.zscan(bigZsetKey, cursor, scanParams);
        List<Tuple> tupleList = scanResult.getResult();
        if (tupleList != null && !tupleList.isEmpty()) {
            for (Tuple tuple : tupleList) {
                jedis.zrem(bigZsetKey, tuple.getElement());
            }
        }
        cursor = scanResult.getStringCursor();
    } while (!"0".equals(cursor));
    
    //Delete bigkey
    jedis.del(bigZsetKey);
}