Explain how Redis implements Jiangsu Quick Three Program Development Penguin 1279829431 [Source Link] dashengba.com The principle of distributed ID is explained in java language.
Here we divide the distributed id into three parts: millisecond time, the number of nodes in redis cluster, and the self-incremental sequence value of each redis node in every millisecond.
Then because Windows is 64-bit, and then the first integer must be zero, so the maximum value is 63-bit 11111111111111111111111111111111111111111111111111111111111111111 11. Here, we allocate 41 bits as milliseconds, then 12 bits as redis nodes, and then 10 bits as redis nodes in each. Self-incremental Sequence Value of One Millisecond
The millisecond that 41-bit binary 11111111111111111111111111111111111111 converts to 10-bit is 21990232551, and then we convert 21990232551 to 2039-09-07, that is to say, it can be used for 20 years.
Then 12 bits are used as redis nodes, so the maximum is 11111111111 with 12 bits, that is to say, 4095 redis nodes can be supported.
Then each 10-bit redis node increments its sequence value, which is at most 11 1111 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1. That is to say, each redis node can generate a maximum of 1023 non-repetitive id values per millisecond.
Then we use java code to explain this principle. The following 1565165536640L is a millisecond value, then our redis node is set to 53, and then we set two different self-incremental sequence values, 1 and 1023, respectively. The following results show that in the millisecond of 1565165536640L, the 53 redis node. Two different distributed id values are generated
package io.github.hengyunabc.redis;
import java.text.SimpleDateFormat;
import java.util.Date;
public class Test {
public static void main(String[] args) {
long buildId = buildId(1565165536640L, 53, 1);
System.out.println("Distributed id Yes,"+buildId);
long buildIdLast = buildId(1565165536640L, 53, 1023);
System.out.println("Distributed id Yes,"+buildIdLast);
}
public static long buildId(long miliSecond, long shardId, long seq) {
return (miliSecond << (12 + 10)) + (shardId << 10) + seq;
}
}
public class Test {
public static void main(String[] args) {
long buildId = buildId(1565165536640L, 53, 1);
System.out.println("Distributed id Yes,"+buildId);
long buildIdLast = buildId(1565165536640L, 53, 1023);
System.out.println("Distributed id Yes,"+buildIdLast);
}
public static long buildId(long miliSecond, long shardId, long seq) {
return (miliSecond << (12 + 10)) + (shardId << 10) + seq;
}
}
The results are as follows
Distributed id: 6564780070991352833
Distributed id: 6564780070991353855
Then someone has to say that you are not in line with the distributed id settings. There is no readability at all. Here we can use the following way to get the generation time value of the distributed id.
package io.github.hengyunabc.redis;
import java.text.SimpleDateFormat;
import java.util.Date;
public class Test {
public static void main(String[] args) {
long buildId = buildId(1565165536640L, 53, 1);
parseId(buildId);
long buildIdLast = buildId(1565165536640L, 53, 1023);
parseId(buildIdLast);
}
public static long buildId(long miliSecond, long shardId, long seq) {
return (miliSecond << (12 + 10)) + (shardId << 10) + seq;
}
public static void parseId(long id) {
long miliSecond = id >>> 22;
long shardId = (id & (0xFFF << 10)) >> 10;
System.err.println("Distributed id-"+id+"The generation time is:"+new SimpleDateFormat("yyyy-MM-dd").format(new Date(miliSecond)));
System.err.println("Distributed id-"+id+"In the first place"+shardId+"Number redis Node Generation");
}
}
That's ok ay, haha.
The generation time of distributed id-6564780070991352833 is 2019-08-07.
Distributed id-6564780070991352833 is generated at the 53rd redis node
The generation time of distributed id-6564780070991353855 is 2019-08-07.
Distributed id-6564780070991353855 is generated at the 53rd redis node
Implementation of Distributed id Creation of redis in Cluster Edition
At this time, the ports of my distributed redis cluster are 6380 and 6381, respectively.
The first step is to generate the Evalsha command security sha1 check code. The generation process is as follows.
The first step is to generate the security sha1 check code corresponding to port 6380, first enter the bin directory of redis, and then download the lua script by executing the following command
wget https: //github .com/maqiankun/distributed-id-redis-generator/blob/master/redis-script-node1.lua
Then execute the following command to generate the security sha1 check code corresponding to port 6380. At this time, you can see that it is be6d4e21e9113bf8af47ce72f3da18e00580d402.
./redis-cli -p 6380 script load "$(cat redis-script-node1.lua)"
The first step is to generate the security sha1 check code corresponding to port 6381, first enter the bin directory of redis, and then download the lua script by executing the following command
wget https //github .com/maqiankun/distributed-id-redis-generator/blob/master/redis-script-node2.lua
Then execute the following command to generate the security sha1 check code corresponding to port 6381. At this time, you can see that it is 97f65601d0aaf1a0574da69b1ff3092969c4310e.
./redis-cli -p 6381 script load "$(cat redis-script-node2.lua)"
Then we use the above sha1 check code and the following code to generate the distributed id
The project pictures are as follows
The code for the IdGenerator class is as follows
package io.github.hengyunabc.redis;
import java.util.ArrayList;
import java.util.List;
import org.apache.commons.lang3.tuple.Pair;
import redis.clients.jedis.Jedis;
import redis.clients.jedis.JedisPool;
import redis.clients.jedis.exceptions.JedisConnectionException;
public class IdGenerator {
/**
* JedisPool, luaSha
*/
List<Pair<JedisPool, String>> jedisPoolList;
int retryTimes;
int index = 0;
private IdGenerator(List<Pair<JedisPool, String>> jedisPoolList,
int retryTimes) {
this.jedisPoolList = jedisPoolList;
this.retryTimes = retryTimes;
}
static public IdGeneratorBuilder builder() {
return new IdGeneratorBuilder();
}
static class IdGeneratorBuilder {
List<Pair<JedisPool, String>> jedisPoolList = new ArrayList();
int retryTimes = 5;
public IdGeneratorBuilder addHost(String host, int port, String luaSha) {
jedisPoolList.add(Pair.of(new JedisPool(host, port), luaSha));
return this;
}
public IdGenerator build() {
return new IdGenerator(jedisPoolList, retryTimes);
}
}
public long next(String tab) {
for (int i = 0; i < retryTimes; ++i) {
Long id = innerNext(tab);
if (id != null) {
return id;
}
}
throw new RuntimeException("Can not generate id!");
}
Long innerNext(String tab) {
index++;
int i = index % jedisPoolList.size();
Pair<JedisPool, String> pair = jedisPoolList.get(i);
JedisPool jedisPool = pair.getLeft();
String luaSha = pair.getRight();
Jedis jedis = null;
try {
jedis = jedisPool.getResource();
List<Long> result = (List<Long>) jedis.evalsha(luaSha, 2, tab, ""
+ i);
long id = buildId(result.get(0), result.get(1), result.get(2),
result.get(3));
return id;
} catch (JedisConnectionException e) {
if (jedis != null) {
jedisPool.returnBrokenResource(jedis);
}
} finally {
if (jedis != null) {
jedisPool.returnResource(jedis);
}
}
return null;
}
public static long buildId(long second, long microSecond, long shardId,
long seq) {
long miliSecond = (second * 1000 + microSecond / 1000);
return (miliSecond << (12 + 10)) + (shardId << 10) + seq;
}
public static List<Long> parseId(long id) {
long miliSecond = id >>> 22;
long shardId = (id & (0xFFF << 10)) >> 10;
List<Long> re = new ArrayList<Long>(4);
re.add(miliSecond);
re.add(shardId);
return re;
}
}
Example's code is shown below. The purpose of the following while loop is to print multiple distributed id s. The following tab variable is the parameter in the evalsha command. It can be defined according to its own requirements.
package io.github.hengyunabc.redis;
import java.text.SimpleDateFormat;
import java.util.Date;
import java.util.List;
public class Example {
public static void main(String[] args) {
String tab = "Here it is. evalsha The parameters in the command are defined arbitrarily";
IdGenerator idGenerator = IdGenerator.builder()
.addHost("47.91.248.236", 6380, "be6d4e21e9113bf8af47ce72f3da18e00580d402")
.addHost("47.91.248.236", 6381, "97f65601d0aaf1a0574da69b1ff3092969c4310e")
.build();
int hello = 0;
while (hello<3){
long id = idGenerator.next(tab);
System.out.println("Distributed id value:" + id);
List<Long> result = IdGenerator.parseId(id);
System.out.println("Distributed id The generation time is:" + new SimpleDateFormat("yyyy-MM-dd").format(new Date(result.get(0))) );
System.out.println("redis node:" + result.get(1));
hello++;
}
}
}
At this point, the print results are as follows
Distributed id value: 6564819854640022531
Distributed id generation time is 2019-08-07
redis node: 1
Distributed id value: 6564819855189475330
Distributed id generation time is 2019-08-07
redis node: 0
Distributed id value: 6564819855361442819
Distributed id generation time is 2019-08-07
redis node: 1
Here redis cluster version of the distributed id is done, perfect
Is the distributed ID implemented by Redis cluster suitable for distributed id?
I think Redis Cluster to achieve distributed ID can be used for our basic development, but I still think it has the following two problems:
1: Here we can compare the mechanism of database self-increasing ID in the previous article. In fact, Redis cluster can be said to solve the performance problem of creating distributed ID in database cluster, but it is still difficult for Redis cluster system to expand horizontally. If we want to add Redis nodes to Redis cluster in the future, we will still meet with database set. Node expansion of a group is just as cumbersome.
2: Also, if you don't use Redis in your project, then you need to introduce new components, which is also a troublesome problem.