One background
The traditional method of generating id can be realized by the self-increment of database, but it is not suitable in distributed environment. Dependence on databases can easily lead to singletons.
Why not use UUID? When you read someone's introduction on the internet, you should analyze it from two aspects:
In the case of large concurrency, UUID will duplicate.
2.UUID is instantaneous and its meaning is unclear. From a business point of view, if used as an order, the user queries the order in the case of data fragmentation, which is likely to be scattered in multiple databases and difficult to query.
The requirement of global unique id is high:
No single point of failure.
Good performance, millisecond return.
It is easy to store and divide DB in sequence.
Second, zookeeper is used to generate global unique id.
2.1 Generation of serial numbers using Zookeeper's znode data version
Client adopts: zkClient( https://github.com/adyliu/zkclient)
<dependency> <groupId>com.github.adyliu</groupId> <artifactId>zkclient</artifactId> <version>2.1.1</version> </dependency>
java code implementation
public class ZKSeqTest {
//Create the'/ createSeq'node to store Seq in advance CreateMode.PERSISTENT
public static final String SEQ_ZNODE = "/seq";
//Distributed seq generation through znode data version
public static class Task1 implements Runnable {
private final String taskName;
public Task1(String taskName) {
this.taskName = taskName;
}
@Override
public void run() {
ZkClient zkClient = new ZkClient("192.168.190.36:2181", 3000, 50000);
Stat stat =zkClient.writeData(SEQ_ZNODE, new byte[0], -1);
int versionAsSeq = stat.getVersion();
System.out.println(taskName + " obtain seq=" +versionAsSeq );
zkClient.close();
}
}
public static void main(String[] args) {
// TODO Auto-generated method stub
//main
final ExecutorService service = Executors.newFixedThreadPool(20);
for (int i = 0; i < 10; i++) {
service.execute(new Task1("[Concurrent-" + i + "]"));
}
}
}public class ZKLock {
//Create the node "/ lock" CreateMode.PERSISTENT of the lock object in advance
public static final String LOCK_ZNODE = "/lock";
//Distributed Lock for Distributed seq Generation
public static class Task2 implements Runnable, IZkChildListener {
private final String taskName;
private final ZkClient zkClient;
private final String lockPrefix = "/loc";
private final String selfZnode;
public Task2(String taskName) {
this.taskName = taskName;
zkClient = new ZkClient("192.168.190.36:2181", 30000, 50000);
selfZnode = zkClient.createEphemeralSequential(LOCK_ZNODE + lockPrefix, new byte[0]);
}
@Override
public void run() {
createSeq();
}
private void createSeq() {
Stat stat = new Stat();
byte[] oldData = zkClient.readData(LOCK_ZNODE, stat);
byte[] newData = update(oldData);
zkClient.writeData(LOCK_ZNODE, newData);
System.out.println(taskName + selfZnode + " obtain seq=" + new String(newData));
}
private byte[] update(byte[] currentData) {
String s = new String(currentData);
int d = Integer.parseInt(s);
d = d + 1;
s = String.valueOf(d);
return s.getBytes();
}
@Override
public void handleChildChange(String parentPath,
List<String> currentChildren) throws Exception {
// TODO Auto-generated method stub
}
}
public static void main(String[] args) {
final ExecutorService service = Executors.newFixedThreadPool(20);
for (int i = 0; i < 10; i++) {
service.execute(new Task2("[Concurrent-" + i + "]"));
}
service.shutdown();
}
}Using znode with serial number to realize
[java] view plain copy
Back-end logs are temporary nodes that are automatically deleted after the session ends.
Three Open Source Solutions
There are also better open source implementations on the Internet that are worth learning from.
3.1Flikr
Self-increment Based on int/bigint
Excellent: Low development cost
Bad: If you need high performance, you need a dedicated MySQL cluster for generating self-increasing ID s only. Usability is not strong
3.2 Snowflake
Twitter uses zookeeper to implement a global ID generation service snowflake, https://github.com/twitter/snowflake, which can generate a globally unique 64bit ID.
The composition of the generated ID:
Time -- The first 41 bit s represent time, accurate to milliseconds, and can represent 69 years of data. Machine ID -- expressed in 10 bit s, which means 1024 machines can be deployed Sequence Number - Represented in 12 bit s, which means that each machine can generate up to 4096 ID s per millisecond
Excellent: high availability, fast speed, id saves more information.
Bad: zookeeper and independent snowflake dedicated servers need to be introduced
3.3instagram
instagram refers to flickr's scheme, combines twitter's experience, and utilizes the characteristics of Postgres database to realize a simpler and more reliable ID generation service.
Use 41 bit s for storage time, accurate to milliseconds, and can last 41 years.
Use 13 bit s to store the logical fragment ID.
Using 10 bit s to store self-growing IDs means that up to 1024 IDs can be generated per machine per millisecond.
Excellent: Low development cost
Poor: Storage process based on postgreSQL, poor versatility
There is also a redis-based global id generation scheme: http://blog.csdn.net/hengyunabc/article/details/44244951
Reference resources:
http://blog.csdn.net/bohu83/article/details/51457961