catalogue
1. Distributed lock case implemented by native Zookeeper
1) Distributed lock implementation
2. Case of distributed lock implemented by cursor framework
The concept of distributed lock: for example, "process"
1"
When using this resource, you will first obtain the lock,
"
process
1"
The resource will be locked after obtaining the lock
Remain exclusive so that other processes cannot access the resource“
process
1"
After using up the resource, release the lock and let other processes obtain the lock. Through this lock mechanism, we can ensure that multiple processes in the distributed system can access the critical resource orderly. Then we call this lock in the distributed environment distributed lock.
1. Distributed lock case implemented by native Zookeeper
1) Distributed lock implementation
public class DistributedLock { private final String connectString = "hadoop102:2181,hadoop103:2181,hadoop104:2181"; private final int sessionTimeout = 2000; private final ZooKeeper zk; private CountDownLatch connectLatch = new CountDownLatch(1); private CountDownLatch waitLatch = new CountDownLatch(1); private String waitPath; private String currentNode; public DistributedLock() throws IOException, InterruptedException, KeeperException { //Get connection zk = new ZooKeeper(connectString, sessionTimeout, new Watcher() { @Override public void process(WatchedEvent watchedEvent) { //connectLatch can be released if zk it is connected //Note that the package guided by Event is Zookeeper if (watchedEvent.getState() == Event.KeeperState.SyncConnected){ connectLatch.countDown(); } //Watchlatch needs to be released //If the event of deleting a node occurs and the deleted path is the path of the monitored node, it is released if (watchedEvent.getType() == Event.EventType.NodeDeleted && watchedEvent.getPath().equals(waitPath)){ waitLatch.countDown(); } } }); //After waiting for zk normal connection, go to the next procedure connectLatch.await(); //Determine whether the root node / locks exists Stat stat = zk.exists("/locks", false); if(stat == null){ //Create root node zk.create("/locks", "locks".getBytes(), ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT); } } //Lock zk public void zklock(){ //Create the corresponding temporary node with sequence number try { //The create method returns the node path. Here is the current node currentNode = zk.create("/locks/" + "seq-", null, ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.EPHEMERAL_SEQUENTIAL); //Judge whether the created node is the smallest sequence number node. If so, obtain the lock; If not, listen to the previous node of its serial number List<String> children = zk.getChildren("/locks", false); //If children have only one node, get the value directly. If there are multiple nodes, you need to judge who has the smallest serial number if(children.size() == 1){ return; }else{ //Sort by sequence number Collections.sort(children); //Get node name seq-00000000 //Cut off / locks / and the rest is the name String thisNode = currentNode.substring("/locks/".length()); //Get the position of the node in the children collection through seq-00000000 int index = children.indexOf(thisNode); //Determine whether there is a problem with the data if(index == -1){ System.out.println("Data exception"); }else if(index == 0){ //The first node can obtain the lock return; }else{ //You need to listen for the change of the previous node //waitPath is the previous path of the current node waitPath = "/locks/" + children.get(index - 1); zk.getData(waitPath, true, null); //Waiting for listening waitLatch.await(); return; } } } catch (KeeperException e) { e.printStackTrace(); } catch (InterruptedException e) { e.printStackTrace(); } } //Unlock public void unZklock(){ //Delete node try { zk.delete(currentNode, -1); } catch (InterruptedException e) { e.printStackTrace(); } catch (KeeperException e) { e.printStackTrace(); } } }
2) Distributed lock test
(1) Create two threads
public class DistributedLockTest { public static void main(String[] args) throws InterruptedException, IOException, KeeperException { final DistributedLock lock1 = new DistributedLock(); final DistributedLock lock2 = new DistributedLock(); new Thread(new Runnable() { @Override public void run() { try { lock1.zklock(); System.out.println("Thread 1 acquire lock"); Thread.sleep(5 * 1000); lock1.unZklock(); System.out.println("Thread 1 release lock"); } catch (InterruptedException e) { e.printStackTrace(); } } }).start(); new Thread(new Runnable() { @Override public void run() { try { lock2.zklock(); System.out.println("Thread 2 acquire lock"); Thread.sleep(5 * 1000); lock2.unZklock(); System.out.println("Thread 2 release lock"); } catch (InterruptedException e) { e.printStackTrace(); } } }).start(); } }
(2) Observe console changes
2. Case of distributed lock implemented by cursor framework
1) Problems in the development of native Java API
(
1
)The session connection is asynchronous and needs to be handled by yourself. For example, use
CountDownLatch
(2)
Watch
Otherwise, the registration cannot be repeated
(3) The complexity of development is still relatively high
(4) Multi node deletion and creation are not supported. You need to recurse yourself
2) Cursor is a special framework for solving distributed locks, which solves the problems encountered in the development of native Java APIs.
Please check the official documents for details:
https://curator.apache.org/index.html
3) Curator case practice
(
1
)Add dependency
In POM Add the following content to the XML:
<dependency> <groupId>org.apache.curator</groupId> <artifactId>curator-framework</artifactId> <version>4.3.0</version> </dependency> <dependency> <groupId>org.apache.curator</groupId> <artifactId>curator-recipes</artifactId> <version>4.3.0</version> </dependency> <dependency> <groupId>org.apache.curator</groupId> <artifactId>curator-client</artifactId> <version>4.3.0</version> </dependency>
pom.xml complete code:
<?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion> <groupId>org.example</groupId> <artifactId>zookeeper</artifactId> <version>1.0-SNAPSHOT</version> <dependencies> <dependency> <groupId>junit</groupId> <artifactId>junit</artifactId> <version>RELEASE</version> </dependency> <dependency> <groupId>org.apache.logging.log4j</groupId> <artifactId>log4j-core</artifactId> <version>2.8.2</version> </dependency> <dependency> <groupId>org.apache.zookeeper</groupId> <artifactId>zookeeper</artifactId> <version>3.5.7</version> </dependency> <dependency> <groupId>org.apache.curator</groupId> <artifactId>curator-framework</artifactId> <version>4.3.0</version> </dependency> <dependency> <groupId>org.apache.curator</groupId> <artifactId>curator-recipes</artifactId> <version>4.3.0</version> </dependency> <dependency> <groupId>org.apache.curator</groupId> <artifactId>curator-client</artifactId> <version>4.3.0</version> </dependency> </dependencies> </project>
(2) Code implementation
public class CuratorLockTest { public static void main(String[] args) { //Create distributed lock 1 InterProcessMutex lock1 = new InterProcessMutex(getCuratorFramework(), "/locks"); //Create distributed lock 2 InterProcessMutex lock2 = new InterProcessMutex(getCuratorFramework(), "/locks"); new Thread((new Runnable() { @Override public void run() { try { //Here, the same thread is obtained twice. In order to verify, you can enter the same thread repeatedly lock1.acquire(); System.out.println("Thread 1 acquire lock"); lock1.acquire(); System.out.println("Thread 1 acquires the lock again"); Thread.sleep(5 * 1000); lock1.release(); System.out.println("Thread 1 release lock"); lock1.release(); System.out.println("Thread 1 releases the lock again"); } catch (Exception e) { e.printStackTrace(); } } })).start(); new Thread((new Runnable() { @Override public void run() { try { lock2.acquire(); System.out.println("Thread 2 acquire lock"); lock2.acquire(); System.out.println("Thread 2 acquires the lock again"); Thread.sleep(5 * 1000); lock2.release(); System.out.println("Thread 2 release lock"); lock2.release(); System.out.println("Thread 2 releases the lock again"); } catch (Exception e) { e.printStackTrace(); } } })).start(); } // Distributed lock initialization private static CuratorFramework getCuratorFramework() { //Retry strategy, initial test time 3 seconds, retry 3 times ExponentialBackoffRetry policy = new ExponentialBackoffRetry(3000, 3); CuratorFramework client = CuratorFrameworkFactory.builder().connectString("hadoop102:2181,hadoop103:2181,hadoop104:2181") .connectionTimeoutMs(2000) .sessionTimeoutMs(2000) .retryPolicy(policy).build(); //Start client client.start(); System.out.println("zookeeper Start successful"); return client; } }
(3) Console display