Recommended reading: Java Xiaobai advanced architect learning route
1, Introduction
ZooKeeper is a distributed, open-source distributed application coordination service. It is an open-source implementation of Chubby of Google and an important component of Hadoop and Hbase. It is a software that provides consistency services for distributed applications. Its functions include configuration maintenance, domain name service, distributed synchronization, group service and so on.
The goal of ZooKeeper is to encapsulate complex and error prone key services and provide users with simple and easy-to-use interfaces and systems with efficient performance and stable functions.
ZooKeeper contains a simple set of primitives that provide interfaces between Java and C.
In the code version of ZooKeeper, interfaces of distributed exclusive lock, election and queue are provided. The code is in $zookeeper_home\src\recipes. There are two versions of distributed locks and queues: Java and C, and there is only java version for election.
2, Principle
ZooKeeper is based on Fast Paxos algorithm. Paxos algorithm has the problem of livelock, that is, when multiple proposers submit alternately, they may be mutually exclusive, resulting in no proposer can submit successfully. However, Fast Paxos has made some optimization and elected a leader (leader). Only the leader can submit the proposer. See Fast Paxos for the specific algorithm. Therefore, to understand ZooKeeper, you must first understand Fast Paxos.
Basic operation process of ZooKeeper:
- Elect leaders.
- Synchronize data.
- There are many algorithms in the process of electing leaders, but the election standards to be achieved are consistent.
- The Leader should have the highest execution ID, similar to root permission.
- Most of the machines in the cluster get the response and accept the selected Leader.
3, Characteristics
In zookeeper, znode is a node similar to the Unix file system path. You can store or obtain data from this node. If the Flag is set to EPHEMERAL when creating a znode, when the node creating the znode loses connection with zookeeper, the znode will no longer exist in zookeeper, and zookeeper uses Watcher to detect event information. When the client receives event information, such as connection timeout, node data change and child node change, it can call the corresponding behavior to process the data. Zookeeper's Wiki page shows how to use zookeeper to handle event notification, queue, priority queue, lock, shared lock, revocable shared lock and two-stage submission.
So what can Zookeeper do? A simple example: suppose we have 20 search engine servers (each responsible for a part of the search task in the total index), a total server (responsible for sending search requests to the servers of these 20 search engines and merging the result set), and a standby total server (responsible for replacing the total server when the total server goes down), A web CGI (send a search request to the main server). 15 of the search engine's servers provide search services, and 5 servers are generating indexes. The servers of these 20 search engines often have to ask the server providing the search service to stop providing the service and start generating the index, or the server generating the index has completed the index generation and can provide the search service. Using Zookeeper can ensure that the total server automatically senses how many servers provide search engines and sends search requests to these servers. When the total server goes down, the standby total server will be automatically enabled.
4, spring cloud integrates Zookeeper
This paper will introduce how to use Zookeeper to realize service discovery in the micro service framework. This service discovery mechanism can be used as the registration center of cloud services. Provide a Spring Boot integration for applications through Spring Cloud Zookeeper, which integrates Zookeeper into the Spring environment through automatic configuration and binding.
1. Install Zookeeper
download
The official version of Apache is 3.4.8. Download address: http://mirrors.cnnic.cn/apache/zookeeper/zookeeper-3.4.8/zookeeper-3.4.8.tar.gz
install
Extract it to the specified directory D: softbookkeeper-3.4.8, and modify the zoo_sample.cfg file name (D: softbookkeeper-3.4.8conf) is zoo CFG (you can also copy it out) mainly modifies the log location. The specific configuration files are as follows:
# The number of milliseconds of each tick tickTime=2000 # The number of ticks that the initial # synchronization phase can take initLimit=10 # The number of ticks that can pass between # sending a request and getting an acknowledgement syncLimit=5 # the directory where the snapshot is stored. # do not use /tmp for storage, /tmp here is just # example sakes. dataDir=D:\\zookeeper\\data dataLogDir=D:\\zookeeper\\log # the port at which the clients will connect clientPort=2181 # the maximum number of client connections. # increase this if you need to handle more clients #maxClientCnxns=60 # # Be sure to read the maintenance section of the # administrator guide before turning on autopurge. # # http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance # # The number of snapshots to retain in dataDir #autopurge.snapRetainCount=3 # Purge task interval in hours # Set to "0" to disable auto purge feature #autopurge.purgeInterval=1
- tickTime: this time is used as the time interval between Zookeeper servers or between clients and servers to maintain heartbeat, that is, a heartbeat will be sent every tickTime.
- dataDir: as the name suggests, it is the directory where Zookeeper saves data. By default, Zookeeper also saves the log files that write data in this directory.
- dataLogDir: as the name suggests, it is the directory where Zookeeper saves log files
- clientPort: this port is the port where the client connects to the zookeeper server. Zookeeper will listen to this port and accept the client's access requests.
start-up
Enter the bin directory and start zkserver CMD, a java process will be started in this script. Double click the file to start it.
2. Build service provider
We will create a service provider by adding the pring cloud starter zookeeper discovery dependency and introducing the @ enablediscovery client annotation into the main program. The specific content of the service is to return "Hello World!" through the GET request character string.
pom.xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.yiidian</groupId>
<artifactId>spring-cloud-zookeeper-proivder</artifactId>
<version>1.0-SNAPSHOT</version>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.0.1.RELEASE</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<java.version>1.8</java.version>
</properties>
<dependencies>
<!-- SpringBoot yes Web support SpringMVC Related functions, json Conversion functions, etc -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-zookeeper-discovery</artifactId>
</dependency>
</dependencies>
<!-- definition SpringCloud edition -->
<repositories>
<repository>
<id>spring-snapshots</id>
<name>Spring Snapshots</name>
<url>https://repo.spring.io/snapshot</url>
<snapshots>
<enabled>true</enabled>
</snapshots>
</repository>
<repository>
<id>spring-milestones</id>
<name>Spring Milestones</name>
<url>https://repo.spring.io/milestone</url>
<snapshots>
<enabled>false</enabled>
</snapshots>
</repository>
</repositories>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-dependencies</artifactId>
<version>Finchley.M9</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
</project>
Startup class
Annotate our main class with @ EnableDiscoveryClient, which will automatically publish the HelloWorld application.
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.cloud.client.discovery.EnableDiscoveryClient;
/**
* Service provider startup class
*/
@SpringBootApplication
@EnableDiscoveryClient
public class ProviderStarter {
public static void main(String[] args) {
SpringApplication.run(ProviderStarter.class,args);
}
}
Controller
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;
/**
* controller
*/
@RestController
public class HelloController {
@GetMapping("/hello")
public String hello(){
return "Hello World";
}
}
application.yml
server:
port: 9001
spring:
application:
name: Provider
cloud:
zookeeper:
discovery:
enabled: true
connect-string: localhost:2181
logging:
level:
org.apache.zookeeper.ClientCnxn: WARN
3. Build service consumers
Now let's create a REST service consumer that uses the spring Netflix Feign Client to invoke the service.
pom.xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.yiidian</groupId>
<artifactId>spring-cloud-zookeeper-consumer</artifactId>
<version>1.0-SNAPSHOT</version>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.0.1.RELEASE</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<java.version>1.8</java.version>
</properties>
<dependencies>
<!-- SpringBoot yes Web support SpringMVC Related functions, json Conversion functions, etc -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-zookeeper-discovery</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-openfeign</artifactId>
</dependency>
</dependencies>
<!-- definition SpringCloud edition -->
<repositories>
<repository>
<id>spring-snapshots</id>
<name>Spring Snapshots</name>
<url>https://repo.spring.io/snapshot</url>
<snapshots>
<enabled>true</enabled>
</snapshots>
</repository>
<repository>
<id>spring-milestones</id>
<name>Spring Milestones</name>
<url>https://repo.spring.io/milestone</url>
<snapshots>
<enabled>false</enabled>
</snapshots>
</repository>
</repositories>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-dependencies</artifactId>
<version>Finchley.M9</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
</project>
Startup class
Like the service provider, add the @ EnableDiscoveryClient annotation in the main program.
package com.yiidian.consumer;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.cloud.client.discovery.EnableDiscoveryClient;
import org.springframework.cloud.openfeign.EnableFeignClients;
/**
*Service consumer startup class
*/
@SpringBootApplication
@EnableDiscoveryClient
@EnableFeignClients
public class ConsumerStarter {
public static void main(String[] args) {
SpringApplication.run(ConsumerStarter.class,args);
}
}
Define Feign remote interface
By introducing the spring cloud starter openfeign component package, we use the declarative service call method to call the remote service, annotate an interface with @ FeignClient ("service name") and automatically connect it to our application, so that we can access the service programmatically.
import org.springframework.cloud.openfeign.FeignClient;
import org.springframework.web.bind.annotation.GetMapping;
/**
* Feign Remote interface
*/
@FeignClient("Provider")
public interface HelloClient {
@GetMapping("/hello")
public String hello();
}
Controller
The following is a simple service controller class, which consumes the service by injecting the interface helloWorldClient object and calling the service provider's interface, and displays its return value in the response.
import com.yiidian.consumer.client.HelloClient;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;
/**
* Consumer controller
*/
@RestController
public class GreetingController {
@Autowired
private HelloClient helloClient;
@GetMapping("/get-greeting")
public String greeting() {
return helloClient.hello();
}
}
application.yml
server:
port: 9002
spring:
application:
name: Consumer
cloud:
zookeeper:
discovery:
enabled: true
connect-string: localhost:2181
logging:
level:
org.apache.zookeeper.ClientCnxn: WARN
4. Testing
HelloWorld REST service registers itself in Zookeeper, and Greeting service discovers and invokes HelloWorld service through declarative client.
Now we can run these two services and access them in the browser http://localhost:9002/get -After the meeting, the following information will be returned:
hello world
5, Summary
In this article, we saw how to use Spring Cloud Zookeeper to realize service discovery, and registered a service named Hello World in Zookeeper. Then a service consumer Greeting is implemented by declarative service invocation to discover and use the service.
By the way, let's introduce the differences between Zookeeper and Eureka. The service governance mechanism implemented by Spring Cloud Eureka emphasizes AP in CAP principle, namely usability and reliability, while Zookeeper emphasizes CP (consistency and reliability). Eureka sacrifices a certain degree of consistency in order to achieve higher service availability. In extreme cases, Eureka would rather accept failure instances than lose "healthy" instances. For example, when the network of the service registry breaks down, because all service instances cannot maintain a continuous heartbeat, all service instances will be eliminated in the service governance emphasizing CP, Eureka will trigger the protection mechanism and retain all nodes at this time to realize the scenario that services can still call each other.
Recommended reading: Java Xiaobai advanced architect learning route