Zookeeper
zookeeper action
effect:
- Middleware provides coordination services, assists in computing, assists in managing storage and other functions
- Acting on the distributed system and giving full play to its advantages, it can serve big data
- Support java and provide client Api of java and c language
characteristic:
- Consistency: data consistency. Data is received in batches in order
- Atomicity: transactions either succeed or fail without localization
- Single view: the client links to any node in the cluster, and the data is consistent
- Reliability: the operation status of zk is saved on the server every time
- Real time: the client can read the latest data of zk server
Function embodiment:
- After the Master node is elected and the Master node hangs up, the slave node will take over the work and ensure that this node is unique. This is also the so-called summit mode, so as to ensure that the cluster is highly available.
- Unified profile management, that is, if only one server needs to be deployed, the same profile can be synchronously updated to all other servers. This operation is particularly used in cloud computing
- Publish and subscribe, similar to message queue MQ(amq, rmq...), dubbo publishers store data on znode, and subscribers will read the data.
- Distributed locks are provided. Different processes compete for resources in a distributed environment, similar to locks between multiple threads
- Cluster management ensures strong data consistency in the cluster
What is a distributed system
- Many computers form a whole, which is consistent and processes the same request
- Each internal computer can communicate with each other
- A request from the client to the server will go through a very long time from the end of the response
Publish package file list
# Executable file drwxr-xr-x 2 andrew andrew 4096 3 May 17, 2021 bin # configuration file drwxr-xr-x 2 andrew andrew 4096 1 January 18:18 conf # Document path drwxr-xr-x 5 andrew andrew 4096 3 May 17, 2021 docs # Dependent jar package drwxrwxr-x 2 andrew andrew 4096 1 January 18:07 lib -rw-r--r-- 1 andrew andrew 11358 3 May 17, 2021 LICENSE.txt drwxrwxr-x 2 andrew andrew 4096 1 January 18:20 logs -rw-r--r-- 1 andrew andrew 432 3 May 17, 2021 NOTICE.txt -rw-r--r-- 1 andrew andrew 2214 3 May 17, 2021 README.md -rw-r--r-- 1 andrew andrew 3570 3 May 17, 2021 README_packaging.md
Profile zoo_sample.cfg
# The number of milliseconds of each tick # Time unit used for calculation. For example, session timeout N*tickTime tickTime=2000 # The number of ticks that the initial # synchronization phase can take # It is used for the cluster. The initialization time allowed to connect from the node and synchronize to the master node is expressed in multiple of tickTime initLimit=10 # The number of ticks that can pass between # sending a request and getting an acknowledgement # It is used for the length of message sending, request and response time between the master node and the slave node in the cluster. syncLimit=5 # the directory where the snapshot is stored. # do not use /tmp for storage, /tmp here is just # example sakes. # Default data saving location, and finally redefined dataDir=/tmp/zookeeper # If the default location is not defined, it is consistent with the location of dataDir dataLogDir=/tmp/zookeeper # the port at which the clients will connect clientPort=2181 # the maximum number of client connections. # increase this if you need to handle more clients #maxClientCnxns=60 # # Be sure to read the maintenance section of the # administrator guide before turning on autopurge. # # http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance # # The number of snapshots to retain in dataDir #autopurge.snapRetainCount=3 # Purge task interval in hours # Set to "0" to disable auto purge feature #autopurge.purgeInterval=1 ## Metrics Providers # # https://prometheus.io Metrics Exporter #metricsProvider.className=org.apache.zookeeper.metrics.prometheus.PrometheusMetricsProvider #metricsProvider.httpPort=7000 #metricsProvider.exportJvmInfo=true
Start and stop zk services
# configuration file # Add the conf to the zoo_sample.cfg is modified to zoo CFG and modify the configuration file # Start zk ./zkServer.sh start # Stop zk ./zkServer.sh stop
Start and stop zk client
# Via / zkcli SH start zk client
zookeeper basic data model
- It is a tree structure, similar to the tree developed by the front end JS component
- It can be understood as the directory structure of Linux
- Each node is called znode. It can have child nodes or data
- Each node is divided into temporary node and permanent node. The temporary node disappears after the client disconnects
- Each zk node has its own version number, and node information can be displayed through the command line
- Whenever the node data changes, the version number of the node will be accumulated (leguan lock)
- Delete / modify obsolete nodes. If the version numbers do not match, an error will be reported
- The data stored in each zk node should not be too large, just a few k
- The node can set the permission acl and restrict the user's access through the permission
zk characteristic principle
Session Fundamentals
- There is a session between the client and the server
- A timeout can be set for each session
- The heartbeat ends and the session expires
- If the session expires, the temporary node becomes invalid
Watcher mechanism
- For each node operation, there will be a supervisor - > watcher
- When a monitored object (znode) changes, the watcher event is triggered
- Zk's watcher is one-time and is destroyed immediately after triggering
- Adding, deleting, modifying and querying parent nodes and child nodes can trigger their watcher events
- Different types of operations trigger different watcher events
- (child) node creation event
- (child) node deletion event
- (child) node data change event
Composition of ACL
- When the IP address specified for IP is set, IP access is restricted, for example, IP:192.168.1.1:[permissions]
- Super: represents the super administrator and has all permissions
zk client commands
Client connection
# Default local link 127.0.0.1 ./zkCli.sh
View znode node
# ls + path to view the contents of the corresponding directory [zk: localhost:2181(CONNECTED) 4] ls / [zookeeper] # get, stat command # ls2 ==> ls + stat [zk: localhost:2181(CONNECTED) 8] ls -s / [zookeeper] cZxid = 0x0 ctime = Thu Jan 01 08:00:00 CST 1970 mZxid = 0x0 mtime = Thu Jan 01 08:00:00 CST 1970 pZxid = 0x0 cversion = -1 dataVersion = 0 aclVersion = 0 ephemeralOwner = 0x0 dataLength = 0 numChildren = 1 # Use the stat command to view the corresponding path status information [zk: localhost:2181(CONNECTED) 9] stat / cZxid = 0x0 ctime = Thu Jan 01 08:00:00 CST 1970 mZxid = 0x0 mtime = Thu Jan 01 08:00:00 CST 1970 pZxid = 0x0 cversion = -1 dataVersion = 0 aclVersion = 0 ephemeralOwner = 0x0 dataLength = 0 numChildren = 1 # get node content information
Close client connection
Just Ctrl+C to exit
set command
Optimistic lock control can be realized by following the version number behind the data. Only the version number is given to the right person can it be set successfully
# Setting changes a node [zk: localhost:2181(CONNECTED) 6] set /data/name # Use optimistic locking to set the value of a node [zk: localhost:2181(CONNECTED) 22] get -s /data/name 123 cZxid = 0x7 ctime = Mon Jan 03 23:25:59 CST 2022 mZxid = 0x1a mtime = Wed Jan 05 21:48:36 CST 2022 pZxid = 0x7 cversion = 0 dataVersion = 15 aclVersion = 0 ephemeralOwner = 0x0 dataLength = 3 numChildren = 0 # The specified version must be consistent with the current version. If it is inconsistent, an error will be reported [zk: localhost:2181(CONNECTED) 23] set /data/name 123 -v 15 # After the previous step is executed, the version here will change to 16. If you set it according to 15, the setting will certainly fail [zk: localhost:2181(CONNECTED) 24] set /data/name 123 -v 15 version No is not valid : /data/name
Delete node delete command
[zk: localhost:2181(CONNECTED) 25] delete /data/name [zk: localhost:2181(CONNECTED) 26] get /date/name org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /date/name