zookeeper for Notebook Series Installed in Single-machine Analog Cluster

Posted by neugi on Fri, 17 Dec 2021 23:57:53 +0100

0. Purpose

The zookeeper single-node construction method is relatively simple, in practice, it is also generally used to set up a cluster. Due to resource constraints, you don't want to install too many virtual machines or docker, so you can simulate cluster building on one machine.

1. Download zookeeper and unzip

Official download is good. I am a person who likes the new and hates the old. With the latest, I like to try the latest version. The following files are extracted:

2. Edit Profile

First step into the conf directory, and we'll see a sample configuration file, zoo_sample.cfg

# The number of milliseconds of each tick for the duration of each clock cycle
tickTime=2000 
# The number of ticks that the initial 
# synchronization phase can take to initialize the clock cycle of the synchronization section
initLimit=10
# The number of ticks that can pass between  
# sending a request and getting an acknowledgement  
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just 
# example sakes.
dataDir=/tmp/zookeeper
# the port at which the clients will connect
clientPort=2181
# the maximum number of client connections.
# increase this if you need to handle more clients
#maxClientCnxns=60
#
# Be sure to read the maintenance section of the 
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
#autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature
#autopurge.purgeInterval=1

## Metrics Providers
#
# https://prometheus.io Metrics Exporter
#metricsProvider.className=org.apache.zookeeper.metrics.prometheus.PrometheusMetricsProvider
#metricsProvider.httpPort=7000
#metricsProvider.exportJvmInfo=true

There are notes on what the configuration means. Let's start with three configuration files:
zoo_1.cfg

tickTime=2000
initLimit=10
syncLimit=5
dataDir=/tmp/zk/data1/
clientPort=2181

server.1=localhost:10086:8886
server.2=localhost:10087:8887
server.3=localhost:10088:8888

The content here needs to be explained:

  • dataDir
    Here is the directory where the data is stored. I specified it in the / tmp directory, then created a zk directory by myself, and a new data1 directory in the zk directory, indicating that the data for the first node is stored here.
  • clientPort
    This is the port number, which is 2181
  • server.1=localhost:10086:8888
    Server in front of here. 1 represents node 1 followed by ip address plus port 10086 for data synchronization and other communication between leader and follower; The latter 8888 is used for the leader node election port, which is used for voting communication.

Create a second profile
zoo_2.cfg

tickTime=2000
initLimit=10
syncLimit=5
dataDir=/tmp/zk/data2/
clientPort=2182

server.1=localhost:10086:8886
server.2=localhost:10087:8887
server.3=localhost:10088:8888

Just use a different port.

zoo_3.cfg

tickTime=2000
initLimit=10
syncLimit=5
dataDir=/tmp/zk/data3/
clientPort=2183

server.1=localhost:10086:8886
server.2=localhost:10087:8887
server.3=localhost:10088:8888

3. Directory Creation

We create another file, myid, under / tmp/zk/data1, which fills in a unique id to represent this node. Here we simply write 1.
Similarly, create myid write 2 in the / tmp/zk/data2 directory, myid write 3 in the / tmp/zk/data3 directory.

4. Start

Start them separately. Since starting on one machine, you need to specify a configuration file, using the command

././zkServer.sh start  ../conf/zoo_1.cfg

Start services on three ports in turn

././zkServer.sh start  ../conf/zoo_2.cfg
././zkServer.sh start  ../conf/zoo_3.cfg

At this time, if not unexpectedly, we should all start up, so let's have a look at their status:

yy@host-192-168-0-26:~/Downloads/apache-zookeeper-3.7.0-bin/bin$ sudo ./zkServer.sh status  ../conf/zoo_3.cfg 
/usr/bin/java
ZooKeeper JMX enabled by default
Using config: ../conf/zoo_3.cfg
Client port found: 2183. Client address: localhost. Client SSL: false.
Mode: follower
yy@host-192-168-0-26:~/Downloads/apache-zookeeper-3.7.0-bin/bin$ sudo ./zkServer.sh status  ../conf/zoo_2.cfg 
/usr/bin/java
ZooKeeper JMX enabled by default
Using config: ../conf/zoo_2.cfg
Client port found: 2182. Client address: localhost. Client SSL: false.
Mode: leader
yy@host-192-168-0-26:~/Downloads/apache-zookeeper-3.7.0-bin/bin$ sudo ./zkServer.sh status  ../conf/zoo_1.cfg 
/usr/bin/java
ZooKeeper JMX enabled by default
Using config: ../conf/zoo_1.cfg
Client port found: 2181. Client address: localhost. Client SSL: false.
Mode: follower

As you can see above, node 2 is the leader node, and nodes 1 and 3 are the follower nodes. (How do leader nodes elect? Dig a hole and fill it if necessary)

5. Client Connection

Connect zookeeper using client

./zkCli.sh #Connect to local zookeeper server
./zkCli.sh -server ip:port #Connect to the specified server

Here I'm just lazy and don't specify a server to connect to. By default, I choose the first one. After a successful connection, a bunch of details:

2021-08-31 18:08:14,829 [myid:] - INFO  [main:Environment@98] - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
2021-08-31 18:08:14,830 [myid:] - INFO  [main:Environment@98] - Client environment:java.io.tmpdir=/tmp
2021-08-31 18:08:14,830 [myid:] - INFO  [main:Environment@98] - Client environment:java.compiler=<NA>
2021-08-31 18:08:14,830 [myid:] - INFO  [main:Environment@98] - Client environment:os.name=Linux
2021-08-31 18:08:14,830 [myid:] - INFO  [main:Environment@98] - Client environment:os.arch=amd64
2021-08-31 18:08:14,830 [myid:] - INFO  [main:Environment@98] - Client environment:os.version=4.15.0-48-generic
2021-08-31 18:08:14,830 [myid:] - INFO  [main:Environment@98] - Client environment:user.name=yayuan
2021-08-31 18:08:14,831 [myid:] - INFO  [main:Environment@98] - Client environment:user.home=/home/yy
2021-08-31 18:08:14,831 [myid:] - INFO  [main:Environment@98] - Client environment:user.dir=/home/yy/Downloads/apache-zookeeper-3.7.0-bin/bin
2021-08-31 18:08:14,831 [myid:] - INFO  [main:Environment@98] - Client environment:os.memory.free=232MB
2021-08-31 18:08:14,833 [myid:] - INFO  [main:Environment@98] - Client environment:os.memory.max=245MB
2021-08-31 18:08:14,833 [myid:] - INFO  [main:Environment@98] - Client environment:os.memory.total=245MB
2021-08-31 18:08:14,838 [myid:] - INFO  [main:ZooKeeper@637] - Initiating client connection, connectString=localhost:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@4fccd51b
2021-08-31 18:08:14,842 [myid:] - INFO  [main:X509Util@77] - Setting -D jdk.tls.rejectClientInitiatedRenegotiation=true to disable client-initiated TLS renegotiation
2021-08-31 18:08:14,849 [myid:] - INFO  [main:ClientCnxnSocket@239] - jute.maxbuffer value is 1048575 Bytes
2021-08-31 18:08:14,860 [myid:] - INFO  [main:ClientCnxn@1726] - zookeeper.request.timeout value is 0. feature enabled=false
Welcome to ZooKeeper!
2021-08-31 18:08:14,881 [myid:localhost:2181] - INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@1171] - Opening socket connection to server localhost/0:0:0:0:0:0:0:1:2181.
2021-08-31 18:08:14,881 [myid:localhost:2181] - INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@1173] - SASL config status: Will not attempt to authenticate using SASL (unknown error)
2021-08-31 18:08:14,889 [myid:localhost:2181] - INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@1005] - Socket connection established, initiating session, client: /0:0:0:0:0:0:0:1:53802, server: localhost/0:0:0:0:0:0:0:1:2181
JLine support is enabled
2021-08-31 18:08:14,925 [myid:localhost:2181] - INFO  [main-SendThread(localhost:2181):ClientCnxn$SendThread@1438] - Session establishment complete on server localhost/0:0:0:0:0:0:0:1:2181, session id = 0x100761b51a00000, negotiated timeout = 30000

WATCHER::

WatchedEvent state:SyncConnected type:None path:null
[zk: localhost:2181(CONNECTED) 0] 

5.1 Create Nodes

Use create to create nodes, but nodes are divided into permanent, temporary, and sequential nodes:

create [-s][-e] path data acl
 Where,-s or-e Specify the node characteristics, order, or temporary nodes, respectively, or create persistent nodes if not specified; acl Used for permission control.

5.1. 1 Create Sequential Nodes

[zk: localhost:2181(CONNECTED) 0] create -s /zk-sorted first
Created /zk-sorted0000000000

A zk-sorted node is created under the root node, content is first. But when stored, zookeeper adds 0000000000 to distinguish it.

5.1. 2 Create a temporary node

[zk: localhost:2181(CONNECTED) 3] create -e /zk-tmp tmp
Created /zk-tmp

A zk-tmp node is created under the root node, which is tmp. After disconnecting this node, the load has recently disappeared. It only exists temporarily.

5.1. 3 Create permanent nodes

[zk: localhost:2181(CONNECTED) 5] create /zk-per per
Created /zk-per

5.2 Read Node

Read nodes typically have ls or get

[zk: localhost:2181(CONNECTED) 6] ls /
[zk-per, zk-sorted0000000000, zk-tmp, zookeeper]

ls followed by a path.

[zk: localhost:2181(CONNECTED) 7] get /zk-per
per

Getts are also followed by paths.
(There are two trees in front of Luxun Jiamen, one is a date tree and the other is a date tree. Manual dog head)

5.3 Update Node

Update with set

[zk: localhost:2181(CONNECTED) 8] set /zk-per per-plus

You need to specify the path and content to change. What I've changed is to change the content in the / zk-per node to per-plus to check it

[zk: localhost:2181(CONNECTED) 9] get /zk-per
per-plus

The content has been changed, isn't it detailed enough? I don't see the version change, do I? see

[zk: localhost:2181(CONNECTED) 10] get -s /zk-per
per-plus
cZxid = 0x200000004
ctime = Tue Aug 31 18:29:56 CST 2021
mZxid = 0x200000005
mtime = Tue Aug 31 18:35:30 CST 2021
pZxid = 0x200000004
cversion = 0
dataVersion = 1
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 8
numChildren = 0

As you can see here, dataVersion has become 1.

5.4 Delete Nodes

Simple operation
delete plus path

[zk: localhost:2181(CONNECTED) 11] delete /zk-per 
[zk: localhost:2181(CONNECTED) 12] get /zk-per
org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /zk-per
[zk: localhost:2181(CONNECTED) 13] ls /
[zk-sorted0000000000, zk-tmp, zookeeper]

As you can see, the node is deleted.
It is worth noting that if a deleted node has a child node, it cannot be deleted. You must delete the child node before deleting the parent node.

Topics: Zookeeper Middleware