kafka installation and configuration (cross platform: both windows and linux Installation, including zk and jdk installation)

Posted by ok on Fri, 15 Oct 2021 23:39:33 +0200

1.jdk installation

1) windows install jdk

1. Download

Open the following link: http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html , enter the JDK1.8 download official website, or directly Baidu JDK1.8, or enter the download official website.

After entering the official website, download the JDK of the corresponding system version according to the given information. Here, take the installation of jdk-8u60-windows-x64 as an example.

Here we choose jdk-17_windows-x64_bin.exe Download

2. Installation

① After downloading, double-click to install, as shown in the following figure:

② Change the JDK installation path (or install it on Disk C by default). Here I install D:\Software\Java\jdk1.8.0_60 \ directory (see yourself), and then next.


Click next

Click close

3. Environment configuration

JDK environment variable configuration. First configure JAVA_HOME: right click the computer icon, and then click the Properties menu item at the bottom of the menu, or continuously open the control panel - system and security - system.

Click the Advanced tab in the pop-up dialog box, and then click the environment variable below:


Click the "new" button in the system variable in the environment variable dialog box, enter JAVA_HOME in the variable name in the pop-up new system variable, and enter the root directory of JDK in the variable value: D:\Software\Java\jdk-17, and then click OK to return to the environment variable dialog box.


Check whether there is a "classpath" variable in the system variable. If not, create a new variable with the variable name classpath and the variable value.;% JAVA_HOME%\lib; note that this variable value starts with an English period and ends with a good score. Then click OK to return to the environment variable dialog box.


Finally, find the Path variable in the system variable. Note that this time, click the Edit button. The last variable value in the pop-up dialog box must be the last. Add the following string:;% JAVA_HOME%\bin;%JAVA_HOME%\jre\bin. Note that the first semicolon in front is a semicolon. If there is no Path variable, add the Path variable. The adding steps are the same as before and will not be repeated.

Then confirm to return to the desktop, open the "command prompt" (run as administrator), enter java – version, and the output shown in the figure below shows that the jdk installation is successful.

2) Installing jdk for linux

1. Check whether the system has a jdk and delete it without skipping

java -version
rpm -qa | grep java
 It may be found that:
java-1.7.0-openjdk-1.7.0.111-2.6.7.8.el7.x86_64
java-1.8.0-openjdk-1.8.0.102-4.b14.el7.x86_64
java-1.8.0-openjdk-headless-1.8.0.102-4.b14.el7.x86_64
java-1.7.0-openjdk-headless-1.7.0.111-2.6.7.8.el7.x86_64
 Delete example
rpm -e --nodeps java-1.7.0-openjdk-1.7.0.111-2.6.7.8.el7.x86_64
rpm -e --nodeps java-1.8.0-openjdk-1.8.0.102-4.b14.el7.x86_64
rpm -e --nodeps java-1.8.0-openjdk-headless-1.8.0.102-4.b14.el7.x86_64
rpm -e --nodeps java-1.7.0-openjdk-headless-1.7.0.111-2.6.7.8.el7.x86_64

2. Download

Note: version 221 jdk,adopt wget The download needs to be verified and can be downloaded after logging in through the official website
**jdk1.8 221 edition:
Link: https://pan.baidu.com/s/1RCEJfWgTkAE1pVm-DeLrzw
 Extraction code: iu94

3. Installation

mkdir /etc/java
cd /etc/java
tar -zxvf jdk-8u221-linux-x64.tar.gz
vim /etc/profile

export JAVA_HOME=/etc/java/jdk1.8.0_221
CLASSPATH=.:$JAVA_HOME/jre/lib/rt.jar:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
export PATH=$PATH:$JAVA_HOME/bin

Make configuration effective
source /etc/profile

testing environment
java -version

2.zookeeper installation

  • download

Download File: http://zookeeper.apache.org/releases.html As the file I downloaded: zookeeper-3.4.14.tar.gz

1) windows installation

1) Unzip zookeeper to D:\zookeeper-3.4.14 directory

Open: D:\zookeeper-3.4.14\conf, copy zoo_sample.cfg, rename it to zoo.cfg, open the zoo.cfg file (the file I opened with notepad + +), find the following information and make modifications

hold dataDir Change the value of to“./zookeeper-3.4.14/data"

(2) Add zookeeper environment variable

ZOOKEEPER_HOME: D:\zookeeper-3.4.14

New in Path, add;% ZOOKEEPER_HOME%\bin

(3) Run zookeeper

(4) Open the cmd command window to execute zkserver

Note: do not close the cmd window here

2) linux Installation

linux opens the port portal , port 2181 needs to be opened here

1.tar -xzvf file directory - C target directory. See the following figure:

2. Configuration

cd conf/

We can see that there is a log4j log file and a zoo_sample file. Zoo_sample is the configuration template, so we can use the template as a reference for our configuration. Directly copy one, named zoo.fg.

vim zoo.cfg

tickTime:Heartbeat configuration, unit: ms, two failed heartbeats are down. 
initLimit:How many heartbeats are allowed after other server initialization.
dataDir:The folder used to store the snapshot of the in memory database and the cluster myid Files also exist in this folder (Note: a configuration file can only contain one dataDir Words, even if it is annotated.)
clientPort=2181 # Service listening port, which can be modified as needed

Here you can modify dataDir, which is the place where zk's data is stored. Just set it

3. Start

cd ../bin
zkCli.sh -server 127.0.0.1:2181,-server 127.0.0.1:2181 This is a service server of ip And ports)

3. Installation and startup kafka

1) windows installation startup kafka

3.1 download installation files: http://kafka.apache.org/downloads.html

3.2 extract the file D:\program\kafka_2.11-2.0.0

3.3 open D:\program\kafka_2.11-2.0.0\config

3.4 open server.properties from a text editor

3.5 change the value of log.dirs to "log. Dirs = D: \ program \ kafka_2.11-2.0.0 \ Kafka logs"

3.6 open cmd

3.7 enter Kafka file directory: CD / D D D: \ program \ kafka_2.11-2.0.0 \ bin \ windows

3.8 enter and execute to open kafka:

kafka-server-start ../../config/server.properties

2) linux Installation startup kafka

1.tar -zxvf kafka_2.11-2.3.0.tgz
2.cd config modify the configuration as needed
① Modify where logs are stored

log.dirs = /root/3rd/kafka_2.11-2.3.0/log

3. Simple stand-alone startup kafka

cd bin
kafka-server-start.sh .../config/server.propirty

Topics: Linux Windows kafka Middleware