Using Kafka under Windows

Posted by daverules on Mon, 20 Jan 2020 18:22:24 +0100

1, Prepare ZooKeeper

Download the latest stable version from the official website: 3.5.6 . After downloading, extract it to the local location, find zoo_sample.cfg in the conf directory, rename it to zoo.cfg, and modify the data storage directory in it:


Start with git command line client:

## Test start, normal
$ ./ start
ZooKeeper JMX enabled by default
Using config: D:\hecg\apache-zookeeper-3.5.6-bin\conf\zoo.cfg
Starting zookeeper ... STARTED
## Test stop, normal
$ ./ stop
ZooKeeper JMX enabled by default
Using config: D:\hecg\apache-zookeeper-3.5.6-bin\conf\zoo.cfg

2, Prepare Kafka

The version downloaded from the official website is: kafka_2.12-2.4.0

2.1 modify configuration file
  • Modify the kafka configuration file in the config directory and the log storage directory:

  • Modify the zookeeper configuration file in the config directory, and modify the data storage directory:

2.2 start ZooKeeper:
$ bin/ config/
[2020-01-20 14:46:01,282] INFO Reading configuration from: config/ (org.apache.zookeeper.server.quorum.QuorumPeerConfig)
[2020-01-20 14:46:01,283] WARN config\ is relative. Prepend.\ to indicate that you're sure! (org.apache.zookeeper.server.quorum.QuorumPeerConfig)
[2020-01-20 14:46:01,284] INFO clientPortAddress is (org.apache.zookeeper.server.quorum.QuorumPeerConfig)
2.3 start Kafka Server:
$ bin/ config/
[2020-01-20 14:50:57,961] INFO Registered kafka:type=kafka.Log4jController MBean (kafka.utils.Log4jControllerRegistration$)
[2020-01-20 14:50:58,388] INFO starting (kafka.server.KafkaServer)
[2020-01-20 14:50:58,389] INFO Connecting to zookeeper on localhost:2181 (kafka.server.KafkaServer)
[2020-01-20 14:50:58,407] INFO [ZooKeeperClient Kafka server] Initializing a new session to localhost:2181. (kafka.zookeeper.ZooKeeperClient)
[2020-01-20 14:50:58,412] INFO Client environment:zookeeper.version=3.5.6-c11b7e26bc554b8523dc929761dd28808913f091, built on 10/08/2019 20:18 GMT (org.apache.zookeeper.ZooKeeper)
2.4 create Topic
  • To create a Topic:

    $ bin/ --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 1 --topic test
  • List all topics:

    $ bin/ --list --bootstrap-server localhost:9092

During this period, there is a little episode: the configuration file cannot be loaded into log4j: after the configuration file is written, it is ready

$ bin/ --list --bootstrap-server localhost:9092
log4j:ERROR Could not read configuration file from URL [file:/d/hecg/kafka_2.12-2.4.0/bin/../config/]. \d\hecg\kafka_2.12-2.4.0\bin\..\config\

## Modify line 194
2.5 sending and receiving message test
  • Specify Topic to send message:

    $ bin/ --broker-list localhost:9092 --topic test
    >This is a message
    >This is another message
    >hello hecg
  • Specify the Topic to receive the message:

    $ bin/ --bootstrap-server localhost:9092 --topic test --from-beginning
    This is a message
    This is another message
    hello hecg

Topics: Programming Zookeeper kafka Apache log4j