Integrated kafka of spring boot

The installation and principles of kafka and zookeeper were mentioned in another blog post. I won't talk about this. I'll just talk about how to integrate kafka in spring boot project. In this article, I will mainly talk about two ways: Method 1: We use spring-integration-kafka, a plug-in of Kafka integration in spring. We mainly talk about ...

Posted by mdgottlieb on Thu, 30 May 2019 20:52:02 +0200

Elastic search 429, logstash does not update kafka queue status

Recently, a new problem has arisen. When using filebeat to load historical log files, a large amount of data will be added to the kafka queue. logstash will read a large number of event s from the queue and send them to the ES side. The initial manifestations are: logstash keeps sending messages to ES, and the data in ES keeps increasing. The ...

Posted by !jazz on Sat, 25 May 2019 20:26:52 +0200

docker builds ELK, logstash reads data from kafka

Set up ELK to do log analysis for recording. Installation of kafka and docker is not explained here. First, build elastic search docker run -d elasticsearch es has an official image, so there is no other requirement, so you can run an elastic search container, but I specify the port mapping and configuration file and data file volume, so ...

Posted by pliant on Tue, 21 May 2019 23:09:47 +0200

Distributed Project Mapping Server Data Mapping

In the last section, the implementation of CoAp client and server, the data is also sent to kafka in the format defined by the installation, followed by the implementation of Mapping server, which maps physical device data to abstract devices and gives data business meaning. iot-mapping Building iot-mapping module, introducing kafka public modu ...

Posted by nominator on Sat, 11 May 2019 20:02:02 +0200