Real time log analysis

Germeng Society AI: Keras PyTorch MXNet TensorFlow PaddlePaddle deep learning practice (updated from time to time) 4.4 real time log analysis Learning objectives target Master the connection between Flume and Kafka We have collected the log data into hadoop, but when doing real-time ana ...

Posted by pontiac007 on Thu, 18 Jun 2020 06:33:47 +0200

Concept and application of Spring Statemachine

From https://www.jianshu.com/p/9ee887e045dd , thank you 1 Finite-state machine 1.1 definition of state machine FSM (finite state machine, FSM), also known as FSM, is a mathematical model that represents finite states, their transitions and actions. Finite state machine embodies two points: first ...

Posted by echoindia756 on Thu, 18 Jun 2020 05:48:33 +0200

ELK log system theory and several schemes

Log system scene In general, we need to do log analysis scenarios: directly in the log file, grep and awk can get the information they want. However, in large-scale scenarios, this method is inefficient, facing problems such as how to archive too much logs, how to do too slow text search, and how t ...

Posted by mschrank on Sat, 13 Jun 2020 13:10:05 +0200

SpringBoot Distributed Task Middleware Development with video tutorials (hands-on instructions on developing and using middleware)

Author: Little FuBlog: https://bugstack.cn Sediment, sharing, growth, so that you and others can gain something!_ Distributed task DcsSchedule middleware, Github address: https://github.com/fuzhengwei/schedule-spring-boot-starter Distributed Task DcsSchedule Console, Github Address: https://github.com/fuzhengwei/itstack-middleware-control D ...

Posted by quak on Thu, 11 Jun 2020 03:44:00 +0200

Zookeeper implements service registration / Discovery (recommended Collection)

  what that? Zookeeper is frequently used in distributed development, but many frameworks encapsulate it. Beginners may not understand its working principle well. This article demonstrates the simple demo of using zookeeper to realize service registration and service discovery, hoping to achieve th ...

Posted by anushka on Tue, 09 Jun 2020 10:05:39 +0200

zabbix alarm information pushed to kafka

Application scenario Due to the high security requirements of the company where the friend is located, zabbix's network environment cannot be connected to the Internet, so it is not possible to send the alarm directly to some instant messaging tools through zabbix, which requires sending the alarm message to some middleware and forwarding it t ...

Posted by lanas on Tue, 26 May 2020 19:11:42 +0200

Simple application of Dubbo+ZooKeeper

1. What we want to achieve ​ Provider: the service provider that exposes the service. I create a Module named news provider to implement it. Consumer: the service consumer that calls the remote service. I create a Module named news consumer to implement it. Registry: the Registration Center for service registration and discovery, which is imp ...

Posted by xenoalien on Wed, 06 May 2020 08:31:48 +0200

spring boot integrated zookeeper registry

ZooKeeper is a distributed service framework and a sub project of Apache Hadoop. It is mainly used to solve some data management problems often encountered in distributed applications, such as: unified naming service, state synchronization service, cluster management, distributed application configuration item management, etc ZooKeeper is a tr ...

Posted by ph0ngwh0ng on Thu, 16 Apr 2020 11:54:18 +0200

Interpretation of FlinkKafkaConsumer011 method

FlinkKafkaConsumer011.setStartFromEarliest() /** *Specifies that consumer reads from the earliest offset of all partitions. *This method causes consumer to ignore any submitted offset s in Zookeeper / Kafka brokers. * *When consumer is restored from checkpoint or savepoint, this method does not work. In this case, the offset in the r ...

Posted by malec on Fri, 10 Apr 2020 13:33:32 +0200

Solve Bug Path - dubbo apps cannot reconnect zookeeper

Preface Dubbo is a mature and widely used framework.Rao is such that in some extreme cases dubbo-based applications will also have the problem of not being able to reconnect zookeeper.Because this problem is easy to cause relatively large failures, the author has spent a lot of time to locate, and now will write a blog to share the investigatio ...

Posted by rschneid on Tue, 07 Apr 2020 05:02:01 +0200