SpringBoot integrates Prometheus to realize application monitoring

Posted by SteveMellor on Mon, 14 Feb 2022 11:18:24 +0100

1. Introduction to micrometer

Micrometer provides a general API for performance data collection on the Java platform. Applications only need to use the general API of micrometer to collect performance indicators. Micrometer will be responsible for completing the adaptation with different monitoring systems. This makes it easy to switch the monitoring system. Micrometer also supports pushing data to multiple different monitoring systems. Micrometer is similar to SLF4J in log system.

There are two core concepts in Micrometer, namely, meter and meter registry. Let's take a look at these two concepts respectively.

1.1 Meter

Meter is used to collect performance indicator data (Metris). There are four types of meters: Counter, Gauge, Timer and Summary.

Each Meter has its own name, and the Meter can specify a series of tags. Tag is in the form of key value, so we can filter indicators according to tag. In addition to the unique tag of each Meter, you can also add a general tag through MeterRegistry.

MeterRegistry.Config config = simpleMeterRegistry.config();
    config.commonTags("tag1","value1","tag2","value2");

Counter

Counter can only increase the value. The count value represented by counter is of double type. By default, the increased value is 1.0

@Autowired
private SimpleMeterRegistry simpleMeterRegistry;

@Bean
public Counter counter1(){
    return Counter.builder("test.count1").register(simpleMeterRegistry);
}

@Bean
public Counter counter2(){
    return simpleMeterRegistry.counter("test.count2");
}

@Test
public void test(){
    counter1.increment();
}

Gauge

Cauge is a value representing a single change, such as temperature and air pressure. The difference from Counter is that the value of Gauge does not always increase

public void guage(){
    Gauge.builder("guaua1", this::getValue).register(simpleMeterRegistry);
}

public double getValue(){
    return ThreadLocalRandom.current().nextDouble();
}

Once a gauge object is created, its values cannot be modified manually. Gauge returns the current value at each sampling

Timer

Timer is usually used to record the duration of an event. Timer will record two types of data, the number of events and the total duration. Timer provides different ways to record duration. The first method is to use the record() method to record the running time of Runnable and Callable objects, and the second method is to use timer Sample to save the timing status

public void record(){
    Timer timer = simpleMeterRegistry.timer("record");
    timer.record(() -> {
        try {
            Thread.sleep(3000);
        }catch (Exception e){
            e.printStackTrace();
        }
    });
}

public void sample(){
    Timer.Sample sample = Timer.start();
    new Thread(()->{
        try {
            Thread.sleep(3000);
        }catch (Exception e){
            e.printStackTrace();
        }
        sample.stop(simpleMeterRegistry.timer("sample"));
    });
}

summary

Summary is used to record the distribution of indicators. According to the value of each indicator, summary allocates the value to the corresponding bucket. The default bucket value of Micrometer ranges from 1 to long MAX_ Value, you can control the bucket range through minimumExpectedValue and maximumExpectedValue. If the indicator value is small, you can also set a value through scale to enlarge the value

public void summary(){
    DistributionSummary summary = DistributionSummary.builder("summary")
            .maximumExpectedValue(10L)
            .minimumExpectedValue(1L)
            .publishPercentiles(0.5, 0.75, 0.9)
            .register(simpleMeterRegistry);

    summary.record(1.0);
    summary.record(5.0);
    summary.record(4.5);
    summary.record(3.0);

    System.out.println(summary.takeSnapshot());
}

1.2 meter registry

Meter registry is responsible for creating and maintaining meters. Each monitoring system has its own unique registry.

SimpleMeterRegistry is a memory based registry. It does not support exporting data to the monitoring system. It is mainly used for local development and testing.

Micrometer supports multiple different monitoring systems. Multiple meter registries can be combined through compositemeter registry, which allows data to be published to multiple monitoring systems at the same time.

public void compositeRegistry(){
        CompositeMeterRegistry compositeMeterRegistry = new CompositeMeterRegistry();
        compositeMeterRegistry.add(new SimpleMeterRegistry());
        compositeMeterRegistry.add(new SimpleMeterRegistry(new SimpleConfig() {
            @Override
            public String get(String s) {
                return null;
            }

            //Add prefix
            @Override
            public String prefix() {
                return "simple";
            }
        },Clock.SYSTEM));

        Counter counter = compositeMeterRegistry.counter("test");
        counter.increment();
    }
}

Micrometer itself provides a static global registry Metrics golbalRegistry. This registry is a composite registry. Gauges created using static methods in the Metrics class will be added to this global registry

public void globalRegistry(){
    Metrics.addRegistry(simpleMeterRegistry);
    Counter global = Metrics.counter("global");
    global.increment();
}

2.SpringBoot Actuator

Micrometer is the performance indicator collection library provided by Spring Boot by default. SpringBoot Actuator provides automatic configuration of micrometer. Introduce SpringBoot Actuator into the project,

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-actuator</artifactId>
</dependency>

Add the following configuration in the configuration file

Actuator can provide default services externally, * indicates that all services are displayed

management.endpoints.web.exposure.include=*

Start project, access http://8080/actuator , you can see all the monitoring provided by the actor

{
  "_links": {
    "self": {
      "href": "http://localhost:8080/actuator",
      "templated": false
    },
    "auditevents": {
      "href": "http://localhost:8080/actuator/auditevents",
      "templated": false
    },
    "beans": {
      "href": "http://localhost:8080/actuator/beans",
      "templated": false
    },
    "caches-cache": {
      "href": "http://localhost:8080/actuator/caches/{cache}",
      "templated": true
    },
    "caches": {
      "href": "http://localhost:8080/actuator/caches",
      "templated": false
    },
    "health": {
      "href": "http://localhost:8080/actuator/health",
      "templated": false
    },
    "health-component": {
      "href": "http://localhost:8080/actuator/health/{component}",
      "templated": true
    },
    "health-component-instance": {
      "href": "http://localhost:8080/actuator/health/{component}/{instance}",
      "templated": true
    },
    "conditions": {
      "href": "http://localhost:8080/actuator/conditions",
      "templated": false
    },
    "configprops": {
      "href": "http://localhost:8080/actuator/configprops",
      "templated": false
    },
    "env": {
      "href": "http://localhost:8080/actuator/env",
      "templated": false
    },
    "env-toMatch": {
      "href": "http://localhost:8080/actuator/env/{toMatch}",
      "templated": true
    },
    "info": {
      "href": "http://localhost:8080/actuator/info",
      "templated": false
    },
    "loggers": {
      "href": "http://localhost:8080/actuator/loggers",
      "templated": false
    },
    "loggers-name": {
      "href": "http://localhost:8080/actuator/loggers/{name}",
      "templated": true
    },
    "heapdump": {
      "href": "http://localhost:8080/actuator/heapdump",
      "templated": false
    },
    "threaddump": {
      "href": "http://localhost:8080/actuator/threaddump",
      "templated": false
    },
    "prometheus": {
      "href": "http://localhost:8080/actuator/prometheus",
      "templated": false
    },
    "metrics": {
      "href": "http://localhost:8080/actuator/metrics",
      "templated": false
    },
    "metrics-requiredMetricName": {
      "href": "http://localhost:8080/actuator/metrics/{requiredMetricName}",
      "templated": true
    },
    "scheduledtasks": {
      "href": "http://localhost:8080/actuator/scheduledtasks",
      "templated": false
    },
    "httptrace": {
      "href": "http://localhost:8080/actuator/httptrace",
      "templated": false
    },
    "mappings": {
      "href": "http://localhost:8080/actuator/mappings",
      "templated": false
    }
  }
}

visit http://localhost:8080/actuator/metrics , you can see the monitoring indicators collected by Actuator by default, including JVM related indicators (memory usage, garbage collection), tomcat related indicators, database connection pool or system related indicators

{
  "names": [
    "jvm.memory.max",
    "jvm.threads.states",
    "process.files.max",
    "jvm.gc.memory.promoted",
    "system.load.average.1m",
    "jvm.memory.used",
    "jvm.gc.max.data.size",
    "jvm.gc.pause",
    "jvm.memory.committed",
    "system.cpu.count",
    "logback.events",
    "tomcat.global.sent",
    "jvm.buffer.memory.used",
    "tomcat.sessions.created",
    "jvm.threads.daemon",
    "system.cpu.usage",
    "jvm.gc.memory.allocated",
    "tomcat.global.request.max",
    "tomcat.global.request",
    "tomcat.sessions.expired",
    "jvm.threads.live",
    "jvm.threads.peak",
    "tomcat.global.received",
    "process.uptime",
    "tomcat.sessions.rejected",
    "process.cpu.usage",
    "http.server.requests",
    "tomcat.threads.config.max",
    "jvm.classes.loaded",
    "jvm.classes.unloaded",
    "tomcat.global.error",
    "tomcat.sessions.active.current",
    "tomcat.sessions.alive.max",
    "jvm.gc.live.data.size",
    "tomcat.threads.current",
    "process.files.open",
    "jvm.buffer.count",
    "jvm.buffer.total.capacity",
    "tomcat.sessions.active.max",
    "tomcat.threads.busy",
    "process.start.time"
  ]
}

3.Prometheus

Micrometer supports Prometheus. Micrometer provides the Prometheus meter registry, which is used to convert indicators to Prometheus format indicators. First, you need to introduce dependencies into the pom file

<dependency>
    <groupId>io.micrometer</groupId>
    <artifactId>micrometer-registry-prometheus</artifactId>
</dependency>
management.endpoint.prometheus.enabled=true
management.metrics.export.prometheus.enabled=true
docker run -d -p 9090:9090 -v ~/Documents/config/prometheus.yml:/etc/prometheus/prometheus.yml prom/prometheus
scrape_configs:
 - job_name: prometheus-test
   metrics_path: /actuator/prometheus
   static_configs:
   - targets: ['192.168.66.66:8080']

http://localhost:9090/targets You can see the Prometheus acquisition configuration

4. Customize Metric

We can use Prometheus client to customize the metric

import io.prometheus.client.CollectorRegistry;
import io.prometheus.client.Counter;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;

import javax.annotation.PostConstruct;


@Service
public class PrometheusMeter {

    @Autowired
    private CollectorRegistry collectorRegistry;

    // Define name as prometheus_counter of counter
    public Counter prometheusCounter(){
        return Counter.build().name("prometheus_counter").help("prometheus counter test")
                .register(collectorRegistry);
    }

    @PostConstruct
    public void init(){
        Counter counter = prometheusCounter();
        new Thread(()-> {
            while (true){
                counter.inc();
                try {
                    Thread.sleep(5000);
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }
            }
        }).start();
    }
}

After starting the project, you can see the index Prometheus just defined on the Prometheus query page_ counter

5. Summary

  1. Micrometer integrates multiple monitoring systems, including Prometheus. Micrometer uses Meter to collect data, and uses different meterregistries to integrate with different monitoring systems
  2. SpringBoot Actuator integrates Micrometer and defines many default metrics, which can be used in http://localhost:8080/actuator/metrics see
  3. SpringBoot Actuator can import the collected indicators into Prometheus through Micrometer

Topics: Java Spring Boot Back-end Microservices