Monitoring springboot projects with skywalking

Posted by SsirhC on Wed, 05 Jan 2022 00:39:07 +0100

 

1. Introduction to SkyWalking

SkyWalking is a domestic open source call chain analysis and application monitoring analysis tool based on bytecode injection.
It is characterized by supporting a variety of plug-ins, strong UI function and no code intrusion at the access end.
At present, most manufacturers are used and the version is updated quickly. It has become the top project of Apache foundation.
 
What are its functions?
  • A variety of monitoring means. Monitoring data can be obtained through language probe and service mesh.
  • Multiple language automatic probes. Including Java NET Core and node JS.
  • Lightweight and efficient. There is no need for big data platform and a large number of server resources.
  • modularization. UI, storage and cluster management all have a variety of optional mechanisms.
  • Support alarm.
  • Excellent visualization solution.

Its overall architecture

The whole architecture is divided into four parts: top, bottom, left and right:

Here, we abandon the Metric index and focus on the tracking link related functions.

  • The upper sub Agent is responsible for collecting link information from the application and sending it to SkyWalking OAP server. Currently, it supports tracking data information provided by SkyWalking, Zikpin, Jaeger, etc. At present, we use SkyWalking Agent to collect SkyWalking tracking data and transfer it to the server.
  • The lower part is divided into SkyWalking OAP: it is responsible for receiving the tracking data information sent by the Agent, then analyzing (Analysis Core), storing it in the external Storage, and finally providing the Query function.
  • Right: Tracing data Storage. At present, it supports ES, MySQL, Sharding Sphere, TiDB and H2 memory. At present, we use ES, mainly considering that the SkyWalking development team's own production environment mainly adopts ES.
  • The left part is divided into} SkyWalking UI: responsible for providing console, viewing links, etc.

2. Download and install skywalking

download

Download address

 

Installation

Unzipped directory

 

3. Start

Execute startup.exe in the bin directory Bat to start the service

Execute startup After bat, the following two services will be started:
(1) Skywalking collector: a tracking information collector, which collects the collection information of the client through gRPC/Http. The default port of Http is 12800 and gRPC is 11800.
(2) Skywalking webapp: the default port of the management platform page is 8080

4. Configuration information

It can be used to set the data storage. If it is local, it can also be used as the default without modification. It is only used to view the effect of integrating skywalking with springboot project without modification.

4.1 OAP configuration file

config/application.yml

It mainly configures SkyWakling cluster mode and data storage. The content of the configuration file is as follows

core:
  default:
  restHost: ${SW_CORE_REST_HOST:0.0.0.0}
  restPort: ${SW_CORE_REST_PORT:12800}
  restContextPath: ${SW_CORE_REST_CONTEXT_PATH:/}
  gRPCHost: ${SW_CORE_GRPC_HOST:0.0.0.0}
  gRPCPort: ${SW_CORE_GRPC_PORT:11800}
  downsampling:
    - Hour
    - Day
    - Month
  # Set a timeout on metric data. After the timeout has expired, the metric data will automatically be deleted.
  recordDataTTL: ${SW_CORE_RECORD_DATA_TTL:90} # Unit is minute
  minuteMetricsDataTTL: ${SW_CORE_MINUTE_METRIC_DATA_TTL:90} # Unit is minute
  hourMetricsDataTTL: ${SW_CORE_HOUR_METRIC_DATA_TTL:36} # Unit is hour
  dayMetricsDataTTL: ${SW_CORE_DAY_METRIC_DATA_TTL:45} # Unit is day
  monthMetricsDataTTL: ${SW_CORE_MONTH_METRIC_DATA_TTL:18} # Unit is month
storage:
  h2:
    driver: ${SW_STORAGE_H2_DRIVER:org.h2.jdbcx.JdbcDataSource}
    url: ${SW_STORAGE_H2_URL:jdbc:h2:mem:skywalking-oap-db}
    user: ${SW_STORAGE_H2_USER:sa}
    #  elasticsearch:
    #    # nameSpace: ${SW_NAMESPACE:""}
    #    clusterNodes:     ${SW_STORAGE_ES_CLUSTER_NODES:localhost:9200}
    #    indexShardsNumber: ${SW_STORAGE_ES_INDEX_SHARDS_NUMBER:2}
    #    indexReplicasNumber: ${SW_STORAGE_ES_INDEX_REPLICAS_NUMBER:0}
    #    # Batch process setting, refer to https://www.elastic.co/guide/en/elasticsearch/client/java-api/5.5/java-docs-bulk-processor.html
    #    bulkActions: ${SW_STORAGE_ES_BULK_ACTIONS:2000} # Execute the bulk every 2000 requests
    #    bulkSize: ${SW_STORAGE_ES_BULK_SIZE:20} # flush the bulk every 20mb
    #    flushInterval: ${SW_STORAGE_ES_FLUSH_INTERVAL:10} # flush the bulk every 10 seconds whatever the number of requests
    #    concurrentRequests: ${SW_STORAGE_ES_CONCURRENT_REQUESTS:2} # the number of concurrent requests
receiver-register:
    default:
receiver-trace:
    default:
bufferPath: ${SW_RECEIVER_BUFFER_PATH:../trace-buffer/}  # Path to trace buffer files, suggest to use absolute path
bufferOffsetMaxFileSize: ${SW_RECEIVER_BUFFER_OFFSET_MAX_FILE_SIZE:100} # Unit is MB
bufferDataMaxFileSize: ${SW_RECEIVER_BUFFER_DATA_MAX_FILE_SIZE:500} # Unit is MB
bufferFileCleanWhenRestart: ${SW_RECEIVER_BUFFER_FILE_CLEAN_WHEN_RESTART:false}
sampleRate: ${SW_TRACE_SAMPLE_RATE:10000} # The sample rate precision is 1/10000. 10000 means 100% sample in default.
receiver-jvm:
    default:
#service-mesh:

5. Deploy probes

5.1.IDEA deployment probe

Modify VM running parameters of project startup

1. Click Run - > Edit configurations in the menu bar

 

 

 

2. Add the following parameters to VM options:

-javaagent:C:\Users\ke\Desktop\apache-skywalking-apm-6.6.0\apache-skywalking-apm-bin\agent\skywalking-agent.jar 
-Dskywalking.agent.service_name=service-myapp
-javaagent: used to specify the probe path
-Dskywalking.agent.service_name: used to rewrite agent / config / agent Service name in config configuration file
attach -Dskywalking.collector.backend_service: used to rewrite agent / config / agent Service address in config configuration file After startup, you can see the following startup log:
  INFO 2020-02-13 14:57:36:310 main SnifferConfigInitializer : Config file found in C:\Users\ke\Desktop\apache-skywalking-apm-6.6.0\apache-skywalking-apm-bin\agent\config\agent.config.
5.2. Java command line startup mode
java -javaagent:C:\Users\ke\Desktop\apache-skywalking-apm-6.6.0\apache-skywalking-apm-bin\agent/skywalking-agent.jar=-Dskywalking.agent.service_name=service-myapp,-Dskywalking.collector.backend_service=localhost:11800 -jar service-myapp.jar
5.3. Script startup

Script writing (can be used after modification)

[root@basenode demo]# vi startup.sh 

#!/bin/sh
# SkyWalking Agent configuration
export SW_AGENT_NAME=springboot-skywalking-wudl #Agent name, usually ` spring application. name`
export SW_AGENT_COLLECTOR_BACKEND_SERVICES=192.168.1.180:11800 #Configure the Collector address.
export SW_AGENT_SPAN_LIMIT=2000 #Configure the maximum number of spans for the link. The default is 300. 
export JAVA_AGENT=-javaagent:/opt/module/skywalking-apm-bin-es7/agent/skywalking-agent.jar
java $JAVA_AGENT -jar /opt/module/demo/user-center-0.0.1-SNAPSHOT.jar #jar startup

function

[root@basenode demo]# sh startup.sh 
DEBUG 2021-09-12 18:01:30:050 main AgentPackagePath : The beacon class location is jar:file:/opt/module/skywalking-apm-bin-es7/agent/skywalking-agent.jar!/org/apache/skywalking/apm/agent/core/boot/AgentPackagePath.class. 
INFO 2021-09-12 18:01:30:051 main SnifferConfigInitializer : Config file found in /opt/module/skywalking-apm-bin-es7/agent/config/agent.config. 

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::        (v2.2.2.RELEASE)

2021-09-12 18:01:39.170  INFO 5590 --- [           main] c.wudl.usercenter.UserCenterApplication  : Starting UserCenterApplication v0.0.1-SNAPSHOT on basenode with PID 5590 (/opt/module/demo/user-center-0.0.1-SNAPSHOT.jar started by root in /opt/module/demo)
2021-09-12 18:01:39.177  INFO 5590 --- [           main] c.wudl.usercenter.UserCenterApplication  : The following profiles are active: dev
2021-09-12 18:01:43.948  INFO 5590 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat initialized with port(s): 9909 (http)
2021-09-12 18:01:44.050  INFO 5590 --- [           main] o.apache.catalina.core.StandardService   : Starting service [Tomcat]
2021-09-12 18:01:44.050  INFO 5590 --- [           main] org.apache.catalina.core.StandardEngine  : Starting Servlet engine: [Apache Tomcat/9.0.29]
2021-09-12 18:01:44.196  INFO 5590 --- [           main] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring embedded WebApplicationContext
2021-09-12 18:01:44.197  INFO 5590 --- [           main] o.s.web.context.ContextLoader            : Root WebApplicationContext: initialization completed in 4819 ms
2021-09-12 18:01:47.260  INFO 5590 --- [           main] o.s.s.concurrent.ThreadPoolTaskExecutor  : Initializing ExecutorService 'applicationTaskExecutor'
2021-09-12 18:01:47.956  INFO 5590 --- [           main] t.m.m.autoconfigure.MapperCacheDisabler  : Clear tk.mybatis.mapper.util.MsUtil CLASS_CACHE cache.
2021-09-12 18:01:47.956  INFO 5590 --- [           main] t.m.m.autoconfigure.MapperCacheDisabler  : Clear tk.mybatis.mapper.genid.GenIdUtil CACHE cache.
2021-09-12 18:01:47.957  INFO 5590 --- [           main] t.m.m.autoconfigure.MapperCacheDisabler  : Clear tk.mybatis.mapper.version.VersionUtil CACHE cache.
2021-09-12 18:01:47.957  INFO 5590 --- [           main] t.m.m.autoconfigure.MapperCacheDisabler  : Clear EntityHelper entityTableMap cache.
2021-09-12 18:01:48.365  INFO 5590 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat started on port(s): 9909 (http) with context path ''
2021-09-12 18:01:48.370  INFO 5590 --- [           main] c.wudl.usercenter.UserCenterApplication  : Started UserCenterApplication in 11.576 seconds (JVM running for 18.419)

6. Simple test

After that, you can go to the SkyWalking UI to check whether the link collection is successful.

1. First, use the browser to access http://127.0.0.1:8079/demo/echo Address and request the API provided by the Spring Boot application. Because we need to track the link.

2. Then, continue using the browser and open http://127.0.0.1:8080/ Address and enter the SkyWalking UI interface. As shown in the figure below:

Here, we will see three very important concepts in SkyWalking:

  • Service: represents a series or set of workloads that provide the same behavior to requests. When using the Agent or SDK, you can define the name of the service. If not defined, SkyWalking will use the name you defined on the platform (e.g. Istio).

    Here, we can see that the service of the Spring Boot application is "demo application", that is, we are in the environment variable SW_ AGENT_ As defined in name +.

  • Service instance: each workload in the above set of workloads is called an instance. Like pods in Kubernetes, a service instance is not necessarily a process on the operating system. But when you use Agent, a service instance is actually a real process on the operating system.

    Here, we can see that the service of Spring Boot application is {agent_name}-pid:{pid}@{hostname}, which is automatically generated by the Agent. About it, we are 「5.1 hostname」 In the section, there is further explanation, fat friends can have a look.

  • Endpoint: the request path received by a specific service, such as the URI path of HTTP and the class name + method signature of gRPC service.

    Here, we can see an endpoint of the Spring Boot application, which is the API interface / demo/echo.

3. Then, click the "topology map" menu to enter the view topology map interface. As shown in the figure below:

4. After that, click the "tracking" menu to enter the interface of viewing link data. As shown in the figure below:

 

7. Others

You can also set up a skywalking cluster environment. The steps are as follows:

To build a SkyWalking} cluster environment, the steps are as follows:

  • The first step is to build a cluster of Elasticsearch services.
  • The second step is to build a cluster of registration centers. At present, SkyWalking supports Zookeeper, Kubernetes, Consul and Nacos as registration centers.
  • Step 3: build a cluster of SkyWalking OAP services, and refer to SkyWalking documentation - cluster management , register the SkyWalking OAP service with the registry.
  • Step 4: start a Spring Boot application and configure SkyWalking Agent. In addition, set the SW of SkyWalking Agent_ AGENT_ COLLECTOR_ BACKEND_ For services} address, you need to set multiple address arrays of SkyWalking OAP services.
  • Step 5: build a cluster of SkyWalking UI services and use Nginx for load balancing. In addition, set the SkyWalking UI's {collector ribbon. When listofservers} addresses, you also need to set multiple address arrays of SkyWalking OAP services.

Set alarm, etc.

 

Reference:

https://skywalking.apache.org/zh/2020-04-19-skywalking-quick-start/

https://www.jianshu.com/p/5524b4545421

https://www.jianshu.com/p/5bb6e9d289c8

Topics: skywalking