Comparable to JMeter Net pressure measuring tool - Crank beginner level chapter

Posted by KingIsulgard on Tue, 08 Mar 2022 08:22:26 +0100

1. Preface

Crank Yes The benchmark infrastructure used by the. NET team to run benchmarks, including (but not limited to) from TechEmpower Web The framework benchmark scenario is 2021 NET Conf conference introduced a new project, its predecessor is Benchmarks.

One of the goals of Crank is to provide developers with a tool that makes it very easy for them to deal with performance and measure potential improvements. Some of these functions are:

Deployment and benchmarking are based on NET or Docker container

By specifying Net project (local path or git remote warehouse address), which supports direct deployment or application deployment through Docker for benchmarking)

Through Yml configuration, the results can be stored not only in JSON and SQL Server, but also in csv files for charts

At present, some small partners are already proposing to store support in es

Support changing the Franework environment of custom applications to test the performance in different environments
Collect diagnostic trace information

2. Core composition

Crank consists of Agent and Controller

The Controller is the scheduler of tasks, which can schedule load tasks and output results

Agent is the benchmark agent, which is the actual executor of tasks. It receives and executes tasks from the Controller.

3. Installation

If you want to work well first, you must first sharpen your tools. First, let's learn how to install crank and how to verify whether the installation is successful

3.1. preparation

install .NET 5.0.
Open shell: install Crank Controller

Installation command:

dotnet tool update Microsoft.Crank.Controller --version "0.2.0-alpha.21567.1" --global

Verify command:

crank

Open shell: install Crank Agent

Installation command:

dotnet tool update Microsoft.Crank.Agent --version "0.2.0-alpha.21567.1" --global

Verify command:

crank-agent

3.2. Summary

For the convenience of reading, in this article, the Crank Controller is abbreviated as Crank and the Crank Agent is abbreviated as Agent

Agent and Crank need to be installed according to the actual situation, which can be divided into the following situations:

Just to learn from Crank, if there is no separate test environment, you need to install Agent and Controller respectively
If the Agent has a separate test environment, there is no need to install the Agent locally, just install the Controller
If the Agent has a separate test environment and the pressure test task is triggered and executed by ci, there is no need to install any configuration locally. You can complete the pressure plan by building ci task

Open shell: View Agent and Controller versions

dotnet tool list -g

4. Basic knowledge

4.1. variables: parameters

variables are divided into two types: local parameters and global parameters. The parameters in the root node are global parameters and those in other nodes are local parameters.

Example:

hello. benchmarks. The serverPort and path under YML > scenarios > Hello load > variables node and the serverAddress under profiles > local > variables node are local parameters

scenarios:
  hello:
    application:
      job: server
    load:
      job: bombardier
      variables:
        serverPort: 5000
        path: /

profiles:
  local:
    variables:
      serverAddress: localhost

bombardier. YML > variables > headers are global parameters

variables:
  headers:
    none: ''
    plaintext: '--header "Accept: text/plain,text/html;q=0.9,application/xhtml+xml;q=0.9,application/xml;q=0.8,*/*;q=0.7" --header "Connection: keep-alive"'
    html: '--header "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8" --header "Connection: keep-alive"'
    json: '--header "Accept: application/json,text/html;q=0.9,application/xhtml+xml;q=0.9,application/xml;q=0.8,*/*;q=0.7" --header "Connection: keep-alive"'
    connectionclose: '--header "Connection: close"'
    ---------------------------------------------------------------------

4.2. profiles: Configuration

profiles are actually configuration file information. profiles can be used multiple times, which can be found in the document introduce.

Usage: crank [options]

Options:
  -?|-h|--help                       Show help information

  These options are not specific to a Job

  ----------------------------------------------------------------------
  --profile <profile>                Profiles to apply. Can be used multiple times.

Naming rules: recommended * profiles.yml

4.3. jobs: tasks

Define what we want to do as a job. Reuse after convenience. The thing here refers to a kind of thing, not a specific thing.

For example, Microsoft's built-in defined bombardier is a job that is created through bombardier Benchmark it and record and output the results. In fact, it doesn't care which interface is benchmarked.

According to the application source, job s can be divided into remote and local.

Local source:

jobs:
  server: #Task name, which can be named according to the role of the task
    source: #Task source
      localFolder: ../hello 
      project: hello.csproj #To build NET project
    readyStateText: Application started. #Text in the console notifying the server that it has started

The local source localFolder is the relative path of the currently running crank --config command. After the task starts, the local project will be sent to the agent and then the task will be executed.

Remote source

jobs:
  server:
    source:
      repository: https://github.com/dotnet/crank
      branchOrCommit: main #Branch of a task performed by a remote source
      project: samples/hello/hello.csproj #To build NET project file name, format: relative path of relative root + project name csproj
    readyStateText: Application started.

The remote source will send the warehouse information to the Agent. The Agent will download the warehouse and switch to the specified branch before executing the construction task and starting the project

4.4. scenarios: Scenarios

Jobs care about one kind of things, but specific things don't care. Who cares about the specific things? That's right. That's the scene, that is, scenarios. Scenarios benchmark the specified scene through multiple job s and do the arrangement of specific tasks

4.5. imports: imports

Imports provides us with the possibility of yml reuse. With the support of imports, we can extract the public yml into a separate yml, and import the used yml through imports, which is similar to the import of js and css

4.6. Summary

In crank, variables and profiles are not necessary, but because of their existence, we can develop with the idea of object-oriented. We can complete the benchmark test by adding variables or specifying configuration. This will be explained in detail in the actual combat later

5. Getting started

After the previous study, we also have a certain understanding of the basic configuration of crank. In the next time, we will first try to learn the Sample prepared by the official for us. The following tutorial will also explain the function of each configuration in detail. We hope to understand the basic working principle of crank through the following study

5.1. Start Agent

crank-agent --dotnethome "/home/{your-account}/dotnet"

5.1.0.1. Start the Agent and specify the dotnet environment

Format: crank agent – dotnethome "dotnet installation address"
crank-agent --dotnethome "C:\Program Files\dotnet" (windows)
crank-agent --dotnethome "/usr/share/dotnet" (Linux)

When starting the agent, we strongly recommend that you add the dotnethome configuration and specify the environment for the agent to run, so as to avoid getting stuck due to environmental problems when running the task install sdk here
Because the local dotnet of the demo machine uses the installation path of / home/gushoudao/dotnet, the commands running in the video are different, and the path needs to be adjusted according to the local actual situation. (for video recording reasons, the agent will be stopped after recording. We don't need to exit after starting in real use. Once we exit the agent, we can't execute the task.)

5.1.0.2. Start the Agent and specify not to clean up temporary files

Crank agen -- no cleanup (specify not to clean up temporary files)

The default agent will delete the temporary files generated during the execution of the current task after the execution of the task

5.1.0.3. Start the Agent and specify the maximum duration of the build task

crank-agent --build-timeout

The maximum duration of the default build task is 10 minutes

More configuration Click to view

5.2. New hello benchmarks. YML configuration

Configuration file source code from hello.benchmarks.yml

imports:
  - https://raw.githubusercontent.com/doddgu/crank/sample/src/Microsoft.Crank.Jobs.Bombardier/bombardier.yml

jobs:
  server:
    source:
      repository: https://github.com/doddgu/crank
      branchOrCommit: sample
      project: samples/hello/hello.csproj
    readyStateText: Application started.

scenarios:
  hello:
    application:
      job: server
    load:
      job: bombardier
      variables:
        serverPort: 5000
        path: /

profiles:
  local:
    variables:
      serverAddress: localhost
    jobs: 
      application:
        endpoints: 
          - http://localhost:5010
      load:
        endpoints: 
          - http://localhost:5010

5.3. Start task

Start agent (open and put aside):

crank-agent --dotnethome "/usr/share/dotnet"

Start task (start a new Shell):

git clone https://github.com/doddgu/crank.git
cd crank
git checkout sample
crank --config ./samples/hello/hello.original.benchmarks.yml --scenario hello --load.framework net5.0 --application.framework net5.0

Then we wait a moment and the following results will be output

crank-agent:

crank:

| load                  |                |
| --------------------- | -------------- |
| CPU Usage (%)         | 39             |  CPU Utilization rate
| Cores usage (%)       | 631            |  Multicore CPU Utilization rate
| Working Set (MB)      | 35             |  Memory usage
| Private Memory (MB)   | 35             |  The amount of private memory used by the process
| Build Time (ms)       | 4,853          |  How long (in milliseconds) does it take to build the application
| Start Time (ms)       | 386            |  How long (in milliseconds) does it take to start the application
| Published Size (KB)   | 66,731         |  The size of the published application (KB)
| .NET Core SDK Version | 5.0.403        |  .Net Core SDK edition
| ASP.NET Core Version  | 5.0.12+0bc3c37 |  .Net Core edition
| .NET Runtime Version  | 5.0.12+7211aa0 |  .Net Runtime version
| First Request (ms)    | 172            |  The first request takes time (here the request is Get)
| Requests              | 2,086,594      |  Total number of requests sent
| Bad responses         | 0              |  Bad requests (response status code is not 2)**Not 3**)
| Mean latency (us)     | 1,833          |  Average delay time
| Max latency (us)      | 89,001         |  Maximum delay time
| Requests/sec          | 138,067        |  Number of requests supported per second
| Requests/sec (max)    | 255,442        |  Maximum supported requests per second

When you can output the above information, it proves that you have successfully run through the whole process

We can clearly see the test results under the scenario hello above, including CPU utilization, multi-core CPU utilization, memory utilization and the number of requests executed per second

At this moment, do you suddenly feel that this crank is very powerful? Although it's not clear how to do it, it's really great!! At this moment, are you interested in it and want to know what it can do and why it can output the above results?

6. Conclusion

In order to ensure that the follow-up operation according to the document will not be unavailable due to the update, the source code is forked from the official source, most of which are from the samples and some files provided by the official. In order to better meet personal habits, adjustments will be made to a certain extent.

Source address: https://github.com/doddgu/cra...

Reference link: