1, Interpretation of performance test terms in the traditional sense

Pressure test:

Stress test is to evaluate the operation of the system when the system is at or above the expected load. The stress test focuses on the processing capacity of the system under peak load or beyond the maximum load. As the pressure level gradually increases, the system performance should decline slowly as expected, but should not collapse. Stress testing can also find the critical point of system collapse, so as to find the weak links in the system.

Capacity test:

Determine the maximum number of users that the system can process online at the same time, and make the system bear the excess data capacity to find out whether it can process correctly.　　

Limit test:

Load test under excessive users.

 

2, Reasonable definition of performance test:

Performance test: according to the performance indicators of the system, establish the performance test model, formulate the performance test scheme, formulate the monitoring strategy, execute the performance scenario under the scenario conditions, analyze and judge the performance bottleneck and optimize, and finally obtain the performance results to evaluate whether the performance indicators of the system meet the set value.

　　

Summarize the concept of performance testing:

 

 

1. Performance test demand index

Some people say that when we are working on the project, there are no indicators. The boss only says that the system is crushed to death. It sounds childish, but there are not a few such scenes. In my opinion, crushing the system is also an indicator. As for what means you use to "crush" the system, that is the problem of implementation.
You can use many means to tell your boss that the system is dead before it is pressed! This is also your contribution. The definition of "with indicators" is theoretically reasonable, and the indicators should be:Time index, capacity index and resource utilization index.　　

Suppose the company starts performance testing from 0

• The first stage: do a good job of performance test and get the performance index value (usually the first performance test is also called single benchmark test)
• The second stage: assuming that the performance is worse than before, which performance index values do not meet the expected values, we need to analyze what the problem is

Version iteration, conduct the second performance test, and run the previous performance script again

• Some performance index values will be obtained. Compare the performance index values of the previous version to see whether there is optimization (performance change)
• Assuming that it is normal for 120 people to place orders at the same time and 150 talents are abnormal, the interface has been optimized

 

2. Performance testing requires models

What is type? It is an abstraction of the real scene and can tell the performance tester what the business model looks like. For example, we have 100 kinds of businesses, but not every business needs concurrency. Maybe only 50 businesses have concurrency, so we need to count these concurrent businesses. Which business has more concurrency and which business has less concurrency, so we should control this proportion when doing stress.

We need to choose a way suitable for our system business logic and do things with the lowest cost and the fastest time.

 

3. There should be a scheme for performance test

There are several key points in the contents specified in the scheme, which areTest environment, test data, test model, performance index, stress strategy, admission and exit and schedule risk.Basically, these contents are enough, and the specific information of these contents needs to be accurate.
You might say, why don't you have a test plan? My suggestion is to draw test plans separately with project management tools, such as Project or OmniPlan Tools like that. This is because in the scheme, when writing the test plan, basically only one milestone can be written. Further refinement is to add several major stage items.
However, planning with project management tools is different. It can not only subdivide items, but also track the dynamic progress of each work. It can set up before and after dependencies and fill in resources and costs to calculate project deviation.　

 

4. There should be monitoring in the performance test

The monitoring of this part should have the ability of layering and segmentation, as well as the ability of global monitoring and directional monitoring

 

5. There should be scenarios in the performance test

Performance scenarios should also be classified. In my limited work experience, performance scenarios have never exceeded these classifications

1,Benchmark performance scenario: what needs to be done here is the capacity of a single transaction to prepare for the mixed capacity (don't tell me that a script running three or five times in the last few threads is called a benchmark test. In my opinion, it is only a pre execution before scenario execution to determine whether there are basic script and scenario design problems, which can't be called a classification).
2,Capacity performance scenario: this link must be the core performance execution part. According to the business complexity, many scenarios will be designed in this part, which will not be expanded in detail in the concept part. I will explain them in detail in the following articles.
3,Stability performance scenarios: stability testing must be a classification of performance scenarios. But now in the actual project, the stability test is basically not consistent with the production. In the stability test, it is obvious that the core element is time (the business model has been determined in the capacity scenario), and the setting of time should come from the operation and maintenance cycle, not from the psychological security of bosses, products and architectures.
4,Abnormal performance scenario: the premise of abnormal performance scenario is pressure. Under pressure and flow, simulate anomalies. The definition of this exception is very broad. We'll talk about it in detail in the next article.

　　

6. Analysis, tuning and test result report in performance test

Among the performance test engineers, there are not many people who can make bottleneck judgment, performance analysis and optimization, so many people in other positions position the positioning of performance test, that is, performance verification, does not include the tuning part. So many performance projects are defined in a week or two. This kind of project is basically personality verification, which can not be called a complete performance project. After the tuning part is added, the performance project will become complex. For most teams, it may take a long time to analyze bottlenecks, which will involve correlation analysis, trend analysis, evidence chain search and so on.

The performance items are divided into the following categories:

New system performance test: such projects generally require to test the maximum capacity of the system, otherwise there is no bottom on the line.

Performance test of new version of old system: such projects are generally compared with the old version. As long as the performance does not decline, the capacity can be calculated according to the historical data, and the tuning requirements are generally small.

New system performance test optimization class: this kind of system not only needs to test the maximum capacity, but also needs to be tuned to the best.

　　

The responsibilities of the performance team are as follows.

1)Performance verification: only performance verification is performed for a given index. Third party testing organizations basically do this.

2)Performance test: for a given system, a comprehensive performance test can get the maximum capacity of the system, but it does not involve tuning.

3)performance testing  + Analysis and tuning: conduct a comprehensive performance test for a given system, and tune the system to the optimal state at the same time

7. The performance test must have a result report

How are performance results defined? With the previous definition of monitoring and the process of scenario execution, the generated data will be sorted into the result report. This document work is also very important, which is an important aspect of whether the performance team is professional or not. It's not about sorting out one Word，Just beautify the format. Test reports need to be reported or filed.