Psychology and economics of software testing (1)

1. Psychology of software testing

In some cases, the attitude of the tester may be more important than the actual test process itself.

(1) Misunderstanding of "testing"

Software testing is the process of proving that there are no errors in software.

The purpose of software testing is to prove that the software can correctly complete its predetermined functions.

Software testing is the process of building confidence that the software has done what it should do.

(2) Correct understanding

Whenever you test a program, you should think of adding some value to the program. For example, improve the reliability or quality of the program.

Testing is the process of executing programs to find errors.

Software testing is a destructive process, even a "sadistic" process.

There are two kinds of software errors: (1) the program does not achieve the expected function. (2) The program did what it shouldn't do.

2. Economics of software testing

Generally speaking, it is impractical and often impossible to find all errors in the program.

2.1 black box test

Black box testing is an important testing strategy, also known as data-driven testing or input / output driven testing. When using this test method, treat the program as a black box. The test goal has nothing to do with the internal mechanism and structure of the program, but focuses on finding the environmental conditions under which the program does not operate correctly according to its specifications. In this method, the test data comes entirely from the software specification (there is no need to understand the internal structure of the program).

If you want to use this method to find all errors in the program, the criterion is "exhaustive input test", taking all possible input conditions as test cases.

Exhaustive input testing cannot be achieved. Two meanings: first, we cannot test a program to ensure that it is error free. 2, A basic problem to be considered in software testing is the economics of software testing. In other words, because exhaustive testing is impossible, the goal of test investment is to maximize the number of problems found through limited test cases, so as to achieve the best test effect.

2.2 white box test

White box testing, also known as logic driven testing, allows us to check the internal structure of the program. This test strategy checks the logical structure of the program and obtains test data. (program specifications are often ignored).

If all possible control flow paths in the program are executed using test cases, the program may be fully tested. This method is called exhaustive path testing.

There are two problems in exhaustive path testing: first, the number of different logical paths in the program may reach astronomical numbers, and the actual implementation is impossible and impractical.

2, Although we can test all paths in the program, there may still be errors in the program.

reason:

1. Even exhaustive path testing can never guarantee that the program meets its design specifications.

For example: to write an ascending sorting program, but it is wrong to write a descending sorting program. In this way, even the exhaustive path test is of little value. The program itself is a wrong program and does not meet the design specifications.

2. The program may have problems due to the lack of some paths.

The exhaustive path test cannot find which required paths are missing.

3. Exhaustive path testing may not expose data sensitive errors.

For example, the program should compare whether the two values converge, that is, check whether the difference between the two values is less than a given value, and only executing each path in the program does not necessarily find the error.

if(a - b < c)
	System.out.println("a - b < c");

The original intention of the program is to a - b Absolute sum of c Comparison, to find such errors, depends on a，b And just executing each path in the program does not necessarily find the error.

Principles of software testing

Principle 1: a necessary part of a test case is the definition of the expected output or result.

The expected output of the program is precisely defined in advance, and people are encouraged to carefully check all the output. Therefore, a test case must include two parts:
1.Description of the input data of the program.
2.An accurate description of the correct output of the program under the above input data.

Principle 2: programmers should avoid testing their own programs.

When a programmer "constructively" designs and writes a program, it is difficult for him to suddenly change his perspective and review the program with a "destructive" perspective.
There are two reasons:
1.Most programmers can't test their programs effectively because they can't change their way of thinking to try to expose the mistakes in their programs. In addition, programmers may subconsciously avoid finding mistakes for fear of being punished by colleagues, bosses, customers, or supervisors of programs or systems being developed.
2.Another reason is that there are errors in the program because the programmer mistakenly understands the difficult definition or specification. If this is the case, programmers may test their programs with the same misunderstanding.

The above arguments are not suitable for "debugging", which will be more effective if it is completed by the programmer.

Principle 3: organizations that write software should not test software they write.

Because a software project or programming organization is an organic organization, it has similar psychological problems with individual programmers.

Principle 4: the results of each test should be thoroughly checked.

Principle 5: test cases should be written not only according to valid and expected inputs, but also according to invalid and unexpected inputs.

Many problems suddenly exposed in software products are found when programs run in some new or unexpected way. Therefore, test cases for unexpected and invalid input situations seem to find problems better than those for valid input situations.

Principle 6: checking whether the program "didn't do what it should do" is only half of the test, and the other half of the test is checking whether the program "did what it shouldn't do".

This principle is the inevitable result of the previous principle. We must check whether the procedure has any undesirable negative effects.

Principle 7: test cases should be avoided to be discarded after use, unless the software itself is a one-time software.

Since test cases are discarded after use, once the software needs to be retested (for example, after correcting a mistake or making some improvement), these test cases must be redesigned. This is often the case, because redesigning test cases requires a lot of work, and people always avoid doing so. Therefore, the retest of the program is rarely as strict as the last time. This means that if a change to the program causes a previously executable part of the program to fail, the failure is often not found. Keep test cases and re execute them when other parts of the program change. This is what we call "regression testing".

Principle 8: when planning test work, you should not acquiesce in the assumption that errors will not be found.

Testing is the process of executing a program to find errors, not to prove that the program runs correctly.

Principle 9: the possibility of more errors in a part of the program is directly proportional to the number of errors found in that part.

Errors always tend to exist together, and in a specific program, some parts are more prone to errors than others, although no one can give a good explanation for this phenomenon.

Principle 10: software testing is a highly creative and intellectually challenging job.

Several important test principles (summary)

(1)Software testing is a process performed to find errors.
(2)Try to avoid coders testing their own programs.
(3)Good test cases can be highly sensitive to undetected errors.
(4)Successful test cases can find unknown errors.
(5)Successful test cases need to carefully define the expected values of input and output.
(6)Successful testing requires careful study and analysis of test results.