A test engineer walks into a bar and asks for a beer;
A test engineer walks into a bar and asks for a cup of coffee;
A test engineer walked into a bar and ordered 0.7 glasses of beer;
A test engineer walks into a bar and asks for -1 beer;
A test engineer walked into a bar and ordered 2 ^ 32 glasses of beer;
A test engineer walked into a bar and asked for a glass of foot washing water;
A test engineer walks into a bar and asks for a glass of lizard;
A test engineer walked into a bar and asked for a copy asdfQwer@24dg !&* (@；
A test engineer walks into a bar and asks for nothing;
A test engineer walks into a bar, goes out, comes in through the window, goes out through the back door and goes in through the sewer;
A test engineer walked into a bar, went out and in and out, and finally beat the boss outside;
A test engineer walked into a bar and asked for a cup of hot roller;
A test engineer walked into a bar and ordered a NaN cup Null;
A test engineer rushed into a bar and ordered 500T beer, coffee, foot washing water, wild cat wolf tooth stick and milk tea;
A test engineer dismantled the bar;
A test engineer disguised as a boss walked into a bar and ordered 500 BEERS without paying;
Ten thousand test engineers roared past outside the bar;
A test engineer walked into a bar and asked for a glass of beer '; DROP TABLE bar;
The test engineers left the bar with satisfaction.
Then a customer ordered a fried rice and the bar blew up.
The above is a popular joke about testing on the Internet. Its main core idea is that you can never fully test all the problems.
In software engineering, testing is an extremely important part, and its proportion can usually be the same as that of coding, or even greatly exceed that of coding. So in Golang, how to write the test well and correctly? This paper will make some brief introduction to this problem. The current article will be divided into two parts:
- Some basic writing methods and tools of Golang test
- How to write "correct" tests? Although this part of the code is written in golang, its core idea is not limited to language
Due to space issues, this article will not cover performance testing, and will be discussed in another article later.
The test is not lost: WeChat official account (programmer Xiao Hao) (mainly sharing the learning resources of software testing, helping to change, advance, and become a senior test engineer. Software test exchange group: 175317069)
Why write tests
Let's take an inappropriate example. Testing is also code. We assume that the probability of bugs when writing code is p (0 < p < 1). If we write tests at the same time, the probability of bugs on both sides is (we think the two events are independent of each other)
P (code Bug) * P (test Bug) = P ^ 2 < p
For example, if p is 1%, the probability of writing a bug at the same time is only 0.01%.
Testing is also code, and it is possible to write bug s, so how to ensure the correctness of testing? Write tests for tests? Continue writing tests for tests?
We define t(0) as the original code, any i, i > 0, t(i+1) as the necessary condition for the test of t(i), and t(i+1) is correct. Then for all i, i > 0, t(i) is correct, it is necessary condition for t(0)...
Type of test
There are many kinds of tests. We only select a few tests that are important to ordinary developers for a brief description.
White box test, black box test
Firstly, the test methods can be divided into white box test and black box test (of course, there is also the so-called gray box test, which is not discussed here)
- White box testing: white box testing is also known as transparent box testing and structure testing. It is one of the main methods of software testing, also known as structure testing, logic driven testing or testing based on the program itself. Test the internal structure or operation of the application, not the function of the application. In white box testing, test cases are designed from the perspective of programming language. The tester inputs data, verifies the flow path of the data flow in the program, and determines the appropriate output, similar to the node in the test circuit.
- Black box testing: black box testing is one of the main methods of software testing. It can also be called function testing, data-driven testing or specification based testing. The tester does not understand the internal situation of the program and does not need to have special knowledge of the code, internal structure and programming language of the application program. Only know the input and output of the program and the function of the system, which is to test the software interface, function and external structure from the perspective of the user, without considering the internal logical structure of the program.
The unit tests we write are generally white box tests because we have a full understanding of the internal logic of the test object.
Unit test, integration test
From the dimension of testing, it can be divided into unit testing and integration testing:
- In computer programming, unit testing, also known as module testing, is a test to verify the correctness of program modules. The program unit is the smallest testable part of the application. In process programming, a unit is a single program, function, process, etc; For object-oriented programming, the smallest unit is the method, including the method in the base class, abstract class, or derived class.
- Integration testing, also known as assembly testing, is the testing of the correctness of the system interface by assembling the program modules in a one-time or value-added way. Integration testing is generally carried out after unit testing and before system testing. Practice shows that sometimes modules can work alone, but it can not guarantee that they can work at the same time when assembled.
Unit test can be black box test, integration test can also be white box test
regression testing
- Regression testing is a kind of software testing, which aims to test whether the original function of the software remains intact after modification.
Regression testing is mainly to maintain the invariance of software. Let's give an example to illustrate. For example, we found that there was a problem in the running process of the software, and an issue was opened on gitlab. After that, we can also locate the problem. We can write a test (the name of the test can be accompanied by the ID of the issue) to reproduce the problem (the version of the code fails to run the test result). Then we fix the problem and run the test again. The test result should be successful. Then every time we run the test, we can ensure that the same problem will not recur by running the test.
A basic test
Let's first look at a code of Golang:
// add.go
package add
func Add(a, b int) int {
return a + b
}
Copy codeA test case can be written as:
// add_test.go
package add
import (
"testing"
)
func TestAdd(t *testing.T) {
res := Add(1, 2)
if res != 3 {
t.Errorf("the result is %d instead of 3", res)
}
}
Copy codeOn the command line, we use go test
go test Copy code
At this time, go will execute all the operations under the directory_ test. If the test is successful, the suffix in the test will be as follows:
% go test PASS ok code.byted.org/ek/demo_test/t01_basic/correct 0.015s Copy code
Suppose we modify the Add function to the wrong implementation at this time
// add.go
package add
func Add(a, b int) int {
return a - b
}
Copy codeExecute the test command again
% go test
--- FAIL: TestAddWrong (0.00s)
add_test.go:11: the result is -1 instead of 3
FAIL
exit status 1
FAIL code.byted.org/ek/demo_test/t01_basic/wrong 0.006s
Copy codeYou will find that the test fails.
Execute only one test file
So if we want to test only this file, enter
go test add_test.go Copy code
Command line output will be found
% go test add_test.go # command-line-arguments [command-line-arguments.test] ./add_test.go:9:9: undefined: Add FAIL command-line-arguments [build failed] FAIL Copy code
This is because we do not have the code attached with the test object. After modifying the test, we can get the correct output:
% go test add_test.go add.go ok command-line-arguments 0.007s Copy code
Several writing methods of test
Subtest
Generally speaking, when we test a function or method, we may need to test many different case s or marginal conditions. For example, we write two tests for the above Add function, which can be written as:
// add_test.go
package add
import (
"testing"
)
func TestAdd(t *testing.T) {
res := Add(1, 0)
if res != 1 {
t.Errorf("the result is %d instead of 1", res)
}
}
func TestAdd2(t *testing.T) {
res := Add(0, 1)
if res != 1 {
t.Errorf("the result is %d instead of 1", res)
}
}
Copy codeTest results: (use - v for more output)
% go test -v === RUN TestAdd --- PASS: TestAdd (0.00s) === RUN TestAdd2 --- PASS: TestAdd2 (0.00s) PASS ok code.byted.org/ek/demo_test/t02_subtest/non_subtest 0.007s Copy code
Another way is to write it in the form of sub test
// add_test.go
package add
import (
"testing"
)
func TestAdd(t *testing.T) {
t.Run("test1", func(t *testing.T) {
res := Add(1, 0)
if res != 1 {
t.Errorf("the result is %d instead of 1", res)
}
})
t.Run("", func(t *testing.T) {
res := Add(0, 1)
if res != 1 {
t.Errorf("the result is %d instead of 1", res)
}
})
}
Copy codeExecution result:
% go test -v
=== RUN TestAdd
=== RUN TestAdd/test1
=== RUN TestAdd/#00
--- PASS: TestAdd (0.00s)
--- PASS: TestAdd/test1 (0.00s)
--- PASS: TestAdd/#00 (0.00s)
PASS
ok code.byted.org/ek/demo_test/t02_subtest/subtest 0.007s
Copy codeYou can see that the tests will be classified according to the nested structure in the output. There is no limit on the number of nested layers of sub tests. If the test name is not written, the serial number will be automatically given as its test name in order (for example, #00 above)
Subtest friendly to IDE (Goland)
One way to write a test is:
tcList := map[string][]int{
"t1": {1, 2, 3},
"t2": {4, 5, 9},
}
for name, tc := range tcList {
t.Run(name, func(t *testing.T) {
require.Equal(t, tc[2], Add(tc[0], tc[1]))
})
}
Copy codeThere seems to be no problem, but one drawback is that this test is not IDE friendly:
We cannot re execute a single test when there is an error, so it is recommended to write each t.Run independently as far as possible, for example:
f := func(a, b, exp int) func(t *testing.T) {
return func(t *testing.T) {
require.Equal(t, exp, Add(a, b))
}
}
t.Run("t1", f(1, 2, 3))
t.Run("t2", f(4, 5, 9))
Copy code
Test subcontracting
We add. Above Go and add_test.go files are in the same directory, and the package name at the top is add. In the process of writing the test, you can also enable a package name different from that of the non test file. For example, we now change the package name of the test file to add_test:
// add_test.go
package add_test
import (
"testing"
)
func TestAdd(t *testing.T) {
res := Add(1, 2)
if res != 3 {
t.Errorf("the result is %d instead of 3", res)
}
}
Copy codeWhen you execute go test at this time, you will find
% go test # code.byted.org/ek/demo_test/t03_diffpkg_test [code.byted.org/ek/demo_test/t03_diffpkg.test] ./add_test.go:9:9: undefined: Add FAIL code.byted.org/ek/demo_test/t03_diffpkg [build failed] Copy code
Because the package name has changed, we can no longer access the Add function. At this time, we can Add import:
// add_test.go
package add_test
import (
"testing"
. "code.byted.org/ek/demo_test/t03_diffpkg"
)
func TestAdd(t *testing.T) {
res := Add(1, 2)
if res != 3 {
t.Errorf("the result is %d instead of 3", res)
}
}
Copy codeWe can use the above method to import the functions in the package. However, with this method, you will not be able to access the functions (starting with lowercase) that are not exported in the package.
Test tool library
github.com/stretchr/testify
We can use powerful testify to facilitate us to write tests. For example, the above tests can be written in this library:
// add_test.go
package correct
import (
"testing"
"github.com/stretchr/testify/require"
)
func TestAdd(t *testing.T) {
res := Add(1, 2)
require.Equal(t, 3, res)
/*
must := require.New(t)
res := Add(1, 2)
must.Equal(3, res)
*/
}
Copy codeIf the execution fails, you will see the following output on the command line:
% go test
ok code.byted.org/ek/demo_test/t04_libraries/testify/correct 0.008s
--- FAIL: TestAdd (0.00s)
add_test.go:12:
Error Trace: add_test.go:12
Error: Not equal:
expected: 3
actual : -1
Test: TestAdd
FAIL
FAIL code.byted.org/ek/demo_test/t04_libraries/testify/wrong 0.009s
FAIL
Copy codeThe library provides formatted error details (stack, error value, expected value, etc.) to facilitate our debugging.
github.com/DATA-DOG/go-sqlmock
Where sql needs to be tested, you can use go sqlmock to test
- Advantage: no need to rely on Database
- Disadvantages: it is separated from the specific implementation of the database, so it is necessary to write more complex test code
github.com/golang/mock
Powerful mock library for interface. For example, we want to test the function ioutil ReadAll
func ReadAll(r io.Reader) ([]byte, error) Copy code
We mock an IO Reader
// Package: output package name // destination: output file // IO: package of mock object // Reader: interface name of mock object mockgen -package gomock -destination mock_test.go io Reader Copy code
You can see mock in the directory_ test. Go file contains an IO The mock implementation of reader can be used to test ioutil Reader, for example
ctrl := gomock.NewController(t)
defer ctrl.Finish()
m := NewMockReader(ctrl)
m.EXPECT().Read(gomock.Any()).Return(0, errors.New("error"))
_, err := ioutil.ReadAll(m)
require.Error(t, err)
Copy codenet/http/httptest
Usually, when we test the server code, we will start the service first and then start the test. The official httptest package provides us with a convenient way to start a service instance to test.
other
Other testing tools can be found in awesomego#testing
How to write a test
The above describes the basic tools and writing methods of testing. We have completed the "must first sharpen its tools". Next, we will introduce how to "do good things".
Concurrent testing
At ordinary times, when writing services, we must basically consider concurrency. When we use ide testing, the IDE will not actively test the concurrency state by default. So how to ensure that the code we write is concurrency safe? Let's take an example. For example, we have a counter that counts.
type Counter int32
func (c *Counter) Incr() {
*c++
}
Copy codeObviously, this counter is not safe in the case of concurrency, so how do we write a test to test the concurrency of this counter?
import (
"sync"
"testing"
"github.com/stretchr/testify/require"
)
func TestA_Incr(t *testing.T) {
var a Counter
eg := sync.WaitGroup{}
count := 10
eg.Add(count)
for i := 0; i < count; i++ {
go func() {
defer eg.Done()
a.Incr()
}()
}
eg.Wait()
require.Equal(t, count, int(a))
}
Copy codeBy executing the above tests several times, we find that sometimes the test result returns OK and sometimes the test result returns FAIL. That is, even if a test is written, it may be marked as passed in a test. So is there any way to find the problem directly? The answer is to add - race flag during the test
-race flag is not suitable for benchmark test
go test -race Copy code
At this time, the terminal will output:
WARNING: DATA RACE
Read at 0x00c00001ca50 by goroutine 9:
code.byted.org/ek/demo_test/t05_race/race.(*A).Incr()
/Users/bytedance/go/src/code.byted.org/ek/demo_test/t05_race/race/race.go:6 +0x6f
code.byted.org/ek/demo_test/t05_race/race.TestA_Incr.func1()
/Users/bytedance/go/src/code.byted.org/ek/demo_test/t05_race/race/race_test.go:18 +0x66
Previous write at 0x00c00001ca50 by goroutine 8:
code.byted.org/ek/demo_test/t05_race/race.(*A).Incr()
/Users/bytedance/go/src/code.byted.org/ek/demo_test/t05_race/race/race.go:6 +0x85
code.byted.org/ek/demo_test/t05_race/race.TestA_Incr.func1()
/Users/bytedance/go/src/code.byted.org/ek/demo_test/t05_race/race/race_test.go:18 +0x66
Goroutine 9 (running) created at:
code.byted.org/ek/demo_test/t05_race/race.TestA_Incr()
/Users/bytedance/go/src/code.byted.org/ek/demo_test/t05_race/race/race_test.go:16 +0xe4
testing.tRunner()
/usr/local/Cellar/go/1.15/libexec/src/testing/testing.go:1108 +0x202
Goroutine 8 (finished) created at:
code.byted.org/ek/demo_test/t05_race/race.TestA_Incr()
/Users/bytedance/go/src/code.byted.org/ek/demo_test/t05_race/race/race_test.go:16 +0xe4
testing.tRunner()
/usr/local/Cellar/go/1.15/libexec/src/testing/testing.go:1108 +0x202
Copy codego actively prompts that we have found a race state in our code. At this time, we have to repair the code
type Counter int32
func (c *Counter) Incr() {
atomic.AddInt32((*int32)(c), 1)
}
Copy codeAfter the repair is completed, we test with - race again, and our test passes successfully!
Golang native concurrency testing
The test class of golang is testing T has a method Parallel(). All the methods invoked in the test are labeled as concurrency, but note that if you need to use the result of the concurrent test, you must wrap it in the outer layer with an additional test function.
func TestA_Incr(t *testing.T) {
var a Counter
t.Run("outer", func(t *testing.T) {
for i := 0; i < 100; i++ {
t.Run("inner", func(t *testing.T) {
t.Parallel()
a.Incr()
})
}
})
t.Log(a)
}
Copy codeIf there is no t.Run in the third line, the print result of line 11 will be incorrect
Testing by Golang T there are many other practical methods. You can check them yourself. We won't discuss them in detail here
Correct test return value
As a gopher, I usually have to write a lot of if err= Nil, when testing the error returned by a function, we have the following example
type I interface {
Foo() error
}
func Bar(i1, i2 I) error {
i1.Foo()
return i2.Foo()
}
Copy codeThe Bar function wants to process i1 and i2 inputs in turn and return when the first error is encountered, so we wrote a test that looks "correct"
import (
"errors"
"testing"
"github.com/stretchr/testify/require"
)
type impl string
func (i impl) Foo() error {
return errors.New(string(i))
}
func TestBar(t *testing.T) {
i1 := impl("i1")
i2 := impl("i2")
err := Bar(i1, i2)
require.Error(t, err) // assert err != nil
}
Copy codeThe test result "looks" perfect, and the function returns an error correctly. But in fact, we know that the return value of this function is wrong, so we should slightly modify the test and treat error as a return value to verify the content, rather than simply judge nil
func TestBarFixed(t *testing.T) {
i1 := impl("i1")
i2 := impl("i2")
err := Bar(i1, i2)
// Both can be written
require.Equal(t, errors.New("i1"), err)
require.Equal(t, "i1", err.Error())
}
Copy codeAt this time, we can find that there are errors in the code and need to be repaired. Similarly, it can be applied to other return values. We should not only make some simple judgments, but also make "accurate value" judgments as much as possible.
Test input parameters
We discussed the test return value above, and the input value also needs to be tested. This point is mainly combined with gomock. For example, our code is as follows:
type I interface {
Foo(ctx context.Context, i int) (int, error)
}
type bar struct {
i I
}
func (b bar) Bar(ctx context.Context, i int) (int, error) {
i, err := b.i.Foo(context.Background(), i)
return i + 1, err
}
Copy codeWe want to test whether the bar class is correct. We call the Foo method in the method. We use gomock to mock the mock implementation of the I interface we want:
mockgen -package gomock -destination mock_test.go io Reader Copy code
Next, we wrote a test:
import (
"context"
"testing"
. "code.byted.org/ek/testutil/testcase"
"github.com/stretchr/testify/require"
)
func TestBar(t *testing.T) {
t.Run("test", TF(func(must *require.Assertions, tc *TC) {
impl := NewMockI(tc.GomockCtrl)
i := 10
j := 11
ctx := context.Background()
impl.EXPECT().Foo(ctx, i).
Return(j, nil)
b := bar{i: impl}
r, err := b.Bar(ctx, i)
must.NoError(err)
must.Equal(j+1, r)
}))
}
Copy codeThe test run was successful, but in fact, we looked at the code and found that the context in the code was not passed correctly, so how should we correctly test this situation? One way is to write a similar test and modify the context in the test Background() is another context:
t.Run("correct", TF(func(must *require.Assertions, tc *TC) {
impl := NewMockI(tc.GomockCtrl)
i := 10
j := 11
ctx := context.WithValue(context.TODO(), "k", "v")
impl.EXPECT().Foo(ctx, i).
Return(j, nil)
b := bar{i: impl}
r, err := b.Bar(ctx, i)
must.NoError(err)
must.Equal(j+1, r)
}))
Copy codeAnother way is to add random test elements.
Add random elements to the test
The same is the above test. We will make some modifications
import (
"context"
"testing"
randTest "code.byted.org/ek/testutil/rand"
. "code.byted.org/ek/testutil/testcase"
"github.com/stretchr/testify/require"
)
t.Run("correct", TF(func(must *require.Assertions, tc *TC) {
impl := NewMockI(tc.GomockCtrl)
i := 10
j := 11
ctx := context.WithValue(context.TODO(), randTest.String(), randTest.String())
impl.EXPECT().Foo(ctx, i).
Return(j, nil)
b := bar{i: impl}
r, err := b.Bar(ctx, i)
must.NoError(err)
must.Equal(j+1, r)
}))
Copy codeIn this way, it can largely avoid that some edge case s caused by fixed test variables are easy to be incorrectly measured as correct. If you go back to the previous example of Add function, you can write
import (
"math/rand"
"testing"
"github.com/stretchr/testify/require"
)
func TestAdd(t *testing.T) {
a := rand.Int()
b := rand.Int()
res := Add(a, b)
require.Equal(t, a+b, res)
}
Copy codeModified input parameters
If we modify the previous Bar example
func (b bar) Bar(ctx context.Context, i int) (int, error) {
ctx = context.WithValue(ctx, "v", i)
i, err := b.i.Foo(ctx, i)
return i + 1, err
}
Copy codeThe functions are basically the same, except that the ctx passed to the Foo method becomes a sub context. At this time, the previous tests cannot be executed correctly. So how to judge that the passed context is a sub context of the top-level context?
Judgment by handwriting
One method is to pass a context to Bar in the test Withvalue, and then judge whether the received context has a specific kv in the implementation of Foo
t.Run("correct", TF(func(must *require.Assertions, tc *TC) {
impl := NewMockI(tc.GomockCtrl)
i := 10
j := 11
k := randTest.String()
v := randTest.String()
ctx := context.WithValue(context.TODO(), k, v)
impl.EXPECT().Foo(gomock.Any(), i).
Do(func(ctx context.Context, i int) {
s, _ := ctx.Value(k).(string)
must.Equal(v, s)
}).
Return(j, nil)
b := bar{i: impl}
r, err := b.Bar(ctx, i)
must.NoError(err)
must.Equal(j+1, r)
}))
Copy codegomock.Matcher
Another way is to implement gomock Matcher interface
import (
randTest "code.byted.org/ek/testutil/rand"
)
t.Run("simple", TF(func(must *require.Assertions, tc *TC) {
impl := NewMockI(tc.GomockCtrl)
i := 10
j := 11
ctx := randTest.Context()
impl.EXPECT().Foo(ctx, i).
Return(j, nil)
b := bar{i: impl}
r, err := b.Bar(ctx, i)
must.NoError(err)
must.Equal(j+1, r)
}))
Copy coderandTest. The main code of context is as follows:
func (ctx randomContext) Matches(x interface{}) bool {
switch v := x.(type) {
case context.Context:
return v.Value(ctx) == ctx.value
default:
return false
}
}
Copy codegomock will automatically use this interface to judge the matching of input parameters.
Test functions with many child calls
Let's look at the following functions:
func foo(i int) (int, error) {
if i < 0 {
return 0, errors.New("negative")
}
return i + 1, nil
}
func Bar(i, j int) (int, error) {
i, err := foo(i)
if err != nil {
return 0, err
}
j, err = foo(j)
if err != nil {
return 0, err
}
return i + j, nil
}
Copy codeThe logic here looks relatively simple, but if we imagine that Bar's logic and foo logic are very complex and contain more branches of logic, then we will encounter two problems in testing.
- When testing the Bar function, you may need to consider the return values of various foo functions, and the input parameters need to be specially constructed according to the requirements of foo
- It may be necessary to repeat a large number of tests to the foo scenario, which is the same as the foo itself
So how to solve this problem? I'll give you an idea here, although it may not be the optimal solution. Hope for a better solution can be put forward in the comment area. My idea is to change the foo function from a fixed function to a variable function pointer, which can be dynamically replaced during testing
var foo = func(i int) (int, error) {
if i < 0 {
return 0, errors.New("negative")
}
return i + 1, nil
}
func Bar(i, j int) (int, error) {
i, err := foo(i)
if err != nil {
return 0, err
}
j, err = foo(j)
if err != nil {
return 0, err
}
return i + j, nil
}
Copy codeSo when testing Bar, we can replace foo:
func TestBar(t *testing.T) {
f := func(newFoo func(i int) (int, error), cb func()) {
old := foo
defer func() {
foo = old
}()
foo = newFoo
cb()
}
t.Run("first error", TF(func(must *require.Assertions, tc *TC) {
expErr := randTest.Error()
f(func(i int) (int, error) {
return 0, expErr
}, func() {
_, err := Bar(1, 2)
must.Equal(expErr, err)
})
}))
t.Run("second error", TF(func(must *require.Assertions, tc *TC) {
expErr := randTest.Error()
first := true
f(func(i int) (int, error) {
if first {
first = false
return 0, nil
}
return 0, expErr
}, func() {
_, err := Bar(1, 2)
must.Equal(expErr, err)
})
}))
t.Run("success", TF(func(must *require.Assertions, tc *TC) {
f(func(i int) (int, error) {
return i, nil
}, func() {
r, err := Bar(1, 2)
must.NoError(err)
must.Equal(3, r)
})
}))
}
Copy codeThe above writing method can test foo and Bar separately
- After using this method, you may need to write more code related to mock (this part can be combined with gomock)
- When testing concurrency in this method, you need to consider whether your mock function handles concurrency correctly
- The necessary condition for the overall correctness of this test is that the test of foo function is correct, and the mock of foo function is consistent with the behavior of the correct foo function, so it is necessary to write an additional overall test without mock foo function when necessary
Test coverage
When writing tests, we often mention a word, coverage. So what is test coverage?
Test coverage is a software measurement in software testing or software engineering, which indicates the proportion of software programs tested. Coverage is a way to judge the rigor of testing. There are many different types of test coverage: code coverage, feature coverage, scenario coverage, screen item coverage, module coverage. Each coverage assumes that the system under test has a morphological benchmark. Therefore, when the system changes, the test coverage will also change.
In general, we can think that the higher the test coverage, the more comprehensive our test coverage, and the higher the effectiveness of the test.
Test coverage of Golang
In golang, we test the coverage while testing the code by attaching the - cover flag
% go test -cover PASS coverage: 100.0% of statements ok code.byted.org/ek/demo_test/t10_coverage 0.008s Copy code
We can see that the current test coverage is 100%.
100% test coverage is not equal to the correct test
The higher the test coverage is, the incorrect the test is. We will give examples in several cases.
Input and output are not tested correctly
This has been mentioned above. You can refer to the example of "correct test return value" above. In the example, the test coverage reaches 100%, but the code problem is not correctly tested.
Not covering all branch logic
func AddIfBothPositive(i, j int) int {
if i > 0 && j > 0 {
i += j
}
return i
}
Copy codeThe coverage of the following test cases is 100%, but not all branches are tested
func TestAdd(t *testing.T) {
res := AddIfBothPositive(1, 2)
require.Equal(t, 3, res)
}
Copy codeExceptions / boundary conditions are not handled
func Divide(i, j int) int {
return i / j
}
Copy codeThe Divide function does not handle the case where the divisor is 0, while the coverage of unit tests is 100%
func TestAdd(t *testing.T) {
res := Divide(6, 2)
require.Equal(t, 3, res)
}
Copy codeThe above example shows that 100% test coverage does not really "100% cover" all code operations.
Statistical method of coverage
The statistical method of test coverage is generally: the number of lines of code executed in the test / the total number of lines of code tested. However, in the actual operation of the code, the probability of each line running and the severity of errors are also different, so we should not superstitious about coverage while pursuing high coverage.
The test is not afraid of repeated writing
The repeated writing here can be regarded as the antonym of "code reuse" to a certain extent. We mainly from the following aspects.
Write similar test cases repeatedly
As long as the test cases are not completely consistent, we can think that even the more similar test cases are meaningful. There is no need to delete them specifically for the sake of code simplification. For example, we test the Add function above
func TestAdd(t *testing.T) {
t.Run("fixed", func(t *testing.T) {
res := Add(1, 2)
require.Equal(t, 3, res)
})
t.Run("random", func(t *testing.T) {
a := rand.Int()
b := rand.Int()
res := Add(a, b)
require.Equal(t, a+b, res)
})
}
Copy codeAlthough the second test seems to cover the first test, there is no need to specifically delete the first test. The more tests, the more reliable our code will be.
Repeatedly write the definition and logic in the (source) code
For example, we have a code
package add
const Value = 3
func AddInternalValue(a int) int {
return a + Value
}
Copy codeTest as
func TestAdd(t *testing.T) {
res := AddInternalValue(1)
require.Equal(t, 1+Value, res)
}
Copy codeIt looks perfect, but if the Value of the internal variable Value is accidentally changed one day, the test can't reflect the change and find the error in time. If we write
func TestAdd(t *testing.T) {
const value = 3
res := AddInternalValue(1)
require.Equal(t, 1+value, res)
}
Copy codeYou don't have to worry about not finding changes in constant values.