Java excellent testing framework - TestNG

Posted by abdbuet on Mon, 21 Feb 2022 13:14:51 +0100

Why do we need TestNG?

Java has several test frameworks. JUnit is a common one. The default test framework of Spring series is JUnit. TestNG is also a test framework of Java. It has similar functions to JUnit, but supports more annotations and functions.

we often use JUnit for unit testing, but we prefer TestNG for automated testing.

TestNG build

in Maven project, you can add TestNG dependency package, and you can choose the version by yourself:

		<dependency>
            <groupId>org.testng</groupId>
            <artifactId>testng</artifactId>
            <version>7.1.0</version>
            <scope>test</scope>
        </dependency>

TestNG annotation and function

the annotation of TestNG is similar to JUnit, but it will be richer and more detailed:

annotation	describe
@Test	Mark all public void methods or a method in a class as test cases.
@BeforeSuite	For suite tests, run only once before all tests in this suite are executed.
@AfterSuite	For suite tests, run only once after all tests in this suite have been executed.
@BeforeClass	Run only once before calling the current class.
@AfterClass	Run after calling the current class, only once.
@BeforeTest	For suite tests, run before the test methods belonging to all classes within the tag are executed.
@AfterTest	For suite tests, run after the test methods of all classes belonging to the tag have been executed.
@BeforeGroups	Run before calling the first test method belonging to the group.
@AfterGroups	Run after calling the last test method that belongs to the group for execution.
@BeforeMethod	Annotated methods will be run before each test method.
@AfterMethod	It runs before each test method is executed.
@DataProvider	Marks a method, a Test method that provides data. The annotated method must return an Object [], where each Object [] can be assigned in the parameter list of the Test method. For this @ Test method, if you want to receive data from this DataProvider, you need to use a DataProvider name equal to the name of this annotation.
@Factory	As a factory, the Object that returns the test class of TestNG will be used for the marked method. The method must return Object [].
@Listeners	Define a listener for a test class.
@Parameters	Used to pass parameters to the @ Test method.

TestNG configuration file

compared with JUnit, the most obvious difference of TestNG is the addition of a configuration file.

with the existence of this configuration file, it is more convenient to manage the execution of test cases.

1. Add a configuration file

in the project root directory, add TestNG XML format configuration file. The file name is arbitrary. There can be multiple configuration files.

maven project can directly run the configuration file after specifying the configuration file path in the pom file:

			<plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>3.0.0-M4</version>
                <configuration>
                    <suiteXmlFiles>
                        <suiteXmlFile>./testng.xml</suiteXmlFile>
                    </suiteXmlFiles>
                </configuration>
            </plugin>

the content of TestNG configuration is not complex. The configuration here is not the dependence of TestNG startup, but the execution mode of test cases.

2. Configure Test Suite (suit)

one configuration file has and can only have one test suite. The common attributes of the suit tag are as follows. name is mandatory and will be displayed in the final test report.

attribute	describe
name	The name of the suite, which is a mandatory property
verbose	The level or level of detail of the operation. The level is 0-10, of which 10 is the most detailed
parallel	Whether TestNG runs different threads to run the suite. The default is none. Other levels are methods, tests, classes and instances
thread-count	If parallel mode is enabled (ignoring other methods), the number of threads used
annotations	The type of comment used in the test
time-out	The default timeout used on all test methods in this test
preserve-order	Used to configure whether test cases are executed in sequence

3. Configure test cases

divided by groups (annotated with @ Test (groups = {"group1"}), the group name marked with the include tag will be executed, and the group marked with the exclude tag will not be executed:

		<groups>
  			<run>
     			<include name = "includedGroupName" />
     			<exclude name = "excludedGroupName" />
  			</run>
		</groups>

it is divided into three levels: package, class and method:
package:

		<packages>
            <package name = "com.hhm.demo1.*" />
            <package name = "com.hhm.demo2.*" />
        </packages>

class:

		<classes>
            <class name="com.hhm.demo1.DemoTest1"/>
            <class name="com.hhm.demo1.DemoTest2"/>
		</classes>

method:

		<classes>
            <class name="com.hhm.demo1.DemoTest1">
                <methods>
                    <include name="testMethodName"></include>
                </methods>
            </class>
        </classes>

example of complete configuration:

<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >
<suite name="First suite" verbose="1" >
    <test name="MyTest" parallel="classes" thread-count="4">
        <packages>
            <package name="com.hhm.demo1.*"></package>
            <package name="com.hhm.demo2.*"></package>
        </packages>
    </test>
</suite>

after the configuration is completed, you can directly run the configuration file to execute the test case. It can also be executed through the test command of Maven lifecycle.

TestNG test

1. Abnormal test

the expected Exception Test is added by adding the expected Exception after the @ Test annotation. The example is as follows:

	@Test(expectedExceptions = ArithmeticException.class)
    public void divisionWithException() {
        int i = 1 / 0;
        System.out.println("After division the value of i is :"+ i);
    }

2. Ignore the test

sometimes the use case we write is not ready, or the test doesn't want to run this use case, so it's obviously unwise to delete it. Then you can ignore it by commenting @ Test(enabled = false), and this use case won't run. The following example:

public class DemoTest1{

    @Test(enabled=false)
    public void TestNgLearn1() {
        System.out.println("this is TestNG test case1");
    }
    
    @Test
    public void TestNgLearn2() {
        System.out.println("this is TestNG test case2");
    }
}

3. Timeout test

"timeout" means that if the unit test takes longer than the specified number of milliseconds, TestNG will abort it and mark it as failed. This item is often used for performance testing. The following is an example:

public class DemoTest2{

    @Test(timeOut = 5000) // time in mulliseconds
    public void testThisShouldPass() throws InterruptedException {
        Thread.sleep(4000);
    }

    @Test(timeOut = 1000)
    public void testThisShouldFail() {
        while (true){
            // do nothing
        }
    }
}

4. Parametric test

4.1 using XML configuration file to provide

the code is as follows:

public class DemoTest2 {

    @Parameters({"param1", "param2"})
    public void TestNgLearn1(String param1, int param2) {
        System.out.println("this is TestNG test case1, and param1 is:"+param1+"; param2 is:"+param2);
        Assert.assertFalse(false);
    }
}

XML configuration is as follows:

<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >
<suite name="First suite" verbose="1" >
    <test name="MyTest">
	    <parameter name="param1" value="1011111" />
	    <parameter name="param2" value="10" />
        <classes>
      		<class name="com.hhm.demo1.DemoTest2"/>
    	</classes>
    </test>
</suite>

4.2 passing parameters using @ DataProvider

it should be noted here that the types of parameters passed must be consistent, and the function with @ DataProvider annotation must return Object [] [], which should be noted here. When multiple sets of data are provided, the test case will be executed corresponding times.

public class DemoTest2 {

    @DataProvider(name = "provideNumbers")
    public Object[][] provideData() {
        return new Object[][] { { 10, 20 }, { 100, 110 }, { 200, 210 } };
    }
	
    @Test(dataProvider = "provideNumbers")
    public void TestNgLearn1(int param1, int param2) {
        System.out.println("this is TestNG test case1, and param1 is:"+param1+"; param2 is:"+param2);
        Assert.assertFalse(false);
    }
}

5. Dependency testing

sometimes we need to call test cases in order, so there are dependencies between test cases. TestNG supports dependencies between test cases, and uses dependOnMethods and dependsOnGroups to implement dependency testing. In the following cases, when TestNgLearn1() case fails, TestNgLearn2() case will not be executed:

public class DemoTest1 {

    public void TestNgLearn1() {
        System.out.println("this is TestNG test case1");
        Assert.assertFalse(true);
    }
    
    @Test(dependsOnMethods= {"TestNgLearn1"})
    public void TestNgLearn2() {
        System.out.println("this is TestNG test case2");
    }
}

TestNG test report

after using maven to execute the test case, the test report file will be generated in the project root directory \ target \ surefire reports directory, and click index HTML files can be viewed in the browser.

the report here is relatively simple and not beautiful enough. We can combine Allure plug-in to generate more beautiful test reports.

Topics: software testing Testing

Programmer Think