1, Introduction

TestNG is a JUnit and NUnit inspired testing framework designed to simplify a wide range of testing requirements, from unit testing to interface testing. However, some new functions have been introduced to make it more powerful and easier to use, such as:

• notes.
• Run tests in the thread pool and provide various available strategies (single thread, multi thread, etc.)
• The code tested is multithread safe
• Flexible test configuration
• Support data-driven testing (using @ DataProvider)
• Support parameterization
• Powerful execution model (no longer TestSuite)
• Supported by various tools and plug-ins (Eclipse, IDEA, Maven, etc.).
• Embed BeanShell scripts for greater flexibility
• Default JDK function for runtime and logging (no dependencies)
• Dependency method of application server testing
• TestNG aims to cover all categories of testing: unit, interface, end-to-end, integration, etc

2, Quick Demo

Writing a test usually requires three steps:

• Write test code and insert TestNG comments.
• In TestNG Add information about the test (such as class name, group to run, etc.) to the XML.
• Run TestNG.

A quick example

package example1;

import org.testng.annotations.*;

public class SimpleTest {
 
 @BeforeClass
 public void setUp() {
   // Call this code when instantiating
 }
 
 @Test(groups = { "fast" })
 public void aFastTest() {
   System.out.println("Fast test");
 }
 
 @Test(groups = { "slow" })
 public void aSlowTest() {
    System.out.println("Slow test");
 }
}
Copy code

After building the test class and before running any test methods, the method setUp() is called. In this example, we will run the group quickly, so aFastTest() will be called and aSlowTest() will be skipped.

matters needing attention:

• There is no need to extend classes or implement interfaces.
• Although the above example uses the JUnit convention, our methods can be called any name, which is a comment that tells TestNG what they are.
• Test methods can belong to one or more groups.
• After compiling the test class into the build directory, you can invoke the test using the command line, ant task (as shown below), or XML file

<project default="test">
 
 <path id="cp">
   <pathelement location="lib/testng-testng-5.13.1.jar"/>
   <pathelement location="build"/>
 </path>

 <taskdef name="testng" classpathref="cp"
          classname="org.testng.TestNGAntTask" />
          
 <target name="test">
   <testng classpathref="cp" groups="fast">
     <classfileset dir="build" includes="example1/*.class"/>
   </testng>
 </target>
 
</project>
Copy code

Call it with ant

c:> ant
Buildfile: build.xml
 
test:
[testng] Fast test
[testng] ===============================================
[testng] Suite for Command line test
[testng] Total tests run: 1, Failures: 0, Skips: 0
[testng] ===============================================
 
 
BUILD SUCCESSFUL
Total time: 4 seconds
 Copy code

You can then browse the test results:

start test-output\index.html (on Windows)
Copy code

The concepts used in this document are as follows:

• The test suite is represented by an XML file. It can contain one or more tests and is defined by the < suite > tag.
• Tests are represented by < test > and can contain one or more TestNG classes.
• The TestNG class is a Java class that contains at least one TestNG annotation. It is represented by the < class > tag and can contain one or more test methods.
• A Test method is a Java method annotated by @ Test in the source.

TestNG tests can be configured through the @ BeforeXXX and @ AfterXXX annotations, which allow some Java logic to be executed before and after a point that is one of the items listed above.

3, Basic notes

The following is an overview of the comments available in TestNG and their properties.

annotation	describe
@BeforeSuite	Before all the tests in the suite are run on the annotated method, they are run only once.
@AfterSuite	After all the tests in the suite are run after the annotation method, they are run only once.
@BeforeClass	The annotation method runs only once before calling the first test method of the current class.
@AfterClass	After calling the first test method of the current class, the annotation method runs only once
@BeforeTest	Annotated methods will run before all test methods belonging to classes within the < test > tag run.
@AfterTest	Annotated methods will run after all test methods belonging to classes within the < test > tag are run.
@BeforeGroups	The configuration method will run the group list before. This method is guaranteed to run shortly before calling the first test method belonging to any of these groups.
@AfterGroups	This configuration method will run the group list after. This method is guaranteed to run shortly after calling the last test method belonging to any of these groups.
@BeforeMethod	Annotation methods will run before each test method.
@AfterMethod	Annotation methods will run after each test method.
alwaysRun	For before methods (beforeSuite, beforeTest, beforeTestClass and beforeTestMethod, but not beforeGroups): if set to true, this configuration method will run regardless of which group it belongs to. For after methods (afterSuite, afterClass,...): If it is set to true, even if one or more methods previously called failed or skipped, the configuration method will be run.
dependsOnGroups	List of groups on which this method depends.
dependsOnMethods	List of methods this method depends on.
enabled	Whether to enable methods on this class / method.
groups	List of groups to which this class / method belongs.
inheritGroups	If true, this method will belong to the group specified in the @ Test annotation at the class level.
onlyForGroups	Only available for @ BeforeMethod and @ AfterMethod. If specified, this setup / teardown method is called only if the corresponding test method belongs to one of the listed groups.
When placed on the superclass of TestNG class, the above annotation will also be inherited. For example, this is useful for centralizing test settings for multiple test classes in a common superclass.

In this case, TestNG ensures that the "@ Before" method is executed in the order of inheritance (first the highest superclass, then inheritance), while the "@ After" method is executed in the opposite order (upward inheritance chain).

@dataProvider: marks the method as providing data for the Test method. Remember a method to provide data for the Test method. The annotation method must return an Object [], where each Object [] can be assigned to the parameter list of the Test method. The @ Test method to receive data from this dataProvider requires a dataProvider name equal to this annotation name.

attribute	describe
name	The name of this data provider. If not provided, the name of this data provider is automatically set to the name of the method.
parallel	If set to true, tests generated using this data provider will run in parallel. The default value is false.

@Factory: mark the method as a factory and return the Object to be used by TestNG as the Test class. The method must return Object [].

@Listeners: define listeners on the test class.

attribute	describe
value	Extension org testng. An array of classes for itestnglistener.

@Parameters: describes how to pass parameters to the @ Test method.

attribute	describe
value	A list of variables used to populate this method parameter.

@Test: mark a class or method as part of a test.

attribute	describe
alwaysRun	If set to true, the test method will always run even if it depends on the failed method.
dataProvider	The name of the data provider for this test method.
dataProviderClass	Find the class of the data provider. If not specified, the data provider is found on the class of the current test method or one of its base classes. If this property is specified, the data provider method must be static on the specified class.
dependsOnGroups	List of groups on which this method depends.
dependsOnMethods	List of methods this method depends on.
description	Description of this method.
enabled	Whether to enable methods on this class / method.
expectedExceptions	List of exceptions expected to be thrown by the test method. If no exceptions or different exceptions are thrown in this list, the test will be marked as failed.
groups	List of groups to which this class / method belongs.
invocationCount	The number of times this method should be called.
invocationTimeOut	The maximum number of milliseconds that this test should take for the cumulative time of all call counts. If invocationCount is not specified, this property is ignored.
priority	The priority of this test method. Lower priorities will be prioritized.
successPercentage	Expected success percentage of this method
singleThreaded	If set to true, all methods on this test class are guaranteed to run in the same thread, even if the test is currently running with parameter = "methods". This property can only be used at the class level. If it is used at the method level, it will be ignored. Note: this property was once called order (now deprecated).
timeOut	The maximum number of milliseconds that should be used for this test.
threadPoolSize	The thread pool size of this method. This method will be called from multiple threads specified by invocationCount.
Note: if invocationCount is not specified, this property is ignored
##4, Common assertion methods
In order to judge whether the test case is executed successfully, TestNG provides a specific assertion class, which contains various forms of assertion methods.
method	describe
:---------	:-----
assertTrue	Judge whether it is true
assertFalse	Judge whether it is false
assertSame	Determine whether the reference addresses are the same
assertNotSame	Judge whether the reference addresses are different
assertNull	Determine whether it is null
assertNotNull	Determine whether it is not null
assertEquals	To determine whether they are equal, objects of type Object need to implement hashCode and equals methods
assertNotEquals	Judge whether it is unequal
assertNoOrder	Judge whether the order of neglect is equal
##5, TestNG xml
You can call TestNG in several different ways:

• Use TestNG XML file
• ant
• maven, such as mvn clean test -U -Dxml=xmlFileName
• command line

This section describes TestNG XML format (you'll find documentation on ant and the command line below).

This is an example TestNG XML file

<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >
  
<suite name="Suite1" verbose="1" >
  <test name="Nopackage" >
    <classes>
       <class name="NoPackageTest" />
    </classes>
  </test>
 
  <test name="Regression1">
    <classes>
      <class name="test.sample.ParameterSample"/>
      <class name="test.sample.ParameterTest"/>
    </classes>
  </test>
</suite>
Copy code

Specify package name

<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >
 
<suite name="Suite1" verbose="1" >
  <test name="Regression1"   >
    <packages>
      <package name="test.sample" />
   </packages>
 </test>
</suite>
Copy code

In this example, TestNG will execute the package test All classes in sample, and only those with TestNG annotations are retained.

Specify the groups and methods to include and exclude

<test name="Regression1">
  <groups>
    <run>
      <exclude name="brokenTests"  />
      <include name="checkinTests"  />
    </run>
  </groups>
  
  <classes>
    <class name="test.IndividualMethodsTest">
      <methods>
        <include name="testMethod" />
      </methods>
    </class>
  </classes>
</test>
Copy code

In TestNG Define a new group in XML and specify other details in the attribute, such as whether to run tests in parallel, how many threads to use, whether to run tests, and so on By default, TestNG will run tests in the order in the XML file. If you want the classes and methods listed in this file to run in an unpredictable order, set the preserve order property to false

<test name="Regression1" preserve-order="false">
  <classes>
 
    <class name="test.Test1">
      <methods>
        <include name="m1" />
        <include name="m2" />
      </methods>
    </class>
 
    <class name="test.Test2" />
 
  </classes>
</test>
Copy code

6, Test methods, test classes and test groups

1. Test method

The Test method is annotated with @ Test. Unless in TestNG Set allow return values to true in XML, otherwise the method that exactly returns the value with @ Test annotation will be ignored:

<suite allow-return-values="true">
or
<test allow-return-values="true">
Copy code

2. Test group

TestNG allows you to perform complex test method groupings. You can not only declare that a method belongs to a group, you can also specify a group that contains other groups. You can then call TestNG and ask to include a specific group (or regular expression) while excluding another group. This provides maximum flexibility for partitioned testing, and if you want to run two different sets of tests continuously, you don't need to recompile anything.

Group in TestNG XML file, which can be found under the < test > or < suite > tag. The group specified in the tag applies to all < test > tags below. Note that groups are cumulative in these tags: if you specify group "a" in < suite > and "b" in < test >, you will include "a" and "b".

For example, at least two types of tests are common

• Check in test. You should run these tests before submitting new code. They should usually be fast and ensure that no basic functions are broken.
• Function test. These tests should cover all functions of the software and run at least once a day, although ideally they want to run continuously.

Typically, check-in tests are a subset of functional tests. TestNG allows you to use test group assignments in a very intuitive way. For example, you can build tests by the fact that the entire test class belongs to the "functest" group, and some methods belong to the "checkint" group:

public class Test1 {
  @Test(groups = { "functest", "checkintest" })
  public void testMethod1() {
  }
 
  @Test(groups = {"functest", "checkintest"} )
  public void testMethod2() {
  }
 
  @Test(groups = { "functest" })
  public void testMethod3() {
  }
}
Copy code

TestNG call

<test name="Test1">
  <groups>
    <run>
      <include name="functest"/>
    </run>
  </groups>
  <classes>
    <class name="example1.Test1"/>
  </classes>
</test>
Copy code

All the test methods in this class will be run, while calling checkint will only run testMethod1() and testMethod2().

This is another example, this time using regular expressions. Assuming that some test methods should not run on Linux, the test will be as follows

@Test
public class Test1 {
  @Test(groups = { "windows.checkintest" })
  public void testWindowsOnly() {
  }
 
  @Test(groups = {"linux.checkintest"} )
  public void testLinuxOnly() {
  }
 
  @Test(groups = { "windows.functest" )
  public void testWindowsToo() {
  }
}
Copy code

You can use the following TestNG XML start Windows only method:

<test name="Test1">
  <groups>
    <run>
      <include name="windows.*"/>
    </run>
  </groups>
 
  <classes>
    <class name="example1.Test1"/>
  </classes>
</test>
Copy code

Note: TestNG uses regular expressions instead of wildmats

Method groups can also exclude or contain individual methods

<test name="Test1">
  <classes>
    <class name="example1.Test1">
      <methods>
        <include name=".*enabledTestMethod.*"/>
        <exclude name=".*brokenTestMethod.*"/>
      </methods>
     </class>
  </classes>
</test>
Copy code

This can be used to disable a single method without recompiling anything, but it is not recommended to use this technology too much, because if you start refactoring your Java code (regular expressions used in regular expressions), it will crash your test framework. The label may no longer match your method).

3. Group

Groups can also include other groups. These groups are called MetaGroups. For example, you might want to define the group "all" that contains "checkint" and "functest". 'functest' itself will contain 'windows' and' linux 'groups, while' checkintest 'will contain only' windows'. Here is how to define it in the properties file:

<test name="Regression1">
  <groups>
    <define name="functest">
      <include name="windows"/>
      <include name="linux"/>
    </define>
  
    <define name="all">
      <include name="functest"/>
      <include name="checkintest"/>
    </define>
  
    <run>
      <include name="all"/>
    </run>
  </groups>
  
  <classes>
    <class name="test.sample.Test1"/>
  </classes>
</test>
Copy code

4. Exclusion group

TestNG allows you to include groups and exclude them.

For example, it is often common to temporarily interrupt testing due to recent changes and there is no time to repair the damage. However, you do want a clean run of functional tests, so you need to deactivate these tests, but remember to reactivate them. A simple way to solve this problem is to create a group called "broken" and make these test methods belong to it. For example, in the above example, I know that testMethod2() is broken now, so I want to disable it:

@Test(groups = {"checkintest", "broken"} )
public void testMethod2() {
}
Copy code

What I need to do now is to exclude this group from the run:

<test name="Simple example">
  <groups>
    <run>
      <include name="checkintest"/>
      <exclude name="broken"/>
    </run>
  </groups>
  
  <classes>
    <class name="example1.Test1"/>
  </classes>
</test>
Copy code

This will get a clean test run and track which tests are broken and need to be repaired later.

Note: you can also disable tests one by one using the "enabled" attribute on the @ Test and @ Before / After annotations.

5. Partial group

You can define groups at the class level and then add groups at the method level:

@Test(groups = { "checkin-test" })
public class All {
 
  @Test(groups = { "func-test" )
  public void method1() { ... }
 
  public void method2() { ... }
}
Copy code

In this class, method2() is part of the "checkin test" group, which is defined at the class level, while method1() belongs to "checkin test" and "func test".

7, Parameterization

The test method does not have to be parameterless. You can use any number of Parameters on each test method and instruct TestNG to pass the correct Parameters using the @ Parameters annotation.

There are two ways to set these parameters:

• Use TestNG xml
• Programmatically.

1,testng. Parameters in XML

If you use simple values for parameters, you can use TestNG Specify them in XML:

@Parameters({ "first-name" })
@Test
public void testSingleString(String firstName) {
  System.out.println("Invoked testString " + firstName);
  assert "Cedric".equals(firstName);
}
Copy code

In this code, we specify that the parameter firstName of the Java method should receive the value of the XML parameter named first name. This XML parameter is in TestNG Defined in XML:

<suite name="My suite">
  <parameter name="first-name"  value="Cedric"/>
  <test name="Simple example">
  <-- ... -->
Copy code

@Before / After and @ Factory annotations can use the same technique:

@Parameters({ "datasource", "jdbcDriver" })
@BeforeMethod
public void beforeTest(String ds, String driver) {
  m_dataSource = ...;                              // Query data source value
  m_jdbcDriver = driver;
}
Copy code

This time, the two Java parameters ds and driver will receive the values assigned to the attributes datasource and jdbc driver, respectively. You can declare parameters as Optional using the Optional annotation:

@Parameters("db")
@Test
public void testNonExistentParameter(@Optional("mysql") String db) { ... }
Copy code

If in TestNG If the parameter named "db" cannot be found in the XML file, the test method will receive the default value specified in the @ Optional annotation: "mysql". The @ Parameters can be placed in the following locations:

• On any method that already has @ Test, @ Before / After or @ Factory annotations.
• There is at most one constructor for the test class. In this case, TestNG will call this specific constructor and initialize the parameter to TestNG. When the test class needs to be instantiated The value specified in the XML. This function can be used to initialize a field in a class to a value that the test method will then use.

be careful:

• XML parameters are mapped to Java parameters in the same order as in comments, and TestNG issues an error if the numbers do not match.
• Parameter is scoped. In TestNG In XML, you can declare them under the < suite > tag or < test >. If two parameters have the same name, it is the priority parameter defined in < test >. This is convenient if you need to specify parameters that apply to all tests and override their values for only some tests.

2. Use parameters for DataProviders

If you need to pass complex parameters or parameters that need to be created from Java (complex objects, objects read from property files or databases, etc.), in TestNG The parameters specified in the XML may not be enough. In this case, you can use the data provider to provide the values required for the test. A data provider is a method on a class that returns an array of objects. This method uses the @ DataProvider annotation:

//This method will provide data to any test method that declares that its Data Provider
//is named "test1"
@DataProvider(name = "test1")
public Object[][] createData1() {
 return new Object[][] {
   { "Cedric", new Integer(36) },
   { "Anne", new Integer(37)},
 };
}
 
//This test method declares that its data should be supplied by the Data Provider
//named "test1"
@Test(dataProvider = "test1")
public void verifyData1(String n1, Integer n2) {
 System.out.println(n1 + " " + n2);
}
Copy code

Will print

Cedric 36
Anne 37
 Copy code

A @ Test method specifies the data provider properties associated with the data provider. This name must correspond to a @ DataProvider (name = '...') with a matching name Methods on the same class of annotations. By default, the data provider is found in the current Test class or one of the base classes. If you want to place the data provider in a different class, you need to use a static method or a class with a non arg constructor and specify the class that can be found in the dataProviderClass property:

public class StaticProvider {
  @DataProvider(name = "create")
  public static Object[][] createData() {
    return new Object[][] {
      new Object[] { new Integer(42) }
    };
  }
}
 
public class MyTest {
  @Test(dataProvider = "create", dataProviderClass = StaticProvider.class)
  public void test(Integer n) {
    // ...
  }
}
Copy code

Data providers also support injection. TestNG will inject using the test context. The Data Provider method can return one of the following two types: a set of Object arrays (Object []). The size of the first dimension is the number of times the test method is called, and the size of the second dimension contains the Object array methods that must be compatible with the parameter types tested. This is the case shown in the above example. One iteration < Object []] >. The only difference from Object [] [] is that Iterator allows you to create test data lazily. TestNG calls the Iterator, and then calls the test methods one by one using the parameters returned by the Iterator. This is particularly useful if you have many parameter sets to pass to the method and you do not want to create all parameter sets in advance. The following is an example of this feature:

@DataProvider(name = "test1")
public Iterator<Object[]> createData() {
  return new MyIterator(DATA);
}
Copy code

If you declare @ DataProvider as Java lang.reflect. Method as the first parameter, TestNG will pass the current test method for this first parameter. This is especially useful when multiple test methods use the same @ DataProvider and you want it to return different values based on the test method that provides data for it. For example, the following code prints the name of the test method in its @ DataProvider:

@DataProvider(name = "dp")
public Object[][] createData(Method m) {
  System.out.println(m.getName());  // print test method name
  return new Object[][] { new Object[] { "Cedric" }};
}
 
@Test(dataProvider = "dp")
public void test1(String s) {
}
 
@Test(dataProvider = "dp")
public void test2(String s) {
}
Copy code

Therefore, the following is displayed:

test1
test2
 Copy code

Data providers can run in parallel with parallel properties:

@DataProvider(parallel = true)
// ...
Copy code

Parallel data providers running from XML files share the same thread pool, which is 10 by default. You can modify this value in the < suite > tag of the XML file:

<suite name="Suite1" data-provider-thread-count="20" >
Copy code

If you want to run several specific data providers in different thread pools, you need to run them from other XML files.

8, Dependency

Sometimes you need to call test methods in a specific order. Here are some examples: before running more test methods, ensure that a certain number of test methods have been completed and successfully executed. To initialize the test, you also want this initialization method to be a test method (Methods marked with @ Before / After will not be part of the final report). TestNG allows you to specify dependencies using comments or XML.

1. Annotated dependencies

You can annotate the discovered @ Test using the properties dependsOnMethods or dependsOnGroups.

There are two dependencies:

• Hard dependency. All methods you depend on must run and run successfully. If you have at least one fault in your dependencies, you will not call it in the report and mark it as SKIP.
• Soft dependency. You will always run after the methods you depend on, even if some of them fail. This is useful when you just want to ensure that your Test methods run in a specific order, but their success does not really depend on the success of others. Soft dependencies are obtained by adding "alwaysRun = true" in the @ Test annotation.

The following is an example of a hard dependency:

@Test
public void serverStartedOk() {}
 
@Test(dependsOnMethods = { "serverStartedOk" })
public void method1() {}
Copy code

In this example, method1() is declared as dependent on the method serverStartedOk(), which guarantees that serverStartedOk() is always called first.

You can also have methods that depend on the entire group:

@Test(groups = { "init" })
public void serverStartedOk() {}
 
@Test(groups = { "init" })
public void initEnvironment() {}
 
@Test(dependsOnGroups = { "init.*" })
public void method1() {}
Copy code

In this example, method1 () is declared to depend on any group that matches the regular expression "init * *", which ensures that the method serverStartedOk () and initEnvironment () will always be called before method1 ().

Note: as mentioned earlier, for methods belonging to the same group, the call order is not guaranteed to be the same in the test run.

If the dependent method fails and you have a hard dependency on it (alwaysRun = false, which is the default), the dependent method will not be marked as FAIL, but as SKIP. The skipped method will be reported in the final report (the color is neither red nor green in HTML), which is important because the skipped method does not necessarily FAIL.

Both dependsOnGroups and dependsOnMethods accept regular expressions as arguments. For dependsOnMethods, if you rely on methods that happen to have multiple overloaded versions, all overloaded methods will be called. If you only want to call one of the overloaded methods, you should use dependsOnGroups.

By default, dependent methods are grouped by class. For example, if method b () depends on method a (), and you have several instances of classes containing these methods (because of the factory of the data provider), the call order is as follows:

a(1)
a(2)
b(2)
b(2)
Copy code

TestNG will not run b() until all instances call their a() method. In some cases, you may not want this to happen, such as testing logging in and out of Web browsers in various countries. In this case, you need to order the following:

signIn("us")
signOut("us")
signIn("uk")
signOut("uk")
Copy code

For this sort, you can use XML attributes on an instance by instance basis. This attribute is valid on < suite > or < test >

  <suite name="Factory" group-by-instances="true">
or
  <test name="Factory" group-by-instances="true">
Copy code

2. Dependencies in XML

Can be found in TestNG Specify a group dependency in the XML file. Use the < dependencies > tag to do this:

<test name="My suite">
  <groups>
    <dependencies>
      <group name="c" depends-on="a  b" />
      <group name="z" depends-on="c" />
    </dependencies>
  </groups>
</test>
Copy code

The < dependencies on > attribute described above contains a list of groups separated by spaces.

9, Factory

The factory allows dynamic creation of tests. For example, suppose you want to create a test method that will visit a page on a website multiple times and want to call it with different values:

public class TestWebServer {
  @Test(parameters = { "number-of-times" })
  public void accessPage(int numberOfTimes) {
    while (numberOfTimes-- > 0) {
     // access the web page
    }
  }
}
Copy code

<test name="T1">
  <parameter name="number-of-times" value="10"/>
  <classes>
    <class name= "TestWebServer" />
  </classes>
</test>
 
<test name="T2">
  <parameter name="number-of-times" value="20"/>
  <classes>
    <class name= "TestWebServer"/>
  </classes>
</test>
 
<test name="T3">
  <parameter name="number-of-times" value="30"/>
  <classes>
    <class name= "TestWebServer"/>
  </classes>
</test>
Copy code

This will soon become unmanageable, so you should use the factory

public class WebTestFactory {
  @Factory
  public Object[] createInstances() {
   Object[] result = new Object[10]; 
   for (int i = 0; i < 10; i++) {
      result[i] = new WebTest(i * 10);
    }
    return result;
  }
}
Copy code

Now the new test class

public class WebTest {
  private int m_numberOfTimes;
  public WebTest(int numberOfTimes) {
    m_numberOfTimes = numberOfTimes;
  }
 
  @Test
  public void testServer() {
   for (int i = 0; i < m_numberOfTimes; i++) {
     // access the web page
    }
  }
}
Copy code

Your TestNG XML only needs to reference the class containing the factory method, because the test instance itself will be created at run time

<class name="WebTestFactory" />
Copy code

Alternatively, if you build test suite instances programmatically, you can add factories in the same way as testing

TestNG testNG = new TestNG();
testNG.setTestClasses(WebTestFactory.class);
testNG.run();
Copy code

The factory method can receive parameters like @ Test and @ Before / After, and it must return Object []. The returned objects can be any class (not necessarily the same class as the factory class), and they don't even need to contain TestNG annotations (in this case, they will be ignored by TestNG).

Factories can also be used with data providers, which you can take advantage of by placing @ Factory annotations on regular methods or constructors. The following is an example of a constructor Factory:

@Factory(dataProvider = "dp")
public FactoryDataProviderSampleTest(int n) {
  super(n);
}
 
@DataProvider
static public Object[][] dp() {
  return new Object[][] {
    new Object[] { 41 },
    new Object[] { 42 },
  };
}
Copy code

This example will cause TestNG to create two test classes, calling the constructor with the value 41 and the other 42.

10, Ignore test

TestNG allows you to ignore all @ Test methods:

• A class (or)
• Specific package (or)
• In a package and all its sub packages

Use the new comment @ Ignore. Using the @ Ignore annotation at the method level is functionally equivalent to @ Test (enabled = false). This is an example of how to Ignore all tests in a class.

import org.testng.annotations.Ignore;
import org.testng.annotations.Test;
 
@Ignore
public class TestcaseSample {
 
    @Test
    public void testMethod1() {
    }
 
    @Test
    public void testMethod2() {
    }
}
Copy code

The @ Ignore annotation has a higher priority than the annotation of the @ Test method. When @ Ignore is placed on a class, all tests in that class will be disabled. To Ignore all tests in a particular package, simply create package info Java and add the @ Ignore annotation to it. Here is an example:

@Ignore
package com.testng.master;
 
import org.testng.annotations.Ignore;
Copy code

This will lead to com testng. All @ Test methods in the master package and all its sub packages are ignored.

11, Parallelism and timeout

TestNG can be instructed to run tests in separate threads in various ways.

1. Parallel Suite

This is useful if you run multiple suite files (for example, "java org.testng.TestNG testng1.xml testng2.xml") and want each suite to run in a separate thread. You can specify the size of the thread pool using the following command line flags:

java org.testng.TestNG -suitethreadpoolsize 3 testng1.xml testng2.xml testng3.xml
 Copy code

The corresponding ant task name is suitethreadpoolsize.

2. Parallel testing, classes and methods

In the attribute tag parallel to < suite >, you can take one of the following values:

<suite name="My suite" parallel="methods" thread-count="5">
<suite name="My suite" parallel="tests" thread-count="5">
<suite name="My suite" parallel="classes" thread-count="5">
<suite name="My suite" parallel="instances" thread-count="5">
Copy code

• Parameter = "methods": TestNG will run all test methods in different threads. Dependent methods will also run in separate threads, but they will follow the order you specify.
• parallel = "tests": TestNG will run all the methods in the same < test > tag in the same thread, but each < test > tag will be in a separate thread. This allows you to group all non thread safe classes in the same < test > and ensure that they will run in the same thread, while using TestNG to run tests with as many threads as possible.
• parallel = "classes": TestNG will run all methods in the same class in the same thread, but each class will run in a separate thread
• parallel = "instances": TestNG will run all methods in the same instance in the same thread, but two methods on two different instances will run in different threads.

In addition, the property thread count allows you to specify the number of threads that should be allocated for this execution. Note: @ Test attribute timeOut is valid in both parallel and non parallel modes.

You can also specify that the @ Test method should be called from a different thread. You can use the threadPoolSize property to achieve this result:

@Test(threadPoolSize = 3, invocationCount = 10,  timeOut = 10000)
public void testServer() {
Copy code

In this example, the function testServer will be called ten times from three different threads. In addition, a timeout of ten seconds ensures that all threads do not permanently block this thread.

12, Rerun failed tests

Each time a test fails in the suite, TestNG creates a file named TestNG failed. In the output directory XML file. This XML file contains the necessary information to rerun only those methods that failed, allowing you to quickly reproduce the failure without running the entire test. Therefore, a typical session will be as follows:

java -classpath testng.jar;%CLASSPATH% org.testng.TestNG -d test-outputs testng.xml
java -classpath testng.jar;%CLASSPATH% org.testng.TestNG -d test-outputs test-outputs\testng-failed.xml
 Copy code

Notice that TestNG failed XML will contain all the necessary dependent methods so that you can guarantee that the failed methods will run without any SKIP failures.

Sometimes, you might want TestNG to automatically retry the Test if it fails. In these cases, you can use the retry analyzer. When you bind the retry analyzer to a Test, TestNG will automatically call the retry analyzer to determine whether TestNG can retry the Test case again to try to see whether the Test that has just failed now passes. Here's how to use the retry analyzer: build org testng. The implementation of the iretryanalyzer interface binds this implementation to the @ Test annotation. For example, @ Test (retryAnalyzer = LocalRetry.class). The following is an example implementation of the retry analyzer. The Test can be retried up to three times.

import org.testng.IRetryAnalyzer;
import org.testng.ITestResult;
 
public class MyRetry implements IRetryAnalyzer {
 
  private int retryCount = 0;
  private static final int maxRetryCount = 3;
 
  @Override
  public boolean retry(ITestResult result) {
    if (retryCount < maxRetryCount) {
      retryCount++;
      return true;
    }
    return false;
  }
}
import org.testng.Assert;
import org.testng.annotations.Test;
 
public class TestclassSample {
 
  @Test(retryAnalyzer = MyRetry.class)
  public void test2() {
    Assert.fail();
  }
}
Copy code

13, YAML file

TestNG supports YAML as an alternative to specifying suite files. For example, the following XML file:

<suite name="SingleSuite" verbose="2" thread-count="4">
 
  <parameter name="n" value="42" />
 
  <test name="Regression2">
    <groups>
      <run>
        <exclude name="broken" />
      </run>
    </groups>
 
    <classes>
      <class name="test.listeners.ResultEndMillisTest" />
    </classes>
  </test>
</suite>
Copy code

This is its YAML version:

name: SingleSuite
threadCount: 4
parameters: { n: 42 }
 
tests:
  - name: Regression2
    parameters: { count: 10 }
    excludedGroups: [ broken ]
    classes:
      - test.listeners.ResultEndMillisTest
 Copy code

This is TestNG's own suite file and its YAML counterpart. You may find the YAML file format easier to read and maintain. The TestNG Eclipse Plug-in can also recognize YAML files.

Note: by default, TestNG does not introduce YAML related libraries into your classpath. Therefore, depending on your build system (Gradle / Maven), you need to add an explicit reference to the YAML Library in the build file.

For example, if Maven is used, it needs to be in POM Add the following dependencies to the XML file:

<dependency>
  <groupid>org.yaml</groupid>
  <artifactid>snakeyaml</artifactid>
  <version>1.23</version>
</dependency>