Python automated test framework, who is your only one?

Posted by jannz on Mon, 03 Jan 2022 18:17:28 +0100

Python's unique advantages have created a series of test frameworks. In front of these test frameworks, which is better or worse? How to choose?

As the saying goes, "there is no best, there is only the most suitable". Today, we will compete with the four mainstream automated testing frameworks frequently used in Python to select the one that is most suitable for your current project needs.

  • Robot Framework

  • Unittest

  • Nose2

  • Pytest

Introduction to software testing and development, full set of information sharing and explanation

Preliminary study on Framework


Robot Framework

Robot Framework, referred to as RF, is an open source automated testing framework based on Python, which creates test cases in the form of keywords.

Its tabular use case writing mode, rich libraries and toolsets, as well as its powerful functions such as parallel testing, make RF very popular among testers.

RF is mainly used in acceptance testing and test driven development.

In addition, RF also supports automated testing of desktop applications, mobile applications, Web applications, etc. on cross platforms such as Windows, Mac OS and Linux.


*** Settings ***Library SeleniumLibrary*** Variables ***${SERVER} localhost:7272${BROWSER} Firefox${DELAY} 0${VALID USER} demo${VALID PASSWORD} mode${LOGIN URL} http://${SERVER}/${WELCOME URL} http://${SERVER}/welcome.html${ERROR URL} http://${SERVER}/error.html*** Keywords ***Open Browser To Login Page Open Browser ${LOGIN URL} ${BROWSER} Maximize Browser Window Set Selenium Speed ${DELAY}Login Page Should Be Open Title Should Be Login PageGo To Login Page Go To ${LOGIN URL} Login Page Should Be OpenInput Username [Arguments] ${username} Input Text username_field ${username}Input Password [Arguments] ${password} Input Text password_field ${password}Submit Credentials Click Button login_buttonWelcome Page Should Be Open Location Should Be ${WELCOME URL} Title Should Be Welcome Page


Unittest is an automated unit test framework based on Python. It belongs to the Python standard library and supports the reuse of test suites.

Just import the Unittest library into the test script, and testers can customize test classes, create test cases, and pass Unittest Main() runs all test cases.


import unittestdef add(x, y): return x + yclass Test(unittest.TestCase): def test_add_001(self): self.assertEquals(add(4, 5), 9) def test_add_002(self) self.assertNotEqual(add(1,2),10)if __name__ == '__main__': unittest.main()

Common packages / methods are as follows:


Nose2 inherits from nose. It is also a Python based unit testing framework and can be regarded as an extension of Unittest framework. Therefore, test cases written by Unittest can be run under nose2.

Nose2 has a rich set of plug-ins, including writing test cases, exception handling and other functions. Compared with the Unittest and Robot Framework mentioned above, it is not so popular, but it is still a useful open source testing framework.


from mynum import *import nosedef add(x, y): return x + ydef test_add_integers(): assert add(5, 3) == 8def test_add_floats(): assert add(1.5, 2.5) == 4def test_add_strings():, add, 'hello', 'nose2')if __name__ == '__main__':

Common packages / methods are as follows:


Pytest is another very popular open source testing framework for Python. Its syntax is simple and has rich plug-ins. It can be applied to many test types, such as function test, API test, database and UI test.


import pytestdef test_demo_method1(): x = 1 y = 2 assert x+1 == y, "test pass"def test_demo_method2(): x = 6 y = 3 assert x-1 == y+2, "test failed"

Common packages / methods are as follows:


The following is @ pytest Take the fixture decorator as an example. Take a brief look at its initialization function:

(1) Create a file separately, which contains a method decorated with @ pytest.fixture, which returns a list data.

import pytest@pytest.fixturedef supply_AA_BB_CC(): aa = 25 bb = 35 cc = 45 print("This is a separate file[],Contains fixture label") return[aa,bb,cc]

(2) Call the modified method in to obtain the initialization test data.

import pytestdef test_withAA(supply_AA_BB_CC): zz = 35 assert supply_AA_BB_CC[0]== zz, "Verification failed. They are not equal"def test_withBB(supply_AA_BB_CC): zz = 35 assert supply_AA_BB_CC[1]== zz, "Verification failed. They are not equal"def test_withCC(supply_AA_BB_CC): zz = 35 assert supply_AA_BB_CC[2]== zz, "Verification failed. They are not equal"

(3) The operation results are shown as follows:

---------------------------- Captured stdout setup ----------------------------This is a separate file[],Contains fixture label=========================== short test summary info ===========================FAILED - AssertionError: Verification failed, both are not valid...FAILED - AssertionError: Verification failed, both are not valid...========================= 2 failed, 1 passed in 1.20s =========================

Frame selection


After having a basic understanding of the four mainstream Python automation testing frameworks, how to choose the one suitable for the current project?

In order to be targeted and understand the advantages and limitations of each framework, it is the first choice to select the best Python testing framework. Let's explore it.

Robot Framework




While comparing the advantages and limitations of the test framework, it needs to be considered in combination with the test types:

  • Function test: Robot Framework, Pytest, Unittest

  • Behavior driven testing: Behave and Lettuce (these two are not covered in this article)



Through the above analysis and comparison, combined with the automation test requirements of the actual project, the best option can be obtained after comprehensive consideration, which is more conducive to the progress of the automation project, achieve twice the result with half the effort, truly implement the automation framework effectively and find the only one that belongs to you.

Topics: Python