Automated Test Framework - pytest
pytest is one of Python's most popular unit testing frameworks that helps you write test scripts more easily and supports a variety of complex test scenarios that can be used for app testing as well as function testing
Official documents: https://docs.pytest.org/en/latest/
pytest has the following advantages:
- Allow assert s
- Automatically identify test scripts, classes, functions
- Can be used to manage small or parameter type test data or resources
- Compatible with unittest and nose test framework
- Support for Python 2.7/Python 3.4+
- Rich plugin support, more than 315 plugin support
pytest installation
pip install -U pytest
If you are prompted for the following error, it means that the version of pip is too old and needs to be updated below:
Could not find a version that satisfies the requirement pytest (from versions: ) No matching distribution found for pytest
Update method:
easy_install --upgrade pip
Official examples
Prepare a test_sample.py as follows:
def inc(x): return x + 1 def test_answer(): assert inc(3) == 5
Execute in the directory where the file is located:
pytest
Here we will explain:
The py test scripts all have the file name test_xxx.py;
The inc method is a self-increasing function defined by us that returns the passed parameter by adding 1.
test_answer is a test function that we write, in which we validate the results using the basic assert assert assert, named after test_xxx
The results are as follows:
============================================================ test session starts ============================================================ platform darwin -- Python 2.7.15, pytest-4.1.0, py-1.7.0, pluggy-0.8.0 rootdir: /Users/jackey/Documents/iOS/code/iOS-Auto/Agent_Test, inifile: collected 1 item test_sample.py F [100%] ================================================================= FAILURES ================================================================== ________________________________________________________________ test_answer ________________________________________________________________ def test_answer(): > assert inc(3) == 5 E assert 4 == 5 E + where 4 = inc(3) test_sample.py:5: AssertionError ========================================================= 1 failed in 0.05 seconds ========================================================== (wda_python) bash-3.2$
Error when executing to assert inc(3) == 5
Executing py test finds the test file for test_xx.py in the current directory and subdirectories, goes into the test file and finds the test function that starts with test_xx to begin execution
Executing py test-q test_xxx.py is a script to execute
In one example, the test specifies an error: (Assert that a certain exception is raised)
import pytest def f(): raise SystemExit(1) def test_mytest(): with pytest.raises(SystemExit): f()
Execute instructions:
pytest -q test_sysexit.py
Output:
(wda_python) bash-3.2$ pytest -q test_sysexit.py . [100%] 1 passed in 0.04 seconds (wda_python) bash-3.2$
If you want to develop more than one test method, you can write the method into one class
class TestClass(object): def test_one(self): x = 'this' assert 'h' in x def test_two(self): x = 'hello' assert hasattr(x, 'check')
pytest automatically recognizes test methods in classes, and it doesn't require us to create subclasses or real instances. The results are as follows:
(wda_python) bash-3.2$ pytest -q test_sample.py .F [100%] ================================================================== FAILURES ================================================================== _____________________________________________________________ TestClass.test_two _____________________________________________________________ self = <test_sample.TestClass object at 0x102e151d0> def test_two(self): x = 'hello' > assert hasattr(x, 'check') E AssertionError: assert False E + where False = hasattr('hello', 'check') test_sample.py:8: AssertionError 1 failed, 1 passed in 0.08 seconds (wda_python) bash-3.2$
In addition to executing pytest directly in the script path, you can use the following methods
python -m pytest xxx.py
Stop when the first (or N th) error occurs
pytest -x # stop after first failure pytest --maxfail=2 # stop after two failures
Run Execution Test Script
pytest test_mod.py
Run all scripts in the specified directory
pytest testing/
Run a test method that contains the specified keyword, which can be file name, class name, test function name
pytest -k "MyClass and not method"
Execute the node id to run the test script, and each collected test method is assigned a specified id. We can run the executed test method in one of the following ways:
# To run a specific test within a module pytest test_mod.py::test_func # To run a test within a class pytest test_mod.py::TestClass::test_method
Different ways of printing logs
pytest --showlocals # show local variables in tracebacks pytest -l # show local variables (shortcut) pytest --tb=auto # (default) 'long' tracebacks for the first and last # entry, but 'short' style for the other entries pytest --tb=long # exhaustive, informative traceback formatting pytest --tb=short # shorter traceback format pytest --tb=line # only one line per failure pytest --tb=native # Python standard library formatting pytest --tb=no # no traceback at all
Test Report
pytest is a complete test report by default, we can add-r tag to show a short test report, you can match the parameters
Here is the full list of available characters that can be used: f - failed E - error s - skipped x - xfailed X - xpassed p - passed P - passed with output a - all except pP
Can be used with multiple parameters
Debug mode
pytest --pdb
Example:
(wda_python) bash-3.2$ pytest --pdb ========================================================== test session starts =========================================================== platform darwin -- Python 2.7.15, pytest-4.1.0, py-1.7.0, pluggy-0.8.0 rootdir: /Users/jackey/Documents/iOS/code/iOS-Auto/Agent_Test, inifile: collected 3 items test_sample.py .F >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> traceback >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> self = <test_sample.TestClass object at 0x10e928610> def test_two(self): x = 'hello' > assert hasattr(x, 'check') E AssertionError: assert False E + where False = hasattr('hello', 'check') test_sample.py:8: AssertionError >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> entering PDB >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > /Users/jackey/Documents/iOS/code/iOS-Auto/Agent_Test/test_sample.py(8)test_two() -> assert hasattr(x, 'check') (Pdb) print x hello (Pdb) print hasattr(x,'check') False (Pdb)
You can also specify the number of failures to start entering debug:
pytest -x --pdb # drop to PDB on first failure, then end test session pytest --pdb --maxfail=3 # drop to PDB for first three failures
Any failed exception information is stored in sys.last_value, sys.last_type, and sys_last_traceback
The contents of the last error can be obtained in debug by
(Pdb) import sys (Pdb) sys.last_traceback.tb_lineno 1357 (Pdb) sys.last_value AssertionError(u"assert False\n + where False = hasattr('hello', 'check')",) (Pdb)
Enter debug mode at the beginning of execution
pytest --trace
Enter next to execute next step, exit to exit
Setting breakpoints in scripts
import pdb pdb.set_trace()
For example:
import pdb class TestClass(object): def test_one(self): x = 'this' pdb.set_trace() assert 'h' in x def test_two(self): x = 'hello' assert hasattr(x, 'check')
Get the slowest n test steps to execute
pytest --durations=10
======================================================= slowest 10 test durations ======================================================== (0.00 durations hidden. Use -vv to show these durations.)
However, if all scripts run less than 0.01s, they will not be displayed unless the -vv parameter is taken
pytest --durations=10 -vv
Output results:
======================================================= slowest 10 test durations ======================================================== 0.00s call test_sample.py::TestClass::test_two 0.00s setup test_sysexit.py::test_mytest 0.00s setup test_sample.py::TestClass::test_two 0.00s setup test_sample.py::TestClass::test_one 0.00s teardown test_sample.py::TestClass::test_two 0.00s teardown test_sample.py::TestClass::test_one 0.00s call test_sysexit.py::test_mytest 0.00s teardown test_sysexit.py::test_mytest 0.00s call test_sample.py::TestClass::test_one =================================================== 1 failed, 2 passed in 0.06 seconds =================================================== (wda_python) bash-3.2$
Save log to specified file
pytest --resultlog=path
Disabling plugins
To disable loading specific plugins at invocation time, use the -p option together with the prefix no:.
Example: to disable loading the plugin doctest, which is responsible for executing doctest tests from text files, invoke pytest like this:
pytest -p no:doctest
We can also start pytest in the pytestdemo script:
import pytest pytest.main()
Executing python pytestdemo.py executes pytest
main() does not throw an exception to SystemExit, but returns exitcode, which has six exitcodes
Exit code 0: All tests were collected and passed successfully Exit code 1: Tests were collected and run but some of the tests failed Exit code 2: Test execution was interrupted by the user Exit code 3: Internal error happened while executing tests Exit code 4: pytest command line usage error Exit code 5: No tests were collected
Let's try to add printing
import pytest print pytest.main()
Output:
(wda_python) bash-3.2$ python pytestDemo.py ========================================================== test session starts =========================================================== platform darwin -- Python 2.7.15, pytest-4.1.0, py-1.7.0, pluggy-0.8.0 rootdir: /Users/jackey/Documents/iOS/code/iOS-Auto/Agent_Test, inifile: collected 3 items test_sample.py .F [ 66%] test_sysexit.py . [100%] ================================================================ FAILURES ================================================================ ___________________________________________________________ TestClass.test_two ___________________________________________________________ self = <test_sample.TestClass object at 0x1038ba650> def test_two(self): x = 'hello' > assert hasattr(x, 'check') E AssertionError: assert False E + where False = hasattr('hello', 'check') test_sample.py:11: AssertionError =================================================== 1 failed, 2 passed in 0.05 seconds =================================================== 1 (wda_python) bash-3.2$
We can also pass parameters in main:
pytest.main(['-q','test_sample.py'])
Add a plugin to pytest.main, adding print information at the beginning and end of execution as shown in the following example
import pytest
class MyPlugin(object):
def pytest_sessionfinish(self):
print '*** Test run reporting finishing'
def pytest_sessionstart(self):
print '*** Test run report beginning'
pytest.main(['-q','test_sample.py'], plugins=[MyPlugin()])
Output:
(wda_python) bash-3.2$ python pytestDemo.py
*** Test run report beginning
.F [100%]*** Test run reporting finishing
================================================================ FAILURES ================================================================
___________________________________________________________ TestClass.test_two ___________________________________________________________
self = <test_sample.TestClass object at 0x1090843d0>
def test_two(self):
x = 'hello'
> assert hasattr(x, 'check')
E AssertionError: assert False
E + where False = hasattr('hello', 'check')
test_sample.py:11: AssertionError
1 failed, 1 passed in 0.05 seconds
Origin: https://www.cnblogs.com/zhouxihi/p/10244320.html