preface
Testers who can test and understand programming are still scarce, and most organizations may not invest in this aspect. Therefore, let ordinary test engineers simply learn to carry out automated testing, which is still in market demand.
Before, I was confused that ordinary test engineers could not deeply participate in the case preparation of automated testing, so I searched and found a pytest plug-in called pytest play. Using this plug-in, you can carry out automated work as long as you write YAML files according to the rules, and there is almost no introduction to the plug-in in in Chinese. Therefore, I specially wrote this article, Let me give you an in-depth introduction.
What can you do
Today's protagonist is pytest play, which is a plug-in of a well-known automated testing framework pytest. So, what can this plug-in do for us?
According to the official documents, pytest play can be used to do some automated work, including automated testing.
What are the characteristics
There are many tools that can do automated testing. What are the characteristics of pytest play?
First, non-technical users are available. This should be the biggest feature and advantage of pytest play. For the majority of testers who are not proficient in programming, as long as they learn some relatively simple syntax rules, they can complete the automatic test task by writing clear format text files, which is still very fragrant.
Secondly, test configuration. As we know, YAML is a good file format. We must have used it to write the data and parameters of test cases. When using pytest play, we just need to write the test actions, data and assertions.
Thirdly, it is extensible. Pytest play itself, as an extension of pytest, is designed to be extensible. At present, play is known_ Selenium plug-in, which can drive the browser for Web page testing, with play_ The requests driver initiates HTTP requests for HTTP interface testing and plays_ SQL can execute SQL and assert the results. Of course, you can also develop your own plug-ins.
The above are the main features of pytest play. Other features are easy to install, easy to use, parameterization, integration report, integration with test management tools, smooth learning curve and so on.
How to use
Pytest play can be used in two ways, one without Python and the other. This can meet the needs of different levels.
With test_ The file with the yml extension at the beginning will be automatically recognized and run, or the entire test set can be run with the name or keyword.
No Python mode
The following is an example of a Python free test project, which contains only an optional environment variable file env-ALPHA.yml and a test script file test_login.yml.
$ tree . ├── env-ALPHA.yml (OPTIONAL) └── test_login.yml
The following is the content of the environment variable configuration file. As external parameters, it can be used to run tests in different environments.
$ cat env-ALPHA.yml pytest-play: base_url: https://www.yoursite.com
The following is a test plan file that contains actions, assertions, and metadata (optional).
$ cat test_login.yml
---
markers:
- login
test_data:
- username: siteadmin
password: siteadmin
- username: editor
password: editor
- username: reader
password: reader
---
- comment: Access the initial page
type: get
provider: selenium
url: "$base_url"
- comment: Click the login link
locator:
type: id
value: personaltools-login
type: clickElement
provider: selenium
- comment: enter one user name
locator:
type: id
value: __ac_name
text: "$username"
type: setElementText
provider: selenium
- comment: Input password
locator:
type: id
value: __ac_password
text: "$password"
type: setElementText
provider: selenium
- comment: Click the login button
locator:
type: css
value: ".pattern-modal-buttons > input[name=submit]"
type: clickElement
provider: selenium
- comment: Wait for the page to load
locator:
type: css
value: ".icon-user"
type: waitForElementVisible
provider: selenium
The sample file is divided into two sections with -.
The first segment is the metadata segment, which is not required. You can use markers to set several tags for the current test, which is convenient to filter and run the test according to the tags. You can also use test_data configures the case run data. In this example, three groups of test data are configured, and the whole test will run three times.
The second paragraph is the test script, which uses - to configure five behaviors.
Comment: a comment that briefly describes this item. It can be output to the report for easy viewing and editing. It is not necessary.
provider: used to specify the command to be executed. In this case, selenium will call the selenium plug-in.
type: it can be used as a subcommand of the provider. For example, waitForElementVisible is waiting for HTML elements to be visible.
locator: used to locate HTML elements. type is the location method and value is the location parameter.
Python mode
This method can be used for behavior driven testing using pytest BDD.
The following example is test_ The content of login.py defines the name test_ The login function, called the data test scheme, finally calls the execute_ of play. The raw method runs, extra_ The variables parameter is an environment variable class parameter.
def test_login(play):
data = play.get_file_contents(
'my', 'path', 'etc', 'login.yml')
play.execute_raw(data, extra_variables={})
Operation result report
Use the – junit XML parameter on the command line to generate a result report in junit format.
--junit-xml results.xml
In the report, you can see the errors of each test case, the commands executed each time, and the time consumption. You will see each command line output by default, unless the - s or – capture=no parameter is specified at run time.
The following is an example of an execution report.
<?xml version="1.0" encoding="utf-8"?>
<testsuite errors="0" failures="0" name="pytest" skipped="0" tests="1" time="0.360">
<testcase classname="test_assertion.yml" file="test_assertion.yml" name="test_assertion.yml" time="0.326">
<system-out>{'expression': '1 == 1', 'provider': 'python', 'type': 'assert', '_elapsed': 0.0003077983856201172}
{'expression': '0 == 0', 'provider': 'python', 'type': 'assert', '_elapsed': 0.0002529621124267578}
</system-out>
</testcase>
</testsuite>
The result report can be customized. You can output custom attributes and execution time in the report.
Play is used below_ In the example where the requests plug-in tests the HTTP interface, a custom attribute configuration for collecting indicators is inserted before the assertion. The provider is metrics and the custom attribute name is categories_time, the corresponding indicator type is record_elapsed (similar time). In addition, the example asserts against custom attributes.
test_data: - category: dev - category: movie - category: food --- - type: GET provider: play_requests url: https://api.chucknorris.io/jokes/categories expression: "'$category' in response.json()" - provider: metrics type: record_elapsed name: categories_time - type: assert provider: python expression: "variables['categories_time'] < 2.5" comment: You can use it in assertions categories_time
The output report is as follows. As you can see, categories are output_ Time custom attribute and its value.
<?xml version="1.0" encoding="utf-8"?>
<testsuite errors="0" failures="0" name="pytest" skipped="0" tests="3" time="2.031">
<testcase classname="test_categories.yml" file="test_categories.yml" name="test_categories.yml0" time="0.968">
<properties>
<property name="categories_time" value="0.5829994678497314"/>
</properties>
<system-out>{'expression': "'dev' in response.json()", 'provider': 'play_requests', 'type': 'GET', 'url': 'https://api.chucknorris.io/jokes/categories', '_elapsed': 0.5829994678497314}
{'name': 'categories_time', 'provider': 'metrics', 'type': 'record_elapsed', '_elapsed': 3.3855438232421875e-05}
{'comment': 'You can use it in assertions categories_time', 'expression': "variables['categories_time'] < 2.5", 'provider': 'python', 'type': 'assert', '_elapsed': 0.0006382465362548828}
</system-out>
</testcase>
<testcase classname="test_categories.yml" file="test_categories.yml" name="test_categories.yml1" time="0.481">
<properties>
<property name="categories_time" value="0.4184422492980957"/>
</properties>
<system-out>{'expression': "'movie' in response.json()", 'provider': 'play_requests', 'type': 'GET', 'url': 'https://api.chucknorris.io/jokes/categories', '_elapsed': 0.4184422492980957}
{'name': 'categories_time', 'provider': 'metrics', 'type': 'record_elapsed', '_elapsed': 2.09808349609375e-05}
{'comment': 'You can use it in assertions categories_time', 'expression': "variables['categories_time'] < 2.5", 'provider': 'python', 'type': 'assert', '_elapsed': 0.000553131103515625}
</system-out>
</testcase>
<testcase classname="test_categories.yml" file="test_categories.yml" name="test_categories.yml2" time="0.534">
<properties>
<property name="categories_time" value="0.463592529296875"/>
</properties>
<system-out>{'expression': "'food' in response.json()", 'provider': 'play_requests', 'type': 'GET', 'url': 'https://api.chucknorris.io/jokes/categories', '_elapsed': 0.463592529296875}
{'name': 'categories_time', 'provider': 'metrics', 'type': 'record_elapsed', '_elapsed': 2.09808349609375e-05}
{'comment': 'You can use it in assertions categories_time', 'expression': "variables['categories_time'] < 2.5", 'provider': 'python', 'type': 'assert', '_elapsed': 0.00054931640625}
</system-out>
</testcase>
</testsuite>
How to reuse test steps
In different test schemes, there should be reusable test steps, so that when modifying, you can change one place. Pytest play takes this into account and realizes the reuse of test steps by using the include provider.
In the following example, both provider and type are include keywords, and the test scheme reuses the common test steps specified in path.
- provider: include type: include path: "/some-path/included-scenario.yml"
Of course, if there are variables in the test steps, they can be configured under the common root directory of these test scripts.
Default command (reuse of commands)
Similar to type inheritance in code, pytest play also provides a mechanism for reusing test commands.
If play is used_ When requests tests multiple interfaces of the same service, the same header needs to be passed in every call. In this case, the default command can be used.
In the following example, Section 1 uses the store of the python command_ The variable subcommand defines a variable named bear with a value of bear. Store using Python command in Section 2_ Variable subcommand, defined with the name play_ The default command of requests sets the public HTTP header in expression. Play is directly reused in Section 3_ The requests command eliminates the need to define HTTP headers.
- provider: python
type: store_variable
name: bearer
expression: "'BEARER'"
- provider: python
type: store_variable
name: play_requests
expression: "{'parameters': {'headers': {'Authorization': '$bearer'}}}"
- provider: play_requests
type: GET
comment: this is an authenticated request!
url: "$base_url"
Declare variables, assertions
As you can see in the above example, we can use the python command store in the script_ The variable subcommand declares variables for use in subsequent steps.
In the following example, a variable named foo is declared, and its value is set through an expression expression expression. The expression is computable and the final value is 2. In the subsequent test steps, assertions are made through the assert subcommand of the python command.
- provider: python type: store_variable expression: "1+1" name: foo - provider: python type: assert expression: variables['foo'] == 2
sleep
During the test process, it is sometimes necessary to pause the operation for a period of time, and pytest play is also supported. The following example will pause for 2 seconds.
- provider: python type: sleep seconds: 2
Execute expression
Pytest play supports the simple execution of an expression through the exec subcommand of the python command.
- provider: python type: exec expression: "1+1"
loop
Complex test steps are not supported. Pytest play supports running a test step in a loop.
In the following example, the subcommand is while, indicating that this is a loop. The adjacent expression is the condition for entering the cycle. The variable countdown is required to be greater than or equal to 0. Timeout indicates that the timeout of the cycle is 2.3 seconds, poll indicates that it is judged every 0.1 seconds, sub_commands represents the loop body, which contains the test steps to be executed in the loop.
- provider: python
type: while
expression: variables['countdown'] >= 0
timeout: 2.3
poll: 0.1
sub_commands:
- provider: python
type: store_variable
name: countdown
expression: variables['countdown'] - 1
Condition step
Pytest play also supports judging whether to skip the current test step according to conditions. In the following example, skip_ The expression in condition is a condition to skip the current command.
- provider: include type: include path: "/some-path/assertions.yml" skip_condition: variables['cassandra_assertions'] is True
Assert cumulative time
Pytest play maintains a named_ The elapsed variable indicates the elapsed time after the start of the test, which can be used in assertions.
- type: GET provider: play_requests url: https://api.chucknorris.io/jokes/categories expression: "'dev' in response.json()" - type: assert provider: python expression: "variables['_elapsed'] > 0"
Time tracking
Sometimes, we need to know that each test command takes time to execute, and pytest play is also supported.
In the following example, through the subcommand record of the metrics command_ elapsed_ start,record_elapsed_stop respectively record the start and end time of the step and give it to load respectively_ Time and live_ search_ There are two indicators.
- provider: selenium
type: get
url: https://www.plone-demo.info/
- provider: metrics
type: record_elapsed_start
name: load_time
- provider: selenium
type: setElementText
text: plone 5
locator:
type: id
value: searchGadget
- provider: metrics
type: record_elapsed_stop
name: load_time
- provider: metrics
type: record_elapsed_start
name: live_search_time
- provider: selenium
type: waitForElementVisible
locator:
type: css
value: li[data-url$="https://www.plone-demo.info/front-page"]
- provider: metrics
type: record_elapsed_stop
name: live_search_time
The following is the result report after operation. You can see the values of the two indicators, which are about 1.1s and 1.09s respectively.
<?xml version="1.0" encoding="utf-8"?>
<testsuite errors="0" failures="0" name="pytest" skipped="0" tests="1" time="13.650">
<testcase classname="test_search.yml" file="test_search.yml" name="test_search.yml" time="13.580">
<properties>
<property name="load_time" value="1.1175920963287354"/>
<property name="live_search_time" value="1.0871295928955078"/>
</properties>
<system-out>{'provider': 'selenium', 'type': 'get', 'url': 'https://www.plone-demo.info/', '_elapsed': 9.593282461166382}
{'name': 'load_time', 'provider': 'metrics', 'type': 'record_elapsed_start', '_elapsed': 1.1682510375976562e-05}
{'locator': {'type': 'id', 'value': 'searchGadget'}, 'provider': 'selenium', 'text': 'plone 5', 'type': 'setElementText', '_elapsed': 1.1019845008850098}
{'name': 'load_time', 'provider': 'metrics', 'type': 'record_elapsed_stop', '_elapsed': 1.9788742065429688e-05}
{'name': 'live_search_time', 'provider': 'metrics', 'type': 'record_elapsed_start', '_elapsed': 1.0013580322265625e-05}
{'locator': {'type': 'css', 'value': 'li[data-url$="https://www.plone-demo.info/front-page"]'}, 'provider': 'selenium', 'type': 'waitForElementVisible', '_elapsed': 1.060795545578003}
{'name': 'live_search_time', 'provider': 'metrics', 'type': 'record_elapsed_stop', '_elapsed': 2.3603439331054688e-05}
</system-out>
</testcase>
</testsuite>
Measurement time can be used to predict system behavior and compare performance under different branches or scenarios. In addition to the above methods provided by pytest play, you can also install pytest StatsD plug-in, StatsD and graphite Web integration to achieve this purpose.
StatsD and graphite web can be deployed using Docker. See:
https://graphite.readthedocs.io/en/latest/install.html
The pytest statsd plug-in installation commands are as follows:
pip install pytest-play[statsd]
The usage is as follows:
--stats-d [--stats-prefix play --stats-host http://myserver.com --stats-port 3000]
– stats-d is used to enable statistics, – stats host specifies the HTTP address of the target server receiving indicator data, – stats port specifies the port of the target server, – stats prefix is used to distinguish the source when the target server is public.
To collect and send indicator data, you can use record_elapsed,record_elapsed_start,record_elapsed_stop and other commands, you can also use record_property command, but metric must be provided at the same time_ Type additional parameter. The specific usage is as follows:
- provider: metrics type: record_property name: categories_time expression: "variables['_elapsed']*1000" metric_type: timing - provider: metrics type: record_property name: fridge_temperature expression: "4" metric_type: gauge
If metric is not provided_ Type, the indicator data will not be sent to StatsD.
The following is an example of the final generated statistical graph.
HTTP interface response time example
Browser performance example
performance testing
You can use the pytest play test scheme as a performance test using bzt/Taurus.
The specific method is to add a bzt/Taurus YAML file. Note that the file name cannot be test_ start. Here is a complete example:
https://github.com/pytest-dev/pytest-play/tree/features/examples/bzt_performance
settings:
artifacts-dir: /tmp/%Y-%m-%d_%H-%M-%S.%f
execution:
- executor: pytest
scenario: pytest-run
iterations: 1
scenarios:
pytest-run:
# additional-args: --stats-d --stats-prefix play
script: scripts/
services:
- module: shellexec
prepare:
- pip3 install -r https://raw.githubusercontent.com/davidemoro/pytest-play-docker/master/requirements.txt
The operation method is as follows:
docker run --rm -it -v $(pwd):/src --user root --entrypoint "bzt" davidemoro/pytest-play bzt.yml
After the command is executed, you can see bzt start and run each test scenario:
Use dynamic data directly in the message body
Store is usually used_ Variable subcommand to provide variable support. Variables in this way can be used in multiple places. Sometimes, when you need to temporarily generate a value when sending a message (REST or MQTT), you need to use {! Expression!} to declare an expression. An example is as follows.
- comment: python expressions in mqtt payload (without declaring variables)
provider: mqtt
type: publish
host: "$mqtt_host"
port: "$mqtt_port"
endpoint: "$mqtt_endpoint/$device_serial_number"
payload: '{
"measure_id": [124],
"obj_id_L": [0],
"measureType": ["float"],
"start_time": {! int(datetime.datetime.utcnow().timestamp()*1000) !},
"bin_value": [1]
}'
extend
Pytest play provides a set of extension mechanism to support custom commands.
Extend the command as follows:
command = {'type': 'print', 'provider': 'newprovider', 'message': 'Hello, World!'}
You also need to implement the command provider:
from pytest_play.providers import BaseProvider
class NewProvider(BaseProvider):
def this_is_not_a_command(self):
""" Commands should be command_ prefixed """
def command_print(self, command):
print(command['message'])
def command_yetAnotherCommand(self, command):
print(command)
And register in setup.py:
entry_points={
'playcommands': [
'print = your_package.providers:NewProvider',
],
},
Example
Here are some examples of addresses:
- https://github.com/pytest-dev/pytest-play/tree/master/examples
- https://github.com/davidemoro/pytest-play-docker/tree/master/tests
- https://github.com/davidemoro/pytest-play-plone-example
plug-in unit
The existing third-party plug-ins are as follows:
- play_selenium: test Web pages through Selenium/Splinter driven browsers
- play_requests: drives Python's requests library to send requests and test the HTTP interface
- play_sql: supports accessing SQL database and asserting
- play_cassandra: support Cassandra distributed NoSQL database
- play_dynamodb: query and assert AWS DynamoDB
- play_ websockets: support websockets
- play_mqtt: provides support for MQTT
OK, that's all for pytest play
The following is the supporting information. For the friends doing [software testing], it should be the most comprehensive and complete war preparation warehouse. This warehouse has also accompanied me through the most difficult journey. I hope it can also help you!
Finally, it can be in the official account: the sad spicy bar! Get a 216 page interview document of Software Test Engineer for free. And the corresponding video learning tutorials for free!, It includes basic knowledge, Linux essentials, Shell, Internet program principles, Mysql database, special topics of packet capture tools, interface test tools, test advanced Python programming, Web automation test, APP automation test, interface automation test, advanced continuous test integration, test architecture, development test framework, performance test, security test, etc.
Don't fight alone in learning. It's best to stay together and grow together. The effect of mass effect is very powerful. If we study together and punch in together, we will have more motivation to learn and stick to it. You can join our testing technology exchange group: 914172719 (there are various software testing resources and technical discussions)
Friends who like software testing, if my blog is helpful to you and if you like my blog content, please click "like", "comment" and "collect" for three times!
Haowen recommendation
What kind of person is suitable for software testing?