Before doing the interface test, we need to find out the details of the tested interface according to the interface document or packet capturing interface data, including the coding format of the request parameters, so as to send the request in the corresponding parameter format. For example, if the encoding method of the request body specified by an interface is application/json, when requesting the interface, the request parameter format must be json format, and the request will not succeed if other encoding methods are used.

Then, what are the common encoding methods of the request subject in http requests? What format of request parameters should be used for each encoding method in python? This is what we want to understand in this blog.

content-type

In order to understand the encoding format of request parameters, it is necessary to understand the content type field in the HTTP request header.

Function of content type

In the HTTP protocol, the message usually includes two parts: the request head and the request body. The body can be empty. For example, the GET request puts the request parameters in the request URL rather than in the body.

The request parameters in the POST request are placed in the body. The end receiving the request (that is, the server) needs to know what type of data the transmitted body is and what encoding method it adopts in order to parse the data accordingly. At this time, it is necessary to use content type in the request header to indicate the media type of the body.

media type

Media types (commonly referred to as Multipurpose Internet Mail Extensions or MIME types) are a standard used to represent the nature and format of documents, files, or byte streams. It's in IETF RFC 6838 It is defined and standardized in.

Browsers usually use MIME types (rather than file extensions) to determine how to handle URL s, so it is important for the Web server to add the correct MIME type to the response header. If the configuration is incorrect, the browser may misinterpret the contents of the file, the website will not work properly, and the downloaded file will also be handled incorrectly. The content transmitted from the browser also needs to specify the media type, and the server can analyze the corresponding data according to the media type.

Common media types are as follows:

In addition, there are multipart types, which represent the types of file types in subdivided fields. They often correspond to different MIME types, which are used for file upload. They are divided into the following two types:

multipart/form-data
multipart/byteranges

Here is only a brief description. For the specific use details of each type, you can find the relevant information by yourself.

In short, in the interface test, the content type in the request header is used to inform the server of the encoding method of the request body. Because the body of the GET request is empty, there is no content type field in the GET request.

get request parameter format

We already know that the request parameters of the GET request are directly placed in the URL, and there is no need for content type to specify the media type. The encoding format of the request parameters in the GET request is query string params.

query string params

explain

In the format of query string params, the parameters will be passed in the form of url string, that is? The string after is its request parameter, with & as the separator. The parameter is written in the following way:? Key = value & key = value, spliced after the url. It is usually used for GET requests. In addition, some other request methods can also use this format.

Open Baidu in the browser and search for a page of white paper - blog Garden. You can also see that the format of request parameters in Payload is Query String Parameters through F12 packet capture, as shown in the following figure:

The URL link opens https://www.baidu.com/s?ie=utf -8 & WD = give you a page of white paper - blog Garden.

python code sending request

If you use requests in python Get() to send a get request to the example above, you need to use the parameter params, and the parameter value is in the format of dict (Dictionary). The example is as follows:

import requests

url = "http://www.baidu.com/s"
params = {"wd": "Here's a blank page for you-Blog Garden", "ie": "utf-8"}
res = requests.get(url=url, params=params)
print(res.text)

The coding format of the request body in the GET request is fixed, and there is basically no need to determine its coding format in the interface test, which is relatively simple.

post request parameter format

From the previous article Send post request , we know that the request parameters of POST requests in HTTP protocol have different encoding formats.

The server usually knows the encoding method of the message body in the request according to the content type field in the request header, and then parses the corresponding method of the request parameters.

There are four common encoding methods for post requests:

1. application/x-www-form-urlencoded

2. multipart/form-data

3. application/json

4. text/xml

application/x-www-form-urlencoded

application/x-www-form-urlencoded is the most common way to submit data through POST, which is used to submit form data. Native browser

If the enctype attribute is not set for the form, the data will be submitted in the application/x-www-form-urlencoded mode by default.

explain

When the POST request uses application/x-www-form-urlencoded to encode the request parameters, it has the following characteristics:

1. The value of content type in the request header is: application/x-www-form-urlencoded

2. The request parameters will be encoded in the way of key1 = value1 & key2 = Value2, and both key and value are URL transcoded

3. After receiving the request, the server will parse the request parameters in this encoding format in a corresponding way

4. Although the request parameters are encoded in the form of key1 = value1 & key2 = Value2, the encoded content will be spliced after the URL in the GET request.

Take the request for the login interface of TesterHome network as an example:

The content type in the Request Headers is application/x-www-form-urlencoded; charset=UTF-8.

python code sending request

Use requests. In python When post () requests the interface in the above picture, it needs to use the parameter data, and the parameter value is in dict (Dictionary) format. The code is as follows:

import requests

'''
Request header content-type by application/x-www-form-urlencoded
'''

data = {
	"user[login]": "account number",
	"user[password]": "password",
	"user[remember_me]": 0,
	"commit": "Sign in"
}
headers = {
	"user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/98.0.4758.82 Safari/537.36",
    "content-type": "application/x-www-form-urlencoded; charset=UTF-8"
}
url = "https://testerhome.com/account/sign_in"

# The coding format is application/x-www-form-urlencoded;charset=UTF-8, using the data parameter, the parameter value is dict,
res = requests.post(url=url, headers=headers, data=data)
print(res.text)

multipart/form-data

Multipart / form data is also a common POST data submission method for uploading files. When using forms to upload files, we must make the enctype of the form equal to multipart / form data.

explain

When the POST request uses multipart / form data to encode the request parameters, it has the following characteristics:

1. Use boundary to split different fields

2. The message body is divided into several parts with similar structure according to the number of fields. Each part starts with -- boundary, followed by the content description information and the specific content of the field (text or binary). If you are transferring a file, you should also include the file name and file type information

3. The message body ends with -- boundary --

Examples are as follows:

POST http://www.example.com HTTP/1.1
Content-Type:multipart/form-data; boundary=----WebKitFormBoundary8G1vtgT1pXWqqHzV

------WebKitFormBoundary8G1vtgT1pXWqqHzV
Content-Disposition: form-data; name="txt"

title
------WebKitFormBoundary8G1vtgT1pXWqqHzV
Content-Disposition: form-data; name="file"; filename="blog.png"
Content-Type: image/png

PNG ... content of blog.png ...
------WebKitFormBoundary8G1vtgT1pXWqqHzV--

python code sending request

For example, we are requesting Niu Tu net Content type: multipart / form data in Requests Headers; Boundary = --- webkitformboundary4aa3zrkovwuivmx0, as shown in the following figure:

Using requests. In python Post () requests this interface. The code example is as follows:

import requests

'''
Request header content-type by multipart/form-data
'''

def post_mulitpart_form_data():
    headers = {
        "user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/98.0.4758.82 Safari/537.36",
        "content-type": "multipart/form-data; boundary=----WebKitFormBoundary4aA3ZrkOVwUIvmx0"
    }
    url = "https://www.niupic.com/api/upload"
    
    filepath = "./dianzan.jpg"
    # Read the contents of the uploaded file
    files = {"file": open(filepath, "rb")}
    # Use the files parameter to receive the requested content, that is, the read uploaded file content
    res = requests.post(url=url, headers=headers, files=files).content
    print(json.loads(res))


if __name__ == '__main__':
    post_mulitpart_form_data()

The operation results are as follows:

C:\Users\xiaoqq\AppData\Local\Programs\Python\Python37\python.exe E:/blog/python Interface automation/flask_demo/test.py
{'status': 'success', 'code': 200, 'data': 'https://i.niupic. com/images/2022/02/20/9V3n. Jpg ',' MSG ':' upload succeeded! '}

Process finished with exit code 0

application/x-www-form-urlencoded and multipart / form data, the two data encoding methods of POST requests, are supported by the browser natively, and the native forms in the current standard are also supported Only these two methods are supported (specified by the enctype attribute of the element, the default is application/x-www-form-urlencoded. In fact, enctype also supports text/plain, but it is rarely used).

application/json

In the common business scenarios of general companies, application/json is very common. It is used to tell the server that the message body is the serialized json string, that is, the data transmitted from the front end to the server is in json format.

In other words, if the content type in the request header is application/json, when we use tools such as postman or python script to simulate the request excuse, the request parameters also need to be converted into json format before sending the request. Code examples are as follows:

import requests
import json

'''
Request header content-type by application/json
'''

headers = {"Content-Type": "application/json;charset=utf8"}
url = "http://127.0.0.1:5000/login"
_data = {
    "username": "lilei",
    "password": "123456"
}

# The json parameter, json, is used here=_ data
res = requests.post(url=url, headers=headers, json=_data).text
# Of course, you can use the data parameter first_ Convert data to json format, that is, data = json dumps(_data)
# json.dumps() converts dict format to JSON format
res = requests.post(url=url, headers=headers, data=json.dumps(_data)).text
print(res)

text/xml

In the post request, the encoding format of some request bodies is text/xml, that is, the corresponding value of the content type field in the request header is text/xml. For such an interface, we need to use the parameters in xml format to send the request.

Use requests When sending a post request whose request parameter is in xml format, post () only needs to write the body part in the xml file as a string type. When encountering a line break, it will be followed by a backslash and assign this string to the data parameter. Code examples are as follows:

import requests

'''
Request header content-type by text/xml
'''

def post_text_xml():
    headers = {"Content-Type": "text/xml"}
    url = "http://httpbin.org/post"

    body = '<?xml version="1.0" encoding = "UTF-8"?>' \
           '<COM>' \
           '<REQ name="Here's a blank page for you">' \
           '<USER_ID></USER_ID>' \
           '<COMMODITY_ID>111111</COMMODITY_ID>' \
           '<SESSION_ID>asdfghjklfr0123</SESSION_ID>' \
           '</REQ>' \
           '</COM>'

    res = requests.post(url=url, headers=headers, data=body.encode("utf-8")).text
    print(res)


if __name__ == '__main__':
    post_text_xml()

The operation results are as follows:

C:\Users\xiaoqq\AppData\Local\Programs\Python\Python37\python.exe E:/blog/python Interface automation/flask_demo/test.py
{
  "args": {}, 
  "data": "<?xml version=\"1.0\" encoding = \"UTF-8\"?><COM><REQ name=\"\u7ed9\u4f60\u4e00\u9875\u767d\u7eb8\"><USER_ID></USER_ID><COMMODITY_ID>111111</COMMODITY_ID><SESSION_ID>asdfghjklfr0123</SESSION_ID></REQ></COM>", 
  "files": {}, 
  "form": {}, 
  "headers": {
    "Accept": "*/*", 
    "Accept-Encoding": "gzip, deflate", 
    "Content-Length": "182", 
    "Content-Type": "text/xml", 
    "Host": "httpbin.org", 
    "User-Agent": "python-requests/2.24.0", 
    "X-Amzn-Trace-Id": "Root=1-6211ebd3-2cc90293777649ba01e50b08"
  }, 
  "json": null, 
  "origin": "101.71.37.212", 
  "url": "http://httpbin.org/post"
}


Process finished with exit code 0

summary

Here we only introduce the common request parameter encoding formats of GET request and POST request in HTTP protocol, how to determine the encoding format of request parameters, and what format parameters need to be used when sending requests using requests in python code.

We summarize as follows:

1. GET request. The request parameter encoding format is query string params, requests GET () uses the params parameter when sending a request. The value of params (i.e. request parameter) is in dictionary format

2. The encoding format of the POST request body needs to be determined according to the content type field in the request header

3. content-type: application/x-www-form-urlencoded，requests. When sending a request, post () uses the data parameter, and the value of data (i.e. request parameter) is in dictionary format

4. content-type: multipart/form-data，requests. When sending a request, post () uses the files parameter, and the value of files (i.e. request parameter) is the content of the uploaded file read

5. content-type: application/json，requests. When sending a request, post () uses the JSON parameter. The JSON value (i.e. request parameter) is in dictionary format, or you can also use the data parameter, but at this time, you need to convert the request parameter to JSON format first

6. content-type: text/xml，requests.post() uses the data parameter when sending the request. The value of data (i.e. the request parameter) is the body part of XML

Refer to the sample code for specific scripting methods.