1. What are requests?
requests is a Python third-party library that is particularly convenient for handling URL resources
2. Install requests
pip3 install requests
If Permission denied installation fails, please add sudo and try again
3. Use requests
3.1 get request method
3.1.1 basic get request
import requests response = requests.get('http://httpbin.org/get') print(response.text)
3.1.2 get request with parameters
For with parameters, pass in a dict as the params parameter:
import requests
def loginGet():
r=requests.get(
url='http://47.95.142.233:8000/login/auth/',
params={"name":"wuya","age":18})
print('Protocol status code',r.status_code)
#The data type of the dictionary is obtained
print('Returns the data type of the dictionary',r.json())
#Get the requested address information
print('Get information about the requested address',r.url)
#Got a string
print('Gets the data type of the string',r.text)
#What you get is bytest Data type for
print('What you get is bytest Data type for',r.content)
#Get response header
print('Response header',r.headers)
Note: the response data is generally obtained by r.text. If the response data is in JSON 'format, use r.json()
To inquire the home address of the telephone number:
import requests
r=requests.get(
url='http://ws.webxml.com.cn/WebServices/MobileCodeWS.asmx/getMobileCodeInfo',
params={"mobileCode":"15191084297","userId":""})
print('Protocol status code', r.status_code)
print('Get information about the requested address', r.url)
print('Response header', r.headers)
print('Gets the data type of the string',r.text)
3.2 post request method
3.2.1 JSON data format:
import json
def loginPost():
r = requests.post(
url='http://47.95.142.233:8000/login/auth/',
json={"username":"15191084297","password":"123456"},
headers={'content-type': 'application/json'})
print(json.dumps(r.json(),ensure_ascii=False,indent=True))
3.2.2 form data format:
r=requests.post(
url='http://ws.webxml.com.cn/WebServices/MobileCodeWS.asmx/getMobileCodeInfo',
data={"mobileCode":"15191084297","userId":""},
headers={'content-type':'application/x-www-form-urlencoded'})
print('Gets the data type of the string',r.text)
Difference between json and data:
When to use json
When the request parameter is JSON, the JSON parameter is used. When the request parameter is JSON, but the data parameter is used, the request parameter needs to be serialized
When to use data
When the request parameter is a form, use data. When the request parameter is in JSON format, the request parameter should be serialized
Pull hook net actual combat:
r = requests.post(
url='https://www.lagou.com/jobs/v2/positionAjax.json',
data={
'data': 'Iphu%2FZ6ba7xjPqUDKO03NaTge36Zu7CANc5xZfNf0Brv4y1pJEeEuKl1PTDANiyGvMDYzLyoZqP7pPX3T9vFbzaVfuiXlpLScGwoDEJ6YlgCeyA38q5TJ%2FeCp2OmMywUQalg1pLh5vpFJIazFh7A6ovCMqZFGFB2DmsZXn22k4LdQJhEnS8Y4rmCv6a15mFX'},
headers={'content-type': 'application/x-www-form-urlencoded',
'Cookie': 'RECOMMEND_TIP=true; PRE_UTM=; PRE_LAND=https%3A%2F%2Fwww.lagou.com%2F; user_trace_token=20220119144853-8c70d9f2-7f4f-4097-ab6f-77a57190e796; LGUID=20220119144853-fa5d8ce2-6aa5-4fd0-9353-a266309a766c; privacyPolicyPopup=false; _ga=GA1.2.673573823.1642574934; _gid=GA1.2.1607293937.1642574935; LGSID=20220119144854-49e4e6e0-cb73-4e52-b471-58cefe6e79dc; PRE_HOST=cn.bing.com; PRE_SITE=https%3A%2F%2Fcn.bing.com%2F; sajssdk_2015_cross_new_user=1; WEBTJ-ID=20220119145841-17e712165122f0-02258ae20ef9a5-5e181655-1327104-17e71216513558; JSESSIONID=ABAAABAABEIABCI41426992A613A6F1F2FAAD0189C97FFD; sensorsdata2015session=%7B%7D; Hm_lvt_4233e74dff0ae5bd0a3d81c6ccf756e6=1642574935,1642575522; index_location_city=%E5%85%A8%E5%9B%BD; X_HTTP_TOKEN=68221908d10068cd4255752461329f451998b7de45; Hm_lpvt_4233e74dff0ae5bd0a3d81c6ccf756e6=1642575525; TG-TRACK-CODE=index_search; LGRID=20220119145857-272d10f0-d02a-451c-b154-f0074a3fb737; sensorsdata2015jssdkcross=%7B%22distinct_id%22%3A%2217e711873e74c8-0e9dec4a6c3c5a-5e181655-1327104-17e711873e818b%22%2C%22first_id%22%3A%22%22%2C%22props%22%3A%7B%22%24latest_traffic_source_type%22%3A%22%E8%87%AA%E7%84%B6%E6%90%9C%E7%B4%A2%E6%B5%81%E9%87%8F%22%2C%22%24latest_search_keyword%22%3A%22%E6%9C%AA%E5%8F%96%E5%88%B0%E5%80%BC%22%2C%22%24latest_referrer%22%3A%22https%3A%2F%2Fcn.bing.com%2F%22%2C%22%24os%22%3A%22Windows%22%2C%22%24browser%22%3A%22Chrome%22%2C%22%24browser_version%22%3A%2297.0.4692.71%22%7D%2C%22%24device_id%22%3A%2217e711873e74c8-0e9dec4a6c3c5a-5e181655-1327104-17e711873e818b%22%7D',
'User-Agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.71 Safari/537.36 Edg/97.0.1072.62',
'referer':'https://www.lagou.com/wn/jobs?labelWords=&fromSearch=true&suginput=&kd=%25E6%25B5%258B%25E8%25AF%2595%25E5%25BC%2580%25E5%258F%2591%25E5%25B7%25A5%25E7%25A8%258B%25E5%25B8%2588'})
print('Protocol status code:',r.status_code)
print('Return protocol data:',r.text)
print(json.dumps(r.json(),indent=True,ensure_ascii=False))
4. Detailed explanation of requests parameters
headers
Fill in the information of the request header. The information of the request header is the data type of the dictionary
cookies
In the request header, you need to bring the value of the set cookie returned from the response header of the server
51cto actual combat:
def login(): r=requests.post( url='https://home.51cto.com/index?reback=https%3A%2F%2Fhome.51cto.com&iframe=0&is_go_to_user_set_mobile=1', data={"_csrf":"ZXk0VTBJaXENEEE5eCsjN1U9ZGZ0LwI7LBRjAQM7Dh1IDAAeagoZRw", "LoginForm[username]":"15191084297", "LoginForm[password]":"cy980716", "show_qr":"0"}, headers={'content-type':'application/x-www-form-urlencoded', 'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36', 'Referer': '//home.51cto.com/index', 'Cookie': 'www51cto=02B0203B122F2012E0EC7BAC6020A4C2pHub; _csrf=6a44440f3396ae1244c0004595444c2bc471690ec46ad7c1fa772eb23bb216daa%3A2%3A%7Bi%3A0%3Bs%3A5%3A%22_csrf%22%3Bi%3A1%3Bs%3A32%3A%22hiulHbJF0DP3DfkJImWT3rgl-u4KZCp6%22%3B%7D; _ourplusFirstTime=122-1-13-15-38-6; sajssdk_2015_cross_new_user=1; sensorsdata2015jssdkcross=%7B%22distinct_id%22%3A%2217e525f5627a34-0113622e3482c8-36657407-1296000-17e525f5628112a%22%2C%22first_id%22%3A%22%22%2C%22props%22%3A%7B%22%24latest_traffic_source_type%22%3A%22%E7%9B%B4%E6%8E%A5%E6%B5%81%E9%87%8F%22%2C%22%24latest_search_keyword%22%3A%22%E6%9C%AA%E5%8F%96%E5%88%B0%E5%80%BC_%E7%9B%B4%E6%8E%A5%E6%89%93%E5%BC%80%22%2C%22%24latest_referrer%22%3A%22%22%7D%2C%22%24device_id%22%3A%2217e525f5627a34-0113622e3482c8-36657407-1296000-17e525f5628112a%22%7D; Hm_lvt_110fc9b2e1cae4d110b7959ee4f27e3b=1642059408,1642084977; pub_cookietime=2592000; lastlogin=on; Hm_lpvt_110fc9b2e1cae4d110b7959ee4f27e3b=1642085620; Hm_lvt_844390da7774b6a92b34d40f8e16f5ac=1642085057,1642085464,1642085663,1642086193; callback_api_url=https%3A%2F%2Fhome.51cto.com%2Findex; pub_wechatopen=DgZDExYRBjVaYERiLmkgAWt0GV0kZWIHGhIBMTwLMWwlN0oVAxAPBVlec2I2aSVMR1ZESR9RfHkzalAMalZTB1ZdDQdTAwdVVz0DB1cLVgwCBQ1X; PHPSESSID=8gfk7kgeg4jjo1a6vkfk2vcc22; once_p=ce2eb9; _ourplusReturnCount=18; _ourplusReturnTime=122-1-13-23-12-4; login_from=home.51cto.com; reg_from=home.51cto.com; Hm_lpvt_844390da7774b6a92b34d40f8e16f5ac=1642086725'}) print(r.status_code) print(r.text) print(r.cookies) # r.cookies A simple understanding is to get the information in the response header set-cookie The pointer returned to the client, that is sessionID return r.cookies def profile(): r=requests.get( url='https://edu.51cto.com/center/user/info/get-user-info', # Through the request header cookie Bring back sessionID Send to the server for authentication cookies=login()) print(r.status_code) print(r.text)
files
Parameters for file upload
auth
Permission verification
import requests from requests.auth import HTTPBasicAuth api=requests.get( url='http://101.43.158.84:5000/v1/api/books', auth=HTTPBasicAuth(username='admin',password='admin'), timeout=180) print(api.status_code) print(json.dumps(api.json(),ensure_ascii=False,indent=True))
timeout
It mainly refers to that the response data of the server is very slow, but sometimes very fast, so it is recommended to add tiaout at this time, which is equivalent to implicit waiting. Another is when requests appear in network requests exceptions. When the connecttimeout is wrong, it means that this parameter needs to be used
import requests from requests.auth import HTTPBasicAuth api=requests.get( url='http://101.43.158.84:5000/v1/api/books', auth=HTTPBasicAuth(username='admin',password='admin'), timeout=180) print(api.status_code) print(json.dumps(api.json(),ensure_ascii=False,indent=True))
verify
When your code has no problem, but the request is the problem of returning ssl and 403, it means that you need to set the parameters of verify, such as the return error: requests exceptions. SSLError. Then you need to bring this parameter
import requests from requests.auth import HTTPBasicAuth api=requests.get( url='http://101.43.158.84:5000/v1/api/books', auth=HTTPBasicAuth(username='admin',password='admin'), timeout=180,verify=False) print(api.status_code) print(json.dumps(api.json(),ensure_ascii=False,indent=True))
5. Solution of dynamic parameters in requests
The first way to write:
imoport requests import json def login(): dict1={"username":"15191084297","password":"123456"} r=requests.post( url='http://47.95.142.233:8000/login/auth/', data=json.dumps(dict1), headers={'content-type':'application/json'}) return r.json()['token'] def index(): r=requests.get( url='http://47.95.142.233:8000/interface/index', headers={"Authorization":"JWT {0}".format(login())}) print(r.text) assert r.json()['count']['api']==4
The second way to write:
imoport requests
import json
def login(): dict1={"username":"15191084297","password":"123456"} r=requests.post( url='http://47.95.142.233:8000/login/auth/', data=json.dumps(dict1), headers={'content-type':'application/json'}) return r.json()['token'] def index(): r=requests.get( url='http://47.95.142.233:8000/interface/index', headers={"Authorization":"JWT {token}".format(token=login())}) print(r.text) assert r.json()['count']['api']==4