Python Learning Requests Library

Posted by BIOSTALL on Sat, 13 Jul 2019 20:12:38 +0200

The Requests Library in python is an encapsulated python utility that requests http and returns the corresponding results

Initial Requests Library

Requests Environment Preparation

  • Install pip
    https://pypi.python.org/pypi/pip#downloads
    Click on the connection above to download the pip, unzip to the local directory when finished, and enter the unzip directory under the cmd command line to execute the following commands:
python setup.py install

When you execute the pip command from the command line, you will find that the command is still not found.

We need to configure the pip environment variable. This command is in python's installation path, and I'm here "C:\Python27\Scripts", so our pip command will work

  • Install Requests Library

Resolve eclipse's failure to introduce requests

Above, we've installed the requests library, but we found that when eclipse was introduced, there was still an error like this:

Unused import: requests
Unresolved import: requests

The following steps can be taken to solve the problem:

  • Download python request source code https://github.com/kennethreitz/requests
  • Copy the downloaded requests library code to python's installation root directory

  • Go to the directory and install the requests package

  • eclipse configuration

    Our eclipse now correctly identifies the requests Library

Test Server

By now, the requests on the requester side have been configured, so there are many different servers to request. Here the requests author has written a server side himself, which we can use to test whether the requests we write are valid
http://httpbin.org/

urllib is simple to use

In python we have a system library to access the url. Here I use this library to access the link above and print out the current header and body

import urllib2
import urllib

url_ip = 'http://httpbin.org/ip'

def use_urllib2():
    response = urllib2.urlopen(url_ip)
    print '>>>>Response Headers:'
    print response.info()
    print '>>>>Response Body:'
    print ''.join([line for line in response.readlines()])


use_urllib2()   

The effect is as follows:

Adding request parameters using urllib2

import urllib2
import urllib

def use_params_urllib2():
    url_get = "http://httpbin.org/get"
    # Build Request Parameters
    params = urllib.urlencode({'param1':'hello','param2':'world'})
    print '>>>Request params is :'
    print params
    respsonse = urllib2.urlopen('?'.join([url_get,'%s']) % params)

    print '>>>>Response Headers:'
    print respsonse.info()
    print '>>>>Code is :'
    print respsonse.getcode()
    print '>>>>Response Body:'
    print ''.join([line for line in respsonse.readlines()])

use_params_urllib2()

Use requests

Request url using requests

import request

url_ip = 'http://httpbin.org/ip'
def use_requests():
    response = requests.get(url_ip)
    print '>>>>Response Headers:'
    print response.headers
    print '>>>>Response Body:'
    print response.text

use_requests()   

Passing parameters using requests

import requests

url_ip = 'http://httpbin.org/ip'
def use_params_requests():
    params = {'param1':'hello','param2':'world'}
    print '>>>Request params is :'
    print params
    response = requests.get(url_get, params)

    print '>>>>Response Headers:'
    print response.headers
    print '>>>>Code is :'
    print response.status_code
    print '>>>>Response Body:'
    print response.json()

use_params_requests()

Use of the requests Library

Here we use the documentation provided to us by github to learn how to use the requests Library
https://developer.github.com/v3/users/

As you can see, here we use'/users/username'to get the corresponding user information

import requests
import json

url = 'https://api.github.com'

#Build url, add endpoint after URL
def build_uri(endpoint):
    print '/'.join([url,endpoint])
    return '/'.join([url,endpoint])

# Indent the printed json by four spaces
def better_print(json_str):
    return json.dumps(json.loads(json_str), indent=4)

def request_method():
    response = requests.get(build_uri('users/mockingbirds'))
    print better_print(response.text)

if __name__ == '__main__':
    request_method()

The output is as follows:

Requests library requests with parameters

Modifying user information using the patch method

requests.get('url', params={'params1':'hello'})

Modify user parameters using requests pass parameters

def json_request():
    response = requests.patch(build_uri('user'), auth=('mockingbirds','Fill in here github User Password'),json={'company':'tencent'})
    print better_print(response.text)
    print response.request.headers
    print response.request.body
    print response.status_code

if __name__ == '__main__':
    json_request()


You can see that it has been correctly modified at this point

Use the post method to increase mailbox addresses

In addition, we can use the post method to add two mailbox addresses corresponding to the account based on the api github provided to us

def json_post():
    response = requests.post(build_uri('user/emails'), auth=('mockingbirds','github User Password'),json=['hello@github.com','world@github.com'])
    print better_print(response.text)
    print response.request.headers
    print response.request.body
    print response.status_code

if __name__ == '__main__':
    json_post()

Request library request exception handling

def request_except():
    NETWORK_STATUS = True 
    try:
        requests.get('https://developer.github.com', timeout=10)
    except requests.exceptions.ConnectTimeout:
        NETWORK_STATUS = False
    print(NETWORK_STATUS)

if __name__ == '__main__':
    request_except()

requests download file

import requests

url = 'https://img1.doubanio.com/view/photo/photo/public/p2453514758.jpg'

def download_file():
    response = requests.get(url, stream=True)
    from contextlib import closing
    with closing(requests.get(url, stream=True)) as response:
        if response.status_code == 200:
            with open('file.jpg','wb') as fd:
                for fun in response.iter_content(128):
                    fd.write(fun)

if __name__ == '__main__':
    download_file()

requests library http authentication

Previously, we needed to pass in the user's password manually to get and modify the user's email list. This is really unsafe. We can generate a token in github, which is managed by GitHub itself, but we can use this token for authentication-related operations.
https://github.com/settings/tokens/new

def base_oauth():
    headers = {'Authorization':'token f12100a704408ee8bc4fe74a04a612570067adb8'}
    response = requests.get('https://api.github.com/user/emails', headers=headers)
    print response.request.headers
    print response.text
    print response.status_code

if __name__ == '__main__':
    base_oauth()

The effect is as follows:

Instead of manually specifying headers, the requests library provides us with an encapsulated method

from requests.auth import AuthBase

class GithubAuth(AuthBase):

    def __init__(self, token):
        self.token = token

    def __call__(self, r):
        r.headers['Authorization'] = ''.join(['token', self.token])
        return r    

def oauth_impove():
    auth = GithubAuth('f12100a704408ee8bc4fe74a04a612570067adb8');
    response = requests.get('https://api.github.com/user/emails', auth=GithubAuth('f12100a704408ee8bc4fe74a04a612570067adb8'))
    print response.text

if __name__ == '__main__':
    oauth_impove()

ok, that's it in a few days

Topics: github Python JSON pip