Python requests Module

Python requests is a commonly used HTTP request library that can conveniently send HTTP requests to websites and obtain response results.

The requests module isurllibmore concise than the module.

To send HTTP requests using requests, you need to import the requests module first:

import requests

After importing, you can send HTTP requests. Use the methods provided by requests to send an HTTP request to the specified URL, for example:

Example

# Import the requests package
import requests

# Send request
x = requests.get('https://www.example.com/')

# Return webpage content
print(x.text)

After each requests call, a response object is returned. This object contains specific response information, such as status code, response headers, response content, etc.:

print(response.status_code)  # 获取响应状态码
print(response.headers)  # 获取响应头
print(response.content)  # 获取响应内容

More response information is as follows:

Attribute or Method Description
apparent_encoding Encoding
close() Closes the connection to the server
content Returns the response content in bytes
cookies Returns a CookieJar object containing cookies sent from the server
elapsed Returns a timedelta object containing the amount of time elapsed from sending the request to receiving the response, which can be used to test response speed. For example, r.elapsed.microseconds indicates how many microseconds it takes for the response to arrive.
encoding The encoding used to decode r.text
headers Returns the response headers in dictionary format
history Returns a list of response objects containing the request history (url)
is_permanent_redirect Returns True if the response is a permanently redirected URL, otherwise returns False
is_redirect Returns True if the response has been redirected, otherwise returns False
iter_content() Iterates over the response
iter_lines() Iterates over the lines of the response
json() Returns a JSON object of the result (the result must be written in JSON format, otherwise an error will be raised)
links Returns the parsed header links of the response
next Returns the PreparedRequest object for the next request in the redirect chain
ok Checks the value of "status_code", returns True if it is less than 400, returns False if it is not less than 400
raise_for_status() If an error occurs, the method returns an HTTPError object
reason Description of the response status, such as "Not Found" or "OK"
request Returns the request object that requested this response
status_code Returns the HTTP status code, such as 404 and 200 (200 is OK, 404 is Not Found)
text Returns the response content as Unicode type data
url Returns the URL of the response

Example

# Import the requests package
import requests

# Send request
x = requests.get('https://www.example.com/')

# Return HTTP status code
print(x.status_code)

# Description of the response status
print(x.reason)

# Return encoding
print(x.apparent_encoding)

The output result is as follows:

200
OK
utf-8

Request a JSON data file and return JSON content:

Example

# Import the requests package
import requests

# Send request
x = requests.get('https://www.example.com/try/ajax/json_demo.json')

# Return JSON data
print(x.json())

The output result is as follows:

{'name': '网站', 'num': 3, 'sites': [{'name': 'Google', 'info': ['Android', 'Google 搜索', 'Google 翻译']}, {'name': 'Example', 'info': ['Example', 'Example工具', 'Example微信']}, {'name': 'Taobao', 'info': ['淘宝', '网购']}]}

requests Methods

The requests methods are as follows:

Method Description
delete(url, args) Sends a DELETE request to the specified URL
get(url, params, args) Sends a GET request to the specified URL
head(url, args) Sends a HEAD request to the specified URL
patch(url, data, args) Sends a PATCH request to the specified URL
post(url, data, json, args) Sends a POST request to the specified URL
put(url, data, args) Sends a PUT request to the specified URL
request(method, url, args) Sends the specified request method to the specified URL

Send a GET request using requests.request():

Example

# Import the requests package
import requests

# Send request
x = requests.request('get', 'https://www.example.com/')

# Return webpage content
print(x.status_code)

The output result is as follows:

200

Set request headers:

Example

# Import the requests package
import requests

 
kw = {'s':'python tutorial'}

# Set request headers
headers = {"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.99 Safari/537.36"}
 
# params accepts query parameters as a dictionary or string. Dictionary types are automatically converted to URL encoding, no need for urlencode()
response = requests.get("https://www.example.com/", params = kw, headers = headers)

# View response status code
print (response.status_code)

# View response header character encoding
print (response.encoding)

# View the complete URL address
print (response.url)

# View response content, response.text returns data in Unicode format
print(response.text)

The output result is as follows:

200
UTF-8
https://www.example.com/?s=python+%E6%95%99%E7%A8%8B

... 其他内容...

The post() method can send a POST request to the specified URL. The general format is as follows:

requests.post(url, data={key: value}, json={key: value}, args)

  • urlThe URL to request.

  • dataThe data parameter is a dictionary, tuple list, bytes, or file object to send to the specified URL.

  • jsonThe json parameter is a JSON object to send to the specified URL.

  • argsOther parameters, such as cookies, headers, verify, etc.

Example

# Import the requests package
import requests

# Send request
x = requests.post('https://www.example.com/try/ajax/demo_post.php')

# Return webpage content
print(x.text)

The output result is as follows:

<p style='color:red;'>本内容是使用 POST 方法请求的。</p><p style='color:red;'>请求时间：
2022-05-26 17:30:47</p>

POST request with parameters:

Example

# Import the requests package
import requests

# Form parameters, parameter names are fname and lname
myobj = {'fname': 'EXAMPLE','lname': 'Boy'}

# Send request
x = requests.post('https://www.example.com/try/ajax/demo_post2.php', data = myobj)

# Return webpage content
print(x.text)

The output result is as follows:

<p style='color:red;'>你好，EXAMPLE Boy，今天过得怎么样？</p>

Additional Request Parameters

When sending a request, we can attach additional parameters to the request, such as request headers, query parameters, request body, etc., for example:

headers = {'User-Agent': 'Mozilla/5.0'}  # 设置请求头
params = {'key1': 'value1', 'key2': 'value2'}  # 设置查询参数
data = {'username': 'example', 'password': '123456'}  # 设置请求体
response = requests.post('https://www.example.com', headers=headers, params=params, data=data)

The above code sends a POST request with additional request headers, query parameters, and request body.

In addition to basic GET and POST requests, requests also supports other HTTP methods, such as PUT, DELETE, HEAD, OPTIONS, etc.

Other Extensions