Python Requests
Python Requests Library is a simple and powerful tool to send HTTP requests and interact with web
resources. It allows you to easily send GET, POST, PUT, DELETE, PATCH, HEAD requests to web servers,
handle responses, and work with REST APIs and web scraping tasks.
Why do we need Requests Library
Simplifies HTTP Requests: Makes sending GET, POST, PUT, DELETE, etc., as easy as calling a
function.
Manages Sessions & Auth: Handles cookies, sessions, headers, and various authentication
methods.
Ideal for REST APIs: Perfect for consuming or testing RESTful services.
Supports All HTTP Methods: Full support for common and custom HTTP verbs.
Built-in SSL & Error Handling: Automatically manages SSL verification and error handling.
Installation
To install requests library via pip, use the following command:
pip install requests
Syntax
[Link](url, params={key: value}, **kwargs)
Parameter:
url: The URL you want to send the request to. (Required)
params: Dictionary or bytes to be sent in the query string for GET requests. (Optional)
**kwargs: Additional optional arguments such as headers, cookies, authentication, timeout,
proxies, verify (SSL), stream, etc.
Return Type: It returns a response object.
Making a Simple GET Request
Let's try making a get request to URL: "[Link]
import requests
response = [Link]("[Link]
print(response.status_code)
Output
200
Explanation:
[Link]() sends a GET request to the specified URL.
response.status_code returns the HTTP status code (200 means success).
Sending GET Requests with Parameters
Let's demonstrate how to make a GET request to an endpoint. The GET method sends encoded user
information appended to the page request.
Example: Let's try making a request to github's APIs for example purposes.
import requests
response = [Link]("[Link]
print(response.status_code)
print([Link])
Output
Output of Sending GET request with parameter
HTTP Request Methods
Method Description
GET Retrieve information from the server
Send data to the server to create/update
POST
resources
PUT Replace the target resource with new data.
The DELETE method deletes the specified
DELETE
resource
HEAD Retrieve headers only (no body)
PATCH Apply partial modifications to a resource
Response object
Whenever you send a request, the server returns a Response object. It contains useful information like
status code, headers, content, and more. Response object can be used to imply lots of features,
methods, and functionalities.
Example:
import requests
response = [Link]('[Link]
print([Link])
print(response.status_code)
Output
Output of Response object
Explanation:
[Link] returns the final URL after redirections.
response.status_code shows the HTTP status of the request.
Status code 200 indicates that request was made successfully.
Response Methods
Method Description
[Link] returns a dictionary of
[Link]
response headers.
[Link] returns the encoding used to
[Link]
decode [Link].
[Link] returns a timedelta object
[Link] with the time elapsed from sending the request
to the arrival of the response.
[Link]() closes the connection to the
[Link]()
server.
[Link] returns the content of the
[Link]
response, in bytes.
Method Description
[Link] returns a CookieJar object with
[Link]
the cookies sent back from the server.
[Link] returns a list of response
[Link]
objects holding the history of request (url).
response.is_permanent_redirect returns True if
response.is_permanent_redirect the response is the permanent redirected url,
otherwise False.
response.is_redirect returns True if the response
response.is_redirect
was redirected, otherwise False.
response.iter_content() iterates over the
response.iter_content()
[Link].
[Link]() returns a JSON object of the
[Link]() result (if the result was written in JSON format, if
not it raises an error).
[Link] [Link] returns the URL of the response.
[Link] returns the content of the
[Link]
response, in unicode.
response.status_code returns a number that
response.status_code indicates the status (200 is OK, 404 is Not
Found).
[Link] returns the request object that
[Link]
requested this response.
[Link] [Link] returns a text corresponding to
Method Description
the status code.
response.raise_for_status() returns an HTTPError
response.raise_for_status() object if an error has occurred during the
process.
[Link] returns True if the status code is less
[Link] than 400 and greater than or equal to 200;
otherwise, it returns False
[Link] [Link] returns the header links.
POST Request Example
import requests
payload = {'username': 'test', 'password': 'test123'}
response = [Link]("[Link] data=payload)
print([Link])
Output
Output of POST request
Explanation:
Sends form data to the server.
data=payload sends the data in the request body.
The server echoes back the data for testing.
Authentication using Python Requests
Authentication refers to giving a user permissions to access a particular resource. Since, everyone can't
be allowed to access data from every URL, one would require authentication primarily. To achieve this
authentication, typically one provides authentication data through Authorization header or a custom
header defined by server.
Requests module makes this simple using HTTPBasicAuth.
Example:
import requests
from [Link] import HTTPBasicAuth
response = [Link]('[Link] auth=HTTPBasicAuth('user', 'pass'))
print(response.status_code)
Output
401
Explanation:
Replace 'user' and 'pass' with your credentials (username and password).
Automatically handles HTTP Basic Authentication.
SSL Certificate Verification
Requests verifies SSL certificates for HTTPS requests, just like a web browser. SSL Certificates are small
data files that digitally bind a cryptographic key to an organization's details. Often, an website with a SSL
certificate is termed as secure website. By default, SSL verification is enabled, and Requests will throw a
SSLError if it’s unable to verify the certificate.
Accessing a site with invalid SSL:
Let us try to access a website with an invalid SSL certificate, using Python requests
import requests
response = [Link]('[Link] verify=False)
print(response.status_code)
Output
Output of Accessing a site with valid SSL
Explanation: Setting verify=False disables SSL verification (not recommended for production).
Providing a custom certificate:
The above website doesn't have SSL setup so it raises this error, one can also pass the link to the
certificate for validation via python requests only.
import requests
response = [Link]('[Link] verify='/path/to/certfile')
print(response.status_code)
This would work in case the path provided is correct for SSL certificate for [Link].
Session Objects
Session objects allow you to persist settings across multiple requests, such as headers, cookies, and
connection pooling. So if several requests are being made to the same host, the underlying TCP
connection will be reused, which can result in a significant performance increase. A session object all the
methods as of requests.
Using Session Objects
Let us illustrate use of session objects by setting a cookie to a url and then making a request again to
check if cookie is set.
import requests
# Create a session object
session = [Link]()
# Set a cookie
[Link]('[Link]
# Access the cookie in the next request
response = [Link]('[Link]
print([Link])
Output
Output of Session objects
Explanation:
Session() maintains cookies across requests.
Connection pooling improves performance when making multiple requests to the same server.
For more, visit - Session Objects – Python requests
Error Handling with Requests
If this code doesn't print anything, it means request was successful and no errors occurred during the
process.
import requests
try:
response = [Link]("[Link] timeout=5)
response.raise_for_status()
except [Link] as errh:
print("HTTP Error:", errh)
except [Link] as errc:
print("Connection Error:", errc)
except [Link] as errt:
print("Timeout Error:", errt)
except [Link] as err:
print("Something Else:", err)
Explanation:
Always handle errors like connection issues, timeouts, or bad responses.
raise_for_status() raises an exception for HTTP errors automatically.