Introduction to Proxies
A proxy server acts as an intermediary between your computer and the internet. It allows you to make
requests to the internet on behalf of your computer, hiding the actual origin of the request.
Different Types of Proxies
HTTP Proxies: Primarily used for web browsing, these handle web traffic and are the most common
type of proxies used.
SOCKS Proxies: More versatile than HTTP proxies, they can handle various types of traffic, including
email, FTP, and torrents.
SSL Proxies: Specifically designed to handle HTTPS traffic, offering a secure connection between the
client and the proxy server.
Transparent Proxies: These are mainly used for caching purposes and do not provide anonymity as
they forward the user’s original IP address.
Setting up http_proxy and https_proxy using
Environment Variables
Environment variables such as http_proxy , https_proxy , ftp_proxy , and no_proxy can be configured to
define proxies in a Linux environment. Here’s how you can set up these variables with examples:
Without Username and Password
You can set up HTTP and HTTPS proxies by exporting them as environment variables. Open a terminal and
input the following commands:
bash
export http_proxy=[Link]
export https_proxy=[Link]
Example:
bash
export http_proxy=[Link]
export https_proxy=[Link]
With Username and Password
When authentication is required, include the username and password in the environment variable.
bash
export http_proxy=[Link]
export https_proxy=[Link]
Example:
bash
export http_proxy=[Link]
export https_proxy=[Link]
Create Stratis File System (thin provision) with examples
ALSO READ
(CentOS/RHEL 8)
With Domain, Username, and Password
If you need to include a domain along with the username and password, the format would look like this:
ADVERTISEMENT
bash
export http_proxy=[Link]
export https_proxy=[Link]
Example:
bash
export http_proxy=[Link]
export https_proxy=[Link]
Special character handling in http_proxy and
https_proxy
When setting the http_proxy and https_proxy environment variables in Linux, you might encounter
situations where the username or password includes special characters. Special characters, such as ! , @ , # ,
$ , % , ^ , & , * , ( , ) , { , } , [ , ] , : , ; , " , ' , < , > , ? , / , and \ , need to be handled carefully
because they can be interpreted differently by the shell or as part of the URL.
With more complex and robust handling of special characters in username or password follow How to setup
http or https proxy with special characters in username and password
Special characters should be URL-encoded to ensure that they are interpreted correctly. URL encoding
replaces special characters with a "%" followed by two hexadecimal digits corresponding to the ASCII code of
the character.
Examples of Encoding Special Characters:
@ would become %40
# would become %23
$ would become %24
: would become %3A
/ would become %2F
& would become %26
If either the username or password contains special characters, they need to be URL-encoded.
bash
export http_proxy=[Link]
export https_proxy=[Link]
Example:
ADVERTISEMENT
bash
export http_proxy=[Link]
export https_proxy=[Link]
Here, the password "$&@pass" is URL-encoded as "%24%26%40pass" .
Setting Environment Variables Permanently
The previous examples I shared covers setting http_proxy and https_proxy temporarily i.e. they are non
persistent and will not survive a reboot. Setting environment variables permanently can be crucial to ensure
that your proxy configurations persist across system reboots and terminal sessions.
1. Individual User:
Open .bashrc or .bash_profile in your home directory.
bash
vi ~/.bashrc
Add the export lines at the end of the file.
bash
export http_proxy=[Link]
export https_proxy=[Link]
Save and exit the editor, then source the file to apply the changes immediately.
bash
source ~/.bashrc
2. System-wide:
ALSO READ List All Active SSH connections in Linux [8 Methods]
Open /etc/environment using a text editor.
bash
sudo vi /etc/environment
Add the proxy configurations without the export command.
bash
http_proxy="[Link]
https_proxy="[Link]
Save and exit.
ADVERTISEMENT
Verification
After editing the necessary files and sourcing them, you can use the echo command to verify if the
environment variables are set correctly.
bash
echo $http_proxy
echo $https_proxy
Accessing URLs using Curl/Wget with Proxies
Accessing URLs using curl or wget with proxies involves passing the proxy details as part of the command.
Both curl and wget have options that allow you to specify a proxy to be used for the request.
Using Curl with Proxies
Basic Usage without Authentication:
bash
curl -x [Link] [Link]
With Username and Password Authentication:
bash
curl -U username:password -x [Link] [Link]
If you have Environment Variables Set:
bash
curl -x $http_proxy [Link]
Here, curl uses the http_proxy environment variable that has been set previously.
Using Wget with Proxies
Basic Usage without Authentication:
bash
wget -e use_proxy=yes -e http_proxy=[Link]
[Link]
With Username and Password Authentication:
You can include the username and password in the URL.
ADVERTISEMENT
bash
wget -e use_proxy=yes -e
http_proxy=[Link] [Link]
If you have Environment Variables Set:
wget automatically uses the environment variables. So, you just need to execute a regular wget command,
and it will use the proxy settings from the environment variables.
bash
wget [Link]
Ensure that the http_proxy , https_proxy , and other related environment variables are correctly set.
Bypassing Proxy for Specific Domains/Addresses
(NO_PROXY)
The no_proxy environment variable allows for exceptions to be made in proxy configurations, letting you
bypass the proxy for specific domains or IP addresses. This can be essential for intranet resources or local
development environments that should not be accessed via a proxy.
The no_proxy variable is a comma-separated list of domain names or IP addresses that should bypass
the proxy.
Wildcards can be used for broader matches. For instance, .local will match all hosts in the .local
domain.
It's also possible to specify ports along with domains or IP addresses.
ALSO READ Exclude Users from Match Group in SSHD [SOLVED]
1. Bypassing for Specific Domains
bash
export no_proxy=".[Link],.local,localhost"
This setting bypasses the proxy for all subdomains of [Link] , all domains ending in .local , and
localhost .
2. Bypassing for Specific IP Addresses
bash
export no_proxy="[Link],[Link]"
This configuration bypasses the proxy for traffic going to the specified IP addresses.
3. Bypassing for Localhost and Local Network
ADVERTISEMENT
bash
export no_proxy="localhost,[Link],[Link]/8"
Here, all traffic to localhost , [Link] , and the 10.x.x.x network bypasses the proxy.
4. Bypassing for Specific Ports
bash
export no_proxy="localhost:8080,[Link]"
In this example, traffic to localhost on port 8080 and [Link] on port 9090 will bypass the proxy.
You might want to automate the no_proxy configuration by adding the export command to a startup script
like ~/.bashrc or ~/.bash_profile .
bash
echo 'export no_proxy=".[Link],.local,localhost"' >> ~/.bashrc
source ~/.bashrc
You can verify the no_proxy configuration by echoing the variable:
bash
echo $no_proxy
Summary
In this comprehensive guide, we delved into the multifaceted process of setting up and managing proxies in a
Linux environment, focusing on the utilization of http_proxy and https_proxy environment variables.
Beginning with a foundational understanding, the guide elucidated the definitions, purposes, and various
types of proxies. It meticulously navigated through the steps of configuring proxies, handling special
characters in credentials, and establishing a permanent proxy setup for persistent configurations across
different shells such as bash, zsh, and csh.
The tutorial also explored advanced usage scenarios like accessing URLs via command-line tools, employing
scripts for automated setups, and implementing specific configurations to bypass proxies for designated
domains and addresses. The guide accentuated the customization and flexibility that environment variables
offer in tailoring proxy configurations to meet specific network requirements and preferences. For a deeper
dive and continuous learning, it also provided a reservoir of essential further reading resources, including
official documentation, man pages, and community forums, aiming to foster a well-rounded and up-to-date
understanding of proxy management in Linux.