Understanding IP Security (IPSec) Basics
Understanding IP Security (IPSec) Basics
Importance of IPSec
IPSec (Internet Protocol Security) is important because it helps keep your data safe and secure
when you send it over the Internet or any network. Here are some of the important aspects why
IPSec is Important:
• IPSec is often used in Virtual Private Networks (VPNs) to create secure, private
connections.
Features of IPSec
• Integrity: IPSec provides integrity by ensuring that IP packets have not been modified or
corrupted during transmission.
• Key management: IPSec provides key management services, including key exchange
and key revocation, to ensure that cryptographic keys are securely managed.
• Flexibility: IPSec can be configured to provide security for a wide range of network
topologies, including point-to-point, site-to-site, and remote access connections.
IPSec (Internet Protocol Security) is used to secure data when it travels over the Internet. IPSec
works by creating secure connections between devices, making sure that the information
exchanged is kept safe from unauthorized access. IPSec majorly operates in two ways
i.e. Transport Mode and Tunnel Mode.
To provide security, IPSec uses two main protocols: AH (Authentication Header) and ESP
(Encapsulating Security Payload). Both protocols are very useful as Authentication
Header verifies the data that whether it comes from a trusted source and hasn’t been changed,
and ESP has the work of performing authentication and also encrypts the data so that it becomes
difficult to read.
For Encryption, IPSec uses cryptographic keys. It can be created and shared using a process
called IKE (Internet Key Exchange), that ensures that both devices have the correct keys to
establish a secure connection.
When two devices communicate using IPSec, the devices first initiate the connection by sending
a request to each other. After that, they mutually decide on protection of data
using passwords or digital certificates. Now, they establish the secure tunnel for
communication. Once the tunnel is set up, data can be transmitted safely, as IPSec is encrypting
the data and also checking the integrity of the data to ensure that data has not been altered. After
the communication is finished, the devices can close the secure connection. In this way, the
IPSec works.
IPSec is a protocol suite used in securing communication using the Internet Protocol such that
each packet communicated in the course of a particular session is authenticated and encrypted.
The process of establishing an IPSec connection involves two main phases:
In phase 1, the main aim is to establish the secure channel the IKE tunnel, which is used to further
negotiations. Phase 1 can operate in one of two modes:
• Main Mode: Main Mode is a six-message exchange procedure that is more secure than
Basic Mode, although at the cost of a longer session, since identity information is
transmitted during negotiations.
• Aggressive Mode: Aggressive Mode takes lesser time with the exchange of three
messages and is less secure since more information like identity is disclosed during the
course of negotiation.
Phase 2 is called Quick Mode and its aim is to negotiate the IPSec Security Associations after the
construction of a secure IKE tunnel has been made. There are two modes in Phase 2.
• Tunnel Mode: This mode encapsulates the whole of the original IP packet including the
header and data. It is mostly deployed in the site to site VPNs.
• Transport Mode: By this mode, only the actual data to be transmitted is encrypted and
the header part of the IP packets remain unaltered. It is mainly employed in end to end
communication between hosts.
2. Authentication Header (AH): It also provides data integrity, authentication, and anti-replay
and it does not provide encryption. The anti-replay protection protects against the unauthorized
transmission of packets. It does not protect data confidentiality.
IP Header
IPSec Encryption
IPSec encryption is a software function that encrypts data to protect it from unauthorized access.
An encryption key encrypts data, which must be decrypted. IPSec supports a variety of
encryption algorithms, including AES, Triple DES etc. IPSec combines asymmetric
and symmetric encryption to provide both speed and security during data transmission.
In asymmetric encryption, the encryption key is made public, while the decryption key remains
private. Symmetric encryption employs the same public key to encrypt and decrypts data. IPSec
builds a secure connection using asymmetric encryption and then switches to symmetric
encryption to speed up data transmission.
IPSec VPN
VPN(Virtual Private Network) is a networking software that enables users to browse the internet
anonymously and securely. An IPSec VPN is a type of VPN software that uses the IPSec protocol
to establish encrypted tunnels over the internet. It offers end-to-end encryption, which means
that data is broken down at the computer and then collected at the receiving server.
IPSec Architecture
IPSec (IP Security) architecture uses two protocols to secure the traffic or data flow. These
protocols are ESP (Encapsulation Security Payload) and AH (Authentication Header). IPSec
Architecture includes protocols, algorithms, DOI, and Key Management. All these components
are very important in order to provide the three main services:
• Confidentiality
• Authentication
• Integrity
IP Security Architecture:
• Sequence Number: Unique Sequence numbers are allotted to every packet so that on
the receiver side packets can be arranged properly.
• Payload Data: Payload data means the actual data or the actual message. The Payload
data is in an encrypted format to achieve confidentiality.
• Padding: Extra bits of space are added to the original message in order to ensure
confidentiality. Padding length is the size of the added bits of space in the original
message.
• Next Header: Next header means the next payload or next actual data.
3. Encryption algorithm: The encryption algorithm is the document that describes various
encryption algorithms used for Encapsulation Security Payload.
Authentication Header covers the packet format and general issues related to the use of AH for
packet authentication and integrity.
5. Authentication Algorithm: The authentication Algorithm contains the set of documents that
describe the authentication algorithm used for AH and for the authentication option of ESP.
6. DOI (Domain of Interpretation): DOI is the identifier that supports both AH and ESP protocols.
It contains values needed for documentation related to each other.
7. Key Management: Key Management contains the document that describes how the keys are
exchanged between sender and receiver.
Secure Sockets Layer (SSL) is an Internet security protocol that encrypts data to ensure secure
communication between devices over a network. Originally developed by Netscape in 1995, SSL
provides privacy, authentication and data integrity for online communications. SSL is the
predecessor of TLS (Transport Layer Security), which is now the standard protocol for secure
communications on the Internet.
Working of SSL
2. Authentication: SSL uses a handshake process to authenticate both the client and
server, ensuring each party is legitimate and not an imposter.
3. Data Integrity: SSL digitally signs transmitted data to detect any tampering, ensuring that
the data received is exactly what was sent.
Importance of SSL
Before SSL, web data was transmitted in plaintext, making it vulnerable to interception. SSL
solves this by:
SSL Protocols
SSL consists of several protocols, each handling a different aspect of secure communication
• Application data is divided into fragments, optionally compressed and appended with a
Message Authentication Code (MAC).
2. Handshake Protocol
Four phases:
• Client and server exchange hello packets, protocol versions and cipher suites.
• Signals that pending cryptographic parameters from the handshake should now become
active.
• Fatal alerts (level 2): Critical errors, such as handshake failures, bad record MAC or
illegal parameters, which terminate the connection.
Secure Electronic Transaction or SET is a security protocol designed to ensure the security and
integrity of electronic transactions conducted using credit cards. Unlike a payment system, SET
operates as a security protocol applied to those payments. It uses different encryption and
hashing techniques to secure payments over the internet done through credit cards. The SET
protocol was supported in development by major organizations like Visa, Mastercard, and
Microsoft which provided its Secure Transaction Technology (STT), and Netscape which provided
the technology of Secure Socket Layer (SSL).
SET protocol restricts the revealing of credit card details to merchants thus keeping hackers and
thieves at bay. The SET protocol includes Certification Authorities for making use of standard
Digital Certificates like X.509 Certificate.
Requirements in SET: The SET protocol has some requirements to meet, some of the important
requirements are:
• SET also needs to provide interoperability and make use of the best security mechanisms.
Participants in SET: In the general scenario of online transactions, SET includes similar
participants:
1. Cardholder - customer
2. Issuer - customer financial institution
3. Merchant
SET functionalities:
• Provide Authentication
• Provide Message Integrity: SET doesn't allow message modification with the help of
signatures. Messages are protected against unauthorized modification using RSA digital
signatures with SHA-1 and some using HMAC with SHA-1,
Dual Signature: The dual signature is a concept introduced with SET, which aims at connecting
two information pieces meant for two different receivers :
You might think sending them separately is an easy and more secure way, but sending them in a
connected form resolves any future dispute possible. Here is the generation of dual signature:
Purchase Request Generation: The process of purchase request generation requires three
inputs:
• Dual Signature
Since we used Customer's private key in encryption here we use KUC which is the public key of
the customer or cardholder for decryption 'D'.
Payment Authorization and Payment Capture: Payment authorization as the name suggests is
the authorization of payment information by the merchant which ensures payment will be
received by the merchant. Payment capture is the process by which a merchant receives
payment which includes again generating some request blocks to gateway and payment gateway
in turn issues payment to the merchant.
Intrusion is when an attacker gets unauthorized access to a device, network, or system. Cyber
criminals use advanced techniques to sneak into organizations without being detected.
Intrusion Detection System (IDS) observes network traffic for malicious transactions and sends
immediate alerts when it is observed. It is software that checks a network or system for malicious
activities or policy violations. Each illegal activity or violation is often recorded either centrally
using an SIEM system or notified to an administration. IDS monitors a network or system for
malicious activity and protects a computer network from unauthorized access from users,
including perhaps insiders. The intrusion detector learning task is to build a predictive model (i.e.
a classifier) capable of distinguishing between ‘bad connections’ (intrusion/attacks) and ‘good
(normal) connections’.
• Address Spoofing: Hiding the source of an attack by using fake or unsecured proxy
servers making it hard to identify the attacker.
• Coordinated Attack: Using multiple attackers or ports to scan a network, confusing the
IDS and making it hard to see what is happening.
• An IDS (Intrusion Detection System) monitors the traffic on a computer network to detect
any suspicious activity.
• It analyzes the data flowing through the network to look for patterns and signs of abnormal
behavior.
• The IDS compares the network activity to a set of predefined rules and patterns to identify
any activity that might indicate an attack or intrusion.
• If the IDS detects something that matches one of these rules or patterns, it sends an alert
to the system administrator.
• The system administrator can then investigate the alert and take action to prevent any
damage or further intrusion.
• Network Intrusion Detection System (NIDS): Network intrusion detection systems (NIDS)
are set up at a planned point within the network to examine traffic from all devices on the
network. It performs an observation of passing traffic on the entire subnet and matches the
traffic that is passed on the subnets to the collection of known attacks. Once an attack is
identified or abnormal behavior is observed, the alert can be sent to the administrator. An
example of a NIDS is installing it on the subnet where firewalls are located in order to see if
someone is trying to crack the firewall.
• Host Intrusion Detection System (HIDS): Host intrusion detection systems (HIDS) run on
independent hosts or devices on the network. A HIDS monitors the incoming and outgoing
packets from the device only and will alert the administrator if suspicious or malicious activity
is detected. It takes a snapshot of existing system files and compares it with the previous
snapshot. If the analytical system files were edited or deleted, an alert is sent to the
administrator to investigate. An example of HIDS usage can be seen on mission-critical
machines, which are not expected to change their layout.
• Hybrid Intrusion Detection System: Hybrid intrusion detection system is made by the
combination of two or more approaches to the intrusion detection system. In the hybrid
intrusion detection system, the host agent or system data is combined with network
information to develop a complete view of the network system. The hybrid intrusion detection
system is more effective in comparison to the other intrusion detection system. Prelude is an
example of Hybrid IDS.
• Fragmentation: Dividing the packet into smaller packet called fragment and the process
is known as fragmentation. This makes it impossible to identify an intrusion because
there can't be a malware signature.
• Packet Encoding: Encoding packets using methods like Base64 or hexadecimal can hide
malicious content from signature-based IDS.
• Encryption: Several security features such as data integrity, confidentiality, and data
privacy, are provided by encryption. Unfortunately, security features are used by malware
developers to hide attacks and avoid detection.
• Signature-Based Method: Signature-based IDS detects the attacks on the basis of the
specific patterns such as the number of bytes or a number of 1s or the number of 0s in
the network traffic. It also detects on the basis of the already known malicious instruction
sequence that is used by the malware. The detected patterns in the IDS are known as
signatures. Signature-based IDS can easily detect the attacks whose pattern (signature)
already exists in the system but it is quite difficult to detect new malware attacks as their
pattern (signature) is not known.
An Intrusion Prevention System is also known as an Intrusion Detection and Prevention System.
It is a network security application that monitors network or system activities for malicious
activity. The major functions of intrusion prevention systems are to identify malicious activity,
collect information about it, report it, and attempt to block or stop it.
An IPS typically records information related to observed events, notifies security administrators
of important events, and produces reports. Many IPS can also respond to a detected threat by
attempting to prevent it from succeeding. They use various response techniques, such as
stopping the attack itself, changing the security environment, or altering the attack's content.
An IPS works by analyzing network traffic in real-time and comparing it against known attack
patterns and signatures. When the system detects suspicious traffic, it blocks it from entering the
network.
1. Inline Deployment
An IPS is placed directly in the path of network traffic between your internal network and the
internet or right behind a firewall. This position allows it to inspect every packet that flows through
and immediately block anything suspicious before it causes harm.
2. Traffic Preprocessing
Before deep inspection begins, the IPS organizes the raw data:
• It normalizes traffic formats to prevent attackers from hiding threats using clever
encoding tricks.
The IPS doesn’t just look at surface-level information. This “deep packet inspection” helps the IPS
understand both the technical structure and the intent behind the traffic. It dives into:
• Network layer: Where packets come from and where they're going.
• Application layer: What kind of data is being transmitted — like login info or file transfers.
4. Detection Mechanisms
Once packets are unpacked and understood, the IPS checks them using different techniques:
• Anomaly-Based Detection: Flags anything that deviates from normal behavior — like an
unusual spike in traffic.
• Behavior-Based Detection: Watches for odd or suspicious actions over time, such as
repeated failed logins.
• Terminate sessions
An effective IPS isn't a “set it and forget it” system. It needs regular updates and fine-tuning to:
Types of IPS
1. Network-Based IPS: Installed at the network perimeter, it monitors all traffic entering
and exiting the network.
2. Host-Based IPS: Installed on individual hosts, it monitors traffic flowing in and out of the
host.
An IPS is an essential tool for network security. Here are some reasons why:
• Protection Against Known and Unknown Threats: An IPS can block known threats and also
detect and block unknown threats that haven't been seen before.
• Real-Time Protection: An IPS can detect and block malicious traffic in real-time,
preventing attacks from doing any damage.
• Compliance Requirements: Many industries have regulations that require the use of an
IPS to protect sensitive information and prevent data breaches.
• Increased Network Visibility: An IPS provides increased network visibility, allowing you to
see what's happening on your network and identify potential security risks.
• Network-based intrusion prevention system (NIPS): It monitors the entire network for
suspicious traffic by analyzing protocol activity.
• Network behavior analysis (NBA): It examines network traffic to identify threats that
generate unusual traffic flows, such as distributed denial of service attacks, specific
forms of malware and policy violations.
Information security threats represent actions or events that can compromise the confidentiality,
integrity, or availability of data and systems. These threats originate from various sources,
including individuals, organised groups, or even natural disasters.
Information security programs are built around three core objectives, commonly known as
the CIA (Confidentiality, Integrity, Availability):
• Availability: Guarantees that information and systems remain accessible when needed.
This requires coordination between network operations, development teams, incident
response, and policy management to prevent service disruptions.
Malware Threats
Malware is malicious software made to damage or take over your computer. It can spread fast
and cause big problems.
• Viruses are self-replicating programs that attach themselves to other programs or files
on host computers. They spread through infected media like songs, videos, or software
downloads. Notable examples include file viruses, macro viruses, boot sector viruses,
and stealth viruses.
• Worms share self-replicating capabilities with viruses but operate independently without
attaching to host programs. Their key distinction is network awareness—they can travel
between connected computers, typically consuming system resources and slowing
performance rather than causing direct damage.
• Adware may not be explicitly malicious, but it violates user privacy by displaying
advertisements on desktops or within programs. These programs often accompany free
software and monitor user interests to display targeted ads. Attackers can embed
malicious code within adware to monitor system activities and potentially compromise
machines.
• Spyware consists of programs that monitor computer activities and transmit collected
information to interested parties. These programs are typically distributed through
trojans, viruses, or worms. Keyloggers represent a common spyware example, recording
user keystrokes with timestamps to capture sensitive information like usernames,
passwords, and credit card details.
• Zombies operate similarly to spyware but focus on awaiting commands from hackers
rather than stealing information. These infected systems can be remotely controlled to
participate in coordinated attacks.
Some attacks trick or scare you into paying money or giving access. Protecting yourself means
staying alert and using good security tools.
• Rootkits are designed to gain administrative privileges within user systems. Once root
access is obtained, attackers can steal private files, access sensitive data, or maintain
persistent system control.
• Identity and Access Management (IAM): Manages user access to systems and data. IAM
solutions include single sign-on (SSO) and multi-factor authentication (MFA).
• Security Information and Event Management (SIEM): Security Information and Event
Management (SIEM) Collects and analyzes security-related data to detect and respond
to threats.
• Physical Security: Protects physical assets (e.g., servers, data centers) through access
controls, surveillance, and alarms.
A firewall is a network security system, available as hardware or software, that monitors and
controls incoming and outgoing traffic based on predefined rules. It acts like a security guard,
filtering data packets to either:
Importance of Firewalls
A firewall is the first line of defense in cybersecurity, acting as a security barrier between internal
systems and external networks. It forces all traffic through a single checkpoint, where data
packets are monitored, filtered, and either allowed or blocked based on predefined rules.
Firewalls are essential because they:
• Prevent Unauthorized Access: Like a locked door with a guard, only trusted users and
traffic are allowed through.
• Block Malicious Traffic: Harmful data such as viruses, phishing attempts, or denial-of-
service (DoS) attacks are stopped before reaching the system.
• Protect Sensitive Information: Safeguards personal and business data from theft or
accidental leaks.
Working of Firewall
A firewall inspects all incoming and outgoing traffic and decide whether to allow or block it.
1. All data packets entering or leaving the network must first pass through the firewall.
2. The firewall examines each packet against predefined security rules set by the
organization.
4. Blocked or unusual traffic is recorded in logs, and real-time alerts may be generated for
serious threats.
5. Since it is not possible to define every rule, the firewall applies a default policy (accept,
reject, or drop). Setting the default policy to drop or reject is considered best practice to
prevent unauthorized access.
Default Policy: A firewall needs a default action (accept, reject, or drop) for traffic not covered by
rules. For example, if no rule exists for SSH, the default applies. To prevent unauthorized access,
it is best set to drop or reject.
1) Network Placement
Network Security is the process of protecting networks, systems, and data from unauthorized
access, attacks, and damage.
• Circuit-Level Gateway
Validates if a connection is successfully established (like a TCP handshake), but doesn't
inspect actual data. Like confirming someone is calling from a known number without
listening to the call.
2) Network Firewall
A Network Firewall secures the perimeter, while a host-based firewall protects the endpoint.
• Network Firewall
Protects a whole network—usually placed at the entry/exit point between your internal
systems and the internet. Picture it like a guard standing at your building's main
entrance.
• Host-Based Firewall
Installed on individual devices like laptops, servers, or mobile phones. It protects only
that one system. Think of it as having a security app that watches over just your phone,
not the whole office.
All three of these firewall types work to control network access, but they differ in their placement
and scope.
• Perimeter Firewall
Sits at the edge of your network, filtering traffic coming in and out from the internet. Like
a fence with a gate that controls who gets into your property.
• Internal Firewall
Placed between different segments inside your network, such as departments or
sensitive zones. Imagine every department in a company having a door lock with access
rules.
• Distributed Firewall
Instead of one firewall at the edge, security rules are applied at multiple endpoints
across the network. Like installing security alarms in every room of your house rather
than just at the main door.
4) Form Factors
• Hardware Firewall
A physical box or appliance that connects to your network. Often used in large or office
environments. Think of it like a security gate at the main entrance—visible, strong, and
standalone.
• Software Firewall
Installed as a program on a device or server. Easier to set up and ideal for individuals or
virtual setups. Like installing a firewall app on your laptop to control its own internet
access.
Working of Firewalls
Firewalls can control and monitor the amount of incoming or outgoing traffic of our network. The
data that comes into our network is in the form of packets (small units of data). It is difficult to
identify whether a packet is safe, giving hackers and intruders opportunities to flood networks
with viruses, malware, spam, and more.