What Is an IPsec VPN? How It Works, Modes and Uses

An IPsec VPN is a virtual private network that uses IPsec (Internet Protocol Security) to encrypt and authenticate the packets that travel between two gateways, between a device and a gateway, or between two devices across a network such as the internet. IPsec is a suite of protocols standardised by the Internet Engineering Task Force (IETF), and because it works at the network layer, it can protect traffic from any application.

IPsec also appears in VPN app settings as “IKEv2/IPsec” or “L2TP/IPsec“, where Internet Key Exchange version 2 (IKEv2) or the older version 1 (IKEv1) sets up the keys and IPsec protects the data.

What is IPsec?

IPsec is a set of IETF standards for protecting IP packets at the network layer (Layer 3), so it can protect any IP traffic, whatever application produced it. The policy on each device or gateway decides which traffic is protected.

IPsec has three protocols: the Authentication Header (AH), the Encapsulating Security Payload (ESP) and Internet Key Exchange (IKE).

The IETF published the first IPsec standards in 1995. RFC 4301, published in December 2005, sets out the current architecture.

Is IPsec the same as a VPN?

No. IPsec is a set of security protocols, and an IPsec VPN is one way of using them. A VPN can also be built on other protocols, such as WireGuard or Transport Layer Security (TLS), and the details of how a VPN works depend on which one it uses.

What does IPsec protect?

IPsec’s main protections are encryption of the data (ESP only), integrity checks that show a packet was not changed, proof of which device sent it, and rejection of replayed packets when the receiver turns that check on.

IPsec does not hide everything: the outer IP header shows the two tunnel endpoints, so anyone watching the network can see that a tunnel exists, and IPsec has only limited ways to disguise how much traffic flows and when.

A VPN between two gateways, the devices at the edge of each network, protects traffic only between them, not inside the network at either end.

Types of IPsec VPN

The US National Institute of Standards and Technology (NIST) describes four main architectures in its guide to IPsec VPNs: gateway-to-gateway, remote access, host-to-host and mesh. The two most familiar are site-to-site (gateway-to-gateway) and remote access.

Gateway-to-gateway (site-to-site). Gateways at two locations join their networks into a site-to-site VPN. The VPN is typically transparent to users, and their devices need no VPN client software. Gateways choose which traffic enters the tunnel either with policy rules that list the networks at each end (policy-based) or by routing traffic to a virtual tunnel interface (route-based). The large cloud providers each offer a managed VPN gateway that uses IPsec, and some of them also accept IKEv1.

Remote access (also called host-to-gateway). In a remote access VPN, each remote device is set up as an IPsec client of the organisation’s gateway. NIST calls this an increasingly common architecture, and somewhat complex to manage in terms of users and devices. Protection ends at the gateway.

Host-to-host. Two hosts (individual computers or servers) protect the traffic between themselves. NIST notes that host-to-host VPNs can be resource-intensive to implement and maintain.

Mesh. NIST uses this name for a large-scale host-to-host deployment, in which every host sets up its own IPsec connection to each host it talks to. Products sold as mesh VPNs also connect devices directly to each other, using protocols such as WireGuard.

The parts of IPsec: AH, ESP and IKE

AH and ESP protect packets, and IKE sets up the keys and security associations they use.

AH vs ESP

ESP is the main IPsec protocol and the one IPsec VPNs rely on. It can encrypt and AH cannot. AH also does not work through network address translation (NAT), which lets a router share one public address between many devices, and NIST says AH should not be implemented or deployed.

Where only integrity checking is wanted, the IETF prefers ESP without encryption to AH, because AH does not work through NAT.

Security associations

A security association (SA) is an agreement between two devices on how to protect traffic: which algorithms, which keys and for how long. Each SA is identified by a number called the Security Parameter Index (SPI).

The lifetime can be a time, a byte count or both. IKEv2 does not negotiate it: each end applies its own, and before it runs out the SA is replaced by a new one or closed.

SAs, with their keys, algorithm settings and sequence counters, are kept in the Security Association Database (SAD). The Security Policy Database (SPD) decides for each packet whether to protect it, let it through unprotected, or drop it.

IKE

Internet Key Exchange authenticates the two ends and creates the SAs, so keys do not have to be configured by hand. IKE first sets up its own SA, the IKE SA, which protects IKE’s messages; the SAs it then creates for ESP are called child SAs. IKEv2 is the current version, defined in RFC 7296 (October 2014), an Internet Standard. The IETF deprecated the older IKEv1 in 2023.

The two ends prove their identities with certificates, pre-shared keys or the Extensible Authentication Protocol (EAP). With EAP, the server normally still proves its identity with a certificate.

How an IPsec VPN connection works

IKE normally sets up an IPsec VPN connection in two exchanges; ESP then carries the data until the tunnel closes, with the SAs renewed along the way.

1. Traffic matches a policy. A packet matches a “protect” rule in the gateway’s or device’s policy. If no SA exists yet, IKE is asked to create one.
2. Key exchange (IKE_SA_INIT). The two ends agree on the encryption and other algorithms for the IKE SA and exchange the values they need to create shared keys: Diffie-Hellman values and random numbers called nonces. They also check whether a NAT device sits between them.
3. Authentication (IKE_AUTH). This exchange is encrypted, apart from its message header. Each end proves its identity, and the first child SA for the data, with its own algorithms, is created inside it. The two exchanges normally take four messages.
4. ESP carries the data. In tunnel mode, each original packet is encrypted, given an integrity check and placed inside a new packet addressed to the other end.
5. SAs are renewed. Before an SA expires, one end uses a CREATE_CHILD_SA exchange to replace it with a new SA that has fresh keys. A fresh Diffie-Hellman exchange is required when the IKE SA is rekeyed and optional for child SAs, which is how IKEv2 can provide perfect forward secrecy.
6. The tunnel closes. A Delete message in an INFORMATIONAL exchange removes the SAs.

What are Phase 1 and Phase 2?

Phase 1 and Phase 2 are IKEv1 names. In IKEv1, Phase 1 (Main Mode or Aggressive Mode) set up a protected channel between the two ends, and Phase 2 (Quick Mode) set up the IPsec SAs.

RFC 7296 maps the names onto IKEv2. IKE_SA_INIT and IKE_AUTH are what IKEv1 called Phase 1, and the first child SA is created inside IKE_AUTH rather than in a separate phase. Later child SAs are created with CREATE_CHILD_SA, which does part of the job IKEv1 called Phase 2. IKEv2 has no Aggressive Mode.

Several vendors, Microsoft Azure among them, still label IKEv2 settings this way. In those screens, “Phase 1” holds the settings for the IKE connection itself, and “Phase 2” holds those for the child SAs that carry data.

What must match for an IPsec VPN to connect?

An IPsec VPN comes up only when both ends run the same IKE version, accept at least one common set of algorithms, accept each other’s identity and credentials, and agree on which traffic belongs in the tunnel.

• Algorithms. For the IKE SA (encryption, integrity where the cipher does not include it, pseudorandom function and Diffie-Hellman group) and for the child SAs. The initiator offers the sets it accepts, and the responder picks one.
• Authentication. How each end proves who it is (certificate, pre-shared key or EAP; the two ends can use different methods), the credentials, and the identity each end expects from the other.
• Traffic. The addresses or networks at each end, called traffic selectors in IKEv2. The responder can narrow them to a subset, so the two sides need compatible selectors, not identical ones.

When one of these does not match, the connection fails with an IKEv2 error notification: NO_PROPOSAL_CHOSEN for algorithms, AUTHENTICATION_FAILED for authentication and TS_UNACCEPTABLE for traffic selectors. SA lifetimes do not need to match, because IKEv2 does not negotiate them. A tunnel can also fail because a network blocks IPsec’s ports or mishandles fragmented packets.

Tunnel mode vs transport mode

Tunnel mode protects the whole original packet behind a new IP header and is the mode VPN gateways must use for the traffic they carry. Transport mode protects only the payload and is typically used between two hosts.

In tunnel mode, the outer header names the two IPsec devices, such as two gateways, while the inner header keeps the packet’s original source and destination. For example, a computer at a branch office (10.1.0.5) sends a packet to a server at head office (10.2.0.20). The branch gateway (198.51.100.1) encrypts the whole packet and places it inside a new one addressed to the head-office gateway (203.0.113.1), so the internet in between sees only the two gateway addresses.

Under L2TP/IPsec, the Layer 2 Tunneling Protocol (L2TP) provides the tunnel, so IPsec normally runs in transport mode.

Germany’s Federal Office for Information Security (BSI) generally advises preferring tunnel mode when using ESP, because it encrypts the entire original packet.

What ports does IPsec use?

IPsec normally uses UDP port 500 for IKE, UDP port 4500 when NAT is in the path, and its own IP protocol numbers for the data (ESP 50, AH 51), rather than a single port.

ESP is its own IP protocol, so it travels inside UDP only when NAT traversal is in use, or inside TCP with the TCP fallback. A device behind a router that uses NAT therefore sends its ESP packets over UDP 4500.

How IPsec gets through NAT

IPsec gets through NAT by placing ESP inside UDP. ESP packets carry no port numbers, so a router that shares one public address between many devices has no reliable way to track them. Firewalls can also be configured to pass IPsec inside UDP but not plain ESP, or the other way round.

• Detection. In the first IKEv2 exchange, each side sends hashes of the address and port it sent the packet from and the address and port it sent it to. The other side compares them with the packet that actually arrived. If they differ, a NAT device is in the path.
• Encapsulation. Both ends then put ESP inside UDP on port 4500, and IKE moves to the same port.
• Keepalives. While the tunnel is idle, the device behind the NAT sends a one-byte keepalive packet, typically every 20 seconds, so the NAT router does not drop its record of the connection.

AH cannot be carried this way, because its check covers the addresses that NAT rewrites. IPsec’s headers, plus the UDP header when NAT is in the path, leave less room for data in each packet; how much depends on the cipher and the mode.

A network that blocks UDP 500, UDP 4500 and ESP stops IPsec, unless both ends are set up to carry IKE and IPsec over TCP. That option was first specified in 2017 and is now defined in RFC 9329 (2022). It is meant as a fallback, uses TCP port 4500 by default (other ports only when both ends are configured for them), and running TCP inside TCP can slow traffic. Obfuscated VPNs take a different approach, disguising VPN traffic so that such networks do not recognise it.

Is IPsec secure?

The UK’s National Cyber Security Centre (NCSC) recommends IPsec for VPN access, citing built-in clients and interoperability, and security agencies’ guidance is about using it well: strong algorithms only, IKEv2 rather than IKEv1, strong keys or certificates, correct configuration, and gateways that are kept patched.

• Algorithms. The NCSC’s recommended IKEv2 profile uses AES-128-GCM for encryption, HMAC-SHA-256 as the pseudorandom function and Diffie-Hellman group 19 or 14. Current IETF guidance says implementations must not include DES, MD5-based algorithms or the weakest Diffie-Hellman groups, and discourages the other small Diffie-Hellman groups; separate IETF guidance for ESP also discourages 3DES. A gateway that still accepts weak algorithms can be pushed into using them, which the US National Security Agency (NSA) calls a downgrade attack.
• IKE version. The IETF deprecated IKEv1 in 2023, and the NCSC’s IPsec guidance says it should no longer be used.
• Authentication. The NCSC strongly recommends certificates and accepts pre-shared keys only by exception, with at least 128 bits of entropy and unique to each group of devices. Weak or widely shared pre-shared keys can be guessed or misused, and anyone who knows the key can pose as another device.
• Configuration and patching. The NSA’s 2020 guidance on configuring IPsec VPNs advises avoiding default settings, removing unused or non-compliant algorithm suites and patching gateways routinely.
• Gateway software. The IPsec-related entries in the US Cybersecurity and Infrastructure Security Agency’s (CISA) catalogue of exploited vulnerabilities are bugs in vendors’ gateway and operating system software, not weaknesses in the IPsec standards.
• Post-quantum. Post-quantum protection for IKEv2 is arriving: a post-quantum pre-shared key can be mixed into the keys IKEv2 derives, an extra key exchange can run alongside Diffie-Hellman, and a standard for using ML-KEM in IKEv2 was awaiting publication in October 2026.

Is IPsec outdated?

No. IKEv1 is deprecated, but IPsec with IKEv2 remains in current use. Windows, macOS, iOS and Android all include a built-in IKEv2/IPsec client, and the large cloud providers’ managed VPN gateways use IPsec. NIST’s 2020 guide adds that IKEv2 configurations have improved to the point where IPsec VPNs are as easy to set up and use as Secure Sockets Layer (SSL) VPNs. Newer options, WireGuard and TLS-based VPNs, compete with it.

Advantages and disadvantages of an IPsec VPN

IPsec’s strengths come from being an open standard built into operating systems. Its weaknesses come from relying on ports and protocols that networks can block or that NAT disturbs, and from the management some deployments need.

Advantages

• Open standard. Equipment from different vendors can usually connect when both ends are configured with matching settings.
• Built into operating systems. Windows, macOS, iOS and Android include IKEv2/IPsec clients.
• Works below the applications. It can protect any IP traffic, whatever application produced it.
• Long-established. The IETF has published IPsec standards since 1995, and the current architecture since 2005.

Disadvantages

• Can be blocked. A network that blocks its ports and ESP stops it, unless both ends are set up for the TCP fallback.
• NAT needs extra handling. ESP must go inside UDP, and AH cannot cross NAT.
• Some setups take ongoing work. NIST describes remote access VPNs as somewhat complex to manage in terms of users and devices, and host-to-host VPNs as resource-intensive to implement and maintain.
• Less control per application. Because it works at the network layer, it gives less control over individual applications than protection built into them.
• Per-packet overhead. Extra headers leave less room for data in each packet.

IPsec VPN vs SSL VPN, WireGuard and OpenVPN

IPsec is one of several VPN protocols. It is defined by IETF standards; WireGuard and OpenVPN are each defined by their own projects.

• SSL VPN. An SSL VPN runs each vendor’s own protocol inside TLS, or Datagram TLS (DTLS), which runs over UDP. NIST’s 2008 guide to SSL VPNs said there were no standards for any type of SSL VPN. IPsec products from different vendors, by contrast, can usually work together. The NCSC sees very little difference in risk between IPsec and TLS VPNs, other things being equal.
• WireGuard. A protocol built on the Noise framework, which its designer presents as an alternative to IPsec. It uses one fixed set of algorithms instead of negotiating them, and runs over UDP only.
• OpenVPN. Uses TLS to set up its connection and its own format to carry data, over a single UDP or TCP port.

IPsec in your VPN app or phone settings

When a VPN app or phone lists “IKEv2/IPsec” or “L2TP/IPsec”, IPsec is what protects the traffic.

• IKEv2/IPsec: IKEv2 sets up the keys and SAs.
• L2TP/IPsec: L2TP carries the connection and the older IKEv1 sets up the keys. Because IKEv1 is deprecated, choose IKEv2/IPsec where an app or phone offers both. Apple still lists L2TP/IPsec on iOS, iPadOS and macOS. Current versions of Android’s built-in VPN settings offer only IKEv2/IPsec types for new profiles. Microsoft announced on 8 October 2024 that future Windows Server versions will not accept incoming L2TP connections to their built-in VPN server, while Windows can still make outgoing ones.
• IPsec identifier: on a Samsung phone running One UI 8.5 (checked October 2026), the built-in VPN form has an “IPSec identifier” field. It is the identity the phone sends to the server in IKEv2’s identification payload, and the server uses it to look up which settings and credentials apply to that phone. Samsung’s form lets it be left empty (shown as “Not used”); whether the connection then works depends on what the server expects. Apple devices call the equivalent the local identifier.

Frequently asked questions

What is the main purpose of IPsec?

To protect IP packets as they cross a network. IPsec can keep their contents confidential, show that they were not changed on the way, prove which device sent them and reject copies an attacker replays. It does this at the network layer, below the applications.

What does IPsec stand for?

Internet Protocol Security. It is not one protocol but a suite of standards from the Internet Engineering Task Force (IETF). It covers packet protection (AH and ESP) and key exchange (IKE). The first IPsec standards appeared in 1995, and the architecture in use today dates from 2005.

Is IPsec layer 2 or layer 3?

Layer 3, the network layer. IPsec provides its security at the IP layer, so it can protect any protocol carried over IP, whatever application uses it. ESP and AH are protocols of their own, normally carried directly inside IP rather than over TCP or UDP.

What are the three protocols used in IPsec?

AH, ESP and IKE. IKE runs first: it checks each end’s identity and sets up the keys. ESP then encrypts and integrity-checks the data, and is the protocol IPsec VPNs rely on. AH checks packets without encrypting them; it is optional in the standards, and NIST advises against deploying it.

What encryption does an IPsec VPN use?

There is no single answer: the two ends agree on algorithms when they set up or renew a connection, from the sets each is configured to accept. One current example is the UK NCSC’s recommended IKEv2 profile, built on AES-128-GCM. Older algorithms such as DES and MD5-based ones are ruled out by current IETF guidance.

Does an IPsec VPN hide your IP address?

It depends on where your traffic leaves the tunnel. A VPN app that routes your traffic through its provider’s server shows sites that server’s address instead of yours. A company VPN does the same with the company’s address, unless split tunnelling sends your internet traffic out directly. Anyone watching your connection still sees you are connected to the VPN server.

Is IPsec required for IPv6?

No longer. Earlier IETF guidance required every IPv6 device to implement IPsec. RFC 6434 (December 2011) lowered that to a recommendation, and RFC 8504 (2019) keeps it as one. The requirement was about implementing IPsec, not about using it for all IPv6 traffic.

What is an IPsec identifier in VPN settings?

It is the identity your phone presents to the VPN server when an IKEv2 connection starts. The server can use it to choose your settings and credentials; when it does, the value must match. IKEv2 allows forms such as an email address, a domain name or an IP address. Samsung labels it “IPSec identifier” and lets it be left empty.

Can an IPsec VPN be blocked?

Yes. A network can stop it by blocking UDP ports 500 and 4500 and IP protocol 50 (ESP). IPsec can then fall back to TCP, by default on port 4500, but only if both ends were configured for that in advance; the fallback is not negotiated during the connection.