What Is VXLAN?

VXLAN (Virtual Extensible Local Area Network) is a network virtualization protocol that helps scale network operations across the cloud and other virtualized service infrastructures, such as data centers. VXLAN encapsulates Layer 2 Ethernet frames in UDP packets to transport network traffic by tunneling over Layer 3. By using VXLAN, networking is more scalable and fault-resistant than traditional VLAN systems, for performance in massive, virtualized environments.

Because VXLAN encapsulates network traffic in UDP packets, it can operate over any IP network that supports UDP. This enables VXLAN to span large, distributed cloud networks regardless of their physical network layout or the geographic distance between underlying network nodes.

Overcome VLAN scalability limits

Traditional VLANs have 12-bit IDs and are limited to 4,094 unique segments—in practice, closer to 4,000, as hardware vendors typically reserve some IDs. This segment constraint falls well below the number of virtual networks required to support modern cloud and multi-tenant network environments. VXLAN uses a 24-bit segment identifier called a VXLAN Network Identifier (VNI), which means you can define around 16 million logical network segments.

Easier workload mobility between data centers

VXLAN tunneling allows you to move virtual machines between physical servers or entire data centers, without changing their addressing structure. From the destination network device’s perspective, it does not leave its LAN, regardless of changes in hardware or its location in the underlay network. This mobility makes cloud server migrations and backups easier to manage.

Simplify network isolation in multi-tenant environments

Modern server virtualization technology has made using shared physical infrastructure the standard model for data centers. However, to maintain security and compliance, you need to make sure that traffic for each tenant within the data center infrastructure remains isolated. VXLAN technology allows you to create and segment millions of broadcast domains. Each domain is isolated from the others to help prevent data from accidentally being routed to the wrong organization’s server.

Reduce complexity in virtualized environments

As VXLANs allow for millions of isolated segments, the separation of tenancies is far easier than VLAN segmentation. Using VXLAN, tenancies are portable to other sites without address reconfiguration, and broadcasting works as expected per tenant.

Transport across diverse hardware networks

Because VXLAN encapsulates Ethernet frames within UDP packets, it can run over any Layer 3 IP network, regardless of the underlying hardware the network uses. VXLANs can be used with a wide range of modern switches, routers, and hypervisors. That means whether you're using your own on-premises infrastructure or connecting to a cloud provider’s infrastructure, such as AWS, VXLANs allow you to operate a consistent overlay network.

To create large-scale virtualized networks, VXLAN uses a combination of encapsulation, tunneling, and intelligent forwarding.

Encapsulation

VXLAN works by encapsulating an Ethernet frame within a UDP packet so it can traverse an IP network. In other words, it allows OSI Layer 2 communication between virtual machines or other cloud resources through a tunnel over a Layer 3 network. The original Ethernet frame is wrapped with a VXLAN header that includes the VNI, that specific VXLAN’s unique identifier, and is placed inside a UDP packet for transport.

VXLAN Tunnel Endpoints (VTEPs)

The devices responsible for encapsulating and decapsulating traffic at the network edge are called VXLAN Tunnel Endpoints (VTEPs). A VTEP can be implemented in hardware, such as in a top-of-rack switch (TOR), or in software, such as on a hypervisor. When a VTEP receives a frame from a local system, it checks the destination based on the VNI and MAC address and then encapsulates and forwards the packet to the appropriate remote VTEP. Upon arrival, the destination VTEP strips off the headers and delivers the original frame to the target system identified by its MAC address.

Control plane and data plane separation

VXLANs purposefully separate control and data plane functions to improve scalability. The data plane handles the actual encapsulation and forwarding of data. The control plane handles tunneling and routing. Modern VXLAN deployments often use centralized or distributed control planes, such as EVPN (Ethernet VPN) with MP-BGP, for mapping MAC addresses to VTEPs. This reduces broadcast traffic, which is how traditional VLANs learn MAC addresses, and which can be inefficient in modern, massive data centers.

Multicast vs. unicast replication modes

VXLANs use multicast replication or unicast replication to handle BUM (broadcast, unknown-unicast, and multicast) traffic. Multicast mode uses the Layer 3 IP multicast network infrastructure to replicate BUM packets. It sends a single copy from the source VTEP to all other relevant VTEPs on the network. Unicast replication requires the source VTEP to create and send multiple individual unicast copies of the packet, one for each destination VTEP.

A VXLAN’s ability to support up to 16 million logical networks makes it ideal for many use cases in cloud architecture and modern data centers.

Multi-tenancy for cloud providers

Cloud providers can use VXLANs to isolate customer workloads, or tenants, on shared physical infrastructure. Each tenant gets its own VXLAN segment and VNI, making it private and separate from all other tenants’ traffic.

Data center network virtualization technology

Traditional VLANs, with only 4094 possible segments, are insufficient for the needs of modern, large-scale data centers. Thanks to their 24-bit addressing, VXLANs allow you to create a vast number of isolated virtual networks for different applications, teams, or services. They also support microsegmentation, which allows you to set fine-grained security policies for different workloads.

Recovery and workload migration

Because they decouple networks from physical infrastructure, VXLANs make it easier to migrate virtual machines (VMs) and containers between geographically dispersed data centers. You can move VMs to regions where they’ll operate more effectively without changing IP addresses or disrupting active connections or maintenance activity.

Hybrid cloud connectivity and network extension

Organizations connecting on-premises environments to the cloud can unpack VXLAN packets at the network boundary, then transport and route them with services such as AWS Direct Connect and AWS Transit Gateway. On the cloud side, you can use other network virtualization and isolation services, such as AWS Virtual Private Cloud.

Implementing VXLANs requires thoughtful planning and preparation. These are some of the most common challenges you will want to consider.

Encapsulation overhead and performance impact

VXLAN traffic adds 50 bytes of overhead to each IPv4 Ethernet frame, and 70 bytes for IPv6. Although this may not affect some environments, at scale, it increases bandwidth consumption and can reduce throughput for high-performance workloads.

The VXLAN encapsulation and decapsulation process also requires more CPU or hardware resources. While modern hardware VTEPs handle this efficiently, if not properly optimized, software-based implementations on hypervisors can increase latency.

Control plane complexity

VXLAN is an overlay protocol, which means it doesn’t define how MAC-to-VTEP mappings are learned or distributed within your Layer 3 networks. Without a robust control plane such as EVPN (Ethernet VPN), networks rely on data-plane learning, which can lead to inefficient flooding of broadcast, unknown unicast, and multicast (BUM) traffic.

Operational visibility and troubleshooting

Because VXLAN networks are overlays that sit over the physical underlay, it can be more challenging to identify the source of network events and perform troubleshooting. Traditional network tools can't see inside encapsulated VXLAN packets, making it difficult to trace issues across layers. You want to maintain modern, integrated monitoring solutions that can provide visibility into both the overlay and underlay networks.

Hardware VTEP support requirements

Not all network devices support VXLAN termination (VTEPs). Deploying VXLANs at scale often requires modern switches or routers. Older or lower-end equipment can lack this support and limit your deployment options.

VXLAN is one of several network overlay technologies designed to extend Layer 2 networks over Layer 3 infrastructure. While these technologies all share similar purposes, they differ in their design and use cases.

VXLAN vs. GENEVE

GENEVE (Generic Network Virtualization Encapsulation) is a newer, flexible overlay protocol designed to support features such as service chaining and in-band telemetry. Where VXLAN uses a fixed-length header, GENEVE uses a variable-length, extensible header. Within this header, in addition to routing data, it can also carry contextual metadata. VXLAN remains more widely adopted in production environments due to its existing, broad hardware and software support.

VXLAN vs. NVGRE

NVGRE (Network Virtualization using Generic Routing Encapsulation) was Microsoft’s alternative to VXLAN, using GRE (Generic Routing Encapsulation) instead of UDP. Both use 24-bit identifiers, named a TNI in NVGRE versus a VNI in VXLAN, the TNI allowing for larger-scale segmentation. However, using UDP, the VXLAN protocol integrates more easily with most existing network infrastructure. As a result, VXLAN has seen broader industry adoption over NVGRE, especially outside Microsoft-centric cloud environments.

VXLAN vs. SD-WAN Overlays

SD-WAN also uses an overlay model, but for a different purpose. Although VXLAN was designed for data center network virtualization and for extending Layer 2 domains within or between data centers, SD-WAN creates secure, application-aware overlays across WAN links. For example, a common use case of SD-WAN is to connect branch offices to headquarters or the cloud.

These are not mutually exclusive technologies. You might use VXLAN internally for server virtualization, and use SD-WAN to securely connect remote sites to those resources. You can carry VXLAN traffic over an SD-WAN backbone.

AWS supports organizations with existing VXLAN architectures or migrating data center implementations to the cloud. Explore our composable network services to connect VXLAN with AWS services.

AWS Transit Gateway helps you design and implement networks at scale by acting as a cloud router. AWS Transit Gateway supports VXLAN, connecting VPCs and on-premises networks through a central hub.

Amazon Virtual Private Cloud (VPC) supports traffic mirroring that allows you to capture inbound and outbound traffic from your network interfaces, copying it to your VPC destinations with VXLAN, including Amazon EC2 instances and Network Load Balancer.

AWS Direct Connect allows you to create a dedicated network connection between your VXLAN data center and AWS regions, for hybrid cloud performance.

Get started with VXLAN networking on AWS by creating a free account today.