Overlay Networks — containers across many hosts

A bridge network stops dead at the edge of one host. When your app spreads across several servers, you need a network that stretches across all of them too.

The problem

You've got containers on host A and containers on host B. Each has a private IP handed out by its own host's local bridge — and odds are they're both sitting on 172.20.0.2. Same address, different machines, no way for them to talk to each other directly.

The idea

An overlay network is a virtual network that rides on top of the real network already connecting your hosts. Every container on the overlay gets a unique IP from the overlay's own address space. So when container A on host 1 fires a packet at container B on host 2, that packet gets tucked inside an ordinary host-to-host packet, shipped across, unwrapped on the far side, and handed over — like a letter travelling inside an envelope.

Overlay network across two hosts (VXLAN encapsulation)
Host 1 (10.0.1.5) Container A 10.100.0.2 Container B 10.100.0.3 Host 2 (10.0.1.6) Container C 10.100.0.4 Container D 10.100.0.5 Overlay: my-app-overlay (10.100.0.0/16) Every container sees a flat network, regardless of host. Traffic between hosts is encapsulated (VXLAN or IPIP) over the physical network.

How the packet actually travels

Let's follow one packet. Container A (10.100.0.2 on host 1) pings container C (10.100.0.4 on host 2):

  1. A writes a packet addressed to 10.100.0.4.
  2. The overlay driver on host 1 notices that destination lives on host 2. So it encapsulates the whole packet inside a UDP/VXLAN packet running from 10.0.1.5 → 10.0.1.6.
  3. The physical network carries that VXLAN packet over to host 2 like any other traffic.
  4. Host 2's overlay driver peels off the VXLAN wrapper and hands the inner packet to container C.

Neither container has any clue the wrapping and unwrapping happened. As far as they're concerned, they're both sitting on one flat network with nothing in between.

Where you'll encounter overlays

Docker Swarm

Built-in overlay driver. docker network create --driver overlay.

Kubernetes CNIs

Most cluster network plugins are overlay-based: Flannel (VXLAN), Calico (IPIP or BGP), Cilium (eBPF), Weave.

Nomad

HashiCorp Nomad uses Consul Connect / native overlays for cross-node service comms.

Cloud provider VPCs

Your VPC is basically an overlay too — the cloud provider virtualises the whole network on top of their physical fabric, you just never see the seams.

Overlay vs underlay

  • Underlay — the real physical network that actually moves packets between your hosts.
  • Overlay — the virtual network your containers think they're on, riding along on top of the underlay.
Extra latency, extra header

All that wrapping tacks on ~50 bytes per packet plus a sliver of CPU. For most workloads you'll never feel it. But if you're pushing serious throughput, that tax can start to add up — which is why some very-high-traffic setups drop back to routed / BGP-based networking to skip the encapsulation entirely.

Try it yourself (Docker Swarm)

# On host 1 — init a swarm
docker swarm init

# On host 2 — join it (use the token printed by init)
docker swarm join --token <TOKEN> <HOST1_IP>:2377

# On any manager — create an overlay network
docker network create --driver overlay --attachable my-app-overlay

# Run a service across the swarm
docker service create --network my-app-overlay --replicas 3 nginx

Key takeaways

  • Overlay networks make containers on many hosts look like they're on one flat network.
  • They work by encapsulating container packets inside host-to-host packets (VXLAN, IPIP, WireGuard).
  • Kubernetes CNIs, Docker Swarm, and Nomad all use overlays under the hood.
  • Small overhead per packet — usually invisible; sometimes worth engineering around.