Home / AWS

AWSHow-toAdvanced

Advisory: hybrid connectivity on AWS - Site-to-Site VPN, Direct Connect and resilient designs

A design guide for connecting data centres and offices to AWS. Choose between VPN and Direct Connect by bandwidth, latency consistency and cost, pick a resiliency model that matches the business impact, decide on gateways and routing, and plan encryption, monitoring and failover tests.

Deepak Bhardwaj3 min read

Applies toAWS Site-to-Site VPN (incl. accelerated VPN), AWS Direct Connect (dedicated and hosted connections), Direct Connect gateway, Transit Gateway, AWS Cloud WAN

VPN or Direct Connect?

Site-to-Site VPN Direct Connect
Path Internet (encrypted IPsec) Private circuit to an AWS DX location
Bandwidth Per-tunnel limit (around 1.25 Gbps); ECMP across tunnels with TGW 50 Mbps–100 Gbps per connection
Latency Variable Consistent
Setup time Minutes Weeks (circuit provisioning)
Encryption Built in Not by default: MACsec (supported ports) or VPN over DX
Cost Hourly per connection + internet egress Port hours + lower data transfer out rates + circuit costs

Typical pattern: Direct Connect as primary for production, VPN as backup or for small sites.

Resiliency models

Model Design Fits
Development/test One DX connection or VPN only Non-critical
High resiliency Two connections at two different DX locations Most production
Maximum resiliency Two connections at each of two DX locations (four total), separate devices Critical workloads
DX + VPN backup DX primary, VPN failover over internet Cost-sensitive production (backup bandwidth lower)

The AWS Direct Connect Resiliency Toolkit in the console walks through these models.

Gateways and routing

On-prem routers ──DX── DX location ──Transit VIF── DX gateway ── Transit Gateway ── VPCs (many accounts/regions)
               └─VPN (backup) ─────────────────────────────────┘
  • Transit VIF + DX gateway + Transit Gateway for many VPCs/accounts.
  • Private VIF + DX gateway + virtual private gateways for simpler setups.
  • BGP design: advertise summarised prefixes; use AS path prepending / local preference so DX is preferred and VPN is backup; know the prefix limits per VIF and TGW.
  • Avoid overlapping CIDRs between on-prem and VPCs. Plan IPAM early.

Encryption

  • Compliance needs encryption in transit across the provider network? MACsec on dedicated connections where supported, or IPsec VPN over a public/transit VIF.
  • Application-level TLS remains good practice either way.

DNS

Hybrid DNS with Route 53 Resolver inbound/outbound endpoints and forwarding rules shared via RAM (see the Route 53 private hosted zone troubleshooting post).

Monitoring and testing

  • CloudWatch: ConnectionState, ConnectionBpsEgress/Ingress, ConnectionLightLevelTx/Rx for DX; TunnelState for VPN.
  • Alarms on state changes and on utilisation > 70% (plan upgrades before saturation).
  • Failover testing: DX resiliency testing (failover test feature brings down BGP sessions on a VIF for a period). Run it at least yearly and after design changes.

Design review questions

  • What's the business impact of losing hybrid connectivity for 1 hour? For 1 day?
  • Do both paths share any single point of failure (same building, same provider, same router)?
  • What bandwidth does the backup path give, and what happens to traffic when it carries everything?
  • How long does BGP failover take, and do applications survive it?

Warning

Two DX connections from the same provider into the same DX location aren't high resiliency. A fibre cut or location issue takes both. Diversity of location, provider and devices is the point.

Checklist

  • VPN vs DX chosen by bandwidth, latency and cost
  • Resiliency model matched to business impact
  • Gateway topology and BGP preferences designed
  • Non-overlapping IP plan
  • Encryption decision documented
  • Monitoring, alarms and failover tests scheduled

Related fixes