Home / OCI

OCIRunbookAdvanced

Sev1: OCI FastConnect down — virtual circuit, BGP and DRG routing checks

Troubleshoot an OCI FastConnect outage from the provider and cross-connect state through the virtual circuit, BGP session, advertised routes and DRG route tables, and confirm IPSec VPN failover takes over.

Deepak Bhardwaj2 min read

Applies toOCI FastConnect (partner and colocation), Dynamic Routing Gateway (DRG), Site-to-Site VPN, OCI CLI 3.x

Symptoms

  • On-premises to OCI traffic fails, or shifts to the backup IPSec VPN.
  • OCI Monitoring shows the virtual circuit BGP session down, or no bytes in/out.

1. Virtual circuit state

oci network virtual-circuit list --compartment-id $C \
  --query 'data[].{name:"display-name", id:id, state:"lifecycle-state", bgp:"bgp-session-state", provider:"provider-state", svcProvider:"service-type"}' --output table
Field Expected If not
lifecycle-state PROVISIONED Being changed or deleted — check Audit
provider-state ACTIVE (partner) Provider side down — call the partner with the circuit OCID
bgp-session-state UP Step 3

2. Colocation: cross-connect

oci network cross-connect list --compartment-id $C \
  --query 'data[].{name:"display-name", state:"lifecycle-state", location:"location-name"}' --output table
oci network cross-connect-status get --cross-connect-id <id>

The status shows interface state and light levels. Down interface or no light → physical issue at the colocation (cross-connect, optics, your router port).

3. BGP

oci network virtual-circuit get --virtual-circuit-id $VC \
  --query 'data.{asn:"customer-asn", cross:"cross-connect-mappings", bw:"bandwidth-shape-name", routing:"routing-policy"}'

Compare with your router: BGP peer IPs (the /30 or /31 in the cross-connect mappings), your ASN, MD5 key, and VLAN. After maintenance, a changed MD5 key or VLAN is a common cause.

router# show bgp summary
router# show bgp neighbor 10.255.0.1 advertised-routes

Check you're within the documented limit of routes you can advertise per BGP session; exceeding it can bring the session down.

4. DRG routing

oci network drg-attachment list --compartment-id $C --drg-id $DRG --output table
oci network drg-route-table list --drg-id $DRG --output table
oci network drg-route-rule list --drg-route-table-id <table-ocid> --query 'data[].{dest:destination, type:"route-type", next:"next-hop-drg-attachment-id"}' --output table
  • The VCN attachment's DRG route table must have the on-prem prefixes (learned dynamically from the virtual circuit attachment's import distribution).
  • VCN route tables must send on-prem CIDRs to the DRG.
  • Recent DRG route distribution edits can silently drop routes — check Audit for UpdateDrgRouteDistribution.

5. Failover to VPN

oci network ip-sec-connection list --compartment-id $C --output table
oci network ip-sec-tunnel list --ipsc-id <ipsc-ocid> --query 'data[].{ip:"vpn-ip", status:status, bgp:"bgp-session-info"."bgp-state"}' --output table

With both paths up, OCI prefers FastConnect for the same prefix. When FastConnect is down, confirm the VPN tunnels are UP, BGP over the tunnels is UP, and your on-prem router now prefers the VPN path.

Prevent it next time

  • Two virtual circuits on separate cross-connects or providers (redundant FastConnect), plus a VPN backup.
  • Alarms on the virtual circuit BgpSessionState metric and on IPSec tunnel state.
  • Test failover twice a year; document the router changes needed.

Related fixes