Home / OCI

OCITroubleshootingIntermediate

OCI Network Path Analyzer: find the route or security rule that blocks traffic

Use OCI Network Path Analyzer from the Console or CLI to find which route table, security list or NSG stops traffic between two endpoints, then check the parts it can't see.

Deepak Bhardwaj9 min read

Applies toOCI Networking (VCN, DRG, LPG), Network Path Analyzer, OCI CLI 3.x

An application can't reach its database, an on-premises client times out against a load balancer, or a new subnet "can't see" a peered VCN. You can click through route tables, security lists and NSGs one by one, or you can ask Network Path Analyzer (NPA) to walk the path for you and tell you which hop allows or denies the traffic.

NPA does not send packets. It reads your network configuration (route tables, DRG route tables, security lists, NSGs and Zero Trust Packet Routing policies) and models the path. That makes it safe to run in production and fast, but it also means it can't see anything inside the operating system.

When to use it

Symptom NPA helps? What to look at
Connection timed out between two OCI hosts Yes Forward and return path, security rules on both VNICs
Works one way, fails the other Yes Run bidirectional; compare forward vs return hops
On-prem to OCI over FastConnect or VPN times out Yes, up to the DRG DRG route table, VCN route table for the DRG attachment
Connection refused Usually no The port is reachable but nothing listens, or the OS firewall rejects
DNS name doesn't resolve No DNS, not routing
Peered VCN in another region or tenancy Partly Expect Indeterminate for cross-region RPC and cross-tenancy LPG

Tip

"Timed out" points to routing or security rules, which NPA covers. "Refused" usually means the packet arrived and the host answered with a reset, so check the service and the OS firewall first.

Prerequisites: IAM policies

NPA needs two sets of permissions: one for your admins, and one that lets the NPA service read your network configuration. Oracle recommends granting both at the tenancy level, because compartment-level grants can make results less exact.

allow group NetworkAdmins to manage vn-path-analyzer-test in tenancy

allow any-user to inspect compartments in tenancy where all { request.principal.type = 'vnpa-service' }
allow any-user to read instances in tenancy where all { request.principal.type = 'vnpa-service' }
allow any-user to read virtual-network-family in tenancy where all { request.principal.type = 'vnpa-service' }
allow any-user to read load-balancers in tenancy where all { request.principal.type = 'vnpa-service' }
allow any-user to read network-security-group in tenancy where all { request.principal.type = 'vnpa-service' }
allow any-user to read zpr-family in tenancy where all { request.principal.type = 'vnpa-service' }

Warning

NPA results show your network and security layout in detail. Give manage vn-path-analyzer-test to the network and on-call groups, not to everyone.

If the service statements are missing, results are often Indeterminate or a hop is marked as not evaluated. Check those before you trust a "no route" result.

Run an analysis in the Console

  1. Go to Networking → Network Command Center → Network Path Analyzer and choose Create path analysis.
  2. Pick the protocol (TCP, UDP or ICMP) and the destination port.
  3. Choose the source and destination. You can use an IP address, an instance VNIC, a load balancer or listener, a network load balancer, a subnet or an on-premises address.
  4. Leave bidirectional analysis on for TCP and UDP. It's on by default and checks the return path too.
  5. Run it. Each hop shows whether the routing and the security rules allowed or denied the traffic, with a link to the rule or route table involved.

Save the analysis if it's a path you'll check again, such as app subnet to DB on 1521, or on-prem to the load balancer on 443. Saved tests can be re-run after every network change.

Run an analysis with the OCI CLI

The CLI is useful in runbooks and when you need the result in a ticket. Generate the input templates first so you get the current JSON shape for your CLI version:

oci vn-monitoring path-analysis get-path-analysis-adhoc \
  --generate-param-json-input source-endpoint > src.json
oci vn-monitoring path-analysis get-path-analysis-adhoc \
  --generate-param-json-input protocol-parameters > proto.json

A typical app-to-database check looks like this:

cat > src.json <<'EOF'
{ "type": "IP_ADDRESS", "address": "10.0.1.15" }
EOF

cat > dst.json <<'EOF'
{ "type": "IP_ADDRESS", "address": "10.0.2.30" }
EOF

cat > proto.json <<'EOF'
{ "type": "TCP", "destinationPort": 1521 }
EOF

cat > opts.json <<'EOF'
{ "isBiDirectionalAnalysis": true }
EOF

oci vn-monitoring path-analysis get-path-analysis-adhoc \
  --compartment-id "$COMPARTMENT_OCID" \
  --protocol 6 \
  --source-endpoint file://src.json \
  --destination-endpoint file://dst.json \
  --protocol-parameters file://proto.json \
  --query-options file://opts.json \
  --wait-for-state SUCCEEDED --wait-for-state FAILED

--protocol is the IP protocol number: 6 for TCP, 17 for UDP, 1 for ICMP.

If an IP range overlaps across VCNs, use a SUBNET endpoint with subnetId set, or a COMPUTE_INSTANCE/VNIC endpoint with the instanceId/vnicId, so NPA knows which address you mean. Other endpoint types include ON_PREM, LOAD_BALANCER, LOAD_BALANCER_LISTENER, NETWORK_LOAD_BALANCER and VLAN.

The command runs as a work request. Fetch the result with the work request OCID:

WR=ocid1.vnmonitoringworkrequest.oc1..example   # from the output above

oci vn-monitoring work-request get --work-request-id "$WR"

oci vn-monitoring work-request-result list \
  --work-request-id "$WR" --result-type PATH_ANALYSIS --all > npa-result.json

# If the work request FAILED:
oci vn-monitoring work-request-error list --work-request-id "$WR"

The result JSON lists the forward and return paths hop by hop, with the routing and security decision at each hop. Field names can change between API versions, so search the file for the deny or "no route" entry rather than relying on a fixed --query path. Work requests are kept for about 12 hours, so attach the JSON to the incident ticket.

Reading the result

Denied by a security rule

The hop names the security list or NSG that has no matching rule. Things to check:

  • Ingress on the destination. The source CIDR or source NSG and the destination port must be allowed.
  • Egress on the source. Teams often add a tight egress rule to an NSG and forget that the old "allow all egress" lived in the subnet's default security list. That list may have been cleaned up since.
  • Stateless rules. A stateless ingress rule needs a matching stateless egress rule for the return traffic. A bidirectional analysis shows this as a deny on the return path.
# Show the rules on an NSG named in the result
oci network nsg rules list --nsg-id "$NSG_OCID" --all \
  --query 'data[].{dir:direction,proto:protocol,src:source,dst:destination,stateless:"is-stateless",tcp:"tcp-options"}' \
  --output table

No route

The hop shows the route table with no matching rule. Common causes:

Cause Fix
Subnet route table has no rule for the peer CIDR Add a route rule to the DRG or LPG
DRG attachment uses a DRG route table that doesn't import the spoke Fix the import distribution or add a static route
Return route is missing on the destination subnet Add the reverse route; NPA shows this on the return path
Traffic should go through a firewall but the ingress route table on the hub attachment is missing Attach a VCN ingress route table that points to the firewall's private IP 
oci network route-table get --rt-id "$RT_OCID" \
  --query 'data."route-rules"[].{dest:destination,target:"network-entity-id"}' --output table

Indeterminate

NPA couldn't evaluate part of the path. Common reasons: the service IAM statements are missing, the path crosses a cross-region RPC or cross-tenancy LPG, or the address is IPv6, which isn't supported. Fix the policies if they're the cause. Otherwise check the far side manually.

NPA says "allowed" but it still fails

NPA covers the OCI network configuration only. When every hop is allowed, look at the hosts:

# On the destination: is anything listening on the port?
sudo ss -ltnp | grep ':1521'

# OS firewall on Oracle Linux
sudo firewall-cmd --list-all

# From the source: an actual connection attempt
nc -vz -w 5 10.0.2.30 1521

Also check:

  • Network firewall policy. NPA shows the routing through an OCI Network Firewall, but you still need to check the firewall's own traffic logs to see whether its policy allows the session.
  • On-premises side. NPA models the path up to the DRG. Your edge firewall and the on-prem routing need separate checks.
  • Source/destination check. If traffic goes through an appliance VM, its VNIC must have Skip source/destination check enabled.

Caution

Don't open 0.0.0.0/0 ingress "to test". Add the specific source CIDR and port that NPA flagged, re-run the analysis, then remove anything temporary.

Prevent it next time

  • Save NPA tests for critical paths, such as app to DB, on-prem to the load balancer, and spoke to the hub firewall. Re-run them after every route table, DRG or NSG change.
  • Make NPA a step in network change runbooks. Run it before the change for a baseline and after it to confirm.
  • Prefer NSGs with source NSG references over CIDR-based rules in shared security lists. Then a missing rule is easy to spot.
  • Keep the NPA service policies in your landing zone Terraform so new tenancies and regions get them by default.
  • Check the current NPA limitations in the documentation (IPv6, cross-region and cross-tenancy paths, tenancy size) before you rely on it for a given path.

Related fixes