OCI instance metrics missing or plugins INVALID: troubleshooting Oracle Cloud Agent

#Symptom

One or more of these on an OCI compute instance:

• The instance Metrics page shows no data for CPU, memory, disk or network.
• Alarms on the oci_computeagent namespace go to No data or never fire.
• The Oracle Cloud Agent tab shows plugins as INVALID, or plugins you enabled never reach RUNNING.
• Bastion managed SSH, Run Command, OS Management Hub or Custom Logs stop working on that instance.

All of these depend on Oracle Cloud Agent, a service on the instance that runs the plugins and calls OCI APIs. If the agent stops, can't reach OCI, or is rejected, everything built on it goes quiet together.

#Quick triage

#1. Confirm from the outside

Check plugin status without logging in:

oci instance-agent plugin list \
  --compartment-id ocid1.compartment.oc1..xxxx \
  --instanceagent-id ocid1.instance.oc1..xxxx \
  --all --output table

Check whether any CPU data arrived in the last hour:

oci monitoring metric-data summarize-metrics-data \
  --compartment-id ocid1.compartment.oc1..xxxx \
  --namespace oci_computeagent \
  --query-text 'CpuUtilization[5m]{resourceId = "ocid1.instance.oc1..xxxx"}.mean()' \
  --start-time "$(date -u -d '-1 hour' +%Y-%m-%dT%H:%M:%SZ)" \
  --end-time "$(date -u +%Y-%m-%dT%H:%M:%SZ)"

An empty data array means OCI isn't receiving metrics from the agent. Check the agent-level settings too:

oci compute instance get --instance-id ocid1.instance.oc1..xxxx \
  --query 'data."agent-config"'

If is-monitoring-disabled or are-all-plugins-disabled is true, someone turned it off. Go to step 5.

#2. Is the agent installed and running?

Log in (SSH, a Bastion port-forwarding session, or the serial console if SSH is also broken).

Oracle Linux:

rpm -q oracle-cloud-agent
systemctl is-enabled oracle-cloud-agent; systemctl is-active oracle-cloud-agent
sudo systemctl status oracle-cloud-agent oracle-cloud-agent-updater --no-pager

Ubuntu (installed as a snap):

snap list oracle-cloud-agent
snap services oracle-cloud-agent

Windows Server (PowerShell as administrator):

Get-WmiObject -Class Win32_Product | Where-Object Name -eq "Oracle Cloud Agent"
sc.exe query "OCA" | findstr "STATE"

If it's stopped, restart it:

sudo systemctl restart oracle-cloud-agent      # Oracle Linux
sudo snap restart oracle-cloud-agent           # Ubuntu

net stop OCA; net start OCA                    # Windows

If the agent isn't installed (common on custom or imported images), install it using the steps in the Oracle Cloud Agent documentation for your OS. On Oracle Linux it's the oracle-cloud-agent package.

On domain-joined Windows instances, check that the agent's virtual account still has the Log on as a service right. A domain GPO can remove it, and then the service won't start or runs without access.

#3. Can the agent reach OCI?

First, the instance metadata service. The agent reads its identity and config from it:

curl -s -H 'Authorization: Bearer Oracle' http://169.254.169.254/opc/v2/instance/ | head -40

You should get JSON that includes agentConfig. A timeout here usually means a host firewall rule or a proxy setting is catching the link-local address.

Next, the route out. The agent calls public OCI endpoints, so the instance's subnet needs one of:

• a service gateway with a route rule for All Services in Oracle Services Network (the usual choice for private subnets),
• a NAT gateway, or
• an internet gateway (public subnets with a public IP).

Check the subnet's route table:

SUBNET=$(oci compute instance list-vnics --instance-id ocid1.instance.oc1..xxxx \
  --query 'data[0]."subnet-id"' --raw-output)
RT=$(oci network subnet get --subnet-id "$SUBNET" --query 'data."route-table-id"' --raw-output)
oci network route-table get --rt-id "$RT" --query 'data."route-rules"' --output table

Also check that the security list or NSG allows egress TCP 443 to that destination (the Oracle Services Network service CIDR label, or 0.0.0.0/0).

If outbound traffic must go through a proxy, give the agent the proxy settings. On Oracle Linux:

sudo EDITOR=vi systemctl edit oracle-cloud-agent
# add:
# [Service]
# Environment="http_proxy=http://proxy.internal:3128"
# Environment="https_proxy=http://proxy.internal:3128"
# Environment="no_proxy=localhost,127.0.0.1,169.254.169.254"
sudo EDITOR=vi systemctl edit oracle-cloud-agent-updater   # same block
sudo systemctl daemon-reload
sudo systemctl restart oracle-cloud-agent oracle-cloud-agent-updater

Warning

Always keep 169.254.169.254 in no_proxy. If metadata requests go to the proxy, the agent can't get its credentials and every plugin fails.

#4. Check the monitoring log for auth and clock errors

sudo tail -30 /var/log/oracle-cloud-agent/plugins/gomon/monitoring.log

Windows Server 2019 and later:

Get-Content -Tail 30 C:\Windows\ServiceProfiles\OCA\AppData\Local\OracleCloudAgent\plugins\gomon\monitoring.log

Fix the clock on Oracle Linux:

grep -E '^(server|pool)' /etc/chrony.conf     # expect: server 169.254.169.254 iburst
sudo systemctl restart chronyd && chronyc tracking
sudo systemctl restart oracle-cloud-agent

#5. Re-enable plugins in the instance config

Console: Compute → Instances → instance → Oracle Cloud Agent tab, toggle the plugin to Enabled, or click Start plugins if all plugins were stopped. Changes can take up to 10 minutes.

From the CLI, generate the JSON template, edit it, then apply:

oci compute instance update --generate-param-json-input agent-config > agent-config.json
# edit, for example:
# {
#   "isMonitoringDisabled": false,
#   "areAllPluginsDisabled": false,
#   "pluginsConfig": [
#     { "name": "Compute Instance Monitoring", "desiredState": "ENABLED" },
#     { "name": "Bastion", "desiredState": "ENABLED" }
#   ]
# }
oci compute instance update --instance-id ocid1.instance.oc1..xxxx \
  --agent-config file://agent-config.json

Caution

Compare against the current agent-config (step 1) before applying, and list every plugin you want to keep enabled. Disabling Compute Instance Monitoring stops metrics, and any autoscaling policy that uses them stops working.

#6. Still broken: collect a diagnostic bundle

Agent version 1.14.0 and later include a diagnostic tool:

cd /usr/libexec/oracle-cloud-agent/ocatools && sudo ./diagnostic

cd "C:\Program Files\Oracle Cloud Agent\ocatools"; .\diagnostic.ps1

Attach the resulting archive to an Oracle support request.

#Prevent it next time

• Give private subnets a service gateway route as standard. Don't rely on a NAT gateway that might be removed in a cost review.
• Build custom images with Oracle Cloud Agent installed and enabled, and with chrony pointed at 169.254.169.254.
• Exclude 169.254.169.254 from proxies and host firewall rules in your baseline config.
• Create an alarm that fires on missing data, for example CpuUtilization[5m].absent(), so silent agents get noticed before the next incident.
• Manage agent_config in Terraform so plugin settings don't drift after console changes.