Home / OCI

OCIHow-toIntermediate

Transformation: from scattered server logs to central observability with OCI Logging and Logging Analytics

Stop SSH-ing into servers to read logs. Collect OS, application and OCI service logs centrally, parse and search them in Logging Analytics, build dashboards and alerts, and set retention and cost controls.

Deepak Bhardwaj3 min read

Applies toOCI Logging, Service Connector Hub, OCI Logging Analytics, Management Agent / Oracle Cloud Agent, OCI Monitoring

Target picture

Signal Path
OCI service logs (LB, Object Storage, VCN flow, Functions) OCI Logging (service logs)
OS and app logs on instances Custom logs via Oracle Cloud Agent (unified monitoring agent)
On-prem and other clouds Management Agent → Logging Analytics
Audit _Audit log, always on
Analysis, parsing, dashboards, ML clustering Logging Analytics
Metrics and alarms OCI Monitoring

1. Turn on service logs

oci logging log-group create --compartment-id $C --display-name lg-prod
oci logging log create --log-group-id <lg-ocid> --display-name lb-access --log-type SERVICE --is-enabled true \
  --configuration '{"source":{"sourceType":"OCISERVICE","service":"loadbalancer","resource":"<lb-ocid>","category":"access"},"compartmentId":"'$C'"}'

Do the same for VCN flow logs on key subnets, Object Storage write logs on sensitive buckets, and API Gateway execution logs.

2. Collect OS and app logs

Create a custom log with an agent configuration that targets a dynamic group of instances and lists file paths (/var/log/messages, /u01/app/logs/*.log, Windows event channels). Instances need the Custom Logs Monitoring plugin of the Oracle Cloud Agent enabled and a dynamic group policy allowing use log-content.

3. Send to Logging Analytics

Enable Logging Analytics in the region and onboard. Then use a Service Connector to move logs from OCI Logging into a Logging Analytics log group:

oci sch service-connector create --compartment-id $C --display-name logs-to-la \
  --source '{"kind":"logging","logSources":[{"compartmentId":"'$C'","logGroupId":"<lg-ocid>"}]}' \
  --target '{"kind":"loggingAnalytics","logGroupId":"<la-log-group-ocid>"}'

Logging Analytics ships many built-in parsers (Linux syslog, Apache, NGINX, Oracle DB alert log, WebLogic). Write custom parsers only for your own app formats.

4. Search and dashboards

Example queries in Logging Analytics:

'Log Source' = 'OCI Load Balancer Access Logs' and Status >= 500 | timestats count by 'Backend Server'
'Log Source' = 'Linux Syslog Logs' and Severity in (error, critical) | stats count by Entity | sort -count
* | link 'Log Source' | classify 'Message'      -- cluster similar messages to find new error patterns

Build a dashboard per service: error rate, top errors, latency from access logs.

5. Alerts

Create detection rules (scheduled saved searches) that emit metrics to Monitoring, then alarms on those metrics to Notifications (email, Slack via HTTPS, PagerDuty).

6. Retention and cost

  • Logging keeps logs for a configurable period; Logging Analytics charges by storage — set archiving for older data and purge policies.
  • Filter noisy sources (debug logs) at the agent or with Service Connector function tasks.

Tip

Tag entities (hosts, databases) consistently in Logging Analytics so dashboards and searches can filter by app and environment.

Checklist

  • Service logs on for LBs, flow logs, buckets, gateways
  • Custom logs from all instances via agent
  • Service Connector to Logging Analytics; parsers assigned
  • Dashboards and detection rules with alarms
  • Retention, archiving and noise filters set

Related fixes