Home / OCI

OCIRunbookAdvanced

Sev1: OCI Base Database (VM DB system) out of storage — scale DATA/RECO and buy time safely

Recover an Oracle database on an OCI VM DB system that stops because DATA or RECO storage is full. Confirm which area is full, free space safely, scale the DB system storage online and set alarms so it doesn't happen again.

Deepak Bhardwaj2 min read

Applies toOCI Base Database Service (VM DB systems, ASM or LVM storage), Oracle Database 19c/23ai, OCI CLI 3.x, dbcli

Symptoms

  • ORA-01653 / ORA-01688 (unable to extend), ORA-19809 / ORA-19815 (recovery area full), ORA-00257 (archiver stuck — no new connections).
  • Applications hang on commits; the DB system shows healthy in the console.

1. Which area is full?

-- ASM-based systems
SELECT name, total_mb, free_mb, ROUND(free_mb/total_mb*100,1) pct_free FROM v$asm_diskgroup;

-- Recovery area
SELECT space_limit/1024/1024/1024 limit_gb, space_used/1024/1024/1024 used_gb,
       space_reclaimable/1024/1024/1024 reclaimable_gb FROM v$recovery_file_dest;
SELECT file_type, percent_space_used, percent_space_reclaimable FROM v$recovery_area_usage;

On LVM-based systems, check df -h for /u02 (data) and /u03 (reco).

Full Quick relief Durable fix
RECO (archive logs) Back up and delete archive logs Scale storage, fix backup schedule
RECO limit only (diskgroup has space) Raise db_recovery_file_dest_size Size it properly
DATA Add space to tablespace if diskgroup has room Scale storage, purge/partition data

2. Quick relief for RECO

If backups go to Object Storage via automatic backups, archive logs older than the last backup are reclaimable. Check backups are actually succeeding first.

rman target /
CROSSCHECK ARCHIVELOG ALL;
DELETE NOPROMPT ARCHIVELOG ALL BACKED UP 1 TIMES TO DEVICE TYPE SBT COMPLETED BEFORE 'SYSDATE-1';

Caution

Never delete archive logs that are not backed up, and never delete logs a Data Guard standby still needs. Check v$archived_log applied status for the standby destination first.

Raise the limit if the diskgroup has space:

ALTER SYSTEM SET db_recovery_file_dest_size = 500G SCOPE=BOTH;

3. Scale DB system storage

oci db system get --db-system-id $DBS --query 'data.{data:"data-storage-size-in-gbs", reco:"reco-storage-size-in-gb", state:"lifecycle-state", storage:"db-system-options"."storage-management"}'

oci db system update --db-system-id $DBS --data-storage-size-in-gbs 2048 --wait-for-state AVAILABLE

Storage scaling for VM DB systems is online, but check the documented increments and maximums for your shape and storage type. RECO is sized from DATA on many configurations.

4. DATA: extend the tablespace

SELECT tablespace_name, ROUND(SUM(bytes)/1024/1024/1024,1) gb FROM dba_data_files GROUP BY tablespace_name;
ALTER TABLESPACE app_data ADD DATAFILE '+DATA' SIZE 50G AUTOEXTEND ON NEXT 1G MAXSIZE 200G;

On LVM systems, add the datafile under the data mount path instead of +DATA.

5. Verify

  • Archiver running: SELECT dest_name, status, error FROM v$archive_dest WHERE status <> 'INACTIVE';
  • Apps reconnect; no new ORA- errors in the alert log: adrci exec="show alert -tail 50".

Prevent it next time

  • OCI Monitoring alarms on the DB system storage utilisation metrics (database management / oci_database namespace) at 75% and 90%.
  • Confirm automatic backups succeed daily — failed backups are the usual reason RECO fills.
  • Capacity review monthly: growth per tablespace.