Home / OCI

OCIHow-toBeginner

OCI Object Storage bulk transfers - fast uploads with multipart, bulk-upload and sync from the OCI CLI

Moving many files or very large files into OCI Object Storage is slow with one-by-one put. Use multipart uploads, bulk-upload with parallelism, and sync for repeat runs, then verify with checksums and counts.

Deepak Bhardwaj4 min read

Applies toOCI Object Storage (Standard and Infrequent Access tiers); OCI CLI 3.x on Linux and Windows

Pick the right command

Task Command
One file, any size oci os object put (multipart is automatic above a threshold)
A folder of files, one-off oci os object bulk-upload
Keep a folder and a bucket in step, repeatedly oci os object sync
Download a whole prefix oci os object bulk-download
Copy between buckets/regions oci os object copy (server side)

Get your namespace first. Many commands need it:

oci os ns get --query data --raw-output

Large single files (multipart)

oci os object put -bn backups --file /data/db_full_20261026.dmp \
  --name db/2026/10/db_full_20261026.dmp \
  --part-size 128 --parallel-upload-count 8 \
  --storage-tier InfrequentAccess
  • --part-size is in MiB. Larger parts mean fewer requests. Increase it for very large files, because there's a maximum number of parts per object (check the current limit).
  • --parallel-upload-count sets how many parts go at once. Raise it until your network link is full.
  • If the upload is interrupted, list and clean up incomplete multipart uploads, which otherwise keep using storage:
oci os multipart list -bn backups --all
oci os multipart abort -bn backups --object-name db/2026/10/db_full_20261026.dmp --upload-id <id>

Tip

Add a lifecycle rule that aborts uncommitted multipart uploads after a few days, so failed uploads don't quietly cost money.

Many files (bulk-upload)

oci os object bulk-upload -bn app-archive --src-dir /data/archive \
  --object-prefix archive/2026/ \
  --parallel-upload-count 16 \
  --include "*.csv" --include "*.gz" \
  --no-overwrite
Option Use
--object-prefix Put everything under a "folder"
--include / --exclude Glob filters, repeatable
--no-overwrite Skip objects that already exist (good for retries)
--overwrite Replace without prompting
--dry-run Show what would be uploaded

The summary at the end lists uploaded, skipped and failed files. Re-run with --no-overwrite to retry only the failures.

Repeat runs (sync)

sync compares the local directory with the bucket and uploads only new or changed files:

oci os object sync -bn app-archive --src-dir /data/archive --prefix archive/ --dry-run
oci os object sync -bn app-archive --src-dir /data/archive --prefix archive/

With --delete, objects that no longer exist locally are removed from the bucket:

oci os object sync -bn app-archive --src-dir /data/archive --prefix archive/ --delete --dry-run

Caution

Always run --delete with --dry-run first. A wrong --src-dir (for example an empty mount) would delete the whole prefix. Enable versioning on buckets where this risk matters.

Download and copy

# whole prefix to a local folder
oci os object bulk-download -bn app-archive --prefix archive/2026/ --download-dir /restore --parallel-operations-count 16

# server-side copy to another region (no data through your machine)
oci os object copy -bn app-archive --source-object-name archive/2026/big.gz \
  --destination-region eu-frankfurt-1 --destination-bucket app-archive-dr
oci os work-request get --work-request-id <id-from-output>

Verify

# count and total size under a prefix
oci os object list -bn app-archive --prefix archive/2026/ --all \
  --query "{count: length(data), bytes: sum(data[].size)}"

# compare with local
find /data/archive -type f | wc -l
du -sb /data/archive

For single-part objects, oci os object head returns an opc-content-md5 you can compare with openssl dgst -md5 -binary file | base64. Multipart objects return a multipart MD5 (opc-multipart-md5) instead, which isn't the MD5 of the whole file.

Speed troubleshooting

Symptom Check
Slow from an OCI instance Use a service gateway so traffic stays on the OCI network
Slow from on-premises FastConnect vs internet; raise parallelism; check the proxy
Many small files are slow Raise --parallel-upload-count; tar small files first if you can
429 errors Too much parallelism for the bucket; lower it and retry
Auth errors on a long run Session token expired; use API key or instance principal auth for automation

For automation on an OCI instance, use instance principal authentication rather than a user API key:

oci os object sync -bn app-archive --src-dir /data/archive --auth instance_principal

That needs a dynamic group and a policy such as Allow dynamic-group archivers to manage objects in compartment data where target.bucket.name='app-archive'.

Checklist

  • Right command for the job (put, bulk-upload, sync, copy)
  • Part size and parallelism tuned
  • Lifecycle rule to abort stale multipart uploads
  • --dry-run before any --delete
  • Counts and sizes verified
  • Instance principal for scheduled transfers

Related fixes