OCI capacity reservations: hold compute capacity for DR and scale-out

#The goal

You have a launch that must succeed at a known moment: a DR failover, a planned scale-out, or a GPU workload in a busy availability domain (AD). A capacity reservation sets host capacity aside for your tenancy, so that launch doesn't depend on what is free at the time.

This post covers creating one, checking what you actually got, using it, and removing it. If a launch is already failing, start with the Out of host capacity runbook instead.

#Check the rules first

Capacity reservations aren't available for Free Tier accounts, confidential computing, preemptible instances, dedicated virtual machine hosts, burstable instances or extended memory VM instances.

Note

The billing percentages come from Oracle's capacity reservation documentation at the time of writing. Check the current terms before you commit a budget to a large reservation.

#Set up access

Allow group capacity-admins to manage compute-capacity-reservations in compartment prod
Allow group app-deployers to use compute-capacity-reservations in compartment prod

manage lets a group create and delete reservations. use is what a group needs to launch instances into one. Separate them so a deployment team can't delete the reservation or add to the bill.

#Create the reservation

See which shapes you can reserve in the AD:

C=ocid1.compartment.oc1..xxxx
AD="Uocm:AP-MUMBAI-1-AD-1"

oci compute capacity-reservation instance-shape list \
  --compartment-id "$C" --availability-domain "$AD" --all --output table

Describe what you want in a file. This example reserves ten 4-OCPU E5 Flex instances, split across two fault domains:

[
  {
    "instanceShape": "VM.Standard.E5.Flex",
    "instanceShapeConfig": { "ocpus": 4, "memoryInGBs": 32 },
    "faultDomain": "FAULT-DOMAIN-1",
    "reservedCount": 5
  },
  {
    "instanceShape": "VM.Standard.E5.Flex",
    "instanceShapeConfig": { "ocpus": 4, "memoryInGBs": 32 },
    "faultDomain": "FAULT-DOMAIN-2",
    "reservedCount": 5
  }
]

Create it:

oci compute capacity-reservation create \
  --compartment-id "$C" \
  --availability-domain "$AD" \
  --display-name cr-dr-web-ad1 \
  --instance-reservation-configs file://reservation.json \
  --wait-for-state ACTIVE

For the DR region, add --region <dr-region> and use an AD name from that region.

#Confirm you got everything

Don't skip this step. A successful create doesn't mean the full count was reserved.

CR=ocid1.capacityreservation.oc1..xxxx

oci compute capacity-reservation get --capacity-reservation-id "$CR" \
  --query 'data.{state:"lifecycle-state",reserved:"reserved-instance-count",used:"used-instance-count"}' \
  --output table

# Per-configuration detail
oci compute capacity-reservation get --capacity-reservation-id "$CR" \
  --query 'data."instance-reservation-configs"'

Compare reserved with the count you asked for. If it is short:

• Try another fault domain or AD for the balance.
• Try a different shape family that your workload can run on.
• Use a separate configuration for the missing part, so the capacity you already hold stays in place.

Warning

oci compute capacity-reservation update removes any capacity configuration you leave out of --instance-reservation-configs. Always start from the current list, then edit it. Omitting a configuration gives that capacity back, and you may not be able to get it again.

# Save the current configs as a starting point for edits
oci compute capacity-reservation get --capacity-reservation-id "$CR" \
  --query 'data."instance-reservation-configs"' > current-configs.json

The saved output uses CLI field names, so convert it to the camelCase format shown earlier before you pass it back. Add the new configuration and keep all the existing ones. Then apply it:

oci compute capacity-reservation update --capacity-reservation-id "$CR" \
  --instance-reservation-configs file://reservation-new.json

#Use the capacity

#Single instance

oci compute instance launch \
  --compartment-id "$C" \
  --availability-domain "$AD" \
  --fault-domain FAULT-DOMAIN-1 \
  --shape VM.Standard.E5.Flex \
  --shape-config '{"ocpus":4,"memoryInGBs":32}' \
  --subnet-id "$SUBNET" \
  --image-id "$IMAGE" \
  --display-name dr-web-01 \
  --capacity-reservation-id "$CR" \
  --wait-for-state RUNNING

#Existing on-demand instance

The reservation needs a matching configuration with unused capacity. Test on a non-production instance first and do this in a change window.

oci compute instance update --instance-id "$ID" --capacity-reservation-id "$CR"

# Take it back out of the reservation
oci compute instance update --instance-id "$ID" --capacity-reservation-id ""

#Instance pools

Put the reservation in the pool's instance configuration:

{
  "instanceType": "compute",
  "launchDetails": {
    "compartmentId": "ocid1.compartment.oc1..xxxx",
    "shape": "VM.Standard.E5.Flex",
    "shapeConfig": { "ocpus": 4, "memoryInGBs": 32 },
    "capacityReservationId": "ocid1.capacityreservation.oc1..xxxx",
    "sourceDetails": { "sourceType": "image", "imageId": "ocid1.image.oc1..xxxx" },
    "createVnicDetails": { "assignPublicIp": false }
  }
}

oci compute-management instance-configuration create \
  --compartment-id "$C" --display-name ic-web-reserved \
  --instance-details file://instance-config.json

#OKE managed node pools

Add capacityReservationId to the placement configuration:

oci ce node-pool create ... --placement-configs '[{
  "availabilityDomain": "Uocm:AP-MUMBAI-1-AD-1",
  "subnetId": "ocid1.subnet.oc1..xxxx",
  "capacityReservationId": "ocid1.capacityreservation.oc1..xxxx"
}]'

#Default reservations

With --is-default-reservation true, matching launches in that AD use the reservation automatically. The rules:

• One default reservation per AD, and it must be in the root compartment.
• Users need use permission on it.
• If it can't be used, the instance falls back to on-demand capacity.

A launch can opt out by passing --capacity-reservation-id "".

Tip

Use explicit reservations for DR. A default reservation lets any matching launch consume capacity meant for failover. The fallback also hides the problem, because the launch still succeeds.

#Find waste and clean up

Each unused slot costs 85% of a running instance. If you reserve 10 and run 4, you pay the standard rate for the 4 and 85% for the other 6. Review this monthly:

oci compute capacity-reservation list --compartment-id "$C" --all \
  --query 'data[].{name:"display-name",ad:"availability-domain",reserved:"reserved-instance-count",used:"used-instance-count"}' \
  --output table

# What is running in a reservation
oci compute capacity-reservation instance list \
  --capacity-reservation-id "$CR" --all --output table

Stopped instances keep their reserved capacity, so stopping one doesn't free a slot. Terminating an instance returns the capacity to the reservation.

To delete a reservation, first terminate its instances or move them out:

oci compute capacity-reservation delete --capacity-reservation-id "$CR"

Caution

Deleting a reservation gives the capacity back straight away. Recreating it later only gets what is free at that moment, which may be nothing in a constrained AD. Confirm that no DR plan or pool depends on it first.

#Common problems

#Prevent it next time

• Create DR reservations when you build the DR site, not during the incident.
• Reserve in the DR region, and run a test launch into the reservation in each DR drill.
• Tag reservations with an owner and a review date, and review reserved against used monthly.
• Check your service limits as well. A reservation holds host capacity, but it doesn't replace your limits and quotas planning.