Home / AI & GenAI

AI & GenAITroubleshootingIntermediate

OCI Generative AI: policies, regions and common API errors

Set up IAM policies for OCI Generative AI, call the chat API with the right compartment and endpoint, and fix 404 NotAuthorizedOrNotFound, 400 and 429 errors.

Deepak Bhardwaj2 min read

Applies toOCI Generative AI service (on-demand and dedicated AI clusters), OCI SDK for Python, OCI CLI

1. IAM policy

Without a policy, calls fail with 404 NotAuthorizedOrNotFound. OCI returns the same error whether the resource doesn't exist or you aren't allowed to see it.

allow group GenAI-Users to use generative-ai-family in compartment AI-Prod

For an application running on a compute instance or OKE, use a dynamic group or workload identity instead of user keys:

allow dynamic-group app-instances to use generative-ai-family in compartment AI-Prod

Use manage generative-ai-family only for administrators who create dedicated AI clusters and endpoints.

2. Region and endpoint

The service is available in selected regions only, and model availability differs per region. The inference endpoint has the form:

https://inference.generativeai.<region>.oci.oraclecloud.com

Calling a region where the model isn't offered returns 404 or a model-not-found error. Check the model list in the console for that region.

3. A minimal Python call

import oci
from oci.generative_ai_inference import GenerativeAiInferenceClient
from oci.generative_ai_inference.models import (
    ChatDetails, OnDemandServingMode, GenericChatRequest, UserMessage, TextContent)

config = oci.config.from_file()            # or a signer for instance/resource principals
client = GenerativeAiInferenceClient(config,
    service_endpoint="https://inference.generativeai.us-chicago-1.oci.oraclecloud.com")

resp = client.chat(ChatDetails(
    compartment_id="ocid1.compartment.oc1..aaaa...",
    serving_mode=OnDemandServingMode(model_id="<model name or OCID from the console>"),
    chat_request=GenericChatRequest(
        messages=[UserMessage(content=[TextContent(text="Summarise this incident: ...")])],
        max_tokens=600, temperature=0.2)))
print(resp.data.chat_response.choices[0].message.content[0].text)

Cohere models use CohereChatRequest instead of GenericChatRequest; check the request type for the model family you call.

4. Errors

Status Cause Fix
401 NotAuthenticated Wrong key, fingerprint, tenancy or clock skew Check ~/.oci/config, sync time (NTP)
404 NotAuthorizedOrNotFound Missing policy, wrong compartment, wrong region or model id Section 1–2
400 InvalidParameter Wrong request type for the model, max tokens above the limit, input too long Match the request class to the model; trim the prompt
429 TooManyRequests On-demand limits Retry with backoff; dedicated AI cluster for steady high volume

5. On-demand vs dedicated AI clusters

On-demand is shared capacity billed per use. For predictable latency, fine-tuned models or high steady volume, a dedicated AI cluster with an endpoint gives you reserved capacity. Call it with DedicatedServingMode(endpoint_id=...).

Tip

Use instance or resource principals (oci.auth.signers.InstancePrincipalsSecurityTokenSigner()) for apps on OCI. No API keys on servers means nothing to rotate or leak.

Related fixes