Home / OCI

OCITroubleshootingIntermediate

OCI Functions invoke failing with 502, 504 or 429: match the error code to the fix

Function calls fail with 502, 504 or 429 and the status alone doesn't explain why. Read the error code, check logs and metrics, then fix the image pull, subnet, timeout, memory or throttling cause.

Deepak Bhardwaj8 min read

Applies toOCI Functions (container-image functions), OCI CLI, Fn Project CLI, OCI Registry, OCI Logging, OCI Monitoring

Symptom

A function invocation, or the API Gateway route, Events rule or Connector Hub flow in front of it, fails. The client shows a status such as 502 or 504. The same status covers many unrelated causes, so don't start with the code. Get the error code first.

1. Get the exact error

Invoke the function directly. This removes API Gateway and other callers from the path.

export FN=<function-ocid>

oci fn function invoke --function-id "$FN" --file "-" --body ''

--body is required even when empty. Synchronous CLI calls stop at the global --read-timeout, 60 seconds by default, so add --read-timeout 310 for slow functions.

With the Fn CLI, turn on debug output. DEBUG must be uppercase:

DEBUG=1 fn invoke orders-app enrich-order

Failures come back in this form:

{"code":"FunctionInvokeImageNotAvailable","message":"Failed to pull function image"}

2. Match the code to the cause

Status Code or message Likely cause Fix
502 FunctionInvokeExecutionFailed ("function failed") Bug or unhandled exception in your code Read the function logs
502 FunctionInvokeImageNotAvailable Image missing, or no route to the registry Check the image and subnet (section 5)
502 FunctionInvokeSubnetOutOfIPs Subnet has no free IPs Free or add address space (section 6)
502 FunctionInvokeSubnetNotAvailable Subnet or DHCP options deleted or not authorized Restore the resource or point the application at a valid subnet
502 FunctionInvokeSubnetConfigError Custom DNS resolver unreachable or broken Fix the resolver in the VCN
502 FunctionInvokeSyslogUnavailable Application syslog URL invalid or unreachable Fix or clear the URL (section 8)
502 FunctionInvokeResponseBodyTooLarge Response over 6 MB Return less, or write to Object Storage and return a reference
502 FunctionInvokeContainerInitFail Container didn't start, often an old FDK Update the FDK, read the logs
502 FunctionInvokeTooManyMatchingDGs Function matches too many dynamic groups Narrow the matching rules (section 9)
503 FunctionInvokeServiceUnavailable Service capacity Retry with jittered backoff. For GENERIC_ARM applications, consider GENERIC_X86 or GENERIC_X86_ARM
504 FunctionInvokeTimeout Run exceeded the function timeout Raise the timeout or go detached (section 7)
504 FunctionInvokeContainerInitTimeout Memory too low or slow container start Raise memory (section 7)
504 FunctionInvokeImagePullTimeout Image too large or slow to pull Shrink the image
413 FunctionInvokeRequestContentTooLarge Request over 6 MB Send a reference instead of the payload
429 TooManyRequests Tenancy at its concurrent memory limit Back off and check the limit (section 8)
444 No message, client sees a timeout Connection interrupted during execution Invoke again

A 404 with NotAuthorizedOrNotFound usually means a wrong function OCID or an invoker without permission. A human or group caller needs use fn-invocation:

Allow group app-operators to use fn-invocation in compartment orders

3. Check function and application settings

oci fn function get --function-id "$FN" --query \
  'data.{state:"lifecycle-state",image:image,digest:"image-digest",memory:"memory-in-mbs",timeout:"timeout-in-seconds",app:"application-id"}'

export APP=<application-ocid>

oci fn application get --application-id "$APP" --query \
  'data.{state:"lifecycle-state",subnets:"subnet-ids",shape:shape,syslog:"syslog-url"}'

4. Read the logs and metrics

Invocation logs are off until you enable them. In the Console, open the application, select Monitoring, find Function Invocation Logs in the Logs section and choose Enable Log from the actions menu. Logs show default start and end messages, plus whatever your code prints.

oci logging-search search-logs \
  --time-start "$(date -u -d '-1 hour' +%FT%TZ)" \
  --time-end "$(date -u +%FT%TZ)" \
  --search-query 'search "<compartment-ocid>/<log-group-ocid>/<log-ocid>" | where subject = '"'"'enrich-order'"'"' | sort by datetime desc'

subject holds the function name and source holds the application name.

Metrics need no setup and show which errors are most frequent:

oci monitoring metric-data summarize-metrics-data -c "$C" --namespace oci_faas \
  --query-text 'FunctionResponseCount[5m]{responseType = "Error"}.groupBy(ErrorCode).sum()' \
  --start-time "$(date -u -d '-6 hours' +%FT%TZ)" \
  --query 'data[].{code:dimensions.ErrorCode,total:sum("aggregated-datapoints"[].value)}'

5. Image pull failures

First confirm the image and digest from section 3 exist in the registry:

oci artifacts container image list -c "$REPO_COMPARTMENT" \
  --repository-name orders/enrich-order --all \
  --query 'data.items[].{name:"display-name",digest:digest}' --output table

Then check the path from the application's subnet:

oci network subnet get --subnet-id "$SUBNET" --query \
  'data.{routeTable:"route-table-id",securityLists:"security-list-ids"}'

oci network route-table get --rt-id "$RT" --query 'data."route-rules"'
Subnet type Route rule needed Egress needed
Public 0.0.0.0/0 to an internet gateway Stateful egress to All <region> Services In Oracle Services Network
Private All <region> Services In Oracle Services Network to a service gateway Same egress rule

If the image and network are both correct, compare your policies with the Policy Builder template "Let users create, deploy, and manage functions and applications". Older Oracle tutorials also grant the service read repos access. The exact statements have changed over time, so check the current template rather than copying one from an old post.

6. Subnet out of IPs

Oracle's sizing guideline is one IP per 14 GB of concurrent function memory, plus 3 IPs reserved per subnet:

required IPs = (3 x subnets) + (sum of memory GB x max concurrent executions per function) / 14

Example: 512 MB at 2,000 concurrent executions is 1,000 GB. That needs 72 IPs plus 3, so 75 in total, and a /25 is enough. Oracle says this can overestimate, and other services in the subnet need addresses too. Check what else is consuming the range, and confirm who owns those resources before you remove anything.

7. Timeouts and memory

Setting Default Allowed values
Timeout (sync) 30 seconds Up to 300 seconds
Timeout (detached) Same as sync value 5 to 3,600 seconds
Memory 128 MB 128, 256, 512, 1024, 2048, 3072 MB
oci fn function update container-function --function-id "$FN" \
  --memory-in-mbs 512 --timeout-in-seconds 120

The value set on the service takes precedence over func.yaml, so check the live value with the get command from section 3 instead of trusting the file. Oracle recommends at least 256 MB for Java functions.

Work that can't finish in 300 seconds doesn't fit a sync call. Use a detached invocation and a longer detached timeout:

oci fn function invoke --function-id "$FN" --file "-" --body '{}' --fn-invoke-type detached

Caution

Retrying a timed-out function can repeat work that already completed, because the function may have changed data before the timeout. Make handlers idempotent before adding automatic retries to anything that writes.

8. Throttling and syslog

For 429s, check how much concurrent memory is allocated, using AllocatedTotalConcurrency. The limit is shared across the tenancy, so one busy function can throttle others. Retry with backoff, and ask Oracle Support to raise the limit if it persists.

To test whether syslog is the cause of FunctionInvokeSyslogUnavailable, clear the URL:

fn update app orders-app --syslog-url ''

Warning

This stops log delivery to that endpoint. Record the old URL from section 3 and restore it after you fix the endpoint or the subnet's outbound access.

9. Function code gets NotAuthorizedOrNotFound

The invocation works, but the function's SDK calls fail. Check that a dynamic group matches the function and that a policy grants that group access:

ALL {resource.type = 'fnfunc', resource.compartment.id = '<compartment-ocid>'}

Allow dynamic-group orders-functions to read secret-bundles in compartment orders

Keep matching rules narrow. A function matching too many dynamic groups fails with FunctionInvokeTooManyMatchingDGs.

Prevent it next time

  • Enable invocation logs when you create each application, and print useful context in handlers.
  • Alarm on errors by code, not only on total invocations:
FunctionResponseCount[5m]{responseType = "Error"}.groupBy(ErrorCode).sum() > 0
  • Alarm on responseType = "Throttled" for functions that users depend on.
  • Size function subnets with the formula above and revisit them when concurrency grows.
  • Don't delete registry images that a live function still references.