LLM API calls fail with CERTIFICATE_VERIFY_FAILED behind a corporate proxy: CA bundles and proxy settings for the OpenAI SDK, boto3 and Azure SDKs

#Symptom

An LLM call works from your laptop at home, or from a cloud VM, but fails from the office network, a build agent or a locked-down server. Typical errors:

• openai.APIConnectionError: Connection error. with a cause of ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate in certificate chain
• botocore: SSLError: SSL validation failed for https://bedrock-runtime.<region>.amazonaws.com/ ... unable to get local issuer certificate
• azure-identity: ClientAuthenticationError or ServiceRequestError with the same CERTIFICATE_VERIFY_FAILED text when fetching a token from login.microsoftonline.com
• Calls hang and then time out, with no TLS error at all (proxy not being used)

The usual cause is a TLS-inspecting proxy (Zscaler, Netskope, Palo Alto, Blue Coat and similar). It decrypts traffic and re-signs it with a corporate root CA. Your OS trusts that root. Python usually doesn't, because most HTTP libraries ship their own CA bundle (certifi) and ignore the Windows or macOS certificate store.

Caution

Do not "fix" this with verify=False. It turns off certificate checks for traffic that carries API keys and prompts, and it tends to survive into production. Trust the corporate root CA instead.

#Quick checks

#1. Who signed the certificate you actually receive?

openssl s_client -connect api.openai.com:443 -servername api.openai.com -showcerts </dev/null 2>/dev/null \
  | openssl x509 -noout -issuer -subject

If the issuer is your company's CA rather than a public CA, the connection is being inspected. Repeat for <resource>.openai.azure.com, bedrock-runtime.<region>.amazonaws.com and login.microsoftonline.com.

On Windows, PowerShell gives the same answer:

$tcp = New-Object Net.Sockets.TcpClient('api.openai.com',443)
$ssl = New-Object Net.Security.SslStream($tcp.GetStream(),$false,{$true})
$ssl.AuthenticateAsClient('api.openai.com')
$ssl.RemoteCertificate.Issuer
$tcp.Close()

#2. Which CA bundle does Python use?

python -c "import certifi, ssl; print(certifi.where()); print(ssl.get_default_verify_paths())"
python -c "import httpx, openai, botocore; print(httpx.__version__, openai.__version__, botocore.__version__)"

The httpx version matters, see below.

#3. Is a proxy required, and is it set?

env | grep -i -E '^(https?|no|all)_proxy='

Get-ChildItem Env: | Where-Object Name -match 'PROXY'
netsh winhttp show proxy

Many networks use a PAC file that browsers pick up automatically. Python libraries don't read PAC files; they need HTTPS_PROXY set explicitly.

#Symptom → cause → fix

#Fix

#Step 1: Get the corporate root CA as PEM

Ask the network team for the root (and any intermediate) in PEM format, or export it. On Windows:

Get-ChildItem Cert:\LocalMachine\Root | Where-Object Subject -match 'Zscaler|YourCorp' | Select-Object Subject, Thumbprint
$c = Get-Item Cert:\LocalMachine\Root\<THUMBPRINT>
"-----BEGIN CERTIFICATE-----`n" + [Convert]::ToBase64String($c.RawData,'InsertLineBreaks') + "`n-----END CERTIFICATE-----" | Out-File -Encoding ascii C:\certs\corp-root.pem

Build one bundle that trusts both public CAs and the corporate root, so the same file works on and off the corporate network:

cat "$(python -c 'import certifi; print(certifi.where())')" /etc/ssl/corp/corp-root.pem > /etc/ssl/corp/ca-bundle.pem

Warning

Don't append to certifi's own cacert.pem in site-packages. The next pip install --upgrade certifi overwrites it and the error comes back.

#Step 2: OpenAI and Azure OpenAI SDK (httpx)

The OpenAI Python SDK uses httpx. From httpx 0.28, verify no longer accepts a file path string and the SSL_CERT_FILE / SSL_CERT_DIR environment variables are no longer read automatically. Pass an SSL context:

import os, ssl
import httpx
from openai import OpenAI, DefaultHttpxClient

ctx = ssl.create_default_context(cafile=os.environ.get("LLM_CA_BUNDLE", "/etc/ssl/corp/ca-bundle.pem"))

client = OpenAI(
    http_client=DefaultHttpxClient(
        verify=ctx,
        proxy=os.environ.get("HTTPS_PROXY"),   # omit if trust_env picks it up
        timeout=httpx.Timeout(60.0, connect=10.0),
    )
)
print(client.models.list().data[0].id)

AzureOpenAI accepts the same http_client argument. DefaultHttpxClient keeps the SDK's default timeouts and connection limits, so prefer it over a bare httpx.Client. Note that proxies= was removed in httpx 0.28; use proxy=.

An alternative is the truststore package, which makes Python use the operating system's certificate store. If the corporate root is already deployed to Windows or macOS by policy, this needs no PEM file:

import truststore
truststore.inject_into_ssl()   # call once, before creating any clients

from openai import OpenAI
client = OpenAI()

#Step 3: Entra ID tokens and other requests-based SDKs

azure-identity and azure-core use requests by default, which reads REQUESTS_CA_BUNDLE:

export REQUESTS_CA_BUNDLE=/etc/ssl/corp/ca-bundle.pem
python -c "from azure.identity import DefaultAzureCredential; print(DefaultAzureCredential().get_token('https://cognitiveservices.azure.com/.default').expires_on)"

If that prints a timestamp, token acquisition through the proxy works. truststore.inject_into_ssl() also covers requests.

#Step 4: boto3 and Bedrock

export AWS_CA_BUNDLE=/etc/ssl/corp/ca-bundle.pem
export HTTPS_PROXY=http://proxy.corp.example:8080
aws bedrock list-foundation-models --region us-east-1 --query 'modelSummaries[0].modelId'

Or per profile in ~/.aws/config:

[profile genai]
region = us-east-1
ca_bundle = /etc/ssl/corp/ca-bundle.pem

In code you can pass it per client:

import boto3
from botocore.config import Config

brt = boto3.client(
    "bedrock-runtime",
    region_name="us-east-1",
    verify="/etc/ssl/corp/ca-bundle.pem",
    config=Config(proxies={"https": "http://proxy.corp.example:8080"}),
)

#Step 5: Proxy and NO_PROXY

export HTTPS_PROXY=http://proxy.corp.example:8080
export HTTP_PROXY=$HTTPS_PROXY
export NO_PROXY=localhost,127.0.0.1,.internal.corp.example,.privatelink.openai.azure.com,169.254.169.254

Keep the instance metadata address and private endpoint domains in NO_PROXY. Otherwise managed identity and instance role credentials fail, or private endpoint traffic leaves through the internet proxy. NO_PROXY matching rules differ slightly between libraries; test each SDK you use.

#Step 6: Containers and Node.js

In a container, copy the PEM into the image and set the variables there, not on the host:

COPY corp-root.pem /usr/local/share/ca-certificates/corp-root.crt
RUN update-ca-certificates
ENV REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt \
    AWS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt \
    LLM_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt

Node.js apps using the OpenAI or AWS JavaScript SDKs read NODE_EXTRA_CA_CERTS=/path/corp-root.pem at process start.

#Verify

Run one small call per provider from the failing host:

python - <<'EOF'
import os, ssl, httpx
ctx = ssl.create_default_context(cafile=os.environ["LLM_CA_BUNDLE"])
for url in ["https://api.openai.com/v1/models", "https://login.microsoftonline.com/common/v2.0/.well-known/openid-configuration"]:
    r = httpx.get(url, verify=ctx, proxy=os.environ.get("HTTPS_PROXY"), timeout=15)
    print(url, r.status_code)   # 401 on /v1/models without a key still proves TLS works
EOF

A 401 from the models endpoint is fine here: it means the TLS handshake and proxy path worked and only authentication is missing.

#Prevent it next time

• Ship the corporate CA bundle in your base images and set REQUESTS_CA_BUNDLE, AWS_CA_BUNDLE and your own app variable in one place.
• Pin httpx and the OpenAI SDK in lock files and read release notes for SSL changes before upgrading.
• Ask the network team to exclude LLM API endpoints from TLS inspection where policy allows. Inspection can also break streaming responses.
• Add a startup self-check that calls each provider's endpoint and logs the certificate issuer, so a proxy change shows up as one clear log line instead of a vague Connection error.
• Fail builds that contain verify=False with a simple grep in CI.