Why teams move
- Identity: Entra ID and managed identities instead of shared API keys.
- Network: private endpoints, no public egress needed.
- Data handling and region choice under your Azure agreements (check current Azure OpenAI data-privacy terms).
- Billing, quotas and policies in your Azure subscription.
1. Inventory current use
Scan repos and secrets stores for api.openai.com, OPENAI_API_KEY and SDK usage:
git grep -n -E "api\.openai\.com|OPENAI_API_KEY|from openai import|new OpenAI\(" -- '*.py' '*.ts' '*.js' '*.cs'
For each app record: models used, features (chat, embeddings, Assistants, Batch, fine-tuned models), volume (tokens/day) and peak RPM.
2. Create resources and deployments
In Azure, you call a deployment (your name) of a model version:
| OpenAI model name | Azure deployment |
|---|---|
gpt-4o | chat-main → gpt-4o, version pinned |
text-embedding-3-large | embed-large → text-embedding-3-large |
Choose deployment type (Standard / Global / Data Zone / Provisioned) by data residency and capacity needs, and request quota for peak tokens per minute. Feature availability (latest models, some APIs) can lag or differ by region. Check before committing.
3. Change the client
Recent OpenAI SDKs support Azure via the /openai/v1/ endpoint style or the AzureOpenAI client. Example with Entra ID:
from openai import AzureOpenAI
from azure.identity import DefaultAzureCredential, get_bearer_token_provider
token_provider = get_bearer_token_provider(DefaultAzureCredential(), "https://cognitiveservices.azure.com/.default")
client = AzureOpenAI(
azure_endpoint="https://aoai-prod.openai.azure.com",
azure_ad_token_provider=token_provider,
api_version="2024-10-21", # use a current GA version
)
resp = client.chat.completions.create(model="chat-main", messages=[{"role": "user", "content": "Hello"}])
model= takes the deployment name, not the OpenAI model name. Check the SDK docs for the current recommended pattern; the Azure endpoint and API versioning have evolved.
Grant the app's managed identity Cognitive Services OpenAI User on the resource, then disable key auth when all apps are moved.
4. Things that differ
| Area | Watch for |
|---|---|
| Content filtering | Azure applies content filters by default; legitimate prompts may be blocked; tune filter configs where allowed |
| Rate limits | Per-deployment TPM/RPM quota; handle 429 with retries |
| Features | Some APIs (e.g. Assistants, Batch, Realtime) have different availability or API versions |
| Fine-tuned models | Re-fine-tune on Azure; OpenAI fine-tunes don't transfer |
| Tokens and output | Same model family → same behaviour, but verify with your eval set |
5. Put a gateway in front (recommended)
Route all apps through API Management: one endpoint, per-app subscription keys or identities, token limits, logging and multi-region failover.
6. Cut over app by app
- Dev/test environment on Azure first; run the app's evaluation and integration tests.
- Production behind a feature flag or config switch; watch errors, latency, filter blocks.
- Revoke the old OpenAI keys after the rollback window; remove them from secret stores.
Tip
Set up cost tracking per app from day one (gateway token metrics with app dimension). Without it, moving to one shared Azure resource makes cost allocation harder, not easier.
Checklist
- All OpenAI usage inventoried
- Deployments, versions and quota in place
- Clients on Entra ID / managed identity
- Content filter and feature differences tested
- Gateway, cost attribution, old keys revoked