1. Placement
- Platform (connectivity subscription): hub VNet, firewall, private DNS zones, and the AI gateway (APIM).
- AI platform spoke (shared): Foundry resource(s), shared model deployments, and shared AI Search where appropriate.
- Application spokes: each app's compute (App Service, AKS, Container Apps), its own data stores and its own Foundry projects.
2. Resources and their private endpoints
| Resource | Private endpoint sub-resource | Private DNS zone |
|---|---|---|
| Foundry / AI Services / Azure OpenAI | account | privatelink.cognitiveservices.azure.com, privatelink.openai.azure.com, privatelink.services.ai.azure.com |
| AI Search | searchService | privatelink.search.windows.net |
| Storage (blob) | blob | privatelink.blob.core.windows.net |
| Key Vault | vault | privatelink.vaultcore.azure.net |
| Cosmos DB (agent threads / state, if used) | Sql | privatelink.documents.azure.com |
Link the private DNS zones to the hub (or use DNS Private Resolver) so spokes and on-premises clients resolve the private IPs.
Set public network access = Disabled on every one of these resources.
3. Service-to-service access
Services call each other with managed identities (AI Search to Storage, Foundry to Search, and so on). Allow them through the trusted services setting or shared private links, not by opening public access.
| Identity | Role | On |
|---|---|---|
| App managed identity | Cognitive Services OpenAI User (or Azure AI User) | Foundry / OpenAI |
| App managed identity | Search Index Data Reader | AI Search |
| AI Search managed identity | Storage Blob Data Reader | Storage |
| AI Search managed identity | Cognitive Services OpenAI User | OpenAI (integrated vectorisation) |
Disable local (key) authentication on AI Services and Search (disableLocalAuth: true) once all clients use Entra ID.
4. Policy guardrails
- Deny public network access for Cognitive Services, Search and Storage.
- Allowed locations, and allowed models or deployment SKUs if you need to restrict them.
- Require diagnostic settings to Log Analytics.
- Deny key-based authentication on Cognitive Services.
5. Agents and outbound traffic
If you use agents or tools that call the internet (Bing grounding, web APIs), decide where egress goes: through the firewall with an allow-list. Use the Foundry network-secured agent setup (bring-your-own VNet and resources) for private agent workloads.
6. Operations
- Diagnostic logs from every AI resource to Log Analytics.
- Quota is tracked per subscription, region and model. Plan it (see the GPU and quota planning post).
- Content filters configured per deployment, with any change approved (see the responsible AI post).
Tip
Most "it works from my laptop but not from the app" problems in private Foundry setups are DNS. Run
nslookup <resource>.openai.azure.comfrom the app subnet: it must return a private 10.x address.