Advisory: a well-architected review for GenAI workloads - questions per pillar for production readiness

#Reliability

• Is there a fallback when the primary model/region is unavailable or rate-limited (see the HA endpoints and fallback posts)?
• Are timeouts, retries with backoff and circuit breakers in place?
• Can the RAG index be rebuilt or restored, and how long does it take (see the vector index DR post)?
• Is there a degraded mode (safe mode, cached answers, human handoff)?

#Security

• Has the app been threat-modelled for LLM risks (see the OWASP threat model post)?
• Are identities used instead of API keys? Are endpoints private?
• Does retrieval enforce user permissions?
• Are tools least-privilege, with approvals for risky actions?
• Are prompts and logs free of secrets and redacted for PII?

#Cost

• Is cost tracked per feature/team with budgets and alerts?
• Are model sizes chosen by evaluation (routing for mixed traffic)?
• Are prompt caching, output limits and retrieval budgets tuned?
• Is there a cost kill switch?

#Performance

• Are latency SLOs defined (TTFT, total) and met?
• Is streaming working end to end?
• Is capacity (quota/PTU/GPU) planned for peaks?

#Operational excellence

• Are prompts, configs and evaluation sets versioned and deployed through CI/CD?
• Is tracing in place for prompts, retrieval and tool calls (with sampling)?
• Are runbooks ready for model retirement, quality regression, injection incidents and outages?
• Can you roll back to the last good prompt/model in one step?

#AI quality and lifecycle

• Is there an offline evaluation set with release gates?
• Are online quality signals reviewed regularly?
• Are model versions pinned, with a process for upgrades and retirements?
• Is ingestion freshness monitored for RAG sources?

#Responsible AI

• Is the use case registered with a risk tier and approvals?
• Are users told they're interacting with AI and how to report problems?
• Are content safety controls configured and tested?
• Are kill switches and their owners defined (see the kill switch series)?

#Running the review

Tip

Make "eval set exists and gates releases" and "rollback in one step" go-live blockers. They prevent most avoidable GenAI production incidents.

#Checklist

• All pillars plus AI quality and responsible AI covered
• Evidence attached
• Risks rated
• Top actions owned and dated
• Re-review scheduled