Safe mode for a GenAI assistant: what to switch to when the model's answers can't be trusted

#The situation

Screenshots of your assistant giving a dangerous instruction, a false policy statement or an offensive reply are circulating. You need to stop it happening again now, but switching the assistant off completely might also cut a support channel thousands of people rely on.

Safe mode gives you steps between "normal" and "off".

#Safe-mode levels

#One flag, several behaviours

SAFE_MODE = flags.get_int("assistant-safe-mode", default=0)   # read per request, short cache

def answer(question: str, user) -> dict:
    if SAFE_MODE >= 5:
        return {"hidden": True}
    if SAFE_MODE == 4:
        return handoff_message(user)
    docs = retrieve(question, user)
    if SAFE_MODE == 3:
        return {"type": "links", "items": [d.link for d in docs[:5]]}
    if SAFE_MODE == 2:
        if not docs or docs[0].score < GROUNDING_MIN_SCORE:
            return {"type": "text", "text": "I couldn't find that in our help articles.", "links": []}
        return grounded_answer(question, docs, temperature=0, require_citations=True)
    if SAFE_MODE == 1:
        if not topic_allowed(question):
            return {"type": "text", "text": "I can only help with questions about our products and services."}
        return generate(question, docs, prompt="system_strict_v3", temperature=0.2)
    return generate(question, docs, prompt="system_v7", temperature=0.5)

Every level must work even if the model provider is misbehaving: levels 3–5 make no model call.

#Stricter filtering for level 1

• Raise content filter severity settings (Azure OpenAI content filtering configuration, Bedrock Guardrails) on a separate deployment or guardrail, and route level 1 traffic to it. Changing filters on the shared deployment affects every app.
• Add an output check: a second classifier or a rule list on the generated text before it's shown.

#Grounded-only (level 2) checks

• Require citations to retrieved passages; drop answers without them.
• Groundedness evaluation on the answer (Azure AI Content Safety groundedness detection, or an LLM-as-judge check) with a strict threshold.
• Temperature 0.

#What to tell users

Agree the wording with the product owner and legal in advance:

• Level 2–3: "Answers are limited to our help articles right now."
• Level 4: "Our assistant is temporarily unavailable. Here's how to reach a person."

Don't pretend nothing changed. Users notice.

#During the incident

1. Set the level (usually 2 or 3 straight away).
2. Collect the problem conversations: prompt, retrieved docs, prompt version, model version.
3. Reproduce in a test environment.
4. Fix (prompt, filter, data source, guardrail) and evaluate with the reproduction cases added to your test set.
5. Step back down one level at a time, watching reports and quality metrics.

Tip

Keep a "golden set" of the incident's bad conversations. Every future release must pass it.

#Checklist

• Levels defined and agreed with product owner
• One flag controls the level; checked per request
• Levels 3–5 make no model calls
• Separate strict deployment/guardrail for level 1
• User messages pre-approved
• Drill done for at least levels 2 and 4