Symptoms
- A user reports seeing salary data, another customer's files or confidential documents in an answer or citation.
- Citations link to documents the user can't open directly.
1. Contain immediately
- Disable the assistant, or switch it to a restricted index that contains only public/all-staff content.
- If one source (for example an HR SharePoint site) is involved, remove it from retrieval now:
POST https://<search>.search.windows.net/indexes/docs/docs/index?api-version=2024-07-01
api-key: <admin-key>
Content-Type: application/json
{ "value": [ { "@search.action": "delete", "id": "<doc-chunk-id>" } ] }
For bulk removal, delete by source in your ingestion tool or rebuild the index without that source.
Caution
Treat this as a personal data or confidentiality incident from the start. Involve security and the data protection officer early; there may be legal deadlines for notification.
2. Find how trimming failed
Typical design: each chunk stores allowed groups or tenant ID, and every query adds a filter from the user's identity. Check each link:
| Check | Typical bug |
|---|---|
| Is the filter applied on every query path? | A new feature (summaries, "related docs", agent tool) queries without the filter |
| Is the filter built from the authenticated identity? | Taken from a request parameter the client can change |
| Are ACLs in the index correct? | Ingestion copied "Everyone" or lost permissions on folder moves |
| Group membership up to date? | Cached group list; user removed from group still sees content |
| Empty filter handling | No groups → filter omitted → everything returned |
Example of a fail-open bug and the fix:
# BAD: no groups means no filter at all
flt = f"group_ids/any(g: search.in(g, '{','.join(groups)}'))" if groups else None
# GOOD: no groups means nothing restricted is visible
if not groups:
groups = ["__none__"]
flt = f"group_ids/any(g: search.in(g, '{','.join(groups)}'))"
Also check whether the LLM tool layer (agents) can call search without the user's context.
3. Measure the exposure
From retrieval logs or traces, list queries where returned chunk ACLs don't include the requesting user:
SELECT r.request_id, r.user_id, r.created_at, c.doc_id, c.acl_groups
FROM retrieval_log r
JOIN retrieval_log_chunks c ON c.request_id = r.request_id
WHERE r.created_at >= '2026-10-20'
AND NOT (c.acl_groups && r.user_groups) -- PostgreSQL array overlap
AND NOT ('all-staff' = ANY(c.acl_groups));
The output is the incident's exposure list: which users saw which documents. If you don't log retrieved chunk IDs, say so in the report and add that logging now.
4. Fix and verify
- Enforce the filter in one server-side function that every retrieval path must call; fail closed.
- Re-sync ACLs from the source system and re-index affected sources.
- Write automated tests: for a set of test users, assert that restricted documents never appear for queries designed to retrieve them.
def test_hr_docs_hidden_from_engineering():
res = retrieve("what is the salary band for level 5", user=ENGINEERING_TEST_USER)
assert all("hr-restricted" not in c.acl_groups for c in res)
5. Re-enable
Bring sources back one at a time, starting with low-sensitivity ones, after the tests pass and security signs off.
Prevent it next time
- Fail-closed filters enforced server-side, never built from client input.
- Log retrieved chunk IDs and ACLs per request (without full text) for audit.
- Red-team new features that add retrieval paths, including agent tools.
- Keep highly sensitive sources out of general assistants; use separate indexes.