Redact PII from LLM prompts and logs with Presidio

#Goal

Tickets, chat transcripts and log lines that you send to an LLM often contain customer names, email addresses, phone numbers, card numbers and internal account IDs. The same text then lands in prompt logs, traces and evaluation datasets. This post sets up one redaction step that runs before the LLM call and before anything is logged.

The approach:

1. Detect PII with Presidio Analyzer (NER model plus pattern recognizers).
2. Replace it with Presidio Anonymizer, either permanently (for logs) or with placeholders you can swap back (for the user-facing answer).
3. Test it against real samples, because detection is probabilistic.

#Install

python -m venv .venv && . .venv/bin/activate
pip install presidio_analyzer presidio_anonymizer
python -m spacy download en_core_web_lg

Presidio is now maintained under the data-privacy-stack organisation. Containers are published to GitHub Container Registry; the older mcr.microsoft.com/presidio-* images are no longer updated.

docker run -d -p 5002:3000 ghcr.io/data-privacy-stack/presidio-analyzer:<release-tag>
docker run -d -p 5001:3000 ghcr.io/data-privacy-stack/presidio-anonymizer:<release-tag>

curl -s -X POST http://localhost:5002/analyze \
  -H "Content-Type: application/json" \
  -d '{"text": "Call Priya Shah on +44 20 7946 0958", "language": "en"}'

Tip

Pin a release tag instead of latest. A new model or recognizer version changes what gets detected, and you want that change to go through testing like any other release.

#Detect and mask in Python

from presidio_analyzer import AnalyzerEngine, PatternRecognizer, Pattern
from presidio_anonymizer import AnonymizerEngine
from presidio_anonymizer.entities import OperatorConfig

analyzer = AnalyzerEngine()          # loads en_core_web_lg; create once, reuse
anonymizer = AnonymizerEngine()

# Internal identifiers the built-in recognizers don't know about
analyzer.registry.add_recognizer(PatternRecognizer(
    supported_entity="CUSTOMER_ID",
    patterns=[Pattern(name="customer_id", regex=r"\bCUST-\d{6}\b", score=0.9)],
))

ENTITIES = ["PERSON", "EMAIL_ADDRESS", "PHONE_NUMBER", "CREDIT_CARD",
            "IBAN_CODE", "IP_ADDRESS", "CUSTOMER_ID"]

def redact(text: str) -> str:
    results = analyzer.analyze(text=text, entities=ENTITIES,
                               language="en", score_threshold=0.4)
    return anonymizer.anonymize(
        text=text,
        analyzer_results=results,
        operators={
            "DEFAULT": OperatorConfig("replace", {"new_value": "<REDACTED>"}),
            "CREDIT_CARD": OperatorConfig("mask", {"masking_char": "*",
                                                   "chars_to_mask": 15,  # keeps the last 4 of a spaced 16-digit number
                                                   "from_end": False}),
            "EMAIL_ADDRESS": OperatorConfig("hash", {"hash_type": "sha256"}),
        },
    ).text

print(redact("Priya Shah (CUST-482913, priya@contoso.com) says card 4111 1111 1111 1111 was charged twice."))

Operators you will use most:

Hashing an email without a salt is not anonymisation: anyone with a list of addresses can hash them and match. Treat hashed values as personal data.

#Reversible placeholders for the LLM round trip

If the model must refer to the customer in its answer ("Hi Priya, your refund..."), replace each value with a numbered placeholder, send that to the LLM, and swap the real values back into the response only.

def pseudonymize(text: str):
    mapping = {}                      # (entity_type, value) -> placeholder

    def operator_for(entity_type):
        def to_token(value: str) -> str:
            key = (entity_type, value)
            if key not in mapping:
                mapping[key] = f"<{entity_type}_{len(mapping) + 1}>"
            return mapping[key]
        return OperatorConfig("custom", {"lambda": to_token})

    results = analyzer.analyze(text=text, entities=ENTITIES,
                               language="en", score_threshold=0.4)
    masked = anonymizer.anonymize(
        text=text,
        analyzer_results=results,
        operators={e: operator_for(e) for e in ENTITIES},
    ).text
    reverse = {token: value for (_, value), token in mapping.items()}
    return masked, reverse

def restore(answer: str, reverse: dict) -> str:
    for token, value in reverse.items():
        answer = answer.replace(token, value)
    return answer

masked, reverse = pseudonymize(ticket_text)
answer = call_llm(system_prompt, masked)       # your existing client code
log.info("llm_call prompt=%s answer=%s", masked, answer)   # only masked text is logged
return restore(answer, reverse)

Keep reverse in memory for the request only. Don't write it to logs, caches or traces, or you have rebuilt the PII store you were trying to avoid.

Warning

Add an instruction to the system prompt such as "Keep placeholders like exactly as written." Models sometimes reformat placeholders (PERSON_1, <Person 1>), and then restore silently leaves them in the answer. Check for leftover <..._n> patterns and fall back to a generic greeting if any remain.

#Redact logs at the source

Application logs and LLM traces are where PII usually leaks, because they're kept longest and read by the most people. Add a logging filter so nothing unredacted is written, even by code paths you didn't review.

import logging

class PiiRedactingFilter(logging.Filter):
    def filter(self, record: logging.LogRecord) -> bool:
        record.msg = redact(record.getMessage())
        record.args = ()
        return True

handler = logging.StreamHandler()
handler.addFilter(PiiRedactingFilter())
logging.getLogger("llm").addHandler(handler)

The NER model adds latency to every log line. Apply the filter to the logger that records prompts and responses, not to the root logger of a high-volume service. For OpenTelemetry traces, redact before you set span attributes that hold prompt or completion text.

#Test it before you trust it

Detection misses things (unusual names, non-English text, IDs in odd formats) and over-matches others (product names tagged as PERSON). Build a small labelled set from real, approved samples and measure it.

samples = [
    ("Reset MFA for John O'Neill, john.oneill@fabrikam.com", ["PERSON", "EMAIL_ADDRESS"]),
    ("Server 10.20.4.17 is down, ticket for CUST-000123", ["IP_ADDRESS", "CUSTOMER_ID"]),
]
for text, expected in samples:
    found = {r.entity_type for r in analyzer.analyze(text=text, entities=ENTITIES, language="en")}
    missing = set(expected) - found
    print("MISS" if missing else "ok  ", sorted(missing), text[:50])

#Prevent it next time

• Put redaction in the shared LLM client or gateway, so every app gets it without opting in.
• Run the labelled test set in CI and fail the build if the miss rate rises after a Presidio, spaCy or regex change.
• Combine with the provider's controls where available, such as Amazon Bedrock Guardrails sensitive information filters or Azure AI Language PII detection, but keep your own redaction for logs, which provider filters don't cover.
• Set short retention on prompt and response logs, and restrict who can read them.
• Review a sample of redacted logs each month. New ticket templates and ID formats appear without anyone telling the platform team.