Detect PII with Presidio and Google Sensitive Data Protection

Introduction

When a user pastes a customer email, a support transcript, or a receipt screenshot into an LLM prompt, raw PII can reach model context, provider logs, and downstream stores within milliseconds. Teams that ship without a dedicated detection layer typically discover the leak only when a regulator or end customer reports a name, phone number, or credit card surfacing in a model response — by which point the data has already left the perimeter. This lesson shows how to detect personally identifiable information in text and images by combining Microsoft Presidio with Google Sensitive Data Protection (SDP). You will learn how Presidio's AnalyzerEngine runs a registry of recognizers over text to emit RecognizerResult entities with confidence scores, and how Google SDP's managed infoType catalog covers entities — including image-based PII — that Presidio cannot. By the end you will be able to wire both systems into a single PII protection pipeline that handles real-time inline scanning and batch compliance audits.

Key Terminology

• AnalyzerEngine: Presidio's top-level entry point that orchestrates a registry of recognizers over input text and returns a list of RecognizerResult detections.
• RecognizerResult: Presidio's output object containing the detected entity_type, start/end character offsets, and a confidence score between 0.0 and 1.0.
• infoType: Google SDP's named identifier for a class of sensitive data (such as EMAIL_ADDRESS or CREDIT_CARD_NUMBER) that the managed API knows how to detect in text or images.

Concepts

Why Dual Detection Systems

No single PII detection system covers every entity type, format, and modality. Microsoft Presidio excels at real-time text scanning with zero API costs and supports custom recognizer extensions. Google Sensitive Data Protection (SDP) provides broader coverage with over 150 built-in infoTypes and uniquely supports image-based PII detection for receipts, screenshots, and scanned documents. A production PII protection pipeline uses both: Presidio for real-time inline scanning where latency matters, and SDP for batch compliance audits and image scanning where thoroughness outweighs speed.

Code Walkthrough

Building on the dual-detection design above, the code below stands up each engine — Presidio's AnalyzerEngine for inline text and Google SDP for managed text/image inspection — and shows what each returns.

Microsoft Presidio Architecture

Presidio's detection engine is the presidio-analyzer, which finds PII entities in text using a pipeline of recognizers. The analyzer consults a registry of recognizers, each responsible for detecting specific entity types (person names, phone numbers, credit cards, and so on), and returns a span and confidence score for every match.

Installing and Configuring Presidio

The implementation below initializes a presidio-analyzer with its default recognizer registry, then scans a sample string for a set of PII entity types and prints each detected entity with its type, confidence score, and character span.

Code snippetpython
1# Install Presidio components
2# pip install presidio-analyzer spacy
3# python -m spacy download en_core_web_lg
4
5from presidio_analyzer import AnalyzerEngine, RecognizerRegistry
6
7# Initialize the analyzer with default recognizers
8analyzer = AnalyzerEngine()
9
10# Analyze text for PII entities
11text = "John Smith called from 555-123-4567 about order #12345"
12results = analyzer.analyze(
13    text=text,
14    entities=["PERSON", "PHONE_NUMBER", "EMAIL_ADDRESS",
15              "CREDIT_CARD", "US_SSN", "IP_ADDRESS"],
16    language="en"
17)
18
19for result in results:
20    print(f"Entity: {result.entity_type}, "
21          f"Score: {result.score:.2f}, "
22          f"Start: {result.start}, End: {result.end}, "
23          f"Text: {text[result.start:result.end]}")

• Install comments — the commented pip install presidio-analyzer spacy and python -m spacy download en_core_web_lg lines, noting the spaCy English model the analyzer relies on for PERSON detection.
• Import — pulls AnalyzerEngine and RecognizerRegistry from presidio_analyzer.
• analyzer = AnalyzerEngine() — instantiates the analyzer with the default recognizer registry; no custom configuration required.
• text — the sample input string containing a person name and a phone number for the analyzer to scan.
• analyzer.analyze(...) — runs detection for the target entity types with language="en", returning a list of RecognizerResult objects.
• for result in results — iterates the detections and prints each one's entity_type, confidence score, character span, and the underlying text slice.

The analyzer returns RecognizerResult objects containing the entity type, confidence score (0.0 to 1.0), and character positions in the source text. Default recognizers handle standard PII types:

Each RecognizerResult carries a confidence score between 0.0 and 1.0 reflecting how certain the contributing recognizer is. SpaCy NER scores depend on the model's training, regex-only recognizers assign fixed scores based on pattern strength and surrounding context, and checksum-validated patterns like credit card numbers receive high scores (0.85–1.0) when the Luhn algorithm confirms validity. Downstream code should filter on a score_threshold (typically 0.5 in production) before acting on a detection.

Google Sensitive Data Protection Configuration

Google SDP (formerly Cloud DLP) provides a managed API for detecting sensitive data. Unlike Presidio, SDP requires no local model deployment and supports image inspection:

The create_dlp_client and inspect_text functions below call Google SDP: inspect_text submits the text with the infoTypes to look for and returns the API's findings — each carrying an info_type, a likelihood, and the matched quote.

Code snippetpython
1import google.cloud.dlp_v2 as dlp
2
3def create_dlp_client():
4    """Create a Google Cloud DLP client."""
5    return dlp.DlpServiceClient()
6
7def inspect_text(client, project_id: str, text: str) -> list:
8    """Inspect text for PII using Google SDP."""
9    parent = f"projects/{project_id}"
10
11    # Configure which infoTypes to look for
12    info_types = [
13        {"name": "PERSON_NAME"},
14        {"name": "EMAIL_ADDRESS"},
15        {"name": "PHONE_NUMBER"},
16        {"name": "CREDIT_CARD_NUMBER"},
17        {"name": "US_SOCIAL_SECURITY_NUMBER"},
18    ]
19
20    inspect_config = {
21        "info_types": info_types,
22        "min_likelihood": dlp.Likelihood.POSSIBLE,
23        "include_quote": True,
24    }
25
26    item = {"value": text}
27
28    response = client.inspect_content(
29        request={
30            "parent": parent,
31            "inspect_config": inspect_config,
32            "item": item,
33        }
34    )
35
36    findings = []
37    for finding in response.result.findings:
38        findings.append({
39            "info_type": finding.info_type.name,
40            "likelihood": dlp.Likelihood(finding.likelihood).name,
41            "quote": finding.quote,
42            "location": {
43                "start": finding.location.byte_range.start,
44                "end": finding.location.byte_range.end,
45            }
46        })
47    return findings

• Import — brings in google.cloud.dlp_v2, which exposes the SDP service client and likelihood enum.
• create_dlp_client — constructs and returns a DlpServiceClient, the managed-API entry point.
• inspect_text signature — takes the client, project_id, and text, and builds the projects/{project_id} parent path SDP requires for any inspection request.
• info_types — the target infoTypes to scan for: PERSON_NAME, EMAIL_ADDRESS, PHONE_NUMBER, CREDIT_CARD_NUMBER, US_SOCIAL_SECURITY_NUMBER.
• inspect_config — pairs the chosen infoTypes with a POSSIBLE minimum-likelihood threshold and include_quote=True so the matched substring is returned.
• item — wraps the input text as an SDP-inspectable value.
• client.inspect_content(...) — calls SDP with the parent path, config, and item.
• for finding in response.result.findings — collects each finding into a dictionary capturing the info_type, likelihood name, matched quote, and byte-range location.

SDP additionally accepts a byte_item instead of value in the item payload, which lets the same managed API inspect images (receipts, screenshots, scanned forms) — a capability Presidio does not provide.

Comparing Presidio vs Google SDP

Processing a test corpus of 200 customer support messages through both systems reveals important differences:

The recommended strategy is to use both systems for their respective strengths:

• Presidio for real-time inline scanning: Every user prompt passes through Presidio before reaching the LLM. The 5-15ms latency is acceptable for interactive applications, and the zero API cost makes it viable for high-throughput deployments.
• SDP for batch compliance audits: Nightly or weekly batch jobs scan stored conversations, uploaded images, and log archives through SDP for comprehensive coverage. The higher latency and per-call cost are acceptable for batch processing where thoroughness matters more than speed.

You'll know the walkthrough works when running the Presidio snippet prints a PERSON detection for "John Smith" and a PHONE_NUMBER detection for "555-123-4567" each with score >= 0.85, and the Google SDP call returns findings whose likelihood is POSSIBLE or higher for the same entity classes. Cross-check by feeding both detectors a string with no PII — both should return an empty list.

---

Do's and Don'ts

Do's

1. ✓Do download en_core_web_lg before instantiating AnalyzerEngine — Presidio initializes without error even when the spaCy model is absent, but PERSON detections silently return nothing; skipping python -m spacy download en_core_web_lg means every name in a support transcript or customer email passes through the analyzer undetected.
2. ✓Do apply a per-entity score_threshold when filtering RecognizerResult objects — checksum-validated types like CREDIT_CARD reach scores of 0.85–1.0 when the Luhn algorithm confirms the pattern and can tolerate a tighter cutoff, while regex-only recognizers like IP_ADDRESS assign fixed scores based on context and benefit from the looser 0.5 baseline; collapsing all entity types to a single threshold discards the precision signal the scoring model already gives you.
3. ✓Do set include_quote=True in the SDP inspect_config and capture byte_range from each finding — without include_quote, DlpServiceClient.inspect_content returns only the info_type name and the Likelihood enum with no matched substring, making compliance audit logs category labels without evidence; byte_range.start/end ties each finding to its exact position in the inspected content so redaction or review can act on the right span.

Don'ts

1. ✗Don't route image payloads through Presidio's AnalyzerEngine — AnalyzerEngine.analyze accepts only text strings, so printed names, credit card numbers, and receipts embedded in images bypass detection entirely and reach model context unscanned; images must be sent to Google SDP's inspect_content, which covers image-native infoTypes that Presidio has no equivalent for.
2. ✗Don't treat a post-threshold empty RecognizerResult list as confirmation that no PII is present — a PHONE_NUMBER match at score 0.3 that falls below a 0.5 cutoff is suppressed, not disproved; downstream code must log pre-filter detections separately, because an empty filtered list means "nothing confident enough to act on," not "nothing found."
3. ✗Don't call DlpServiceClient.inspect_content on every synchronous prompt — Google SDP is a paid, network-bound API billed per character inspected; putting inspect_text in the real-time inline path adds a managed-API round-trip and per-character cost to every user request; use Presidio's in-process AnalyzerEngine for the synchronous hot path and reserve SDP for images and offline batch compliance audits.