Build custom PII recognizers for domain data

Introduction

When you put a hosted-LLM proxy in front of regulated data, Presidio's stock recognizers cover names and emails but miss the patterns your security team actually cares about — medical record numbers, internal IP ranges, project codes. A single un-redacted MRN in a prompt is a HIPAA incident; a leaked internal IP exposes network topology to whoever logs the model's traffic. By the end of this lesson you'll be able to build, register, and measure custom Presidio recognizers that catch your organization's domain-specific PII before it ever reaches the model.

Key Terminology

• PatternRecognizer: Presidio base class that wraps one or more regex Pattern objects plus a context word list; sufficient when an entity can be detected purely by regex with confidence boosted by nearby terms (e.g. PATIENT_ID, PROJECT_CODE).
• EntityRecognizer: Presidio base class for custom detectors whose logic exceeds regex — you override analyze() to add validation (RFC 1918 octet ranges for INTERNAL_IP) and compute scores from a context window.
• RecognizerRegistry: the container an AnalyzerEngine reads from; load_predefined_recognizers() seeds it with Presidio's built-ins, then add_recognizer() plugs in each custom class so they run in the same pipeline.
• Precision / recall gate: the per-entity deployment bar (precision ≥ 0.90, recall ≥ 0.85) computed by evaluate() against a labeled corpus; recognizers below the bar are tuned before they sit behind the LLM proxy.

Concepts

Why Custom Recognizers Are Necessary

Standard PII detection covers universal entity types like names, email addresses, and phone numbers. However, every organization has domain-specific sensitive data that standard NER models cannot detect. A hospital system considers medical record numbers (MRN-123456) to be highly sensitive PII. A defense contractor treats internal network IP ranges (10.x.x.x) as classified information. A technology company considers internal project codes (PROJ-ENG-2024) to be confidential. None of these patterns appear in Presidio's default recognizer registry, yet exposing them to a hosted LLM constitutes a data breach under the organization's security policies.

Presidio's extensible recognizer architecture allows registering custom recognizers that run alongside built-in ones, ensuring domain-specific patterns are detected with the same pipeline and scoring infrastructure.

Code Walkthrough

This section pulls the concept above into runnable form: subclass PatternRecognizer for the regex-driven entity types (PATIENT_ID, PROJECT_CODE), subclass EntityRecognizer for INTERNAL_IP where pattern matching needs octet validation and a manual context boost, then register all three with an AnalyzerEngine and measure precision/recall against a labeled corpus.

Defining the Three Custom Recognizers

PatientIDRecognizer and ProjectCodeRecognizer only need patterns plus context words, so PatternRecognizer is enough. InternalIPRecognizer overrides analyze() because RFC 1918 detection needs per-octet range validation that regex alone cannot express, and it applies a manual context boost.

Code snippetpython
1import re
2from typing import List
3from presidio_analyzer import (
4    EntityRecognizer,
5    PatternRecognizer,
6    Pattern,
7    RecognizerResult,
8)
9
10class PatientIDRecognizer(PatternRecognizer):
11    """Detects medical record numbers (MRN-XXXXXX)."""
12
13    PATTERNS = [
14        Pattern("mrn_strict", r"\bMRN-\d{6}\b", 0.85),
15        Pattern("mrn_loose", r"\bMRN[\s-]?\d{6}\b", 0.65),
16    ]
17    CONTEXT = ["patient", "medical", "record", "hospital", "mrn", "chart"]
18
19    def __init__(self):
20        super().__init__(
21            supported_entity="PATIENT_ID",
22            patterns=self.PATTERNS,
23            context=self.CONTEXT,
24            supported_language="en",
25            name="PatientIDRecognizer",
26        )
27
28class ProjectCodeRecognizer(PatternRecognizer):
29    """Detects internal project codes (PROJ-XXX-NNNN)."""
30
31    PATTERNS = [
32        Pattern("project_strict", r"\bPROJ-[A-Z]{3}-\d{4}\b", 0.90),
33        Pattern("project_loose", r"\bPROJ[\s-][A-Z]{3}[\s-]\d{4}\b", 0.70),
34    ]
35    CONTEXT = ["project", "sprint", "jira", "ticket", "milestone", "roadmap"]
36
37    def __init__(self):
38        super().__init__(
39            supported_entity="PROJECT_CODE",
40            patterns=self.PATTERNS,
41            context=self.CONTEXT,
42            supported_language="en",
43            name="ProjectCodeRecognizer",
44        )
45
46class InternalIPRecognizer(EntityRecognizer):
47    """Detects RFC 1918 private IPs with octet validation + context boost."""
48
49    INTERNAL_IP_PATTERN = re.compile(
50        r"\b("
51        r"10\.\d{1,3}\.\d{1,3}\.\d{1,3}"
52        r"|172\.(?:1[6-9]|2\d|3[01])\.\d{1,3}\.\d{1,3}"
53        r"|192\.168\.\d{1,3}\.\d{1,3}"
54        r")\b"
55    )
56    CONTEXT_WORDS = ["server", "ip", "network", "host", "ssh", "internal", "subnet"]
57
58    def __init__(self):
59        super().__init__(
60            supported_entities=["INTERNAL_IP"],
61            supported_language="en",
62            name="InternalIPRecognizer",
63        )
64
65    def load(self) -> None:
66        pass
67
68    def analyze(self, text: str, entities: List[str], nlp_artifacts=None):
69        results = []
70        for match in self.INTERNAL_IP_PATTERN.finditer(text):
71            octets = match.group().split(".")
72            if not all(0 <= int(o) <= 255 for o in octets):
73                continue
74            window = text[max(0, match.start() - 50):match.end() + 50].lower()
75            boost = sum(1 for w in self.CONTEXT_WORDS if w in window) * 0.05
76            results.append(RecognizerResult(
77                entity_type="INTERNAL_IP",
78                start=match.start(),
79                end=match.end(),
80                score=min(1.0, 0.75 + boost),
81            ))
82        return results

The CONTEXT / CONTEXT_WORDS lists are what turn a noisy regex (an "MRN-123456" tracking ID, a public-looking 10.x address in a network blog post) into a high-confidence hit when surrounding text actually mentions a hospital or a server. The InternalIPRecognizer override shows the escape hatch: when pattern matching alone produces false positives a regex can't filter (here: octets must be 0-255), drop to EntityRecognizer and implement analyze() directly.

Registering and Measuring the Recognizers

With the classes defined, register them alongside Presidio's predefined recognizers and evaluate against a labeled corpus so you have a deployment gate. The canonical bar: precision ≥ 0.90 (lower over-redacts and degrades LLM output) and recall ≥ 0.85 (lower leaks PII silently).

Code snippetpython
1from dataclasses import dataclass
2from typing import Dict
3from presidio_analyzer import AnalyzerEngine, RecognizerRegistry
4
5def build_analyzer() -> AnalyzerEngine:
6    registry = RecognizerRegistry()
7    registry.load_predefined_recognizers()
8    registry.add_recognizer(PatientIDRecognizer())
9    registry.add_recognizer(InternalIPRecognizer())
10    registry.add_recognizer(ProjectCodeRecognizer())
11    return AnalyzerEngine(registry=registry)
12
13@dataclass
14class RecognizerMetrics:
15    entity_type: str
16    tp: int = 0
17    fp: int = 0
18    fn: int = 0
19
20    @property
21    def precision(self) -> float:
22        d = self.tp + self.fp
23        return self.tp / d if d else 0.0
24
25    @property
26    def recall(self) -> float:
27        d = self.tp + self.fn
28        return self.tp / d if d else 0.0
29
30    @property
31    def f1(self) -> float:
32        p, r = self.precision, self.recall
33        return 2 * p * r / (p + r) if (p + r) else 0.0
34
35def evaluate(analyzer: AnalyzerEngine, entity_type: str, corpus: List[Dict]) -> RecognizerMetrics:
36    m = RecognizerMetrics(entity_type=entity_type)
37    for sample in corpus:
38        expected = {(e["start"], e["end"]) for e in sample["entities"] if e["type"] == entity_type}
39        detected = {(r.start, r.end) for r in analyzer.analyze(
40            text=sample["text"], entities=[entity_type], language="en",
41        )}
42        m.tp += len(expected & detected)
43        m.fp += len(detected - expected)
44        m.fn += len(expected - detected)
45    return m
46
47analyzer = build_analyzer()
48text = "Patient MRN-845723 called from 10.42.88.15 about PROJ-ENG-4521."
49for r in analyzer.analyze(
50    text=text,
51    entities=["PATIENT_ID", "INTERNAL_IP", "PROJECT_CODE"],
52    language="en",
53):
54    print(f"{r.entity_type}: {text[r.start:r.end]!r} score={r.score:.2f}")
55# PATIENT_ID:   'MRN-845723'    score=0.90
56# INTERNAL_IP:  '10.42.88.15'   score=0.80
57# PROJECT_CODE: 'PROJ-ENG-4521' score=0.90

You'll know it works when analyzer.analyze(...) returns all three entity types from the sample sentence with scores ≥ 0.80, and evaluate(...) against your labeled corpus reports precision ≥ 0.90 and recall ≥ 0.85 for each entity type — below those thresholds, tune the patterns or context words before promoting the recognizers behind the LLM proxy.

---

Do's and Don'ts

Do's

1. ✓Do use tiered Pattern confidence scores — define a strict pattern at 0.85 (e.g., mrn_strict) and a loose pattern at 0.65 (e.g., mrn_loose) so Presidio's context boost from surrounding words like "patient" or "hospital" can elevate a marginal match without anchoring every hit at the ceiling and hiding false positives.
2. ✓Do validate each octet as 0 <= int(o) <= 255 inside InternalIPRecognizer.analyze() — the RFC 1918 regex correctly constrains the 172.16–31.x.x range but cannot express numeric bounds on the third and fourth octets, so a string like 10.999.0.1 passes the pattern; the integer check is the only reliable gate.
3. ✓Do implement load() as a pass stub whenever you subclass EntityRecognizer — Presidio's RecognizerRegistry calls load() on every recognizer at initialization time, and omitting the method raises TypeError: Can't instantiate abstract class InternalIPRecognizer before the engine processes a single character.

Don'ts

1. ✗Don't skip registry.load_predefined_recognizers() before calling registry.add_recognizer() — RecognizerRegistry starts empty, so omitting the call silently removes Presidio's built-in PERSON, EMAIL_ADDRESS, and PHONE_NUMBER detectors while your PATIENT_ID/INTERNAL_IP/PROJECT_CODE recognizers are live, creating blind spots for stock PII the proxy was already catching.
2. ✗Don't implement INTERNAL_IP detection as a PatternRecognizer subclass — doing so forces you to express RFC 1918 membership entirely in regex, which cannot compare octet integers to numeric ranges; the result is false positives on addresses like 10.999.0.1 that the analyze() override in EntityRecognizer eliminates with a two-line all(0 <= int(o) <= 255 ...) guard.
3. ✗Don't deploy a recognizer to the proxy-side AnalyzerEngine before measuring it against a labeled corpus — a PATIENT_ID recognizer at precision 0.85 over-redacts one in seven hits, replacing real MRNs with <PATIENT_ID> tokens that the LLM then reasons over as placeholders; a recall below 0.85 silently passes live MRNs through to the model log, making the redaction layer a HIPAA liability rather than a control.