Measure precision and recall before choosing a PII detector

A detector score is only meaningful when the evaluation explains what was counted. “Accuracy” can hide whether the system missed a home address, removed ordinary text or merely classified a document as containing something sensitive.

For personal-data detection, start with the unit of evaluation. A token, an exact entity span, a document and a completed redaction are different units. Choose the one that answers the decision you need to make, and report other relevant outcomes separately.

Define the labels and matching rule

A hypothetical support workflow needs names, contact details and precise addresses removed before messages are shared with an approved reviewer. The evaluation team labels the expected entities and their exact positions.

The team must decide how a prediction matches a label. Does it require the exact span and correct category? Is partial overlap reported separately? What happens when the detector marks only one word of a multiword name?

These choices can change the score without changing the system. Record them before comparing configurations.

Presidio’s evaluation guidance describes evaluation against labelled data and measures such as precision and recall. It does not remove the need to define the task’s entity policy and matching rules.

Read precision and recall together

Precision asks how many predicted positives were correct. Recall asks how many actual positives were found. The scikit-learn metric documentation gives the standard definitions:

• Precision = true positives divided by true positives plus false positives.
• Recall = true positives divided by true positives plus false negatives.

For an illustrative arithmetic example, suppose a small synthetic set contains twelve labelled entities. A detector finds eight correctly, misses four and also marks two ordinary spans. Its precision is eight out of ten, or 80%. Its recall is eight out of twelve, approximately 66.7%.

These are invented counts to explain the calculation, not observed product results. They show why one metric cannot replace the other. Most detections can be correct while a substantial part of the labelled content remains missed.

A false-positive rate is a different measure: false positives divided by false positives plus true negatives. It is not one minus precision. In span detection, defining the universe of possible negative spans can itself be difficult, so avoid reporting a rate without a clear denominator.

Inspect the category results

An overall score can conceal a weak category. If most examples contain easily recognised contact details, good performance on those examples can dominate a smaller collection of difficult names or addresses.

Report the counts and errors for categories that matter to the workflow. Where relevant, also separate language, script, document type and input channel.

Micro and macro averages summarise groups differently, as the scikit-learn documentation explains. State which average is used and show enough supporting counts for the reader to interpret it. A category with very few examples should not acquire false authority from a neatly formatted percentage.

Undefined cases also need a rule. If there are no predicted entities or no labelled entities in a group, explain how the metric library handles the result rather than silently converting uncertainty into a favourable number.

Connect scores to the output

A correct detection does not prove that redaction was applied correctly. A missed entity may be caught by another control. An unnecessary detection may remove information essential to the recipient’s task.

Evaluate the transformed output as a separate stage. For document workflows, include extraction and rendering errors. For a routing workflow, inspect whether the decision actually prevented an unapproved transfer.

The acceptance threshold should follow the consequences of errors and the available fallback. Do not choose a threshold simply because it produces the highest single aggregate score.

A recogniser’s confidence score is also not automatically a calibrated probability of correctness for your data. Choose thresholds using development or validation cases, then confirm the fixed choice on separate held-out evaluation data. This keeps the final check independent of the tuning decision.

Require a reproducible comparison

Before choosing a detector, ask for:

• The entity policy, label guide and matching rule.
• The origin and composition of the evaluation set.
• Separate development and held-out evaluation data.
• Configuration, model, recogniser and threshold versions.
• Precision, recall and supporting counts for important groups.
• Examples of consequential misses and unnecessary transformations.
• Evidence that the final workflow, not only the detector, behaves as intended.

A credible comparison leaves enough detail for another person to repeat the calculation and challenge its interpretation. The score should support a bounded deployment decision, with visible limitations for categories and conditions that remain insufficiently tested.