Place privacy checks where an AI prompt actually leaves the system

A privacy check placed just before the final model call can be too late. The document may already have been sent to a remote OCR service, converted into embeddings by another provider or copied into a request trace.

To find the right control points, follow the data through the actual application. A screen labelled “local processing” or a diagram containing one model box is not enough. The meaningful boundary is each place where data becomes available to a different service, audience or retention regime.

Draw the outbound paths first

A hypothetical support assistant accepts a message and an attachment, retrieves policy guidance and drafts a reply. Its data path may include attachment storage, text extraction, retrieval, prompt construction, model inference and observability.

For each step, record the fields received, the destination, the purpose and the copies created. Include failed requests and background jobs. Retry queues and diagnostic reports are still data paths even when customers never see them.

NIST’s Generative AI Profile addresses privacy risks and the need to understand relevant components and third-party dependencies. Use that risk-management perspective to build the inventory. It does not certify a particular architecture.

Put a decision before the transfer

The support assistant may have an approved reason to send a complaint’s general meaning to a model but no reason to send a home address. The control at the model boundary should enforce that specific policy.

However, if an external OCR service receives the full attachment first, a later address replacement cannot undo that earlier transfer. The extraction path needs its own decision: use an approved service and permitted input, transform before upload where feasible, or reject or route the document differently.

Presidio’s image-redaction documentation includes OCR-based processing options. That is a reminder to inspect the configured extraction service rather than assuming all redaction happens before any remote processing.

The same reasoning applies to embeddings. An embedding request can transmit source text before retrieval ever occurs. Record that request as a separate processing step.

Inspect the assembled payload

Prompt construction can reintroduce data that an earlier check removed. Retrieved passages, conversation history, tool results and system-generated context can all be added later.

A useful control evaluates the final assembled payload at the outbound adapter, using a policy appropriate to the destination. Earlier checks remain useful for reducing exposure along the way, but they do not replace inspection of later additions.

Keep that inspection from creating another copy of the sensitive payload. Logs should normally record the decision and useful identifiers without storing the full rejected content. OpenTelemetry’s sensitive-data guidance recommends minimising sensitive collection and describes processing controls for telemetry.

If a rejected payload needs authorised review, define a restricted review store with its own access and retention. Do not place it in general application logs for convenience.

Give failure a designed outcome

A detector may time out, a policy service may be unavailable or a message may exceed a configured limit. Decide what each condition means before deployment.

For the support assistant, a safe outcome could be saving a restricted draft for later processing, using an approved local path or asking the operator to remove unnecessary content. The choice depends on the service and the consequences of delay.

Automatically forwarding the original payload on a privacy-check error undermines the intended boundary. Conversely, blocking every workflow indefinitely may create unacceptable operational pressure. Design a fallback that the team can actually use and test it under outage conditions.

Automated detection also has a coverage limit. Presidio explicitly does not guarantee complete detection. Combine detection with data minimisation, field restrictions and destination controls rather than treating a clean detector result as universal permission.

Verify the route with a controlled case

Use a synthetic support case containing distinctive test markers in the message, attachment and retrieved context. Observe which approved endpoints receive each marker and which stores retain it. Include a failed request and a retry.

The release evidence should answer:

• Which service receives the original message or document first?
• Which data fields are permitted at each outbound boundary?
• Can prompt assembly introduce unchecked content after an earlier transformation?
• What happens when a privacy decision cannot be obtained?
• Do traces, queues and error reports follow the same minimisation policy?
• Can the team reproduce the observed data path from configuration and test evidence?

The result is an accountable route for data through the application. A privacy check earns its value by governing a real transfer, with a defined purpose and observable outcome.