Your AI stack might be leaking PII

Everyone is racing to add AI. Almost no one can say where the data went.

Every clever new feature you bolt on — a chatbot, a summariser, a “smart” search box — is another place your customers’ personal data can quietly walk out the door. Not through a dramatic breach. Through an API call nobody mapped, a contract clause nobody read, or a vector database that can never truly forget. The scary part is not that it happens. It is that most teams could not prove it didn’t.

Here is how to think about it — and how to build so the answer is easy.

Where is your data right now?

Three questions. If you cannot answer all three off the top of your head, you have a gap:

• Where does each API call go? Every call to a cloud model is data leaving your building.
• Do your contracts actually allow it? Not what the salesperson said — what the terms say.
• Would your legal team approve if they saw the full picture today?

Cloud APIs are shared custody

The moment you send data to someone else’s model, you are in a shared-custody arrangement — and the paperwork matters more than the technology. One missed clause is not a technicality; under GDPR it is fines of up to 4% of global revenue. So the questions get sharper: does the contract really say what you think it says? What about data residency — which country is that data sitting in? “We assumed it was fine” is not a defence.

The expensive problem: RAG and embeddings

This is the one that catches people out. To make AI answer questions about your business, teams turn documents into embeddings — mathematical vectors — and store them. The trouble: embeddings are effectively irreversible. Once a customer’s personal data is baked into vectors, deleting it on request is expensive, and sometimes practically impossible.

The fix is architectural, not a patch: keep the PII separate. Let the model work with anonymised signals only, and keep the identifying detail somewhere you can actually delete from.

Harnessing the AI without exposing the data

Three principles turn “we hope it’s fine” into “here’s exactly how it works”:

1. Anonymise before the model ever sees it.
2. Keep PII in relational storage, not in embeddings — where you can query, update, and delete it cleanly.
3. Let tools work on signals, not raw data.

In practice that means the model never sees “John Doe, 42 Oak St, 5 kids.” It sees “parent, 3–5 children, high engagement.” Same useful signal. None of the liability.

Personalisation without the risk

You do not have to trade personalisation for safety. Split the job:

1. AI drafts the template using anonymised signals.
2. A deterministic step inserts the real name and details at the very end.
3. The template is never trained on PII.

The payoff is the part your legal team will love: personalised output, zero PII in your embeddings, and “the right to be forgotten” becomes one SQL delete — not an archaeology project. That is a system you can defend.

Could you draw the diagram in ten minutes?

Here is the regulator’s real question: show me every place customer PII touches your AI system. Could you draw that diagram in ten minutes? If not, you do not have a technology problem — you have a data-governance gap, and it only gets more expensive to close the longer it sits there.

That is exactly what the checklist below is for: it walks your stack through data-flow mapping, contract review, storage and embedding audits, removal compliance, and the architecture patterns that make all of it easier.