You Can Subscribe to Any AI Service. You Can Never Truly Trace Your Data.

When we built CasePilot, our document verification platform for mortgage brokers, we could tell you exactly where a customer’s payslip sat at any point in the pipeline. AWS London, encrypted at rest and in transit, IAM scoped to the microservice, audit trail on every access. If a broker asked “where is my client’s bank statement right now?”, we had an answer. A provable one.

Then generative AI arrived, and the moment you send that same document to a third-party model API, the honest answer to that question becomes: I don’t know.

That’s the uncomfortable trade at the heart of the current AI gold rush. You can subscribe to any AI service in about ninety seconds with a corporate card. What you cannot buy, at almost any price, is genuine traceability of your data once it crosses the API boundary.

The subscription is easy. The provability isn’t.

Signing up is frictionless by design. Paste an API key, send a prompt, get a response. The vendor’s job is to make that as smooth as possible, and they’re very good at it.

But smooth onboarding hides three questions that matter enormously in regulated industries, and they’re the same three questions I’ve written about before:

Where does the data physically reside while it’s being processed?
Who, or what, can access it?
Can you prove it?

With traditional infrastructure we can answer all three with logs, region pins, and IAM policies. With most AI APIs you can answer none of them with certainty. You can read the vendor’s policy page. You can read their data processing addendum. But a policy is a promise, not a proof. As engineers, we know the difference. A promise is a config someone can change; a proof is something you can point an auditor at.

What “we don’t train on your data” actually means

Every major provider now says some version of “we don’t train on your business data by default.” That’s genuinely better than where we were two years ago, and it’s worth acknowledging.

But read the sentences around it carefully:

“By default” implies a setting exists that changes it, which implies the plumbing to use your data for training already exists and is one toggle away.
Retention windows. Many APIs retain your prompts and outputs for a period, often around 30 days, for “abuse monitoring.” That means your data sits in someone else’s storage, readable by their staff or automated systems, for weeks after the request completes.
Sub-processors. Your prompt may pass through a chain of providers you never signed a contract with. The model vendor, their cloud host, their moderation vendor, their logging stack. Each is a link where your data lives briefly and where you have zero visibility.
Human review. “We may review a sample of interactions to improve safety” is common. Somewhere, a person you’ll never meet may read the exact text you sent.

None of these are necessarily scandalous. They’re often reasonable engineering and safety decisions. The problem is that you are trusting a description of a system you cannot inspect. You’ve swapped provable data handling for a vendor’s assurance about it.

The CLOUD Act problem hasn’t gone away, it’s got an API now

I’ve written before about the US CLOUD Act and how it lets US law enforcement compel US-headquartered companies to hand over data regardless of where the servers physically sit. AWS London, Azure UK South, all US-owned, all in scope.

The major AI labs are almost all US-headquartered. OpenAI, Anthropic, Google. When you pipe customer data through their APIs, you’ve added a US-jurisdiction processor to your data flow, often without a UK or EU region option that actually isolates the processing end to end. For most consumer workloads that’s an acceptable risk. For an NHS trust, a law firm, or a financial services company under DORA, it’s a question their procurement team will absolutely ask, and “trust me, the vendor says it’s fine” is not an answer that wins the contract.

Why this is harder than normal cloud

With normal cloud infrastructure, sovereignty is at least solvable. Pin the region, own the account, enforce boundaries with SCPs and IAM, and you can draw a line around your data. Hard work, but tractable.

AI services break the model in a way that’s genuinely new:

The data leaves your control by definition. The whole point of the API is that your data goes to their model. You can’t process it “in your VPC” unless the vendor offers that, and most of the frontier models don’t.
The processing is opaque. You can’t see what happens inside the inference call. There’s no equivalent of a CloudTrail log for “what did the model do with token 4,000 of my prompt.”
The output is derived, not stored. Even if nothing is retained, your data has already shaped a response, been embedded into a vector, or been chunked into a RAG index that now lives somewhere. Deletion becomes philosophical.

So even a well-intentioned vendor with a spotless policy can’t give you what infrastructure gives you: a log you can point at and say “here is exactly what happened to this piece of data.”

I’m not anti-AI. I build with it. But I treat the traceability gap as a design constraint, not an afterthought. For anyone handling sensitive data:

Classify before you send. Not every workload needs a frontier model. Decide which data classes are allowed to leave your boundary at all, and enforce that in code, not in a policy doc nobody reads.
Prefer models you can run inside your own boundary for sensitive workloads. Open-weight models on infrastructure you control (self-hosted on your own cloud account, or a provider that offers in-VPC inference) give you back the audit trail. The model may be a little less capable. The provability is worth it.
Redact and tokenise at the edge. Strip or pseudonymise personal data before it hits the API. If the model never sees the real payslip, the traceability question shrinks dramatically.
Read the DPA, not the marketing page. Retention windows, sub-processors, training toggles, human review, jurisdiction. Get it in the contract, and get the enterprise tier that lets you turn retention off if you need to.
Log your side of the boundary. You can’t see inside the model, but you can log every prompt you send and every response you get, with the classification of the data attached. That’s your evidence trail, and it’s the part you actually control.
Be able to answer the three questions. Where does the data go, who can touch it, and can you prove it? If sending data to an AI service means you can’t answer those, that’s a decision to make deliberately, not by accident because signing up was easy.

The uncomfortable truth

The AI vendors have made subscribing trivial and traceability optional. That’s a rational business decision for them, and it works fine for a huge amount of low-stakes work. But it quietly shifts the burden of proof onto you, and if you’re in a regulated sector, the regulator doesn’t accept “the vendor said so” any more than they accept it for your cloud provider.

When we processed mortgage documents on CasePilot, we could point at a region and say “your data is here, and only here.” The whole value of that statement is that it was provable. The AI era asks us to give that up in exchange for capability, and often we should, but only with our eyes open.

You can subscribe to any AI service you like. Just don’t confuse the ease of signing up with knowing where your data went. Those are very different things, and the gap between them is exactly where trust, compliance, and reputation live.