Training AI on customer data is possible under GDPR once you have answered three questions in order: what your legal role is for that data, which legal basis covers the new purpose, and whether the people in the data could reasonably expect it. If you hold the data on a customer's behalf, their contract decides first. If you rely on legitimate interest, you need a documented assessment, a working opt-out and masking before any record reaches a training set.
Training here covers fine-tuning, training your own classifier and building evaluation sets from production data. This is general information, not legal advice; your data protection officer (DPO) or counsel should confirm the position for your product.
Can you train AI on customer data under GDPR?
Often, yes, but GDPR has no special exemption for AI. The European Data Protection Board (EDPB) addressed the question in Opinion 28/2024 on AI models, adopted in December 2024. It recalls that there is no hierarchy between legal bases and that the controller must choose and justify one, then focuses on legitimate interest.
The opinion deliberately leaves out special category data (Article 9), automated decisions (Article 22) and the purpose compatibility test (Article 6(4)), while noting they may still matter. Health details inside support tickets raise questions it doesn't answer.
Are you the controller or the processor for this data?
This decides who gets to say yes. A B2B software company is usually the controller for account and usage data about its own users. For content its customers upload, such as tickets, documents or call recordings, it is usually a processor under a data processing agreement (DPA).
The EDPB guidelines on controller and processor state that a processor may not process data for its own purposes. Under Article 28(10), a processor that starts determining purposes and means is treated as a controller for that processing. Training a shared model on customer content serves your purpose, so you take on the controller's duties for it.
| Data | Your usual role | What has to be true before training |
|---|---|---|
| Your users' account and usage data | Controller | A legal basis, updated notices and a documented assessment |
| Customer content, used only for that customer | Processor | The customer's documented instruction, such as a per-tenant model they enabled |
| Customer content, used to improve a shared model | Controller for the training | Contract terms that allow it, your own legal basis and transparency toward the people in the data |
| Data anonymized to the GDPR standard | Outside GDPR | Evidence that the anonymization holds |
Read the DPAs customers actually signed before engineering starts. A negotiated clause forbidding any use beyond service delivery can split your dataset in two.
Which legal basis fits: consent or legitimate interest?
Consent works when people can refuse without losing the service, and you must stop using their data when they withdraw. Legitimate interest under Article 6(1)(f) avoids asking every user but requires the three-step test that Opinion 28/2024 walks through:
- Identify the interest. It must be lawful, clearly articulated, and real and present. Developing a conversational agent to assist users is the EDPB's own example. "Exploring AI opportunities" is too vague.
- Test necessity. The processing must serve that interest with no less intrusive alternative, and the amount of personal data must be proportionate. If a model trained on redacted tickets performs as well as one trained on raw tickets, the raw version fails.
- Balance the interests. Your interest must not be overridden by the rights and freedoms of the people in the data. Their reasonable expectations carry much of the weight.
When you rely on legitimate interest, the right to object under Article 21 applies. Build the objection path before the first training run; leaving someone out is easier than removing them later.
Is a privacy policy update enough?
Usually not. The EDPB says a privacy policy mention doesn't necessarily mean people can reasonably expect the processing. The relationship, the service and how the data was collected all count. A customer who wrote to support about a billing error doesn't expect that message to shape a model sold to other companies.
US companies face a similar test. In February 2024 the Federal Trade Commission (FTC) warned on its technology blog that it may be unfair or deceptive to start using consumer data for AI training and disclose it only through a surreptitious, retroactive change to the terms of service or privacy policy.
The safer sequence works in both regions: announce the change prominently, apply it to data collected after the notice, offer a real opt-out, and only then start building training sets.
Is a model trained on personal data anonymous?
Not by default. Opinion 28/2024 says AI models trained on personal data cannot, in all cases, be considered anonymous. Both the likelihood of extracting personal data from the model and of obtaining it through queries must be insignificant, considering all the means reasonably likely to be used. The EDPB points to evidence such as testing against membership inference, regurgitation of training data and model inversion.
Treat a fine-tuned model as a store of personal data unless you can show otherwise. In engineering terms:
- A shared model fine-tuned on several customers' tickets can repeat one customer's text to another. OWASP covers this exposure in LLM02: Sensitive Information Disclosure and recommends sanitizing data before it enters training.
- Erasure requests reach the model. Record which dataset version trained which model, so you can remove a person and retrain.
- Model weights and training sets need the same access controls as the production database they came from.
How to prepare customer data for AI training
Before the first record leaves production:
- Map each data source to your role, the DPA terms and the legal basis. Exclude tenants whose contracts forbid the use.
- Record the legitimate interest assessment and the DPO's advice. Our guide to running a DPIA for an AI feature covers when an impact assessment is also required.
- Filter out special category data, such as health information, and data about children, unless counsel has confirmed a basis.
- Drop fields the task doesn't need. A ticket router needs the message text, not the sender's name or email address.
- Mask or replace identifiers. The EDPB lists substituting fake names and email addresses as a mitigating measure where the content matters more than the author.
- Leave time between collection and training. The opinion names a reasonable delay as a safeguard, giving people a window to object.
- Offer an opt-out that applies before processing starts, enforce it in the pipeline and test it.
- Version every dataset with lineage to its source records, and set a retention period for training copies.
Also ask whether you need training at all. Retrieval over a customer's own documents can keep data out of model weights, as explained in choosing between RAG and fine-tuning.
Common mistakes when training on customer data
- Treating training as "service improvement" under a DPA clause written before anyone planned to train models.
- Exporting raw production tables to a notebook and planning to redact later. The unredacted copy is the exposure.
- Keeping evaluation sets built from real conversations indefinitely in a shared drive.
- Offering an opt-out that stops future collection but leaves the person in existing training sets.
- Declaring the model anonymous without testing it for extraction.
- Never revisiting the assessment. Review it when you add a data source, move to shared models or hear of the model reproducing personal data.
If you want an outside view of where personal data flows through your AI plans, our AI Readiness & Governance Assessment covers data protection and GDPR touchpoints, the security of AI tools and vendors, and a risk register. Your legal counsel confirms final legal positions.
Frequently asked questions
Does sending customer data to an LLM API count as training?
Not by itself. Sending data to get a response is inference, which still needs a legal basis and a processing agreement with the provider. It becomes training if the provider's terms let it use your inputs to improve its models, so confirm the terms in writing.
Can we fine-tune a separate model for each customer?
Yes, and it often simplifies the legal position. If a customer enables a model trained and used only for its own data and users, you can usually act as its processor. Record that instruction and delete the model when the contract ends.
What if someone asks us to delete their data after training?
Delete it from source systems and training sets, and mark the dataset versions that contained it. Whether you must retrain depends on whether the model still holds that data, which the EDPB treats case by case. A regular retraining schedule from cleaned datasets is easier to defend than one-off fixes.