AI in healthcare: patient data, hosting rules and the GDPR
Short answer
A healthcare organisation may use AI, but patient data does not belong in a public chatbot. Health data is a special category under the GDPR, medical confidentiality also applies to software suppliers, and several countries set their own hosting rules: NEN 7510 in the Netherlands, a C5 attestation for cloud use in Germany, HDS certification in France. The safest route is AI that runs inside the care provider’s own environment, with a log of who used which data.
What went wrong at a GP practice
In 2024 the Dutch Data Protection Authority warned about data breaches caused by AI chatbots. One of the reports came from a GP practice: an employee had, against company agreements, entered patients’ medical data into an AI chatbot. The regulator stressed that medical data is especially sensitive and therefore has extra legal protection.
The example shows where the risk in healthcare lies. Not with malicious staff, but with busy staff who want to summarise a referral letter or rewrite a letter to a patient, and use the tool they know. See also Shadow AI.
Which rules apply to AI with patient data?
| Rule | What it requires of AI |
|---|---|
| Medical confidentiality | Patient data only for those directly involved in the patient’s care. An outside AI vendor is not one of them. |
| GDPR, Articles 9 and 35 | Health data is a special category of personal data. Large-scale processing or new technology usually requires a DPIA. |
| Netherlands: NEN 7510, 7512 and 7513 | Mandatory for care providers: information security, secure data exchange and logging of access to patient data. |
| Germany: § 393 SGB V | Processing health and social data with cloud services requires a provider with a C5 attestation, type 2 since 1 July 2025. |
| France: HDS (art. L.1111-8 CSP) | Hosting health data for a care provider, and administering the system that holds it, requires HDS certification. From 27 September 2026 storage must be in the EU or EEA. |
| Medical Device Regulation (EU 2017/745) | Software intended for diagnosis or treatment can be a medical device, with its own requirements and certification. |
| EU AI Act | AI in medical devices is high-risk; after the Digital Omnibus those rules apply from 2 August 2028. AI literacy and transparency apply now. |
Access logging deserves extra attention. Rules such as NEN 7513 ask you to record who viewed which patient data and when. When staff paste data into an outside chatbot, that log is incomplete by definition: the access happens outside your systems.
Where AI can safely help in healthcare
AI does not have to start with diagnosis to add value. Most of the gain is in administration and support, where no medical decision depends on the model:
- summarising referral and discharge letters for handover;
- drafts of letters to patients in plain language, checked by a clinician;
- searching and answering questions in protocols, guidelines and internal procedures;
- structuring free text for quality registries and reports;
- supporting the service desk and the secretariat.
As soon as AI is intended to support diagnostic or treatment decisions, the Medical Device Regulation comes into play. That is a different track, with its own certification.
How to use AI safely with patient data
- Keep processing inside your own environment. Run the model on your organisation’s infrastructure, so no outside AI vendor receives patient data. That also makes your DPIA more manageable, and keeps you out of cloud rules such as C5 and HDS for that processing.
- Link access to roles. A department gets only the sources and automations that belong to its work, in line with medical confidentiality.
- Log everything. Who used which model with which source, and when.
- Block what does not belong. Safeguards that stop patient numbers and other identifying patterns where they are not needed.
- Keep a human in charge. AI makes a draft; a clinician reviews and signs.
werqly places a dedicated AI node in your organisation’s server room for this. You set roles per department, safeguards and the audit log in werqly AI Control. For patient data we recommend your own server room: werqly holds neither HDS nor C5 certification and its NEN 7510 certification is in progress; werqly Cloud suits work without patient data. More on our approach for care providers is on AI for healthcare.
Frequently asked questions
Questions on this topic
May healthcare staff use ChatGPT?
Not with patient data in a public or personal version. An outside party then gains unauthorised access to special-category data, which the Dutch Data Protection Authority treats as a data breach. For general questions without patient data the organisation can allow it, with a clear policy.
Does a hospital need HDS or C5 for AI?
Only if an outside provider hosts or administers the health data: HDS in France, a C5 attestation for cloud services in Germany. When the model runs on the hospital’s own infrastructure and its own IT department manages it, that processing stays in-house.
Is AI in healthcare a medical device?
Only if the software is intended for diagnosis, prevention, monitoring or treatment. AI for administration, summaries of letters or searching protocols usually is not. Do not use such tools for treatment decisions either.
Do we need a DPIA for AI with patient data?
As a rule, yes. Health data is a special category of personal data, and generative AI counts as new technology. Processing on your own node without transfer to third parties does make the DPIA more manageable.
Sources
- Dutch Data Protection Authority — Caution: use of AI chatbot may lead to data breaches (2024)
- Germany — § 393 SGB V, cloud use in healthcare
- France — Agence du Numérique en Santé: HDS certification
- Regulation (EU) 2017/745 on medical devices
This article is general information, not legal advice. Rules and guidance change; check the sources listed or ask your lawyer or DPO if in doubt.