An AI chatbot for customer service: transparency, liability and data

Transparency: customers must know it is AI

Article 50 of the AI Act requires AI systems that interact directly with people to be designed so that people know they are dealing with AI, unless that is already obvious to a reasonably well-informed person. This duty has applied since 2 August 2026.

In practice: state at the start of the conversation that the customer is talking to an AI assistant, and do not give the assistant a human name and photo that suggest otherwise. Whoever builds the chatbot or puts it into service under their own name is often the provider for this duty.

You are liable for what the chatbot says

In Moffatt v. Air Canada (February 2024), the airline’s chatbot told a customer he could claim a bereavement discount after the trip. That contradicted the policy. Air Canada argued that the chatbot was a separate entity responsible for its own statements. The Canadian tribunal rejected that: the company had not taken reasonable care that the chatbot gave accurate information, and had to compensate the customer.

The lesson applies in Europe too: a customer may rely on what your website says, even when a language model phrases it. See AI hallucinations.

How to set up a customer service chatbot properly

1. Answers from your own sources. Let the chatbot answer from your current terms, rates and procedures, not from the model’s memory. See AI on your own documents.
2. Limits on topics. For questions about amounts, contracts, complaints or legal matters, let the chatbot refer to an employee.
3. Always a route to a human. Make it visible how a customer reaches an employee.
4. Limited data. Ask only for what is needed, and do not keep conversations longer than necessary.
5. Test and measure. Test with real questions, including difficult and misleading ones. Measure how often answers are wrong and how often customers still call.

Customer conversations are customer data

Customers tell a chatbot all kinds of things: addresses, customer numbers, complaints, sometimes their health or financial situation. If the chatbot runs at an external AI service, those conversations are stored there. That requires a data processing agreement, an assessment of transfers and often a DPIA. See A DPIA for generative AI.

With werqly, the language model for your customer service runs on a dedicated node. Customer conversations and the connection to your CRM stay within your environment, and the audit log shows which source belonged to which answer. See What is private AI?