What is private AI? Meaning, forms and when you need it

Private AI in one sentence

With an AI service such as ChatGPT, Gemini or Claude, your question goes to the vendor’s servers. The model runs there, your text is processed there, and the vendor’s terms apply. With private AI, the model runs on hardware you own, or that is reserved for you alone. Processing happens inside your own security perimeter.

That sounds technical, but it changes three things that matter to a CISO or DPO:

• Where the data is. Not with a third party, but on your own node.
• Who can get access to it. Only the people you give access. There is no vendor that can be compelled to hand data over, because the vendor never had it.
• Who sets the rules. Your IT department decides which models are available, which data is blocked and how long logs are kept. Not someone else’s product terms.

Which forms of private AI are there?

The term is used loosely, including by vendors who mean something else by it. It helps to distinguish four forms, from most to least your own:

Only the first two are private AI in the strict sense: the model and the data sit on hardware nobody else uses. A private cloud tenant or a business subscription offers better terms than a free chatbot, but processing still happens at an outside party. For organisations dealing with the CLOUD Act question, professional secrecy or sector rules on health data and outsourcing, that is exactly the difference that counts. More on this in Microsoft 365 Copilot or your own AI environment.

Which models run on private AI?

Private AI uses open-weight models: language models whose makers publish the weights, so you can run them on your own hardware. Well-known families are Llama (Meta), Mistral (from the French company Mistral AI) and Qwen (Alibaba). The closed models behind ChatGPT, Gemini and Claude cannot be self-hosted; they exist only at the vendor.

For most business tasks the larger open-weight models are good enough: summarising, rewriting, answering questions about your own documents, drafting emails, extracting data from text. Where the gap with the best closed models is largest, in complex reasoning and long programming tasks, you trade some performance for control. That is a deliberate trade-off, not a hidden drawback.

What it takes technically to run such a model, from GPU memory to operations, is covered in An on-premises LLM server: what do you need?

When is private AI the right choice?

Private AI costs more than a per-user AI subscription. You pay for your own hardware and for control. It is therefore not the right choice for everyone. It fits best when one or more of these apply:

• You process special-category or confidential data. Patient records, client files under professional secrecy, financial customer data, citizens’ personal data.
• Your sector sets rules for outsourcing. Think of health-data hosting rules in healthcare, lawyers’ duty of confidentiality or DORA for financial institutions.
• Staff already use AI, out of your sight. Then the question is not whether AI is used, but where. See Shadow AI.
• You do not want to depend on a single vendor. Prices, terms and availability of an AI service can change. Your own environment with open-weight models stays yours.
• You must be able to show what happens to data. An auditor, regulator or client does not ask whether you have a policy, but what demonstrably happened. With your own logging you can show it.

If none of these apply, a business AI subscription with good agreements is often enough, and cheaper.

What needs to be in place besides the model?

Putting a model on a server is the easy part. What makes private AI usable and defensible is what surrounds it:

• Access by role, linked to your identity provider, so finance gets a different model and different rights than the service desk.
• Safeguards that stop patterns such as national ID numbers, IBANs or source code before they reach a model.
• Logging and an audit trail: who, which model, when, with a retention period you set and export to your SIEM.
• A policy on insight: how much does the organisation see of what staff ask? Anonymous, per department or per person? That is a decision for the board, the DPO and the works council, not a vendor’s default.
• Updates and model management: testing and rolling out new model versions, without anything leaving the building.

werqly delivers this as a whole: a dedicated AI node in your server room (or, if you do not have one, in a locked rack in our European datacentre) with werqly AI Control as the management console. Your IT department runs it, or leaves that to werqly. How that works in practice is on the AI Control page.