Monitoring AI in production: what a local authority measures, keeps and reassesses every year
Short answer
An AI application changes after roll-out: use shifts, sources age, a new model becomes available. The AI Act requires deployers of high-risk systems to monitor operation, report incidents and keep logs for at least six months. For every application, high-risk or not, the same approach is sensible: a handful of figures a month, a reporting point for errors, spot checks on output and an annual reassessment.
What the AI Act asks
Article 26 places duties on deployers of high-risk AI: use according to the provider’s instructions, human oversight by competent people, monitoring operation, informing the provider and the supervisor of risks and incidents, and keeping the automatically generated logs for at least six months. For applications outside high risk this is not mandatory, but it is what a supervisor, the auditor and the council expect when something goes wrong. See DPIA and fundamental rights assessment for the assessment beforehand; this article is about afterwards.
What you measure
| Figure | What it says | Source |
|---|---|---|
| Use per role | Who uses it for what; shifts point to new use that has not been assessed | Audit log |
| Correction rate | How often staff change the output before use | Spot check or confirmation step |
| Source references | Share of answers with a valid source; a drop means outdated sources | Automatic check |
| Reports | Number and nature of reported errors, time to resolution | Reporting point |
| Outcomes per group | For applications affecting residents: differences between groups | Periodic analysis |
| Availability | Outages and response time | Operations |
See Testing algorithms for discrimination for the outcomes per group.
The routine
- Monthly an overview of the figures to the application owner and the data protection officer.
- Quarterly a sample of output, reviewed by professionals against the source.
- On every change of model, source or use: a short re-check and a note in the register.
- Annually a reassessment: is the risk classification still right, are the measures still sufficient, is the application still needed.
- On an incident a fixed route: report, freeze the output, root cause, notification to provider and supervisor if high-risk, and feedback to those affected.
See Rolling out an AI assistant in local government for the roll-out that precedes this routine.
How werqly fits in
werqly AI Control supplies the audit log per role, model and source, with a retention period you set yourself. Model changes are an explicit choice of the authority, not a silent update by a provider, so every change is a moment for a re-check. See AI for local government.
Frequently asked questions
Questions on this topic
How long must AI logs be kept?
For high-risk AI at least six months under Article 26 of the AI Act, unless other legislation requires longer. For other applications you decide, in line with records and data protection law; in practice a year is a workable starting point.
What is an incident that must be reported?
For high-risk AI: a malfunction or event leading to harm to health, safety or fundamental rights. For every application a lower internal threshold is sensible: every error that affected a resident is recorded and fed back.
Who owns the monitoring?
The application owner in the line, with the data protection officer and the security officer as second readers. Not the supplier and not IT alone: the professional department knows what good output is.
What if a new model becomes available?
Treat it as a change: test on the same sample as the current model, record the differences, re-check, and only then switch. A provider that replaces models silently makes this impossible.
Sources
- Artificialintelligenceact.eu — Article 26: Obligations of deployers
- Dutch Algorithm Framework — Algoritmes.overheid.nl (Dutch)
This article is general information, not legal advice. Rules and guidance change; check the sources listed or ask your lawyer or DPO if in doubt.