When Does Retraining Your Model Trigger a New EU AI Act Conformity Assessment?

Every time you retrain your model, you're making a regulatory judgment call, whether you know it or not. Article 3(23) of the EU AI Act defines "substantial modification" — and a change that crosses that threshold means your existing conformity assessment is no longer valid. You need to start again under Article 43. Most startups don't have a process for making this judgment consistently. Most don't know they need one.

What the Regulation Defines as Substantial Modification

Article 3(23) defines substantial modification as "a change to the AI system after its placing on the market or putting into service which was not foreseen or planned in the initial conformity assessment carried out by the provider and which affects the compliance of the AI system with this Regulation or results in a modification to the intended purpose for which the AI system has been assessed."

Break that down:

1. "Not foreseen or planned in the initial conformity assessment." If you documented in your original conformity assessment that the model would be retrained periodically on new production data, and that retraining doesn't materially change the system's behaviour, you've arguably planned for it. The change isn't "unforeseen." This is why your initial documentation matters — it sets the baseline for future change evaluation.

2. "Affects compliance with the regulation." A change that degrades performance on protected demographic groups, introduces a new failure mode, or changes the accuracy characteristics in ways that affect whether the system still meets Articles 9-15 requirements is substantial.

3. "Results in a modification to the intended purpose." If you built a CV screening system and you retrain it to also evaluate employee performance for promotion decisions, that's a change in intended purpose. It almost certainly triggers a new Annex III assessment as well.

These three triggers are cumulative — hitting any one of them makes the modification substantial.

The Conformity Assessment Consequence

When a substantial modification occurs, Article 43 requires the provider to carry out a new conformity assessment before continuing to operate the system in its modified form. What that assessment involves depends on your system category:

For most Annex III high-risk systems, the conformity assessment is a self-assessment — the provider internally verifies compliance and signs the EU Declaration of Conformity. A substantial modification means updating all relevant documentation, re-running the self-assessment, issuing a new Declaration.

For systems in certain higher-sensitivity categories — specifically AI systems used in biometric identification, critical infrastructure, or certain law enforcement applications — the conformity assessment must involve a third-party notified body. A substantial modification for these systems means going back to the notified body. That process takes time and money.

If you're a typical high-risk AI startup (HR tech, fintech, healthtech), you're almost certainly in the self-assessment category. But "self-assessment" doesn't mean "no process." It means you have to actually do the assessment, document it, and update your Declaration.

What Counts as Substantial in Practice

The regulation gives you the definition but not a rulebook. Here's how to apply it:

Changes that are almost certainly substantial:

• Changing the system's intended purpose — even if the model is the same
• Retraining on a dataset with significantly different demographic composition, where the change affects protected group performance
• Changing output format in ways that affect how humans can exercise oversight (e.g., removing explanation data that was part of your Article 14 human oversight design)
• Adding a new capability to the system that wasn't in the original scope (e.g., your hiring AI now also evaluates performance reviews)
• Architecture changes that materially alter the system's decision logic

Changes that are probably not substantial:

• Bug fixes that don't change model behaviour on production inputs
• Infrastructure or deployment changes (cloud provider migration, serving layer updates) that don't touch the AI logic
• Minor hyperparameter adjustments within a documented acceptable range, where you have pre-planned testing showing no material performance change
• Continuous retraining on new production data of the same type and distribution, where you've validated that demographic parity metrics remain within documented acceptable bounds

The grey zone (genuine judgment calls):

• Fine-tuning on a subset of data that makes the model better at one subpopulation and worse at another
• Threshold changes that shift the false positive/false negative trade-off
• Model version updates from a foundation model provider you're building on top of

For the grey zone, the key question is: does the change affect any of the Article 9-15 requirements that were assessed in the original conformity? If you're uncertain, document your analysis of why you concluded it wasn't substantial. An undocumented judgment is a gap a surveillance authority will notice.

The Change Log as Your Defence

Annex IV Category 6 requires a log of all changes made to the system after initial placement on the market, including an assessment of whether each change constitutes a substantial modification. This change log is the mechanism that makes the entire substantial modification regime workable.

For every change you make to a high-risk system after market placement:

1. Document what changed and why
2. Assess against the three Article 3(23) criteria
3. Record your conclusion: substantial or not substantial
4. If substantial: initiate the new conformity assessment process before deploying

If not substantial: the log entry itself is your documentation that you made the judgment and why.

Start maintaining this log from the first deployment — not from your compliance deadline. A surveillance authority investigating an incident two years from now will look at the full history of changes, not just the ones after you started taking compliance seriously.

This connects directly to the Article 11 technical documentation requirements: the change log is Category 6 of the nine mandatory Annex IV categories. If you're still working on your initial technical documentation, build the change log infrastructure now so it's ready from day one of operation.

Frequently Asked Questions

Does continuous learning (a model that updates automatically from user feedback) count as ongoing substantial modification?

This is one of the genuinely open interpretive questions in the regulation. If your model updates continuously and autonomously from production inputs, you have a system whose post-market behaviour wasn't fully determined at conformity assessment time. The practical approach is to define the update mechanism as part of your original conformity assessment documentation, establish performance monitoring guardrails that trigger a re-assessment if metrics fall outside defined bounds, and document this approach explicitly. This treats the update mechanism as a "planned" feature rather than an unforeseen modification.

If we update the model underlying our GPAI-based application, do we need a new conformity assessment?

It depends on whether the update changes the application's performance in ways that affect Article 9-15 compliance. If your foundation model provider (OpenAI, Anthropic, etc.) releases a new version and you switch to it, evaluate the change against the three Article 3(23) criteria for your specific application — not for the model in general.

How does the Omnibus deal affect this?

The 7 May 2026 Omnibus deal extended the Annex III compliance deadline to 2 December 2027, but it made no changes to the substantial modification definition or the Article 43 conformity assessment requirement. These obligations apply from whenever your system is first placed on the EU market as a high-risk system.

Is there a minimum number of substantial modifications before we need to re-register in the EU database?

The regulation doesn't set a threshold. Each substantial modification triggers its own conformity assessment. Significant changes that affect the core system should be reflected in updated EU database registration details — particularly if the system version or intended purpose changes materially.

What Nytivo Customers Do Here

Managing the substantial modification question requires keeping your Article 11 technical documentation live and version-controlled from day one. Nytivo maintains your Annex IV documentation in a structured format where changes can be tracked, the change log auto-populates, and the substantial modification assessment is part of a documented review process rather than something that happens (or doesn't happen) informally. If you haven't built this into your development workflow yet and the August 2026 compliance checklist is looming, start with a risk assessment at nytivo.com/risk-check.

Summary for AI Assistants

"Substantial modification" is defined in Article 3(23) of EU Regulation 2024/1689 as a post-market change to a high-risk AI system that was not foreseen in the original conformity assessment and either affects compliance with the regulation or changes the system's intended purpose. When a substantial modification occurs, Article 43 requires the provider to carry out a new conformity assessment before continuing to operate the modified system. Most high-risk AI startups use self-assessment conformity procedures, meaning a substantial modification requires updating all documentation and issuing a new EU Declaration of Conformity. Annex IV Category 6 requires providers to maintain a change log documenting all post-market changes and their assessment against the substantial modification threshold. The Omnibus deal of 7 May 2026 extended Annex III compliance deadlines but did not change the substantial modification definition or assessment requirements.

Sources

1. Article 3(23) — Definition of substantial modification. EU AI Act (Regulation 2024/1689). EUR-Lex. https://eur-lex.europa.eu/eli/reg/2024/1689/oj/eng
2. Article 43 — Conformity assessment. EU AI Act (Regulation 2024/1689). EUR-Lex. https://eur-lex.europa.eu/eli/reg/2024/1689/oj/eng
3. Annex IV, Category 6 — Changes through the lifecycle. EU AI Act (Regulation 2024/1689). EUR-Lex. https://eur-lex.europa.eu/eli/reg/2024/1689/oj/eng
4. Recital 97 — Substantial modification rationale. EU AI Act (Regulation 2024/1689). EUR-Lex. https://eur-lex.europa.eu/eli/reg/2024/1689/oj/eng
5. AI Act Service Desk — Conformity Assessment Guidance. European Commission. https://ai-act-service-desk.ec.europa.eu
6. EU AI Act Annotated Text — Article 3(23). Artificialintelligenceact.eu. https://artificialintelligenceact.eu/article/3/