The Conformity Assessment Rests on a Table the Industry Just Kicked Out

Article 43 of the AI Act is a quiet article. It instructs providers of high-risk AI systems to perform a conformity assessment — in most cases an internal one, in some cases involving a notified body — before putting the system on the Union market. The architecture is familiar to anyone who has ever certified a medical device or a children's toy. You test the thing. You document that you tested the thing. You keep the file. A regulator, in theory, can ask to see the file.

The assumption underneath is that the thing can be tested. That assumption is now being contested by the people who build the things.

At a conference in San Francisco last week, the chief policy officer of one of the largest open-model platforms described the current state of model evaluation in terms that any Brussels official reading the transcript should find unsettling. Benchmarks, she said, are an evolving science. We do not know how to meaningfully compare them to each other. They start to get ineffective relatively quickly. Reproducibility — the baseline requirement of any test that regulators are supposed to rely on — remains, in her phrasing, one of the stickier aspects of the field.

This is not a dissident view from a critic of the industry. This is the policy chief of a platform that hosts a significant share of the world's openly available models, speaking at an industry showcase, telling the room that the measurement apparatus is not ready.

Meanwhile, the solution the labs have proposed in place of state regulation — embedded third-party evaluators, so-called neutral observers placed inside AI organizations — has itself become a subject of satire. When one of the better-known American labs appointed a global consultancy as its first external evaluator, the reaction from technically literate observers was something between disbelief and amusement. A consultancy with a sprawling government-services practice is not, on any serious reading, a neutral party to the regulation of its own clients.

And then, on the same day, the largest enterprise software company in the world released a security product built on the premise that AI agents inside a company should be treated as insider threats. The chief executive's framing was blunt: frontier models cannot be trusted. The product assumes the AI deployed under contract, paid for and provisioned by the enterprise, may behave like a rogue employee. This is not a marketing flourish. It is the operational posture the vendor is now selling.

Stack these three disclosures and look at what the AI Act asks of a compliance officer in Lyon on Monday morning. She is asked to certify, under Article 16, that her firm's high-risk system meets the requirements of Chapter III, Section 2 — risk management, data governance, accuracy, robustness, cybersecurity. She is asked to run evaluations whose scientific status is openly described as immature by the industry's own policy leads. She is asked to rely, for third-party assurance, on an ecosystem of evaluators whose qualifications, in the same policy lead's words, need a lot more standardization. And she is asked to attest to the robustness of a system whose own vendor advises her to treat it as a potential insider threat.

This is not a drafting problem. The text of the Act is, in places, quite good. It is an enforcement problem of a particular kind: the measurement instruments the Act presupposes do not yet exist at the fidelity the Act requires. The AI Office in Brussels, the national market surveillance authorities, the notified bodies — none of them can audit what the industry cannot reliably measure.

Which capital will say this first? Paris has an interest in sounding the alarm, because its national champion is one of the few European frontier labs and would benefit from a slower, more rigorous conformity regime. Berlin's instinct, filtered through its industrial base, will be to push for pragmatic codes of practice that let deployment continue. Dublin, host to most of the American labs' European entities, has every reason to keep quiet. The Commission will not reopen the file. It cannot. The political capital spent on passing the Act forbids admitting that its central technical premise is contested.

The transmission to the Gulf is already underway. Firms in Riyadh and Abu Dhabi building Arabic-language models for European deployment are quietly asking their European counsel what, exactly, a conformity assessment looks like when the benchmarks are not reproducible. Their counsel does not know. Nobody does.

A regulation whose core test the regulated industry has publicly described as not yet science is a regulation whose first enforcement action will be its most important. We will learn, that day, whether the Act has teeth or whether it has a very elegant mouth.