Home / Glossary

High-risk AI system

A high-risk AI system is a category defined by the EU AI Act, Regulation (EU) 2024/1689: an AI system used as a safety component of a product already covered by EU product legislation, or one whose purpose falls inside the regulation's annexed list of high-risk uses.

Auditrek is not open yet, so anything on this page about what it does is what it is being built to do.

Definition

A high-risk AI system, in the sense the EU AI Act uses the phrase, is an AI system that reaches the regulation's high-risk tier by one of two routes: it is intended to be used as a safety component of a product, or is itself a product, covered by the EU product-safety legislation the Act lists, and requires third-party conformity assessment under that legislation; or its intended purpose falls within one of the use-case areas the Act sets out in its annex on high-risk uses.

That annexed list is organised by domain rather than by technique. It reaches, among others, biometric identification and categorisation, safety components in critical infrastructure, access to education and vocational training, employment and worker management including CV screening, access to essential private and public services such as creditworthiness evaluation of natural persons and risk pricing in life and health insurance, and certain law-enforcement, migration and administration-of-justice uses.

Two things follow that compliance officers tend to reach for early. First, the classification attaches to the intended purpose, not to the model architecture: the same gradient-boosted tree is unremarkable in demand forecasting and in scope when it scores loan applicants. Second, the regulation contains a filter for the annexed list, allowing a provider who considers a listed use case not to pose a significant risk of harm to health, safety or fundamental rights to document that assessment rather than treat the system as high-risk. The documentation of that reasoning is the point. An undocumented conclusion is indistinguishable, from the outside, from no assessment at all.

Nothing on this page is legal advice, and a classification question with money attached belongs with qualified regulatory counsel and the Official Journal text, not with a vendor glossary.

What the classification changes, and a worked example

Once a system sits in the high-risk tier, a cluster of obligations follows it: a risk management system, data and data-governance requirements, technical documentation, automatic recording of events over the system's lifetime under the record-keeping article, transparency and instructions for use directed at deployers, human oversight, accuracy and robustness expectations, conformity assessment before the system goes on the market, and post-market monitoring afterwards. Providers and deployers carry different parts of that load, and an organisation buying a model and putting it into its own underwriting flow can find itself holding more of it than it expected.

An example

A mid-market consumer lender runs an affordability model behind its application flow. The model returns a score and a confidence value; an underwriter sees both and can override the recommendation. The model itself is ordinary. The use, creditworthiness evaluation of natural persons, is the kind of use the annexed list names, so the lender is not arguing about whether the maths is sophisticated enough to matter. It is answering narrower questions. Which system version produced this applicant's score, on which date. What the input was, without retaining the raw applicant file. Whether a human reviewed the output, and whether that is recorded anywhere a supervisory authority would accept. Whether the record of the decision can be shown not to have been edited after the complaint arrived.

That is the practical shape of the high-risk label for most mid-market teams: not a philosophical question about risk, an evidentiary one about what the organisation can still produce eighteen months later.

Related terms

  • Provider and deployer. The Act splits duties between the party that develops an AI system and places it on the market under its own name, and the party that uses it under its own authority. Substantially modifying a purchased system, or putting your own name on it, can move you from one column to the other.
  • Annexed high-risk use cases. The domain-organised list that forms the second route into the high-risk tier. Most commercial AI in finance, insurance, HR and healthcare triage that ends up in scope arrives through this list rather than through product-safety legislation.
  • General-purpose AI model. A separate track in the regulation, with its own obligations for model providers. A general-purpose model is not high-risk by definition; building it into a high-risk use case is what brings the use-case obligations into play.
  • Conformity assessment and CE marking. The pre-market procedure by which a high-risk system is declared to meet the regulation's requirements, and the mark that records it.
  • Record-keeping and logging. The obligation to design high-risk systems for automatic recording of events across their lifetime. Our sibling guide walks the fields an AI log has to contain, article by article, rather than restating the definition here.
  • Human oversight. The requirement that high-risk systems be designed so natural persons can oversee them effectively. Whether oversight happened is an evidence question, which is why the override flag is worth treating as a stored field rather than a process claim.
  • Post-market monitoring. The provider-side duty to keep watching a system's behaviour after it is in use, and to feed what is found back into risk management.
  • Tamper-evident log. A record structure where each entry commits to the one before it, usually by hash, so that a later edit is detectable rather than invisible. Adjacent to, but not the same as, append-only storage.

Questions

Who decides whether an AI system is high-risk?

In the first instance the provider does, by assessing its system's intended purpose against the regulation's classification rules and annexed list, and documenting the conclusion. National market surveillance and supervisory authorities can take a different view later, which is why the written assessment, and the records that support it, matter more than the conclusion on its own.

Is a large language model a high-risk AI system?

Not by virtue of being a large language model. The regulation handles general-purpose AI models on a separate track with their own provider obligations. A general-purpose model wired into an annexed use case, for example screening job applicants, brings that use case's obligations with it.

Is high-risk the same as prohibited?

No. The Act has a separate tier of prohibited practices that may not be placed on the market or used at all. High-risk systems are permitted, subject to the requirements attached to them. The two tiers are often conflated in summaries; the distinction is in the regulation's own text.

Does a system stop being high-risk if a human makes the final call?

Human involvement does not remove a system from the tier. Human oversight is one of the requirements that applies to high-risk systems, not an exemption from the classification. What it does change is what you need to be able to show, namely that the oversight was real and that it was recorded.

When do the high-risk obligations start applying?

The regulation applies in phases, with different provisions taking effect on different dates set out in its final articles, and the high-risk timeline has been the subject of continuing political discussion. Read the dates from the Official Journal text and from the European Commission's own regulatory framework page rather than from secondary summaries, including this one.

What evidence do supervisory authorities expect for a high-risk system?

The regulation describes documentation and logging duties in structural terms rather than prescribing a file format, so the practical answer is whatever lets you reconstruct what a system did, when, on which version, and with what human involvement. Our Article 12 guide covers the fields that follow from the record-keeping obligation.

Sources

Join the waitlist

Leave your work email and we will send the payload schema, the verification endpoint documentation and a sample Article 12 and 13 report before you talk to anyone. Pilot cohort conversations start with your own pipeline, not a demo script.

One email when it launches, and one now to confirm your address. Unsubscribe any time. Privacy.