Europe AI Act is catching up on Generative AI
The AI Act draft was approved by EU Parliament. Similarly to GDPR it’s a piece of legislation everyone dealing with AI should be aware of.
On June 14th the European Parliament voted for the adoption of the Artificial Intelligence Act with a vast majority. A fundamental piece of regulation that will come into action by 2026, after a final discussion with the EU Countries and the European Council, where the text could be further amended. It’s a bold and for some aspects clumsy, first large-scale attempt to provide a legal framework for AI. Also a catch-up exercise, given that the legislators had to quickly adapt to the new disruptive scenario of Generative AI, triggering an outcry from the business world over the risks for competition and innovation.
The AI Act's goal is to lay down the conditions for the development and deployment of AI products, fostering innovation and ensuring that those systems are safe, fair, and trusted. After all, this is exactly what some of the makers of those AI systems recently advocated for. Clearly, as with any regulation, the main risk is to achieve the opposite of the intended goal, by imposing a burden on AI makers, which can give an even greater advantage to companies outside this legislation.
What does the regulation say?
The AI Act follows a risk-based approach, with differential regulation requirements, and prohibited models at the top of the ladder. Those are mostly models that belong to the dystopian movie screenwrite notepad (but that are unfortunately available in some countries), for example:
- Models that deliver subliminal content in ways that can cause psychological harm.
- Models exploiting the vulnerabilities of a specific group of persons due to their age, physical or mental disability,y
- “Social Scoring” systems, evaluating the trustworthiness of natural persons over a certain period of time based on their social behavior.
- Real-Time biometric AI for law enforcement (with some exceptions).
However, the biggest chunk of regulations concerns the High-Risk models, identified by Art 7 of the draft as systems that “pose a risk of harm to the health and safety, or a risk of adverse impact on fundamental rights”. Those are listed in Annex III of the draft and in continuous evolution. If your model belongs to the list in the image below, then it will have to abide by a set of strict requirements and go through an assessment process, either carried out internally or by an external examiner. Although some researchers argue that there will be very few cases where a company will require an external examiner.
The requirements of High-Risk models are listed in articles 9 to 15 of the draft, they refer to the governance and correct management of the model, more specifically:
- The need for a risk management system, to be documented, reviewed, and periodically updated. For example, what is an impact of a false positive? What happens if the model is subject to an adversarial attack? (Art. 9)
- Quality Data and Data Governance. Models should use a train, validation, and test set. Data transformation and features engineering documented. Model bias monitored and corrected, etc. (Art. 10)
- Technical Documentation is drafted before the model is put into service. (Art. 11)
- Model events should be logged and records kept for a sufficient amount of time. (Art. 12)
- Transparent and clear information to users, including expected accuracy, how the data was collected, how the model is monitored and improved, and how it should be used. (Art. 13)
- Human oversight, with the provision of “appropriate human-machine interface tools, that can be effectively overseen by natural persons during the period in which the AI system is in use”. (Art. 14)
- Proper accuracy, robustness, and cybersecurity. (Art.15).
This is one of the most controversial points for businesses since the requirements and assessment may represent an extra layer of bureaucracy that could inflict an extra cost and may delay model deployment in production by months. Some analysts go as far as estimating that the total impact of the regulation will cost the economy up to 31 Billion euros in the next 5 years and slow down the pace of AI investment. On the other end, as a citizen and user of AI models, I would be appalled to discover that the algorithm deciding if I need to lose my job or not didn’t go through a process as rigorous as the one listed in articles 9–15 of the Act.
The AI Act doesn’t regulate in depth all the other models, with some exceptions for the Generative AI Models, which we will review later. For the models that interact with humans or generatively manipulate video/image/audio, considered Medium Risk, the Act states in Art. 52 some transparency obligations. In a nutshell, users should be notified that they are interacting with a machine or that an image was generated by an algorithm.
As a general principle, the AI Act strives to support innovation without putting a high burden on small-scale start-ups, following the principle of “proportionality”, which is a recurring word in the document. I found interesting the provision of “AI regulatory sandboxes” for research and development. As stated in Art. 53 those are “controlled environment that facilitates the development, testing, and validation of innovative AI systems for a limited time before their placement on the market or putting into service pursuant to a specific plan”. How will they work in practice is not very clear though and is left to future definition, and we know that the devil is in the details.
The AI Act and Generative AI
The advent and tremendous popularity of Chat GPT forced the regulator to quickly come up with some legislative provisions, in the amendments to the initial draft. They defined the “Foundation Models”, which are not considered High-Risk models, but yet have to follow some extra requirements than Medium Risk models. This added some unnecessary extra confusion, although I don’t feel like blaming the MPs, given that even the fathers of AI disagree on key concepts and about the risks of this AI.
Foundation Models are multitasking and general-purpose models like the LLMs (ChatGPT, LLama, Palm2, etc). As per Recital 60 in the amendments those are: “models often trained on a broad range of data sources and large amounts of data to accomplish a wide range of downstream tasks, including some for which they were not specifically developed and trained”. In the amended draft the regulator included Article 28b with a set of obligations and requirements for the providers of foundation models, regardless of their open-source nature. Although Recital 60g, states that: “These specific requirements and obligations do not amount to considering foundation models as high-risk AI systems”, article 28b was put in the High-Risk models section and includes very similar requirements, with some additions. On top of requirements for risk mitigation, data quality, technical specifications, and safety measures, Foundation Models will need to be energy efficient: “ They shall be designed with capabilities enabling the measurement and logging of the consumption of energy and resources, and, where technically feasible, other environmental impacts “.
And what about the data they used for training? Art 28b make it clear that also the Foundation Models are subject to the transparency requirements of Article 52. And more of that, they should “without prejudice to national or Union legislation on copyright, document and make publicly available a sufficiently detailed summary of the use of training data protected under copyright law”. This is the part that generated the stronger reaction: will the likes of OpenAI withdraw from Europe as it will be impossible to comply? In my opinion, it will not be the case since it should be sufficient to document the source of data with a detailed description of the data semantics. Moreover, I struggle to deem unreasonable the request of providing clarity on which copyright-protected data, if any, was used for training some LLMs.
Stanford University researchers from the CRFM, scored some of the most popular LLMs according to the AI Act requirements. They found that almost all the models fall short of expectations, especially on Copyrighted Data, Energy, and Compute resources disclosure. The authors concluded anyway that compliance with the regulation is feasible and that it will improve transparency and trust in those models. Which in turn will benefit the entire ecosystem.
Parting Thoughts
I believe the AI Act will set the scene for many other similar regulations globally. The technology is evolving at a faster pace than governments' capability to implement laws that protect consumers, but the AI Act managed to keep the right equilibrium and provided a framework that doesn’t get into the trap of over-regulating. In fairness, it could have gone even further on transparency and data protection. Also, it’s striking that models for military applications were considered completely out of scope.
Some rules, like the ones defining high-risk models assessment and notification to local authorities, may represent a burden that might actually slow down models development and deployment. It’s critical that the State Members keep the dialogue open on those procedures, involving experts and businesses.
Overall the Act goes in the right direction. AI should be a force for humanity and not a threat. It should be safe, reliable, trusted and fair. It should help humanity towards their common goals and not be used to polarize, discriminate and generate higher inequality.
References
[1] The 2021 draft of the AI Act can be found at this link: https://artificialintelligenceact.eu/the-act/
[2] Annexes to the AI Act: https://artificialintelligenceact.eu/annexes/
[3] Amendments to the AI Act as approved on June 14th: https://www.europarl.europa.eu/resources/library/media/20230516RES90302/20230516RES90302.pdf
