Harmonised rules on Artificial Intelligence (Artificial Intelligence Act) and amending certain Union Legislative Acts

European Union · EU:ENVI-PA-699056

Authoritative source — the official register ↗.

European Parliament 2019-2024 {ENVI}Committee on the Environment, Public Health and Food Safety 2021/0106 (COD) {10/12/2021}10.12.2021

DRAFT OPINION

of the Committee on the Environment, Public Health and Food Safety for the Committee on the Internal Market and Consumer Protection and for the Committee on Civil Liberties, Justice and Home Affairs on the proposal for a regulation of the European Parliament and of the Council laying down harmonised rules on Artificial Intelligence (Artificial Intelligence Act) and amending certain Union Legislative Acts

(COM(2021)0206 – C90146/2021 – 2021/0106(COD))

Rapporteur for opinion: Susana Solís Pérez

PA_Legam

SHORT JUSTIFICATION

The proposal for Regulation of the European Parliament and the Council laying down harmonized rules on Artificial Intelligence (hereinafter “AI Act”) is part of the broader agenda to boost Europe in the digital age and achieve its environmental and climate objectives . This stems from the fact that AI currently plays a role in all aspects of European daily life activities.

AI systems will become more and more embedded into products and services therefore requiring a horizontal legislative approach as set out in the AI Act. The Rapporteur is fully aligned with this as she considers that we must establish the common rules to provide a cross-cutting approach to all sectors, including the healthcare sector. By doing so, the European Union has a chance to lead and set the standards of AI worldwide, as it has already done with data protection through GDPR. The EU could also become a global leader in niche sectors that require a very forward-looking perspective such as the regulation of neurological rights.

Overall the AI act should preserve European values, facilitating the distribution of AI’s benefits across society, protecting individuals, companies and the environment from risks while boosting innovation and employment and making Europe a leader in the field.

In this regard, the Rapporteur wants to emphasize the importance of sandboxes in certain areas (e.g. Health) and how it could be extended to other areas such as Hospitals, Health Authorities and research centers in order to reinforce and expand the leading position of the health system in all the Member States and at EU level. Health is wealth. By applying AI in health using interoperable health data we could further increase this wealth from health systems to society at large. The Rapporteur also highlights the potential implications of AI systems in mental health.

The Rapporteur for the opinion deems that the proposal insufficiently anticipates the risks of not having a common and consistent regulatory approach.

As a horizontal legislative initiative, the proposed AI Act is expected to intersect with several regulations currently in place (e.g GDPR or MDR) and several legislative initiatives that might intersect in the future such as the European Health Data Space. All these initiatives should be aligned with the AI Act to ensure a common and consistent regulatory approach therefore avoiding duplication of functions or discoordination among bodies and authorities at both the EU and Member State level.

The Rapporteur for the opinion is concerned that the AI Act does not provide sufficient protection to the environment.

The Special Eurobarometer 513 Climate change published in 2021 shows that tackling climate and environmental-related challenges is one of the main concerns for European citizens. Therefore, the Rapporteur proposes that the AI Act shall include the environment among the areas that require a high level of protection. In order to do so, the environment has been included in all the recitals and articles together with health, safety and the protection of fundamental rights. This will entail the classification as “high risk AI” of all those systems that can have major negative implications on the environment. At the same time, the Rapporteur has reinforced the right to proper redress mechanisms in case of negative environmental impacts as set out in the Aarhus Convention, and has set the principle of “Do no significant harm” as established in the Taxonomy Regulation as a limit to ensure that AI systems abide with the EU´s high level of environmental standards and rights.

The Rapporteur for the opinion considers that the AI Act shall not just cover users but must expand its scope to end recipients too. Many of the applications mentioned in the proposed AI Act will involve not just users but end recipients. In the case of healthcare applications this distinction is crucial as there is a clear differentiation between the intended use and capabilities of patients and doctors. Therefore, the draft report now includes a new definition of end recipients and grants them the appropriate degree of transparency and provision of specific information.

AMENDMENTS

The Committee on the Environment, Public Health and Food Safety calls on the Committee on the Internal Market and Consumer Protection, as the committee responsible, to take into account the following amendments: <RepeatBlock-Amend> Amendment 1 Proposal for a regulation Recital 1 Text proposed by the Commission Amendment

(1) The purpose of this Regulation is to improve the functioning of the internal market by laying down a uniform legal framework in particular for the development, marketing and use of artificial intelligence in conformity with Union values.

This Regulation pursues a number of overriding reasons of public interest, such as a high level of protection of health, safety and fundamental rights, and it ensures the free movement of AI-based goods and services cross-border, thus preventing Member States from imposing restrictions on the development, marketing and use of AI systems, unless explicitly authorised by this Regulation. (1) The purpose of this Regulation is to improve the functioning of the internal market by laying down a uniform legal framework in particular for the development, marketing and use of artificial intelligence in conformity with Union values.

This Regulation pursues a number of overriding reasons of public interest, such as a high level of protection of health, safety, the environment and fundamental rights, and it ensures the free movement of AI-based goods and services cross-border, thus preventing Member States from imposing restrictions on the development, marketing and use of AI systems, unless explicitly authorised by this Regulation. Or. {EN}en Amendment 2 Proposal for a regulation Recital 1 a (new) Text proposed by the Commission Amendment

(1a) This Regulation should serve as a basis to promote health, wellbeing, prevent diseases, and foster supportive environments for healthy lifestyles. Or. {EN}en Amendment 3 Proposal for a regulation Recital 1 b (new) Text proposed by the Commission Amendment (1b) This Regulation should preserve the European values facilitating the distribution of AI benefits across society, protecting individuals, companies and the environment from risks while boosting innovation and employment and making Europe a leader in the field.

Or. {EN}en Amendment 4 Proposal for a regulation Recital 3

Text proposed by the Commission Amendment

(3) Artificial intelligence is a fast evolving family of technologies that can contribute to a wide array of economic and societal benefits across the entire spectrum of industries and social activities. By improving prediction, optimising operations and resource allocation, and personalising digital solutions available for individuals and organisations, the use of artificial intelligence can provide key competitive advantages to companies and support socially and environmentally beneficial outcomes, for example in healthcare, farming, education and training, infrastructure management, energy, transport and logistics, public services, security, justice, resource and energy efficiency, and climate change mitigation and adaptation.

By improving prediction, optimising operations and resource allocation, and personalising digital solutions available for individuals and organisations, the use of artificial intelligence can provide key competitive advantages to companies and support socially and environmentally beneficial outcomes, for example in access and provision of healthcare, including mental health, protection of biodiversity, farming, education and training, infrastructure management, crisis management, energy, sustainable transport and logistics, public services, security, justice, resource and energy efficiency, and climate change mitigation and adaptation. Or.

{EN}en Amendment 5 Proposal for a regulation Recital 3 a (new) Text proposed by the Commission Amendment

(3a) According to the definition of the World Health Organisation (WHO), "Health is a state of complete physical, mental and social well-being and not merely the absence of disease or infirmity." In order to improve the health of the population in the Union and reduce health inequalities, it is essential not to focus only on physical health. Digital technologies and especially Artificial Intelligence can have a direct negative impact on mental health. At the same time, we must unleash the full potential of AI in the development of prediction, detection, and treatment solutions for mental health.

Or. {EN}en Amendment 6 Proposal for a regulation Recital 3 b (new)

Text proposed by the Commission Amendment (3b) The right to physical and mental health is a fundamental human right and universal health coverage is a Sustainable Development Goal that all signatories have committed to achieve by 2030. Or. {EN}en Amendment 7 Proposal for a regulation Recital 3 c (new) Text proposed by the Commission Amendment (3c) The EU commits to progressing towards the recognition of the right to a clean, healthy and sustainable environment, as laid out in Resolution 48/13 of the U.N. Human Rights Council.

Or. {EN}en Amendment 8 Proposal for a regulation Recital 4

Text proposed by the Commission Amendment

(4) At the same time, depending on the circumstances regarding its specific application and use, artificial intelligence may generate risks and cause harm to public interests and rights that are protected by Union law, whether individual, societal or environmental. Such harm might be material or immaterial, present or future. Or. {EN}en Amendment 9 Proposal for a regulation Recital 4 a (new)

Text proposed by the Commission Amendment (4a) Tackling climate change and environmental-related challenges and reaching the objectives of the Paris Agreement are at the core of the Communication on “The European Green Deal”, adopted on 11 December 2019, where the Commission recalled the role of digital technologies such as artificial intelligence, 5G, cloud and edge computing and the internet of things to accelerate and maximise the impact of policies to deal with climate change mitigation and adaptation, protect the environment and address biodiversity loss.

Or. {EN}en Amendment 10 Proposal for a regulation Recital 4 b (new) Text proposed by the Commission Amendment

(4b) In its White Paper on "Artificial Intelligence - A European approach to excellence and trust" of 19 February 2020, the Commission recalls that artificial intelligence can contribute to finding solutions to some of the most pressing societal challenges, including the fight against climate change, biodiversity loss and environmental degradation and highlights the potential benefits and risks of artificial intelligence in relation to safety, health and wellbeing of individuals.

Or. {EN}en Amendment 11 Proposal for a regulation Recital 4 c (new) Text proposed by the Commission Amendment

(4c) AI applications can bring environmental and economic benefits and strengthen predictive capabilities that contribute to the fight against climate change, to meeting the Sustainable Development Goals (SDGs) and to achieving our target of becoming the first climate-neutral continent. In this sense, the use of AI has the potential to reduce global greenhouse gas emissions by up to 4 % by 2030 but it is important that AI systems and associated machinery are designed sustainably to reduce resource usage and energy consumption, thereby limiting the risks to the environment;

Or. {EN}en Amendment 12 Proposal for a regulation Recital 4 d (new) Text proposed by the Commission Amendment

(4d) Traditional identification of species has been time consuming and costly, which hinders real time biodiversity assessments. The integration of AI systems has the potential to move away from manual sorting and identification of species, which can play a role in animal conservation by allowing authorities to quickly identify, observe and monitor endangered species populations and help inform additional measures if needed for conservation purposes.

Or. {EN}en Amendment 13 Proposal for a regulation Recital 5

Text proposed by the Commission Amendment

(5) A Union legal framework laying down harmonised rules on artificial intelligence is therefore needed to foster the development, use and uptake of artificial intelligence in the internal market that at the same time meets a high level of protection of public interests, such as health and safety and the protection of fundamental rights, as recognised and protected by Union law. To achieve that objective, rules regulating the placing on the market and putting into service of certain AI systems should be laid down, thus ensuring the smooth functioning of the internal market and allowing those systems to benefit from the principle of free movement of goods and services.

By laying down those rules, this Regulation supports the objective of the Union of being a global leader in the development of secure, trustworthy and ethical artificial intelligence, as stated by the European Council33 , and it ensures the protection of ethical principles, as specifically requested by the European Parliament34 . (5) A Union legal framework laying down harmonised rules on artificial intelligence is therefore needed to foster the development, use and uptake of artificial intelligence in the internal market that at the same time meets a high level of protection of public interests, such as health and safety, the environment and the protection of fundamental rights, as recognised and protected by Union law.

To achieve that objective, rules regulating the placing on the market and putting into service of certain AI systems should be laid down, thus ensuring the smooth functioning of the internal market and allowing those systems to benefit from the principle of free movement of goods and services. By laying down those rules, this Regulation supports the objective of the Union of being a global leader in the development of secure, trustworthy and ethical artificial intelligence, as stated by the European Council33, and it ensures the protection of ethical principles, as specifically requested by the European Parliament34.

33 European Council, Special meeting of the European Council (1 and 2 October 2020) – Conclusions, EUCO 13/20, 2020, p. 6. 33 European Council, Special meeting of the European Council (1 and 2 October 2020) – Conclusions, EUCO 13/20, 2020, p. 6. 34 European Parliament resolution of 20 October 2020 with recommendations to the Commission on a framework of ethical aspects of artificial intelligence, robotics and related technologies, 2020/2012(INL). 34 European Parliament resolution of 20 October 2020 with recommendations to the Commission on a framework of ethical aspects of artificial intelligence, robotics and related technologies, 2020/2012(INL). Or.

{EN}en Amendment 14 Proposal for a regulation Recital 6 Text proposed by the Commission Amendment

(6) The notion of AI system should be clearly defined to ensure legal certainty, while providing the flexibility to accommodate future technological developments. The definition should be based on the key functional characteristics of the software, in particular the ability, for a given set of human-defined objectives, to generate outputs such as content, predictions, recommendations, or decisions which influence the environment with which the system interacts, be it in a physical or digital dimension.

AI systems can be designed to operate with varying levels of autonomy and be used on a stand-alone basis or as a component of a product, irrespective of whether the system is physically integrated into the product (embedded) or serve the functionality of the product without being integrated therein (non-embedded). The definition of AI system should be complemented by a list of specific techniques and approaches used for its development, which should be kept up-to–date in the light of market and technological developments through the adoption of delegated acts by the Commission to amend that list.

(6) The notion of AI system should be clearly defined to ensure legal certainty, while providing the flexibility to accommodate future technological developments, such as neurotechnology, which may put mental privacy at risk and require legislative proposals to protect neurodata. The definition should be based on the key functional characteristics of the software, in particular the ability, for a given set of human-defined objectives, to generate outputs such as content, predictions, recommendations, or decisions which influence the environment with which the system interacts, be it in a physical or digital dimension.

Text proposed by the Commission Amendment

(13) In order to ensure a consistent and high level of protection of public interests as regards health, safety and fundamental rights, common normative standards for all high-risk AI systems should be established. Those standards should be consistent with the Charter of fundamental rights of the European Union (the Charter) and should be non-discriminatory and in line with the Union’s international trade commitments.

(13) In order to ensure a consistent and high level of protection of public interests as regards health, safety, the environment and fundamental rights, common normative standards for all high-risk AI systems should be established. Those standards should be consistent with the Charter of fundamental rights of the European Union (the Charter) and should be non-discriminatory and in line with the Union’s international trade commitments. Or. {EN}en Amendment 16 Proposal for a regulation Recital 13 a (new)

Text proposed by the Commission Amendment (13a) AI systems shall fully respect the climate and environmental standards and priorities of the Union and the principle of ‘do no significant harm’ within the meaning of Article 17 of Regulation (EU) 2020/852 on the establishment of a framework to facilitate sustainable investment.

Or. {EN}en Amendment 17 Proposal for a regulation Recital 13 b (new) Text proposed by the Commission Amendment

(13b) AI in the field of health interventions has the potential to improve health outcomes, enhance the quality of care to patients respond to unmet needs, and also to foster the competitiveness of stakeholders and to improve the cost-effectiveness and sustainability of health services and medical care. The EU has the potential to become a leader in the application of AI in the healthcare sector.

Or. {EN}en Amendment 18 Proposal for a regulation Recital 13 c (new) Text proposed by the Commission Amendment

(13c) AI can unlock solutions in the health sector that could save millions of lives, improve our standard of living and improve patient care, especially in diagnosis, prognosis and treatment, patient engagement, adherence, management and follow-up, clinical decision-making, including predictive analytics, screening and optimization of clinical pathways, and pathology. AI can also improve prevention strategies, health system management and in the organization and provision of health services and medical care, including health promotion and disease prevention interventions.

Or. {EN}en Amendment 19 Proposal for a regulation Recital 16

Text proposed by the Commission Amendment

(16) The placing on the market, putting into service or use of certain AI systems intended to distort human behaviour, whereby physical or psychological harms are likely to occur, should be forbidden. Such AI systems deploy subliminal components individuals cannot perceive or exploit vulnerabilities of children and people due to their age, physical or mental incapacities. They do so with the intention to materially distort the behaviour of a person and in a manner that causes or is likely to cause harm to that or another person. The intention may not be presumed if the distortion of human behaviour results from factors external to the AI system which are outside of the control of the provider or the user.

Research for legitimate purposes in relation to such AI systems should not be stifled by the prohibition, if such research does not amount to use of the AI system in human-machine relations that exposes natural persons to harm and such research is carried out in accordance with recognised ethical standards for scientific research. (16) The placing on the market, putting into service or use of certain AI systems intended to distort human behaviour, whereby physical, psychological harms or disruption of the sense of oneself are likely to occur, should be forbidden. Such AI systems deploy subliminal components individuals cannot perceive or exploit vulnerabilities of children and people due to their age, physical or mental incapacities.

They do so with the intention to materially distort the behaviour of a person and in a manner that causes or is likely to cause harm to that or another person or remove ultimate control over personal decision-making, with unknown manipulation from external neurotechnologies. The intention may not be presumed if the distortion of human behaviour results from factors external to the AI system which are outside of the control of the provider or the user.

(27) High-risk AI systems should only be placed on the Union market or put into service if they comply with certain mandatory requirements.

Those requirements should ensure that high-risk AI systems available in the Union or whose output is otherwise used in the Union do not pose unacceptable risks to important Union public interests as recognised and protected by Union law. AI systems identified as high-risk should be limited to those that have a significant harmful impact on the health, safety and fundamental rights of persons in the Union and such limitation minimises any potential restriction to international trade, if any. (27) High-risk AI systems should only be placed on the Union market or put into service if they comply with certain mandatory requirements.

Those requirements should ensure that high-risk AI systems available in the Union or whose output is otherwise used in the Union do not pose unacceptable risks to important Union public interests as recognised and protected by Union law. AI systems identified as high-risk should be limited to those that have a significant harmful impact on the health, safety, the environment and fundamental rights of persons in the Union and such limitation minimises any potential restriction to international trade, if any. Or. {EN}en Amendment 21 Proposal for a regulation Recital 28

Text proposed by the Commission Amendment

(28) AI systems could produce adverse outcomes to health and safety of persons, in particular when such systems operate as components of products. Consistently with the objectives of Union harmonisation legislation to facilitate the free movement of products in the internal market and to ensure that only safe and otherwise compliant products find their way into the market, it is important that the safety risks that may be generated by a product as a whole due to its digital components, including AI systems, are duly prevented and mitigated. For instance, increasingly autonomous robots, whether in the context of manufacturing or personal assistance and care should be able to safely operate and performs their functions in complex environments.

Similarly, in the health sector where the stakes for life and health are particularly high, increasingly sophisticated diagnostics systems and systems supporting human decisions should be reliable and accurate. The extent of the adverse impact caused by the AI system on the fundamental rights protected by the Charter is of particular relevance when classifying an AI system as high-risk.

Those rights include the right to human dignity, respect for private and family life, protection of personal data, freedom of expression and information, freedom of assembly and of association, and non-discrimination, consumer protection, workers’ rights, rights of persons with disabilities, right to an effective remedy and to a fair trial, right of defence and the presumption of innocence, right to good administration. In addition to those rights, it is important to highlight that children have specific rights as enshrined in Article 24 of the EU Charter and in the United Nations Convention on the Rights of the Child (further elaborated in the UNCRC General Comment No.

25 as regards the digital environment), both of which require consideration of the children’s vulnerabilities and provision of such protection and care as necessary for their well-being. The fundamental right to a high level of environmental protection enshrined in the Charter and implemented in Union policies should also be considered when assessing the severity of the harm that an AI system can cause, including in relation to the health and safety of persons. (28) AI systems could produce adverse outcomes to health, the environment and safety of persons.

Consistently with the objectives of Union harmonisation legislation to facilitate the free movement of products in the internal market and to ensure that only safe and otherwise compliant products find their way into the market, it is important that the safety risks that may be generated by a product as a whole due to its digital components, including AI systems, are duly prevented and mitigated. For instance, increasingly autonomous robots, whether in the context of manufacturing or personal assistance and care should be able to safely operate and performs their functions in complex environments.

Similarly, in the health sector where the stakes for life and health are particularly high, increasingly sophisticated diagnostics systems and systems supporting human decisions such as neurological AI applications should be safe, reliable and accurate. The extent of the adverse impact caused by the AI system on the fundamental rights protected by the Charter and negative impact on the environment is of particular relevance when classifying an AI system as high-risk.

Those rights include the right to human dignity, respect for private and family life, protection of personal data, neurological data, freedom of expression and information, freedom of assembly and of association, and non-discrimination, consumer protection, workers’ rights, rights of persons with disabilities, right to an effective remedy and to a fair trial, right of defence and the presumption of innocence, right to good administration and the right to a high level of environmental protection and the improvement of the quality of the environment.

In addition to those rights, it is important to highlight that children have specific rights as enshrined in Article 24 of the EU Charter and in the United Nations Convention on the Rights of the Child (further elaborated in the UNCRC General Comment No. 25 as regards the digital environment), both of which require consideration of the children’s vulnerabilities and provision of such protection and care as necessary for their well-being. The fundamental right to a high level of environmental protection enshrined in the Charter and implemented in Union policies should also be considered when assessing the severity of the harm that an AI system can cause, including in relation to the health and safety of persons and the environment. Or.

{EN}en Amendment 22 Proposal for a regulation Recital 32 Text proposed by the Commission Amendment

(32) As regards stand-alone AI systems, meaning high-risk AI systems other than those that are safety components of products, or which are themselves products, it is appropriate to classify them as high-risk if, in the light of their intended purpose, they pose a high risk of harm to the health and safety or the fundamental rights of persons, taking into account both the severity of the possible harm and its probability of occurrence and they are used in a number of specifically pre-defined areas specified in the Regulation.

The identification of those systems is based on the same methodology and criteria envisaged also for any future amendments of the list of high-risk AI systems. (32) As regards stand-alone AI systems, meaning high-risk AI systems other than those that are safety components of products, or which are themselves products, it is appropriate to classify them as high-risk if, in the light of their intended purpose, they pose a high risk of harm to the health and safety, the environment or the fundamental rights of persons, taking into account both the severity of the possible harm and its probability of occurrence and they are used in a number of specifically pre-defined areas specified in the Regulation.

The identification of those systems is based on the same methodology and criteria envisaged also for any future amendments of the list of high-risk AI systems. Or. {EN}en Amendment 23 Proposal for a regulation Recital 34

Text proposed by the Commission Amendment

(34) As regards the management and operation of critical infrastructure, it is appropriate to classify as high-risk the AI systems intended to be used as safety components in the management and operation of road traffic and the supply of water, gas, heating and electricity, since their failure or malfunctioning may put at risk the life and health of persons at large scale and lead to appreciable disruptions in the ordinary conduct of social and economic activities.

(34) As regards the management and operation of critical infrastructure, it is appropriate to classify as high-risk the AI systems intended to be used as safety components in the management and operation of road traffic, the supply of water and gas, healthcare systems, natural or man-made disaster control mechanisms, heating and electricity, since their failure or malfunctioning may put at risk the life and health of persons and environment at large scale and lead to appreciable disruptions in the ordinary conduct of social and economic activities. Or. {EN}en Amendment 24 Proposal for a regulation Recital 37

Text proposed by the Commission Amendment

(37) Another area in which the use of AI systems deserves special consideration is the access to and enjoyment of certain essential private and public services and benefits necessary for people to fully participate in society or to improve one’s standard of living. In particular, AI systems used to evaluate the credit score or creditworthiness of natural persons should be classified as high-risk AI systems, since they determine those persons’ access to financial resources or essential services such as housing, electricity, and telecommunication services.

AI systems used for this purpose may lead to discrimination of persons or groups and perpetuate historical patterns of discrimination, for example based on racial or ethnic origins, disabilities, age, sexual orientation, or create new forms of discriminatory impacts. Considering the very limited scale of the impact and the available alternatives on the market, it is appropriate to exempt AI systems for the purpose of creditworthiness assessment and credit scoring when put into service by small-scale providers for their own use. Natural persons applying for or receiving public assistance benefits and services from public authorities are typically dependent on those benefits and services and in a vulnerable position in relation to the responsible authorities.

If AI systems are used for determining whether such benefits and services should be denied, reduced, revoked or reclaimed by authorities, they may have a significant impact on persons’ livelihood and may infringe their fundamental rights, such as the right to social protection, non-discrimination, human dignity or an effective remedy. Those systems should therefore be classified as high-risk. Nonetheless, this Regulation should not hamper the development and use of innovative approaches in the public administration, which would stand to benefit from a wider use of compliant and safe AI systems, provided that those systems do not entail a high risk to legal and natural persons.

Finally, AI systems used to dispatch or establish priority in the dispatching of emergency first response services should also be classified as high-risk since they make decisions in very critical situations for the life and health of persons and their property. (37) Another area in which the use of AI systems deserves special consideration is the access to and enjoyment of certain essential private and public services, including healthcare, and benefits necessary for people to fully participate in society or to improve one’s standard of living.

In particular, AI systems used to evaluate the credit score or creditworthiness of natural persons should be classified as high-risk AI systems, since they determine those persons’ access to financial resources or essential services such as housing, electricity, healthcare and telecommunication services. AI systems used for this purpose may lead to discrimination of persons or groups and perpetuate historical patterns of discrimination, for example based on racial or ethnic origins, disabilities, age, sexual orientation, or create new forms of discriminatory impacts.

Considering the very limited scale of the impact and the available alternatives on the market, it is appropriate to exempt AI systems for the purpose of creditworthiness assessment and credit scoring when put into service by small-scale providers for their own use. Natural persons applying for or receiving public assistance benefits and services from public authorities are typically dependent on those benefits and services and in a vulnerable position in relation to the responsible authorities.

If AI systems are used for determining whether such benefits and services should be denied, reduced, revoked or reclaimed by authorities, they may have a significant impact on persons’ livelihood, health and wellbeing, and may infringe their fundamental rights, such as the right to social protection, non-discrimination, human dignity or an effective remedy. Those systems should therefore be classified as high-risk. Nonetheless, this Regulation should not hamper the development and use of innovative approaches in the public administration, which would stand to benefit from a wider use of compliant and safe AI systems, provided that those systems do not entail a high risk to legal and natural persons.

Finally, AI systems used to dispatch or establish priority in the dispatching of emergency first response services, disease prevention, diagnosis, control and treatment should also be classified as high-risk since they make decisions in very critical situations for the life and health of persons and their property. Or. {EN}en Amendment 25 Proposal for a regulation Recital 38

Text proposed by the Commission Amendment

(38) Actions by law enforcement authorities involving certain uses of AI systems are characterised by a significant degree of power imbalance and may lead to surveillance, arrest or deprivation of a natural person’s liberty as well as other adverse impacts on fundamental rights guaranteed in the Charter. In particular, if the AI system is not trained with high quality data, does not meet adequate requirements in terms of its accuracy or robustness, or is not properly designed and tested before being put on the market or otherwise put into service, it may single out people in a discriminatory or otherwise incorrect or unjust manner.

Furthermore, the exercise of important procedural fundamental rights, such as the right to an effective remedy and to a fair trial as well as the right of defence and the presumption of innocence, could be hampered, in particular, where such AI systems are not sufficiently transparent, explainable and documented. It is therefore appropriate to classify as high-risk a number of AI systems intended to be used in the law enforcement context where accuracy, reliability and transparency is particularly important to avoid adverse impacts, retain public trust and ensure accountability and effective redress.

In view of the nature of the activities in question and the risks relating thereto, those high-risk AI systems should include in particular AI systems intended to be used by law enforcement authorities for individual risk assessments, polygraphs and similar tools or to detect the emotional state of natural person, to detect ‘deep fakes’, for the evaluation of the reliability of evidence in criminal proceedings, for predicting the occurrence or reoccurrence of an actual or potential criminal offence based on profiling of natural persons, or assessing personality traits and characteristics or past criminal behaviour of natural persons or groups, for profiling in the course of detection, investigation or prosecution of criminal offences, as well as for crime analytics regarding natural persons.

AI systems specifically intended to be used for administrative proceedings by tax and customs authorities should not be considered high-risk AI systems used by law enforcement authorities for the purposes of prevention, detection, investigation and prosecution of criminal offences. (38) Actions by law enforcement authorities involving certain uses of AI systems are characterised by a significant degree of power imbalance and may lead to surveillance, arrest or deprivation of a natural person’s liberty as well as other adverse impacts on fundamental rights guaranteed in the Charter.

In particular, if the AI system is not trained with high quality data, does not meet adequate requirements in terms of its accuracy or robustness, or is not properly designed and tested before being put on the market or otherwise put into service, it may single out people in a discriminatory or otherwise incorrect or unjust manner.

Furthermore, the exercise of important procedural fundamental rights, such as the right to an effective remedy, including the right to access to justice for environmental matters as established in the Aarhus Convention (Regulation 1367/2006, as amended by Regulation (EU) 2021/1767) and to a fair trial as well as the right of defence and the presumption of innocence, could be hampered, in particular, where such AI systems are not sufficiently transparent, explainable and documented. It is therefore appropriate to classify as high-risk a number of AI systems intended to be used in the law enforcement context where accuracy, reliability and transparency is particularly important to avoid adverse impacts, retain public trust and ensure accountability and effective redress.

Text proposed by the Commission Amendment

(43) Requirements should apply to high-risk AI systems as regards the quality of data sets used, technical documentation and record-keeping, transparency and the provision of information to users, human oversight, and robustness, accuracy and cybersecurity. Those requirements are necessary to effectively mitigate the risks for health, safety and fundamental rights, as applicable in the light of the intended purpose of the system, and no other less trade restrictive measures are reasonably available, thus avoiding unjustified restrictions to trade.

(43) Requirements should apply to high-risk AI systems as regards the quality of data sets used, technical documentation and record-keeping, transparency and the provision of information to users and end recipients, human oversight, and robustness, accuracy and cybersecurity. Those requirements are necessary to effectively mitigate the risks for health, safety, the environment and fundamental rights, as applicable in the light of the intended purpose of the system, and no other less trade restrictive measures are reasonably available, thus avoiding unjustified restrictions to trade. Or. {EN}en Amendment 27 Proposal for a

regulation Recital 45

27 Proposal for a regulation Recital 45 Text proposed by the Commission Amendment

(45) For the development of high-risk AI systems, certain actors, such as providers, notified bodies and other relevant entities, such as digital innovation hubs, testing experimentation facilities and researchers, should be able to access and use high quality datasets within their respective fields of activities which are related to this Regulation. European common data spaces established by the Commission and the facilitation of data sharing between businesses and with government in the public interest will be instrumental to provide trustful, accountable and non-discriminatory access to high quality data for the training, validation and testing of AI systems.

For example, in health, the European health data space will facilitate non-discriminatory access to health data and the training of artificial intelligence algorithms on those datasets, in a privacy-preserving, secure, timely, transparent and trustworthy manner, and with an appropriate institutional governance. Relevant competent authorities, including sectoral ones, providing or supporting the access to data may also support the provision of high-quality data for the training, validation and testing of AI systems.

(45) For the development of high-risk AI systems, certain actors, such as providers, notified bodies and other relevant entities, such as digital innovation hubs, research and scientific institutes, health authorities, hospitals testing experimentation facilities and researchers, should be able to access and use high quality datasets within their respective fields of activities which are related to this Regulation. European common data spaces established by the Commission and the facilitation of data sharing between businesses and with government in the public interest will be instrumental to provide trustful, accountable and non-discriminatory access to high quality data for the training, validation and testing of AI systems.

For example, in health, the European health data space will facilitate non-discriminatory access to health data and the training of artificial intelligence algorithms on those datasets, in a privacy-preserving, secure, timely, transparent and trustworthy manner, and with an appropriate institutional governance. Member States shall put in place incentives to ensure that the data is completely interoperable to unlock the full potential of Europe´s high quality healthcare services, while complying with the GDPR. Relevant competent authorities, including sectoral ones, providing or supporting the access to data may also support the provision of high-quality data for the training, validation and testing of AI systems. Or.

{EN}en Amendment 28 Proposal for a regulation Recital 47

Text proposed by the Commission Amendment

(47) To address the opacity that may make certain AI systems incomprehensible to or too complex for natural persons, a certain degree of transparency should be required for high-risk AI systems. Users should be able to interpret the system output and use it appropriately. High-risk AI systems should therefore be accompanied by relevant documentation and instructions of use and include concise and clear information, including in relation to possible risks to fundamental rights and discrimination, where appropriate.

(47) To address the opacity that may make certain AI systems incomprehensible to or too complex for natural persons, a certain degree of transparency should be required for high-risk AI systems. Users and end recipients should be able to interpret the system output and use it appropriately. High-risk AI systems should therefore be accompanied by relevant documentation and instructions of use and include concise and clear information, including in relation to possible risks to fundamental rights and discrimination, where appropriate. Or. {EN}en Amendment 29 Proposal for a regulation Recital 49

Text proposed by the Commission Amendment

(49) High-risk AI systems should perform consistently throughout their lifecycle and meet an appropriate level of accuracy, robustness and cybersecurity in accordance with the generally acknowledged state of the art. The level of accuracy and accuracy metrics should be communicated to the users.

Text proposed by the Commission Amendment

(50) The technical robustness is a key requirement for high-risk AI systems. They should be resilient against risks connected to the limitations of the system (e.g. errors, faults, inconsistencies, unexpected situations) as well as against malicious actions that may compromise the security of the AI system and result in harmful or otherwise undesirable behaviour. Failure to protect against these risks could lead to safety impacts or negatively affect the fundamental rights, for example due to erroneous decisions or wrong or biased outputs generated by the AI system.

(50) The technical robustness is a key requirement for high-risk AI systems. They should be resilient against risks connected to the limitations of the system (e.g. errors, faults, inconsistencies, unexpected situations) as well as against malicious actions that may compromise the security of the AI system and result in harmful or otherwise undesirable behaviour. Failure to protect against these risks could lead to safety impacts, negative environmental implications, or negatively affect the fundamental rights, for example due to erroneous decisions or wrong or biased outputs generated by the AI system. Or. {EN}en Amendment

31 Proposal for a regulation Recital 59 a (new)

Text proposed by the Commission Amendment (59a) Considering the specific nature and potential uses of AI systems which can be addressed to natural persons who are not users or operators, it is important to ensure the protection of certain rights, notably regarding transparency and the provision of information, to end recipients such as patients of healthcare services, students, consumers, etc. The current legislation should aim at providing the appropriate type and degree of transparency as well as the provision of specific information to end recipients and establish a clear difference with users as it can increase the protection and usability of AI systems and components.

Or. {EN}en Amendment 32 Proposal for a regulation Recital 72

Text proposed by the Commission Amendment

(72) The objectives of the regulatory sandboxes should be to foster AI innovation by establishing a controlled experimentation and testing environment in the development and pre-marketing phase with a view to ensuring compliance of the innovative AI systems with this Regulation and other relevant Union and Member States legislation; to enhance legal certainty for innovators and the competent authorities’ oversight and understanding of the opportunities, emerging risks and the impacts of AI use, and to accelerate access to markets, including by removing barriers for small and medium enterprises (SMEs) and start-ups.

To ensure uniform implementation across the Union and economies of scale, it is appropriate to establish common rules for the regulatory sandboxes’ implementation and a framework for cooperation between the relevant authorities involved in the supervision of the sandboxes. This Regulation should provide the legal basis for the use of personal data collected for other purposes for developing certain AI systems in the public interest within the AI regulatory sandbox, in line with Article 6(4) of Regulation (EU) 2016/679, and Article 6 of Regulation (EU) 2018/1725, and without prejudice to Article 4(2) of Directive (EU) 2016/680.

Participants in the sandbox should ensure appropriate safeguards and cooperate with the competent authorities, including by following their guidance and acting expeditiously and in good faith to mitigate any high-risks to safety and fundamental rights that may arise during the development and experimentation in the sandbox. The conduct of the participants in the sandbox should be taken into account when competent authorities decide whether to impose an administrative fine under Article 83(2) of Regulation 2016/679 and Article 57 of Directive 2016/680.

Participants in the sandbox should ensure appropriate safeguards and cooperate with the competent authorities, including by following their guidance and acting expeditiously and in good faith to mitigate any high-risks to safety, health, the environment and fundamental rights that may arise during the development and experimentation in the sandbox. The conduct of the participants in the sandbox should be taken into account when competent authorities decide whether to impose an administrative fine under Article 83(2) of Regulation 2016/679 and Article 57 of Directive 2016/680. Or. {EN}en Amendment 33 Proposal for a regulation Recital 74

Text proposed by the Commission Amendment

(74) In order to minimise the risks to implementation resulting from lack of knowledge and expertise in the market as well as to facilitate compliance of providers and notified bodies with their obligations under this Regulation, the AI-on demand platform, the European Digital Innovation Hubs and the Testing and Experimentation Facilities established by the Commission and the Member States at national or EU level should possibly contribute to the implementation of this Regulation. Within their respective mission and fields of competence, they may provide in particular technical and scientific support to providers and notified bodies.

(74) In order to minimise the risks to implementation resulting from lack of knowledge and expertise in the market as well as to facilitate compliance of providers and notified bodies with their obligations under this Regulation, the AI-on demand platform, the European Digital Innovation Hubs, the European Institute of Innovation and Technology, and the Testing and Experimentation Facilities established by the Commission and the Member States at national or EU level should possibly contribute to the implementation of this Regulation. Within their respective mission and fields of competence, they may provide in particular technical and scientific support to providers and notified bodies. Or.

{EN}en Amendment 34 Proposal for a regulation Recital 76 a (new) Text proposed by the Commission Amendment

(76a) To ensure that there is a common and consistent approach regarding the deployment and implementation of artificial intelligence systems in the various areas and sectors concerned and to exploit potential synergies and complementarities, the Board should cooperate closely with other relevant sectoral advisory groups established at Union level, such as boards, committees and expert groups, including organisations from the civil society such as NGOs, consumer associations, and industry representatives with competence in areas related to digital technologies or artificial intelligence, such as governance, exchange, access or use and re-use of data, including health data or environmental information, while avoiding duplication of work.

Or. {EN}en Amendment 35 Proposal for a regulation Article 3 – paragraph 1 – point 4 a (new)

Text proposed by the Commission Amendment (4a) ‘end recipient’ means any natural or legal person, other than an operator, to whom the output of an AI system is intended or to whom that output is provided; Or. {EN}en Amendment 36 Proposal for a regulation Article 3 – paragraph 1 – point 14

Text proposed by the Commission Amendment

(14) ‘safety component of a product or system’ means a component of a product or of a system which fulfils a safety function for that product or system or the failure or malfunctioning of which endangers the health and safety of persons or property;

Text proposed by the Commission Amendment

(15) ‘instructions for use’ means the information provided by the provider to inform the user of in particular an AI system’s intended purpose and proper use, inclusive of the specific geographical, behavioural or functional setting within which the high-risk AI system is intended to be used;

(15) ‘instructions for use’ means the information provided by the provider to inform the user and end recipient of in particular an AI system’s intended purpose and proper use, inclusive of the specific geographical, behavioural or functional setting within which the high-risk AI system is intended to be used; Or. {EN}en Amendment 38 Proposal for a regulation Article 3 – paragraph 1 – point 34

Text proposed by the Commission Amendment

(34) ‘emotion recognition system’ means an AI system for the purpose of identifying or inferring emotions or intentions of natural persons on the basis of their biometric data;

(34) ‘emotion recognition system’ means an AI system for the purpose of identifying or inferring emotions or intentions of natural persons on the basis of their biometric or neurological data; Or. {EN}en Amendment 39 Proposal for a regulation Article 7 – paragraph 1 – point b

Text proposed by the Commission Amendment

(b) the AI systems pose a risk of harm to the health and safety, or a risk of adverse impact on fundamental rights, that is, in respect of its severity and probability of occurrence, equivalent to or greater than the risk of harm or of adverse impact posed by the high-risk AI systems already referred to in Annex III.

(b) the AI systems pose a risk of harm to the health, safety of persons, or a risk of adverse impact on fundamental rights or the environment, that is, in respect of its severity and probability of occurrence, equivalent to or greater than the risk of harm or of adverse impact posed by the high-risk AI systems already referred to in Annex III. Or. {EN}en Amendment 40 Proposal for a regulation Article 7 – paragraph 2 – introductory part

Text proposed by the Commission

Amendment

2When assessing for the purposes of paragraph 1 whether an AI system poses a risk of harm to the health and safety or a risk of adverse impact on fundamental rights that is equivalent to or greater than the risk of harm posed by the high-risk AI systems already referred to in Annex III, the Commission shall take into account the following criteria:

2When assessing for the purposes of paragraph 1 whether an AI system poses a risk of harm to the health and safety or a risk of adverse impact on fundamental rights or the environment that is equivalent to or greater than the risk of harm posed by the high-risk AI systems already referred to in Annex III, the Commission shall take into account the following criteria:

Or. {EN}en Amendment 41 Proposal for a regulation Article 7 – paragraph 2 – point c

Text proposed by the Commission Amendment

(c) the extent to which the use of an AI system has already caused harm to the health and safety or adverse impact on the fundamental rights or has given rise to significant concerns in relation to the materialisation of such harm or adverse impact, as demonstrated by reports or documented allegations submitted to national competent authorities;

(c) the extent to which the use of an AI system has already caused harm to the health and safety or adverse impact on the fundamental rights and the environment or has given rise to significant concerns in relation to the materialisation of such harm or adverse impact, as demonstrated by reports or documented allegations submitted to national competent authorities; Or. {EN}en Amendment 42 Proposal for a regulation Article 7 – paragraph 2 – point d

Text proposed by the Commission Amendment

(d) the potential extent of such harm or such adverse impact, in particular in terms of its intensity and its ability to affect a plurality of persons;

(d) the potential extent of such harm or such adverse impact, in particular in terms of its intensity and its ability to affect a plurality of persons, the environment and biodiversity; Or. {EN}en Amendment 43 Proposal for a regulation Article 7 – paragraph 2 – point e

Text proposed by the Commission Amendment

(e) the extent to which potentially harmed or adversely impacted persons are dependent on the outcome produced with an AI system, in particular because for practical or legal reasons it is not reasonably possible to opt-out from that outcome;

(e) the extent to which potentially harmed or adversely impacted persons, including end recipients, are dependent on the outcome produced with an AI system, in particular because for practical or legal reasons it is not reasonably possible to opt-out from that outcome; Or. {EN}en Amendment 44 Proposal for a regulation Article 7 – paragraph 2 – point g

Text proposed by the Commission Amendment

(g) the extent to which the outcome produced with an AI system is easily reversible, whereby outcomes having an impact on the health or safety of persons shall not be considered as easily reversible;

(g) the extent to which the outcome produced with an AI system is easily reversible, whereby outcomes having an impact on the health or safety of persons, the environment or biodiversity, shall not be considered as easily reversible; Or. {EN}en Amendment 45 Proposal for a regulation Article 9 – paragraph 4 – subparagraph 1

Text proposed by the Commission Amendment The risk management measures referred to in paragraph 2, point (d) shall be such that any residual risk associated with each hazard as well as the overall residual risk of the high-risk AI systems is judged acceptable, provided that the high-risk AI system is used in accordance with its intended purpose or under conditions of reasonably foreseeable misuse. Those residual risks shall be communicated to the user.

The risk management measures referred to in paragraph 2, point (d) shall be such that any residual risk associated with each hazard as well as the overall residual risk of the high-risk AI systems is judged acceptable, provided that the high-risk AI system is used in accordance with its intended purpose or under conditions of reasonably foreseeable misuse. Those residual risks shall be communicated to the user and end recipient.

Or. {EN}en Amendment 46 Proposal for a regulation Article 10 – paragraph 3

Text proposed by the Commission

Amendment

3Training, validation and testing data sets shall be relevant, representative, free of errors and complete. They shall have the appropriate statistical properties, including, where applicable, as regards the persons or groups of persons on which the high-risk AI system is intended to be used. These characteristics of the data sets may be met at the level of individual data sets or a combination thereof.

3Training, validation and testing data sets shall be relevant, representative, free of errors and complete. They shall have the appropriate statistical properties, including, where applicable, as regards the persons or groups of persons on which the high-risk AI system is intended to be used, including end recipients. These characteristics of the data sets may be met at the level of individual data sets or a combination thereof.

Or. {EN}en Amendment 47 Proposal for a regulation Article 10 – paragraph 4 Text proposed by the Commission Amendment

4Training, validation and testing data sets shall take into account, to the extent required by the intended purpose, the characteristics or elements that are particular to the specific geographical, behavioural or functional setting within which the high-risk AI system is intended to be used.

4Training, validation and testing data sets shall take into account, to the extent required by the intended purpose, the characteristics or elements that are particular to the specific geographical, environmental, behavioural or functional setting within which the high-risk AI system is intended to be used.

Or. {EN}en Amendment 48 Proposal for a regulation Article 13 – title

Text proposed by the Commission Amendment Transparency and provision of information to users Transparency and provision of information to users and end recipients Or. {EN}en Amendment 49 Proposal for a regulation Article 13 – paragraph 1

Text proposed by the Commission

Amendment

1High-risk AI systems shall be designed and developed in such a way to ensure that their operation is sufficiently transparent to enable users to interpret the system’s output and use it appropriately. An appropriate type and degree of transparency shall be ensured, with a view to achieving compliance with the relevant obligations of the user and of the provider set out in Chapter 3 of this Title.

1High-risk AI systems shall be designed and developed in such a way to ensure that their operation is sufficiently transparent to enable users and end recipients to interpret the system’s output and use it appropriately. An appropriate type and degree of transparency shall be ensured, with a view to achieving compliance with the relevant obligations of the user, end recipient and of the provider set out in Chapter 3 of this Title.

Or. {EN}en Amendment 50 Proposal for a regulation Article 13 – paragraph 3 a (new) Text proposed by the Commission Amendment

3a. High-risk AI systems shall be designed, developed and used in such a way to ensure that the outputs are sufficiently transparent, relevant, accessible and comprehensible to the end recipients in accordance with the intended purpose. Or. {EN}en Amendment 51 Proposal for a regulation Article 14 – paragraph 2 Text proposed by the Commission Amendment

2Human oversight shall aim at preventing or minimising the risks to health, safety or fundamental rights that may emerge when a high-risk AI system is used in accordance with its intended purpose or under conditions of reasonably foreseeable misuse, in particular when such risks persist notwithstanding the application of other requirements set out in this Chapter.

2Human oversight shall aim at preventing or minimising the risks to health, safety or fundamental rights or the environment that may emerge when a high-risk AI system is used in accordance with its intended purpose or under conditions of reasonably foreseeable misuse, in particular when such risks persist notwithstanding the application of other requirements set out in this Chapter.

Or. {EN}en Amendment 52 Proposal for a regulation Article 53 – paragraph 3 Text proposed by the Commission Amendment

3The AI regulatory sandboxes shall not affect the supervisory and corrective powers of the competent authorities. Any significant risks to health and safety and fundamental rights identified during the development and testing of such systems shall result in immediate mitigation and, failing that, in the suspension of the development and testing process until such mitigation takes place.

3The AI regulatory sandboxes shall not affect the supervisory and corrective powers of the competent authorities. Any significant risks to health and safety, fundamental rights and the environment, identified during the development and testing of such systems shall result in immediate mitigation and, failing that, in the suspension of the development and testing process until such mitigation takes place.

Or. {EN}en Amendment 53 Proposal for a regulation Article 53 – paragraph 4 Text proposed by the Commission Amendment

4Participants in the AI regulatory sandbox shall remain liable under applicable Union and Member States liability legislation for any harm inflicted on third parties as a result from the experimentation taking place in the sandbox.

4Participants in the AI regulatory sandbox shall remain liable under applicable Union and Member States liability legislation for any harm inflicted on third parties or the environment as a result from the experimentation taking place in the sandbox.

Or. {EN}en Amendment 54 Proposal for a regulation Article 54 – paragraph 1 – point a – point ii

Text proposed by the Commission Amendment

(ii) public safety and public health, including disease prevention, control and treatment;

(ii) public safety and public health, including disease prevention, diagnosis, control and treatment; Or. {EN}en Amendment 55 Proposal for a regulation Article 54 – paragraph 1 – point a – point iii

Text proposed by the Commission Amendment

(iii) a high level of protection and improvement of the quality of the environment;

(iii) a high level of protection and improvement of the quality of the environment, protection of biodiversity as well as climate change mitigation and adaptation; Or. {EN}en Amendment 56 Proposal for a regulation Article 56 – paragraph 2 – point b

Text proposed by the Commission Amendment

(b) coordinate and contribute to guidance and analysis by the Commission and the national supervisory authorities and other competent authorities on emerging issues across the internal market with regard to matters covered by this Regulation;

(b) coordinate and contribute to guidance and analysis by the Commission and the national supervisory authorities as well as advisory and expert groups, including organisations from the civil society such as NGOs, consumer associations, and industry representatives and other competent authorities on emerging issues across the internal market with regard to matters covered by this Regulation; Or. {EN}en Amendment 57 Proposal for a regulation Article 57 – paragraph 1

Text proposed by the Commission

Amendment

1The Board shall be composed of the national supervisory authorities, who shall be represented by the head or equivalent high-level official of that authority, and the European Data Protection Supervisor. Other national authorities may be invited to the meetings, where the issues discussed are of relevance for them.

1The Board shall be composed of the national supervisory authorities, who shall be represented by the head or equivalent high-level official of that authority, and the European Data Protection Supervisor. Other national authorities including those which are members of relevant advisory and expert groups at Union level, may be invited to the meetings, where the issues discussed are of relevance for them.

Or. {EN}en Amendment 58 Proposal for a regulation Article 57 – paragraph 4 Text proposed by the Commission Amendment

4The Board may invite external experts and observers to attend its meetings and may hold exchanges with interested third parties to inform its activities to an appropriate extent. To that end the Commission may facilitate exchanges between the Board and other Union bodies, offices, agencies and advisory groups.

4The Board may invite external experts and observers to attend its meetings and may hold exchanges with interested third parties including organisations from the civil society such as NGOs, consumer associations and industry representatives to inform its activities to an appropriate extent. To that end the Commission may facilitate exchanges between the Board and other relevant Union bodies, offices, agencies and expert advisory groups.

Or. {EN}en Amendment 59 Proposal for a regulation Article 58 – paragraph 1 – point c a (new) Text proposed by the Commission Amendment

(ca) ensure that there is a common and consistent approach among the different advisory and expert groups established at Union level on matters covered by this Regulation or related to artificial intelligence systems. Or. {EN}en Amendment 60 Proposal for a regulation Article 59 – paragraph 1

Text proposed by the Commission

Amendment

1National competent authorities shall be established or designated by each Member State for the purpose of ensuring the application and implementation of this Regulation. National competent authorities shall be organised so as to safeguard the objectivity and impartiality of their activities and tasks.

Or. {EN}en Amendment 61 Proposal for a regulation Article 59 – paragraph 4 Text proposed by the Commission Amendment

4Member States shall ensure that national competent authorities are provided with adequate financial and human resources to fulfil their tasks under this Regulation. In particular, national competent authorities shall have a sufficient number of personnel permanently available whose competences and expertise shall include an in-depth understanding of artificial intelligence technologies, data and data computing, fundamental rights, health and safety risks and knowledge of existing standards and legal requirements.

4Member States shall ensure that national competent authorities are provided with adequate financial and human resources to fulfil their tasks under this Regulation. In particular, national competent authorities shall have a sufficient number of personnel permanently available whose competences and expertise shall include an in-depth understanding of artificial intelligence technologies, data and data computing, fundamental rights, health and safety risks, environmental risks and knowledge of existing standards and legal requirements.

Or. {EN}en Amendment 62 Proposal for a regulation Article 62 – paragraph 1 – subparagraph 1

Text proposed by the Commission Amendment Providers of high-risk AI systems placed on the Union market shall report any serious incident or any malfunctioning of those systems which constitutes a breach of obligations under Union law intended to protect fundamental rights to the market surveillance authorities of the Member States where that incident or breach occurred.

Providers of high-risk AI systems placed on the Union market shall report any serious incident or any malfunctioning of those systems which constitutes a breach of obligations under Union law intended to protect health, safety, fundamental rights and the environment to the market surveillance authorities of the Member States where that incident or breach occurred.

Or. {EN}en Amendment 63 Proposal for a regulation Article 65 – paragraph 1

Text proposed by the Commission

Amendment

1AI systems presenting a risk shall be understood as a product presenting a risk defined in Article 3, point 19 of Regulation (EU) 2019/1020 insofar as risks to the health or safety or to the protection of fundamental rights of persons are concerned.

Or. {EN}en Amendment 64 Proposal for a regulation Article 67 – paragraph 1 Text proposed by the Commission Amendment

1Where, having performed an evaluation under Article 65, the market surveillance authority of a Member State finds that although an AI system is in compliance with this Regulation, it presents a risk to the health or safety of persons, to the compliance with obligations under Union or national law intended to protect fundamental rights or to other aspects of public interest protection, it shall require the relevant operator to take all appropriate measures to ensure that the AI system concerned, when placed on the market or put into service, no longer presents that risk, to withdraw the AI system from the market or to recall it within a reasonable period, commensurate with the nature of the risk, as it may prescribe.

1Where, having performed an evaluation under Article 65, the market surveillance authority of a Member State finds that although an AI system is in compliance with this Regulation, it presents a risk to the health or safety of persons or the environment, to the compliance with obligations under Union or national law intended to protect fundamental rights or to other aspects of public interest protection, it shall require the relevant operator to take all appropriate measures to ensure that the AI system concerned, when placed on the market or put into service, no longer presents that risk, to withdraw the AI system from the market or to recall it within a reasonable period, commensurate with the nature of the risk, as it may prescribe.

Or. {EN}en Amendment 65 Proposal for a regulation Annex III – paragraph 1 – point 5 – point a

Text proposed by the Commission Amendment

(a) AI systems intended to be used by public authorities or on behalf of public authorities to evaluate the eligibility of natural persons for public assistance benefits and services, including healthcare services, as well as to grant, reduce, revoke, or reclaim such benefits and services; Or. {EN}en Amendment 66 Proposal for a regulation Annex IV – paragraph 1 – point 1 – point g

Text proposed by the Commission Amendment

(g) instructions of use for the user and, where applicable installation instructions;

(g) instructions of use for the user and end recipient and, where applicable installation instructions; Or. {EN}en Amendment 67 Proposal for a regulation Annex IV – paragraph 1 – point 3 Text proposed by the Commission Amendment

3Detailed information about the monitoring, functioning and control of the AI system, in particular with regard to: its capabilities and limitations in performance, including the degrees of accuracy for specific persons or groups of persons on which the system is intended to be used and the overall expected level of accuracy in relation to its intended purpose; the foreseeable unintended outcomes and sources of risks to health and safety, fundamental rights and discrimination in view of the intended purpose of the AI system; the human oversight measures needed in accordance with Article 14, including the technical measures put in place to facilitate the interpretation of the outputs of AI systems by the users; specifications on input data, as appropriate;

3Detailed information about the monitoring, functioning and control of the AI system, in particular with regard to: its capabilities and limitations in performance, including the degrees of accuracy for specific persons or groups of persons on which the system is intended to be used and the overall expected level of accuracy in relation to its intended purpose; the foreseeable unintended outcomes and sources of risks to health and safety, fundamental rights, the environment and discrimination in view of the intended purpose of the AI system; the human oversight measures needed in accordance with Article 14, including the technical measures put in place to facilitate the interpretation of the outputs of AI systems by the users; specifications on input data, as appropriate;

Or. {EN}en </RepeatBlock-Amend>