The EU AI Act: 11 Recommendations for Business

Photo by artJazz on iStock

May 21, 2024

Authors

Richard Wingfield

Director, Technology Sectors, BSR

Richard Wingfield

Director, Technology Sectors, BSR

London

Richard works with tech companies—particularly those based in or with operations in Europe, the Middle East, and Africa—to build human rights considerations and practices into their products, services, and policies. He brings a strong understanding of international human rights law and standards and how to translate the corporate responsibility to respect human rights into practice for companies of different sizes and sectors.

Prior to joining BSR, Richard led the legal and policy team at Global Partners Digital, an international human rights organization focused on the impacts of digital technologies on human rights. He is also a trustee of the Kaleidoscope Trust, a UK-based charity that campaigns for the human rights of LGBTIQ+ people in countries where they are discriminated.

Richard holds a LLB in Law and European Law from the University of Nottingham and is a qualified lawyer in England and Wales.
Hannah Darnton

Director, Technology and Human Rights, BSR

Hannah Darnton

Director, Technology and Human Rights, BSR

San Francisco

Hannah works with multinational companies to align business and human rights strategies and facilitate incorporation of sustainable practices into business operations across sectors.

She focuses on the intersection of human rights and new, disruptive technology and leads the Tech Against Trafficking collaborative initiative.

Prior to joining BSR, Hannah worked with the Skoll Foundation, where she co-led the portfolio and investments team’s efforts to identify social entrepreneurs with the potential to drive large-scale social change. Her work led to over US$20 million in grants and investments between 2015 and 2018. Before Skoll, Hannah spent six years working in anti-human trafficking in West Africa, Southeast Asia, and the Bay Area. She is fluent in French.

Hannah holds a Master’s in NGOs and Development from the London School of Economics and a B.A. in Political Science and French from the University of Michigan. She currently serves on the advisory boards of Oxfam’s Women in Small Enterprise initiative and Convening17.
Asako Nagai

Managing Director, Technology Sectors and Asia-Pacific, BSR

Asako Nagai

Managing Director, Technology Sectors and Asia-Pacific, BSR

Tokyo

With more than 20 years of sustainability experience, Asako advises global technology companies as well as BSR members and partners in Japan to better integrate sustainability into core business strategies.

Prior to joining BSR, Asako was Head of the CSR Management Section at Sony Corporation, and she has more than 10 years of experience leading global sustainability strategies. She has experience in sustainability reporting, stakeholder engagement, supply chain management, and responsible mineral sourcing and human rights. She also played a key role in shaping the industry framework for the Electronic Industry Citizenship Coalition.

Asako holds dual master’s degrees in Business Administration and Science from the Ross School of Business and School of Natural Resources and Environment at the University of Michigan.
J.Y. Hoh

Manager, Technology Sectors, BSR

J.Y. Hoh

Manager, Technology Sectors, BSR

Singapore

J.Y. works with technology companies to incorporate effective human rights practices.

Before joining BSR, J.Y. worked as an international human rights lawyer at NGOs. He was previously a Legal Officer at the Centre for Law and Democracy, an international human rights NGO promoting freedom of expression and the right to information globally. Before that, he was a Staff Lawyer at the Canadian Civil Liberties Association, Canada’s national human rights organization, where he was a recipient of the Law Foundation of Ontario’s Public Interest Articling Fellowship. He also acted as an international trade negotiator for the government of Singapore and was part of the negotiating team that closed the EU-Singapore Free Trade Agreement.

J.Y. holds a BA in Law from the University of Oxford, Somerville College and a LLM (International Law) from the School of Law at the University of California, Berkeley.

Key Points

The EU AI Act applies to all AI systems impacting people in the EU across all sectors.
Businesses can prioritize the order in which provisions take effect, starting with prohibited AI, followed by AI systems that interact with people, and finally high-risk AI systems.
The Technology team discusses how a human rights-based approach will help ensure compliance.

This is the second of a two-part series from BSR’s Technology and Human Rights team on the latest developments around the European Union Artificial Intelligence Act and its implications for business.

With the EU’s Artificial Intelligence Act (the AI Act) soon to come into force, leaders around the world are asking themselves what it will mean for their business. The Act is broad in its scope, applying to a wide range of AI systems and companies at all parts of the AI value chain, from development to use. Therefore, it is crucial for any company that develops or uses AI to understand which requirements of the Act will be applicable. This is an exercise that many legal and compliance teams are already undertaking.

Where prioritization is needed, companies might want to look at the order in which provisions will come into force, focusing initially on ensuring there are no prohibited AI practices (since these rules will be in six months), then looking at requirements around general purpose AI models (twelve months), next AI systems that interact with people (two years), and finally high-risk AI systems (three years).

How a human rights-based approach to AI can help

As with other pieces of tech-related regulation stemming from the EU (such as the Digital Services Act), the protection of human rights (termed “fundamental rights” within the EU’s legal system) weaves its way throughout many of the AI Act’s provisions.

While there has been criticism from civil society that the legislation does not go far enough in, for example, prohibiting mass surveillance and facial recognition technologies, for companies developing and deploying AI systems in the future, an understanding of the potential adverse impacts on human rights of those systems is essential.

The most relevant provisions, which are particularly important for “high-risk” AI systems, are:

Risk management systems: Companies will need to develop and implement risk management systems when high-risk AI systems are being developed, and these systems will need to include the identification and analysis of potential risks to human rights.
Data and data governance: When developing high-risk AI systems that involve training models with data, companies will need to ensure that those models are validated and tested in line with appropriate data governance and management practices, which must include an examination of possible biases that are likely to adversely impact human rights.
Transparency and provision of information to deployers: Companies that deploy high-risk AI systems must design them in such a way as to ensure that their operation is “sufficiently transparent to enable deployers to interpret the system’s output and use it appropriately”. This involves providing instructions for use that include, among other things, details of any potential adverse impacts on human rights.
Human oversight: Companies will need to design and develop high-risk AI systems in such a way that they can be effectively overseen by a human. Such human oversight must aim at preventing or minimizing any risks to human rights that may emerge when the system is used as intended, or where it is “misused” in ways that are reasonably foreseeable (i.e. where it might be used in ways which weren’t intended).
Human rights impact assessments: Before certain high-risk AI systems are deployed (primarily when systems are to be used by the public sector), deployers must undertake an assessment of the impact on human rights that the use of the system may produce. This assessment must include details of the individuals and groups likely to be affected, the specific risks of harm, and what measures are to be taken to mitigate those risks.
Reporting serious incidents: Providers of high-risk AI systems placed on the EU market must report any “serious incident” to the market surveillance authorities of the member states where that incident occurred, and “serious incidents” include breaches of obligations under EU law intended to protect human rights.

The definition of “high-risk” means that some sectors are likely to be more impacted by these requirements of the AI Act than others, such as companies involved in the provision of key public services and infrastructure, or those in the financial services sector using AI to assess a person’s creditworthiness. But some “high-risk” AI systems may be relevant for companies’ internal uses of AI, regardless of sector, such as the use of AI-based emotion recognition technologies to determine an employee’s emotions (such as whether they are bored or unhappy), or the use of AI for recruitment purposes.

These different provisions all require companies to develop a strong understanding of the potential impacts that their development or use of AI may have on human rights, as well as to take steps to mitigate those impacts. As such, the AI Act has overlaps with other existing and upcoming regulations which require human rights due diligence. Companies will be well placed to meet these emerging regulatory requirements by taking a harmonized approach, grounded in human rights, and specifically the UN Guiding Principles on Business and Human Rights (UNGPs), across their business.

In addition to BSR’s support for companies to apply to the UNGPs, we’re developing a range of resources specially on the human rights impacts of AI, including guidance on responsible AI in the financial services, consumer, healthcare and retail sectors, as well as our upcoming sector-wide human rights assessment of generative AI.

Our key recommendations on what companies can do now are:

Put together an inventory of the existing and planned AI use cases within your company.
Undertake human rights due diligence of those existing and planned uses of AI to identify high risk areas.
Ensure you have a clearly defined purpose for each use of AI and consider establishing use limitations.
Establish a governance mechanism for the responsible use of AI within the company, such as internal cross-functional oversight committees, and/or an external advisory councils.
Given many of AI’s risks, connected to privacy and data protection, ensure a high level of data protection within the company reviewing existing measures, policies and processes to ensure they meet the additional risks created by AI
Test AI models for bias and externalities, to mitigate potential discriminatory impacts.
Undertake adversarial testing and red teaming (exercises where the AI system is stress tested to discover how the system might be misused or lead to harmful outcomes).
Provide transparency to users, internally and externally, about how the AI models and systems work.
Integrate feedback through a reporting channel where potential misuse and abuse of AI systems can be reported.
Engage in dialogue with other industry players in your sector.
Engage with external stakeholders throughout the AI life cycle to help inform decisions around the development, sale and use of AI. External stakeholders could also be part of an external advisory council (see recommendation 4).