What the GDPR Shows Us About the Future of AI Regulation

As artificial intelligence grows in power and sophistication, AI-driven applications will continue to gain prominence in many areas of our lives. These systems provide valuable services and offer many benefits, but they also come with risk — not least because many analyze detailed information about individuals. 

This processing of personal information brings many AI systems directly under the scope of the General Data Protection Regulation (GDPR), the influential European Union legislation that shaped global data privacy regulation. The GDPR uses a risk-based approach to regulate the use of personal data in general, rather than specifically within AI systems, but it includes principles that must be complied by anyone developing or using AI systems that process personal data. 

Privacy and ethical concerns have held a prominent place in discourse around regulation due to the central role personal data plays in many AI applications (whether or not that information can be directly linked to a specific person). With new, AI-specific EU regulation on the horizon, this is a timely moment to highlight the role GDPR has and will continue to play in ensuring AI processes and analyzes personal data in an ethical, responsible, and trustworthy way.

4 GDPR principles that impact AI regulations

The GDPR encapsulates the principles of data minimization, fairness and transparency, explainability, integrity, purpose limitation, and accountability to  minimize the risks of potential harm to individuals caused by the use or misuse of personal data. 

A few of these principles are especially relevant to concerns around privacy, ethics, and AI; they provide an invaluable framework to companies who are looking to develop or use AI systems–whether or not they operate in the European Union. 

Accountability

To be accountable, companies must take a proactive approach to privacy compliance and assume full responsibility for their systems’ impacts, intended or otherwise. Measures to achieve compliance and mitigate risk must be established and documented, including Data Protection Impact Assessments (DPIAs). 

Algorithmic decisions always have the potential for negative impacts, whether through poor design, a lack of oversight, or being trained on biased data. Under the GDPR’s risk-based approach, AI-generated decisions with significant material influence over individuals, such as within the legal system, are subject to even more stringent accountability requirements.

Companies must do their best to prevent these negative outcomes by implementing a comprehensive governance structure that includes strategies and procedures to evaluate and mitigate risk throughout the design and development process. 

Fairness

For AI systems to handle personal data fairly, they must not generate outcomes that could negatively impact those whose data was processed, or analyze it in an unjust way. While thankfully, few systems are created with explicitly detrimental goals, unfair data use can and does occur unintentionally. 

The principle of fairness also states that companies must not process information in an unexpected, undisclosed, or ill-intentioned way. However, a more common concern here is around biased data. One particularly well-known example would be Amazon’s attempt at a hiring algorithm that, due to being trained on data from the company’s already male-dominated workforce, was shown to rank male applicants more highly. 

Unfair data use was not Amazon’s intention, but occurred anyway as the AI system replicated the inequalities in its training data. Since such bias is everywhere, organizations that use AI to analyze personal information must evaluate its likely impact on individuals, and continuously reassess their findings as systems evolve.

Data minimization and security 

The phrase ‘data minimization’ might sound alarming or counterintuitive in reference to AI, since typically, AI systems must process large amounts of data to deliver optimal results. But in fact, there’s no conflict — rather than limiting the overall volume of data processed, data minimization calls for a careful and judicious approach to deciding what datasets should include.

Under GDPR regulations, AI systems must not process any more personal data than is necessary to reach their specific goals — but this includes the copious amounts of data required for training, validation, and testing. Security measures must also be taken to avoid the compromising of personal data; two security risks common to AI systems include the loss or misuse of personal data included in training sets, and personal data becoming vulnerable to theft or fraud by software vulnerabilities with the AI system processing it.

Transparency 

Transparent use of data requires individuals’ knowledge and consent that an AI system will handle their personal information, and this consent must be well-informed. Not only should individuals be aware that an AI system will process their information, but they must also be provided with meaningful information about the purposes of the processing, and the AI’s logic involved in doing so, in a clear, concise and easily comprehensible format.

AI systems must also be explainable — that is, decisions produced must follow a clear and consistent logic so users can understand how an algorithm has come to a decision concerning them. Then, individuals must have the right to contest these decisions, through external human intervention if necessary.

The future of AI regulation 

The GDPR is an especially relevant topic as of late, with the EU’s recent proposal of the “Artificial Intelligence Act.” This proposed regulation would prohibit the development and use of AI systems that are considered a threat to the safety, livelihoods and rights of people. Similarly to the GDPR, these guidelines follow a model of encouraging companies to work proactively to address  privacy and ethical concerns during the development process. 

The GDPR set a new standard for data protection laws, including how personal data is processed, stored and managed with AI systems. Now, the EU’s newly proposed AI-specific regulation is likely to have a similar impact on positively shaping the future of this field.