The rise of AI in business is revolutionizing the protection of personal data: chatbots, product recommendations, automatic selection of candidates for recruitment... these AI-based tools raise questions of confidentiality and data security, but also processing bias. And in addition to the GDPR, the entry into force of the recent European regulation on artificial intelligence (AI Act) imposes new legal obligations. How can the Data Protection Officer (DPO) address these challenges?
Artificial intelligence (AI) is experiencing unprecedented growth. It offers revolutionary possibilities for organizations, but raises many concerns around data protection.
In fact, the personal data protection officer (DPD or DPO in its English name) finds himself at the heart of this challenge. Responsible for implementing compliance with European Data Protection Regulation (GDPR) within the organization he has designated him, he must take into account the new AI law (AI Act, in English) voted on March 13, 2024 by the European Parliament.
DPO’s first challenge: assessing data risks
AI and more particularly Generative AI (IAG), rely on the massive exploitation of data to train algorithmic models, the famous large language models (LLM). This collection and processing raises concerns regarding privacy, informed consent and data security.
Personal data must also respect the principle of minimization. The CNIL defines it as follows: the data must be adequate, relevant and limited to what is necessary in relation to the purposes for which they are processed.
Additionally, AI systems can extend or amplify biases present in training data, thereby threatening the principles of fairness and non-discrimination.
For example, the training datasets used to feed facial recognition systems are mainly composed of portraits of white people. These systems will therefore recognize people of color less well.
DPOs must be vigilant regarding risks inherent to AI, as :
- Reidentification of individuals based on supposedly anonymized data
- The use of sensitive data (gender, ethnicity, political opinions, etc.) for training models
- Tracking and targeting individuals for behavioral advertising purposes
- Automated decisions having a significant impact on people (recruitment, credit, etc.)
DPO’s second challenge: ensuring GDPR compliance for AI
Beyond risks, DPOs face operational challenges in ensuring AI systems are GDPR compliant. Among the main difficulties are:
– Assessing the necessity and proportionality of data processing for AI systems
– Conducting data protection impact analyzes (AIPD) for high-risk treatments
– Implementation of appropriate technical and organizational measures (data protection by design, etc.)
– Managing the rights of data subjects (access, rectification, opposition, etc.) in the context of AI
– Supervision and control of subcontractors involved in the development or use of AI
The AI law: a new regulatory framework
Faced with the challenges posed by AI, the European Union proposed the AI Act, a regulation aimed at establishing harmonized rules for the development, marketing and use of “trustworthy” AI systems. This legislative proposal introduces several key concepts that will have a significant impact on the role of DPOs.
AI Risk Classification
The AI Act classifies AI systems into four levels of risk: unacceptable, high, limited and minimal. DPOs will need to be able to identify the level of risk of the AI systems used by their organization and put in place corresponding requirements.
Requirements for high-risk AI systems
For AI systems considered “high risk,” e.g., facial recognition systems, credit scoring systems, etc., the AI Act imposes strict requirements throughout the lifecycle of the system , as :
- Carrying out risk assessments and compliance testing
- Implementation of risk management systems and human controls
- Establishing detailed activity records
- The designation of a person responsible for monitoring the conformity of the system
These requirements will involve close collaboration between DPOs, development teams and AI experts.
Rights of data subjects
The AI Act strengthens the rights of individuals by granting them, in particular, the right to be informed when they interact with an AI system and the right to challenge the decisions made by these systems. DPOs must ensure compliance with these rights and put in place appropriate procedures.
Strengthened role of supervisory authorities
The AI Act gives supervisory authorities (such as data protection authorities) new powers to inspect, monitor and sanction non-compliant AI systems. DPOs will have to collaborate closely with these authorities, in particular the CNIL, and ensure rigorous documentation of their AI-related activities.
Future prospects for DPOs
Faced with the rise of AI and the emergence of regulations such as the AI Act, the role of DPOs is evolving and becoming more strategic than ever. They will work closely with technical, legal and operational teams to integrate data protection and regulatory requirements into the design and deployment of AI systems.
DPOs will need to develop in-depth expertise in the field of AI, on technical, legal and ethical aspects. They will play a key role in raising awareness and training teams, as well as promoting a culture of data protection and responsible AI within their organization.
In short, DPOs are at the heart of AI's data protection challenges. Their ability to anticipate risks, collaborate with different stakeholders and adapt to regulatory developments will be crucial to enable their organization to fully reap the benefits of AI while preserving the fundamental rights and freedoms of individuals.
