Meaningful XAI Based on User-Centric Design Methodology

The Artificial Intelligence Act proposal lies at the heart of the European approach to artificial intelligence (‘AI’). This approach focuses on excellence and trust, as well as aiming to boost research and industrial capacity while ensuring safety and protecting fundamental rights. The EU aims to become a world-class hub for AI that is human-centric and trustworthy, and such an approach needs to be guided by concrete rules. In April 2021, the EU Commission presented its AI package, which includes a communication on fostering a European approach to AI, a review of the Coordinated Plan on Artificial Intelligence with Member States, the AI Act proposal, and a relevant impact assessment.

Building trust in AI involves implementing a legal framework that addresses fundamental rights, safety risks, liability, and sector-specific regulations. By pursuing this approach, the EU seeks to shape the future of AI and create a resilient digital Europe for the Digital Decade. In light of this, CERRE authors Winston Maxwell and Bruno Dumas have produced this latest report on Explainable AI (‘XAI’) which delves into the concept of ‘explainability’ in AI-based systems, putting forward considerations of both “local” and “global” explanations. Local explanations pertain to specific algorithmic outputs and are context-sensitive, while “global” explanations encompass the functioning of the system as a whole.

The report explores the characteristics of end users, their tasks, and the information required to use AI systems. It emphasises explanations are not a one-size-fits-all solution and can sometimes have unintended consequences, such as creating complacency or exacerbating cognitive biases.

Two use cases, one on an educational recommender system and another on explainable AI for scientific discoveries, are presented to illustrate the application of explainability techniques. ‘Meaningfulness’ is a crucial aspect of explanations and its interpretation within the EUs General Data Protection Regulation framework is examined, noting that it is subjective and influenced by user characteristics, context, and task. The concept of ‘meaningfulness’ is further explored through coherence, purpose, and significance, representing levels that contribute to meaningful interactions.

Overall, the report underscores the importance of tailoring explanations to the user and specific task while considering the challenges and potential impact of measuring meaningfulness. The report concludes by making the following recommendations for policymakers:

• The AI proposal should have specific guidance on local explainability in the risk analyses and management systems of AI providers, which should be adapted as necessary by the users of AI systems;
• The AI proposal should adopt a user-centric design methodology as a way of measuring whether a given explanation is “meaningful” in light of its audience and purpose;
• Using the EU Charter to underpin the approach to explainability in AI-based systems, and provide a foundation for seeking a harmonised methodology;
• European standardisation bodies could develop a framework for meaningful explanations in the context of harmonised standards;
• The European Centre for Algorithmic Transparency can also potentially develop a user-centric design methodology;
• The European Commission should pursue the harmonisation of explainable AI requirements across different EU legislation and case law; and
• The European Commission should also pursue the encouragement of user-centric design methodology to achieve meaningful explanations.