User-Centred Design Framework for Explainable AI Systems
A structured user-centered design framework can guide the development of explainable AI (XAI) systems, ensuring transparency and user trust.
Artificial Intelligence Review · 2025
Key Findings
- 01A five-stage framework for user-centered XAI system design (UCXAISD) was identified: contextual inquiry, explanation needs identification, XAI method selection, user interface design, and evaluation and refinement.
- 02The framework integrates traditional UCD processes with specialized XAI elements like user-centricity, transparency, and actionability.
Application
Design takeaway
Integrate a dedicated user-centered design process into the development lifecycle of explainable AI systems, ensuring that user needs for transparency and understanding are met at every stage.
How to apply
Use the five identified stages (contextual inquiry, explanation needs identification, XAI method selection, user interface design, evaluation and refinement) as a checklist and guide for designing any AI system where explainability is a key requirement.
Project actions
- 01Clearly define the user group for your AI system and their specific needs regarding explanations.
- 02Consider how the AI's explanations will be presented visually and interactively to the user.
Method & Evidence
Variables
Strengths & Limitations
Strengths
- +Systematic approach ensures comprehensive coverage of relevant literature.
- +Provides a structured, actionable framework for designers.
Limitations
The specific XAI methods and UI elements will vary greatly depending on the AI's complexity and application domain.
Reliability & validity
The reliability of the systematic review depends on the rigor of the search strategy and inclusion/exclusion criteria. Validity is enhanced by synthesizing findings from multiple studies.
Think critically
How might the 'actionability' aspect of XAI explanations differ across various user expertise levels and task complexities?
Design Principles
"Prioritize user needs for transparency and understanding throughout the design and development of complex AI systems."
As AI systems become more prevalent, their complexity can lead to a lack of user understanding and trust. Implementing a user-centered approach ensures that the design of XAI prioritizes user needs, making AI more accessible and reliable.
What This Means for Your Design
When you design AI systems that need to explain themselves, make sure to involve users from the start to understand what they need to know and how they want to see the explanations.
How to use in your project
- 1.Reference this framework when justifying your design process for AI-related projects, especially when discussing user research and interface design for explainability.
Add to My Project
Quick Cite
(2025). Developing user-centered system design guidelines for explainable AI: a systematic literature review. Artificial Intelligence Review. https://doi.org/10.1007/s10462-025-11363-y Retrieved from https://designdex.org/study/13b3d1f9-e476-40ef-9436-893db38d06e1/user-centred-design-framework-for-explainable-ai-systems
Paragraph starter
This research provides a valuable user-centered design framework for developing explainable AI (XAI) systems. The framework, comprising five key stages—contextual inquiry, explanation needs identification, XAI method selection, user interface design, and evaluation and refinement—offers a structured approach to ensure AI transparency and build user trust. By integrating specialized XAI elements with traditional user-centered design principles, this framework serves as a blueprint for creating AI systems that are both technically advanced and highly user-centric.
Source
Artificial Intelligence Review
Developing user-centered system design guidelines for explainable AI: a systematic literature review
journal · 2025
View sourceQuestions about this research
- What does the research say about user-centred design framework for explainable ai systems?
- Integrate a dedicated user-centered design process into the development lifecycle of explainable AI systems, ensuring that user needs for transparency and understanding are met at every stage. Evidence: Artificial Intelligence Review (2025).
- Why does "User-Centred Design Framework for Explainable AI Systems" matter for design?
- As AI systems become more prevalent, their complexity can lead to a lack of user understanding and trust. Implementing a user-centered approach ensures that the design of XAI prioritizes user needs, making AI more accessible and reliable.
- How can designers apply this research?
- Integrate a dedicated user-centered design process into the development lifecycle of explainable AI systems, ensuring that user needs for transparency and understanding are met at every stage.
- What were the main findings?
- A five-stage framework for user-centered XAI system design (UCXAISD) was identified: contextual inquiry, explanation needs identification, XAI method selection, user interface design, and evaluation and refinement.. The framework integrates traditional UCD processes with specialized XAI elements like user-centricity, transparency, and actionability.
- What research method was used?
- Systematic Literature Review with 27 studies.
- How strong is the evidence?
- Evidence strength is rated Strong effect, based on a 2025 journal from Artificial Intelligence Review.
- What should I do differently in my next project?
- Use the five identified stages (contextual inquiry, explanation needs identification, XAI method selection, user interface design, evaluation and refinement) as a checklist and guide for designing any AI system where explainability is a key requirement.
- What are the limitations?
- The framework is based on a review of existing literature and may not encompass all emergent XAI design challenges.
- Is there evidence that design affects design outcomes?
- A five-stage framework has been developed to guide the user-centered design of AI systems that are explainable, focusing on understanding user context, their need for explanations, selecting appropriate AI methods, designing user interfaces, and iterative refinement. As AI systems become more prevalent, their complexit Source: Artificial Intelligence Review (2025).
- Where does this user research apply?
- Artificial Intelligence System Design It sits within user-centred design research on designdex.org.
Related research topics
design design research · evidence on design · does design improve design outcomes · user studies for designers · design and user findings · user-centred design research evidence