Short answer

Develop AI explanations by first understanding the target non-expert user's cognitive model and then translating expert knowledge into that framework.

Field
User-Centred Design
Source
ACM Transactions on Interactive Intelligent Systems (2023)
Method
Stage-based design method, User Explanation Model, Expert Explanation Model, Target Explanation Model, prototype assessment
Evidence
Strong effect

Designing AI explanations tailored to non-expert users, focusing on their understanding and perception, significantly enhances their trust in AI-driven medical systems. This user-centred design research insight is drawn from a 2023 study published in ACM Transactions on Interactive Intelligent Systems. Using Stage-based design method, user explanation model, expert explanation model, target explanation model, prototype assessment, researchers explored how this design variable affects real-world outcomes. The key design takeaway: Develop AI explanations by first understanding the target non-expert user's cognitive model and then translating expert knowledge into that framework.

Study
User-Centred DesignRecentStrong effect

Meaningful AI Explanations Increase Layperson Trust in Medical Systems

Designing AI explanations tailored to non-expert users, focusing on their understanding and perception, significantly enhances their trust in AI-driven medical systems.

ACM Transactions on Interactive Intelligent Systems · 2023

01

Key Findings

  • 01Laypeople require explanations that align with their understanding and perception, differing from expert explanations.
  • 02A 'Target Explanation Model' integrating layperson understanding and expert practice leads to more meaningful explanations.
  • 03Meaningful explanations positively influence user trust in AI medical systems.
02

Application

Design takeaway

Develop AI explanations by first understanding the target non-expert user's cognitive model and then translating expert knowledge into that framework.

How to apply

When designing any AI-powered tool for a general audience, conduct user research to determine how best to explain its functionality and decisions, prioritizing clarity and relevance over technical completeness.

Project actions

  • 01When designing an AI interface, consider how you will explain its outputs or decisions to a user who doesn't have a background in AI.
  • 02Use user research methods like interviews or surveys to understand what kind of explanations your target users would find helpful and trustworthy.
03

Method & Evidence

AimHow can AI system explanations be designed to be meaningful for non-expert users, and how does this impact their trust perceptions in medical AI systems?
MethodStage-based design method, User Explanation Model, Expert Explanation Model, Target Explanation Model, prototype assessment
ProcedureThe researchers developed models for how laypeople understand AI explanations and how experts practice explanation. They then proposed a Target Explanation Model combining these, leading to design guidelines. A prototype AI explanation system for a breast cancer scenario was built and evaluated for its impact on user trust.
ContextAI in healthcare, medical diagnostic systems, patient-facing AI

Variables

IVType of AI explanation (e.g., expert-focused vs. layperson-focused)
DVUser trust in the AI medical system
CVAI system scenario (e.g., breast cancer diagnosis), user expertise level (non-expert)
04

Strengths & Limitations

Strengths

  • +Addresses a gap in AI explanation research by focusing on non-expert users.
  • +Proposes a practical framework (Target Explanation Model) and design guidelines.
  • +Includes a prototype assessment to validate findings.

Limitations

The complexity of AI systems can make it challenging to create truly 'meaningful' explanations without oversimplifying or misrepresenting the AI's process. User trust can also be influenced by factors beyond the explanation itself.

Reliability & validity

The study's validity is supported by its structured methodology, including model development and prototype assessment. Reliability could be enhanced by replicating the prototype assessment with larger, more diverse samples of non-expert users.

Think critically

To what extent can 'meaningful explanations' for AI be standardized, or are they inherently subjective and context-dependent?

05

Design Principles

"Explanations for AI systems must be contextually relevant and cognitively accessible to the intended user audience to foster trust and understanding."

As AI becomes more integrated into critical fields like healthcare, ensuring user trust is paramount. This research highlights that generic or expert-focused explanations are insufficient for the general public, necessitating a user-centered approach to explanation design to foster adoption and effective use.

06

What This Means for Your Design

If you want people who aren't experts to trust an AI, you need to explain how it works in a way they can easily understand, not in a super technical way.

How to use in your project

  • 1.Reference this study when discussing the importance of user-centered explanation design in your AI-related design projects.
  • 2.Use the findings to justify your design choices for how your AI system communicates with users.
07

Add to My Project

08

Quick Cite

Paragraph starter

This research highlights the critical need for user-centered explanation design in AI systems, particularly for non-expert users. By developing a 'Target Explanation Model' that bridges layperson understanding with expert knowledge, designers can create more meaningful explanations that foster user trust, as demonstrated in a medical AI context. This approach is vital for the successful adoption and effective use of AI technologies across various domains.

09

Source

ACM Transactions on Interactive Intelligent Systems

Meaningful Explanation Effect on User’s Trust in an AI Medical System: Designing Explanations for Non-Expert Users

journal · 2023

View source

Questions About This Research

What does the research say about meaningful ai explanations increase layperson trust in medical systems?
Develop AI explanations by first understanding the target non-expert user's cognitive model and then translating expert knowledge into that framework. Evidence: ACM Transactions on Interactive Intelligent Systems (2023).
Why does "Meaningful AI Explanations Increase Layperson Trust in Medical Systems" matter for design?
As AI becomes more integrated into critical fields like healthcare, ensuring user trust is paramount. This research highlights that generic or expert-focused explanations are insufficient for the general public, necessitating a user-centered approach to explanation design to foster adoption and effective use.
How can designers apply this research?
Develop AI explanations by first understanding the target non-expert user's cognitive model and then translating expert knowledge into that framework.
What were the main findings?
Laypeople require explanations that align with their understanding and perception, differing from expert explanations.. A 'Target Explanation Model' integrating layperson understanding and expert practice leads to more meaningful explanations.. Meaningful explanations positively influence user trust in AI medical systems.
What research method was used?
Stage-based design method, User Explanation Model, Expert Explanation Model, Target Explanation Model, prototype assessment.
How strong is the evidence?
Evidence strength is rated Strong effect, based on a 2023 journal from ACM Transactions on Interactive Intelligent Systems.
What should I do differently in my next project?
When designing any AI-powered tool for a general audience, conduct user research to determine how best to explain its functionality and decisions, prioritizing clarity and relevance over technical completeness.
What are the limitations?
The study focused on a specific medical scenario (breast cancer), and findings may vary for other domains or AI functionalities. The definition of 'meaningful' can be subjective and context-dependent.