Short answer

Prioritize understanding the user's decision-making context and potential cognitive biases over simply implementing AI explanation features when aiming to build trust.

Field
Human Factors
Source
Frontiers in Computer Science (2023)
Method
Experimental study
Sample
380 participants
Evidence
Moderate effect

While human-centered AI explanations can enhance user reliance on AI systems, the specific context and nature of the decision-making task have a more significant impact on overall trust. This human factors research insight is drawn from a 2023 study published in Frontiers in Computer Science. Using Experimental study with 380 participants, researchers explored how this design variable affects real-world outcomes. The key design takeaway: Prioritize understanding the user's decision-making context and potential cognitive biases over simply implementing AI explanation features when aiming to build trust.

Study
Human FactorsRecentModerate effect

Human-Centered AI Explanations Increase Reliance, But Decision Context Dominates Trust

While human-centered AI explanations can enhance user reliance on AI systems, the specific context and nature of the decision-making task have a more significant impact on overall trust.

Frontiers in Computer Science · 2023

01

Key Findings

  • 01Human-centered explanations significantly increased reliance on the AI system.
  • 02The type of decision-making task (e.g., increasing vs. decreasing a price) had a greater influence on trust and reliance than the type of explanation provided.
  • 03Trust and reliance are not always equivalent.
02

Application

Design takeaway

Prioritize understanding the user's decision-making context and potential cognitive biases over simply implementing AI explanation features when aiming to build trust.

How to apply

When designing AI interfaces, conduct user research to understand how the AI's output will be perceived within the user's specific workflow and decision-making goals, and tailor explanations accordingly.

Project actions

  • 01When evaluating AI systems, consider measuring both subjective user feelings and objective user behavior.
  • 02Explore how different framing of a problem can influence user interaction with a proposed design solution.
03

Method & Evidence

AimTo investigate the effects of human-centered post-hoc explanations (feature importance and counterfactuals) on user trust and reliance in AI decision-making, and to compare these effects against the influence of the decision-making context.
MethodExperimental study
ProcedureParticipants were presented with an AI decision-making task under three conditions: a control group, a group receiving feature importance explanations, and a group receiving counterfactual explanations. The nature of the decision-making task (e.g., increasing vs. decreasing a price) was varied. Trust was measured subjectively via questionnaires, and objectively through behavioral reliance.
Sample380 participants
ContextHuman-AI interaction in decision-making tasks

Variables

IV["Type of human-centered explanation (feature importance, counterfactuals, control)","Type of decision-making task (e.g., increasing vs. decreasing price)"]
DV["Subjective trust (questionnaire-based)","Objective trust (behavioral reliance)"]
CV["Participant demographics","AI system interface","Task complexity"]
04

Strengths & Limitations

Strengths

  • +Large sample size providing statistical power.
  • +Comparison of subjective and objective measures of trust.

Limitations

The specific AI task used in the experiment might not perfectly represent all real-world AI applications, and the participant pool may have specific characteristics that limit generalizability.

Reliability & validity

The study's use of both subjective questionnaires and objective behavioral measures enhances the validity of its findings. The large sample size contributes to reliability. However, the specific context of the experiment might limit external validity.

Think critically

If decision context has a greater impact than explanations on trust, how can designers effectively integrate both transparency and contextual awareness into AI systems to foster genuine user confidence?

05

Design Principles

"User trust in AI is a multifaceted construct influenced by both system transparency and the inherent characteristics of the task and its framing."

This research highlights that simply providing explanations for AI decisions is not a guaranteed path to user trust. Designers must consider the broader user experience and the specific goals of the AI within its operational context, as these factors can outweigh the benefits of transparency alone.

06

What This Means for Your Design

Giving users explanations about why an AI made a decision can make them trust and use the AI more, but what the decision is actually about (like making money or losing money) matters even more for how much they trust it.

How to use in your project

  • 1.Use this study to justify the importance of considering user context and decision framing when designing AI-powered solutions.
  • 2.Reference this research when discussing the limitations of relying solely on transparency for building user trust in your design project.
07

Add to My Project

08

Quick Cite

Paragraph starter

This research by Scharowski et al. (2023) demonstrates that while human-centered AI explanations can increase user reliance, the context of the decision-making task itself exerts a more significant influence on trust. This underscores the need for designers to move beyond superficial transparency and deeply consider the user's cognitive landscape and the framing of the problem when developing AI-integrated solutions.

09

Source

Frontiers in Computer Science

Exploring the effects of human-centered AI explanations on trust and reliance

journal · 2023

View source

Questions About This Research

What does the research say about human-centered ai explanations increase reliance, but decision context dominates trust?
Prioritize understanding the user's decision-making context and potential cognitive biases over simply implementing AI explanation features when aiming to build trust. Evidence: Frontiers in Computer Science (2023).
Why does "Human-Centered AI Explanations Increase Reliance, But Decision Context Dominates Trust" matter for design?
This research highlights that simply providing explanations for AI decisions is not a guaranteed path to user trust. Designers must consider the broader user experience and the specific goals of the AI within its operational context, as these factors can outweigh the benefits of transparency alone.
How can designers apply this research?
Prioritize understanding the user's decision-making context and potential cognitive biases over simply implementing AI explanation features when aiming to build trust.
What were the main findings?
Human-centered explanations significantly increased reliance on the AI system.. The type of decision-making task (e.g., increasing vs. decreasing a price) had a greater influence on trust and reliance than the type of explanation provided.. Trust and reliance are not always equivalent.
What research method was used?
Experimental study with 380 participants.
How strong is the evidence?
Evidence strength is rated Moderate effect, based on a 2023 journal from Frontiers in Computer Science.
What should I do differently in my next project?
When designing AI interfaces, conduct user research to understand how the AI's output will be perceived within the user's specific workflow and decision-making goals, and tailor explanations accordingly.
What are the limitations?
The study focused on specific types of post-hoc explanations and a particular decision-making scenario; generalizability to other AI applications or explanation methods may vary.