Short answer

Prioritize designing AI systems where the explanations lead to genuine human understanding and informed action, rather than just providing a technical output.

Field: Modelling
Source: Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery (2019)
Method: Conceptual framework development and case study analysis.
Evidence: Moderate effect

Causability, a measure of how well an AI's explanation can be understood and acted upon by a human, is distinct from explainability, which is a system's ability to provide an explanation. This modelling research insight is drawn from a 2019 study published in Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery. Using Conceptual framework development and case study analysis., researchers explored how this design variable affects real-world outcomes. The key design takeaway: Prioritize designing AI systems where the explanations lead to genuine human understanding and informed action, rather than just providing a technical output.

Study

ModellingHigh ImpactModerate effect

Causability: Measuring the Quality of AI Explanations for Human Understanding

Causability, a measure of how well an AI's explanation can be understood and acted upon by a human, is distinct from explainability, which is a system's ability to provide an explanation.

Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · 2019

Key Findings

01Explainability is a characteristic of the AI system, while causability is a characteristic of the human user's comprehension and ability to act based on the explanation.
02Effective AI deployment requires not just explainable systems but also causable explanations that facilitate human understanding and decision-making.

Application

Design takeaway

Prioritize designing AI systems where the explanations lead to genuine human understanding and informed action, rather than just providing a technical output.

How to apply

When designing interfaces for AI-driven tools, conduct user testing specifically to assess how well users understand and can act upon the AI's explanations.

Project actions

01When evaluating an AI system, consider if its explanations are just jargon or if they genuinely help a user understand and make a decision.
02Think about the user's background knowledge when designing AI explanations.

Method & Evidence

AimTo differentiate between explainability and causability in AI systems and propose a framework for measuring the quality of AI explanations from a human-centric perspective.

MethodConceptual framework development and case study analysis.

ProcedureThe authors define explainability as a system property and causability as a human property. They then illustrate these concepts with a use-case in histopathology, analyzing the interpretation of deep learning models and the quality of human-generated explanations.

ContextArtificial intelligence in medicine, specifically histopathology diagnostics.

Variables

IVType of AI explanation (e.g., technical vs. simplified, visual vs. textual).

DVUser comprehension of the explanation, user confidence in the AI's output, user decision-making accuracy.

CVComplexity of the task, user's domain expertise, interface design elements.

Strengths & Limitations

Strengths

+Introduces a novel and important concept (causability) for evaluating AI explanations.
+Provides a clear conceptual distinction between system and human properties.

Limitations

It can be challenging to objectively measure 'understanding' or 'ability to act' in a user study.

Reliability & validity

Reliability could be improved by using multiple raters for qualitative assessments of understanding. Validity is enhanced by focusing on behavioral outcomes (e.g., decision accuracy) rather than just self-reported understanding.

Think critically

If an AI is highly explainable but users consistently misunderstand its output, is it a failure of the AI or a failure of the user interface design?

Design Principles

"Design AI explanations for human causability, ensuring they are clear, actionable, and foster trust."

In design practice, particularly with complex AI systems, simply generating an explanation (explainability) is insufficient. Designers must ensure these explanations are truly comprehensible and actionable for the intended user, enabling them to trust and effectively utilize the AI's output. This distinction is crucial for developing user-centered AI solutions.

What This Means for Your Design

Imagine an AI tells you why it made a decision. 'Explainability' is the AI's ability to tell you why. 'Causability' is whether *you* actually understand its reason and can use that information to do something useful.

How to use in your project

1.You can discuss how your design aims for causability by ensuring explanations are tailored to your target user's understanding and decision-making process.
2.Use the concepts of explainability and causability to justify design choices for how AI outputs are presented.

Add to My Project

Quick Cite

Paragraph starter

This design project addresses the critical distinction between AI explainability and causability. While explainability refers to the AI's capacity to generate reasons for its outputs, causability focuses on the human user's ability to comprehend and utilize these explanations for informed decision-making. By prioritizing causability, the design ensures that the AI's insights are not merely presented but are effectively integrated into the user's workflow, fostering trust and enhancing practical application.

Source

Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery

Causability and explainability of artificial intelligence in medicine

journal · 2019

View source

Questions About This Research

What does the research say about causability: measuring the quality of ai explanations for human understanding?: Prioritize designing AI systems where the explanations lead to genuine human understanding and informed action, rather than just providing a technical output. Evidence: Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery (2019).
Why does "Causability: Measuring the Quality of AI Explanations for Human Understanding" matter for design?: In design practice, particularly with complex AI systems, simply generating an explanation (explainability) is insufficient. Designers must ensure these explanations are truly comprehensible and actionable for the intended user, enabling them to trust and effectively utilize the AI's output. This distinction is crucial for developing user-centered AI solutions.
How can designers apply this research?: Prioritize designing AI systems where the explanations lead to genuine human understanding and informed action, rather than just providing a technical output.
What were the main findings?: Explainability is a characteristic of the AI system, while causability is a characteristic of the human user's comprehension and ability to act based on the explanation.. Effective AI deployment requires not just explainable systems but also causable explanations that facilitate human understanding and decision-making.
What research method was used?: Conceptual framework development and case study analysis..
How strong is the evidence?: Evidence strength is rated Moderate effect, based on a 2019 journal from Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery.
What should I do differently in my next project?: When designing interfaces for AI-driven tools, conduct user testing specifically to assess how well users understand and can act upon the AI's explanations.
What are the limitations?: The proposed framework is conceptual and requires further empirical validation across diverse AI applications and user groups.