Short answer

Prioritize the development and adoption of a standardized, unambiguous lexicon for Human-AI Interaction within safety-critical design projects to ensure clarity and mitigate risks associated with misinterpretation.

Field: Human Factors
Source: arXiv (Cornell University) (2023)
Method: Systematic Literature Review
Evidence: Strong effect

The lack of standardized terminology for Human-AI Interaction (HAII) in safety-critical industries creates ambiguity, potentially leading to design flaws and compromised system safety. This human factors research insight is drawn from a 2023 study published in arXiv (Cornell University). Using Systematic literature review, researchers explored how this design variable affects real-world outcomes. The key design takeaway: Prioritize the development and adoption of a standardized, unambiguous lexicon for Human-AI Interaction within safety-critical design projects to ensure clarity and mitigate risks associated with misinterpretation.

Study

Human FactorsRecentStrong effect

Inconsistent HAII Terminology Hinders Safety-Critical System Design

The lack of standardized terminology for Human-AI Interaction (HAII) in safety-critical industries creates ambiguity, potentially leading to design flaws and compromised system safety.

arXiv (Cornell University) · 2023

Key Findings

01No single, consistent term is used to describe Human-AI Interaction (HAII) across the literature, with some terms having multiple meanings.
02Seven key factors influence HAII: user characteristics, user perceptions and attitudes, user expectations and experience, AI interface and features, AI output, explainability and interpretability, and usage context.
03User-related subjective metrics (e.g., trust, perceptions) are the most common measures of HAII.
04AI-assisted decision-making is the most frequent primary role of AI-enabled systems.

Application

Design takeaway

How to apply

When designing or evaluating any system involving human-AI collaboration, especially in high-stakes environments, begin by defining and agreeing upon precise terms for all aspects of the interaction. Ensure that user interfaces and AI outputs are designed with the identified influencing factors in mind, and seek to validate HAII through a combination of objective and subjective measures.

Project actions

01When describing your Human-AI interaction, clearly define all terms you use, especially if they could have multiple meanings.
02Consider how user personality, their trust in the AI, and how well they understand the AI's decisions will affect their interaction.
03Think about how you will measure the success of your Human-AI interaction – will it be based on user feelings or objective performance?

Method & Evidence

AimTo identify and analyze the current state of Human-AI Interaction (HAII) research in safety-critical industries, focusing on terminology, AI roles, influencing factors, and measurement methods, to highlight research gaps and propose best practices.

MethodSystematic Literature Review

ProcedureThe researchers conducted a systematic review of existing literature on Human-AI Interaction in safety-critical industries. They analyzed the terminology used, the primary roles of AI systems, factors influencing HAII, and how HAII is measured. They also assessed the capabilities and maturity of AI systems discussed.

ContextSafety-critical industries (e.g., aviation, healthcare, autonomous vehicles)

Variables

IV["Terminology used to describe HAII","Factors influencing HAII (e.g., user characteristics, AI explainability)"]

DV["Quality of Human-AI Interaction","System safety outcomes","User perceptions and trust"]

CV["Industry sector (safety-critical)","Type of AI system"]

Strengths & Limitations

Strengths

+Provides a comprehensive overview of the current state of HAII research in a critical domain.
+Identifies specific factors that influence HAII, offering actionable insights for designers.

Limitations

The literature review might not capture all emerging trends or niche applications of HAII. The identified factors are broad and their relative importance can vary significantly depending on the specific AI system and industry context.

Reliability & validity

The reliability of this review depends on the thoroughness of the literature search and the consistency of the analysis criteria applied by the researchers. Validity is strengthened by the systematic approach to identifying and categorizing findings across multiple studies.

Think critically

Given the identified inconsistencies in HAII terminology, how can designers proactively establish and enforce clear communication protocols within their design teams and with end-users to mitigate potential risks in safety-critical applications?

Design Principles

"Standardize Human-AI Interaction terminology to ensure clarity and reduce potential for error in safety-critical systems."

In safety-critical domains, clear communication and understanding of system behavior are paramount. Ambiguous HAII terminology can lead to misinterpretations of AI capabilities and limitations, impacting user trust, decision-making, and ultimately, operational safety.

What This Means for Your Design

When people work with AI in important jobs (like flying planes or doing surgery), it's really important they understand exactly what the AI is doing and how to talk about it. This study found that people use different words for the same thing, which can be confusing and dangerous. Designers need to make sure everyone uses the same clear language.

How to use in your project

1.Use this research to justify the importance of clearly defining terms related to Human-AI interaction in your design project's introduction or background section.
2.Refer to the identified factors influencing HAII when discussing the user research or design considerations for your project.

Add to My Project

Quick Cite

Paragraph starter

The effectiveness of Human-AI Interaction (HAII) in safety-critical industries is significantly hampered by a lack of standardized terminology, as highlighted by Bach et al. (2023). This ambiguity can lead to critical misunderstandings in system operation and design. Furthermore, factors such as user characteristics, AI explainability, and user perceptions directly influence HAII, necessitating a user-centered approach that accounts for these variables throughout the design and development lifecycle.

Source

arXiv (Cornell University)

Unpacking Human-AI Interaction in Safety-Critical Industries: A Systematic Literature Review

journal · 2023

View source

Questions About This Research

What does the research say about inconsistent haii terminology hinders safety-critical system design?: Prioritize the development and adoption of a standardized, unambiguous lexicon for Human-AI Interaction within safety-critical design projects to ensure clarity and mitigate risks associated with misinterpretation. Evidence: arXiv (Cornell University) (2023).
Why does "Inconsistent HAII Terminology Hinders Safety-Critical System Design" matter for design?: In safety-critical domains, clear communication and understanding of system behavior are paramount. Ambiguous HAII terminology can lead to misinterpretations of AI capabilities and limitations, impacting user trust, decision-making, and ultimately, operational safety.
How can designers apply this research?: Prioritize the development and adoption of a standardized, unambiguous lexicon for Human-AI Interaction within safety-critical design projects to ensure clarity and mitigate risks associated with misinterpretation.
What were the main findings?: No single, consistent term is used to describe Human-AI Interaction (HAII) across the literature, with some terms having multiple meanings.. Seven key factors influence HAII: user characteristics, user perceptions and attitudes, user expectations and experience, AI interface and features, AI output, explainability and interpretability, and usage context.. User-related subjective metrics (e.g., trust, perceptions) are the most common measures of HAII.. AI-assisted decision-making is the most frequent primary role of AI-enabled systems.
What research method was used?: Systematic Literature Review.
How strong is the evidence?: Evidence strength is rated Strong effect, based on a 2023 journal from arXiv (Cornell University).
What should I do differently in my next project?: When designing or evaluating any system involving human-AI collaboration, especially in high-stakes environments, begin by defining and agreeing upon precise terms for all aspects of the interaction. Ensure that user interfaces and AI outputs are designed with the identified influencing factors in mind, and seek to validate HAII through a combination of objective and subjective measures.
What are the limitations?: The review's findings are based on existing literature, which may have its own inherent biases and limitations. The maturity and capabilities of AI systems discussed were also assessed based on the literature, not direct observation.