Short answer

When designing AI tutors or guides, prioritize features that encourage explanation, error diagnosis, and contextual awareness, rather than just direct command delivery.

Field
User-Centred Design
Source
arXiv preprint (2026)
Method
Observational study and automated/interactive evaluation of AI coaching models.
Sample
72 human coaching sessions (22,752 dialogue turns, 28.1 hours of recordings)
Evidence
Strong effect

Current AI coaching models provide more direct instructions but fewer explanations and less context-aware guidance than human coaches, leading to passive learning. This user-centred design research insight is drawn from a 2026 study published in arXiv preprint. Using Observational study and automated/interactive evaluation of ai coaching models. with 72 human coaching sessions (22,752 dialogue turns, 28.1 hours of recordings), researchers explored how this design variable affects real-world outcomes. The key design takeaway: When designing AI tutors or guides, prioritize features that encourage explanation, error diagnosis, and contextual awareness, rather than just direct command delivery.

Study
User-Centred DesignNew This WeekStrong effect

AI Coaching Lacks Explanatory Depth and Visual Grounding Compared to Human Experts

Current AI coaching models provide more direct instructions but fewer explanations and less context-aware guidance than human coaches, leading to passive learning.

arXiv preprint · 2026

01

Key Findings

  • 01AI coaches provide more direct instructions but fewer explanations, error diagnoses, and knowledge-check questions than human coaches.
  • 02AI coaches exhibit poor grounding in the visual context of the software interface, leading to passive user following rather than active learning.
02

Application

Design takeaway

When designing AI tutors or guides, prioritize features that encourage explanation, error diagnosis, and contextual awareness, rather than just direct command delivery.

How to apply

When developing interactive tutorials or AI assistants, incorporate features that allow the AI to explain steps, ask clarifying questions, and respond to user errors with diagnostic feedback.

Project actions

  • 01Consider how your design can provide explanations beyond simple instructions.
  • 02Think about how your interface can help the AI understand the user's current context.
03

Method & Evidence

AimCan state-of-the-art AI models effectively teach humans how to use software, replicating the communication and grounding strategies of human expert coaches?
MethodObservational study and automated/interactive evaluation of AI coaching models.
ProcedureResearchers collected a multimodal dataset of human expert-novice coaching sessions, including dialogue, screen recordings, and input events. They then used this dataset to evaluate the performance of AI models in coaching scenarios, comparing their communication strategies and visual grounding capabilities against human benchmarks.
Sample72 human coaching sessions (22,752 dialogue turns, 28.1 hours of recordings)
ContextSoftware use education and human-computer interaction.

Variables

IVCoaching method (AI vs. Human)
DVUser learning, user engagement, quality of coaching (explanations, diagnoses, grounding)
CVSoftware application, user novice level, task complexity
04

Strengths & Limitations

Strengths

  • +Large, multimodal dataset capturing rich interaction data.
  • +Combines automated and interactive evaluation for a comprehensive assessment.

Limitations

The AI models tested might not represent the absolute cutting edge, and the specific software used might have unique learning curves.

Reliability & validity

The use of a large dataset and multiple evaluation methods enhances reliability. Validity is supported by comparing AI performance to human benchmarks and through interactive user testing.

Think critically

To what extent can AI truly replicate the empathetic and adaptive teaching styles of human experts, especially in complex or novel software domains?

05

Design Principles

"Effective digital coaching requires not only instruction but also explanation, error handling, and contextual grounding to promote genuine user learning."

For designers developing AI-powered educational tools or interfaces, understanding the nuances of effective human coaching is crucial. Simply automating instructions is insufficient; AI needs to replicate the explanatory and diagnostic capabilities of human experts to foster genuine user understanding and engagement.

06

What This Means for Your Design

AI that teaches people how to use computers isn't as good as a human teacher because it gives instructions but doesn't explain things well or understand what's happening on the screen, making people just follow along without really learning.

How to use in your project

  • 1.Use this research to justify the need for explanatory features in your design, arguing that direct instruction alone leads to passive learning.
07

Add to My Project

08

Quick Cite

Paragraph starter

The research by Chen et al. (2026) indicates that current AI coaching models, when compared to human experts, exhibit significant gaps in providing explanatory depth and visual grounding. This leads to a coaching style that prioritizes direct instruction over deeper understanding, resulting in users who passively follow commands rather than actively engaging with the learning material. Therefore, any design aiming to educate users through AI must incorporate features that facilitate comprehensive explanations and context-aware guidance to ensure effective knowledge transfer.

09

Source

arXiv preprint

DigitalCoach: Communication and Grounding Gaps in Human and Agentic Computer Use Coaching

journal · 2026

View source

Questions About This Research

What does the research say about ai coaching lacks explanatory depth and visual grounding compared to human experts?
When designing AI tutors or guides, prioritize features that encourage explanation, error diagnosis, and contextual awareness, rather than just direct command delivery. Evidence: arXiv preprint (2026).
Why does "AI Coaching Lacks Explanatory Depth and Visual Grounding Compared to Human Experts" matter for design?
For designers developing AI-powered educational tools or interfaces, understanding the nuances of effective human coaching is crucial. Simply automating instructions is insufficient; AI needs to replicate the explanatory and diagnostic capabilities of human experts to foster genuine user understanding and engagement.
How can designers apply this research?
When designing AI tutors or guides, prioritize features that encourage explanation, error diagnosis, and contextual awareness, rather than just direct command delivery.
What were the main findings?
AI coaches provide more direct instructions but fewer explanations, error diagnoses, and knowledge-check questions than human coaches.. AI coaches exhibit poor grounding in the visual context of the software interface, leading to passive user following rather than active learning.
What research method was used?
Observational study and automated/interactive evaluation of AI coaching models. with 72 human coaching sessions (22,752 dialogue turns, 28.1 hours of recordings).
How strong is the evidence?
Evidence strength is rated Strong effect, based on a 2026 journal from arXiv preprint.
What should I do differently in my next project?
When developing interactive tutorials or AI assistants, incorporate features that allow the AI to explain steps, ask clarifying questions, and respond to user errors with diagnostic feedback.
What are the limitations?
The study focused on specific software applications and may not generalize to all types of digital learning.