Short answer
When designing conversational AI, consider the full spectrum of user experience, from functional performance to emotional connection and ethical treatment.
- Field
- User-Centred Design
- Source
- Machine Learning and Knowledge Extraction (2026)
- Method
- Integrative Review
- Evidence
- Strong effect
High-quality interaction with conversational AI is judged by users across pragmatic, social-affective, and accountability/inclusion layers, not just task completion. This user-centred design research insight is drawn from a 2026 study published in Machine Learning and Knowledge Extraction. Using Integrative review, researchers explored how this design variable affects real-world outcomes. The key design takeaway: When designing conversational AI, consider the full spectrum of user experience, from functional performance to emotional connection and ethical treatment.
Conversational AI Quality: A Multi-Layered User-Centric Framework
High-quality interaction with conversational AI is judged by users across pragmatic, social-affective, and accountability/inclusion layers, not just task completion.
Machine Learning and Knowledge Extraction · 2026
Key Findings
- 01User judgment of AI interaction quality operates on three distinct layers: pragmatic (usability, task effectiveness, competence), social-affective (social presence, warmth, synchronicity), and accountability/inclusion (transparency, accessibility, fairness).
- 02A four-layer interpretive framework (Capacity, Alignment, Levers, Outcomes) can map success and failure in AI dialogue.
- 03Design levers like anthropomorphism, role framing, and onboarding significantly influence interaction quality.
Application
Design takeaway
When designing conversational AI, consider the full spectrum of user experience, from functional performance to emotional connection and ethical treatment.
How to apply
When developing or evaluating conversational AI, use the identified layers (pragmatic, social-affective, accountability/inclusion) as a checklist for user experience design and testing.
Project actions
- 01When designing an AI interface, think about how it will 'feel' to the user, not just how it functions.
- 02Consider adding elements that build trust, like explaining how the AI works or acknowledging its limitations.
Method & Evidence
Variables
Strengths & Limitations
Strengths
- +Comprehensive review of a large number of studies.
- +Development of a novel, multi-layer framework for understanding AI interaction quality.
Limitations
It can be challenging to objectively measure 'warmth' or 'fairness' in user interactions, requiring careful qualitative and quantitative research methods.
Reliability & validity
The integrative review synthesizes findings from multiple empirical studies, aiming for construct validity by identifying consistent themes. Reliability is addressed through the systematic selection and synthesis process. However, the validity of the synthesized findings depends on the quality and methodology of the original studies.
Think critically
How can a designer balance the desire for anthropomorphism to improve social-affective connection with the need for transparency about the AI's capabilities and limitations?
Design Principles
"Interaction quality in AI is a co-created, dialogic construct that encompasses pragmatic, social-affective, and ethical dimensions."
Designers must move beyond basic usability to consider the emotional and ethical dimensions of AI interactions. Acknowledging these layers allows for the creation of more engaging, trustworthy, and effective AI systems that truly meet user needs.
What This Means for Your Design
When you talk to a computer program that talks back (like a chatbot), it's not just about if it answers your question correctly. People also care about how friendly it seems, if it feels like a real conversation, and if it's honest and fair.
How to use in your project
- 1.Use the multi-layer framework to justify design decisions related to user experience, particularly in the evaluation sections of your design project.
Add to My Project
Quick Cite
Paragraph starter
This research highlights that user perception of conversational AI quality is multi-faceted, encompassing not only pragmatic aspects like usability and task effectiveness but also social-affective elements such as perceived warmth and social presence, and crucial accountability and inclusion factors like transparency and fairness. Therefore, design efforts must extend beyond functional performance to cultivate a holistic user experience that addresses these diverse user judgments.
Source
Machine Learning and Knowledge Extraction
Assessing Interaction Quality in Human–AI Dialogue: An Integrative Review and Multi-Layer Framework for Conversational Agents
journal · 2026
View sourceQuestions About This Research
- What does the research say about conversational ai quality: a multi-layered user-centric framework?
- When designing conversational AI, consider the full spectrum of user experience, from functional performance to emotional connection and ethical treatment. Evidence: Machine Learning and Knowledge Extraction (2026).
- Why does "Conversational AI Quality: A Multi-Layered User-Centric Framework" matter for design?
- Designers must move beyond basic usability to consider the emotional and ethical dimensions of AI interactions. Acknowledging these layers allows for the creation of more engaging, trustworthy, and effective AI systems that truly meet user needs.
- How can designers apply this research?
- When designing conversational AI, consider the full spectrum of user experience, from functional performance to emotional connection and ethical treatment.
- What were the main findings?
- User judgment of AI interaction quality operates on three distinct layers: pragmatic (usability, task effectiveness, competence), social-affective (social presence, warmth, synchronicity), and accountability/inclusion (transparency, accessibility, fairness).. A four-layer interpretive framework (Capacity, Alignment, Levers, Outcomes) can map success and failure in AI dialogue.. Design levers like anthropomorphism, role framing, and onboarding significantly influence interaction quality.
- What research method was used?
- Integrative Review.
- How strong is the evidence?
- Evidence strength is rated Strong effect, based on a 2026 journal from Machine Learning and Knowledge Extraction.
- What should I do differently in my next project?
- When developing or evaluating conversational AI, use the identified layers (pragmatic, social-affective, accountability/inclusion) as a checklist for user experience design and testing.
- What are the limitations?
- The review synthesizes existing research, and the applicability of the framework may vary depending on the specific AI system and its deployment context.