Short answer

When designing AI-powered tools for complex tasks, break down the AI's process into visible, modifiable steps to empower users with greater transparency and control.

Field
User-Centred Design
Source
CHI Conference on Human Factors in Computing Systems (2022)
Method
User Study with interactive system
Sample
20 participants
Evidence
Strong effect

Breaking down complex AI tasks into sequential, modular steps (Chaining) allows users to better understand and influence the AI's process and output. This user-centred design research insight is drawn from a 2022 study published in CHI Conference on Human Factors in Computing Systems. Using User study with interactive system with 20 participants, researchers explored how this design variable affects real-world outcomes. The key design takeaway: When designing AI-powered tools for complex tasks, break down the AI's process into visible, modifiable steps to empower users with greater transparency and control.

Study
User-Centred DesignHigh ImpactStrong effect

Chaining LLM prompts increases user transparency and control in complex AI tasks

Breaking down complex AI tasks into sequential, modular steps (Chaining) allows users to better understand and influence the AI's process and output.

CHI Conference on Human Factors in Computing Systems · 2022

01

Key Findings

  • 01Chaining LLM steps significantly improved the quality of task outcomes.
  • 02Chaining significantly enhanced system transparency.
  • 03Chaining significantly enhanced user controllability.
  • 04Chaining significantly enhanced users' sense of collaboration with the AI.
  • 05Users developed new interaction patterns, such as calibrating model expectations via sub-tasks and debugging unexpected outputs by 'unit-testing' sub-components.
02

Application

Design takeaway

When designing AI-powered tools for complex tasks, break down the AI's process into visible, modifiable steps to empower users with greater transparency and control.

How to apply

For a content generation AI, instead of a single 'Generate Article' button, provide steps like 'Outline Generation', 'Section Drafting', 'Tone Adjustment', and 'Fact Checking', allowing users to review and edit after each step.

Project actions

  • 01When designing an AI feature, think about how to show the user what the AI is doing at each stage, not just the final output.
  • 02Allow users to 'tweak' or 'correct' the AI's work at different points in a multi-step process.
  • 03Consider how breaking down a complex AI task into smaller, manageable sub-tasks can improve user experience.
03

Method & Evidence

AimTo investigate whether chaining large language model (LLM) prompts improves task outcome quality, system transparency, controllability, and sense of collaboration in complex human-AI interaction.
MethodUser Study with interactive system
ProcedureParticipants used an interactive system that allowed them to construct and modify 'AI Chains' (sequences of LLM primitive operations). They interacted with these Chains, observed intermediate results, and could debug or compare different strategies. Their experiences were then evaluated.
Sample20 participants
ContextHuman-AI interaction, Large Language Models (LLMs)

Variables

IVAI interaction paradigm (Chained LLM prompts vs. traditional single-prompt interaction)
DVTask outcome quality, system transparency, user controllability, sense of collaboration, new interaction patterns.
CVSpecific LLM used, type of complex tasks, interactive system interface (excluding the chaining mechanism).
04

Strengths & Limitations

Strengths

  • +Addresses a critical problem in human-AI interaction (lack of transparency and control).
  • +Provides concrete design recommendations (Chaining LLM steps).
  • +Identifies novel user interaction patterns with chained AI systems.

Limitations

The study was conducted with a specific type of AI (LLMs) and a limited number of participants, so the findings might not apply universally to all AI types or larger, more diverse user groups.

Reliability & validity

The study's validity is strengthened by measuring multiple aspects (quality, transparency, control, collaboration) and observing user behaviors. Reliability could be improved by larger sample sizes and standardized task completion metrics.

Think critically

How might the complexity of the 'primitive operations' in an AI Chain impact user understanding and control? Is there a point where too much transparency becomes overwhelming?

05

Design Principles

"Modular Transparency: Expose and enable modification of intermediate AI processing steps to enhance user understanding and control."

Users often struggle to trust or effectively utilize AI systems due to their 'black box' nature, leading to frustration and suboptimal outcomes. By making the AI's reasoning process visible and modifiable at each stage, users gain a sense of agency and predictability, fostering better collaboration and higher quality results.

06

What This Means for Your Design

If you make an AI's thinking process visible step-by-step, people will get better results and feel more in control and collaborative with the AI.

How to use in your project

  • 1.Information architecture for AI systems should prioritize clear navigation and access to 'chain' components or 'step-by-step' process views, rather than just input/output screens.
07

Add to My Project

08

Quick Cite

Paragraph starter

Wu et al. (2022) found that 'AI Chains,' which expose intermediate steps of large language model processes, significantly improve user transparency, controllability, and the quality of task outcomes.

09

Source

CHI Conference on Human Factors in Computing Systems

AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Prompts

journal · 2022

View source

Questions About This Research

What does the research say about chaining llm prompts increases user transparency and control in complex ai tasks?
When designing AI-powered tools for complex tasks, break down the AI's process into visible, modifiable steps to empower users with greater transparency and control. Evidence: CHI Conference on Human Factors in Computing Systems (2022).
Why does "Chaining LLM prompts increases user transparency and control in complex AI tasks" matter for design?
Users often struggle to trust or effectively utilize AI systems due to their 'black box' nature, leading to frustration and suboptimal outcomes. By making the AI's reasoning process visible and modifiable at each stage, users gain a sense of agency and predictability, fostering better collaboration and higher quality results.
How can designers apply this research?
When designing AI-powered tools for complex tasks, break down the AI's process into visible, modifiable steps to empower users with greater transparency and control.
What were the main findings?
Chaining LLM steps significantly improved the quality of task outcomes.. Chaining significantly enhanced system transparency.. Chaining significantly enhanced user controllability.. Chaining significantly enhanced users' sense of collaboration with the AI.
What research method was used?
User Study with interactive system with 20 participants.
How strong is the evidence?
Evidence strength is rated Strong effect, based on a 2022 journal from CHI Conference on Human Factors in Computing Systems.
What should I do differently in my next project?
For a content generation AI, instead of a single 'Generate Article' button, provide steps like 'Outline Generation', 'Section Drafting', 'Tone Adjustment', and 'Fact Checking', allowing users to review and edit after each step.
What are the limitations?
The study involved a relatively small sample size (20 participants) and focused on specific types of complex tasks. Generalizability to all AI applications or larger user populations may require further research.