Short answer

Incorporate explainable AI features into design tools to provide users with targeted, actionable feedback that guides iterative improvement, rather than just presenting scores.

Field: Innovation & Design
Source: CHI Conference on Human Factors in Computing Systems (2022)
Method: User Study
Evidence: Moderate effect

Providing AI-generated explanations alongside quality and diversity scores significantly improves the iterative ideation process for crowdworkers. This innovation & design research insight is drawn from a 2022 study published in CHI Conference on Human Factors in Computing Systems. Using User study, researchers explored how this design variable affects real-world outcomes. The key design takeaway: Incorporate explainable AI features into design tools to provide users with targeted, actionable feedback that guides iterative improvement, rather than just presenting scores.

Study

Innovation & DesignHigh ImpactModerate effect

AI-driven explanations enhance crowd ideation diversity and quality by 20%

Providing AI-generated explanations alongside quality and diversity scores significantly improves the iterative ideation process for crowdworkers.

CHI Conference on Human Factors in Computing Systems · 2022

Key Findings

01AI-generated explanations significantly improved ideation diversity compared to no feedback or score-only feedback.
02Users found the explanations helpful in focusing their improvement efforts and providing clear directions.
03The system successfully predicted ideation quality and diversity.

Application

Design takeaway

Incorporate explainable AI features into design tools to provide users with targeted, actionable feedback that guides iterative improvement, rather than just presenting scores.

How to apply

When designing a platform for idea generation or problem-solving, implement AI that not only scores submissions but also explains the reasoning behind the score and offers concrete suggestions for enhancement.

Project actions

01Consider how you can provide constructive feedback in your design projects, not just critique.
02Explore how AI or algorithms could potentially assist in evaluating or guiding user-generated content.

Method & Evidence

AimHow can AI-generated explanations be leveraged to improve the quality and diversity of crowd-sourced ideations in iterative creative processes?

MethodUser Study

ProcedureResearchers developed an AI system that predicts ideation quality and diversity, and generates explanations (attribution, contrastive attribution, counterfactual suggestions). This system was tested in formative and controlled user studies where participants used the feedback to iteratively improve their ideas.

ContextCreativity support tools and crowd-based ideation platforms.

Variables

IV["Type of feedback provided (no feedback, score-only feedback, score + AI explanations)."]

DV["Quality of ideations.","Diversity of ideations."]

CV["The ideation task itself.","The platform used for ideation.","Participant demographics (potentially)."]

Strengths & Limitations

Strengths

+Directly addresses the scalability issue of human feedback in crowdsourcing.
+Provides concrete examples of AI explanation types (attribution, contrastive, counterfactual).
+Empirically validated through user studies.

Limitations

The complexity of the AI explanations might be difficult to implement for simpler design projects. The study focused on crowdworkers, so results might differ for expert designers.

Reliability & validity

The study's validity is supported by controlled user studies. Reliability could be further assessed by replicating the study with different participant groups or across various ideation tasks.

Think critically

To what extent can AI truly understand and guide subjective creative processes, and what are the risks of over-reliance on AI-generated feedback?

Design Principles

"Feedback mechanisms in creative support tools should be interpretable and actionable, guiding users towards specific improvements rather than simply evaluating outcomes."

This research demonstrates how explainable AI can move beyond simple scoring to offer actionable feedback, guiding users towards more creative and diverse outcomes. This is crucial for designing effective digital tools that support collaborative innovation and problem-solving at scale.

What This Means for Your Design

Imagine you're asking a group of people to come up with new ideas. This study shows that if a computer can explain *why* an idea is good or bad, and suggest *how* to make it better, people will come up with more varied and better ideas than if they just get a score.

How to use in your project

1.Reference this study when discussing how feedback mechanisms in your design can be improved, especially if your project involves user-generated content or iterative design.

Add to My Project

Quick Cite

Paragraph starter

The study by Wang, Venkatesh, and Lim (2022) highlights the significant impact of interpretable AI feedback on iterative ideation. Their research demonstrated that providing AI-generated explanations, which detail the reasoning behind quality and diversity scores and offer specific suggestions for improvement, led to a notable increase in both the diversity and quality of crowd-sourced ideas compared to systems offering only scores or no feedback. This suggests that for design projects involving user ideation or collaborative content creation, incorporating explainable AI can foster a more productive and innovative environment by guiding users effectively through the iterative refinement process.

Source

CHI Conference on Human Factors in Computing Systems

Interpretable Directed Diversity: Leveraging Model Explanations for Iterative Crowd Ideation

journal · 2022

View source

Questions About This Research

What does the research say about ai-driven explanations enhance crowd ideation diversity and quality by 20%?: Incorporate explainable AI features into design tools to provide users with targeted, actionable feedback that guides iterative improvement, rather than just presenting scores. Evidence: CHI Conference on Human Factors in Computing Systems (2022).
Why does "AI-driven explanations enhance crowd ideation diversity and quality by 20%" matter for design?: This research demonstrates how explainable AI can move beyond simple scoring to offer actionable feedback, guiding users towards more creative and diverse outcomes. This is crucial for designing effective digital tools that support collaborative innovation and problem-solving at scale.
How can designers apply this research?: Incorporate explainable AI features into design tools to provide users with targeted, actionable feedback that guides iterative improvement, rather than just presenting scores.
What were the main findings?: AI-generated explanations significantly improved ideation diversity compared to no feedback or score-only feedback.. Users found the explanations helpful in focusing their improvement efforts and providing clear directions.. The system successfully predicted ideation quality and diversity.
What research method was used?: User Study.
How strong is the evidence?: Evidence strength is rated Moderate effect, based on a 2022 journal from CHI Conference on Human Factors in Computing Systems.
What should I do differently in my next project?: When designing a platform for idea generation or problem-solving, implement AI that not only scores submissions but also explains the reasoning behind the score and offers concrete suggestions for enhancement.
What are the limitations?: The effectiveness of specific explanation types (attribution, contrastive, counterfactual) may vary depending on the user and the complexity of the ideation task. Generalizability to all types of creative tasks needs further investigation.