Short answer

Design AI-augmented systems for crowdsourcing with a focus on motivating users through clear incentives and allowing for a gradual increase in AI reliance as users become more comfortable.

Field: Innovation & Design
Source: Journal of Product Innovation Management (2023)
Method: Behavioral experimental study
Sample: 629 participants
Evidence: Mixed findings

Gig workers, often lacking expert knowledge, are more likely to adopt AI for idea evaluation when financial and social incentives are present, and their aversion to AI diminishes over time. This innovation & design research insight is drawn from a 2023 study published in Journal of Product Innovation Management. Using Behavioral experimental study with 629 participants, researchers explored how this design variable affects real-world outcomes. The key design takeaway: Design AI-augmented systems for crowdsourcing with a focus on motivating users through clear incentives and allowing for a gradual increase in AI reliance as users become more comfortable.

Study

Innovation & DesignRecentMixed findings

Gig workers' AI adoption in idea evaluation is influenced by incentives and time, not expertise.

Gig workers, often lacking expert knowledge, are more likely to adopt AI for idea evaluation when financial and social incentives are present, and their aversion to AI diminishes over time.

Journal of Product Innovation Management · 2023

Key Findings

01Gig workers' lay status did not inherently lead to an appreciation of AI in idea evaluation.
02Financial and social incentives, as well as information about AI functionality, showed mixed support for influencing AI adoption.
03Crowdvoters' aversive behavior towards AI faded over time.

Application

Design takeaway

Design AI-augmented systems for crowdsourcing with a focus on motivating users through clear incentives and allowing for a gradual increase in AI reliance as users become more comfortable.

How to apply

When designing a crowdsourcing platform that incorporates AI for tasks like idea screening or evaluation, implement tiered reward systems and provide clear, concise explanations of the AI's capabilities and benefits. Monitor user engagement over time to identify patterns of increasing AI reliance.

Project actions

01Consider how users might react to AI assistance in your design project.
02Think about what would motivate users to adopt a new technology or feature.

Method & Evidence

AimTo investigate the factors influencing the adoption of AI-enabled systems by gig workers in crowdvoting scenarios for idea evaluation.

MethodBehavioral experimental study

ProcedureParticipants (gig workers) were tasked with predicting the success or failure of crowd-generated ideas. In multiple rounds, they could choose to delegate their decision-making to an AI system or make their own evaluations, with incentive-compatible rewards.

Sample629 participants

ContextCrowdvoting for idea evaluation in an open innovation context.

Variables

IV["Presence of financial incentives","Presence of social incentives","Provision of information about AI functionality","Time/number of rounds"]

DV["AI adoption rate (delegation to AI)","Aversive behavior towards AI"]

CV["Participant type (gig workers)","Task (predicting idea success/failure)","Incentive-compatible rewards structure"]

Strengths & Limitations

Strengths

+Large sample size (n=629) increases generalizability.
+Use of incentive-compatible rewards ensures genuine decision-making.

Limitations

The specific type of AI and the task complexity can significantly impact user adoption. Generalizing findings across different domains might be challenging.

Reliability & validity

The behavioral experiment with incentive-compatible rewards likely enhances the ecological validity of the findings. The large sample size contributes to statistical reliability. However, the specific context of crowdvoting might limit generalizability.

Think critically

To what extent can the observed fading of aversive behavior be attributed to genuine trust in the AI versus a pragmatic acceptance driven by incentives?

Design Principles

"Incentivize and habituate AI adoption in collaborative innovation processes."

Understanding how non-expert crowds interact with AI in innovation processes is crucial for designing effective crowdsourcing platforms. This insight highlights that simply providing AI tools is insufficient; strategic integration considering human psychological factors and motivational drivers is key to successful adoption.

What This Means for Your Design

People who do gig work might not trust AI at first to help them judge ideas, but if you give them money or social rewards, they're more likely to try it. Also, the more they use the AI, the less they dislike it over time.

How to use in your project

1.Reference this study when discussing the human-AI interaction in your design project, particularly if your design involves AI assistance for evaluation or decision-making tasks.

Add to My Project

Quick Cite

Paragraph starter

Research indicates that in collaborative innovation contexts, users such as gig workers may exhibit initial aversions to AI-augmented decision-making. However, factors like financial incentives, social recognition, and the provision of clear information about AI functionality can foster adoption. Furthermore, prolonged exposure to AI systems can lead to a reduction in user apprehension over time, suggesting that habituation plays a significant role in AI acceptance within design projects.

Source

Journal of Product Innovation Management

The AI‐augmented crowd: How human crowdvoters adopt AI (or not)

journal · 2023

View source

Questions About This Research

What does the research say about gig workers' ai adoption in idea evaluation is influenced by incentives and time, not expertise?: Design AI-augmented systems for crowdsourcing with a focus on motivating users through clear incentives and allowing for a gradual increase in AI reliance as users become more comfortable. Evidence: Journal of Product Innovation Management (2023).
Why does "Gig workers' AI adoption in idea evaluation is influenced by incentives and time, not expertise." matter for design?: Understanding how non-expert crowds interact with AI in innovation processes is crucial for designing effective crowdsourcing platforms. This insight highlights that simply providing AI tools is insufficient; strategic integration considering human psychological factors and motivational drivers is key to successful adoption.
How can designers apply this research?: Design AI-augmented systems for crowdsourcing with a focus on motivating users through clear incentives and allowing for a gradual increase in AI reliance as users become more comfortable.
What were the main findings?: Gig workers' lay status did not inherently lead to an appreciation of AI in idea evaluation.. Financial and social incentives, as well as information about AI functionality, showed mixed support for influencing AI adoption.. Crowdvoters' aversive behavior towards AI faded over time.
What research method was used?: Behavioral experimental study with 629 participants.
How strong is the evidence?: Evidence strength is rated Mixed findings, based on a 2023 journal from Journal of Product Innovation Management.
What should I do differently in my next project?: When designing a crowdsourcing platform that incorporates AI for tasks like idea screening or evaluation, implement tiered reward systems and provide clear, concise explanations of the AI's capabilities and benefits. Monitor user engagement over time to identify patterns of increasing AI reliance.
What are the limitations?: The study focused on gig workers, and findings may not generalize to expert evaluators. The specific AI system used and the nature of the ideas evaluated could also influence adoption rates.