Short answer

When designing AI systems for social impact, actively consider how your technical abstractions might obscure or misrepresent social realities, and involve social actors in the design process.

Field: Innovation & Design
Source: Academic Publication (2019)
Method: Conceptual analysis and synthesis of existing research from computer science and Science and Technology Studies (STS).
Evidence: Strong effect

Over-reliance on technical abstractions in AI development can obscure real-world social impacts, leading to systems that fail to achieve desired fairness and justice. This innovation & design research insight is drawn from a 2019 study published in Academic Publication. Using Conceptual analysis and synthesis of existing research from computer science and science and technology studies (sts)., researchers explored how this design variable affects real-world outcomes. The key design takeaway: When designing AI systems for social impact, actively consider how your technical abstractions might obscure or misrepresent social realities, and involve social actors in the design process.

Study

Innovation & DesignHigh ImpactStrong effect

Abstraction in AI can lead to unintended societal consequences

Over-reliance on technical abstractions in AI development can obscure real-world social impacts, leading to systems that fail to achieve desired fairness and justice.

Academic Publication · 2019

Key Findings

01Technical abstractions can create a mismatch between the designed system and its real-world social context.
02These abstractions can lead to interventions that are ineffective, inaccurate, or misguided.
03Focusing on process rather than solely on technical solutions, and including social actors in abstraction boundaries, can mitigate these issues.

Application

Design takeaway

When designing AI systems for social impact, actively consider how your technical abstractions might obscure or misrepresent social realities, and involve social actors in the design process.

How to apply

Before finalizing an AI system's design, conduct a 'social impact abstraction review' to identify potential blind spots created by technical simplifications. Engage diverse user groups and domain experts early and continuously in the design process.

Project actions

01When defining the scope of your design project, explicitly consider the social context it will operate within.
02Be critical of how you simplify complex problems; ask yourself what real-world factors might be lost in your abstraction.

Method & Evidence

AimHow do common computer science abstractions in AI design create 'traps' that hinder the achievement of fairness and justice in sociotechnical systems?

MethodConceptual analysis and synthesis of existing research from computer science and Science and Technology Studies (STS).

ProcedureThe authors analyze how core computer science concepts like abstraction and modular design are applied in the fair-ML community. They identify five common 'traps' resulting from these technical approaches and draw upon STS literature to explain their occurrence and propose mitigation strategies.

ContextDevelopment and deployment of AI/machine learning systems in social contexts.

Variables

IVUse of technical abstractions (e.g., modular design, data abstraction) in AI system design.

DVEffectiveness, accuracy, and fairness of the AI system in its social context.

CVSpecific domain of application (e.g., hiring, loan applications), type of AI algorithm used.

Strengths & Limitations

Strengths

+Highlights a critical, often overlooked, aspect of AI development.
+Connects computer science principles with social science insights.

Limitations

It can be challenging to fully anticipate all social impacts, and involving all relevant stakeholders may be impractical for a design project.

Reliability & validity

The conceptual nature of the paper means reliability and validity are assessed through the coherence of the arguments and the strength of the synthesis of existing research, rather than empirical measurement.

Think critically

How can designers balance the need for technical simplification with the imperative to address complex social realities?

Design Principles

"Design sociotechnical systems with context-aware abstractions that integrate social actors and processes."

Designers and engineers often use abstraction as a tool to simplify complex systems. However, when designing AI for social contexts, these abstractions can inadvertently hide critical social factors, leading to unintended negative outcomes and undermining the very goals of fairness and equity the system aims to achieve.

What This Means for Your Design

When you build computer programs that affect people, just focusing on the code can make you miss important real-world problems. It's like building a bridge without thinking about the river's currents or the people who will use it.

How to use in your project

1.Discuss how your chosen design abstractions might limit the system's effectiveness or fairness in real-world scenarios.
2.Justify design decisions by explaining how they account for or mitigate potential negative social impacts.

Add to My Project

Quick Cite

Paragraph starter

The design process for this project acknowledges the potential pitfalls of technical abstraction in sociotechnical systems. By oversimplifying complex social realities, technical solutions can inadvertently lead to unfair or ineffective outcomes. Therefore, this design prioritizes a process-oriented approach, incorporating feedback from diverse user groups to ensure that the final solution is contextually relevant and addresses the intended social goals without creating new problems.

Source

Academic Publication

Fairness and Abstraction in Sociotechnical Systems

journal · 2019

View source

Questions About This Research

What does the research say about abstraction in ai can lead to unintended societal consequences?: When designing AI systems for social impact, actively consider how your technical abstractions might obscure or misrepresent social realities, and involve social actors in the design process. Evidence: Academic Publication (2019).
Why does "Abstraction in AI can lead to unintended societal consequences" matter for design?: Designers and engineers often use abstraction as a tool to simplify complex systems. However, when designing AI for social contexts, these abstractions can inadvertently hide critical social factors, leading to unintended negative outcomes and undermining the very goals of fairness and equity the system aims to achieve.
How can designers apply this research?: When designing AI systems for social impact, actively consider how your technical abstractions might obscure or misrepresent social realities, and involve social actors in the design process.
What were the main findings?: Technical abstractions can create a mismatch between the designed system and its real-world social context.. These abstractions can lead to interventions that are ineffective, inaccurate, or misguided.. Focusing on process rather than solely on technical solutions, and including social actors in abstraction boundaries, can mitigate these issues.
What research method was used?: Conceptual analysis and synthesis of existing research from computer science and Science and Technology Studies (STS)..
How strong is the evidence?: Evidence strength is rated Strong effect, based on a 2019 journal from Academic Publication.
What should I do differently in my next project?: Before finalizing an AI system's design, conduct a 'social impact abstraction review' to identify potential blind spots created by technical simplifications. Engage diverse user groups and domain experts early and continuously in the design process.
What are the limitations?: The paper focuses on conceptual issues rather than empirical testing of specific interventions. The identified 'traps' are broad categories that may manifest differently in various applications.