Algorithmic Bias Amplifies Societal Prejudices in Interactive Systems
Automated systems, influenced by their underlying algorithms and data, can inadvertently perpetuate and even amplify existing societal biases and prejudices.
Science Technology & Human Values · 2023
Key Findings
- 01AI technologies and smart environments can replicate and reinforce existing prejudices.
- 02Automated systems can actively recalibrate and manipulate the contexts within which humans and nonhumans operate.
Application
Design takeaway
Proactively audit and test automated systems for bias, and design with a conscious effort to counteract rather than replicate societal prejudices.
How to apply
When designing AI-powered features, conduct thorough bias audits on training data and algorithm outputs, and consider diverse user groups during testing.
Project actions
- 01When researching user needs, actively look for potential areas where bias might creep into automated features.
- 02Consider how the 'smart' environment you are designing might influence user behavior in unintended ways.
Method & Evidence
Variables
Strengths & Limitations
Strengths
- +Explores the lived experience of users interacting with automation.
- +Connects technological design to broader social and material contexts.
Limitations
The specific context of an art event might not fully represent typical commercial or everyday use of automated systems.
Reliability & validity
Reliability may be limited due to the qualitative nature and specific context of the event. Validity is strengthened by grounding the analysis in theoretical frameworks and exploring lived experiences.
Think critically
To what extent can designers truly 'de-bias' an algorithm when the societal data it learns from is inherently biased?
Design Principles
"Design for equity: Ensure automated systems do not perpetuate or amplify discrimination."
Understanding how algorithms interact with social practices and material realities is crucial for designers. This insight highlights the ethical responsibility to identify and mitigate bias in AI-driven products and services to ensure equitable user experiences.
What This Means for Your Design
Computers and smart devices can sometimes be unfair because the rules they follow (algorithms) are based on information that already has unfairness in it, making the technology repeat those unfair patterns.
How to use in your project
- 1.Use this research to justify the need for user testing that specifically looks for discriminatory outcomes in your design.
Add to My Project
Quick Cite
(2023). Judders, Jabs, and Slips: Automation, Anomalies, Movements. Science Technology & Human Values. https://doi.org/10.1177/01622439231183334 Retrieved from https://designdex.org/study/af6a631b-5b1d-4845-a763-b4ea478cb582/algorithmic-bias-amplifies-societal-prejudices-in-interactive-systems
Paragraph starter
This study highlights how automated systems can inadvertently perpetuate and amplify societal prejudices. When developing interactive technologies, it is crucial to consider the potential for algorithmic bias, as demonstrated by the findings that AI can replicate existing biases and actively manipulate user contexts. Therefore, a thorough examination of data sources and algorithmic logic is essential to ensure equitable and unbiased user experiences.
Source
Science Technology & Human Values
Judders, Jabs, and Slips: Automation, Anomalies, Movements
journal · 2023
View sourceQuestions about this research
- What does the research say about algorithmic bias amplifies societal prejudices in interactive systems?
- Proactively audit and test automated systems for bias, and design with a conscious effort to counteract rather than replicate societal prejudices. Evidence: Science Technology & Human Values (2023).
- Why does "Algorithmic Bias Amplifies Societal Prejudices in Interactive Systems" matter for design?
- Understanding how algorithms interact with social practices and material realities is crucial for designers. This insight highlights the ethical responsibility to identify and mitigate bias in AI-driven products and services to ensure equitable user experiences.
- How can designers apply this research?
- Proactively audit and test automated systems for bias, and design with a conscious effort to counteract rather than replicate societal prejudices.
- What were the main findings?
- AI technologies and smart environments can replicate and reinforce existing prejudices.. Automated systems can actively recalibrate and manipulate the contexts within which humans and nonhumans operate.
- What research method was used?
- Qualitative analysis of user experiences and emergent behaviors within an interactive event..
- How strong is the evidence?
- Evidence strength is rated Strong effect, based on a 2023 journal from Science Technology & Human Values.
- What should I do differently in my next project?
- When designing AI-powered features, conduct thorough bias audits on training data and algorithm outputs, and consider diverse user groups during testing.
- What are the limitations?
- Findings are based on a specific artistic and social context, which may not be generalizable to all automated systems.
- Is there evidence that societal prejudices affects design outcomes?
- The study found that automated systems, through their design and deployment, can mirror and intensify societal prejudices, and also actively shape user environments in ways that can be manipulative. Understanding how algorithms interact with social practices and material realities is crucial for designers. This insight Source: Science Technology & Human Values (2023).
- Where does this automated systems research apply?
- Interactive art installation event exploring automation and identity. It sits within user-centred design research on designdex.org.
Related research topics
societal prejudices design research · evidence on societal prejudices · does societal prejudices improve design outcomes · automated systems studies for designers · societal prejudices and automated systems findings · user-centred design research evidence