Short answer
When designing interactive systems, anticipate how features might be misused for harassment and build in protective mechanisms from the start.
- Field
- User-Centred Design
- Source
- Proceedings of the ACM on Human-Computer Interaction (2023)
- Method
- Qualitative analysis
- Evidence
- Strong effect
The design of live streaming platforms, specifically their affordances, can inadvertently facilitate coordinated hate raids, highlighting the need for proactive moderation strategies embedded within the design process. This user-centred design research insight is drawn from a 2023 study published in Proceedings of the ACM on Human-Computer Interaction. Using Qualitative analysis, researchers explored how this design variable affects real-world outcomes. The key design takeaway: When designing interactive systems, anticipate how features might be misused for harassment and build in protective mechanisms from the start.
Platform affordances enable coordinated hate raids, necessitating moderation-by-design.
The design of live streaming platforms, specifically their affordances, can inadvertently facilitate coordinated hate raids, highlighting the need for proactive moderation strategies embedded within the design process.
Proceedings of the ACM on Human-Computer Interaction · 2023
Key Findings
- 01Hate raids are human-bot coordinated attacks that exploit live streaming systems to target marginalized streamers.
- 02Marginalized streamers experience significant harm with inadequate platform support.
- 03Current moderation is primarily technical, yet streamers face challenges balancing moderation with participation due to their marginalized status.
Application
Design takeaway
When designing interactive systems, anticipate how features might be misused for harassment and build in protective mechanisms from the start.
How to apply
Before launching new features or platforms, conduct a 'misuse analysis' to identify potential avenues for harassment and design countermeasures.
Project actions
- 01When researching user needs, also consider potential negative uses of your design.
- 02Think about how your design could be used to exclude or harm certain groups.
Method & Evidence
Variables
Strengths & Limitations
Strengths
- +Focuses on a novel and under-researched form of online harassment.
- +Provides a theoretical lens (affordances) to explain the phenomenon.
- +Offers practical design recommendations (moderation-by-design).
Limitations
It can be difficult to predict all possible ways a design might be misused, and some harmful behaviours may only emerge after a product is widely adopted.
Reliability & validity
The qualitative nature of the study relies on interpretation of user discussions, which can be subjective. Validity is strengthened by focusing on a specific, observable phenomenon (hate raids) and using established theoretical concepts (affordances).
Think critically
To what extent can designers be held responsible for the misuse of their creations, and where does user responsibility begin?
Design Principles
"Design for safety by considering potential misuse and integrating proactive moderation."
Understanding how system features can be exploited for harassment is crucial for creating safer online environments. Designers must consider the potential for misuse of platform affordances and integrate moderation considerations from the outset of product development.
What This Means for Your Design
The way a website or app is built can accidentally help people be mean to others, so designers need to think about safety when they create new features.
How to use in your project
- 1.Use this research to justify the inclusion of specific safety features or moderation strategies in your design proposal.
- 2.Reference this study when discussing the ethical considerations of your design project.
Add to My Project
Quick Cite
Paragraph starter
This research highlights the critical role of platform affordances in enabling coordinated online harassment, such as hate raids on live streaming platforms. The study found that system designs can inadvertently facilitate attacks, particularly against marginalized users, underscoring the necessity of a 'moderation-by-design' approach. This principle suggests that safety and moderation should be integral to the design process from its inception, rather than being an add-on, to proactively mitigate potential abuse and ensure a more inclusive user experience.
Source
Proceedings of the ACM on Human-Computer Interaction
Hate Raids on Twitch: Understanding Real-Time Human-Bot Coordinated Attacks in Live Streaming Communities
journal · 2023
View sourceQuestions About This Research
- What does the research say about platform affordances enable coordinated hate raids, necessitating moderation-by-design?
- When designing interactive systems, anticipate how features might be misused for harassment and build in protective mechanisms from the start. Evidence: Proceedings of the ACM on Human-Computer Interaction (2023).
- Why does "Platform affordances enable coordinated hate raids, necessitating moderation-by-design." matter for design?
- Understanding how system features can be exploited for harassment is crucial for creating safer online environments. Designers must consider the potential for misuse of platform affordances and integrate moderation considerations from the outset of product development.
- How can designers apply this research?
- When designing interactive systems, anticipate how features might be misused for harassment and build in protective mechanisms from the start.
- What were the main findings?
- Hate raids are human-bot coordinated attacks that exploit live streaming systems to target marginalized streamers.. Marginalized streamers experience significant harm with inadequate platform support.. Current moderation is primarily technical, yet streamers face challenges balancing moderation with participation due to their marginalized status.
- What research method was used?
- Qualitative analysis.
- How strong is the evidence?
- Evidence strength is rated Strong effect, based on a 2023 journal from Proceedings of the ACM on Human-Computer Interaction.
- What should I do differently in my next project?
- Before launching new features or platforms, conduct a 'misuse analysis' to identify potential avenues for harassment and design countermeasures.
- What are the limitations?
- The study is based on discussions within a specific online community (r/Twitch) and may not represent all live streaming platforms or user experiences.