Short answer
Actively listen to and analyze the intuitive explanations marginalized users create for moderation outcomes, as these 'folk theories' are critical indicators of systemic bias and areas for improvement.
- Field
- User-Centred Design
- Source
- ACM Transactions on Social Computing (2023)
- Method
- Qualitative Interview Study
- Sample
- 24 participants
- Evidence
- Moderate effect
Marginalized social media users develop intuitive explanations, or 'folk theories,' for content moderation decisions that often highlight systemic biases and misalignments with community guidelines. This user-centred design research insight is drawn from a 2023 study published in ACM Transactions on Social Computing. Using Qualitative interview study with 24 participants, researchers explored how this design variable affects real-world outcomes. The key design takeaway: Actively listen to and analyze the intuitive explanations marginalized users create for moderation outcomes, as these 'folk theories' are critical indicators of systemic bias and areas for improvement.
Marginalized users' 'folk theories' reveal platform moderation biases
Marginalized social media users develop intuitive explanations, or 'folk theories,' for content moderation decisions that often highlight systemic biases and misalignments with community guidelines.
ACM Transactions on Social Computing · 2023
Key Findings
- 01Marginalized users develop folk theories to explain content moderation outcomes.
- 02These theories are informed by users' perceptions of the platform's 'spirit' and their marginalized identities.
- 03Folk theories often address content removal despite apparent adherence to guidelines and perceived bias against marginalized users within moderation policies.
Application
Design takeaway
Actively listen to and analyze the intuitive explanations marginalized users create for moderation outcomes, as these 'folk theories' are critical indicators of systemic bias and areas for improvement.
How to apply
When designing or evaluating content moderation systems, conduct qualitative research with diverse user groups to uncover their 'folk theories' about how moderation works and identify potential areas of bias or unfairness.
Project actions
- 01When researching user experiences, consider how different groups might develop unique ways of understanding system behaviors.
- 02Look for patterns in user explanations that go beyond official documentation or stated policies.
Method & Evidence
Variables
Strengths & Limitations
Strengths
- +Focuses on underrepresented user groups.
- +Utilizes qualitative methods to capture nuanced user experiences.
Limitations
Qualitative data can be subjective; ensure rigorous analysis to identify themes rather than relying on anecdotal evidence. Sample size may limit generalizability.
Reliability & validity
The qualitative nature of the study provides rich, in-depth data (validity), but the findings' generalizability may be limited (reliability). Triangulation of findings across participants enhances confidence.
Think critically
How might the concept of 'platform spirit' influence the development of folk theories, and what are the ethical implications for designers when these spirits are perceived as discriminatory?
Design Principles
"Design moderation systems that are transparent, equitable, and demonstrably aligned with community values as perceived by all user groups, especially marginalized ones."
Understanding these user-generated theories provides direct insight into how moderation systems are perceived and experienced by those most affected. This can help design teams identify and rectify unfair or opaque moderation practices, fostering a more equitable and trustworthy platform environment.
What This Means for Your Design
People who are often treated unfairly on social media make up their own rules to understand why their posts get taken down, and these rules often show that the platform's real rules aren't fair.
How to use in your project
- 1.Use the concept of 'folk theories' to frame your analysis of user perceptions of a design's functionality or limitations.
- 2.Incorporate qualitative data from users to support claims about how a design's features are interpreted and explained by different user groups.
Add to My Project
Quick Cite
Paragraph starter
This design project explored how users develop 'folk theories' to explain system behaviors, particularly in contexts where they perceive unfairness. For instance, marginalized social media users create intuitive explanations for content moderation decisions that often highlight perceived biases within platform guidelines and enforcement, demonstrating a critical user-centered perspective on system design.
Source
ACM Transactions on Social Computing
Content Moderation Folk Theories and Perceptions of Platform Spirit among Marginalized Social Media Users
journal · 2023
View sourceQuestions About This Research
- What does the research say about marginalized users' 'folk theories' reveal platform moderation biases?
- Actively listen to and analyze the intuitive explanations marginalized users create for moderation outcomes, as these 'folk theories' are critical indicators of systemic bias and areas for improvement. Evidence: ACM Transactions on Social Computing (2023).
- Why does "Marginalized users' 'folk theories' reveal platform moderation biases" matter for design?
- Understanding these user-generated theories provides direct insight into how moderation systems are perceived and experienced by those most affected. This can help design teams identify and rectify unfair or opaque moderation practices, fostering a more equitable and trustworthy platform environment.
- How can designers apply this research?
- Actively listen to and analyze the intuitive explanations marginalized users create for moderation outcomes, as these 'folk theories' are critical indicators of systemic bias and areas for improvement.
- What were the main findings?
- Marginalized users develop folk theories to explain content moderation outcomes.. These theories are informed by users' perceptions of the platform's 'spirit' and their marginalized identities.. Folk theories often address content removal despite apparent adherence to guidelines and perceived bias against marginalized users within moderation policies.
- What research method was used?
- Qualitative Interview Study with 24 participants.
- How strong is the evidence?
- Evidence strength is rated Moderate effect, based on a 2023 journal from ACM Transactions on Social Computing.
- What should I do differently in my next project?
- When designing or evaluating content moderation systems, conduct qualitative research with diverse user groups to uncover their 'folk theories' about how moderation works and identify potential areas of bias or unfairness.
- What are the limitations?
- The study relies on self-reported experiences and perceptions, which may be subject to individual interpretation and recall bias. The findings may not be generalizable to all social media platforms or all types of marginalized users.