Short answer

Prioritize seeking out and, where possible, demanding research that demonstrates transparency in methodology and inclusivity in participant sampling to ensure the validity and applicability of findings in design projects.

Field: Innovation & Design
Source: Child Development (2023)
Method: Meta-scientific analysis of research artifacts.
Sample: 2615 posters
Evidence: Moderate effect

A meta-scientific analysis of research posters reveals a significant underutilization of transparent and inclusive research practices, impacting the rigor and generalizability of findings. This innovation & design research insight is drawn from a 2023 study published in Child Development. Using Meta-scientific analysis of research artifacts. with 2615 posters, researchers explored how this design variable affects real-world outcomes. The key design takeaway: Prioritize seeking out and, where possible, demanding research that demonstrates transparency in methodology and inclusivity in participant sampling to ensure the validity and applicability of findings in design projects.

Study

Innovation & DesignRecentModerate effect

Open Science Practices Lag in Research Posters, Hindering Reproducibility and Inclusivity

A meta-scientific analysis of research posters reveals a significant underutilization of transparent and inclusive research practices, impacting the rigor and generalizability of findings.

Child Development · 2023

Key Findings

01Research presented was heavily skewed toward quantitative studies by American researchers using Western hemisphere samples.
02Sharing of data/materials, preregistrations, and replications were uncommon.
03The findings highlight a gap between the growing appreciation for metascience issues and their practical implementation in research dissemination.

Application

Design takeaway

How to apply

When reviewing literature for a design project, critically evaluate the methodology for transparency (e.g., are methods clearly described? Is data available?) and sample diversity. Consider how these factors might limit the applicability of the findings to your target user group.

Project actions

01When citing research, note if the authors have made their data or methods available.
02Consider the diversity of the sample in the research you use – does it represent your intended users?

Method & Evidence

AimTo assess the prevalence of inclusive and reproducible science practices within research posters presented at a major scientific conference.

MethodMeta-scientific analysis of research artifacts.

ProcedureResearchers analyzed 2615 research posters from the 2021 SRCD biennial meeting, evaluating the extent to which they incorporated transparent research practices (e.g., data sharing, preregistration, replication studies) and inclusivity (e.g., diverse sample representation).

Sample2615 posters

ContextAcademic research dissemination (psychology/child development).

Variables

IVResearch poster characteristics (e.g., inclusion of data sharing, preregistration, sample demographics).

DVPrevalence of transparent and inclusive research practices.

CVType of research (quantitative/qualitative), researcher nationality, sample origin.

Strengths & Limitations

Strengths

+Large sample size of research posters analyzed.
+Focus on critical metascience issues relevant to scientific rigor.

Limitations

The study only looked at posters, not full papers or other forms of research output. The findings are specific to the field of child development research.

Reliability & validity

The study's reliability is supported by the large sample size and systematic analysis. Validity is enhanced by focusing on concrete indicators of open and reproducible practices.

Think critically

How might the lack of open and reproducible practices in academic research directly lead to flawed or biased design solutions?

Design Principles

"Research underpinning design decisions should strive for transparency, reproducibility, and diverse representation to ensure broad applicability and validity."

In design practice, the ability to reproduce results and understand the context of user research is paramount for building reliable and effective solutions. When research is not transparent or inclusive, it limits the scope of application and can lead to biased design decisions.

What This Means for Your Design

Researchers aren't sharing their data or methods openly enough, and most studies are only about people from Western countries. This makes it hard for others to check their work or use it for different groups of people.

How to use in your project

1.Reference this study when discussing the limitations of existing research or justifying your own transparent research methods.

Add to My Project

Quick Cite

Paragraph starter

Research presented in academic settings, such as the SRCD biennial meeting, often lacks transparency and inclusivity. A meta-scientific investigation of 2615 posters revealed that sharing of data/materials, preregistrations, and replications were uncommon, and studies were heavily skewed toward American researchers using Western hemisphere samples. This limits the reproducibility and generalizability of findings, posing a challenge for designers who rely on such research to inform their practice.

Source

Child Development

A metascience investigation of inclusive, open, and reproducible science practices in research posters at the 2021 SRCD biennial meeting

journal · 2023

View source

Questions About This Research

What does the research say about open science practices lag in research posters, hindering reproducibility and inclusivity?: Prioritize seeking out and, where possible, demanding research that demonstrates transparency in methodology and inclusivity in participant sampling to ensure the validity and applicability of findings in design projects. Evidence: Child Development (2023).
Why does "Open Science Practices Lag in Research Posters, Hindering Reproducibility and Inclusivity" matter for design?: In design practice, the ability to reproduce results and understand the context of user research is paramount for building reliable and effective solutions. When research is not transparent or inclusive, it limits the scope of application and can lead to biased design decisions.
How can designers apply this research?: Prioritize seeking out and, where possible, demanding research that demonstrates transparency in methodology and inclusivity in participant sampling to ensure the validity and applicability of findings in design projects.
What were the main findings?: Research presented was heavily skewed toward quantitative studies by American researchers using Western hemisphere samples.. Sharing of data/materials, preregistrations, and replications were uncommon.. The findings highlight a gap between the growing appreciation for metascience issues and their practical implementation in research dissemination.
What research method was used?: Meta-scientific analysis of research artifacts. with 2615 posters.
How strong is the evidence?: Evidence strength is rated Moderate effect, based on a 2023 journal from Child Development.
What should I do differently in my next project?: When reviewing literature for a design project, critically evaluate the methodology for transparency (e.g., are methods clearly described? Is data available?) and sample diversity. Consider how these factors might limit the applicability of the findings to your target user group.
What are the limitations?: The study focused on poster presentations, which may not fully represent all research conducted or disseminated within the field. The findings are specific to the context of the SRCD meeting in 2021.