Short answer
When conducting user research, clearly articulate and defend your sample size based on the specific goals of your study and the expected richness of data required to inform design, rather than relying on vague notions of 'saturation.'
- Field
- User-Centred Design
- Source
- BMC Medical Research Methodology (2018)
- Method
- Systematic analysis of qualitative research papers.
- Sample
- 234 studies
- Evidence
- Strong effect
This paper highlights the importance of transparently justifying sample size sufficiency in qualitative research, which is crucial for robust user-centred design studies. This user-centred design research insight is drawn from a 2018 study published in BMC Medical Research Methodology. Using Systematic analysis of qualitative research papers. with 234 studies, researchers explored how this design variable affects real-world outcomes. The key design takeaway: When conducting user research, clearly articulate and defend your sample size based on the specific goals of your study and the expected richness of data required to inform design, rather than relying on vague notions of 'saturation.'
Transparent reporting of qualitative sample size increases research credibility in user-centred design
This paper highlights the importance of transparently justifying sample size sufficiency in qualitative research, which is crucial for robust user-centred design studies.
BMC Medical Research Methodology · 2018
Key Findings
- 01The majority of studies (53.5%) did not explicitly justify their sample size.
- 02Justifications, when provided, often referred to 'saturation' (49.8%) or methodological guidelines (10.7%).
- 03Transparency in reporting sample size evaluation was often lacking, hindering assessment of data adequacy.
Application
Design takeaway
When conducting user research, clearly articulate and defend your sample size based on the specific goals of your study and the expected richness of data required to inform design, rather than relying on vague notions of 'saturation.'
How to apply
When planning user interviews for a design project, define what 'saturation' or 'data adequacy' means for your specific research questions (e.g., 'we will stop interviewing when no new themes emerge regarding user pain points with current solutions, and we have heard consistent perspectives from at least 10 users').
Project actions
- 01Before starting user interviews, define what 'enough' data looks like for your project.
- 02Document your reasoning for your sample size in your design journal or report.
Method & Evidence
Variables
Strengths & Limitations
Strengths
- +Systematic review provides a broad overview of current practices.
- +Highlights a critical methodological gap in qualitative research.
Limitations
It's hard to predict exact saturation beforehand. This paper doesn't give a magic number, but rather a process for justification.
Reliability & validity
The reliability of qualitative findings is enhanced when the sample size justification ensures that a consistent and comprehensive understanding of user perspectives has been achieved. Validity is improved by demonstrating that the chosen sample adequately represents the target user group and that the data collected is rich enough to answer the research questions.
Think critically
How might the 'data adequacy' for a product aimed at a very niche user group differ from one aimed at a broad market, and how would this influence your sample size justification?
Design Principles
"Rigorous qualitative research requires transparent and context-specific justification of sample size to ensure data adequacy for design insights."
In design engineering, user-centred design relies heavily on qualitative data from user research (e.g., interviews, observations). Understanding how to justify and report sample sizes ensures the validity and reliability of insights gained, directly impacting design decisions and product success.
What This Means for Your Design
When you do interviews for your design project, don't just pick a random number of people. You need to explain why you chose that many people, showing that you'll get enough information to make good design decisions.
How to use in your project
- 1.In your 'User Research' section, after describing your interview method, state your sample size and provide a clear justification. For example: 'A sample of 8 users was chosen for semi-structured interviews. This number was deemed sufficient to achieve thematic saturation regarding user needs for [product type], as similar studies suggest 6-12 participants are often adequate for identifying core themes in qualitative inquiry (Vasileiou et al., 2018). We aimed for data adequacy, ensuring diverse perspectives were captured until no new significant insights emerged.'
Add to My Project
Quick Cite
Paragraph starter
Vasileiou et al. (2018) emphasize the critical importance of transparently justifying sample size sufficiency in qualitative research, arguing that vague references to 'saturation' are insufficient. For effective user-centred design, designers must articulate how their chosen sample size will yield 'data adequacy' – ensuring enough rich and diverse data to inform design decisions. This approach strengthens the credibility and validity of user research findings, directly impacting the robustness of the design process.
Source
BMC Medical Research Methodology
Characterising and justifying sample size sufficiency in interview-based studies: systematic analysis of qualitative health research over a 15-year period
journal · 2018
View sourceQuestions About This Research
- What does the research say about transparent reporting of qualitative sample size increases research credibility in user-centred design?
- When conducting user research, clearly articulate and defend your sample size based on the specific goals of your study and the expected richness of data required to inform design, rather than relying on vague notions of 'saturation.' Evidence: BMC Medical Research Methodology (2018).
- Why does "Transparent reporting of qualitative sample size increases research credibility in user-centred design" matter for design?
- In IB Design Technology, user-centred design relies heavily on qualitative data from user research (e.g., interviews, observations). Understanding how to justify and report sample sizes ensures the validity and reliability of insights gained, directly impacting design decisions and product success.
- How can designers apply this research?
- When conducting user research, clearly articulate and defend your sample size based on the specific goals of your study and the expected richness of data required to inform design, rather than relying on vague notions of 'saturation.'
- What were the main findings?
- The majority of studies (53.5%) did not explicitly justify their sample size.. Justifications, when provided, often referred to 'saturation' (49.8%) or methodological guidelines (10.7%).. Transparency in reporting sample size evaluation was often lacking, hindering assessment of data adequacy.
- What research method was used?
- Systematic analysis of qualitative research papers. with 234 studies.
- How strong is the evidence?
- Evidence strength is rated Strong effect, based on a 2018 journal from BMC Medical Research Methodology.
- What should I do differently in my next project?
- When planning user interviews for a design project, define what 'saturation' or 'data adequacy' means for your specific research questions (e.g., 'we will stop interviewing when no new themes emerge regarding user pain points with current solutions, and we have heard consistent perspectives from at least 10 users').
- What are the limitations?
- The study focused on health research, so direct applicability to all design contexts might vary, though the principles of qualitative research rigor are universal. It also didn't prescribe an 'ideal' sample size, but rather emphasized justification.