Short answer
Prioritize real-world testing and pilot studies when evaluating AR interfaces, especially for handheld devices, to ensure findings are applicable beyond controlled laboratory conditions.
- Field
- User-Centred Design
- Source
- Frontiers in Robotics and AI (2018)
- Method
- Systematic Review
- Sample
- 291 papers with 369 individual user studies
- Evidence
- Moderate effect
A systematic review of a decade of augmented reality usability studies reveals a significant trend towards handheld devices and a persistent reliance on laboratory settings, indicating a need for more real-world testing and pilot studies. This user-centred design research insight is drawn from a 2018 study published in Frontiers in Robotics and AI. Using Systematic review with 291 papers with 369 individual user studies, researchers explored how this design variable affects real-world outcomes. The key design takeaway: Prioritize real-world testing and pilot studies when evaluating AR interfaces, especially for handheld devices, to ensure findings are applicable beyond controlled laboratory conditions.
Augmented Reality Usability: A Decade of Research Highlights Key Trends and Gaps
A systematic review of a decade of augmented reality usability studies reveals a significant trend towards handheld devices and a persistent reliance on laboratory settings, indicating a need for more real-world testing and pilot studies.
Frontiers in Robotics and AI · 2018
Key Findings
- 01There is a growing trend towards user studies involving handheld AR devices.
- 02The majority of AR usability studies are conducted in laboratory settings.
- 03A significant portion of studies do not involve pilot testing.
- 04The landscape of AR user research has evolved considerably over the decade.
Application
Design takeaway
Prioritize real-world testing and pilot studies when evaluating AR interfaces, especially for handheld devices, to ensure findings are applicable beyond controlled laboratory conditions.
How to apply
When designing and testing AR applications, actively seek opportunities for field studies and incorporate iterative pilot testing phases to uncover usability issues that may not be apparent in a lab setting.
Project actions
- 01When designing an AR experience, think about how it will be used in different environments, not just a controlled lab.
- 02Always plan for a pilot test to catch problems before a full user study.
Method & Evidence
Variables
Strengths & Limitations
Strengths
- +Comprehensive systematic review covering a decade of research.
- +Classification of studies by application area provides structured insights.
Limitations
The review's timeframe (2005-2014) means it doesn't cover the latest AR technologies and research trends, such as advanced head-mounted displays or AI-driven AR experiences.
Reliability & validity
The systematic review methodology aims for high reliability through clear inclusion criteria and classification. Validity is supported by the large number of studies analyzed, providing a broad overview of the research landscape.
Think critically
Given the trend towards laboratory settings, how might the findings of AR usability studies be biased, and what strategies can be employed to mitigate these biases in future research and design?
Design Principles
"Validate AR interface designs through diverse testing environments, including real-world scenarios, to ensure robust usability and user experience."
Understanding the evolution of AR usability research provides valuable context for current design practices. Identifying common methodologies and prevalent research settings helps designers anticipate potential challenges and opportunities when developing and testing AR interfaces.
What This Means for Your Design
Researchers have looked at many studies about how easy it is to use Augmented Reality (AR) over 10 years. They found that more studies are using phones and tablets for AR, but most of these studies happen in a lab, not in the real world. Also, many studies don't do a small test run first.
How to use in your project
- 1.Reference this review when discussing the evolution of AR usability research and justifying your choice of testing methodology, particularly if you are conducting field studies or pilot tests.
Add to My Project
Quick Cite
Paragraph starter
A systematic review of augmented reality usability studies from 2005 to 2014 by Dey et al. (2018) highlights a significant trend towards handheld AR devices but also a persistent reliance on laboratory settings and a lack of pilot testing. This suggests that while AR research has evolved, there is a need to increase the ecological validity of studies by conducting them in real-world environments and to incorporate pilot testing to refine methodologies and identify potential usability issues before larger-scale evaluations.
Source
Frontiers in Robotics and AI
A Systematic Review of 10 Years of Augmented Reality Usability Studies: 2005 to 2014
journal · 2018
View sourceQuestions About This Research
- What does the research say about augmented reality usability: a decade of research highlights key trends and gaps?
- Prioritize real-world testing and pilot studies when evaluating AR interfaces, especially for handheld devices, to ensure findings are applicable beyond controlled laboratory conditions. Evidence: Frontiers in Robotics and AI (2018).
- Why does "Augmented Reality Usability: A Decade of Research Highlights Key Trends and Gaps" matter for design?
- Understanding the evolution of AR usability research provides valuable context for current design practices. Identifying common methodologies and prevalent research settings helps designers anticipate potential challenges and opportunities when developing and testing AR interfaces.
- How can designers apply this research?
- Prioritize real-world testing and pilot studies when evaluating AR interfaces, especially for handheld devices, to ensure findings are applicable beyond controlled laboratory conditions.
- What were the main findings?
- There is a growing trend towards user studies involving handheld AR devices.. The majority of AR usability studies are conducted in laboratory settings.. A significant portion of studies do not involve pilot testing.. The landscape of AR user research has evolved considerably over the decade.
- What research method was used?
- Systematic Review with 291 papers with 369 individual user studies.
- How strong is the evidence?
- Evidence strength is rated Moderate effect, based on a 2018 journal from Frontiers in Robotics and AI.
- What should I do differently in my next project?
- When designing and testing AR applications, actively seek opportunities for field studies and incorporate iterative pilot testing phases to uncover usability issues that may not be apparent in a lab setting.
- What are the limitations?
- The review is limited to studies published between 2005 and 2014, potentially missing more recent advancements and trends in AR usability research.