Short answer
When selecting or developing UX evaluation tools, prioritize those that offer comprehensive coverage, including informational, social, cognitive, and physical dimensions, or consider combining tools to address these gaps.
- Field
- User-Centred Design
- Source
- Interacting with Computers (2024)
- Method
- Systematic Literature Review
- Sample
- 325 publications
- Evidence
- Strong effect
Despite a vast array of UX evaluation tools covering over 300 dimensions, current instruments frequently overlook informational, social, cognitive, and physical aspects of user experience. This user-centred design research insight is drawn from a 2024 study published in Interacting with Computers. Using Systematic literature review with 325 publications, researchers explored how this design variable affects real-world outcomes. The key design takeaway: When selecting or developing UX evaluation tools, prioritize those that offer comprehensive coverage, including informational, social, cognitive, and physical dimensions, or consider combining tools to address these gaps.
UX Evaluation Tools Cover 300+ Dimensions, Yet Neglect Key Areas
Despite a vast array of UX evaluation tools covering over 300 dimensions, current instruments frequently overlook informational, social, cognitive, and physical aspects of user experience.
Interacting with Computers · 2024
Key Findings
- 01104 different UX evaluation tools were identified, classifiable as general or domain-specific.
- 02These tools collectively cover over 300 UX dimensions, categorized into 13 main dimensions.
- 03Informational, social, cognitive, and physical dimensions of UX are less frequently represented in current evaluation tools.
- 04Modularity is an emerging trend in UX evaluation tool development, offering flexibility and efficiency.
Application
Design takeaway
When selecting or developing UX evaluation tools, prioritize those that offer comprehensive coverage, including informational, social, cognitive, and physical dimensions, or consider combining tools to address these gaps.
How to apply
When planning a UX evaluation, map the desired UX dimensions against the capabilities of available tools. If critical dimensions are missing, consider custom questionnaire development or a mixed-methods approach.
Project actions
- 01When choosing a UX evaluation tool for your design project, consider what aspects of user experience are most critical for your specific product.
- 02Don't be afraid to adapt existing tools or create your own questions to cover dimensions that are not well-addressed by standard instruments.
Method & Evidence
Variables
Strengths & Limitations
Strengths
- +Comprehensive systematic review methodology.
- +Analysis of a large number of publications and identified tools.
Limitations
The availability and accessibility of specific UX evaluation tools might vary. The subjective nature of user experience means that even comprehensive tools may yield varied results.
Reliability & validity
The reliability of the findings is supported by the systematic review methodology. Validity is enhanced by the large sample size of publications and the categorization of over 300 UX dimensions, though the interpretation of dimensions can introduce subjectivity.
Think critically
Given the trend towards modularity in UX evaluation, how can designers ensure consistency and comparability across different project evaluations when using a combination of tools or custom modules?
Design Principles
"Comprehensive UX evaluation requires a holistic approach that accounts for all facets of user interaction, not just traditional usability metrics."
For designers and researchers, understanding the full spectrum of user experience is crucial for creating successful products. The proliferation of tools, while offering choice, can lead to confusion and incomplete evaluations if critical, less-covered dimensions are not considered.
What This Means for Your Design
There are many ways to ask users about their experience with a product, but most of them don't ask about everything important, like how the product helps them learn, interact with others, think, or feel physically.
How to use in your project
- 1.Reference this study when discussing the selection of UX evaluation methods, highlighting the need to consider a wide range of UX dimensions and the potential for gaps in standard tools.
Add to My Project
Quick Cite
Paragraph starter
A systematic review of UX evaluation tools (Mortazavi et al., 2024) indicates that while over 300 UX dimensions are covered by existing instruments, crucial aspects such as informational, social, cognitive, and physical dimensions are often underrepresented. This suggests that designers must critically assess the scope of chosen evaluation methods and potentially supplement them to ensure a holistic understanding of user experience.
Source
Interacting with Computers
Exploring the Landscape of UX Subjective Evaluation Tools and UX Dimensions: A Systematic Literature Review (2010–2021)
journal · 2024
View sourceQuestions About This Research
- What does the research say about ux evaluation tools cover 300+ dimensions, yet neglect key areas?
- When selecting or developing UX evaluation tools, prioritize those that offer comprehensive coverage, including informational, social, cognitive, and physical dimensions, or consider combining tools to address these gaps. Evidence: Interacting with Computers (2024).
- Why does "UX Evaluation Tools Cover 300+ Dimensions, Yet Neglect Key Areas" matter for design?
- For designers and researchers, understanding the full spectrum of user experience is crucial for creating successful products. The proliferation of tools, while offering choice, can lead to confusion and incomplete evaluations if critical, less-covered dimensions are not considered.
- How can designers apply this research?
- When selecting or developing UX evaluation tools, prioritize those that offer comprehensive coverage, including informational, social, cognitive, and physical dimensions, or consider combining tools to address these gaps.
- What were the main findings?
- 104 different UX evaluation tools were identified, classifiable as general or domain-specific.. These tools collectively cover over 300 UX dimensions, categorized into 13 main dimensions.. Informational, social, cognitive, and physical dimensions of UX are less frequently represented in current evaluation tools.. Modularity is an emerging trend in UX evaluation tool development, offering flexibility and efficiency.
- What research method was used?
- Systematic Literature Review with 325 publications.
- How strong is the evidence?
- Evidence strength is rated Strong effect, based on a 2024 journal from Interacting with Computers.
- What should I do differently in my next project?
- When planning a UX evaluation, map the desired UX dimensions against the capabilities of available tools. If critical dimensions are missing, consider custom questionnaire development or a mixed-methods approach.
- What are the limitations?
- The review period (2010-2021) may not capture the most recent advancements in UX evaluation tools. The categorization of dimensions is based on the authors' interpretation.