Short answer
When designing or evaluating visual media, prioritize testing with real users and real-world conditions, as these provide the most accurate measure of quality and user satisfaction.
- Field
- User-Centred Design
- Source
- Texas ScholarWorks (Texas Digital Library) (2017)
- Method
- Algorithmic development and subjective user studies
- Sample
- Large-scale (specific numbers not detailed in abstract)
- Evidence
- Strong effect
The perceived quality of images and videos, especially those with authentic, complex distortions, is best assessed by algorithms trained on human subjective evaluations. This user-centred design research insight is drawn from a 2017 study published in Texas ScholarWorks (Texas Digital Library). Using Algorithmic development and subjective user studies with Large-scale (specific numbers not detailed in abstract), researchers explored how this design variable affects real-world outcomes. The key design takeaway: When designing or evaluating visual media, prioritize testing with real users and real-world conditions, as these provide the most accurate measure of quality and user satisfaction.
Human Perception is the Ultimate Metric for Visual Media Quality
The perceived quality of images and videos, especially those with authentic, complex distortions, is best assessed by algorithms trained on human subjective evaluations.
Texas ScholarWorks (Texas Digital Library) · 2017
Key Findings
- 01Authentic, complex distortions in visual media significantly impact perceived quality.
- 02Algorithms trained on human subjective assessments of real-world distortions correlate highly with human perception.
- 03Existing algorithms often fail with diverse, authentic distortions, focusing instead on synthetic ones.
Application
Design takeaway
When designing or evaluating visual media, prioritize testing with real users and real-world conditions, as these provide the most accurate measure of quality and user satisfaction.
How to apply
When developing a new visual feature or optimizing image/video display, conduct user studies with a diverse range of users and real-world viewing conditions to gauge perceptual quality.
Project actions
- 01When evaluating your design's visual output, don't just rely on technical specs; get feedback from potential users.
- 02Consider how different types of users might perceive the quality of your visual elements.
Method & Evidence
Variables
Strengths & Limitations
Strengths
- +Focus on authentic, real-world distortions rather than synthetic ones.
- +Inclusion of large-scale subjective studies using crowdsourcing.
Limitations
It can be challenging to capture all possible 'real-world' distortions, and user perception can vary widely based on individual experience and cultural background.
Reliability & validity
Reliability would be assessed by the consistency of human ratings and the correlation between algorithmic predictions and human ratings. Validity is addressed by using real-world distortions and human perception as the ground truth.
Think critically
To what extent can an algorithm truly capture the nuances of human aesthetic judgment, and where does human judgment remain indispensable in design evaluation?
Design Principles
"Perceptual quality is paramount and should be the primary driver for evaluating and improving visual media."
In design practice, especially for digital products and content creation, understanding how users perceive visual quality is paramount. This research highlights that relying solely on technical metrics or synthetic distortions is insufficient for real-world applications, emphasizing the need for user-centric evaluation in developing quality assessment tools.
What This Means for Your Design
To know if a picture or video looks good to people, you need to ask people what they think, especially if the picture has real-world problems, not just made-up ones.
How to use in your project
- 1.Reference this research when justifying the importance of user testing for visual aspects of your design project.
- 2.Use the findings to explain why subjective user feedback is a critical metric for assessing the success of visual design elements.
Add to My Project
Quick Cite
Paragraph starter
This research underscores the critical role of human perception in assessing visual media quality. By developing algorithms that are grounded in subjective user evaluations of authentic, real-world distortions, it's possible to create more accurate predictors of user experience. This highlights the necessity for design projects involving visual elements to prioritize user-centric testing and feedback to ensure perceived quality aligns with design intent.
Source
Texas ScholarWorks (Texas Digital Library)
Perceptual quality assessment of real-world images and videos
journal · 2017
View sourceQuestions About This Research
- What does the research say about human perception is the ultimate metric for visual media quality?
- When designing or evaluating visual media, prioritize testing with real users and real-world conditions, as these provide the most accurate measure of quality and user satisfaction. Evidence: Texas ScholarWorks (Texas Digital Library) (2017).
- Why does "Human Perception is the Ultimate Metric for Visual Media Quality" matter for design?
- In design practice, especially for digital products and content creation, understanding how users perceive visual quality is paramount. This research highlights that relying solely on technical metrics or synthetic distortions is insufficient for real-world applications, emphasizing the need for user-centric evaluation in developing quality assessment tools.
- How can designers apply this research?
- When designing or evaluating visual media, prioritize testing with real users and real-world conditions, as these provide the most accurate measure of quality and user satisfaction.
- What were the main findings?
- Authentic, complex distortions in visual media significantly impact perceived quality.. Algorithms trained on human subjective assessments of real-world distortions correlate highly with human perception.. Existing algorithms often fail with diverse, authentic distortions, focusing instead on synthetic ones.
- What research method was used?
- Algorithmic development and subjective user studies with Large-scale (specific numbers not detailed in abstract).
- How strong is the evidence?
- Evidence strength is rated Strong effect, based on a 2017 journal from Texas ScholarWorks (Texas Digital Library).
- What should I do differently in my next project?
- When developing a new visual feature or optimizing image/video display, conduct user studies with a diverse range of users and real-world viewing conditions to gauge perceptual quality.
- What are the limitations?
- The abstract does not detail specific limitations of the developed algorithms or the subjective study, but potential limitations could include the representativeness of the 'real-world' distortions and the inherent subjectivity of human perception.