Short answer
When evaluating design concepts, be aware that subjective biases stemming from the evaluator's background and expertise will influence the outcome; actively manage these biases through structured processes and diverse perspectives.
- Field
- Innovation & Design
- Source
- theses.fr (ABES) (2014)
- Method
- Multi-study qualitative and quantitative analysis
- Evidence
- Strong effect
The subjective nature of design creativity means that the evaluator's professional background and experience level profoundly influence their judgment of a design's originality and value. This innovation & design research insight is drawn from a 2014 study published in theses.fr (ABES). Using Multi-study qualitative and quantitative analysis, researchers explored how this design variable affects real-world outcomes. The key design takeaway: When evaluating design concepts, be aware that subjective biases stemming from the evaluator's background and expertise will influence the outcome; actively manage these biases through structured processes and diverse perspectives.
Expertise and background significantly alter design creativity assessments
The subjective nature of design creativity means that the evaluator's professional background and experience level profoundly influence their judgment of a design's originality and value.
theses.fr (ABES) · 2014
Key Findings
- 01Judges' professional backgrounds (designer, art director, design educator, target audience) lead to variations in evaluative criteria.
- 02Levels of design expertise (expert, intermediate, non-expert) impact the criteria considered and the agreement between judges.
Application
Design takeaway
When evaluating design concepts, be aware that subjective biases stemming from the evaluator's background and expertise will influence the outcome; actively manage these biases through structured processes and diverse perspectives.
How to apply
When conducting peer reviews or seeking feedback on design work, explicitly define the criteria for 'creativity' and ensure the review panel includes individuals with different professional roles and varying levels of design experience.
Project actions
- 01When seeking feedback on your design project, ask people with different backgrounds (e.g., a designer, a potential user, a business person) to evaluate it.
- 02Clearly state the criteria you want feedback on, especially regarding creativity, to guide your evaluators.
Method & Evidence
Variables
Strengths & Limitations
Strengths
- +Employs multiple research methods (qualitative and quantitative) for a comprehensive understanding.
- +Investigates two key factors influencing judgment: background and experience.
Limitations
It can be difficult to find and recruit participants with genuinely different professional backgrounds and expertise levels for your design project.
Reliability & validity
The study's validity is supported by its multi-method approach. Reliability could be enhanced by standardizing the stimuli and evaluation tasks across all judges.
Think critically
If creativity is so subjective, how can we ever objectively measure it in design, and what are the implications for design education and professional practice?
Design Principles
"Design evaluation criteria should be context-aware and account for the diverse perspectives of stakeholders, acknowledging that 'creativity' is perceived differently across professional and experience spectrums."
Understanding these biases is crucial for design teams and organizations aiming for objective evaluation of new concepts or products. It highlights the need for diverse review panels and clear, pre-defined assessment criteria to mitigate individual subjectivity.
What This Means for Your Design
How creative a design is seen depends a lot on who is looking at it – their job and how much they know about design changes their opinion.
How to use in your project
- 1.Reference this study when discussing the subjectivity of design evaluation and how to gather diverse feedback for your design project.
Add to My Project
Quick Cite
Paragraph starter
The assessment of design creativity is inherently subjective, influenced by the evaluator's background and expertise. Research indicates that professional roles (e.g., designer, art director, educator) and experience levels (expert vs. non-expert) significantly shape the criteria used and the agreement between judges. Therefore, when seeking feedback for a design project, it is crucial to engage a diverse group of evaluators and to clearly define the parameters of creativity being assessed to mitigate potential biases and ensure a comprehensive evaluation.
Source
theses.fr (ABES)
Creative product assessment in design : Influence of judges’ backgrounds and levels of experience in design
journal · 2014
View sourceQuestions About This Research
- What does the research say about expertise and background significantly alter design creativity assessments?
- When evaluating design concepts, be aware that subjective biases stemming from the evaluator's background and expertise will influence the outcome; actively manage these biases through structured processes and diverse perspectives. Evidence: theses.fr (ABES) (2014).
- Why does "Expertise and background significantly alter design creativity assessments" matter for design?
- Understanding these biases is crucial for design teams and organizations aiming for objective evaluation of new concepts or products. It highlights the need for diverse review panels and clear, pre-defined assessment criteria to mitigate individual subjectivity.
- How can designers apply this research?
- When evaluating design concepts, be aware that subjective biases stemming from the evaluator's background and expertise will influence the outcome; actively manage these biases through structured processes and diverse perspectives.
- What were the main findings?
- Judges' professional backgrounds (designer, art director, design educator, target audience) lead to variations in evaluative criteria.. Levels of design expertise (expert, intermediate, non-expert) impact the criteria considered and the agreement between judges.
- What research method was used?
- Multi-study qualitative and quantitative analysis.
- How strong is the evidence?
- Evidence strength is rated Strong effect, based on a 2014 journal from theses.fr (ABES).
- What should I do differently in my next project?
- When conducting peer reviews or seeking feedback on design work, explicitly define the criteria for 'creativity' and ensure the review panel includes individuals with different professional roles and varying levels of design experience.
- What are the limitations?
- The specific context of graphic design may not generalize to all design disciplines. The study's focus on stated versus actual criteria could also be a limitation.