Short answer
Designers and research managers should advocate for and implement assessment systems that value qualitative contributions, creative processes, and the long-term impact of design solutions, rather than solely focusing on easily quantifiable metrics.
- Field
- Innovation & Design
- Source
- Kent Academic Repository (University of Kent) (2015)
- Method
- Expert Review and Policy Analysis
- Evidence
- Moderate effect
Focusing solely on measurable outputs can inadvertently penalize novel, exploratory, or interdisciplinary design projects that may not yield immediate, quantifiable results. This innovation & design research insight is drawn from a 2015 study published in Kent Academic Repository (University of Kent). Using Expert review and policy analysis, researchers explored how this design variable affects real-world outcomes. The key design takeaway: Designers and research managers should advocate for and implement assessment systems that value qualitative contributions, creative processes, and the long-term impact of design solutions, rather than solely focusing on easily quantifiable metrics.
Over-reliance on quantitative metrics can stifle genuine innovation in design research.
Focusing solely on measurable outputs can inadvertently penalize novel, exploratory, or interdisciplinary design projects that may not yield immediate, quantifiable results.
Kent Academic Repository (University of Kent) · 2015
Key Findings
- 01Quantitative metrics can be useful for certain aspects of research evaluation.
- 02Over-reliance on a narrow set of metrics can lead to unintended consequences, such as gaming the system or discouraging risk-taking.
- 03The complexity and qualitative nature of some research, including design, are not always well-captured by standard metrics.
- 04A more balanced approach, incorporating qualitative assessment and expert judgment, is often necessary.
Application
Design takeaway
Designers and research managers should advocate for and implement assessment systems that value qualitative contributions, creative processes, and the long-term impact of design solutions, rather than solely focusing on easily quantifiable metrics.
How to apply
When proposing or evaluating design research projects, ensure that the assessment criteria acknowledge the value of exploration, user-centered insights, and qualitative outcomes, in addition to any measurable results.
Project actions
- 01When presenting your design project, highlight the innovative aspects of your process and the qualitative user feedback you received, not just the final product's performance metrics.
- 02Consider how your project's success might be measured beyond simple quantitative data, especially if it involves user experience or emotional design.
Method & Evidence
Variables
Strengths & Limitations
Strengths
- +Comprehensive review by a multidisciplinary expert panel.
- +Addresses a critical issue in research management and policy.
Limitations
The specific metrics used in academic research assessment may differ from those prioritized in commercial design contexts.
Reliability & validity
The reliability of the findings stems from the consensus of a multidisciplinary expert panel. Validity is supported by the comprehensive nature of the review and its focus on policy implications.
Think critically
How can design research frameworks be adapted to better capture and reward the inherent creativity, user empathy, and systemic impact of design solutions, moving beyond a purely metric-driven evaluation?
Design Principles
"Evaluate design research holistically, recognizing both quantitative and qualitative contributions to innovation and impact."
Design practice thrives on creativity, experimentation, and the exploration of complex human needs. When assessment frameworks prioritize easily quantifiable metrics, they risk devaluing the very processes that lead to breakthrough solutions. This can discourage designers from pursuing ambitious or unconventional projects.
What This Means for Your Design
Don't just count things; understand the quality and creativity behind design work. Focusing too much on numbers can stop new and brilliant ideas from happening.
How to use in your project
- 1.Reference this insight when discussing the limitations of quantitative data in evaluating the success or impact of your design project, particularly if your project involved novel approaches or subjective user experiences.
Add to My Project
Quick Cite
Paragraph starter
The evaluation of design research necessitates a balanced approach that transcends purely quantitative metrics. As highlighted by Wilsdon et al. (2015), an over-reliance on easily measurable outputs can inadvertently stifle innovation by devaluing the exploratory nature of design and the qualitative insights gained from user-centered processes. Therefore, when assessing the impact of a design project, it is crucial to consider a wider range of criteria, including the novelty of the approach, the depth of user understanding, and the potential for long-term societal or user benefit, rather than solely focusing on easily quantifiable performance indicators.
Source
Kent Academic Repository (University of Kent)
The Metric Tide: Report of the Independent Review of the Role of Metrics in Research Assessment and Management
journal · 2015
View sourceQuestions About This Research
- What does the research say about over-reliance on quantitative metrics can stifle genuine innovation in design research?
- Designers and research managers should advocate for and implement assessment systems that value qualitative contributions, creative processes, and the long-term impact of design solutions, rather than solely focusing on easily quantifiable metrics. Evidence: Kent Academic Repository (University of Kent) (2015).
- Why does "Over-reliance on quantitative metrics can stifle genuine innovation in design research." matter for design?
- Design practice thrives on creativity, experimentation, and the exploration of complex human needs. When assessment frameworks prioritize easily quantifiable metrics, they risk devaluing the very processes that lead to breakthrough solutions. This can discourage designers from pursuing ambitious or unconventional projects.
- How can designers apply this research?
- Designers and research managers should advocate for and implement assessment systems that value qualitative contributions, creative processes, and the long-term impact of design solutions, rather than solely focusing on easily quantifiable metrics.
- What were the main findings?
- Quantitative metrics can be useful for certain aspects of research evaluation.. Over-reliance on a narrow set of metrics can lead to unintended consequences, such as gaming the system or discouraging risk-taking.. The complexity and qualitative nature of some research, including design, are not always well-captured by standard metrics.. A more balanced approach, incorporating qualitative assessment and expert judgment, is often necessary.
- What research method was used?
- Expert Review and Policy Analysis.
- How strong is the evidence?
- Evidence strength is rated Moderate effect, based on a 2015 journal from Kent Academic Repository (University of Kent).
- What should I do differently in my next project?
- When proposing or evaluating design research projects, ensure that the assessment criteria acknowledge the value of exploration, user-centered insights, and qualitative outcomes, in addition to any measurable results.
- What are the limitations?
- The review's findings are primarily focused on academic research assessment and may not directly translate to all industry design contexts.