Short answer

Be wary of assessment frameworks that prioritize quantifiable, market-driven outcomes, as they may stifle true innovation and creativity in design projects.

Field: Innovation & Design
Source: YorkSpace (York University) (2013)
Method: Historical and critical analysis of quality assessment systems.
Evidence: Strong effect

Quality assessment systems, while intended to measure and improve excellence, often inadvertently reinforce existing commercial interests and a standardized workforce rather than fostering groundbreaking advancements. This innovation & design research insight is drawn from a 2013 study published in YorkSpace (York University). Using Historical and critical analysis of quality assessment systems., researchers explored how this design variable affects real-world outcomes. The key design takeaway: Be wary of assessment frameworks that prioritize quantifiable, market-driven outcomes, as they may stifle true innovation and creativity in design projects.

Study

Innovation & DesignHigh ImpactStrong effect

Quality Assessment Frameworks Can Hinder True Innovation by Prioritizing Commercial Value Over Excellence

YorkSpace (York University) · 2013

Key Findings

01Quality assessment systems do not effectively measure or produce qualitative excellence.
02QA systems are instrumental in reproducing commercially oriented research and a more adaptable labor force.
03QA systems were developed to introduce market-like mechanisms into publicly funded education, evaluating the exchange-value of knowledge and graduates.
04These systems serve to rationalize funding and discipline academic institutions and individuals.

Application

Design takeaway

Be wary of assessment frameworks that prioritize quantifiable, market-driven outcomes, as they may stifle true innovation and creativity in design projects.

How to apply

When developing new products or services, consider how their success will be measured. If relying on external assessment, understand the underlying values and potential biases of the evaluators.

Project actions

01When planning your design project, think about how you will measure its success. Is it just about meeting requirements, or is it about creating something truly new and valuable?
02Consider if the criteria you are using to judge your design are limiting your creativity or pushing you towards a more conventional solution.

Method & Evidence

AimTo investigate the actual impact of quality assessment (QA) systems in higher education on the production of qualitative excellence and the nature of research and graduates.

MethodHistorical and critical analysis of quality assessment systems.

ProcedureThe research traces the historical development of QA in the United States, the United Kingdom, and Ontario, analyzing its precursors and its evolution into contemporary measurement and evaluation procedures. It examines how these systems serve specific economic and political interests.

ContextHigher education systems in the United States, the United Kingdom, and Ontario.

Variables

IVQuality assessment frameworks and their historical development.

DVThe nature of research produced, the characteristics of graduates, and the fostering of qualitative excellence.

Strengths & Limitations

Strengths

+Provides a critical historical perspective on the evolution of quality assessment.
+Connects assessment practices to broader economic and political contexts.

Limitations

This study is about higher education, so its direct application to a specific product design project might require careful adaptation of the concepts.

Reliability & validity

The reliability of the findings rests on the historical analysis and critical interpretation of existing systems. Validity is established through the logical connection between the development of QA systems and their purported and actual outcomes.

Think critically

If quality assessment systems are designed to serve commercial interests, how can designers ensure their work pushes boundaries and contributes to genuine progress rather than just meeting market demands?

Design Principles

"Design evaluation should prioritize the generation of novel solutions and genuine user value over adherence to pre-defined commercial metrics."

Understanding how assessment systems are structured and their underlying motivations is crucial for designers and researchers. It highlights the potential for seemingly objective metrics to shape outcomes in ways that may not align with genuine innovation or user benefit, impacting the direction of research and development.

What This Means for Your Design

The way we measure 'quality' in education can actually shape what kind of research and graduates are produced, often favoring things that are commercially useful rather than truly groundbreaking.

How to use in your project

1.You can use this research to critically analyze the evaluation criteria for your design project. Ask yourself if these criteria are truly helping you achieve innovative outcomes or if they are leading you down a more predictable path.

Add to My Project

Quick Cite

Paragraph starter

The research by Newstadt (2013) suggests that systems designed to assess 'quality' can inadvertently prioritize commercial viability over genuine innovation. This highlights the importance of critically examining the evaluation criteria used in design projects to ensure they foster creativity and novel solutions rather than simply reinforcing existing market trends.

Source

YorkSpace (York University)

The Value of Quality: Capital, Class, and Quality Assessment in the Re-making of Higher Education in the United State, the United Kingdom, and Ontario

journal · 2013

View source

Questions About This Research

What does the research say about quality assessment frameworks can hinder true innovation by prioritizing commercial value over excellence?: Be wary of assessment frameworks that prioritize quantifiable, market-driven outcomes, as they may stifle true innovation and creativity in design projects. Evidence: YorkSpace (York University) (2013).
Why does "Quality Assessment Frameworks Can Hinder True Innovation by Prioritizing Commercial Value Over Excellence" matter for design?: Understanding how assessment systems are structured and their underlying motivations is crucial for designers and researchers. It highlights the potential for seemingly objective metrics to shape outcomes in ways that may not align with genuine innovation or user benefit, impacting the direction of research and development.
How can designers apply this research?: Be wary of assessment frameworks that prioritize quantifiable, market-driven outcomes, as they may stifle true innovation and creativity in design projects.
What were the main findings?: Quality assessment systems do not effectively measure or produce qualitative excellence.. QA systems are instrumental in reproducing commercially oriented research and a more adaptable labor force.. QA systems were developed to introduce market-like mechanisms into publicly funded education, evaluating the exchange-value of knowledge and graduates.. These systems serve to rationalize funding and discipline academic institutions and individuals.
What research method was used?: Historical and critical analysis of quality assessment systems..
How strong is the evidence?: Evidence strength is rated Strong effect, based on a 2013 journal from YorkSpace (York University).
What should I do differently in my next project?: When developing new products or services, consider how their success will be measured. If relying on external assessment, understand the underlying values and potential biases of the evaluators.
What are the limitations?: The study focuses specifically on higher education and may not directly translate to all design contexts. The analysis is critical and theoretical, rather than empirical in terms of direct user testing of assessment methods.