Short answer
When conducting or evaluating design research that aims for real-world impact, prioritize principles of relevance, credibility, legitimacy, and effectiveness, ensuring that stakeholder perspectives and practical outcomes are central to the assessment.
- Field
- User-Centred Design
- Source
- Research Evaluation (2015)
- Method
- Systematic Review and Framework Development
- Evidence
- Moderate effect
Evaluating transdisciplinary research requires a framework that extends beyond traditional academic rigor to include stakeholder relevance, integration of diverse knowledge, and demonstrable effectiveness in addressing real-world problems. This user-centred design research insight is drawn from a 2015 study published in Research Evaluation. Using Systematic review and framework development, researchers explored how this design variable affects real-world outcomes. The key design takeaway: When conducting or evaluating design research that aims for real-world impact, prioritize principles of relevance, credibility, legitimacy, and effectiveness, ensuring that stakeholder perspectives and practical outcomes are central to the assessment.
Transdisciplinary Research Quality Framework Enhances Stakeholder Relevance and Problem-Solving Effectiveness
Evaluating transdisciplinary research requires a framework that extends beyond traditional academic rigor to include stakeholder relevance, integration of diverse knowledge, and demonstrable effectiveness in addressing real-world problems.
Research Evaluation · 2015
Key Findings
- 01Traditional academic criteria for research quality are insufficient for transdisciplinary research.
- 02Four main principles for TDR quality are: relevance (social significance, applicability), credibility (integration, reflexivity, rigor), legitimacy (inclusion, fair representation), and effectiveness (problem-solving contributions, social change).
Application
Design takeaway
When conducting or evaluating design research that aims for real-world impact, prioritize principles of relevance, credibility, legitimacy, and effectiveness, ensuring that stakeholder perspectives and practical outcomes are central to the assessment.
How to apply
When planning a design research project, develop an evaluation rubric based on the four principles (relevance, credibility, legitimacy, effectiveness) to guide the research process and assess its success beyond traditional academic metrics.
Project actions
- 01When defining the scope of your design project, consider who the stakeholders are and how their input will be integrated.
- 02Plan how you will measure the 'effectiveness' of your design solution beyond just user satisfaction, considering its real-world impact.
Method & Evidence
Variables
Strengths & Limitations
Strengths
- +Provides a structured framework for evaluating complex, multi-stakeholder research.
- +Integrates both academic rigor and real-world impact into quality assessment.
Limitations
It can be challenging to objectively measure 'social significance' or 'effectiveness' in a design project, and defining 'fair representation' of all stakeholders can be complex.
Reliability & validity
The reliability of this framework would depend on the consistency of application across different evaluators and contexts. Validity is supported by its grounding in a systematic review and practical experience, aiming to capture the multifaceted nature of TDR quality.
Think critically
How might the emphasis on 'effectiveness' and 'social change' in TDR potentially conflict with the pursuit of pure scientific knowledge or aesthetic innovation in design?
Design Principles
"Research quality in applied design contexts is a multi-faceted construct encompassing scientific rigor, stakeholder relevance, knowledge integration, and demonstrable problem-solving effectiveness."
In design practice, projects often involve diverse stakeholders and aim for tangible impact. Understanding how to assess the quality of research that bridges academic knowledge with practical application is crucial for ensuring that design solutions are not only technically sound but also socially significant and effective.
What This Means for Your Design
To judge if research is good when it mixes different subjects and involves people from outside academia, we need to look at more than just how scientifically correct it is. We also need to see if it's useful for society, if it brings different ideas together well, if it's fair to everyone involved, and if it actually helps solve problems.
How to use in your project
- 1.Reference this framework when discussing the criteria you used to evaluate the success or quality of your design research, especially if it involved interdisciplinary collaboration or significant stakeholder input.
Add to My Project
Quick Cite
Paragraph starter
The evaluation of this design research project draws upon principles of transdisciplinary research quality, recognizing that effective solutions require more than just technical merit. Criteria such as relevance to stakeholder needs, credibility through integrated knowledge, legitimacy in representing diverse interests, and demonstrable effectiveness in problem-solving were considered to ensure a holistic assessment of the project's success and impact.
Source
Research Evaluation
Defining and assessing research quality in a transdisciplinary context
journal · 2015
View sourceQuestions About This Research
- What does the research say about transdisciplinary research quality framework enhances stakeholder relevance and problem-solving effectiveness?
- When conducting or evaluating design research that aims for real-world impact, prioritize principles of relevance, credibility, legitimacy, and effectiveness, ensuring that stakeholder perspectives and practical outcomes are central to the assessment. Evidence: Research Evaluation (2015).
- Why does "Transdisciplinary Research Quality Framework Enhances Stakeholder Relevance and Problem-Solving Effectiveness" matter for design?
- In design practice, projects often involve diverse stakeholders and aim for tangible impact. Understanding how to assess the quality of research that bridges academic knowledge with practical application is crucial for ensuring that design solutions are not only technically sound but also socially significant and effective.
- How can designers apply this research?
- When conducting or evaluating design research that aims for real-world impact, prioritize principles of relevance, credibility, legitimacy, and effectiveness, ensuring that stakeholder perspectives and practical outcomes are central to the assessment.
- What were the main findings?
- Traditional academic criteria for research quality are insufficient for transdisciplinary research.. Four main principles for TDR quality are: relevance (social significance, applicability), credibility (integration, reflexivity, rigor), legitimacy (inclusion, fair representation), and effectiveness (problem-solving contributions, social change).
- What research method was used?
- Systematic Review and Framework Development.
- How strong is the evidence?
- Evidence strength is rated Moderate effect, based on a 2015 journal from Research Evaluation.
- What should I do differently in my next project?
- When planning a design research project, develop an evaluation rubric based on the four principles (relevance, credibility, legitimacy, effectiveness) to guide the research process and assess its success beyond traditional academic metrics.
- What are the limitations?
- The framework is a prototype and may require further refinement and validation across diverse transdisciplinary research settings. The specific indicators and measures for each criterion may need adaptation to different project contexts.