Short answer

Implement a structured evaluation framework like QUEST when assessing LLMs for healthcare to ensure safety, reliability, and user trust.

Field: User-Centred Design
Source: npj Digital Medicine (2024)
Method: Literature Review and Framework Development
Sample: 142 studies reviewed
Evidence: Strong effect

A systematic framework, QUEST, can significantly improve the reliability and applicability of human evaluations for large language models in healthcare. This user-centred design research insight is drawn from a 2024 study published in npj Digital Medicine. Using Literature review and framework development with 142 studies reviewed, researchers explored how this design variable affects real-world outcomes. The key design takeaway: Implement a structured evaluation framework like QUEST when assessing LLMs for healthcare to ensure safety, reliability, and user trust.

Study

User-Centred DesignRecentStrong effect

A Structured Framework (QUEST) Enhances Human Evaluation of Healthcare LLMs

A systematic framework, QUEST, can significantly improve the reliability and applicability of human evaluations for large language models in healthcare.

npj Digital Medicine · 2024

Key Findings

01Existing human evaluation practices for healthcare LLMs suffer from gaps in reliability, generalizability, and applicability.
02A structured framework is needed to guide the planning, implementation, and adjudication of LLM evaluations.
03Key evaluation principles should include Quality of Information, Understanding and Reasoning, Expression Style and Persona, Safety and Harm, and Trust and Confidence.

Application

Design takeaway

Implement a structured evaluation framework like QUEST when assessing LLMs for healthcare to ensure safety, reliability, and user trust.

How to apply

When designing or evaluating an LLM for a healthcare context, use the QUEST framework to systematically plan, conduct, and analyze human evaluations, focusing on the five core principles.

Project actions

01When evaluating an AI tool, think about how you will get people to test it and how you will measure their feedback.
02Consider using a framework like QUEST to structure your evaluation process, ensuring you cover all important aspects like accuracy and safety.

Method & Evidence

AimHow can a structured framework improve the reliability, generalizability, and applicability of human evaluations for large language models in healthcare?

MethodLiterature Review and Framework Development

ProcedureThe researchers conducted a comprehensive literature review of 142 studies on human evaluation of LLMs in healthcare. Based on identified gaps, they developed the QUEST framework, outlining principles and phases for evaluation.

Sample142 studies reviewed

ContextHealthcare applications of Large Language Models (LLMs)

Variables

IVThe implementation of the QUEST framework (structured vs. unstructured evaluation)

DVReliability, generalizability, and applicability of human evaluations

CVType of LLM, healthcare specialty, evaluator demographics

Strengths & Limitations

Strengths

+Comprehensive literature review provides a strong foundation.
+Development of a practical, phased framework (QUEST).

Limitations

It can be challenging to recruit a diverse and representative group of healthcare professionals for user testing.

Reliability & validity

The literature review identified a lack of reliability and generalizability in current methods, which the QUEST framework aims to address by providing standardized procedures and clear criteria.

Think critically

How might the 'Expression Style and Persona' principle be interpreted differently by various healthcare professionals (e.g., doctors vs. nurses vs. patients)?

Design Principles

"Human evaluation of AI systems in critical domains should follow a structured, multi-phase approach guided by clearly defined principles."

As AI tools like LLMs become more integrated into healthcare, ensuring their safety and effectiveness through rigorous human evaluation is paramount. A structured approach moves beyond ad-hoc assessments, providing a repeatable and comparable method for judging AI performance in critical medical contexts.

What This Means for Your Design

Testing AI language tools for doctors and patients needs a clear, step-by-step plan to make sure the tests are fair and the results are trustworthy.

How to use in your project

1.Reference the QUEST framework as a model for structuring human evaluation sections of your design project.
2.Use the five evaluation principles (Quality, Understanding, Style, Safety, Trust) as criteria for your own user testing.

Add to My Project

Quick Cite

Paragraph starter

The development and deployment of AI tools in healthcare necessitate rigorous human evaluation. Drawing upon the QUEST framework, this design project employed a structured approach to user testing, focusing on key principles such as the quality of information provided, the AI's understanding and reasoning capabilities, its expression style and persona, potential for safety and harm, and overall trust and confidence. This systematic methodology ensures a comprehensive assessment of the AI's suitability for its intended healthcare application.

Source

npj Digital Medicine

A framework for human evaluation of large language models in healthcare derived from literature review

journal · 2024

View source

Questions About This Research

What does the research say about a structured framework (quest) enhances human evaluation of healthcare llms?: Implement a structured evaluation framework like QUEST when assessing LLMs for healthcare to ensure safety, reliability, and user trust. Evidence: npj Digital Medicine (2024).
Why does "A Structured Framework (QUEST) Enhances Human Evaluation of Healthcare LLMs" matter for design?: As AI tools like LLMs become more integrated into healthcare, ensuring their safety and effectiveness through rigorous human evaluation is paramount. A structured approach moves beyond ad-hoc assessments, providing a repeatable and comparable method for judging AI performance in critical medical contexts.
How can designers apply this research?: Implement a structured evaluation framework like QUEST when assessing LLMs for healthcare to ensure safety, reliability, and user trust.
What were the main findings?: Existing human evaluation practices for healthcare LLMs suffer from gaps in reliability, generalizability, and applicability.. A structured framework is needed to guide the planning, implementation, and adjudication of LLM evaluations.. Key evaluation principles should include Quality of Information, Understanding and Reasoning, Expression Style and Persona, Safety and Harm, and Trust and Confidence.
What research method was used?: Literature Review and Framework Development with 142 studies reviewed.
How strong is the evidence?: Evidence strength is rated Strong effect, based on a 2024 journal from npj Digital Medicine.
What should I do differently in my next project?: When designing or evaluating an LLM for a healthcare context, use the QUEST framework to systematically plan, conduct, and analyze human evaluations, focusing on the five core principles.
What are the limitations?: The proposed framework is derived from a literature review and requires empirical validation through its application in real-world design projects.