Short answer

Incorporate comprehensive software testing and validation throughout the design and development process, and be aware of evolving regulatory requirements for digital health products.

Field: Innovation & Design
Source: Milbank Quarterly (2017)
Method: Database analysis and qualitative evaluation of recall data.
Sample: 627 software devices (1.4 million units) recalled; 12 high-risk recalls evaluated.
Evidence: Strong effect

A significant number of medical device recalls are attributed to software defects, indicating that current regulatory frameworks may not be sufficient to ensure patient safety. This innovation & design research insight is drawn from a 2017 study published in Milbank Quarterly. Using Database analysis and qualitative evaluation of recall data. with 627 software devices (1.4 million units) recalled; 12 high-risk recalls evaluated., researchers explored how this design variable affects real-world outcomes. The key design takeaway: Incorporate comprehensive software testing and validation throughout the design and development process, and be aware of evolving regulatory requirements for digital health products.

Study

Innovation & DesignHigh ImpactStrong effect

Software Defects in Medical Devices Lead to Recalls, Underscoring Regulatory Gaps

A significant number of medical device recalls are attributed to software defects, indicating that current regulatory frameworks may not be sufficient to ensure patient safety.

Milbank Quarterly · 2017

Key Findings

01627 software devices, totaling 1.4 million units, were recalled due to software defects.
0212 of these devices were high-risk recalls, affecting 190,596 units.
03Many high-risk recalled devices entered the market through pathways that did not require evidence of safety or effectiveness.

Application

Design takeaway

Incorporate comprehensive software testing and validation throughout the design and development process, and be aware of evolving regulatory requirements for digital health products.

How to apply

When designing medical devices or health information technology, implement a multi-stage testing protocol for all software components, including unit testing, integration testing, and user acceptance testing, with a focus on failure modes and effects analysis.

Project actions

01When designing a product with software, think about how to test it thoroughly to avoid bugs.
02Consider the regulatory environment for your product and how it might affect your design choices.

Method & Evidence

AimTo quantify the impact of software problems in regulated medical devices and assess the adequacy of current FDA regulatory safeguards and recent legislative changes.

MethodDatabase analysis and qualitative evaluation of recall data.

ProcedureResearchers identified and analyzed medical device recalls primarily due to software defects from 2011-2015 using FDA databases. They categorized recalls by risk level and specifically evaluated high-risk and moderate-risk electronic medical record recalls.

Sample627 software devices (1.4 million units) recalled; 12 high-risk recalls evaluated.

ContextRegulation of health information technology and medical devices.

Variables

IVRegulatory safeguards and legislative changes for medical devices.

DVNumber and risk level of software-related recalls.

CVType of medical device, FDA risk category, submission type.

Strengths & Limitations

Strengths

+Utilizes real-world data from FDA recall databases.
+Focuses on a critical aspect of patient safety in healthcare technology.

Limitations

The study is based on past data and may not reflect current regulatory practices or emerging software development methodologies.

Reliability & validity

The study's reliability is supported by its use of official FDA data. Validity is enhanced by the focus on specific recall reasons (software defects) and risk categories, though the qualitative evaluation of high-risk recalls introduces a degree of subjectivity.

Think critically

To what extent do current design processes adequately address the unique challenges of software development in safety-critical systems?

Design Principles

"Prioritize safety and efficacy through robust design and validation, especially when software is a critical component of a product."

This research highlights a critical area where design and engineering intersect with regulatory oversight. Understanding the prevalence and impact of software-related failures is crucial for developing more robust medical technologies and for informing future regulatory strategies that prioritize user safety.

What This Means for Your Design

Many medical devices have to be recalled because their software has problems. This shows that the rules for checking medical software might not be strong enough to keep people safe.

How to use in your project

1.Use this study to justify the importance of rigorous software testing and validation in your design project, particularly if your project involves technology or healthcare.

Add to My Project

Quick Cite

Paragraph starter

The prevalence of software-related recalls in medical devices, as documented by Ronquillo and Zuckerman (2017), underscores the critical need for comprehensive software validation and verification within the design process. This research highlights that current regulatory safeguards may be insufficient, leading to potential patient harm and necessitating a proactive approach to identifying and mitigating software defects throughout the product lifecycle.

Source

Milbank Quarterly

Software‐Related Recalls of Health Information Technology and Other Medical Devices: Implications for FDA Regulation of Digital Health

journal · 2017

View source

Questions About This Research

What does the research say about software defects in medical devices lead to recalls, underscoring regulatory gaps?: Incorporate comprehensive software testing and validation throughout the design and development process, and be aware of evolving regulatory requirements for digital health products. Evidence: Milbank Quarterly (2017).
Why does "Software Defects in Medical Devices Lead to Recalls, Underscoring Regulatory Gaps" matter for design?: This research highlights a critical area where design and engineering intersect with regulatory oversight. Understanding the prevalence and impact of software-related failures is crucial for developing more robust medical technologies and for informing future regulatory strategies that prioritize user safety.
How can designers apply this research?: Incorporate comprehensive software testing and validation throughout the design and development process, and be aware of evolving regulatory requirements for digital health products.
What were the main findings?: 627 software devices, totaling 1.4 million units, were recalled due to software defects.. 12 of these devices were high-risk recalls, affecting 190,596 units.. Many high-risk recalled devices entered the market through pathways that did not require evidence of safety or effectiveness.
What research method was used?: Database analysis and qualitative evaluation of recall data. with 627 software devices (1.4 million units) recalled; 12 high-risk recalls evaluated..
How strong is the evidence?: Evidence strength is rated Strong effect, based on a 2017 journal from Milbank Quarterly.
What should I do differently in my next project?: When designing medical devices or health information technology, implement a multi-stage testing protocol for all software components, including unit testing, integration testing, and user acceptance testing, with a focus on failure modes and effects analysis.
What are the limitations?: The study period is limited to 2011-2015, and the analysis focuses on recalls, which may not capture all instances of software-related issues that did not result in a recall.