Presentation
Are Our Human Factors Processes Ready for Artificial Intelligence in Medical Devices? Mapping a Literature Review of Hazards Associated With Artificial Intelligence to a Use-Related Risk Analysis
SessionPoster Session 1
DescriptionThe rapid integration of Artificial Intelligence (AI) in medical devices has created urgent questions about appropriate Human Factors (HF) methodologies. This, coupled with a lack of guidance, has left a gap in the framework for HF activities needed for AI systems. This is a presentation from a cross-disciplinary working team utilizing a review of literature to provide further guidance around the application of HF for AI system development, assessment of use-related risks, mitigation, and evaluation. To best explore this, the scope of the review was broadened to consider systems with various levels of autonomy and the use-related issues that arise when an autonomous agent is added to a workflow.
The hypothesis is that the use-related risks associated with the various levels of system autonomy, as they have been identified in other sectors (e.g. aviation, automotive), are applicable to medical systems. This presentation will explore this hypothesis and identify use-related hazards and design considerations critical to the design of AI enabled medical devices.
Considering factors such as over-reliance, loss of situational awareness, appropriate functional allocation, and change fatigue, we will look at the impact to safe use of systems and ask the critical question: For medical devices with autonomous elements (including AI), does our industry’s current Human Factors Engineering practices appropriately evaluate use error causes and mitigations?
This is the work of a cross-disciplinary team across the life science industry, consisting of human factors engineers, clinical specialists, and regulatory affairs with independent review from medical device AI thought leaders. The effort began with a review of concepts and literature from the Massachusetts Institute of Technology (MIT) AI Risk Framework1. From this framework an extraction was conducted for relevant articles based on applicability to Human Factors considerations across the medical product lifecycle. This extraction utilized a lens of safe-and-effective-use and included articles related to users, use scenarios, use errors, causes of use error, mitigations, user interface design principles, and excluded societal impacts and discussions of malicious intent.
From the articles identified, Human Factors themes and key topics were identified as they relate to medical devices design for various levels of autonomy. This team seeks to bring these findings forward and discuss considerations for systems in healthcare that are designed to be Human-in-the-Loop (HITL), Human-on-the Loop (HOTL), or Human-out-of-the-Loop (HOOTL).
To explore this and present a framework for discussion, a review of cleared/approved products within the AI Medical Device Taxonomy2, is used to identify examples for case study. In these case studies, examples are built out using common tools of practice from HF and Risk Management, specifically, examples will be presented from the following tools/methods:
- Use scenario identification
- 'Characteristics for Safety' questions (ref. ISO 14971)
- Task Analysis (TA)/Perception-Cognition-Action (PCA)/ Use-Related Risk Assessment (URRA)
The goal is to drive discussion around medical devices with autonomous elements (including AI) and potential impact to current Human Factors practices.
The hypothesis is that the use-related risks associated with the various levels of system autonomy, as they have been identified in other sectors (e.g. aviation, automotive), are applicable to medical systems. This presentation will explore this hypothesis and identify use-related hazards and design considerations critical to the design of AI enabled medical devices.
Considering factors such as over-reliance, loss of situational awareness, appropriate functional allocation, and change fatigue, we will look at the impact to safe use of systems and ask the critical question: For medical devices with autonomous elements (including AI), does our industry’s current Human Factors Engineering practices appropriately evaluate use error causes and mitigations?
This is the work of a cross-disciplinary team across the life science industry, consisting of human factors engineers, clinical specialists, and regulatory affairs with independent review from medical device AI thought leaders. The effort began with a review of concepts and literature from the Massachusetts Institute of Technology (MIT) AI Risk Framework1. From this framework an extraction was conducted for relevant articles based on applicability to Human Factors considerations across the medical product lifecycle. This extraction utilized a lens of safe-and-effective-use and included articles related to users, use scenarios, use errors, causes of use error, mitigations, user interface design principles, and excluded societal impacts and discussions of malicious intent.
From the articles identified, Human Factors themes and key topics were identified as they relate to medical devices design for various levels of autonomy. This team seeks to bring these findings forward and discuss considerations for systems in healthcare that are designed to be Human-in-the-Loop (HITL), Human-on-the Loop (HOTL), or Human-out-of-the-Loop (HOOTL).
To explore this and present a framework for discussion, a review of cleared/approved products within the AI Medical Device Taxonomy2, is used to identify examples for case study. In these case studies, examples are built out using common tools of practice from HF and Risk Management, specifically, examples will be presented from the following tools/methods:
- Use scenario identification
- 'Characteristics for Safety' questions (ref. ISO 14971)
- Task Analysis (TA)/Perception-Cognition-Action (PCA)/ Use-Related Risk Assessment (URRA)
The goal is to drive discussion around medical devices with autonomous elements (including AI) and potential impact to current Human Factors practices.
Event Type
Poster Presentation
TimeMonday, March 234:45pm - 6:15pm EDT
LocationRhinelander Gallery
Medical and Drug Delivery Devices