Presentation
AI as Your Ally: Accelerating Post Market Use Error Investigations
DescriptionUnder 21 CFR Part 820, medical device manufacturers are required to perform comprehensive investigations to determine the root cause of a product or process non-conformance. These investigations are typically carried out by cross-functional teams of engineering and clinical experts. However, in our experience, human factors (HF) considerations are not consistently integrated into these investigations, which can result in overlooking critical usability issues that affect device performance and safety. To address this gap, we have integrated HF engineers into the investigation process. When use errors are identified, involving HF experts enables a more accurate root cause analysis, ensures alignment with regulatory expectations, and provides valuable insights. This approach not only enhances the quality of investigations but also informs future design improvements aimed at preventing similar use errors.
In order to facilitate this analysis, we have a developed a Use Error Tool based on the UPCARE model (Kaye, R. D, North, R.A., and Peterson, M. K. (2003) UPCARE: An analysis, description, and educational tool for medical device use problems. Proceedings of the 9th Annual International Conference on Industrial Engineering Theory, Applications and Practice. Las Vegas, NV.) The UPCARE model takes its name from six domain areas: 1) Unmet user needs, 2) Perception, 3) Cognition, 4) Actions, 5) Results, and 6) Evaluation. Each of these areas is broken down into targeted questions to evaluate potential user interface design shortcomings that may have led or contributed to the use error. In addition, the Use Error Tool helps to uncover any gaps in the usability process that may have occurred during the development of the user interface relevant to the use error.
The Use Error Tool is completed by a HF / Usability subject matter expert, along with the cross-functional team (typically engineering and clinical representatives). The challenge for the HF engineer is that these investigations are often complex, time-sensitive, and require deep technical and clinical knowledge. The use errors may also involve non-standard or rare workflows, which may be unfamiliar to the HF engineer. In order to assist with these challenges, we have been using generative AI tools (Microsoft Copilot) to assist with completing the Use Error Tool. Specifically, we are experimenting with using Copilot as a way to interpret and simplify investigation narratives, as a sounding board for validating the identified use error root cause(s), and as a brainstorming partner for suggesting potential design improvements to enhance usability and patient safety.
This submission will focus on the details of the Use Error Tool, and how we use it to investigate the root cause of use errors with medical devices. We will also present our experience with integrating Copilot into the investigation process, and key insights on the benefits and limitations of this method.
In order to facilitate this analysis, we have a developed a Use Error Tool based on the UPCARE model (Kaye, R. D, North, R.A., and Peterson, M. K. (2003) UPCARE: An analysis, description, and educational tool for medical device use problems. Proceedings of the 9th Annual International Conference on Industrial Engineering Theory, Applications and Practice. Las Vegas, NV.) The UPCARE model takes its name from six domain areas: 1) Unmet user needs, 2) Perception, 3) Cognition, 4) Actions, 5) Results, and 6) Evaluation. Each of these areas is broken down into targeted questions to evaluate potential user interface design shortcomings that may have led or contributed to the use error. In addition, the Use Error Tool helps to uncover any gaps in the usability process that may have occurred during the development of the user interface relevant to the use error.
The Use Error Tool is completed by a HF / Usability subject matter expert, along with the cross-functional team (typically engineering and clinical representatives). The challenge for the HF engineer is that these investigations are often complex, time-sensitive, and require deep technical and clinical knowledge. The use errors may also involve non-standard or rare workflows, which may be unfamiliar to the HF engineer. In order to assist with these challenges, we have been using generative AI tools (Microsoft Copilot) to assist with completing the Use Error Tool. Specifically, we are experimenting with using Copilot as a way to interpret and simplify investigation narratives, as a sounding board for validating the identified use error root cause(s), and as a brainstorming partner for suggesting potential design improvements to enhance usability and patient safety.
This submission will focus on the details of the Use Error Tool, and how we use it to investigate the root cause of use errors with medical devices. We will also present our experience with integrating Copilot into the investigation process, and key insights on the benefits and limitations of this method.
Event Type
Oral Presentations
TimeTuesday, March 2410:52am - 11:15am EDT
LocationMurray Hill East
Patient Safety Research and Initiatives