Presentation
Patient and Clinician Testing of a Remote Stethoscope in Simulation Lab: A Case Study for Usability and AI Integration Readiness
SessionPoster Session 1
DescriptionBackground & Objective. We evaluated an FDA-cleared, cloud-connected electronic stethoscope without AI functionality as a use case to (1) characterize critical usability tasks in a realistic, simulation-based environment and (2) demonstrate a traceable human factors approach that support future AI integration. Could this study replicate as a model for scaling usability validation of AI-adjacent devices?
Methods. Through simulated workflows involving both human and manikin subjects, the research was conducted at Mayo Clinic’s Multidisciplinary Simulation Laboratory with two cohorts – standardized patients (SPs) and healthcare providers (HCPs). Users completed a prespecified task list under realistic conditions (equipment, setup, and workflow). Minimum enrollment targeted at least 15 SPs and 15 HCPs aligning to the FDA guidance for human factors validation testing. The protocol included inclusion/exclusion criteria, training, and scripted scenarios. Study staff captured SP and HCP task success, observations, and debrief interviews. Would the Sim Lab be operationally ready to evaluate the influx of digital tools and would their resources such as the pool of SPs be representative in general population scenarios?
Governance & Reporting. Study conduct followed a minimal-risk IRB protocol emphasizing de-identification, access controls, and mitigation of device and data risks. All lung sounds were recorded from a manikin to avoid participant exposure. Result reporting was aligned to execution record practices (ISO 13485/21 CFR 820.30). If a new device was tested, is the validation data suitable for governing bodies?
Findings & Implications. Early simulation runs showed successful core operations, surfaced setup challenges (e.g., Wi-Fi connectivity), and demonstrated the value of immediate, simulation-based feedback for AI-readiness assessments of digital health tools and reinforced the role of simulation centers in regulatory-aligned usability testing.
Methods. Through simulated workflows involving both human and manikin subjects, the research was conducted at Mayo Clinic’s Multidisciplinary Simulation Laboratory with two cohorts – standardized patients (SPs) and healthcare providers (HCPs). Users completed a prespecified task list under realistic conditions (equipment, setup, and workflow). Minimum enrollment targeted at least 15 SPs and 15 HCPs aligning to the FDA guidance for human factors validation testing. The protocol included inclusion/exclusion criteria, training, and scripted scenarios. Study staff captured SP and HCP task success, observations, and debrief interviews. Would the Sim Lab be operationally ready to evaluate the influx of digital tools and would their resources such as the pool of SPs be representative in general population scenarios?
Governance & Reporting. Study conduct followed a minimal-risk IRB protocol emphasizing de-identification, access controls, and mitigation of device and data risks. All lung sounds were recorded from a manikin to avoid participant exposure. Result reporting was aligned to execution record practices (ISO 13485/21 CFR 820.30). If a new device was tested, is the validation data suitable for governing bodies?
Findings & Implications. Early simulation runs showed successful core operations, surfaced setup challenges (e.g., Wi-Fi connectivity), and demonstrated the value of immediate, simulation-based feedback for AI-readiness assessments of digital health tools and reinforced the role of simulation centers in regulatory-aligned usability testing.
Authors
Senior Digital Strategist
Event Type
Poster Presentation
TimeMonday, March 234:45pm - 6:15pm EDT
LocationRhinelander Gallery
Digital Health