Presentation
RAG-Enhanced LLM System with Decision Tree for Medical Device Usability Engineering Process
SessionPoster Session 1
DescriptionResearch Background: Medical device manufacturers, particularly small-to-medium enterprises and those lacking regulatory expertise, face significant barriers when attempting to implement IEC 62366-1/-2 usability engineering requirements. The complexity of international standards creates substantial hurdles for manufacturers who struggle to interpret and apply the intricate regulatory framework, leading to prolonged development cycles, extensive reliance on costly consultants, and potential delays in bringing life-saving medical innovations to market.
Research Purpose: This research aims to develop an intelligent decision-tree system that guides manufacturers through systematic usability engineering implementation using RAG-enhanced automated document generation powered by Large Language Models, thereby reducing regulatory barriers and democratizing access to specialized regulatory knowledge through evidence-based AI assistance. Building upon this foundation, future work will expand the system's scope to encompass other essential medical device development standards, including ISO 13485 (Quality Management Systems) and ISO 14971 (Risk Management). This expansion will enable the system to guide manufacturers in generating a wider range of mandatory regulatory documents, fostering a more comprehensive and streamlined understanding of the entire medical device development lifecycle.
Research Methods: We developed a comprehensive four-phase methodology. First, we created an intelligent decision-tree algorithm with 37 decision points that adaptively routes manufacturers through customized usability engineering pathways based on their capabilities and existing documentation status. Second, we implemented this decision tree within an HTML-based web application that collects essential manufacturer information including device specifications, user profiles, use environments, and company capabilities. Third, we integrated RAG-enhanced LLM-based document generation using Gemini Flash and Claude Opus APIs, where the RAG system utilized comprehensive knowledge bases containing IEC 62366-1/-2 international standards and validated usability engineering files from successful regulatory submissions to strengthen LLM performance and ensure regulatory accuracy. The system systematically transforms regulatory requirements into specialized prompts through prompt engineering techniques while leveraging RAG retrieval for contextually relevant regulatory guidance. The system incorporates automated web crawling of FDA MAUDE database for hazard identification and maintains document interconnectivity where generated documents serve as inputs for subsequent process steps. Fourth, we applied this integrated system to four South Korean medical device companies, generating comprehensive usability engineering files for artificial kidney machines, neonatal incubators, AI-based knee X-ray analysis software, and light-guided vocal cord injection devices. Finally, we initiated evaluation by five experts in medical device regulation and usability engineering through structured evaluation.
Research Results: The intelligent decision-tree development proved successful, creating an adaptive guidance system that effectively routes manufacturers through appropriate regulatory pathways. The web application successfully enables regulatory personnel to easily input medical device information and understand the usability engineering process while following systematic guidance. We confirmed the system's capability to generate comprehensive usability engineering files across diverse device categories and complexity levels. The completed documents represent draft-level quality requiring mandatory review and modification by responsible personnel, maintaining appropriate human oversight for safety-critical content. Document generation for four companies' devices was completed successfully, and these documents are currently undergoing expert evaluation to assess completeness and professional quality.
Research Significance: This work establishes a new paradigm for AI-human collaboration in complex regulatory domains, demonstrating how intelligent decision-making can be combined with automated document generation to address critical industry barriers while preserving essential safety oversight mechanisms. By extending this framework to include other pivotal standards like ISO 13485 and ISO 14971, this research paves the way for a more integrated, user-friendly, and holistic system that empowers manufacturers to navigate the entire regulatory landscape with greater efficiency and confidence, ultimately accelerating the delivery of safe and effective medical technologies to the market.
Research Purpose: This research aims to develop an intelligent decision-tree system that guides manufacturers through systematic usability engineering implementation using RAG-enhanced automated document generation powered by Large Language Models, thereby reducing regulatory barriers and democratizing access to specialized regulatory knowledge through evidence-based AI assistance. Building upon this foundation, future work will expand the system's scope to encompass other essential medical device development standards, including ISO 13485 (Quality Management Systems) and ISO 14971 (Risk Management). This expansion will enable the system to guide manufacturers in generating a wider range of mandatory regulatory documents, fostering a more comprehensive and streamlined understanding of the entire medical device development lifecycle.
Research Methods: We developed a comprehensive four-phase methodology. First, we created an intelligent decision-tree algorithm with 37 decision points that adaptively routes manufacturers through customized usability engineering pathways based on their capabilities and existing documentation status. Second, we implemented this decision tree within an HTML-based web application that collects essential manufacturer information including device specifications, user profiles, use environments, and company capabilities. Third, we integrated RAG-enhanced LLM-based document generation using Gemini Flash and Claude Opus APIs, where the RAG system utilized comprehensive knowledge bases containing IEC 62366-1/-2 international standards and validated usability engineering files from successful regulatory submissions to strengthen LLM performance and ensure regulatory accuracy. The system systematically transforms regulatory requirements into specialized prompts through prompt engineering techniques while leveraging RAG retrieval for contextually relevant regulatory guidance. The system incorporates automated web crawling of FDA MAUDE database for hazard identification and maintains document interconnectivity where generated documents serve as inputs for subsequent process steps. Fourth, we applied this integrated system to four South Korean medical device companies, generating comprehensive usability engineering files for artificial kidney machines, neonatal incubators, AI-based knee X-ray analysis software, and light-guided vocal cord injection devices. Finally, we initiated evaluation by five experts in medical device regulation and usability engineering through structured evaluation.
Research Results: The intelligent decision-tree development proved successful, creating an adaptive guidance system that effectively routes manufacturers through appropriate regulatory pathways. The web application successfully enables regulatory personnel to easily input medical device information and understand the usability engineering process while following systematic guidance. We confirmed the system's capability to generate comprehensive usability engineering files across diverse device categories and complexity levels. The completed documents represent draft-level quality requiring mandatory review and modification by responsible personnel, maintaining appropriate human oversight for safety-critical content. Document generation for four companies' devices was completed successfully, and these documents are currently undergoing expert evaluation to assess completeness and professional quality.
Research Significance: This work establishes a new paradigm for AI-human collaboration in complex regulatory domains, demonstrating how intelligent decision-making can be combined with automated document generation to address critical industry barriers while preserving essential safety oversight mechanisms. By extending this framework to include other pivotal standards like ISO 13485 and ISO 14971, this research paves the way for a more integrated, user-friendly, and holistic system that empowers manufacturers to navigate the entire regulatory landscape with greater efficiency and confidence, ultimately accelerating the delivery of safe and effective medical technologies to the market.
Event Type
Poster Presentation
TimeMonday, March 234:45pm - 6:15pm EDT
LocationRhinelander Gallery
Medical and Drug Delivery Devices