Presentation
Uncomplicating complicated team testing: Effective strategies for team testing with robotic assisted surgical devices
SessionRobotics Summit - Session 2
DescriptionThe use of robotic assisted surgical devices (RASDs) requires multiple users to collaborate as a team for effective system use. These users include at least one surgeon and at least one supporting staff, such as a tableside assist, and often more users. The dynamic of a team of users introduces a number of complexities when considering how a system will be used and, in turn, how these complexities can be represented effectively in HF documentation and usability testing of an RASD. This includes understanding how each user’s role might differ depending on the system, facility, and/or operating room (OR) staffing, how the experience of each individual in a team can vary, whether the team members are familiar with one another, how effectively they communicate, etc.
Accounting for these and other variables related to team dynamics can be challenging for manufacturers to navigate throughout the product development process, and they can be especially challenging when ultimately considering how to create realistic environmental and procedural simulations during usability testing for RASDs.
This poses a unique challenge with a complex system like a RASD that requires multiple collaborative users. With this challenge in mind, this presentation will describe considerations and strategies RASD manufacturers can take to develop documentation, conduct usability testing, and ultimately report on HF validation test outcomes as part of a regulatory submission. This presentation will focus on HF validation testing but the content is readily applicable to formative testing as well. In this presentation, we plan to:
• Propose how to maximize the value of early user research and testing to define intended users effectively. This early work creates a foundation to ensure comprehensive and realistic usability testing and risk assessment throughout development, including leading up to potential justifications related to HF validation and/or supplemental test strategies. Early research and testing could include, for example:
o Learning which users perform various tasks, and how to define role responsibilities across documentation
o Researching how task responsibilities vary based on facility, OR staff, and/or experience, including when certain users might perform responsibilities across traditional roles
o Determining the extent to which users typically interact with one another during procedures
o Describing tasks that are collaborative appropriately in the use-related risk analysis and other documentation
o Understanding a system’s intended users as compared to additional individuals who might be present during use, such as a manufacturer representative, and how their role should be defined and represented
• Describe how to approach the complexities of testing with multiple participants at once, including:
o Ensuring participants perform tasks within their expected responsibility during actual use
o Considering how to account for non-users and their equipment, such as anesthesiologists and their carts
o Accounting for variability in OR staffing, including teams who regularly work together versus staff that regularly rotate within facilities
o Understanding which users should be included to ensure complete evaluation of system use and critical tasks
o Outlining the logistics associated with managing team testing, including how many usability test staff may be necessary for effective conduct and observation
o Comparing debrief approaches for teams of participants, including individual break-out sessions versus full team debrief
• Advise how to approach post-testing data analysis, reporting, and supplemental testing, including:
o Documenting manufacturer representative or clinical support specialist interventions, as relevant
o Providing strategies for strong residual risk commentaries that could encompass test artifact related to role responsibilities
o In the case of supplemental HF validation testing, considering whether some users might be appropriately excluded
In discussing these considerations, we will provide case studies that describe how we have implemented these approaches with various team-based systems, with a focus on RASDs.
Accounting for these and other variables related to team dynamics can be challenging for manufacturers to navigate throughout the product development process, and they can be especially challenging when ultimately considering how to create realistic environmental and procedural simulations during usability testing for RASDs.
This poses a unique challenge with a complex system like a RASD that requires multiple collaborative users. With this challenge in mind, this presentation will describe considerations and strategies RASD manufacturers can take to develop documentation, conduct usability testing, and ultimately report on HF validation test outcomes as part of a regulatory submission. This presentation will focus on HF validation testing but the content is readily applicable to formative testing as well. In this presentation, we plan to:
• Propose how to maximize the value of early user research and testing to define intended users effectively. This early work creates a foundation to ensure comprehensive and realistic usability testing and risk assessment throughout development, including leading up to potential justifications related to HF validation and/or supplemental test strategies. Early research and testing could include, for example:
o Learning which users perform various tasks, and how to define role responsibilities across documentation
o Researching how task responsibilities vary based on facility, OR staff, and/or experience, including when certain users might perform responsibilities across traditional roles
o Determining the extent to which users typically interact with one another during procedures
o Describing tasks that are collaborative appropriately in the use-related risk analysis and other documentation
o Understanding a system’s intended users as compared to additional individuals who might be present during use, such as a manufacturer representative, and how their role should be defined and represented
• Describe how to approach the complexities of testing with multiple participants at once, including:
o Ensuring participants perform tasks within their expected responsibility during actual use
o Considering how to account for non-users and their equipment, such as anesthesiologists and their carts
o Accounting for variability in OR staffing, including teams who regularly work together versus staff that regularly rotate within facilities
o Understanding which users should be included to ensure complete evaluation of system use and critical tasks
o Outlining the logistics associated with managing team testing, including how many usability test staff may be necessary for effective conduct and observation
o Comparing debrief approaches for teams of participants, including individual break-out sessions versus full team debrief
• Advise how to approach post-testing data analysis, reporting, and supplemental testing, including:
o Documenting manufacturer representative or clinical support specialist interventions, as relevant
o Providing strategies for strong residual risk commentaries that could encompass test artifact related to role responsibilities
o In the case of supplemental HF validation testing, considering whether some users might be appropriately excluded
In discussing these considerations, we will provide case studies that describe how we have implemented these approaches with various team-based systems, with a focus on RASDs.
Authors
Event Type
TimeSunday, March 2210:30am - 11:00am EDT
LocationMurray Hill