Presentation
‘Frequency-Focused’ Thinking in Qualitative Human Factors Research: Case Studies and Analysis
SessionPoster Session 1
DescriptionAs human factors (HF) practitioners, we often hear quantitative comments from medical device developers after they review the results of a study. For example:
“There were so many use events on this task. That suggests a problem for ‘passing’ validation.”
“The use error only happened twice during the study. We should highlight this in the report to show how unlikely it is to happen in real life.”
The methods generally employed in HF studies in the healthcare space were established to identify the unique usability problems associated with a product in addition to the distinct themes that are associated with their possible root causes. The statistical methods and assumptions used in qualitative research are distinct from those that characterize hypothesis testing, which involves the attempt to extrapolate data to the general population. Therefore, providing statistical evidence about the frequency of use events and the significance of how often a use event occurs are not what these qualitative research methods are designed to do.
Despite this, the frequency of use events observed in an HF study is commonly thought to be a crucial element of the results. Case studies will be included in this poster that provide examples of ‘frequency-focused’ thinking. This project will also examine relationships between frequency and perceived importance of a use event.
The issue of ‘frequency-focused’ (as opposed to ‘harms-focused’) thinking is often problematic as it may distort how HF study results are interpreted. Misguided ‘frequency focused’ analyses may have a negative impact on product development and decision-making about appropriate risk-mitigation measures for medical devices. The poster would reinforce the importance of focusing on harm when thinking about qualitative HF research.
References
International Electrotechnique Commission. Medical devices—part 2: guidance on the application of usability engineering to medical devices. Geneva: International Electrotechnique Commission; 2016. 94p. Report No.: IEC. TR 62366-2, 2016
Alroobaea, Roobaea, and Pam J. Mayhew. "How many participants are really enough for usability studies?." Science and information conference. IEEE, 2014.
Hennink, Monique, and Bonnie N. Kaiser. "Sample sizes for saturation in qualitative research: A systematic review of empirical tests." Social science & medicine 292 (2022): 114523.
“There were so many use events on this task. That suggests a problem for ‘passing’ validation.”
“The use error only happened twice during the study. We should highlight this in the report to show how unlikely it is to happen in real life.”
The methods generally employed in HF studies in the healthcare space were established to identify the unique usability problems associated with a product in addition to the distinct themes that are associated with their possible root causes. The statistical methods and assumptions used in qualitative research are distinct from those that characterize hypothesis testing, which involves the attempt to extrapolate data to the general population. Therefore, providing statistical evidence about the frequency of use events and the significance of how often a use event occurs are not what these qualitative research methods are designed to do.
Despite this, the frequency of use events observed in an HF study is commonly thought to be a crucial element of the results. Case studies will be included in this poster that provide examples of ‘frequency-focused’ thinking. This project will also examine relationships between frequency and perceived importance of a use event.
The issue of ‘frequency-focused’ (as opposed to ‘harms-focused’) thinking is often problematic as it may distort how HF study results are interpreted. Misguided ‘frequency focused’ analyses may have a negative impact on product development and decision-making about appropriate risk-mitigation measures for medical devices. The poster would reinforce the importance of focusing on harm when thinking about qualitative HF research.
References
International Electrotechnique Commission. Medical devices—part 2: guidance on the application of usability engineering to medical devices. Geneva: International Electrotechnique Commission; 2016. 94p. Report No.: IEC. TR 62366-2, 2016
Alroobaea, Roobaea, and Pam J. Mayhew. "How many participants are really enough for usability studies?." Science and information conference. IEEE, 2014.
Hennink, Monique, and Bonnie N. Kaiser. "Sample sizes for saturation in qualitative research: A systematic review of empirical tests." Social science & medicine 292 (2022): 114523.
Event Type
Poster Presentation
TimeMonday, March 234:45pm - 6:15pm EDT
LocationRhinelander Gallery
Medical and Drug Delivery Devices