Presentation
Multimodal Analysis of Patient–Nurse Interactions in bedside vs Virtual Nurse during Patient Education
SessionPoster Session 2
DescriptionINTRODUCTION
The growing nursing shortage in the US and the development of new technologies have led to the widespread use of virtual nursing care models in hospital settings (Russell, 2023). Studying how patients interact with the bedside and virtual nurses during education sessions is critical to improving healthcare delivery and increasing adoption of virtual nursing. This study investigates the patterns of interaction between patients and nurses in conventional and remote settings with a focus on discharge education. we implemented a multimodal study design using eye-tracking and prosodic vocal analysis to measure the level of patient engagement and their emotional response across two delivery modalities, The objective is to compare the two modalities and identify differences between them to inform the design of virtual nursing systems that replicate the effectiveness of in-person interactions, ultimately enhancing patient satisfaction, comprehension, and emotional well-being.
BACKGROUND
Patient education is a critical component for effective condition management, especially for chronic conditions like Chronic Health Failure (Riegel et al, 2009 and Krumholz, 2013). Traditionally delivered by bedside nurses, education sessions are now increasingly facilitated through virtual platforms, where a live nurse provides instruction remotely via a screen (e.g., a television interface). While virtual nursing models offer scalability and efficiency, questions remain about how patients interact with virtual nurses to foster meaningful engagement.
This study addresses this gap by integrating eye-tracking technology to measure visual attention and prosodic vocal analysis to infer emotional states. These tools allow for an original understanding of how patients respond cognitively and emotionally to different educational formats. By comparing bedside and virtual nurse interactions, this research seeks to identify design principles that can improve virtual care delivery and ensure patient-centered outcomes.
APPROACH
The pilot study is conducted at large hospital in the southeastern US and it included 12 hospitalized patients aged 30 and older. Participants were assigned to one of two conditions: receiving education from a bedside nurse or a virtual nurse displayed on a television screen. The discharge education session was conducted in the patient’s hospital room and lasted approximately 2-6 minutes.
Data collection integrated multiple modalities: eye-tracking, and Prosodic vocal analysis. Participants wear eye-tracking glasses to capture gaze direction, fixation duration, and gaze transitions. Audio recordings of the sessions are analyzed for vocal features such as pitch, tone, and rhythm to assess emotional response. All data are anonymized and securely stored. Quantitative analysis is performed using iMotions and Minitab software.
OUTCOME
Preliminary findings suggest differences in patient engagement between bedside and virtual conditions. Bedside sessions foster more consistent eye contact, relaxed vocal tone, and expressive non-verbal cues, while in virtual sessions, engagement varied significantly. Additionally, prosodic vocal analysis revealed minimal emotional differences between the two modalities.
CONCLUSION
This research suggests that patient engagement during education sessions is influenced by delivery modality. Bedside interactions foster stronger emotional connections, while virtual models require thoughtful design to achieve similar outcomes. This pilot completion informs the development of virtual education interfaces that support patient-centered outcomes, emotional connection, and comprehension to ensure that virtual nursing complements rather than compromises the patient experience.
References:
Krumholz, H. M. (2013). Post-hospital syndrome: An acquired, transient condition of generalized risk. The New England Journal of Medicine, 368(2), 100–102. https://doi.org/10.1056/NEJMp1212324
Riegel, B., Moser, D. K., Anker, S. D., Appel, L. J., Dunbar, S. B., Grady, K. L., Gurvitz, M. Z., Havranek, E. P., Lee, C. S., Lindenfeld, J., Peterson, P. N., Pressler, S. J., Schocken, D. D., Whellan, D. J., American Heart Association Council on Cardiovascular Nursing, American Heart Association Council on Clinical Cardiology, American Heart Association Council on Nutrition, Physical Activity, and Metabolism, & American Heart Association Interdisciplinary
Council on Quality of Care and Outcomes Research. (2009). State of the science: Promoting self-care in persons with heart failure: A scientific statement from the American Heart Association. Circulation, 120(12), 1141–1163. https://doi.org/10.1161/CIRCULATIONAHA.109.192628
Russell, M. B. (2023). A hybrid virtual nurse model. Nursing Management, 54(2), 42–49. https://doi.org/10.1097/01.numa.0000918212.05937.de
Key Learning Objectives
After attending this session, the attendees will be able to:
1. Describe how multimodal tools such as eye-tracking and vocal analysis can be used to assess patient engagement and emotional response in clinical education settings.
2. Explain how delivery modality (bedside vs. virtual) influence patient interaction and comprehension during education sessions.
3. Identify design principles for virtual nursing systems that replicate the emotional and cognitive benefits of in-person education, with the objective of improving patient-centered outcomes in virtual healthcare environments.
The growing nursing shortage in the US and the development of new technologies have led to the widespread use of virtual nursing care models in hospital settings (Russell, 2023). Studying how patients interact with the bedside and virtual nurses during education sessions is critical to improving healthcare delivery and increasing adoption of virtual nursing. This study investigates the patterns of interaction between patients and nurses in conventional and remote settings with a focus on discharge education. we implemented a multimodal study design using eye-tracking and prosodic vocal analysis to measure the level of patient engagement and their emotional response across two delivery modalities, The objective is to compare the two modalities and identify differences between them to inform the design of virtual nursing systems that replicate the effectiveness of in-person interactions, ultimately enhancing patient satisfaction, comprehension, and emotional well-being.
BACKGROUND
Patient education is a critical component for effective condition management, especially for chronic conditions like Chronic Health Failure (Riegel et al, 2009 and Krumholz, 2013). Traditionally delivered by bedside nurses, education sessions are now increasingly facilitated through virtual platforms, where a live nurse provides instruction remotely via a screen (e.g., a television interface). While virtual nursing models offer scalability and efficiency, questions remain about how patients interact with virtual nurses to foster meaningful engagement.
This study addresses this gap by integrating eye-tracking technology to measure visual attention and prosodic vocal analysis to infer emotional states. These tools allow for an original understanding of how patients respond cognitively and emotionally to different educational formats. By comparing bedside and virtual nurse interactions, this research seeks to identify design principles that can improve virtual care delivery and ensure patient-centered outcomes.
APPROACH
The pilot study is conducted at large hospital in the southeastern US and it included 12 hospitalized patients aged 30 and older. Participants were assigned to one of two conditions: receiving education from a bedside nurse or a virtual nurse displayed on a television screen. The discharge education session was conducted in the patient’s hospital room and lasted approximately 2-6 minutes.
Data collection integrated multiple modalities: eye-tracking, and Prosodic vocal analysis. Participants wear eye-tracking glasses to capture gaze direction, fixation duration, and gaze transitions. Audio recordings of the sessions are analyzed for vocal features such as pitch, tone, and rhythm to assess emotional response. All data are anonymized and securely stored. Quantitative analysis is performed using iMotions and Minitab software.
OUTCOME
Preliminary findings suggest differences in patient engagement between bedside and virtual conditions. Bedside sessions foster more consistent eye contact, relaxed vocal tone, and expressive non-verbal cues, while in virtual sessions, engagement varied significantly. Additionally, prosodic vocal analysis revealed minimal emotional differences between the two modalities.
CONCLUSION
This research suggests that patient engagement during education sessions is influenced by delivery modality. Bedside interactions foster stronger emotional connections, while virtual models require thoughtful design to achieve similar outcomes. This pilot completion informs the development of virtual education interfaces that support patient-centered outcomes, emotional connection, and comprehension to ensure that virtual nursing complements rather than compromises the patient experience.
References:
Krumholz, H. M. (2013). Post-hospital syndrome: An acquired, transient condition of generalized risk. The New England Journal of Medicine, 368(2), 100–102. https://doi.org/10.1056/NEJMp1212324
Riegel, B., Moser, D. K., Anker, S. D., Appel, L. J., Dunbar, S. B., Grady, K. L., Gurvitz, M. Z., Havranek, E. P., Lee, C. S., Lindenfeld, J., Peterson, P. N., Pressler, S. J., Schocken, D. D., Whellan, D. J., American Heart Association Council on Cardiovascular Nursing, American Heart Association Council on Clinical Cardiology, American Heart Association Council on Nutrition, Physical Activity, and Metabolism, & American Heart Association Interdisciplinary
Council on Quality of Care and Outcomes Research. (2009). State of the science: Promoting self-care in persons with heart failure: A scientific statement from the American Heart Association. Circulation, 120(12), 1141–1163. https://doi.org/10.1161/CIRCULATIONAHA.109.192628
Russell, M. B. (2023). A hybrid virtual nurse model. Nursing Management, 54(2), 42–49. https://doi.org/10.1097/01.numa.0000918212.05937.de
Key Learning Objectives
After attending this session, the attendees will be able to:
1. Describe how multimodal tools such as eye-tracking and vocal analysis can be used to assess patient engagement and emotional response in clinical education settings.
2. Explain how delivery modality (bedside vs. virtual) influence patient interaction and comprehension during education sessions.
3. Identify design principles for virtual nursing systems that replicate the emotional and cognitive benefits of in-person education, with the objective of improving patient-centered outcomes in virtual healthcare environments.
Event Type
Poster Presentation
TimeTuesday, March 244:45pm - 6:15pm EDT
LocationRhinelander Gallery
Digital Health