Presentation
Quantifying Diagnostic Error: A Systems Framework Integrating Retrospective Records and Prospective Socio-Behavioral Metrics
SessionPoster Session 2
DescriptionDiagnostic error is a prevalent and significant issue in healthcare that poses a threat to patient safety and can lead to severe harm. In 2015, the National Academies of Sciences, Engineering, and Medicine (NASEM) published a report titled “Improving Diagnosis in Health Care.” The committee defined diagnostic errors as the failure to (a) establish an accurate and timely explanation of the patient's health problem(s) or (b) communicate that explanation to the patient. The definition highlights the importance of accuracy, timeliness, and communication among stakeholders in fostering a more complete understanding of diseases and diagnoses. This report showed the importance of diagnostic errors; however, there is a lack of a clear and precise framework for defining, capturing, and quantifying these errors, which limits the ability to track and mitigate diagnostic errors. Electronic Health Records (EHRs) are widely used to identify and extract diagnostic errors, primarily emphasizing disease-related factors that impact the accuracy of diagnosis. However, medical records limit exploration of socio-behavioral (patient and clinician intrinsic factors) and system-level (facility, administration, and regulatory factors) factors. EHR-extracted data, while valuable, captures only a partial view of diagnosis; key elements such as clinical reasoning, communication quality, and patient recognition of systems often remain outside the medical record. Collection of socio-behavioral and systems data is typically resource-intensive (surveys, interviews, observations, etc.), leading to exclusion of these factors in diagnostic error studies. A clear, practical human-centric framework is therefore needed to identify and quantify diagnostic error contributory factors.
We developed a framework theoretically grounded in the Systems Engineering Initiative for Patient Safety (SEIPS) model to further elucidate and quantify system factors in the diagnostic process. The SEIPS model provides a holistic and human-centered approach to examining diagnostic safety issues across six domains: Diagnostic Team Members, Tasks, Technologies and Tools, Organization, Physical Environment, and External Environment. We mapped discrete contributory factors for diagnostic error under each SEIPS element and translated them into quantifiable variables that can be extracted and measured from medical records, facilities data, and stakeholder surveys. The framework uses a mixed-method approach that includes: (1) a retrospective review of medical records to quantify SEIPS variables at triggered diagnostic events, (2) a retrospective integration of facility data to include structural and organizational characteristics, and (3) a prospective socio-behavioral component using brief surveys with patients, clinicians and administrators to capture interpersonal, cognitive, and organizational factors that are not found in medical records.
Diagnostic Team Members, including patients and clinicians, form the foundation of the diagnostic process. Retrospective records allow us to capture patient demographics, marital status, distance to the facility, presenting symptoms, and condition severity, which shape how patients enter and interact with the healthcare system. Similarly, facility data documents clinician characteristics such as age, gender, training, role, and experience, which shape their expertise and perspectives that influence how clinicians collect, interpret information, and make decisions. Prospective surveys capture aspects that aren't always reflected in records, such as patient health literacy, disease knowledge, and communication. They also reveal clinician cognitive biases such as overconfidence, anchoring, and delay discounting bias, which can influence reasoning. Together, these complementary perspectives emphasize how human factors affect diagnostic safety.
Tasks within the diagnostic process show how work is organized and executed. Retrospective review of encounter details provides information on visit length, timing, method of entry into the system, referrals, and the number of patients seen per session. These factors help us understand how workflow and workload are structured. Prospective surveys add further depth by assessing how clinicians experience sludge (i.e., administrative burden) can add task complexity, task demands, and time pressure. These combined sources highlight how the structuring of diagnostic work affects accuracy and timeliness.
Technologies and tools are critical in supporting diagnostic decision-making, with the choice of examinations, laboratory tests, and imaging reports shaping the course of care. Retrospective records show which tests were ordered and completed, demonstrating how these decisions guide the diagnostic process and affect clinical judgments.
The organization establishes the infrastructure within which diagnosis occurs. Retrospective facility data provide details on site type and location, clinician and patient volumes, scheduled session lengths, and staff working hours, which together reflect organizational capacity to support diagnostic tasks. Prospective surveys assess perceptions of organizational coordination and communication within the organization, capturing how the team-based care delivery would either facilitate or constrain diagnostic work.
The physical environment significantly shapes diagnostic interactions by influencing both efficiency and ergonomics in care delivery. Factors such as exam room size and accessibility, the condition of diagnostic equipment, and the organization of physical spaces directly affect workflow, provider comfort, and patient safety. Well-designed environments streamline movement and reduce strain, while poorly arranged or maintained spaces can slow care and compromise outcomes.
Finally, the external environment shapes diagnostic interactions through systemic constraints and opportunities. Retrospective data on insurance coverage and payer restrictions document the structural factors that determine access to tests and treatments, showing how broader policy and financial contexts influence the diagnostic process and clinical decisions.
Together, these domains provide a comprehensive, system-oriented framework that links socio-behavioral and system-level factors with data across human, organizational, and environmental conditions to diagnostic outcomes. This integrative approach enables consistent evaluation across settings and supports targeted strategies to reduce diagnostic error and improve patient safety.
We developed a framework theoretically grounded in the Systems Engineering Initiative for Patient Safety (SEIPS) model to further elucidate and quantify system factors in the diagnostic process. The SEIPS model provides a holistic and human-centered approach to examining diagnostic safety issues across six domains: Diagnostic Team Members, Tasks, Technologies and Tools, Organization, Physical Environment, and External Environment. We mapped discrete contributory factors for diagnostic error under each SEIPS element and translated them into quantifiable variables that can be extracted and measured from medical records, facilities data, and stakeholder surveys. The framework uses a mixed-method approach that includes: (1) a retrospective review of medical records to quantify SEIPS variables at triggered diagnostic events, (2) a retrospective integration of facility data to include structural and organizational characteristics, and (3) a prospective socio-behavioral component using brief surveys with patients, clinicians and administrators to capture interpersonal, cognitive, and organizational factors that are not found in medical records.
Diagnostic Team Members, including patients and clinicians, form the foundation of the diagnostic process. Retrospective records allow us to capture patient demographics, marital status, distance to the facility, presenting symptoms, and condition severity, which shape how patients enter and interact with the healthcare system. Similarly, facility data documents clinician characteristics such as age, gender, training, role, and experience, which shape their expertise and perspectives that influence how clinicians collect, interpret information, and make decisions. Prospective surveys capture aspects that aren't always reflected in records, such as patient health literacy, disease knowledge, and communication. They also reveal clinician cognitive biases such as overconfidence, anchoring, and delay discounting bias, which can influence reasoning. Together, these complementary perspectives emphasize how human factors affect diagnostic safety.
Tasks within the diagnostic process show how work is organized and executed. Retrospective review of encounter details provides information on visit length, timing, method of entry into the system, referrals, and the number of patients seen per session. These factors help us understand how workflow and workload are structured. Prospective surveys add further depth by assessing how clinicians experience sludge (i.e., administrative burden) can add task complexity, task demands, and time pressure. These combined sources highlight how the structuring of diagnostic work affects accuracy and timeliness.
Technologies and tools are critical in supporting diagnostic decision-making, with the choice of examinations, laboratory tests, and imaging reports shaping the course of care. Retrospective records show which tests were ordered and completed, demonstrating how these decisions guide the diagnostic process and affect clinical judgments.
The organization establishes the infrastructure within which diagnosis occurs. Retrospective facility data provide details on site type and location, clinician and patient volumes, scheduled session lengths, and staff working hours, which together reflect organizational capacity to support diagnostic tasks. Prospective surveys assess perceptions of organizational coordination and communication within the organization, capturing how the team-based care delivery would either facilitate or constrain diagnostic work.
The physical environment significantly shapes diagnostic interactions by influencing both efficiency and ergonomics in care delivery. Factors such as exam room size and accessibility, the condition of diagnostic equipment, and the organization of physical spaces directly affect workflow, provider comfort, and patient safety. Well-designed environments streamline movement and reduce strain, while poorly arranged or maintained spaces can slow care and compromise outcomes.
Finally, the external environment shapes diagnostic interactions through systemic constraints and opportunities. Retrospective data on insurance coverage and payer restrictions document the structural factors that determine access to tests and treatments, showing how broader policy and financial contexts influence the diagnostic process and clinical decisions.
Together, these domains provide a comprehensive, system-oriented framework that links socio-behavioral and system-level factors with data across human, organizational, and environmental conditions to diagnostic outcomes. This integrative approach enables consistent evaluation across settings and supports targeted strategies to reduce diagnostic error and improve patient safety.
Event Type
Poster Presentation
TimeTuesday, March 244:45pm - 6:15pm EDT
LocationRhinelander Gallery
Patient Safety Research and Initiatives