Using Electronic Health Records to Enhance Predictions of Fall Risk in Inpatient Settings

Using Electronic Health Records to Enhance Predictions of Fall Risk in Inpatient Settings (research paper)

 

NOTE: The research paper below was published in The Joint Commission Journal on Quality and Patient Safety, Volume 46, pages 199-206.

Gil Moskowitz, Natalia N. Egorova, Ariela Hazan, Robert Freeman, David L. Reich, Rosanne M. Leipzig, Using Electronic Health Records to Enhance Predictions of Fall Risk in Inpatient Settings,
The Joint Commission Journal on Quality and Patient Safety, Volume 46, Issue 4, 2020, Pages 199-206, ISSN 1553-7250, https://doi.org/10.1016/j.jcjq.2020.01.009.
(https://www.sciencedirect.com/science/article/pii/S1553725020300325)

Background

Falls are the most common adverse events of hospitalized adults. Traditional validated assessment tools have limited ability to accurately detect patients at high risk for falls. The researchers aim to develop an automated comprehensive risk score to enhance the identification of patients at high risk for falls and examine its effectiveness.

Methods

The enhanced fall algorithm (EFA) was developed from 171,515 hospitalizations and 2,659 falls, in an academic medical center, using hierarchical logistic regression. Routine nursing assessments, labs, medications, demographics, and patients’ location during their hospitalization were gathered from the electronic health record (EHR).

Results

The fall rate was 2.8 per 1,000 patient-days. Morse fall score was the strongest predictor of falls (odds ratio = 7.16, 95% confidence interval = 6.48–7.91), with a model discrimination c-statistic of 0.687. By adding patient demographics, chronic conditions, lab values, and medications, and controlling for patient clustering within units, predication was enhanced and model discrimination increased to 0.805. By applying the enhanced model, we observed redistribution of patient by risk: low-risk group increased from 52.8% to 66.5%, and the high-risk group decreased from 28.0% to 16.2%, with an increase of fall detection from 3.1% to 5.1%.

Conclusion

The EFA redistributes and identifies patients at high risk more accurately than the Morse score alone, decreasing the population of high-risk patients without increasing the rate of falls over time. The EFA requires no addition data collection and automatically updates the patient’s fall risk based on new inputs in the EHR.

Patient falls are the most common adverse safety event in hospitals and health care facilities nationwide and have been a quality-of-care issue for decades.1,2 Falls are the most common adverse events of hospitalized adults.2., 3., 4. Inpatient falls range from 2.3 to 17.1 falls per 1,000 patient-days.5 Approximately 30%–50% of these falls result in injuries, and up to 6% of these injuries are serious, including fractures, subdural hematomas, significant bleeding, and even death.5., 6., 7. Falls without perceived harm can cause distress and anxiety to patients and those surrounding them.7 In addition, falls are a significant financial burden, and the associated expense averages greater than $30,000 per event, due to increased length of stay, treatment costs, and litigation.7,8

Hospitals identify patients at high risk for falls using validated nursing assessments9; one of the most popular is the Morse Fall Scale (MFS).10,11 However, these assessments have relatively high false positive rates and are not very accurate when used in isolation.2,12 Traditional validated fall risk assessment tools have limited accuracy, resulting in a larger cohort of ostensibly high-risk patients and leading to overuse of expensive resources.2 In recent years The Joint Commission has updated its guidelines to include other dimensions of patient treatment and status to current nursing assessments. These changes help create a comprehensive individualized risk assessment that will capture many potential intrinsic and extrinsic factors that have not been previously accounted for.9,13

In this study, we developed an automated comprehensive risk score to enhance the identification of patients at high risk for falls and examine its effectiveness.

Methods

Data Sources

We analyzed a total of 171,515 hospitalizations in a large urban academic medical center from January 1, 2012, to September 30, 2015. Only falls that occurred in inpatient units were included in the study. All psychiatric hospitalizations or hospitalizations of patients who were aged younger than 18 years at the time of admission were excluded.

Falls were identified using a risk management event software package designed to capture all hospital adverse events. Any medical personnel, including nurses and physicians, could enter events into the system. All falls, with or without injury, were included in this analysis, as well as falls in which patients were assisted to the floor (so called near misses) (definitions of types of falls can be found in Supplementary Figure 1, available in online article). All other data, including patient demographics, comorbid conditions, history of prior hospitalizations, height, weight, body mass index (BMI), medications (with time stamps), laboratory test values (with time stamps), nursing assessments, and hospital service, were acquired from the hospital’s data warehouse. The data warehouse is an internal system that extracts data nightly from the electronic health record (EHR) employed by the hospital (Epic Systems Corporation, Verona, Wisconsin), which includes more than 4,000 data points per patient.

For all hospitalizations, we included BMI, labs, and medications (administered in the emergency department or during hospitalization) from the EHR captured up to 12 hours prior to admissions and throughout the hospital stay. For patients without falls, the first record of labs, medications, and nursing assessments, if any, were captured. If a patient’s EHR did not have these inputs, values were assumed to be normal. If a patient had fallen during the hospitalization, data were collected within 24 hours prior to the event. The MFS is used at this medical center to determine each patient’s risk for falling. The MFS consists of the following: history of falling, presence of a second diagnosis, use of ambulatory aid, intravenous therapy/heparin lock, report of gait (normal/bedrest/wheelchair, weak, impaired), and mental status (oriented to own ability, overestimates or forgets limitations).10,11 Each component is scored daily, and the aggregate score determines the risk, with low-risk scores between 0 and 24, medium-risk scores between 25 and 44, and high-risk scores of 45 or greater.

Statistical Analysis

Enhanced Fall Algorithm (EFA) Model: Development and Validation

We assessed the association of falls with the following data: age; gender; race; ethnicity; BMI; chronic conditions as identified with Centers for Medicare & Medicaid Services data warehouse algorithms14; fall risk medications: antihistamines, antiepileptics, antipsychotics, antidepressants, and antianxiety (medications listed in Supplementary Table 1); laboratory test values: albumin (Alb), sodium (Na), chloride (Cl), glucose, aspartate aminotransferase (AST), white blood cell count (WBC), red blood cell count (RBC), potassium (K), and calcium (Ca); and nursing assessments: MFS, sleeping pattern, Braden assessments of friction and sheer, nutrition, activity, moisture, sensory perception, mobility, and admission to a hospital within 30 days prior to hospitalization.

The initial step in developing the algorithm was to evaluate the associated factors for falls using Pearson’s chi square test for categorical variables and a t-test for continuous variables. In addition to these tests, standardized difference was calculated to eliminate the cohort’s large sample size effect on p values. All parameters with a standardized difference of ≥ 0.1 were included in the multivariable model, and the final model included only statistically significant parameters (p < 0.05). A model was developed on data collected from January 1, 2012, through December 31, 2014 (137,627 hospitalizations and 2,161 falls, 2.8 falls per 1,000 patient-days), and validated on data collected from January 1 through September 30, 2015, at an academic medical center (33,888 hospitalizations and 498 falls, 2.2 fall per 1,000 patient-days).

We used a mixed-effect logistic regression with a random hospital service–specific intercept to identify risk factors associated with falls. The outcome was a fall, and predictors included factors associated with falls (listed above). The interclass correlation was 43.4% in the empty model, which decreased to 21.0% after the adjustment for covariates. The performance of the model was assessed using model discrimination (c-statistic) and model calibration. The model was recalibrated using all data from Jan 1, 2012, through September 30, 2015.

To compare the performance of the EFA model and the model based on Morse score only, we calculated predictive probabilities for each observation using EFA and Morse models, constructed and overlaid receiver operating characteristic curves, and compared the difference between the areas under the curves using the ROCCONTRAST statement.

Risk Score Development

Risk scores for falls were developed using the final model’s coefficients. The smallest coefficient was assigned a score of 1. The score was calculated for each covariate by dividing the coefficient of that covariate by the coefficient that was selected as score 1 and then rounding to the nearest integer. The total risk score for a patient is the sum of all risk scores assigned for each covariate plus the score of the intercept corresponding to the hospital service.

We review the performance of EFA and Morse models in the wide range of thresholds and discussed the cutoff values with our hospital clinical experts. Median risk of fall cutoff values was established at the level of prevalence for falls in the institution, while the Youden’s Index identified the cutoff values for high risk of falls (at the probability where the EFA model performance was maximized). Decision curves then compared the net benefit between the Morse and EFA models for different thresholds.15,16 The EFA model outperformed the Morse model within the wide range of probabilities (Supplementary Figure 2).

Distribution of patients by risk of fall to the high-, medium-, and low-risk groups, based on Morse risk categories and EFA score risk-adjusted for patient characteristics (medication, labs, demographics, and nursing assessments) categories were compared using the chi square test.

Model Implementation

The EFA was rolled out in the EHR starting the second quarter of 2016. During the patient intake process, nurses and or hospital care workers collect routine information and input the data into the EHR. When data fields are completed, the patients risk score is calculated automatically. When high risk is identified, the score is prominently displayed on the patient’s chart summary page in the EHR. Additional information regarding risk factors that led to the high risk score can be quickly observed by hovering over the banner (Supplementary Figure 3). After fall risk is assessed, further precautions and hospital protocols can be implemented, such as direct patient observation, depending on the severity of risk. The risk score updates automatically as new data become available.

This study was approved by the Program for the Protection of Human Subjects at the Icahn School of Medicine at Mount Sinai (See https://icahn.mssm.edu/research/pphs.). The approval included a waiver of informed consent.

All statistical analyses were performed using SAS 9.4 (SAS Institute Inc., Cary, North Carolina).

Results

Study Population

Falls were observed in 1.6% of the development cohort’s 137,627 hospitalizations. The fall rate was 2.8 per 1,000 patient-days. Of the patients who fell, 88.0% had only one event per hospitalization, 9.8% had two events per hospitalization, and 2.2% had three or more events per hospitalization. The median length of stay was 12 days (interquartile range [IQR] 6–23 days) for patients with fall(s) and 3 days (IQR 2–6 days) for patients without falls. The majority of the falls occurred on the third day of hospitalization (IQR 1–8 days).

Patients with falls were older than patients without falls (mean age 64 vs. 56, p < 0.001), more likely to be male (52.2% vs. 42.1%, p < 0.001), and had higher rates of chronic conditions, including anemia, chronic kidney disease, chronic obstructive pulmonary disease, diabetes, heart failure, ischemic heart disease, hypertension, rheumatoid arthritis, history of stroke or transient ischemic attack, and depression (Table 1). Patients with a prior 30-day hospitalization were more likely to fall.

Table 1. Development Cohort: Selected Characteristics of Patient With and Without Falls

Factor	Nonfallers n = 135,466 (Mean or %)	Fallers n = 2,161 (Mean or %)	P Value	Standardized Difference
Age, mean (SD)	56 (19)	64 (16)	< 0.001	0.44
Age groups			< 0.001
18–40	26.5	10.2		0.47
41–64	35.8	39.1		0.08
65–79	25.1	31.8		0.16
80+	12.6	18.9		0.18
Male	42.1	52.2	< 0.001	0.20
Race			0.002	0.10
White	46.8	43.2
Black	19.6	23.2
Asian	5.0	4.9
Native Americans	0.2	0.3
Hispanics	27.2	27.0
Other Race	1.2	1.3
Chronic Conditions
Alzheimer’s or Senile Dementia	3.3	6.9	< 0.001	0.16
Anemia	28.5	45.7	< 0.001	0.36
Benign Prostatic Hyperplasia	6.7	10.4	< 0.001	0.13
Chronic Kidney Disease	23.9	41.9	< 0.001	0.39
COPD	7.1	11	< 0.001	0.14
Depression	10.7	22.4	< 0.001	0.32
Diabetes	25.6	34.2	< 0.001	0.19
Heart Failure	15.7	22.9	< 0.001	0.18
Prior Hip/Pelvic Fracture	0.9	2.3	< 0.001	0.11
Hypertension	50.6	66.3	< 0.001	0.33
Hypothyroidism	9.7	12.1	< 0.001	0.08
Ischemic Heart Disease	26.2	31	< 0.001	0.11
Osteoporosis	3.2	4.6	< 0.001	0.07
RA/OA (Rheumatoid Arthritis/Osteoarthritis)	11.5	16.4	< 0.001	0.14
Stroke / Transient Ischemic Attack	2.8	8.7	< 0.001	0.26
Hospitalizations in Prior 30-Days	18	33	< 0.001	0.34
Length of stay, days, median (IQR)	3 (2–6)	12 (6–23)	< 0.001	1.5

COPD, chronic obstructive pulmonary disease; IQR, interquartile range.

Antiepileptic, antianxiety, antihistamine, antipsychotic, and antidepression medication unadjusted rates of use were higher in patients with falls opposed to no falls (Supplementary Table 2). Moreover, abnormal laboratory values and nursing assessments (restless sleep, inadequate nutrition, and friction/shear potential problems) were higher in patients with falls vs. no falls (Supplementary Table 2).

Morse Fall Scores

According to the MFS, patients were divided into three fall risk categories: 52.8% in low (n = 72,646), 19.2% in medium (n = 26,390), and 28.0% in high (n = 38,591). Falls occurred in 0.3% of the low-risk patients, 0.9% of the medium-risk patients, and 3.3% of high-risk patients. The model based on Morse fall score yielded a c-statistic of 0.687 in the derivation cohorts. The Morse fall score diagnostic characteristics (Figure 1) showed a sensitivity of 58.3%, specificity of 72.4%, positive predictive value of 3.3%, and negative predictive value of 99.1%. The accuracy of the test was 72.2%.

Enhanced Fall Algorithm Model

The EFA model incorporates four groups of data: nursing assessments (Morse score, Braden assessment, restless sleep, male gender, and readmissions with 30 days prior), medications (antipsychotic, antiepileptic, antidepressant, antianxiety, and antihistamine drugs), lab values collected (Ca > 10.1, AST [31–40, 41–60, 61–100, > 100], Na < 130, glucose > 165, WBC < 4.3, Cl < 96, Alb [< 2.4 and 2.5–2.7], and RBC < 3.7), and hospital service (Figure 2).

The derivation model for EFA yielded a c-statistic (model discrimination) of 0.805 vs. 0.687 for the model based on Morse fall score only (p < 0.001). A model calibration had a p value > 0.05. Validation of the EFA was done from January 1 through September 30, 2015. No change was observed in the significance of the variables used (a comparison of the two populations can be found in Supplementary Table 3). The correlation between predicted and observed falls was 0.71. We stratified patients by risk of falls into 10 categories. There were no statistical differences between observed and predicted number of falls using these categories; model calibration (Hosmer-Lemeshow test) had p > 0.05 (Figure 3). The model discrimination yielded a c-statistic of 0.836 for EFA vs. 0.668 for the model that included only Morse score (p < 0.001) in the validation data set.

Each component has an assigned score and the total was calculated by the sum of these values. Figure 2 and Supplementary Table 4 display the risk scores developed for the EFA. Patients assessed with a low Morse score received 0 points, patients with a medium Morse score received 23 points, patients with a high Morse score received 53 points. Identification within the Braden assessment of friction/sheer potential problems (16 points) and inadequate nutrition (19 points), as well as restless sleep (25 points), increased the risk for fall. All abnormal laboratory values increased the risk for a fall, several of which increased the risk score by ≥ 10 points: Alb (< 2.4 and 2.5–2.7), Ca > 10.1, AST (31–40, 41–60, 61–100, > 100), glucose > 165, Na < 130, WBC < 4.3, RBC < 3.7, and Cl < 96. AST >100 had the largest score of all lab values, with 28 points. Use of medications affecting the central nervous system significantly increased fall risk: antihistamine (13 points), antianxiety (18 points), antidepression (25 points), antiepileptic (33 points), and antipsychotic (52 points). Being completely immobile, a Braden score component, was the only patient characteristic to decrease falls (-55 points). For hospital services, patients on Obstetrics were least likely to fall (-79 points), while those on Rehabilitation were most likely (86 points).

Cutoff values were determined for the EFA. Patients who scored 64 or less had a low risk for fall, scores between 65 and 119 were medium risk, and scores of 120 or higher were at high risk. When the EFA was used to classify fall risk compared to the MFS, the low risk group increased from 52.8% to 66.5%, medium risk decreased from 19.2% to 17.4%, and high risk decreased from 28.0% to 16.2% (Figure 4). The percentage of falls, using the EFA, decreased in the low risk population (from 0.6% to 0.5%) and increased in the medium and high risk populations (from 1.7% to 2.3% and from 3.3% to 5.1%, respectively, p < 0.001). The EFA vs. traditional Morse showed an increase in accuracy from 72.2% to 84.0% (Figure 1).

The EFA was implemented in seven nursing units between the second quarter of 2016 and the last quarter of 2018. Prior to the transition, these units had a rate of falls of 3.79 per 1,000 patient-days, and their population was responsible for 14.7% of all patient-days in the hospital. Post-transition to the EFA, the rate of falls per 1,000 patient-days decreased to 3.04 while the population increased, responsible for 16.9% of all patient-days in the hospital.

Discussion

The EFA more accurately identifies patients who are at higher risk for falls. By adding patient demographics, chronic conditions, lab values, and medications, and controlling for clustering within units, we were able to improve the discrimination and the accuracy of risk detection, projecting a greater efficiency of nursing resources with the decrease of a high-fall-risk population, as well as a potential decrease of fall-related incidents.

Without the use of the EFA, Morse identified 28.0% as high risk for falls, with falls occurring in 3.3% of this population. The EFA identified 16.2% of patients at high risk for falls, with falls occurring in 5.1% of this population. With the EFA more accurately characterizing less of the population as high risk, the use of costly preventive fall resources can decrease. We observed that after the implementation of the EFA, the rate of falls decreased. This decrease cannot be solely attributed to the EFA. Multiple nursing protocols had been rolled out during this time period and may have also contributed to the decreasing rates.

Prior publications compared fall risk assessments and their model discriminations.9,12 No single stand-alone inpatient assessment tool, in the reviewed articles, had a sensitivity and specificity above 70%.12 Although close, this holds true for the present study as well. As reported earlier, the EFA yielded a sensitivity and specificity of 52.0% and 84.4%, respectively.

In addition to Morse, the EFA includes risk factors that have been identified in previous publications,12,17,18 such as antidepressant and antiepileptic medications and abnormal laboratory values that are significantly associated with falls. Awareness of these factors allows the provider to potentially modify fall risk; for example, by changing medication choice or dosage.19 When the provider cannot alter the patient’s risk factors, a fall prevention protocol may be applied. In agreement with a previous study, age alone was not a significant risk factor for falls.20

A number of studies have incorporated the use of EHRs to create or enhance a predictive fall risk model.21., 22., 23. Hong et al. used a predictive model that included similar risk factors, such as hyponatremia and unit of care. However, this article did not include falls that occurred under the supervision of medical professionals other than nurses. Yokota et al. adopted a model that relies on daily data records from the EHR to determine whether a patient will fall on a given day.23 The model included intensity of nursing care needs for a patient (for example, frequencies of blood pressure measurements, needs of wound care, able to sit up in bed, sustain a sitting position), clinical department, and day of the week. The goal of the model was to predict falls and to reduce the burden of additional assessments. Complementary uses of EHRs have been adapted to alert health care providers of adverse events. Another study examined the doses of high-risk medication given to a patient within 24 hours prior to a fall and changed the default dosage and regimens recommendations in EHRs. Analogous alerts have been incorporated in the EFA via Epic.19

The EHR has been used to validate and determine the most appropriate fall risk score based on the population it services. This is visible in a South Korean study in which researchers determined that the MFS, for their patient population, has the best predictive performance when cutoff values are increased to 51 (originally 45) for high-risk patients.24 For this study’s population we have found that the EFA has the highest predictive performance when the cutoff values for the high risk is 120 (Figure 2).

It should be noted that although the EFA is a validated predictive falls model, it can only identify those who are at high risk for falls; it cannot prevent falls. The proper training and protocols need to be in place for the EFA to work as intended.

This study is not without limitations. The EFA is based on a patient population from one academic medical center. The accuracy of the model is limited by human error in data collection. Although standardized, nursing assessments can differ depending on the interpretation of the assessor. Therefore necessary training is needed to create uniformed assessments to ensure proper data capturing. Nevertheless, even with all risk factors correctly identified and a highly trained and competent staff, it is imperative to note that falls can occur by happenstance and unexpectedly, regardless of patient risk. Precautions should always be taken to prevent falls in the hospital.

Conclusion

The enhanced fall algorithm improves the accuracy of identification of patients at risk for falls compared with use of the Morse Fall Scale alone, labeling fewer patients as high risk but capturing more of those who actually fall during hospitalization. No increase in falls was seen over time. Using the EFA, nurses will be able to concentrate risk reduction efforts on those individuals most likely to fall. Routinely collected information is automatically entered from the EHR and constantly updated, identifying and flagging patients as high risk as their condition changes throughout their hospitalization. There is no further burden of data collection for nurses or other hospital employees. Protocols that are already in use can be used with the EFA, allowing for a seamless transition from Morse fall assessment. We certify that this work is novel.

Acknowledgments

The authors would like to thank the nurses of Mount Sinai Hospital who supported this study and used the enhanced fall algorithm for identification of high-risk patients for falls. They played a crucial role for the data collection of this study and the safety of the patients.

Conflicts of Interest

All authors report no conflicts of interest.