Predictive Analysis of Clinical Status Assessment of Critical Patients Using Electronic Early Warning System Records with Machine Learning

Suhartini Ismail1*, Zahrotul Wardah2, Adi Wibowo3, Satriya Pranata4, Sri Wahyuni5

1Department of Nursing, Unversitas Diponegoro, SH Tembalang, Semarang 1269, Indonesia

2Intensive Care Unit, dr Kanujoso Djatiwibowo General Hospital, Kota Balikpapan, Kalimantan Timur 76115, Indonesia

3Department of Computer Science, Universitas Diponegoro, SH Tembalang, Semarang 1269, Indonesia

4Faculty of Nursing and Health Sciences, Universitas Muhammadiyah, Kota Semarang, Jawa Tengah 50273, Indonesia

5Department of Nursing, Universitas YPIB Majalengka, Kabupaten Majalengka, Jawa Barat 45411, Indonesia

*Corresponding Author’s Email: suhartini.ismail@fk.undip.ac.id

ABSTRACT

Background: Rapid and accurate assessment of a patient’s clinical status in the Emergency Room (ER) is essential for timely intervention and improved outcomes. With advancements in information technology, electronic health records such as the Electronic Early Warning Score (E-EWS) have become invaluable tools in monitoring vital signs and detecting early signs of clinical deterioration. Leveraging machine learning techniques to analyse E-EWS data presents a promising approach to predict critical events including sepsis, acute respiratory distress syndrome (ARDS), cardiac arrest, and mortality. This study focuses on the application of machine learning algorithms to predict patients’ clinical status based on E-EWS records, aiming to enhance early detection and support clinical decision-making in critical care settings. Methods: The research design uses cross-sectional analysis to analyse E-EWS records with machine learning using random forest regression and random forest classification, with a total of 206 respondents, by carrying out six observations at a period of 6 hours, 12 hours, 18 hours, 24 hours, 48 hours to 72 hours. Results: The results showed that the prediction accuracy of the E-EWS record score using machine learning reached 82.26% with an MAE (mean absolute error) of 0.22, in the prediction accuracy of the patient's clinical status in 48 hours (76.19%) and 72 hours (71.43%) and the results of the accuracy of predicting hospital discharge status, the accuracy of E-EWS records and machine learning reached 97.62% with MAE 0.02 indicating that E-EWS records with machine learning with random forest algorithms have the potential to predict patient clinical status and outcomes. Conclusion: E-EWS records based on machine learning can be used to predict future patient conditions using seven EWS parameters that can predict critical patient clinical status assessment.

Keywords: Clinical Status; Critical Patients; E-EWS Record; Machine Learning; Random Forest

INTRODUCTION

Critical patients require timely and appropriate treatment during hospital care. Nurses play a vital role in managing these patients, necessitating proper planning and the application of critical thinking skills to deliver nursing care in accordance with professional standards grounded in scientific evidence. This includes the nursing process stages of assessment, diagnosis, and the effective selection of nursing interventions (DeLaune & Ladner, 2002). One tool currently being developed and implemented in treatment areas for monitoring vital signs is the Early Warning Score (EWS) (Connor, McArthur & Camargo Plazas, 2021). The EWS system is gradually transitioning to an automated electronic platform. Website and network technologies, coupled with internet connectivity, facilitate the expansion of EWS automation for hospital use (Bell et al., 2019). The adoption of websites and Android technology has demonstrated ease of use and effectiveness in addressing challenges related to EWS monitoring for patients in the Emergency Room. The Electronic Early Warning Score Record (E-EWS record) represents an advancement of the EWS system, leveraging website and Android-based technology to enhance the nursing care process. This electronic system enables remote interaction for documenting EWS parameter monitoring results, creating integrated services that simplify information provision and access among healthcare professionals (Gerry et al., 2020).

Information technology with computerised medical decision systems has become an effective method to assist health workers in diagnosing patients quickly and accurately. This computerised system makes predictions or diagnoses on new cases by utilising records and features available from previously known EWS data and can be considered a classification task called machine learning (Mohapatra, Chakravarty & Dash, 2015). Machine learning models can be used to identify and predict diagnoses accurately by analyzing the patient's vital signs within the EWS parameters in determining the clinical status of critical patients (Pirneskoski et al., 2020). The importance of using machine learning algorithms related to EWS scores that nurses use in determining prognosis and determining patient conditions to predict early clinical change status will help reduce further clinical damage and more serious complications in critical patients (Arnold et al., 2019).

Previous research has shown that machine learning models compared with conventional stratification tools based on vital signs results can determine the incidence of sepsis and ARDS in critical patients (Chiew et al., 2019). Machine Learning can help improve the accuracy and quality of triage predictions in determining the clinical status of patients in the Emergency Department. In previous research studies, machine learning technique predictions were better than conventional triage and EWS, by combining the addition of variables outside the Vital Sign parameters in the EWS such as blood sugar. because it has been proven to improve mortality predictions (Pirneskoski et al., 2020).

Research by Wu et al. (2021a) on the implementation of an Electronic National Early Warning System (E-NEWS) highlights its effectiveness as a communication platform among healthcare team members. Their findings demonstrated that E-NEWS significantly reduced adverse events, particularly in relation to cardiopulmonary resuscitation (CPR) on the ward and transfers from the general ward to the ICU. This study aims to analyse the prediction of E-EWS record scores using machine learning to determine patients’ clinical status throughout their hospital stay until discharge after treatment.

METHODOLOGY

Research Design

Quantitative research with a cross-sectional design. Data was taken in August 2023 and October 2023 with a total of 206 samples.

Population and Sample

The population in this study were all patients who came to the emergency room at Dr Kanujoso Djatiwibowo Balikpapan Hospital, East Kalimantan, Indonesia in August and October 2023, a total of 7101.

Sampling used purposive sampling technique with inclusion criteria are: 1. Adult patients who are admitted through the emergency room, aged more than 18 years. 2. Patients with triage priority 1 or priority 2, Exclusion criteria are: 1. Female patients who are pregnant and giving birth, 2. Patients who are discharged from the hospital within ≤ 72 hours and the total sample is 206 samples.

Study Implementation

Application of inclusion and exclusion criteria taken from emergency room census data in August and October, at the stage of excluding patients aged < 18 years and in maternity cases, as well as patients with non-emergency and non-hospitalised conditions, as well as patients who died when in the ER and patients who refuse treatment/discharge against medical advice (DAMA). Then data collection was limited to 4 patients per day to maintain quality in patient observations, observations were carried out six times. Finally, the researchers excluded the set of observations recorded after the event of patient death and discharge before 72 hours.

Patients who meet the inclusion and exclusion criteria are observed for vital signs adjusted to the EWS parameters (respiration rate, systolic blood pressure, pulse, SpO2, temperature, level of consciousness, oxygen use). The first 6 hours of observation were carried out in the ER using the electronic EWS record application, then observations were carried out at 12 hours, 18 hours, 24 hours, 48 hours and 72 hours and continued in the treatment room both when the respondent entered the ICU or was hospitalized for surgery and non-surgical treatment. Data collection is input into Android in the E-EWS record application and entered on the website. Patients with incomplete records and not meeting the inclusion criteria were excluded from the study.

Data Collection Tools

E-EWS Record

Table 1: Physiological Parameters for Early Warning Score Assessment (William, 2019)

EWS score: 0-4: Mild, Score: 5-6: moderate Score ≥7: Severe

In the EWS system, there are 7 vital sign variables used. This includes heart rate, respiratory frequency, body temperature, systolic blood pressure, oxygen saturation level in the blood, and level of consciousness which can be measured by a score of responsiveness to stimuli such as sound, pain, or unresponsiveness, as well as whether the patient is receiving oxygen supplements (Table 1).

Machine Learning

The random forest model is a machine-learning technique that uses many decision trees to produce classification or regression predictions. This algorithm achieves a high level of accuracy because each decision tree carries out classification or regression independently and then the results are combined through voting to produce optimal final results (Breiman, 2001). In this research, the Random Forest algorithm was implemented using the Scikit Learn library in the Python program. The programming used is Python because it is known to have many libraries (Pedregosa et al., 2011). This research was carried out with Google Collaboratory and the libraries used in this research include NumPy, Pandas, and Scikit-Learn.

Ethical Consideration

The research obtained ethical exemption from the Health Research ICS Committee, Dr Kanujoso Djatiwibowo Balikpapan Hospital, Indonesia, with reference number No.06/VII/KEPK-RSKD/2023 on 18th July 2023.

RESULTS

The collected data, comprising 206 patient records with six observation points each, were analysed using Python programming. Frequency distribution analysis was conducted to examine demographic variables such as age, gender, and hospital discharge status. A machine learning approach employing random forest regression was utilised to predict EWS outcomes at multiple intervals—specifically at 6, 12, 18, and 24 hours—with the goal of forecasting the patient’s clinical condition at the 48-hour mark. Further predictions were made for the 72-hour condition and the likelihood of outcomes such as outpatient discharge or mortality. The six EWS observation points adhered to internationally recognised standards for clinical and physiological monitoring in the management of critically ill patients. The performance of the predictive models is illustrated through graphs, accuracy percentages, and MAE values.

Table 2: Frequency Distribution of Respondent Characteristics by Gender, Age, Health Problems, and Clinical Status at Hospital Discharge (n = 206)

Table 2 presents the demographic and clinical characteristics of 206 respondents, including gender distribution, age groups, types of health problems, and clinical status upon hospital discharge. The majority of respondents were aged between 37–74 years, with cardiovascular and innervation-related issues being the most common health problems. Most patients were discharged under outpatient control, while a smaller proportion died during hospitalisation.

Data analysed from 206 patients found 106 (51.45%) male and 100 (48.54%) female. The age in this study was mostly 56-76 years (42.23%). The most common health problems were neurological disorders, with 60 patients (29.12%), while 182 patients (88.32%) had improved clinical status and 24 patients had died (11.65%).

The predictive accuracy of the E-EWS score using the machine learning model reached 82.26%, with a Mean Absolute Error (MAE) of 0.22—indicating a small deviation between the recorded E-EWS scores and those predicted by the random forest regression model. This performance was evaluated using 80% training and 20% test data from a total of 206 records, each with six observations (random state = 0). The comparison between predicted and actual EWS values is illustrated in Figure 1.

Figure 1: Performance of the Random Forest Regression Model in Predicting E-EWS Scores

The accuracy of the E-EWS score prediction and machine learning results for the 48-hour prediction reached 76.19% with an MAE value of 0.33 (the difference between the E-EWS record score and the random forest prediction EWS score). The performance of the random forest model in predicting the EWS score, how much the predicted line is compared to the actual line, is shown in Figure 2.

Figure 2: E-EWS Record Score Classification Results to Predict Patient Clinical Status at 48 Hours

The results of the accuracy of the E-EWS score prediction and machine learning on the 72-hour prediction reached 71.43% with an MAE value of 0.47 (The difference between the E-EWS record score and the Random Forest prediction EWS score). The performance of the random forest model in predicting the EWS score of how much the predicted line compares with the actual line is shown in Figure 3.

Figure 3: E-EWS Record Score Classification Results to Predict Patient Clinical Status at 72 Hours

The results of the accuracy of the E-EWS score prediction and machine learning in predicting hospital discharge reached 97.62% with an MAE value of 0.02 (the difference between the E-EWS record score and the random forest prediction EWS score). The performance of the random forest model in predicting the EWS score, how much the predicted line compares with the actual line, is shown in Figure 4.

Figure 4: E-EWS Record Score Classification Results to Predict Patient Discharge Status from Hospital

DISCUSSION

This study indicates that most patients admitted to the emergency room under Priority 1 and Priority 2 categories are males aged between 56 and 76 years, primarily presenting with neurological conditions. Clinical outcomes at the time of hospital discharge and during outpatient follow-up revealed that the average age of critically ill patients was around 64 years. Notably, age began to significantly influence clinical severity from 40 years onward, peaking between the ages of 75 and 80 (Mahmoodpoor et al., 2022). These findings are consistent with those of Ljunggren et al. (2016), who reported that increasing age is strongly correlated with a higher risk of both one-day and 30-day mortality, and a reduced likelihood of ICU admission for patients over 80 years old.

Another respondent characteristic that plays an important role in the patient's clinical condition is the health problems that the patient is currently experiencing when they enter the emergency room. The health problems in the emergency room in the research study that were the most common were 60 respondents (29.12%), the results of this research were supported by data that the prevalence of non- communicable diseases had increased, including stroke which increased from 7% to 10.9%, chronic kidney disease. which increased from 2% to 3.8%, diabetes mellitus which increased from 6.9% to 8.5%, and hypertension which increased from 22.8% to 34.1%. (Global Health Data Exchanged, 2018).

The prediction results from the random forest regression model, namely prediction accuracy reaching 82.26%, show that the machine learning model can predict the patient's EWS score based on the features entered into the model. The higher the prediction accuracy, the better the algorithm model is in getting patterns and variability in the training data and test data in predicting. The highest recorded EWS score in the dataset was 13, indicating critical conditions (EWS ≥ 7), while the lowest score was 0, corresponding to non-critical or emergency conditions (EWS 0–4), which can be predicted by machine learning models. Previous research explains that vital sign values in EWS score results can be analyzed efficiently by machine learning using algorithms to predict heart attacks in hospitals and the incidence of admission to the Intensive Care Unit (ICU) in unexpected critical conditions (Cho et al., 2020). The difference in predictions between E-EWS record scores and EWS machine learning predictions (17.74%) is due to E-EWS records filling in scores by health workers directly based on the measured parameters so that it can vary between health workers in assessing and giving EWS scores to patients. In the context of previous research, reports regarding the Early Warning Score (EWS) show varied and varied results (Stenhouse et al., 2000).

Health workers in filling out E-EWS are influenced by experience, knowledge, subjectivity and errors in inputting E-EWS record parameters (Bach & Wenz, 2020). Meanwhile, machine learning utilises computerization which uses algorithms to learn patterns from patient data that is larger and more complex to provide more consistent predictions based on structured data, this research is in line with research by Sepideh Jahandideh et al. that machine learning models have been applied to automate the process of identifying patient deterioration in the context of health services to verify that the model machine learning is effective and can predict future clinical conditions (Jahandideh et al., 2023).

Based on Figure 3, the results of the E-EWS record and EWS machine learning predictions with the random forest classification model, namely the prediction accuracy reached 76.19% in predicting the clinical status of patients within 48 hours, showing how well the model can predict the E-EWS level category with an MAE value of 0.31 the model has a relatively low error rate. The results of this study explain that the E-EWS Score and machine learning can be used to predict the patient's clinical status at 48 hours. The results of this study are in line with previous research, namely for AI-based warnings with random forests with vital sign value parameters in the EWS which can predict arrest events. hearts in greater numbers and faster with 80% accuracy, thereby reducing the incidence of false alarms in hospitals at 30 minutes – 24 hours and 48 hours (Wu et al., 2021b). Based on Figure 4, the E-EWS score prediction results for predicting a patient's clinical condition at 72 hours reached 71.43% with an MAE value of 0.47. The results of the two observations showed differences in the predicted results of the E-EWS scores at 48-hour and 72-hour observations. The difference in time to predict clinical status affects prediction accuracy, accuracy increases by more than 10% when predictions are made using vital sign data from the previous 60 minutes (Soudan et al., 2022).

Based on Figure 3, the results of the E-EWS record and EWS machine learning predictions with the random forest classification model, namely the prediction accuracy reached 76.19% in predicting the clinical status of patients within 48 hours, showing how well the model can predict the E-EWS level category with an MAE value of 0.31 the model has a relatively low error rate. The results of this study explain that the E-EWS Score and machine learning can be used to predict the patient's clinical status at 48 hours. The results of this study are in line with previous research, namely for AI-based warnings with random forests with vital sign value parameters in the EWS which can predict arrest events. hearts in greater numbers and faster with 80% accuracy, thereby reducing the incidence of false alarms in hospitals at 30 minutes – 24 hours and 48 hours (Wu et al., 2021b). Based on Figure 4, the E-EWS score prediction results for predicting a patient's clinical condition at 72 hours reached 71.43% with an MAE value of 0.47. The results of the two observations showed differences in the predicted results of the E-EWS scores at 48-hour and 72-hour observations. The difference in time to predict clinical status affects prediction accuracy, accuracy increases by more than 10% when predictions are made using vital sign data from the previous 60 minutes (Soudan et al., 2022).

The accuracy of the E-EWS score prediction on clinical status at hospital discharge reached 97.62% with an MAE value of 0.02. In the confusion matrix value between the E-EWS record score and the EWS score predicted by machine learning on 20% of the test data from 206 data sets, there is the same prediction for outpatient control discharge status of 40 respondents and 1 respondent for death discharge status with a value of 2 respondents' predictions (2.38%) differ. For EWS scores, machine learning predictions can be used to predict future clinical status. Previous research explains that machine learning models can predict the incidence of death in hospitals within 48 hours. MEWS performance decreased while the machine learning model increased in 6 hours and lower in 168 hours, machine learning can predict hospital deaths more than MEWS (Wahyuni et al., 2024).

Machine learning approaches have the potential to provide more accurate and adaptive predictions over time with increasing accuracy. The results of this research are in line with Sankavi Muralitharan, et al. who found that the performance of the two early warning systems showed that the machine learning- based system had a higher level of accuracy than Aggregate-weighted Early Warning Systems (EWSs) (Muralitharan et al., 2021). The use of machine learning is used because it has better performance and can achieve a high level of accuracy. Therefore, the development of Artificial Intelligence in health services requires special attention so that it can help doctors and nurses with patient prognosis and provide optimal service to critical patients (Ependi, Rochim & Wibowo, 2023). Nurses in conducting assessments use E-EWS records which aim to help identify physiological abnormalities before clinical changes occur in patients during a heart attack, so the use of machine learning models can help reduce risk by increasing the detection of patients who have the potential to enter the ICU (Rojas et al., 2018).

Limitation

This study has some limitations. First, the sample size was relatively small (n = 206) and drawn from a single hospital, which may limit the generalisability of the findings. Second, the E-EWS data relied on input from healthcare workers, introducing the potential for human error and variability. Third, only seven EWS parameters were used, excluding other potentially influential clinical indicators such as ECG and laboratory results. Additionally, the study employed a single machine learning algorithm— Random Forest—without comparing its performance against other models. Future research should address these limitations by incorporating larger, multi-centre datasets and exploring diverse predictive variables and algorithm comparisons for broader applicability.

CONCLUSION

The E-EWS (Electronic Early Warning Score) record system plays a crucial role in the early detection of patient deterioration in hospital settings. It serves as a vital tool for initial assessment in emergency rooms, facilitating integrated service delivery and efficient access to clinical information. When supported by machine learning, E-EWS can enhance predictive accuracy by utilising seven key parameters to assess future patient conditions. Compared to manually entered records by healthcare workers, machine learning-enhanced EWS offers greater precision in forecasting patient outcomes. This technology has the potential to assist nurses and other healthcare professionals in identifying patients at risk of clinical deterioration, including the likelihood of cardiac arrest, thus improving patient safety and clinical decision-making.

Building on these findings, future research should explore the integration of additional clinical parameters—such as ECG data and laboratory test results—into machine learning models to further improve the accuracy of EWS predictions. Expanding the dataset with a larger sample size is also recommended to enhance model generalisability and minimise bias. Such advancements could lead to the development of more comprehensive and reliable predictive systems that better support clinical interventions and patient care outcomes.

Conflict of Interest

The authors declare that they have no competing interests.

ACKNOWLEDGEMENT

The authors would like to express their gratitude to all the individuals who supported the study.

REFERENCES

Arnold, J., Davis, A., Fischhoff, B., Yecies, E., Grace, J., Klobuka, A., ... & Hanmer, J. (2019). Comparing the predictive ability of a commercial artificial intelligence early warning system with physician judgement for clinical deterioration in hospitalised general internal medicine patients: A prospective observational study. BMJ Open, 9(10). https://doi.org/10.1136/bmjopen-2019-032187

Bach, R. L., & Wenz, A. (2020). Studying health-related internet and mobile device use using web logs and smartphone records. PLoS One, 15(6). https://doi.org/10.1371/journal.pone.0234663

Bell, D., Baker, J., Williams, C., & Bassin, L. (2021). A trend-based early warning score can be implemented in a hospital electronic medical record to effectively predict inpatient deterioration. Critical Care Medicine, 49(10), e961-e967. https://doi.org/10.1097/CCM.0000000000005064

Breiman, L. (2001). Random forests. Machine Learning, 45, 5-32.https://doi.org/10.1023/A:1010933404324

Chiew, C. J., Liu, N., Tagami, T., Wong, T. H., Koh, Z. X., & Ong, M. E. (2019). Heart rate variability based machine learning models for risk prediction of suspected sepsis patients in the emergency department. Medicine, 98(6). https://doi.org/10.1097/MD.0000000000014197

Cho, K. J., Kwon, O., Kwon, J. M., Lee, Y., Park, H., Jeon, K. H., ... & Oh, B. H. (2020). Detecting patient deterioration using artificial intelligence in a rapid response system. Critical Care Medicine, 48(4), e285-e289. https://doi.org/10.1097/CCM.0000000000004236

Connor, N., McArthur, D., & Camargo Plazas, P. (2021). Reflections on vital sign measurement in nursing practice. Nursing Philosophy, 22(1). https://doi.org/10.1111/nup.12326

DeLaune, S. C., & Ladner, P. K. (2002). Fundamentals of nursing standards & practice. Thomson Learning, London, UK. Retrieved from: https://diglib.globalcollege.edu.et:8080/xmlui/bitstream/handle/123456789/1629/Fundamentals%20of%20Nursing%20Standards%20and%20Practices.pdf?sequence=1&isAllowed=y. Accessed on 19thFebruary 2023.

Ependi, U., Rochim, A. F., & Wibowo, A. (2023). An assessment model for sustainable cities using crowdsourced data based on general system theory: A design science methodology approach. Smart Cities, 6(6), 3032-3059. https://doi.org/10.3390/smartcities6060136

Gerry, S., Bonnici, T., Birks, J., Kirtley, S., Virdee, P. S., Watkinson, P. J., & Collins, G. S. (2020). Early warning scores for detecting deterioration in adult hospital patients : Systematic review and critical appraisal of methodology. BMJ Journal, 1–16. https://doi.org/10.1136/bmj.m1501

Global Health Data Exchanged (2018). Agency of Health Research and Development (Indonesia). Retrieved from: https://ghdx.healthdata.org/organizations/agency-health-research-and-development- indonesia . Accessed on 19th January 2023.

Jahandideh, S., Ozavci, G., Sahle, B. W., Kouzani, A. Z., Magrabi, F., & Bucknall, T. (2023). Evaluation of machine learning-based models for prediction of clinical deterioration: A systematic literature review. International Journal of Medical Informatics, 175. https://doi.org/10.1016/j.ijmedinf.2023.105084

Ljunggren, M., Castrén, M., Nordberg, M., & Kurland, L. (2016). The association between vital signs and mortality in a retrospective cohort study of an unselected emergency department population. Scandinavian Journal of Trauma, Resuscitation and Emergency Medicine, 24, 1-11. https://doi.org/10.1186/s13049-016-0213-8

Mahmoodpoor, A., Sanaie, S., Saghaleini, S. H., Ostadi, Z., Hosseini, M.-S., Sheshgelani, N., Vahedian-Azimi, A., Samim, A., & Rahimi-Bashar, F. (2022). Prognostic value of national early warning score and modified early warning score on intensive care unit readmission and mortality: A prospective observational study. Frontiers in Medicine, 9. https://doi.org/10.3389/fmed.2022.938005

Mohapatra, P., Chakravarty, S., & Dash, P. K. (2015). An improved cuckoo search based extreme learning machine for medical data classification. Swarm and Evolutionary Computation, 24, 25-49. https://doi.org/10.1016/j.swevo.2015.05.003

Muralitharan, S., Nelson, W., Di, S., McGillion, M., Devereaux, P. J., Barr, N. G., & Petch, J. (2021). Machine learning-based early warning systems for clinical deterioration : Systematic Scoping Review. Journal of Medical Internet Research, 23(2). https://doi.org/10.2196/25187

Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., ... & Duchesnay, É. (2011). Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12, 2825- 2830. Retrieved from: https://www.jmlr.org/papers/volume12/pedregosa11a/pedregosa11a.pdf?source=post_page. Accessed on 15th July 2023.

Pirneskoski, J., Tamminen, J., Kallonen, A., Nurmi, J., Kuisma, M., Olkkola, K. T., & Hoppu, S. (2020). Random forest machine learning method outperforms predicting one-day mortality : A retrospective study. Resuscitation Plus, 4. https://doi.org/10.1016/j.resplu.2020.100046

Rojas, J. C., Carey, K. A., Edelson, D. P., Venable, L. R., Howell, M. D., & Churpek, M. M. (2018). Predicting intensive care unit readmission with machine learning using electronic health record data. Annals of the American Thoracic Society, 15(7), 846-853.https://doi.org/10.1513/AnnalsATS.201710-787OC

Soudan, B., Dandachi, F., & Nassif, A. (2022). Attempting cardiac arrest prediction using artificial intelligence on vital signs from Electronic Health Records. Smart Health, 25. https://doi.org/10.1016/j.smhl.2022.100294

Stenhouse, C., Coates, S., Tivey, M., Allsop, P., & Parker, T. (2000). Prospective evaluation of a modified Early Warning Score to aid earlier detection of patients developing critical illness on a general surgical ward. British Journal of Anaesthesia, 84(5). https://doi.org/https://doi.org/10.1093/bja/84.5.663

Wahyuni, S., Pranata, S., Setiaprabhawa, F. B., & Maftuchah, L. (2024). Exploring the trend of technology use and innovation in health care service in hospitals through a bibliometric analysis. Babcock University Medical Journal, 7(1), 51-63. Retrieved from: https://www.ajol.info/index.php/bumj/article/view/266263 . Accessed on 11th March 2023.

Williams, B. (2019). The National Early Warning Score and the acutely confused patient. Clinical Medicine (London), 19(2), 190–191. https://doi.org/10.7861/clinmedicine.19-2-190

Wu, C.-L., Kuo, C.-T., Shih, S.-J., Chen, J.-C., Lo, Y.-C., Yu, H.-H., Huang, M.-D., Sheu, W. H.-H.,& Liu, S.-A. (2021a). Implementation of an electronic national early warning system to decrease clinical deterioration in hospitalized patients at a Tertiary Medical Center. International Journal of Environmental Research and Public Health, 18(9). https://doi.org/10.3390/ijerph18094550

Wu, K.-H., Cheng, F.-J., Tai, H.-L., Wang, J.-C., Huang, Y.-T., Su, C.-M., & Chang, Y.-N. (2021b).Predicting in-hospital mortality in adult non-traumatic emergency department patients: A retrospective comparison of the Modified Early Warning Score (MEWS) and machine learning approach. PeerJ, 9. https://doi.org/10.7717/peerj.11988