Association between Overall Academic Achievement, Cognitive, and Non-Cognitive Variables among Medical and Allied Health Students: A 10-Year Review

Josephine N. Amadi1*, Regidor III Dioso1, Glory B. Obong1, Chinemerem Eleke2, Idris Adewale Ahmed3

1Faculty of Nursing, Lincoln University College, Wisma Lincoln, No. 12-18, Jalan SS 6/12, 47301 Petaling Jaya, Selangor Darul Ehsan, Malaysia

2Department of Nursing Sciences, University of Port Harcourt, Abuja Campus, University of, 500272, Port Harcourt, Nigeria

3Department of Biotechnology, Lincoln University College, Wisma Lincoln, No. 12-18, Jalan SS 6/12, 47301 Petaling Jaya, Selangor Darul Ehsan, Malaysia

*Corresponding Author’s Email: idrisahmed@lincoln.edu.my

ABSTRACT

Background: Medical and allied health institutions employ cognitive measures to select student applicants who can fulfill curriculum components and pass the licensure examination. Secondary school certificate (SSC) grades and preadmission aptitude test (PAT) scores have been in use without systematic evidence. Nevertheless, the connection between non-cognitive elements and academic achievement is not clear. Objectives: This study explored the association between cognitive and non-cognitive aspects of academic achievement in medical and allied health programs. Methods: A narrative review design was applied, and the PubMed/MEDLINE database was searched. The eligibility criteria were undergraduate courses in medical and allied health programs, examining at least one cognitive (SSC scores and preadmission) and non-cognitive (age, gender, parity, parental occupation, lottery, non-cognitive personality test, and letter of recommendation) factor, measuring academic achievement and associational outcomes. Studies were screened by title, abstract, and full text, and data were extracted using a novel data extraction form. Results: About 22 studies were included, involving a sample of 29,152 students were reviewed. There were mixed results on the examined factors, with SSC grades, PAT scores, being female, and being of younger age appearing to have diminishing consistency in the mentioned order across the reviewed studies. There was a paucity of studies on the examined non-cognitive elements. Conclusion: There is insufficient evidence to recommend the exclusive use of any single cognitive or non-cognitive element for admissions decisions. A combination of SSC grades, PAT scores, female gender, and younger age should be considered and weighed in the mentioned order of importance.

Keywords: Academic Success; Aptitude; Certification; Schools; Students

INTRODUCTION

Selecting students who will ultimately become successful in completing a health-related bachelor’s degree is a pressing problem for colleges and educators (Stankus et al., 2019). Given that medical and allied health academic programs are rigorous, educators understand the value of selecting the best-suited students for admission (Zamanzadeh et al., 2020). The selection process aims to identify the applicants who will successfully meet the academic requirements to become licensed healthcare practitioners (Patterson, Griffin, & Hanson, 2018). Many educators view the admission criteria as an essential factor that predicts academic achievement (Mwandigha et al., 2018; Yousafzai & Jamil, 2019). Educators have continued to examine the criteria for admission into medical and allied health programs for their predictive value regarding students’ academic achievement (Roach et al., 2019).

Academic achievement is the completion of educational objectives in one's chosen field of study. It is highly valued in many academic and professional settings and is considered a predictor of future success in the chosen field of practice. Moreover, previous research suggests that academic achievement in medical and allied health academic programs may be influenced by cognitive and non-cognitive elements (Yousafzai & Jamil, 2019). The cognitive elements include secondary school certificate grades and pre-admission aptitude test scores (Liu, Codd, & Mills, 2018). The non-cognitive elements are age, gender, personality, and other demographic and non-demographic variables outside the cognitive elements (Roach et al., 2019). Consequently, the researcher recommends that medical schools and the authorities responsible for selecting and admitting medical students prioritize the consideration of well-informed career decisions. This is because making informed career choices can greatly influence the academic achievements and contentment of medical students (Bekele et al., 2023).

The selection process for medical and allied college applicants involves assessing their cognitive (secondary school certificate grades and pre-admission aptitude test score) and non-cognitive characteristics (demographic details). Where selection based on cognitive elements is governed by test score metrics, selection based on non-cognitive elements is governed by human judgment (Kreiter et al., 2018). Consequently, some researchers suggest that pre-admission aptitude tests alone may not be comprehensive enough, as they often rely on the same constructs as secondary school certificate examinations (Roach et al., 2019; Mwandigha et al., 2018). Furthermore, some researchers argued that cognitive tests may not provide a fair assessment of all individuals as they only evaluate prior scientific knowledge (Kim et al., 2016; Žuljević & Buljan, 2022). A student's cognitive abilities and learning performance may be influenced by whether they are morning or evening types. Blatter and Cajochen's (2007) research found that cognitive performance is influenced by time of day and extended periods of forced wakefulness, affecting tasks related to complexity, memory, and language (Vian et al., 2019). Moreover, though many studies have focused on the relationship between academic performance and secondary school certificate scores and aptitude test results, research concerning the association between demographic characteristics and academic achievement has been limited (Kim et al., 2016; Mwandigha et al., 2018; Yousafzai & Jamil, 2019). To address this issue, some studies have focused on looking at the demographic information of applicants to try and reach a balance between fairness and accuracy (Barber et al., 2022; De-Visser et al., 2018; Žuljević & Buljan, 2022).

In an attempt to reduce student dropout rates in health-related bachelor’s degree programs, studies have examined non-cognitive factors such as age, gender, marital status, parity status, and parent occupation (Callwood et al., 2018; Gale et al., 2016). Nonetheless, due to the lack of systematic evidence, the demographic background has been utilized in an inconsistent manner (Zamanzadeh et al., 2020). The impact of socio-demographic characteristics on academic achievement in health- related colleges has not been sufficiently reported (Wambuguh, Eckfield, & Van-Hofwegen, 2016).

Healthcare educators and professionals consider human judgment an important factor in admitting students to a health-related academic program. However, there is not much systematic evidence regarding the ability of the practice to predict overall student academic achievement at the end of the program. The research problem for this study was articulated using the subjects, phenomena of interest, design, evaluation, and type of research (SPIDER) framework as follows: among medical and health students, are cognitive and non-cognitive elements associated with their overall academic achievement, judging from data from quantitative studies? This study reviewed the evidence of the association between overall academic achievement and cognitive and non- cognitive variables in students of medical and allied health programs.

The significance of research on cognitive and non-cognitive factors influencing academic performance among nursing students is essential (Hollinger-Smith et al., 2023). Understanding the role of cognitive factors will enable nursing educators to focus on cultivating them in nursing students (Hsieh, Wang, & Huang, 2023). Examining non-cognitive factors will inform nurse educators on how to address them to foster academic success among nursing students (Ang et al., 2022). Conducting a 10-year review will offer insights into evolving trends and aid in tailored interventions.

METHODOLOGY

Study Design

This systematic review was conducted in adherence to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement that outlined the optimal standards for reporting systematic reviews (Tetzlaff, Page, & Moher, 2020).

Eligibility Criteria

The selection criteria for the research were outlined based on the population, phenomena of interest reported, outcome, and type of publication. This study included studies of medical and allied students. Studies that examined cognitive (secondary school certification grades and pre- admission) and non-cognitive (age, gender, parity, parental occupation, lottery, non-cognitive personality test, and letter of recommendation) were included. Additionally, studies reporting licensure examination outcomes were included. Only quantitative studies published in peer- reviewed journal articles were utilized.

Studies were excluded if they: (1) incorporated students who were not in medical or healthcare professions; (2) analyzed cognitive aspects such as grade point average in college; and (3) used evaluations of admission criteria as the basis for results.

Search Strategy

The literature search was conducted in the PubMed/Medline database between March 31, 2023, and April 20, 2023. The literature search utilized keywords enhanced with Boolean operators and truncations as follows: (Medical OR Allied) AND (Students) AND (Cognitive OR "Non- Cognitive") AND (Academic*). JA and CE searched the database independently and agreed on the studies to be included by consensus. Additionally, the reference lists of the included studies were hand-searched to identify further relevant studies that met the inclusion criteria. The search was limited to English-language articles published between 2013 and 2023 (10 years).

Selection Process

The literature search query was carried out, and studies were obtained. The two review authors (JA and CE) screened the titles and abstracts to recognize studies that fulfilled the criteria for inclusion. Discrepancies were resolved by consensus among the other three review authors (RB, GBO, and AN). The full texts were evaluated independently by two review authors (JA and CE), and disagreements were discussed and resolved with the other three review authors (RB, GBO, and AN). Twenty-two (22) studies were finally included in the review.

Data Collection Process

One review author (JA) extracted data independently from the included studies using a data extraction form. The extracted data included: (1) the study country; (2) the type of students studied; (3) the research design; (4) the population; (5) the cognitive and non-cognitive elements evaluated; and (6) the outcomes. A second review author (CE) reviewed the extracted data for correctness.

Quality and Certainty Assessment of Included Studies

The 12-item CASP Checklist for quantitative studies was used to evaluate the quality of the studies included in the review (CASP: Critical Appraisal Skill Programme). This tool has been demonstrated to be a dependable instrument for critical appraisal (Ma et al., 2020). Two reviewers (JA and CE) independently rated the quality of each study in the review, giving a score of 0 when the quality criterion was not met and a score of 1 when the quality criterion was met. Any differences between the reviewers were resolved by a third reviewer (AN).

The review involved data extracted from levels II (quasi-experimental studies) and III (retrospective and longitudinal cohort studies) that have a low level of certainty of evidence. The heterogeneity in medical and allied health academic programmes further complicated the certainty level. The CASP Checklist was employed to guarantee the inclusion of the best evidence in the review and improve the level of certainty of the results.

RESULTS

An electronic database search identified 817 studies, and an additional two were identified through a manual search. After screening titles and abstracts, 98 studies were selected for full-text review (eligibility check). Ultimately, 22 studies fulfilled the eligibility criteria and were included in the review, as illustrated in Figure 1.

Figure 1: Preferred Reporting Items for Systematic Review and Meta-Analyses Flow Chart

Table 1: Study Characteristics

Study Characteristics

Table 1 provides a summary of the statistics and findings from each study that was reviewed. Twenty-two studies involving 29,152 students in medical and allied health bachelor's degree programmes were eligible and included in the review. The studies were published between 2013 and 2023 (10 years) and consisted of two quasi-experimental (level II) studies and 20 observational (level III) studies. All the included studies scored above 12 out of 15 on the CASP checklist.

Table 2: Result Synthesis

Narrative Synthesis

Results from Table 2 showed that more than half (56%) of studies on the link between secondary school certificate grades and academic achievement in medical and allied colleges indicated a significant association. Close to half (44%) of the studies that examined preadmission tests showed a significant association with academic achievement. Most studies looking into gender, age, lottery, non-cognitive test interviews, and letters of recommendation found no significant association between the variables and academic achievement.

DISCUSSION

This study is a 10-year review and narrative synthesis of evidence on the association between academic achievement in medical and allied health schools and cognitive and non-cognitive admission criteria. Results from this review demonstrated that a student's secondary school certificate grades are more consistently associated with their overall academic performance in medical and allied health courses than the preadmission aptitude test score. It may be related to the variability in the design of the preadmission aptitude tests used in the countries examined. Where the preadmission aptitude test does suggest an associational relationship, it is weak. It is possible that preadmission aptitude tests could predict success in other relevant outcomes not examined in this review. Nonetheless, significant findings regarding students' secondary school certificate grades were inconsistent across the 16 studies, with some demonstrating a significant association (Alhurishi et al., 2021; Krings et al., 2020; Žuljević and Buljan, 2022) and others finding no significant relationship (Almarabheh et al., 2022; Barber et al., 2022). Findings on preadmission aptitude test scores were also inconsistent across 18 studies, with some showing a significant association (Alhurishi et al., 2021; Almarabheh et al., 2022) and others not (Barber et al., 2022; Krings et al., 2020; Žuljević & Buljan, 2022). Nonetheless, this review brings to light the limitations of utilizing only preadmission aptitude test scores for the selection process of applicants into medical and allied health colleges. Given that secondary school certificate grades showed the most evidence of association with academic achievement, medical and allied health colleges could consider using students' secondary school certificate grades over and above the preadmission aptitude test scores for the student selection process.

Where professional behavior is a primary concern for medical and allied health colleges, one could argue that personality, letter of recommendation, family background, and other demographic factors may play a part; hence, one should consider the non-cognitive individual elements in the student selection process. Across the studies reviewed, few non-cognitive factors were studied in connection with academic achievement in medical and allied health courses. Age, gender, non- cognitive personality test or interview, letter of recommendation, and lottery were the only factors evaluated. Across 13 studies, three found a significant association between academic achievement and the female gender (Adam et al., 2015; Barber et al., 2022; Mercer, Abbott & Puddey, 2013), and one study found a significant association with younger age (Adam et al., 2015). This finding indicates that female students of younger age perform better in medical and allied courses, which are unexpected results. Personality traits, letters of recommendation, and lottery were not found to be consistently associated with academic achievement among medical and allied health students (Price and Park, 2018; Vos et al., 2019). Research on the mentioned non-cognitive elements is scarce, thus reducing their usefulness in predicting academic achievement. Future studies should aim to assess the mentioned non-cognitive elements in relation to academic achievement to yield a thorough understanding of their practical application.

Limitation

This review presents some limitations. The majority of the included studies were level III evidence. Additionally, only a minority of the included studies collect data from three or more cohorts or institutions. This will inevitably affect the strength of the evidence.

CONCLUSION

Based on studies published within the past ten years, there is insufficient evidence to support the exclusive use of cognitive and non-cognitive elements in decisions concerning who should be allowed to enroll in medical and allied health courses. The research team recommends that a combination of secondary school certificate grades, preadmission aptitude test scores, female gender, and younger age be considered and weighed in the mentioned order of importance.

Conflict of Interest

The authors declare that they have no conflict of interests.

ACKNOWLEDGEMENT

The authors are thankful to the institutional authority for completion of the work.

REFERENCES

Adam, J., Bore, M., Childs, R., Dunn, J., Mckendree, J., Munro, D., & Powis, D. (2015). Predictors of professional behaviour and academic outcomes in a UK medical school: A longitudinal cohort study. Medical Teacher, 37(9), 868-880. https://doi.org/10.3109/0142159X.2015.1009023

Al-Alwan, I., Al-Kishi, M., Tamim, H., Magzoub, M., & Elzubeir, M. (2013). Health sciences and medical college preadmission criteria and prediction of in-course academic performance: a longitudinal cohort study. Advances in Health Sciences Education, 18(3), 427-438. https://doi.org/10.1007/s10459-012-9380-1

Alhurishi, S. A., Aljuraiban, G. S., Alshaikh, F. A., Almutairi, M. M., & Almutairi, K. M. (2021). Predictors of students’ academic achievements in allied health professions at King Saud University: a retrospective cohort study. BMC Medical Education, 21(1), 1-7. https://doi.org/10.1186/s12909-021-02525-x

Almarabheh, A., Shehata, M. H., Ismaeel, A., Atwa, H., & Jaradat, A. (2022). Predictive validity of admission criteria in predicting academic performance of medical students: a retrospective cohort study. Frontiers in Medicine, 9, 971926. https://doi.org/10.3389/fmed.2022.971926

Al-Qahtani, M., & Alanzi, T. (2018). Comparisons of the predictive values of admission criteria for academic achievement among undergraduate students of health and non-health science professions: a longitudinal cohort study. Psychology Research and Behavior Management, 12(1), 1-6. https://doi.org/10.2147/PRBM.S183651

Alshammari, F., Saguban, R., Pasay-an, E., Altheban, A., & Al-Shammari, L. (2017). Factors affecting the academic performance of student nurses: A cross-sectional study. Journal of Nursing Education and Practice, 8(1), 60. https://doi.org/10.5430/jnep.v8n1p60

Ang, W., Chew, H., Rusli, K., Ng, W., Zheng, Z., Liaw, S., Ang, N., & Lau, Y. (2022). Spotlight on noncognitive skills: Views from nursing students and educators. Nurse Education Today, 117(1), e105486. https://doi.org/10.1016/j.nedt.2022.105486

Barber, C., Burgess, R., Mountjoy, M., Whyte, R., Vanstone, M., & Grierson, L. (2022). Associations between admissions factors and the need for remediation. Advances in Health Sciences Education, 27(2), 475-489. https://doi.org/10.1007/s10459-022-10097-8

Bekele, A. T., Beza, S. W., Gedamu, S., & Berndt, M. (2023). Predictors of College Academic Achievement for Medical Students: The Case of Gondar University, College of Medicine and Health Sciences, Ethiopia. Advances in Medical Education and Practice, 603-613. https://doi.org/10.2147/AMEP.S406031

Callwood, A., Jeevaratnam, K., Kotronoulas, G., Schneider, A., Lewis, L., & Nadarajah, V. (2018). Personal domains assessed in multiple mini interviews (MMIs) for healthcare student selection: a narrative synthesis systematic review. Nurse Education Today, 64(1), 56–64. https://doi.org/10.1016/j.nedt.2018.01.016

De-Visser, M., Fluit, C., Cohen-Schotanus, J., & Laan, R. (2018). The effects of a non-cognitive versus cognitive admission procedure within cohorts in one medical school. Advances in Health Sciences Education, 23(1), 187-200. https://doi.org/10.1007/s10459-017-9782-1

Edwards, D., Friedman, T., & Pearce, J. (2013). Same admissions tools, different outcomes: a critical perspective on predictive validity in three undergraduate medical schools. BMC Medical Education, 13(1), e173. https://doi.org/10.1186/1472-6920-13-173

Finn, G., Mwandigha, L., Paton, L., & Tiffin, P. (2018). The ability of 'non-cognitive' traits to predict undergraduate performance in medical schools: a national linkage study. BMC Medical Education, 18(1), e93. https://doi.org/10.1186/s12909-018-1201-7

Gale, J., Ooms, A., Grant, R., Paget, K., & Marks-Maran, D. (2016). Student nurse selection and predictability of academic success: the multiple mini interview project. Nurse Education Today, 40(1), 123–127. https://doi.org/10.1016/j.nedt.2016.01.031

Hollinger-Smith, L. M., Patterson, B. J., Morin, K. H., & Scott, C. J. (2023). Cognitive and Noncognitive Factors Influencing Nursing Students’ Academic Success: Structural Equation Model Analysis. Nursing Education Perspectives. https://doi.org/10.1097/01.NEP.0000000000001121

Hsieh, P., Wang, Y., & Huang, T. (2023). Exploring key factors influencing nursing students' cognitive load and willingness to serve older adults: Cross-sectional descriptive correlational study. JMIR Serious Games, 11(1), e43203. https://doi.org/10.2196/43203

Kim, T., Chang, J., Myung, S., Chang, Y., Park, K., Park, W., & Shin, C. (2016). Predictors of undergraduate and postgraduate clinical performance: A longitudinal cohort study. Journal of Surgical Education, 73(4), 715-720. https://doi.org/10.1016/j.jsurg.2016.03.006

Kreiter, C., O'Shea, M., Bruen, C., Murphy, P., & Pawlikowska, T. (2018). A meta-analytic perspective on the valid use of subjective human judgement to make medical school admission decisions. Medical Education Online, 23(1), e1522225. https://doi.org/10.1080/10872981.2018

Krings, R., Huwendiek, S., Walsh, N., Stricker, D., & Berendonk, C. (2020). Predictive power of high school educational attainment and the medical aptitude test for performance during the Bachelor program in human medicine at the University of Bern: a cohort study. Swiss Medical Weekly, 150(1), e20389. https://doi.org/10.4414/smw.2020.20389

Lancia, L., Caponnetto, V., Dante, A., Mattei, A., La-cerra, C., Cifone, M., & Petrucci, C. (2018). Analysis of factors potentially associated with nursing students' academic outcomes: a thirteen- year retrospective multi-cohort study. Nurse Education Today, 70(1), 115-120. https://doi.org/10.1016/j.nedt.2018.08.020

Liu, X., Codd, C., & Mills, C. (2018). Incremental effect of academic predictors on nursing admission assessment. Nurse Educator, 43(6), 292-296.

https://doi.org/10.1097/NNE.0000000000000502

Ma, L., Wang, Y., Yang, Z., Huang, D., Weng, H., & Zeng, A. (2020). Methodological quality (risk of bias) assessment tools for primary and secondary medical studies: what are they and which is better? Military Medical Research, 7(1), e7. https://doi.org/10.1186/s40779-020-00238-8

Mercer, A., Abbott, P., & Puddey, I. (2013). Relationship of selection criteria to subsequent academic performance in an Australian undergraduate dental school. European Journal of Dental Education, 17(1), 39-45. https://doi.org/10.1111/eje.12005

Mwandigha, L., Tiffin, P., Paton, L., Kasim, A., & Bohnke, J. (2018). What is the effect of secondary (high) schooling on subsequent medical school performance? a national, UK-based, cohort study. BMJ Open, 8(5), e020291. https://doi.org/10.1136/bmjopen-2017-020291

Patterson, F., Griffin, B., & Hanson, M. (2018). Opening editorial: selection and recruitment in medical education. MedEdPublish, 7(1), 1. https://doi.org/10.15694/mep.2018.0000222.1

Plouffe, R., Hammond, R., Goldberg, H., & Chahine, S. (2018). What matters from admissions? identifying success and risk among canadian dental students. Journal of Dental Education, 82(5), 515-523. https://doi.org/10.21815/JDE.018.057

Price, M., & Park, S. (2018). Can noncognitive components of admissions data predict dental student performance and postdoctoral program placement? Journal of Dental Education, 82(10), 1051-1058. https://doi.org/10.21815/JDE.018.112

Roach, A., Rose, A., Beiers-jones, W. S., Licaycay, W., & Nielsen, A. (2019). Incorporating group interviews into holistic review in baccalaureate nursing school admissions. Nursing Education Perspectives, 40(2), 125-127. https://doi.org/10.1097/01.NEP.0000000000000338

Sladek, R., Bond, M., Frost, L., & Prior, K. (2016). Predicting success in medical school: a longitudinal study of common Australian student selection tools. BMC Medical Education, 16(1), e187. https://doi.org/10.1186/s12909-016-0692-3

Stankus, J., Hamner, M., Stankey, M., & Mancuso, P. (2019). Successful modeling of factors related to recruitment and retention of prenursing students. Nurse Educator, 44(3), 147-150. https://doi.org/10.1097/NNE.0000000000000579

Tetzlaff, J., Page, M., & Moher, D. (2020). The PRISMA 2020 statement: development of and key changes in an updated guideline for reporting systematic reviews and meta-analyses. Value Health, 23(1), S312–S313. https://doi.org/10.1016/j.jval.2020.04.1154

Vian, C. V., binti Zulkifli, A. J., bin Hishamudin, A. N., Xian, L. F., Barry, J. A. A. H., Yee, O. T., ... & Mohandas, K. (2019). To compare the cognitive ability of preclinical medical students at different times of the working day and correlate this with their morningness-eveningness status. Malaysian Journal of Medical Research (MJMR), 3(1), 29-40. https://doi.org/10.31674/mjmr.2019.v03i01.005

Vos, C., Wouters, A., Jonker, M., de-Haan, M., Westerhof, M., Croiset, G., & Kusurkar, R. (2019). Bachelor completion and dropout rates of selected, rejected and lottery-admitted medical students in the Netherlands. BMC Medical Education , 19(1), e80. https://doi.org/10.1186/s12909-01

Wambuguh, O., Eckfield, M., & Van-hofwegen, L. (2016). Examining the importance of admissions criteria in predicting nursing program success. International Journal of Nursing Education Scholarship, 13(1), 87–96. https://doi.org/10.1515/ijnes-2015-0088

Yousafzai, I., & Jamil, B. (2019). Relationship between admission criteria and academic performance: a correlational study in nursing students. Pakistan Journal of Medical Sciences, 35(3), 858-861. https://doi.org/10.12669/pjms.35.3.217

Zamanzadeh, V., Ghahramanian, A., Valizadeh, L., Bagheriyeh, F., & Lynagh, M. (2020). A scoping review of admission criteria and selection methods in nursing education. BMC Nursing, 19(1), 121. https://doi.org/10.1186/s12912-020-00510-1

Žuljević, M., & Buljan, I. (2022). Academic and non-academic predictors of academic performance in medical school: an exploratory cohort study. BMC Medical Education, 22(1), e366. https://doi.org/10.1186/s12909-022-03436-1