The AI Detector Paradox: A Corpus-Based Analysis of Lexical Features and ZeroGPT Scores in Chinese L2 Undergraduate Writing

Authors

  • Huyue Liao Faculty of School of Education, Lincoln University College, Malaysia
  • Lubna Ali Mohammed Lincoln University College

DOI:

https://doi.org/10.60072/ijeissah.2026.v4i04.004

Abstract

This study examines the "AI Detector Paradox": the association between L2 English lexical features and AI-detection scores. Applying standardized multiple regression to a baseline corpus of 200 verified human-authored undergraduate theses, we analyzed how specific lexical features are associated with ZeroGPT AI-likelihood scores. The results point to a counterintuitive "Penalty of Proficiency." Reliance on formulaic language is not associated with higher scores. Rather, higher lexical diversity (TTR) is significantly associated with higher AI scores (β = 0.188, p < 0.01), while dense academic vocabulary (AWL) paradoxically acts as a protective shield (β = -0.284, p < 0.001). These associations suggest that ZeroGPT scores may penalize signs of L2 lexical development, which could pressure L2 students to fall back on rigid, template-based writing to avoid false accusations.

Keywords:

Academic Integrity, AI Detection Bias, Corpus Analysis, Formulaic Language, L2 Learners

Downloads

Download data is not yet available.

References

Alexander, K., Savvidou, C., & Alexander, C. (2023). Who wrote this essay? Detecting AI-generated writing in second language education in higher education. Teaching English with Technology, 23(2), 25–43. https://doi.org/10.56297/BUKA4060/XHLD5365

Elkhatat, A. M., Elsaid, K., & Almeer, S. (2023). Evaluating the efficacy of AI content detection tools in differentiating between human and AI-generated text. International Journal for Educational Integrity, 19(1), 17. https://doi.org/10.1007/s40979-023-00140-5

Hyatt, J.-P. K., Bienenstock, E. J., Firetto, C. M., Woods, E. R., & Comus, R. C. (2025). Using aggregated AI detector outcomes to eliminate false positives in STEM-student writing. Advances in Physiology Education, 49(3), 486–495. https://doi.org/10.1152/advan.00235.2024

Ibrahim, K. (2023). Using AI-based detectors to control AI-assisted plagiarism in ESL writing: “The Terminator Versus the Machines”. Language Testing in Asia, 13(1), 46. https://doi.org/10.1186/s40468-023-00260-2

Liang, W., Yuksekgonul, M., Mao, Y., Wu, E., & Zou, J. (2023). GPT detectors are biased against non-native English writers. arXiv preprint arXiv:2304.02819.

Maamuujav, U. (2021). Examining lexical features and academic vocabulary use in adolescent L2 students' text-based analytical essays. Assessing Writing, 49, 100540. https://doi.org/10.1016/j.asw.2021.100540

Mitchell, E., Lee, Y., Khazatsky, A., Manning, C. D., & Finn, C. (2023). DetectGPT: Zero-shot machine-generated text detection using probability curvature. In International Conference on Machine Learning (pp. 24950–24962). PMLR. https://proceedings.mlr.press/v202/mitchell23a.html

Pratama, A. R. (2025). The accuracy-bias trade-offs in AI text detection tools and their impact on fairness in scholarly publication. PeerJ Computer Science, 11, e2953. https://doi.org/10.7717/peerj-cs.2953

Walters, W. H. (2023). The effectiveness of software designed to detect AI-generated writing: A comparison of 16 AI text detectors. Open Information Science, 7(1), 20220158. https://doi.org/10.1515/opis-2022-0158

Weber-Wulff, D., Anohina-Naumeca, A., Bjelobaba, S., Foltýnek, T., Guerrero-Dib, J., Popoola, O., Šigut, P., & Waddington, L. (2023). Testing of detection tools for AI-generated text. International Journal for Educational Integrity, 19(1), 26. https://doi.org/10.1007/s40979-023-00146-z

Yan, D., Fauss, M., Hao, J., & Cui, W. (2023). Detection of AI-generated essays in writing assessments. Psychological Test and Assessment Modeling, 65(1), 125–144.

Published

26-10-2026

How to Cite

Liao, H., & Ali Mohammed, L. (2026). The AI Detector Paradox: A Corpus-Based Analysis of Lexical Features and ZeroGPT Scores in Chinese L2 Undergraduate Writing. International Journal of Emerging Issues in Social Science, Arts and Humanities ( IJEISSAH), 4(4), 39-45. https://doi.org/10.60072/ijeissah.2026.v4i04.004

Issue

Section

SI 2026: Transforming Education in the Age of Artificial Intelligence

Metrics