Preliminary Diagnostic Accuracy and Usability of a Rule-Based mHealth Application for Differentiating Migraine and Tension-Type Headache: A Pilot Study

Abstract

Background: Headache disorders are common causes of disability, and differentiating migraine from Tension-Type Headache (TTH) can be challenging because of overlapping clinical features. Mobile Health (mHealth) applications may support structured symptom assessment, but diagnostic performance should be established before such tools are incorporated into clinical triage. Objectives: This pilot study evaluated the preliminary diagnostic accuracy and inter-method agreement of a rule-based mHealth application for differentiating migraine from TTH using International Classification of Headache Disorders, 3^rd edition (ICHD-3) criteria and assessed application usability. In this study, diagnostic accuracy was assessed against an independently established clinical reference diagnosis, while agreement between the application and the clinical reference diagnosis was assessed using Cohen's kappa. Methods: A quantitative, cross-sectional pilot study included 73 participants recruited at the University of Fujairah. The application used a deterministic, logic-based weighted algorithm derived from ICHD-3 symptom criteria. Diagnostic accuracy was evaluated in a subset of 25 participants. After completing the application assessment, these participants attended clinical assessments by physicians from neurology and medicine, who independently applied ICHD-3 criteria without access to the application classification. Sensitivity, specificity, Positive Predictive Value (PPV), Negative Predictive Value (NPV), overall accuracy, and Cohen's kappa were calculated. Reliability in this pilot study refers to agreement with the independent clinical classification as quantified by Cohen's kappa; test-retest reproducibility was not assessed. Usability was assessed using the System Usability Scale (SUS).  Results: In the validation subset, overall accuracy was 88.0% (95% CI: 68.8%-97.5%), sensitivity was 80.0% (95% CI: 44.4%-97.5%), specificity was 93.3% (95% CI: 68.1%-99.8%), PPV was 88.9% (95% CI: 51.8%-99.7%), and NPV was 87.5% (95% CI: 61.7%-98.4%). Agreement with the clinical reference diagnosis was substantial (Cohen's kappa = 0.74). The mean SUS score was 78.6 (SD = 12.3). The wide confidence intervals indicate limited precision because of the small validation subset. Conclusion: The application demonstrated promising preliminary diagnostic accuracy and inter-method agreement with an independently established clinical diagnosis, together with satisfactory usability in this pilot sample. Because the clinical validation subset was small and confidence intervals were wide, these findings should be interpreted cautiously and should not be generalized beyond similar populations. Larger studies are required to confirm diagnostic accuracy and to evaluate test-retest and other forms of reproducibility before routine clinical implementation.