An AI-driven hybrid LSTM–random forest framework for detecting and adaptively responding to modern multi-factor authentication bypass attacks

Abstract

Multi-factor authentication (MFA) is widely deployed to counter credential-based attacks, yet modern bypass techniques — SIM swap fraud, adversary-in-the-middle (AiTM) phishing, session token hijacking, and MFA fatigue — continue to circumvent it by exploiting its static, login-centric design rather than its cryptographic foundations. Reviewed adaptive-authentication and behavioural risk-scoring systems address parts of this problem individually, but few published, operational systems combine log-sequence anomaly detection, contextual risk signals, and adaptive MFA enforcement logic specifically targeting these four bypass patterns within a single evaluated pipeline. This study designs, develops, and evaluates a proof-of-concept AI-driven framework for MFA bypass detection, implemented as a five-layer modular pipeline grounded in Design Science Research methodology. Detection is performed by a two-model ensemble — a two-layer Long Short-Term Memory (LSTM) network modelling temporal authentication sequences and a 100-tree Random Forest classifier operating on engineered per-event features — whose outputs are fused with contextual and behavioural signals into a single risk score mapped to three enforcement actions: Allow, Step-Up MFA, or Block. Trained on the Risk-Based Authentication (RBA) Login Dataset supplemented with synthetically generated, noise-injected attack records, the framework achieves an F1-score of 96.74%, recall of 96.92%, and a false positive rate of 3.47% on a held-out test set, at a mean inference latency of 47.51 ms. A novel Unnecessary Step-Up Rate (USR) of 3.26% quantifies the authentication friction imposed on legitimate users. Ablation and out-of-distribution evaluations confirm the individual contribution of each architectural component and the framework's generalisation beyond its training distribution, positioning it favourably against a static MFA baseline and the closest related end-to-end design. Reported figures reflect a balanced 50/50 evaluation ratio; at realistic attack prevalence, precision degrades substantially unless the decision threshold is recalibrated, a deployment-relevant finding quantified in this revision alongside bootstrap confidence intervals and a corrected feature-ablation analysis.