Mr. Sreekanth K U, Mr. Adwaid R S

DOI: https://doi.org/10.5281/zenodo.23810210

Video Anomaly Detection (VAD) has emerged as a critical research area in intelligent surveillance, public safety, industrial monitoring, and smart city applications. Traditional supervised approaches rely on frame-level annotations, which are labor-intensive, time-consuming, and expensive to obtain, thereby limiting their scalability in large-scale surveillance systems. Weakly Supervised Video Anomaly Detection (WSVAD) addresses this challenge by utilizing only video-level labels while enabling effective temporal anomaly localization. In recent years, multimodal learning has significantly improved detection performance by integrating complementary information from visual, audio, and semantic modalities. This paper presents a comprehensive review of multimodal WSVAD techniques, emphasizing recent advancements in Multiple Instance Learning (MIL), vision-language models, audio-visual learning, knowledge transfer, and semantic alignment strategies. Representative methods are systematically analyzed and compared with respect to their methodologies, benchmark datasets, evaluation metrics, strengths, and limitations. Furthermore, major research challenges, including modality imbalance, limited multimodal datasets, computational complexity, and domain generalization, are discussed. Finally, emerging research directions involving foundation models, multimodal large language models (MLLMs), self-supervised learning, federated learning, and explainable artificial intelligence (XAI) are highlighted to support the development of scalable, robust, and interpretable anomaly detection systems.

Video Anomaly Detection, Weakly Supervised Learning, Multimodal Learning, Semantic Alignment, Knowledge Transfer, Vision-Language Models.

This review presented a comprehensive overview of recent advances in Multimodal Weakly Supervised Video Anomaly Detection (WSVAD). The evolution of the field from traditional handcrafted feature-based methods to deep learning, weakly supervised learning, vision-language models, and multimodal frameworks was examined. The analysis showed that multimodal approaches, which integrate visual, audio, and semantic information, offer improved anomaly localization, robustness, and generalization compared with unimodal methods. Despite these advancements, challenges such as limited multimodal datasets, noisy weak supervision, modality imbalance, computational complexity, and limited interpretability continue to restrict practical deployment. Emerging research directions, including foundation models, multimodal large language models, self-supervised learning, knowledge transfer, semantic alignment, and explainable AI, provide promising opportunities to address these limitations. Overall, multimodal WSVAD represents a promising direction for next-generation intelligent surveillance systems. Continued research toward efficient, scalable, and interpretable multimodal frameworks is expected to enable more reliable anomaly detection and understanding across diverse real-world applications.

W. Sultani, C. Chen, and M. Shah, “Real-World Anomaly Detection in Surveillance Videos,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), 2018, pp. 6479-6488.
[2] Y. Tian, G. Pang, Y. Chen, R. Singh, J. W. Verjans, and G. Carneiro, “Weakly-Supervised Video Anomaly Detection With Robust Temporal Feature Magnitude Learning,” in Proc. IEEE/CVF Int. Conf. Comput. Vis. (ICCV), 2021, pp. 4975-4986.
[3] J.-C. Feng, F.-T. Hong, and W.-S. Zheng, “MIST: Multiple Instance Self-Training Framework for Video Anomaly Detection,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), 2021, pp. 14009-14018.
[4] Y. Chen, Z. Liu, B. Zhang, W. T. Fok, X. Qi, and Y.-C. Wu, “MGFN: Magnitude-Contrastive Glance-and-Focus Network for Weakly-Supervised Video Anomaly Detection,” in Proc. AAAI Conf. Artif. Intell., vol. 37, no. 1, pp. 387-395, 2023.
[5] J. Yu, J. Liu, Y. Cheng, R. Feng, and Y. Zhang, “Modality-Aware Contrastive Instance Learning With Self-Distillation for Weakly-Supervised Audio-Visual Violence Detection,” in Proc. ACM Int. Conf. Multimedia (ACM MM), 2022.
[6] A. Radford et al., “Learning Transferable Visual Models From Natural Language Supervision,” in Proc. Int. Conf. Mach. Learn. (ICML), 2021, pp. 8748-8763.
[7] P. Wu, X. Zhou, G. Pang, L. Zhou, Q. Yan, P. Wang, and Y. Zhang, “VadCLIP: Adapting Vision-Language Models for Weakly Supervised Video Anomaly Detection,” Proc. AAAI Conf. Artif. Intell., vol. 38, no. 6, pp. 6074-6082, 2024.
[8] G. Pang, C. Shen, L. Cao, and A. van den Hengel, “Deep Learning for Anomaly Detection: A Review,” ACM Comput. Surv., vol. 54, no. 2, pp. 1-38, 2021.
[9] R. Mehran, A. Oyama, and M. Shah, “Abnormal Crowd Behavior Detection Using Social Force Model,” in CVPR, 2009, pp. 935-942.
[10] Y. Cong, J. Yuan, and J. Liu, “Sparse Reconstruction Cost for Abnormal Event Detection,” in CVPR, 2011, pp. 3449-3456.
[11] C. Lu, J. Shi, and J. Jia, “Abnormal Event Detection at 150 FPS in MATLAB,” in ICCV, 2013, pp. 2720-2727.
[12] W. Luo, W. Liu, and S. Gao, “A Revisit of Sparse Coding-Based Anomaly Detection in Stacked RNN Framework,” in ICCV, 2017.
[13] W. Luo, W. Liu, and S. Gao, “Remembering History With Convolutional LSTM for Anomaly Detection,” in ICME, 2017, pp. 439-444.
[14] W. Liu, D. Lian, and S. Gao, “Future Frame Prediction for Anomaly Detection: A New Baseline,” in CVPR, 2018.
[15] M. Ravanbakhsh, E. Sangineto, M. Nabi, and N. Sebe, “Training Adversarial Discriminators for Cross-Channel Abnormal Event Detection in Crowds,” in WACV, 2019, pp. 1896-1904.
[16] S. Sabokrou, M. Fathy, M. Hoseini, and R. Klette, “Deep-Anomaly: Fully Convolutional Neural Network for Fast Anomaly Detection in Crowded Scenes,” IEEE Signal Process. Lett., vol. 24, no. 1, pp. 88-92, 2017.
[17] J. Carreira and A. Zisserman, “Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset,” in CVPR, 2017, pp. 4724-4733.
[18] K. Simonyan and A. Zisserman, “Two-Stream Convolutional Networks for Action Recognition in Videos,” in NeurIPS, 2014.
[19] A. Dosovitskiy et al., “An Image Is Worth 16×16 Words: Transformers for Image Recognition at Scale,” in ICLR, 2021.
[20] R. Girdhar, J. Carreira, C. Doersch, and A. Zisserman, “Video Action Transformer Network,” in CVPR, 2019, pp. 244-253.
[21] C. Sun, A. Shrivastava, S. Singh, and A. Gupta, “Revisiting the Unreasonable Effectiveness of Data in Video Recognition,” in ICCV, 2017.
[22] S. Hershey et al., “CNN Architectures for Large-Scale Audio Classification,” in ICASSP, 2017.
[23] A. Alayrac et al., “Self-Supervised Multimodal Versatile Networks,” in NeurIPS, vol. 33, 2020.
[24] A. Miech, D. Zhukov, J.-B. Alayrac, M. Tapaswi, I. Laptev, and J. Sivic, “HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clips,” in ICCV, 2019, pp. 2630-2640.
[25] J. Meng, H. Tian, G. Lin, J.-F. Hu, and W.-S. Zheng, “Audio-Visual Collaborative Learning for Weakly Supervised Video Anomaly Detection,” IEEE Trans. Multimedia, 2025.
[26] K. He, X. Zhang, S. Ren, and J. Sun, “Deep Residual Learning for Image Recognition,” in CVPR, 2016, pp. 770-778.
[27] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet Classification With Deep Convolutional Neural Networks,” Commun. ACM, vol. 60, no. 6, pp. 84-90, 2017.
[28] F. Chollet, “Xception: Deep Learning With Depthwise Separable Convolutions,” in CVPR, 2017.
[29] A. Howard et al., “Searching for MobileNetV3,” in ICCV, 2019.
[30] A. Vaswani et al., “Attention Is All You Need,” in NeurIPS, vol. 30, 2017.
[31] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,” in NAACL-HLT, 2019, pp. 4171-4186.
[32] A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, and I. Sutskever, “Language Models are Unsupervised Multitask Learners,” OpenAI Technical Report, 2019.
[33] M. Zaheer, A. Mahmood, M. H. Khan, M. Astrid, and S.-I. Lee, “CLAWS: Clustering Assisted Weakly Supervised Learning With Normalcy Suppression for Anomalous Event Detection,” in ECCV, 2022.
[34] D. Purwanto, Y.-T. Chen, and W.-H. Fang, “Dance With Self-Attention: A New Look of Conditional Random Fields on Video Anomaly Detection,” IEEE Trans. Multimedia, vol. 24, pp. 2991-3004, 2022.
[35] H. Lv, Z. Yue, Q. Sun, B. Luo, and F. Shen, “BN-WVAD: Bag-Level Normalcy-Guided Weakly Supervised Video Anomaly Detection,” in ICME, 2023.
[36] B. Xu, Y. Wang, J. Wang, Y. Long, and Z. Wang, “Self-Guided Learning for Weakly Supervised Video Anomaly Detection,” in AAAI, 2023.
[37] J. Georgescu, A. Barbalau, R. T. Ionescu, and M. Shah, “Anomaly Detection in Video via Self-Supervised and Multi-Task Learning,” in CVPR, 2021.
[38] T. Wang, H. Snoussi, and F. Smach, “Video Anomaly Detection Based on Deep Prediction Networks,” Pattern Recognit. Lett., vol. 132, pp. 345-352, 2020.
[39] UCF-Crime Dataset Paper (Sultani et al., CVPR 2018).
[40] XD-Violence Dataset: J. Wu et al., “Not Only Look, But Also Listen: Learning Multimodal Violence Detection Under Weak Supervision,” in ECCV, 2020.
[41] ShanghaiTech Campus Dataset: W. Luo et al., 2017.
[42] H. Zhong, J. Li, Z. Zhang, Y. Jiao, and H. Shan, “Open-Vocabulary Video Anomaly Detection,” 2024.
[43] H. Zhong, J. Li, Z. Zhang, and H. Shan, “Uni-VAD: A Unified Framework for Weakly Supervised Video Anomaly Detection,” 2024.
[44] M. Kim, J. Lee, C. Bang, and M. Cho, “CLUE-VAD: Structured Semantic Clues for Explainable Video Anomaly Detection,” 2026.
[45] Y. Kim, T. Kim, W. Lee, and M. Cho, “ASTRA: Joint Latent Representation and Knowledge Transfer for Multimodal Anomaly Detection,” IEEE Access, 2026.

Mr. Sreekanth K U, Mr. Adwaid R S (2026). A Comprehensive Review of Multimodal Weakly Supervised Video Anomaly Detection. International Journal of Computer Techniques, 13(5), 592–604. ISSN: 2394-2231. DOI: https://doi.org/10.5281/zenodo.23810210