Context-Aware Multimodal Voice Assistant for Autonomous Daily Navigation of Visually Impaired Users Using Edge Intelligence | IJCT Volume 13 – Issue 2 | IJCT-V13I2P83
International Journal of Computer Techniques
ISSN 2394-2231
Author
Dr. K. Sundara Velrani, Keerthana Kamalakannan, Jeysri K, Janani A, Hariram S
Abstract
The use of assistive navigation by the visually impaired needs solutions that are real-time, context-sensitive, and dependable; nevertheless, currently, the voice-based assistants used have a high latency rate, little contextual knowledge, and rely on the cloud. The paper presents a context-based multimodal voice assistant, which uses edge intelligence in the context of au- tonomous day-to-day navigation. The model combines YOLOv8, CNN with MobileNetV3 to detect objects in real-time and a Bi-LSTM-based voice intent recognition model. A Multimodal Fusion Transformer (MFT) is used to combine audio, visual and contextual features to make adaptive decisions and sensor fusion of GPS, accelerator, and gyroscope data is used to achieve contextual awareness. This system uses TensorFlow Lite to deploy the deployed system, which supports low-latency (less than 120 ms) and offline usage, and requires the directional feedback of Spatial Audio Rendering. The model has an accuracy of
97.2 percent, precision of 95.8 percent and F1-score of 96.4 percent, which is better than traditional assistive systems, as well as a 30-percent lower response latency. Experimentation has shown higher efficiency and safety of navigation by the user in the real world. The given system indicates that multimodal learning combined with edge computing can greatly improve the consistency, responsiveness, and usability of assistive technologies by people with visual impairment.
Keywords
Assistive technology, Context-Aware Systems, Multimodal Fusion, Edge Computing, voice assistants, visually impaired navigation, object detection, sensor fusion, real-time systems, human–computer interaction.
Conclusion
The study introduces a context-sensitive multimodal voice assistant, which integrates YOLOv8, CNN and MobileNetV3, Bi-LSTM based voice intent detection and a Multimodal Fu- sion Transformer (MFT) to provide autonomous daily naviga- tion to the visually impaired consumer. GPS, accelerator, and gyroscope sensor fusion guarantees the contextual awareness and TensorFlow Lite deployment is compatible with offline operation (low-latency ¡120 ms). Findings indicate that the system is more accurate, precise, recalls and retrieves more compared to traditional voice assistants with a 97.2% accuracy, 95.8% precision, 97% recall, and 96.4% F1-score as well as less responsive by 30%. Spatial Audio Rendering offers dependable and readable directions, which improves safety and effectiveness. Future directions involve experiments with large, heterogeneous data, adaptive learning, interactive feedback, real-time 3D mapping, and wearable or AR-induced prompts to make it even easier and more autonomous to the users with visual impairments.
References
[1]P. Chitra, V. Balamurugan, M. Sumathi, N. Mathan, K. Srilatha and R. Narmadha, ”Voice Navigation Based guiding Device for Visually Im- paired People,” 2021 International Conference on Artificial Intelligence and Smart Systems (ICAIS), Coimbatore, India, 2021, pp. 911-915, doi: 10.1109/ICAIS50930.2021.9395981.
[2]P. Bhosle, P. Pal, V. Khobragade, S. K. Singh and P. Kenekar, ”Smart Navigation System Assistance for Visually Impaired People,” 2022 In- ternational Conference on Futuristic Technologies (INCOFT), Belgaum, India, 2022, pp. 1-5, doi: 10.1109/INCOFT55651.2022.10094458.
[3]R. Suryawanshi, A. Arudra, D. G. V, M. Uthaman, S. P. M and
V. S. Prasad, ”Innovations in Voice Navigation Using Object De- tection to Empower the Visually Impaired,” 2023 3rd International Conference on Smart Generation Computing, Communication and Net- working (SMART GENCON), Bangalore, India, 2023, pp. 1-6, doi: 10.1109/SMARTGENCON60755.2023.10441962.
[4]S. Phatangare, S. Rajeshirke, P. Mule, T. Pawar and S. Morey, ”Naviga- tion Assistance for Visually Impaired People using Machine Learning,” 2024 15th International Conference on Computing Communication and Networking Technologies (ICCCNT), Kamand, India, 2024, pp. 1-8, doi: 10.1109/ICCCNT61001.2024.10724290.
[5]R. Kambhampati et al., ”AI Enabled Voice Assistant for Visually Impaired,” 2024 15th International Conference on Computing Commu- nication and Networking Technologies (ICCCNT), Kamand, India, 2024,
pp. 1-4, doi: 10.1109/ICCCNT61001.2024.10724542.
[6]A. M. Norkhalid, M. A. Faudzi, A. A. Ghapar and F. A. Rahim, ”Mobile Application: Mobile Assistance for Visually Impaired People – Speech Interface System (SIS),” 2020 8th International Conference on Information Technology and Multimedia (ICIMU), Selangor, Malaysia, 2020, pp. 329-333, doi: 10.1109/ICIMU49871.2020.9243450.
[7]S. P, S. N, P. D and U. M. R. N, ”BLIND ASSIST : A One Stop
Mobile Application for the Visually Impaired,” 2021 IEEE Pune Section International Conference (PuneCon), Pune, India, 2021, pp. 1-4, doi: 10.1109/PuneCon52575.2021.9686476.
[8]T. Castro, J. C. Silva and M. Pinheiro, ”WalkTogether – Mobile Application to Enhance Blind People Accessibility: System Design,” 2021 21st International Conference on Computational Science and Its Applications (ICCSA), Cagliari, Italy, 2021, pp. 174-180, doi: 10.1109/ICCSA54496.2021.00032.
[9]Y. Gu, Z. Yu, Z. Shao and W. Wang, ”A Convenient and Efficient Scene Shop Recognition Assistant Tool for the Visually Impaired,” 2022 IEEE Asia-Pacific Conference on Image Processing, Electron- ics and Computers (IPEC), Dalian, China, 2022, pp. 90-94, doi: 10.1109/IPEC54454.2022.9777308.
[10]F. Song, Z. Li, B. Clark, D. Grooms and C. Liu, ”Camera-Based Indoor Navigation in Known Environments with ORB for People with Visual Impairment,” 2020 IEEE Global Humanitarian Technol- ogy Conference (GHTC), Seattle, WA, USA, 2020, pp. 1-8, doi: 10.1109/GHTC46280.2020.9342876.
[11]Z. Yang et al., ”SeeWay: Vision-Language Assistive Navigation for the Visually Impaired,” 2022 IEEE International Conference on Systems, Man, and Cybernetics (SMC), Prague, Czech Republic, 2022, pp. 52- 58, doi: 10.1109/SMC53654.2022.9945087.
[12]X. Hu, A. Song, H. Zeng and D. Chen, ”Intuitive Environmental Percep- tion Assistance for Blind Amputees Using Spatial Audio Rendering,” in IEEE Transactions on Medical Robotics and Bionics, vol. 4, no. 1, pp. 274-284, Feb. 2022, doi: 10.1109/TMRB.2022.3146743.
[13]F. F. Neha and K. H. Shakib, ”Development of a Smartphone-based Real Time Outdoor Navigational System for Visually Impaired People,” 2021 International Conference on Information and Communication Technol- ogy for Sustainable Development (ICICT4SD), Dhaka, Bangladesh, 2021, pp. 305-310, doi: 10.1109/ICICT4SD50815.2021.9397011.
[14]A. Badave, R. Jagtap, R. Kaovasia, S. Rahatwad and S. Kulkarni, ”Android Based Object Detection System for Visually Impaired,” 2020 International Conference on Industry 4.0 Technology (I4Tech), Pune, India, 2020, pp. 34-38, doi: 10.1109/I4Tech48345.2020.9102694.
A. Pardasani, P. N. Indi, S. Banerjee, A. Kamal and V. Garg, ”Smart Assistive Navigation Devices for Visually Impaired People,” 2019 IEEE 4th International Conference on Computer and Com- munication Systems (ICCCS), Singapore, 2019, pp. 725-729, doi: 10.1109/CCOMS.2019.8821654.
How to Cite This Paper
Dr. K. Sundara Velrani, Keerthana Kamalakannan, Jeysri K, Janani A, Hariram S (2026). Context-Aware Multimodal Voice Assistant for Autonomous Daily Navigation of Visually Impaired Users Using Edge Intelligence. International Journal of Computer Techniques, 13(2). ISSN: 2394-2231.
Context-Aware Multimodal Voice Assistant for Autonomous Daily Navigation of Visually Impaired Users Using Edge IntelligenceDownload
Related Posts:
