Sign language serves as a vital communication medium for individuals with hearing and speech impairments; however, limited awareness among the general population often leads to communication barriers and social exclusion. This project presents an intelligent computer vision–based framework for real-time hand gesture recognition .in sign language. The proposed system employs a vision-based approach using a standard webcam to capture hand gestures and utilizes MediaPipe for accurate hand landmark detection and feature extraction. These landmarks are processed to generate skeleton-based representations that minimize the impact of background variations and lighting conditions. Support Vector Machine is trained to classify sign language gestures, focusing on alphabetlevel recognition. To improve classification accuracy, gestures are strategically grouped into multiple classes and further distinguished using geometric relationships between landmarks. The system is implemented with an interactive graphical user interface, enabling real-time gesture recognition with high accuracy and robustness. This framework demonstrates an efficient, cost-effective, and userfriendly solution for automated sign language recognition, contributing to more inclusive human–computer interaction and improved accessibility for the hearing- and speechimpaired community.
Keywords
Sign Language Recognition , Hand Gesture Recognition, Computer Vision, Real-Time Gesture Classification , Support Vector machine
Conclusion
This project presents an intelligent Sign Language Communication System that helps reduce the communication gap between individuals who use sign language and those who do not understand it. By using computer vision and machine learning techniques, the system is able to recognize hand gestures and convert them into meaningful text in real time. The use of MediaPipe for hand landmark detection and an SVM model for gesture classification allows the system to identify sign language alphabets effectively. This approach makes the system practical and accessible since it only requires a standard webcam rather than specialized hardware. Another important aspect of the project is the integration of Natural Language Processing techniques that enhance the generated sentences through spell correction and grammar refinement. This ensures that the output is not only recognized correctly but also presented in a more readable and understandable form. In addition to gesture recognition, the system also provides useful features such as text-tospeech conversion and multilingual translation, allowing users to communicate in different languages and formats. These additional functionalities make the system more flexible and useful for real-world communication scenarios. Overall, the developed system demonstrates how modern technologies such as computer vision, machine learning, and natural language processing can be combined to create an inclusive communication tool. The project provides a simple and cost- effective solution that supports both sign-to-text and text-to- sign interaction, making communication easier for people with hearing or speech impairments. With further improvements and expanded gesture datasets, the system has the potential to become a more advanced assistive technology that can be widely used in educational, social, and professional environments
References
1.Simonyan, K., & Zisserman, A. (2015). Very Deep Convolutional Networks for Large-Scale Image Recognition. International Conference on Learning Representations (ICLR).
2.He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep Residual Learning for Image Recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
3.Zhang, F., Bazarevsky, V., Vakunov, A., et al. (2020). MediaPipe Hands: On-device Real-time Hand Tracking. arXiv preprint arXiv:2006.10214.
4.Krizhevsky, A., Sutskever, I., & Hinton, G. (2012). ImageNet Classification with Deep Convolutional Neural Networks. Advances in Neural Information Processing Systems (NeurIPS).
5.Molchanov, P., Gupta, S., Kim, K., & Kautz, J. (2015). Hand Gesture Recognition with 3D Convolutional Neural Networks. IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).
6.Neverova, N., Wolf, C., Taylor, G., & Nebout, F. (2014). Multi-scale Deep Learning for Gesture Detection and Localization. European Conference on Computer Vision Workshops (ECCVW).
7.Freeman, W. T., & Roth, M. (1995). Orientation Histograms for Hand Gesture Recognition. International Workshop on Automatic Face and Gesture Recognition.
8.Wang, R. Y., & Popović, J. (2009). Real-Time HandTracking with a Color Glove. ACM Transactions on Graphics.
9.Pigou, L., Dieleman, S., Kindermans, P., & Schrauwen,
B. (2015). Sign Language Recognition Using Convolutional Neural Networks. European Conference on Computer Vision Workshops (ECCVW).
10.Ohn-Bar, E., & Trivedi, M. (2014). Hand Gesture Recognition in Real Time for Automotive Interfaces. IEEE Transactions on Intelligent Transportation Systems.
11.Dalal, N., & Triggs, B. (2005). Histograms of Oriented Gradients for Human Detection. IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
12.Bishop, C. M. (2006). Pattern Recognition and Machine Learning. Springer.
13.Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep Learning. MIT Press.
14.Shotton, J., Fitzgibbon, A., Cook, M., et al. (2011). RealTime Human Pose Recognition in Parts from Single Depth Images. IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 15.Mitra, S., & Acharya, T. (2007). Gesture Recognition: A Survey. IEEE Transactions on Systems, Man, and Cybernetics.
16.Cao, Z., Hidalgo, G., Simon, T., Wei, S. E., & Sheikh, Y. (2019). OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI).
17.Vaswani, A., Shazeer, N., Parmar, N., et al. (2017). Attention Is All You Need. Advances in Neural Information Processing Systems (NeurIPS).
18.Hochreiter, S., & Schmidhuber, J. (1997). Long ShortTerm Memory. Neural Computation.
19.Molchanov, P., Yang, X., Gupta, S., Kim, K., Tyree, S., & Kautz, J. (2016). Online Detection and Classification of Dynamic Hand Gestures with Recurrent 3D CNNs. IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
20.Oyedotun, O. K., & Khashman, A. (2017). Deep Learning in Vision-Based Static Hand Gesture Recognition. Neural Computing and Applications.
21.Huang, J., Zhou, W., & Li, H. (2018). Sign Language Recognition Using 3D Convolutional Neural Networks. IEEE International Conference on Multimedia and Expo (ICME).
22.Pigou, L., Dieleman, S., Kindermans, P. J., & Schrauwen, B. (2018). Beyond Temporal Pooling: Recurrence and Temporal Convolutions for Gesture Recognition. International Journal of Computer Vision.
23.Neverova, N., Wolf, C., Taylor, G. W., & Nebout, F. (2016). ModDrop: Adaptive Multi-Modal Gesture Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence.
24.Zhang, X., Liu, Z., & Sun, L. (2020). Hand Gesture Recognition Based on Deep Learning: A Review. Journal of Computer Science and Technology.
25.Chen, X., Wang, G., Guo, H., & Zhang, C. (2019). PoseGuided Hand Gesture Recognition Using CNN. Pattern Recognition Letters.
26.Koller, O., Ney, H., & Bowden, R. (2015). Deep Hand: How to Train a CNN on 1 Million Hand Images for Gesture Recognition. IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).
27.Wan, J., Escalera, S., Guyon, I., et al. (2016). Results and Analysis of ChaLearn LAP Multi-Modal Isolated and Continuous Gesture Recognition. IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).
28.Mittal, A., Zisserman, A., & Torr, P. H. S. (2011). Hand Detection Using Multiple Proposals. British Machine Vision Conference (BMVC).
29.Liu, J., Shahroudy, A., Xu, D., et al. (2017). SkeletonBased Action Recognition Using Spatio- Temporal LSTM Networks. IEEE International Conference on Computer Vision (ICCV).
Yang, W., Zhang, C., & Tian, Y. (2019). Deep Learning for Vision-Based Hand Gesture Recognition: A Survey. IEEE Acces
How to Cite This Paper
P.Suresh, P.Prathap, P.Rohith, Mr.P.Jaya Chandran (2026). An Intelligent Computer Vision Framework for Hand Gesture Recognition in Sign Language. International Journal of Computer Techniques, 13(3). ISSN: 2394-2231.
