Ensuring software reliability through early defect detection and prevention is increasingly important as software systems grow more complex. Automated testing has emerged as an effective solution for producing reliable and efficient code, while machine learning-based approaches, particularly those leveraging natural language models, have gained signifi-cant attention for building advanced defect prediction systems. In this paper, we propose a novel framework for automated software defect prediction that targets eight specific defect types: SIGFPE, NZEC, LOGICAL, SYNTAX, SIGSEGV, SIGABRT,
SEMANTIC, and LINKER. The study utilizes a custom dataset comprising nine classes, including eight categories of program-ming errors and one representing error-free code, with the aim of improving defect detection in code snippets and enhancing software development processes. The proposed method employs a CodeBERT-based model with optimized hyperparameters to achieve superior predictive performance. Comparative evaluation against established models such as RoBERTa, Microsoft Code-BERT, and GPT-2 demonstrates that the proposed approach achieves notable improvements, with accuracy gains of up to 20% in binary classification and 7% in multi-class classification. Experimental findings further validate the effectiveness of neural language models, particularly CodeBERT, in software defect prediction, highlighting their potential to significantly enhance software testing practices and overall code quality within the software development lifecycle.
Keywords
Software Defect Prediction, CodeBERT, Ma-chine Learning, Deep Learning, Natural Language Processing, Automated Testing, Multi-Class and Binary Classification, Soft-ware Reliability, Defect Classification, Code Analysis, Neural Language Models
Conclusion
This study presents an innovative approach to software defect prediction through the use of a CodeBERT-based model, referred to as MSDP. The proposed method demonstrates strong capability in identifying defects across eight commonly occurring categories in software development, thereby contributing to more efficient and effective software testing processes. The experimental setup involved the construction of both binary and multi-class datasets, primarily utilizing code samples derived from C++ programs.
The experimental results highlight the effectiveness of leveraging the pre-trained CodeBERT model to enhance both prediction accuracy and overall productivity. In the case of balanced binary classification, the proposed model achieved an approximate 20% improvement in accuracy compared to existing approaches, successfully distinguishing between ‘Error’ and ‘NoError’ classes. Although the model showed a slight bias toward predicting ‘Error’ instances—likely due to the inherent complexity of error patterns—it still maintained strong overall performance.
When applied to imbalanced datasets, the model exhibited a reduction in overall accuracy; however, it continued to reliably identify both classes, demonstrating resilience to class imbalance and adaptability in challenging scenarios.
For multi-class classification, the model achieved an improvement of approximately 7% in accuracy over baseline
methods, indicating moderate yet meaningful gains. The proposed methodology shows considerable potential in reducing development time while improving the quality of software systems. Furthermore, the analysis of patterns extracted from concise code snippets contributed to improved prediction outcomes, emphasizing the model’s effectiveness in precise defect detection.
Overall, the findings indicate a 20% increase in accuracy for binary classification and a 7% improvement for multi-class classification. The model’s capability to handle imbalanced datasets and its inclination toward detecting error-prone instances reflect the inherent complexity of software defect patterns. In practical applications, the MSDP model enhances software testing efficiency and contributes to the development of higher-quality software systems.
The proposed approach improves software quality by enabling precise defect detection and targeted remediation, which helps in optimizing resource utilization and accelerating the overall development lifecycle.
Currently, our focus is on expanding the dataset to obtain more diverse and comprehensive data, allowing for deeper analysis, improved prediction performance, and broader contributions to software engineering research. A key objective is to enhance defect prediction accuracy by introducing both generic and subclass-level defect classification.
By organizing defects into high-level generic categories and further dividing them into more specific subclasses, we can achieve a detailed understanding of the variety of defects present within a codebase. This fine-grained classification facilitates accurate identification of root causes and supports more effective corrective actions. Additionally, subclass-level categorization enables the design of specialized models tailored to specific defect types, utilizing domain-specific insights to further improve prediction accuracy.
Examining the distribution and characteristics of defects across both generic and subclass categories can provide valuable insights into frequent programming mistakes, potential areas for improving development practices, and evolving trends in software quality. Such insights support better decision-making during development, ultimately leading to more reliable defect prediction and higher-quality software systems.
To further enhance the model, we are exploring the integration of additional attention mechanisms into the existing architecture, as well as the incorporation of complementary neural network techniques to improve performance.
References
[1]A. Kumar, A. Singh, A. K. Nayak, S. Prakash, D. Pradhan, and
J. K. Mishra, “An extreme gradient boosting based automated software defect prediction model using dimensionality reduction technique,” in Computing, Communication and Intelligence, pp. 40–43, CRC Press, 2026.
[2]R. Tamilkodi, P. S. Rani, M. D. Sirisha, G. L. Divya, K. Sagar, and A. Raghuvamsi, “Predicting software defects with an intelligent ensemble-based engine,” in 2025 International Conference on Next Gen-eration Communication & Information Processing (INCIP), pp. 419–424, IEEE, 2025.
[3]L.-f. Chen, K.-x. Cao, S.-p. Zhang, and Q. Dai, “Bagging gradient near-est neighbor-based ivy algorithm optimized svm ensemble learning for heterogeneous software defect prediction,” Cluster Computing, vol. 29, no. 1, p. 72, 2026.
[4]X. Fan, J. Mao, L. Lian, Y. Li, W. Zheng, and Y. Ge, “Software defect prediction method based on stable learning,” Computers, Materials, & Continua, vol. 78, no. 1, p. 65, 2024.
[5]A. Abdu, Z. Zhai, H. A. Abdo, et al., “Semantic and traditional feature fusion using hybrid cnn-mlp,” Scientific Reports, vol. 14, 2024.
[6]M. Ali, T. Mazhar, Y. Arif, et al., “Software defect prediction using ensemble-based model,” IEEE Access, vol. 12, pp. 20376–20395, 2024.
[7]Y. Zhang and N. Liu, “Investigation and research on several key issues of software defect prediction,” IET Software, vol. 2025, no. 1, p. 6615496, 2025.
[8]M. Allamanis, M. Brockschmidt, and M. Khademi, “Learning to repre-sent programs with graphs,” in ICLR, 2018.
[9]U. Alon, M. Zilberstein, O. Levy, and E. Yahav, “Code2vec: Learning distributed representations of code,” Proc. ACM Programming Lan-guages, vol. 3, 2019.
[10]O¨ . F. Arar and K. Ayan, “Software defect prediction using cost-sensitive neural network,” Applied Soft Computing, vol. 33, pp. 263–277, 2015.
[11]X. Chen, D. Zhang, Y. Zhao, Z. Cui, and C. Ni, “Software defect predic-tion: Unsupervised vs supervised methods,” Information and Software Technology, vol. 106, pp. 161–181, 2019.
[12]S. McMurray and A. H. Sodhro, “Ml-based software defect detection for smart healthcare,” Sensors, vol. 23, no. 7, p. 3470, 2023.
[13]M. Choi, S. Jeong, H. Oh, and J. Choo, “End-to-end prediction of buffer overruns from raw source code via neural memory networks,” in IJCAI,
pp. 1546–1553, 2017.
[14]Y. H. Choi and J. Nam, “Wine: Warning miner for improving bug finders,” Information and Software Technology, vol. 155, 2023.
[15]D. Demirci, N. Sahin, M. Sirlancis, and C. Acarturk, “Static malware detection using stacked bilstm and gpt-2,” IEEE Access, vol. 10,
pp. 58488–58502, 2022.
[16]X. Dong, J. Wang, and Y. Liang, “A novel ensemble classifier selection method for software defect prediction,” IEEE Access, vol. 13, pp. 25578–25597, 2025.
[17]Z. Feng, D. Guo, D. Tang, N. Duan, X. Feng, M. Gong, L. Shou, T. Liu,
D.Jiang, and M. Zhou, “Codebert: A pre-trained model for programming and natural languages,” in Findings of EMNLP, pp. 1536–1547, 2020.
[18]Z. Li, D. Zou, S. Xu, X. Ou, H. Jin, S. Wang, Z. Deng, and Y. Zhong, “Vuldeepecker: A deep learning-based system for vulnerability detec-tion,” in NDSS, 2018. [19]B. Isong and E. Igo, “Ensemble learning for software defect prediction: Performance, practicality and future directions,” Journal of Information Systems and Informatics, vol. 7, no. 3, pp. 2245–2291, 2025.
[20]T. Zhou, X. Sun, X. Xia, B. Li, and X. Chen, “Improving defect prediction with deep forest,” Information and Software Technology, vol. 114, pp. 204–216, 2019.
[21]H. Z. Zaidi, U. Ullah, M. Arshad, H. Aljuaid, M. A. Rauf, N. Sarwar, and
R. Sajid, “Machine learning approaches for software defect prediction,” Applied Computational Intelligence and Soft Computing, vol. 2025, no. 1, p. 7933078, 2025.
[22]M. Pradel and K. Sen, “Deepbugs: A learning approach to name-based bug detection,” Proc. ACM Programming Languages, vol. 2, 2018.
[23]S. Kwon, J.-I. Jang, S. Lee, D. Ryu, and J. Baik, “Codebert based software defect prediction for edge-cloud systems,” in International Conference on Web Engineering, pp. 11–21, Springer, 2022.
[24]Z. Li, H. Zhang, Z. Jin, and G. Li, “Well: Applying bug detectors to bug localization via weakly supervised learning,” 2023. arXiv:2305.17384.
[25]S. Mehta and K. S. Patnaik, “Improved prediction of software defects using ensemble machine learning techniques,” Neural Computing and Applications, vol. 33, no. 16, pp. 10551–10562, 2021.
[26]S. Sahar, M. Younas, M. M. Khan, and M. U. Sarwar, “Dp-ccl: A supervised contrastive learning approach using codebert model in software defect prediction,” IEEE Access, vol. 12, pp. 22582–22594, 2024.
[27]J. K. Mishra, D. Pradhan, C. R. Sahoo, R. S. Jyothi, and L. Das, “An automated software defect prediction model using machine learning approaches,” in International Conference on Machine Learning, IoT and Big Data, pp. 176–187, Springer, 2025.
[28]K. Shi, Y. Lu, J. Chang, and Z. Wei, “Pathpair2vec: An ast path pair-based code representation method for defect prediction,” Journal of Computer Languages, vol. 59, 2020.
[29]C. Pan, M. Lu, and B. Xu, “An empirical study on software defect prediction using codebert model,” Applied Sciences, vol. 11, no. 11,
p. 4793, 2021.
[30]J. Zhao, S. Guo, and D. Mu, “Doubigru-a: Software defect detection using attention and bigru,” Computers & Security, vol. 111, 2021.
[31]X. Yang, L. Xiao, J. Su, and B. Huang, “Coco-gan: Codebert-driven collaborative generative adversarial learning for software defect predic-tion,” Software Quality Journal, vol. 34, no. 2, p. 12, 2026.
How to Cite This Paper
Baidehi Jena, Debasish Pradhan, Dolagovinda Mahanta (2026). A CodeBERT-Driven Framework for Multi-Class Software Defect Prediction Using Defect Categorization. International Journal of Computer Techniques, 13(3). ISSN: 2394-2231.
