Federated Learning (FL) has become a critical paradigm for enabling privacy-preserving distributed intelli-gence; however, existing methods such as Federated Averaging (FedAvg), Federated Knowledge Distillation (FedKD), and Model-Contrastive Federated Learning (MOON) largely assume homo-geneous model architectures or lack robust adaptive strategies for reliable knowledge transfer, limiting their applicability in realistic heterogeneous and cross-framework environments. To overcome these limitations, this paper presents SmartFL, a secure, scalable, and universal heterogeneous federated learning framework designed to support seamless collaboration across diverse model architectures and machine learning frameworks, including PyTorch, TensorFlow/Keras, and Scikit-learn. SmartFL introduces a novel hybrid learning paradigm that intelligently integrates partial weight aggregation for structurally compatible models with an adaptive, confidence-aware cross-architecture knowledge distillation mechanism for heterogeneous scenarios, ensuring effective and reliable knowledge transfer. To further enhance robustness, a validation-driven performance preserva-tion strategy selectively incorporates client updates, preventing model degradation and stabilizing convergence. Additionally, SmartFL incorporates secure encrypted communication and model compression techniques to reduce transmission overhead while preserving data confidentiality. A dual-queue asynchronous update mechanism is also proposed to efficiently manage concur-rent client participation, prioritize critical updates, and improve overall system scalability. Extensive experimental evaluations on multi-class image classification benchmarks demonstrate that SmartFL consistently achieves superior or comparable perfor-mance across heterogeneous configurations, while significantly improving knowledge transfer efficiency and training stability. These results establish SmartFL as a practical and high-impact solution for bridging heterogeneous AI systems, advancing secure, efficient, and real-world federated learning deployment.
This paper presents SmartFL, an adaptive heterogeneous federated learning framework capable of improving weak learners while maintaining teacher model stability across di-verse datasets and model architectures. By integrating cross-architecture knowledge distillation, secure communication through compression and encryption, and queue-based scala-bility, SmartFL demonstrates consistent performance improve-ments in both medical imaging and tabular datasets. These re-sults highlight the framework’s potential for real-world multi-institutional healthcare AI, edge computing, and collaborative analytics across heterogeneous systems.
References
[1]H. B. McMahan et al., “Communication-efficient learning of deep networks from decentralized data,” in Proc. AISTATS, 2017.
[2]T. Li, A. K. Sahu, A. Talwalkar, and V. Smith, “Federated optimization in heterogeneous networks,” in Proc. MLSys, 2020.
[3]Q. Yang, Y. Liu, T. Chen, and Y. Tong, “Federated machine learning: Concept and applications,” ACM Trans. Intell. Syst. Technol., 2019.
[4]P. Kairouz et al., “Advances and open problems in federated learning,” Foundations and Trends in Machine Learning, 2021.
[5]D. Li and J. Wang, “Federated learning with heterogeneous models via knowledge distillation,” arXiv preprint arXiv:1910.03581, 2019.
[6]E. Diao et al., “HeteroFL: Computation and communication efficient federated learning for heterogeneous clients,” in Proc. ICLR, 2021.
[7]Y. Lin et al., “Ensemble distillation for robust model fusion in federated learning,” in Proc. NeurIPS Workshops, 2020.
[8]S. Caldas et al., “LEAF: A benchmark for federated settings,” arXiv preprint arXiv:1812.01097, 2018.
[9]G. Hinton, O. Vinyals, and J. Dean, “Distilling the knowledge in a neural network,” arXiv preprint arXiv:1503.02531, 2015.
[10]S. Jeong et al., “Communication-efficient on-device machine learning: Federated distillation and augmentation,” in Proc. NeurIPS Workshops, 2018.
[11]J. Gou et al., “Knowledge distillation: A survey,” Int. J. Comput. Vis., 2021.
[12]Y. Shen et al., “Distilled federated learning,” IEEE Trans. Neural Netw. Learn. Syst., 2022.
[13]Q. Li et al., “Model-contrastive federated learning,” in Proc. CVPR, 2021.
[14]J. Konecˇny´ et al., “Federated learning: Strategies for improving com-munication efficiency,” arXiv preprint arXiv:1610.05492, 2016.
[15]X. Wang et al., “Cross-model federated learning,” IEEE Access, 2020.
[16]A. Fallah, A. Mokhtari, and A. Ozdaglar, “Personalized federated learning with theoretical guarantees,” in Proc. NeurIPS, 2020.
[17]M. Chen et al., “Model-agnostic federated learning,” in Proc. ICML Workshops, 2018.
[18]K. Bonawitz et al., “Practical secure aggregation for privacy-preserving machine learning,” in Proc. ACM CCS, 2017.
[19]M. Abadi et al., “Deep learning with differential privacy,” in Proc. ACM CCS, 2016.
[20]R. Shokri and V. Shmatikov, “Privacy-preserving deep learning,” in Proc. ACM CCS, 2015.
[21]S. Han, H. Mao, and W. J. Dally, “Deep compression: Compressing deep neural networks,” in Proc. ICLR, 2016.
[22]D. Alistarh et al., “QSGD: Communication-efficient SGD via gradient quantization,” in Proc. NeurIPS, 2017.
[23]J. Strom, “Scalable distributed DNN training using commodity GPU cloud computing,” in Proc. Interspeech, 2015.
[24]Y. Zhao et al., “Federated learning with non-IID data,” arXiv preprint arXiv:1806.00582, 2018.
[25]H. Wang et al., “Adaptive federated learning in resource-constrained edge computing systems,” IEEE Trans. Signal Process., 2020.
[26]K. He et al., “Deep residual learning for image recognition,” in Proc. CVPR, 2016.
[27]Z. Liu et al., “Swin transformer: Hierarchical vision transformer using shifted windows,” in Proc. ICCV, 2021.
[28]A. Dosovitskiy et al., “An image is worth 16×16 words: Transformers for image recognition at scale,” in Proc. ICLR, 2021. [29]A. S. Tanenbaum and M. Van Steen, Distributed Systems: Principles and Paradigms, 2nd ed., 2007.
[30]T. Fawcett, “An introduction to ROC analysis,” Pattern Recognition Letters, 2006.
How to Cite This Paper
Dheenadhayalan L, Karthikeyan D P, Vasanthan V (2026). SmartFL: A Secure Universal Heterogeneous Federated Learning Framework with Cross-Architecture Knowledge Distillation and Performance Preservation. International Journal of Computer Techniques, 13(2). ISSN: 2394-2231.
