A Transformer and Deep Embedded Clustering Based Framework for Dynamic Workload Characterization | IJCT Volume 13 – Issue 3 | IJCT-V13I3P115
International Journal of Computer Techniques
ISSN 2394-2231
Author
Rashmi Zambre, Priyanshi Borase
Abstract
Dynamic workload characterization plays a critical role in modern data centers and cloud computing environments, where efficient resource allocation and workload-aware scheduling are essential for maximizing system performance and energy efficiency. Traditional workload characterization approaches based on statistical methods and conventional machine learning models often fail to capture complex temporal dependencies and hidden workload patterns present in large-scale hardware performance datasets. To address this limitation, this paper proposes a novel Transformer and Deep Embedded Clustering (TDEC)-based framework for dynamic workload characterization. The proposed framework integrates a Transformer Encoder with multi-head self-attention for supervised workload classification and Deep Embedded Clustering (DEC) for unsupervised workload grouping and latent feature discovery. The Transformer model effectively captures long-range temporal dependencies in time-series hardware performance metrics, while DEC jointly performs representation learning and clustering to identify hidden workload structures without prior labels. The proposed framework is evaluated using benchmark workload datasets such as SPEC CPU2006 and SPEC CPU2017, utilizing hardware performance metrics collected through EMON monitoring. Experimental results demonstrate that the proposed model significantly improves workload classification accuracy, clustering quality, and computational efficiency compared to conventional CNN + RNN and Autoencoder + K-means based approaches. The findings indicate that the proposed framework provides a scalable, accurate, and intelligent solution for next-generation workload characterization in heterogeneous computing environments.
Keywords
Dynamic Workload Characterization, Transformer Encoder, Deep Embedded Clustering (DEC), Workload Classification, Hardware Performance Metrics, Multi-Head Self-Attention, Representation Learning, Cloud Computing, Resource Allocation, Heterogeneous Computing Environments.
Conclusion
This paper presented a novel Transformer and Deep Embedded Clustering (TDEC)-based framework for dynamic workload characterization in modern computing systems. Unlike traditional deep-learning approaches that rely on CNN + RNN architectures for supervised learning and Autoencoder + K-means for clustering, the proposed framework leverages the superior temporal learning capability of the Transformer Encoder and the joint feature learning and clustering strength of Deep Embedded Clustering. The Transformer model effectively captures both short-term and long-term dependencies in hardware performance sequences through self-attention, while DEC identifies hidden workload groups by learning meaningful latent representations. Experimental evaluation demonstrated that the proposed framework achieves improved workload classification accuracy, faster training performance, and more robust clustering compared to existing methods. Additionally, the model provides enhanced interpretability through attention mechanisms, enabling better understanding of critical workload behaviors. Overall, the proposed approach offers a scalable and intelligent solution for dynamic workload characterization and has strong potential for practical deployment in cloud computing, resource scheduling, and autonomous data center management. Future work will focus on extending the framework to heterogeneous CPU-GPU systems and real-time adaptive workload management environments.
References
[1][1] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention Is All You Need,” in Proc. Advances in Neural Information Processing Systems (NeurIPS), 2017, pp. 5998–6008.
[2][2] J. Devlin, M. W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,” in Proc. NAACL-HLT, 2019, pp. 4171–4186.
[3][3] J. Xie, R. Girshick, and A. Farhadi, “Unsupervised Deep Embedding for Clustering Analysis,” in Proc. ICML, 2016, pp. 478–487.
[4][4] F. Chollet, “Deep Learning with Python,” Manning Publications, 2018.
[5][5] I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning. Cambridge, MA, USA: MIT Press, 2016.
[6][6] T. Chen, H. Li, Q. Yang, and X. Wang, “Transformer-Based Time-Series Representation Learning for Performance Analysis,” IEEE Access, vol. 9, pp. 115432–115445, 2021.
[7][7] Y. Bengio, A. Courville, and P. Vincent, “Representation Learning: A Review and New Perspectives,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 35, no. 8, pp. 1798–1828, Aug. 2013.
[8][8] D. E. Culler and J. P. Singh, Parallel Computer Architecture: A Hardware/Software Approach. Morgan Kaufmann, 1999.
[9][9] J. L. Hennessy and D. A. Patterson, Computer Architecture: A Quantitative Approach, 6th ed. Morgan Kaufmann, 2019.
[10][10] SPEC CPU2006 Benchmark Suite, Standard Performance Evaluation Corporation (SPEC), 2006.
[11][11] SPEC CPU2017 Benchmark Suite, Standard Performance Evaluation Corporation (SPEC), 2017.
[12][12] M. Ferdman et al., “Clearing the Clouds: A Study of Emerging Scale-Out Workloads on Modern Hardware,” in Proc. ASPLOS, 2012, pp. 37–48.
[13][13] C. Delimitrou and C. Kozyrakis, “Quasar: Resource-Efficient and QoS-Aware Cluster Management,” in Proc. ASPLOS, 2014, pp. 127–144.
[14][14] C. Delimitrou and C. Kozyrakis, “Paragon: QoS-Aware Scheduling for Heterogeneous Datacenters,” in Proc. ASPLOS, 2013, pp. 77–88.
[15][15] B. Hindman et al., “Mesos: A Platform for Fine-Grained Resource Sharing in the Data Center,” in Proc. NSDI, 2011, pp. 295–308.
[16][16] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet Classification with Deep Convolutional Neural Networks,” Communications of the ACM, vol. 60, no. 6, pp. 84–90, 2017.
[17][17] S. Hochreiter and J. Schmidhuber, “Long Short-Term Memory,” Neural Computation, vol. 9, no. 8, pp. 1735–1780, 1997.
[18][18] K. He, X. Zhang, S. Ren, and J. Sun, “Deep Residual Learning for Image Recognition,” in Proc. CVPR, 2016, pp. 770–778.
[19][19] T. N. Sainath, O. Vinyals, A. Senior, and H. Sak, “Convolutional, Long Short-Term Memory, Fully Connected Deep Neural Networks,” in Proc. ICASSP, 2015, pp. 4580–4584.
[20][20] M. Zaharia et al., “Apache Spark: A Unified Engine for Big Data Processing,” Communications of the ACM, vol. 59, no. 11, pp. 56–65, 2016.
How to Cite This Paper
Rashmi Zambre, Priyanshi Borase (2026). A Transformer and Deep Embedded Clustering Based Framework for Dynamic Workload Characterization. International Journal of Computer Techniques, 13(3). ISSN: 2394-2231.
