Dr Asha N, Dr Pushpalatha M, Dr Sowmya S B

DOI: https://doi.org/10.5281/zenodo.22031548

Real-world datasets are often characterised by noise, missing values, inconsistencies, and anomalies, which can adversely affect the performance of data mining and machine learning algorithms. Therefore, data preprocessing is an essential step in ensuring data quality and improving the efficiency and accuracy of the analytical process. The time-series pollution dataset is initially preprocessed to ensure data quality and consistency before applying statistical or machine-learning techniques. The preprocessing begins with the identification and treatment of missing or invalid observations, represented as NaN (Not a Number) values. The paper emphasis on incorporating Appropriate imputation techniques to handle missing observations while preserving the temporal characteristics of the dataset. Also, outliers and anomalous observations are detected and treated using suitable statistical methods. Pollution data exhibit extreme observations due to sudden variations in pollutant concentrations, and their distribution may be left-skewed or right-skewed. Therefore, statistical measures such as the Interquartile Range (IQR) are employed to identify and appropriately handle abnormal observations. These preprocessing procedures help produce a reliable and consistent hourly time-series dataset, thereby improving the quality of subsequent analysis and the accuracy of pollution and AQI prediction models. The paper proposes an efficient Data preprocessing framework for time-series air pollution datasets.

Data Preprocessing, Time series Data set, Not a Number(NaN), Interquartile range, Air pollution dataset.

The proposed framework employs statistical data preprocessing techniques to improve the quality, consistency, and reliability of air pollution datasets before predictive modelling. Although the framework is primarily designed for time-series air pollution data, the underlying preprocessing procedures can be adapted and applied to other structured datasets with similar data-quality issues, such as missing values, outliers, inconsistencies, and non-normal distributions. By systematically identifying and treating these data-quality problems, the proposed framework can provide a more reliable input dataset for statistical and machine-learning models, thereby contributing to improved predictive accuracy and reduced prediction error across different application domains.

Wang, L., Zeng, Y., & Chen, T,” Back propagation neural network with adaptive differential evolution algorithm for time series forecasting”, Expert Systems with Applications, Volume 42, pp.855–863, http://doi.org/10.1016/j.eswa.2014.08.018.
[2] Hylleberg, S, “Modelling seasonality”, (1st edition.) Oxford: Oxford University Press.
[3] Makridakis, S., Andersen, A., Carbone, R., Fildes, R., Hibon, M., Lewandowski, R., … Winkler, R, “The accuracy of extrapolation (time series) methods: Results of a forecasting competition”, Journal of Forecasting, Volume 1(2), pp. 111–153. DOI:10.1002/for.3980010202.
[4] Fernández Palomares, N., Álvarez de Prado, L., Menéndez García, L. A., Fernández López, D., Buján, S., & Bernardo Sánchez, A. (2026). “A Reproducible QA/QC, Imputation and Robust-Series Workflow for Air-Quality Monitoring Time Series”, Applied Sciences, 16(7), 3396. https://doi.org/10.3390/app16073396
[5] Khatri P, Shakya KS, Kumar P. “A probabilistic framework for identifying anomalies in urban air quality data”, Environ Sci Pollut Res Int. 2024 Oct;31(49):59534-59570. doi: 10.1007/s11356-024-35006-x. Epub 2024 Oct 2. PMID: 39358655.
[6] Jayaraman, S., T, N., S, A. et al. “Enhancing urban air quality prediction using time-based-spatial forecasting framework”. Sci Rep 15, 4139 (2025). https://doi.org/10.1038/s41598-024-83248-z
[7] Ma Z, Luo W, Jiang J, Wang B, Ma Z, et al. (2023) “Spatial and temporal characteristics analysis and prediction model of PM2.5 concentration based on SpatioTemporal-Informer model”, PLOS ONE 18(6): e0287423. https://doi.org/10.1371/journal.pone.0287423
[8] B.S. Freeman, G. Taylor, B. Gharabaghi, J. “Thé Forecasting air quality time series using Deep learning”, J, Air Waste Manage. Association. 68 (8) (2018) 866–886.
[9] C. C. Aggarwal and P. S. Yu, "Outlier detection for high dimensional data”, ACM SIGMOD International Conference on Management of data (ACM Sigmod 2001), ACM, June 2001, pp. 37- 46, DOI: l0.1145/375663.375668.
[10] X. Zhu and X. Wu, "Class noise vs. attribute noise: a quantitative study of their impacts", Artificial Intelligence Review, Volume. 22, no 3, November 2004, pp. 177-210, DOI: http://doi.org/10.1 007/s 10462-004-0751-8
[11] S. Chen, W. Wang, and H. van Zuylen, "A comparison of outlier detection algorithms for ITS data", Expert Systems with Applications, Volume. 37, no 2, March 2010, pp. 1169-1178, DOI: http://doi.org/10. 10 16/j. eswa. 2009. 06. 008.
[12] Antonio Javier talon Ballesteros, Jose Cristobal Riquelme Santos, “Deleting or Keeping Outliers for Classifier Training?”, Institute of Electrical and Electronics Engineers (IEEE),2015, DOI:c10.1109/nabic.2014.6921892
[13] Kaur, K., Garg, A, “Comparative study of outlier detection algorithms”, IJCA, Volume 147(9), (2016).
[14] J. Brownlee, “Time series forecasting as supervised learning”, Machine Learning”, Mastery,2020, https://machinelearningmastery.com/time-seriesforecasting-supervised-learning/
[15] H. P. Vinutha, B. Poornima and B. M. Sagar “Detection of Outliers Using Interquartile Range Technique from Intrusion Dataset”, Information and Decision Sciences, Advances in Intelligent Systems and Computing 701, 2018, DOI:10.1007/978-981-10-7563-6_53
[16] Mia Hubert and Stephan Van der Veeken “Outlier detection of skewed data”, Article  in  Journal of Chemometrics, March 2008, DOI: 10.1002/cem.1123

Dr Asha N, Dr Pushpalatha M, Dr Sowmya S B (2026). An Efficient Data Preprocessing Framework for Time-Series Air Pollution Data. International Journal of Computer Techniques, 13(4), 236–243. ISSN: 2394-2231. DOI: https://doi.org/10.5281/zenodo.22031548