Arati Patil, Pritam Chaudhari, Ajinkya Shingne, Rohit Nalandwar, Ashish Pawar

DOI: https://doi.org/10.5281/zenodo.24128829

Deep-learning vision models need large labelled datasets, and collecting and annotating real images is slow and expensive. Synthetic data from simulators and rendering pipelines offers a scalable alternative, but models trained only on synthetic images often lose accuracy on real ones, a problem known as the synthetic-to-real domain gap. Domain randomization (DR) attacks the gap by varying rendering parameters such as lighting, textures, backgrounds, camera viewpoint, and object pose so widely that the real world looks like one more variation of the training data. This paper is a critical narrative review of how DR affects synthetic-to-real performance, mainly in object detection. It compares ten studies on their synthetic-data approach, randomization factors, architectures, metrics, findings, and limitations. Across these studies, models trained only on randomized synthetic data reached useful real-world accuracy in some constrained settings, and synthetic pretraining followed by a small amount of real-data fine-tuning often beat real-only training, although the margin can be small when real data is plentiful. Textures, materials, lighting, post-processing, and scene context are each reported as important in individual studies, but the rankings disagree, and differences in datasets, detectors, and metrics prevent a universal ordering of randomization factors. One study also shows that near-perfect synthetic validation scores can sit alongside noticeably lower real-world scores, so synthetic validation alone should not be trusted as a proxy for deployment performance. The review points to gaps in standardized evaluation, controlled factor-level ablation, and reproducibility.

domain randomization, synthetic data, sim-to-real transfer, deep learning, computer vision, object detection, domain adaptation

Domain randomization is a cost-effective way to narrow the synthetic-to-real gap in object detection. By exposing networks to non-photorealistic variation in textures, lighting, backgrounds, and viewpoint during data generation, it can produce detectors that transfer to physical scenes, and in several constrained industrial settings synthetic-only training has delivered strong results. The evidence reviewed here suggests that mixed training, meaning synthetic pretraining followed by a little real fine tuning, is the more dependable route when scenes are open-ended, although its advantage shrinks as real data becomes plentiful. What the evidence does not yet support is a ranking of randomization factors or a trustworthy synthetic only validation protocol. Ablations change one factor at a time, metrics and thresholds differ, test sets are small, and a well-documented case shows near-perfect synthetic validation scores alongside noticeably lower real-world ones. Progress will need shared real test sets, factorial ablations, and transparent rendering pipelines.

[1] S. I. Nikolenko, “Synthetic data for deep learning,” arXiv:1909.11512, 2019.
[2]J. Tobin, R. Fong, A. Ray, J. Schneider, W. Zaremba, and P. Abbeel, “Domain randomization for transferring deep neural networks from simulation to the real world,” in Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst. (IROS), 2017, pp. 23–30.
[3] J. Tremblay et al., “Training deep networks with synthetic data: Bridging the reality gap by domain randomization,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. Workshops (CVPRW), 2018, pp. 969–977.
[4] H. Schieber, K. C. Demir, C. Kleinbeck, S. H. Yang, and D. Roth, “Indoor synthetic data generation: A systematic review,” Comput. Vis. Image Underst., vol. 240, Art. no. 103907, Mar. 2024, doi: 10.1016/j.cviu.2023.103907.
[5] F. Muratore, F. Ramos, G. Turk, W. Yu, M. Gienger, and J. Peters, “Robot learning from randomized simulations: A review,” Front. Robot. AI, vol. 9, Art. no. 799893, Apr. 2022, doi: 10.3389/frobt.2022.799893.
[6] A. Westerski and F. W. Teck, “Synthetic data for object detection with neural networks: State of the art survey of domain randomisation techniques,” ACM Trans. Multimedia Comput. Commun. Appl., doi: 10.1145/3637064. [7] E. Kolve et al., “AI2-THOR: An interactive 3D environment for visual AI,” arXiv:1712.05474, 2017.
[8] M. Denninger et al., “BlenderProc2: A procedural pipeline for photorealistic rendering,” J. Open Source Softw., vol. 8, no. 82, Art. no. 4901, 2023, doi: 10.21105/joss.04901.
[9] G. Martínez-Arellano and M. G. Buck, “A tool for generating and labelling domain randomised synthetic images for object recognition in manufacturing,” presented at the 2nd Eur. Symp. Artif. Intell. Manuf. (ESAIM), Athens, Greece, Oct. 2024.
[10] A. Prakash et al., “Structured domain randomization: Bridging the reality gap by context-aware synthetic data,” in Proc. IEEE Int. Conf. Robot. Autom. (ICRA), 2019, pp. 7249–7255.
[11] Y. Ganin and V. Lempitsky, “Unsupervised domain adaptation by backpropagation,” in Proc. 32nd Int. Conf. Mach. Learn. (ICML), 2015, pp. 1180–1189.
[12] L. Torquato Niño and H. A. A. Gardi, “Synthetic-to-real object detection using YOLOv11 and domain randomization strategies,” arXiv:2509.15045, 2025.
[13] J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You only look once: Unified, real-time object detection,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), 2016, pp. 779–788.
[14] J. Borrego, A. Dehban, R. Figueiredo, P. Moreno, A. Bernardino, and J. Santos-Victor, “Applying domain randomization to synthetic data for object category detection,” arXiv:1807.09834, 2018.
[15] A. Dehban, J. Borrego, R. Figueiredo, P. Moreno, A. Bernardino, and J. Santos-Victor, “The impact of domain randomization on object detection: A case study on parametric shapes and synthetic textures,” in Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst. (IROS), 2019.
[16] X. Zhu et al., “Domain randomization for object detection in manufacturing applications using synthetic data: A comprehensive study,” arXiv:2506.07539, 2025.
[17] S. Hinterstoisser, O. Pauly, H. Heibel, M. Marek, and M. Bokeloh, “An annotation saved is an annotation earned: Using fully synthetic training for object detection,” in Proc. IEEE/CVF Int. Conf. Comput. Vis. Workshops (ICCVW), 2019, pp. 2787–2796, doi: 10.1109/ICCVW.2019.00340.
[18] D. Horváth, G. Erdős, Z. Istenes, T. Horváth, and S. Földi, “Object detection using Sim2Real domain randomization for robotic applications,” IEEE Trans. Robot., vol. 39, no. 2, pp. 1225–1243, Apr. 2023, doi: 10.1109/TRO.2022.3207619.
[19] P. Rawal, M. Sompura, and W. Hintze, “Synthetic data generation for bridging Sim2Real gap in a production environment,” arXiv:2311.11039, 2023.
[20] J. M. Araya-Martinez et al., “SynthRender and IRIS: Open-source framework and dataset for bidirectional sim-real transfer in industrial object perception,” arXiv:2602.21141, 2026.

Arati Patil, Pritam Chaudhari, Ajinkya Shingne, Rohit Nalandwar, Ashish Pawar (2026). Investigating the Impact of Domain Randomization on Synthetic-to-Real Deep Learning Performance. International Journal of Computer Techniques, 13(6), 626–637. ISSN: 2394-2231. DOI: https://doi.org/10.5281/zenodo.24128829