Evaluation of the Reproducibility and Stability of Radiomic Features Derived from Ovarian MRI Phantom Studies

Document Type : Original Paper

Authors

1 Medical Physics Department, School of Medicine, Iran University of Medical Sciences, Tehran, Iran

2 Radiation Science Department, Faculty of Allied Medicine, Iran University of Medical Sciences, Tehran, Iran

3 Department of Radiation Physics, Unit 1420, The University of Texas MD Anderson Cancer Center, 1515 Holcombe Boulevard, Houston, TX 77030, USA

4 Medical Physics Department, School of Medicine, Iran university of Medical Sciences,

10.22038/ijmp.2026.93264.2657

Abstract

Introduction: The reproducibility of radiomic features could be a serious obstacle that limits further applications. This study aims to assess the reproducibility of MRI radiomic features by using a biological ovarian phantom and different acquisition parameters.
Material and Methods: Three 1.5 Tesla MRI scanners from different manufacturers as the first source, and also alterations in imaging parameters, including slice thickness, space between slices, image weight, and fat saturation sequence, as the second source of feature variations, were evaluated. In addition, to evaluate the effect of image normalization on feature reproducibility, all the images were normalized. Ninety-three radiomic features from 6 feature classes, including First-Order, GLCM, GLDM, GLRLM, GLSZM, and NGTDM, were calculated by the 3D-Slicer. Reproducibility of features was measured by COV, ICC, and CCC.
Results: The significant impact of scanner and image weight variation on feature reproducibility is obvious when about 90% and 64% of features showed 20 %< COV, respectively. On the other hand, slice thickness was the least affected source, where 58.8% of features showed excellent reproducibility (COV ≤ 5%). GLRLM showed the best reproducibility against scanner variation (ICC=0.6996 and CCC=0.3503). Also, image normalization has positively affected feature reproducibility in the scanner variation scheme. Additionally, good (5 %< COV≤10%) and intermediate (10 %< COV≤20%) COV groups have increased by normalization.
Conclusion: MRI radiomic features are highly dependent on image acquisition scanner types and imaging parameters, and utilizing biological phantoms can lead to reliable outcomes that make the way of clinical translation of these results easier. Future works should be the priority in the robustness evaluation of radiomic features, and the inconsistent behavior of the image normalization filter needs higher attention.

Keywords

Main Subjects

References

  1. Lambin P, Leijenaar RT, Deist TM, Peerlings J, De Jong EE, Van Timmeren J, et al. Radiomics: the bridge between medical imaging and personalized medicine. Nature reviews Clinical oncology. 2017;14(12):749-62.
  2. Van Timmeren JE, Cester D, Tanadini-Lang S, Alkadhi H, Baessler B. Radiomics in medical imaging—“how-to” guide and critical reflection. Insights into imaging. 2020;11(1):91.
  3. Aerts HJ, Velazquez ER, Leijenaar RT, Parmar C, Grossmann P, Carvalho S, et al. Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach. Nature communications. 2014;5(1):1-9.
  4. Gardin I, Grégoire V, Gibon D, Kirisli H, Pasquier D, Thariat J, et al. Radiomics: principles and radiotherapy applications. Critical reviews in oncology/hematology. 2019;138:44-50.
  5. Kocak B, Keles A, Kose F, Sendur A. Quality of radiomics research: comprehensive analysis of 1574 unique publications from 89 reviews. European Radiology. 2025;35(4):1980-92.
  6. Rizzo S, Botta F, Raimondi S, Origgi D, Fanciullo C, Morganti AG, et al. Radiomics: the facts and the challenges of image analysis. European radiology experimental. 2018;2(1):36.
  7. Lambin P, Rios-Velazquez E, Leijenaar R, Carvalho S, Van Stiphout RG, Granton P, et al. Radiomics: extracting more information from medical images using advanced feature analysis. European journal of cancer. 2012;48(4):441-6.
  8. Tahmasebzadeh A, Yazdani E, Mirshahi R, Naseripour M, Sadeghi M. Machine Learning-Based prediction of local recurrence in uveal melanoma after Ruthenium-106 plaque brachytherapy using ultrasound images and clinical data. Clinical Oncology. 2025:103960.
  9. Traverso A, Kazmierski M, Welch ML, Weiss J, Fiset S, Foltz WD, et al. Sensitivity of radiomic features to inter-observer variability and image pre-processing in Apparent Diffusion Coefficient (ADC) maps of cervix cancer patients. Radiotherapy and Oncology. 2020;143:88-94.
  10. Perniciano A, Loddo A, Di Ruberto C, Pes B. Insights into radiomics: impact of feature selection and classification. Multimedia Tools and Applications. 2025;84(26):31695-721.
  11. Han Y, Wang T, Wu P, Zhang H, Chen H, Yang C. Meningiomas: Preoperative predictive histopathological grading based on radiomics of MRI. Magnetic Resonance Imaging. 2021;77:36-43.
  12. Park JE, Kim HS, Jo Y, Yoo R-E, Choi SH, Nam SJ, et al. Radiomics prognostication model in glioblastoma using diffusion-and perfusion-weighted MRI. Scientific reports. 2020;10(1):4250.
  13. Wang W, Cao K, Jin S, Zhu X, Ding J, Peng W. Differentiation of renal cell carcinoma subtypes through MRI-based radiomics analysis. European radiology. 2020;30(10):5738-47.
  14. Avanzo M, Stancanello J, El Naqa I. Beyond imaging: the promise of radiomics. Physica Medica. 2017;38:122-39.
  15. Mi H, Yuan M, Suo S, Cheng J, Li S, Duan S, et al. Impact of different scanners and acquisition parameters on robustness of MR radiomics features based on women’s cervix. Scientific reports. 2020;10(1):20407.
  16. Reiazi R, Abbas E, Famiyeh P, Rezaie A, Kwan JY, Patel T, et al. The impact of the variation of imaging factors on the robustness of Computed Tomography Radiomic Features: A review. medRxiv. 2020:2020.07. 09.20137240.
  17. Wennmann M, Rotkopf LT, Bauer F, Hielscher T, Kächele J, Mai EK, et al. Reproducible Radiomics Features from Multi‐MRI‐Scanner Test–Retest‐Study: Influence on Performance and Generalizability of Models. Journal of Magnetic Resonance Imaging. 2025;61(2):676-86.
  18. Ammari S, Pitre-Champagnat S, Dercle L, Chouzenoux E, Moalla S, Reuze S, et al. Influence of magnetic field strength on magnetic resonance imaging radiomics features in brain imaging, an in vitro and in vivo study. Frontiers in oncology. 2021;10:541663.
  19. Carré A, Klausner G, Edjlali M, Lerousseau M, Briend-Diop J, Sun R, et al. Standardization of brain MR images across machines and protocols: bridging the gap for MRI-based radiomics. Scientific reports. 2020;10(1):12340.
  20. Isaksson LJ, Raimondi S, Botta F, Pepa M, Gugliandolo SG, De Angelis SP, et al. Effects of MRI image normalization techniques in prostate cancer radiomics. Physica Medica. 2020;71:7-13.
  21. Sun X, Shi L, Luo Y, Yang W, Li H, Liang P, et al. Histogram-based normalization technique on human brain magnetic resonance images from different acquisitions. Biomedical engineering online. 2015;14(1):73.
  22. Du Z, Zhang P, Huang X, Hu Z, Yang G, Xi M, et al. Deeply supervised two stage generative adversarial network for stain normalization. Scientific Reports. 2025;15(1):7068.
  23. Karimi M, Mostaghimi H, Shams S, Mehdizadeh A. Design and production of two-piece thyroid-neck phantom by the concurrent use of epoxy resin and poly (Methyl methacrylate) soft tissue equivalent materials. Journal of biomedical physics & engineering. 2018;8(2):217.
  24. Jones A, Hintenlang D, Bolch W. Tissue‐equivalent materials for construction of tomographic dosimetry phantoms in pediatric radiology. Medical physics. 2003;30(8):2072-81.
  25. Jones A, Hintenlang D, Bolch W. Tissue‐equivalent materials for construction of tomographic dosimetry phantoms in pediatric radiology. Medical physics. 2003;30(8):2072-81.
  26. 2Kikinis R, Pieper SD, Vosburgh KG. 3D Slicer: a platform for subject-specific image analysis, visualization, and clinical support. Intraoperative imaging and image-guided therapy: Springer; 2013. p. 277-89.
  27. Ma S, Xie H, Wang H, Yang J, Han C, Wang X, et al. Preoperative Prediction of Extracapsular Extension: Radiomics Signature Based on Magnetic Resonance Imaging to Stage Prostate Cancer. Mol Imaging Biol. 2020;22(3):711-21.
  28. Van Griethuysen JJ, Fedorov A, Parmar C, Hosny A, Aucoin N, Narayan V, et al. Computational radiomics system to decode the radiographic phenotype. Cancer research. 2017;77(21):e104-e7.
  29. Schurink NW, van Kranen SR, Roberti S, van Griethuysen JJ, Bogveradze N, Castagnoli F, et al. Sources of variation in multicenter rectal MRI data and their effect on radiomics feature reproducibility. European Radiology. 2021:1-11.
  30. Pieper S, Halle M, Kikinis R, editors. 3D Slicer. 2004 2nd IEEE international symposium on biomedical imaging: nano to macro (IEEE Cat No 04EX821); 2004: IEEE.
  31. Peng X, Yang S, Zhou L, Mei Y, Shi L, Zhang R, et al. Repeatability and reproducibility of computed tomography radiomics for pulmonary nodules: a multicenter phantom study. Investigative radiology. 2022;57(4):242-53.
  32. Lee J, Steinmann A, Ding Y, Lee H, Owens C, Wang J, et al. Radiomics feature robustness as measured using an MRI phantom. Scientific reports. 2021;11(1):3973.
  33. Cheong E-N, Park JE, Park SY, Jung SC, Kim HS. Achieving imaging and computational reproducibility on multiparametric MRI radiomics features in brain tumor diagnosis: phantom and clinical validation. European Radiology. 2024;34(3):2008-23.
  34. Jha A, Mithun S, Jaiswar V, Sherkhane U, Purandare N, Prabhash K, et al. Repeatability and reproducibility study of radiomic features on a phantom and human cohort. Scientific reports. 2021;11(1):2055.
  35. Buch K, Kuno H, Qureshi MM, Li B, Sakai O. Quantitative variations in texture analysis features dependent on MRI scanning parameters: A phantom model. Journal of applied clinical medical physics. 2018;19(6):253-64.
  36. Scalco E, Belfatto A, Mastropietro A, Rancati T, Avuzzi B, Messina A, et al. T2w‐MRI signal normalization affects radiomics features reproducibility. Medical physics. 2020;47(4):1680-91.
  37. Schwier M, Van Griethuysen J, Vangel MG, Pieper S, Peled S, Tempany C, et al. Repeatability of multiparametric prostate MRI radiomics features. Scientific reports. 2019;9(1):9441.
  38. Wong J, Baine M, Wisnoskie S, Bennion N, Zheng D, Yu L, et al. Effects of interobserver and interdisciplinary segmentation variabilities on CT-based radiomics for pancreatic cancer. Scientific reports. 2021;11(1):16328.
  39. Traverso A, Wee L, Dekker A, Gillies R. Repeatability and reproducibility of radiomic features: a systematic review. International Journal of Radiation Oncology* Biology* Physics. 2018;102(4):1143-58.
  40. Tixier F, Um H, Young RJ, Veeraraghavan H. Reliability of tumor segmentation in glioblastoma: impact on the robustness of MRI‐radiomic features. Medical physics. 2019;46(8):3582-91.
  41. Alhussen A, Haq MA, Khan AA, Mahendran RK, Kadry S. XAI-RACapsNet: Relevance aware capsule network-based breast cancer detection using mammography images via explainability O-net ROI segmentation. Expert Systems with Applications. 2025;261:125461.
  42. Bologna M, Corino V, Mainardi L. virtual phantom analyses for preprocessing evaluation and detection of a robust feature set for MRI‐radiomics of the brain. Medical physics. 2019;46(11):5116-23.
  43. Archana R, Jeevaraj PE. Deep learning models for digital image processing: a review. Artificial Intelligence Review. 2024;57(1):11.
  44. Pfaehler E, Zhovannik I, Wei L, Boellaard R, Dekker A, Monshouwer R, et al. A systematic review and quality of reporting checklist for repeatability and reproducibility of radiomic features. Physics and imaging in radiation oncology. 2021;20:69-75.
  45. Zwanenburg A, Leger S, Vallières M, Löck S. Image biomarker standardisation initiative. arXiv preprint arXiv:161207003. 2016.
  46. Whybra P, Zwanenburg A, Andrearczyk V, Schaer R, Apte AP, Ayotte A, et al. The image biomarker standardization initiative: standardized convolutional filters for reproducible radiomics and enhanced clinical insights. Radiology. 2024;310(2):e231319.

 

 

 

 

 

 

Iranian Journal of Medical Physics
Volume 23, Issue 1
January and February 2026
Pages 9-17

Files

History

  • Receive Date: 08 December 2025
  • Revise Date: 04 February 2026
  • Accept Date: 18 March 2026

Share

How to cite

Statistics