Deep Learning-based Computer Vision for Radiation Defect Analysis: from Static Defect Segmentation to Dynamic Defect Tracking.

R. Sainju, W. Y. Chen, S. Schaefer, G. Roberts, M. B. Toloczko, R. J. Kurtz, C. H. Henager, D. J. Edwards, M. Li, Y. Zhu*, Microscopy and Microanalysis, 2021, 27 (S1), 1464-1465.


  • Advanced in situ TEM equipped with high-energy ion sources are beginning to produce an ‘avalanche’ of big data, e.g. gigabytes of in situ TEM videos produced in a single irradiation experiment, making it increasingly difficult to extract and quantify the temporal information of radiation defect dynamics
  • Our computer vision model DefectSegNet and associated MATLAB algorithms perform irradiation defect quantification in HT-9 in a more reproducible and reliable manner in just a few seconds
  • Our new multiple object tracking (MOT) computer vision model tracks and quantifies individual defect clusters in Ni irradiated with 1 MeV Kr ions at ANL IVEM in real-time, advancing our understanding of the dynamic evolution of cascade-induced defect clusters


Tracking and Understanding Nanocatalyst Sintering and Regeneration using Deep Learning-assisted In Situ Environmental TEM.

R. Sainju, S. Suib, C. Ding, Y. Zhu, Microscopy and Microanalysis, 2021, 27 (S1), 2216-2217.


  • Unlike conventional sintering studies that rely mostly on measuring averaged nanoparticle size or the overall surface area, the nature of in situ ETEM offers direct real-time visualization of the nanoparticles’ evolution at the nanoscale in response to the different reactive gaseous environments
  • Whether a nanoparticle was sintered or regenerated depends on the intricate interplays among the nanoparticle size, its surrounding nanoparticles, and the reaction conditions
  • In-situ ETEM combined with deep learning-based computer vision holds the potential to register and scale up single-particle level analysis that is critical to the understanding of nanocatalyst regeneration

Microstructural classification of unirradiated LiAlO2 pellets by deep learning methods. Computational Materials Science

K. Pazdernik, N. L. LaHaye, C. M. Artman, Y. Zhu, Computational Materials Science, 181, 2020, 109728


  • A full understanding of unirradiated LiAlO2 microstructure and how it evolves as a result of neutron irradiation is necessary to produce an integrated performance model to predict in-reactor behavior as well as to target strategic experiments
  • We tested a collection of Deep Convolutional Neural Network (DCNN) architectures that have been optimized for image segmentation and selected the best performer to obtain pixel-level classification of the main microstructural features in unirradiated LiAlO2 pellets, including grains, grain boundaries, voids, precipitates, and zirconia impurities

Deep Learning for Semantic Segmentation of Defects in Advanced STEM Images of Steels.   Highlighted on the DOE Fusion Energy Sciences (FES) website: AI Helps Scientists Quantify Irradiation Effects

G. Roberts, S. Y. Haile, R. Sainju, D. J. Edwards, B. Hutchinson, Y. Zhu*, Scientific Reports, 9, 2019, 1-12


  • We demonstrate the feasibility of automated identification of common crystallographic defects (including extended radiation defects) in HT-9 martensitic steel using deep learning semantic segmentation, based on high-quality microscopy data
  • DefectSegNet – a new hybrid CNN architecture with skip connections within and across the encoder and decoder was developed and has proved to be effective at perceptual defect identification with high pixel-wise accuracy
  • Deep-learning semantic segmentation established on advanced microscopy and on optimized CNN architecture offers a path forward to the high-throughput defects quantification needed for rational reactor alloy design

Automated Quantitative Analysis of Extended Irradiation Defects - Dislocations, Voids and Precipitates in Neutron Irradiated HT-9 Steel

R. Sainju, C. Ophus, M.B. Toloczko, D. J. Edwards, Y Zhu*, Microscopy and Microanalysis, 2019


  • A set of MATLAB algorithms were developed for automated identification and quantification of extended irradiation defects like dislocation lines, voids, and precipitates
  • An optimized circular-grid intersection method was employed for dislocation density measurement a reduced systematic error
  • For quantitative analysis of voids and precipitates, we resolved features overlapping with the application of modified the popular circular Hough transformation