Impact of Stochastic Depth on Deterministic and Probabilistic Resnet Models for Weather Modeling
Abstract
This thesis builds on the research that uses Bayesian neural networks to generate Global Precipitation Measurement Microwave Imager data collected by LEO satellites from Advanced Baseline Imager data collected by GEO satellites for the purposes of weather modeling. Specifically, this thesis investigates the efficacy of a stochastic depth (SD) implementation in residual networks (ResNet), both deterministic and probabilistic, to reduce long training times associated with Bayesian neural networks while maintaining model accuracy. We show that overall, ResNets fail to perform better with the implementation of SD with the exception of SD ResNet56 S25, utilizing a survivability probability of 0.25. This resulted in an RMSE of2.863, a 6.83% increase in performance. In our evaluations, SD models did not train faster, with the fastest average time per epoch of 973.69 seconds compared to 960.42 seconds for the base ResNet56. We conclude that SD was unable to provide the expected performance benefits on realistic large-scale satellite data as found in research on smaller datasets.
Document Details
- Document Type
- Technical Report
- Publication Date
- Mar 01, 2022
- Accession Number
- AD1173494
Entities
People
- Cameron P. Woods
Organizations
- Naval Postgraduate School