May 22 - July 28
Comparing OU MAP Real-Time Convection Allowing Ensemble Forecasts Produced During 2021 & 2022 Using Neighborhood & Surrogate Verification Methods
Brett A. Castro, Nicholas A. Gasperoni, and Xuguang Wang
What is already known:
- Convective Allowing models (CAMs) provide detailed, short-range forecasting of convection including identification of storm modes and associated hazards
- CAM ensembles increase forecasting potential by quantifying forecast uncertainty using ensemble spread
- CAM ensemble systems such as the High Resolution Ensemble Forecast System (HREF) have a good track record in the forecasting of convection
- Rapid Refresh Forecast System (RRFS) will replace HREF as a self-contained ensemble CAM using the Finite Volume Cubed Sphere (FV3) within the Unified Forecast System
- The capability of FV3 to model convection compared to existing CAMs is a work in progress which needs further development and verification
What this study adds:
- Utilization of several neighborhood-based verification methods to test forecast accuracy of CAM ensembles using FV3 model core and identify areas of improvement from different years
- Comparison of RRFS-like ensemble systems run in real-time by OU MAP Lab during HWT Spring Forecasting Experiments (SFEs) shows consistent improvement in accuracy from 2021 to 2022 during next-day (12-36h) forecast period
- Superior performance in 2022 seen in both model simulated reflectivity & accumulated precipitation, including enhanced skill as a result of added ensemble spread during 2022
- Mixed results on whether improvement occurred in prediction of severe hazards, more diagnostics and verifications needed to make strong conclusions
Abstract:
During the peak of the 2021 & 2022 Severe Weather season in the Midwest, the University of Oklahoma (OU) Multiscale Data Assimilation and Predictability (MAP) Laboratory ran two Rapid Refresh Forecast System (RRFS)-like systems within the Hazardous Weather Testbed (HWT) in an effort to test their accuracy in forecasting the development and evolution of convection. The model, referred to as the FV3-LAM consists of 10 ensembles, and was initialized at 00z between the first Monday of May and first Monday of June for both 2021 & 2022. This study will analyze the forecasting skill of the model by comparing real time observations with model simulations. Several methods are used to quantitatively examine accuracy. The standard Neighborhood Method, in which an arbitrary radius is chosen containing a set number of grid points, can be used to compare the precipitation and reflectivity within the model to observations during the same period. Additionally, a Surrogate Severe Method is also used, which maps helicity tracks generated within the model and compares them with observed storm reports. From these methods, a Fractions Skill Score (FSS) can be calculated and gives a quantitative measurement of forecast accuracy. Promising trends in model accuracy were observed between the two years, with average skill scores in 2022 outperforming 2021 across most periods. Results from the standard Neighborhood method support improvement in both placement of convection and precipitation. Conclusions based on the surrogate severe method were less concrete and require further study.