As different forecasting models attempt to improve on wind forecasting, it can be difficult to identify which model is better. Betsy Weatherhead worked with the Developmental Testbed Center (DTC) to compare the NMM and ARW models on their wind forecasting abilities. Metrics were developed to help identify small, but significant differences in models. The results were presented to the DTC and have been used in clarifying the magnitude of improvement changes in the fundamental core of the forecasting models.
Differences in both bias and root mean square errors were identified by height and by season.
Improvements in weather forecasting over the years has been a combination of improved models and improved computer capabilities. At each stage of improvement, the measurable performance in forecasting accuracy has been difficult to observe. Using advanced statistical techniques, and well designed tests, these small differences can be identified early in the development process.