Cooperative Institute for Research in Environmental Sciences at the University of Colorado Boulder

Climate Observing System Simulation Experiments


What are Climate Observing System Simulation Experiments (OSSEs)?


Climate OSSEs are a group of experiments that can be carried out to test whether a proposed observing system can adequately address specific climate science questions.  Because the range of climate science questions is broad, the range of approaches is equally broad.  All climate OSSEs address four very clear design questions:

            What accuracy?

            What frequency?

            What spatial resolution?

            What supplementary observations?

A clear metric of success can be addressing a key climate question in a specified time period or with a specified amount of resources.  By addressing all four of the above questions, proposed systems can be analyzed with either of those two success metrics.  Additional metrics that may be considered are, as pointed out in the NAS Continuity Report, the likelihood for success and the likelihood for side benefits.

Here, three types of experiments are outlined: Trend Detection, Sensitivity Studies, and Process Studies.  Since climate OSSEs cannot be evaluated by observations which do not yet exist, climate OSSE studies can utilize data from historical observations, from observations related to the property in question, or from a “Nature Run” (a detailed high resolution model simulation assumed to be the future “truth”) to determine the value of potential future observations. 

1. Trend Detection 

In a Trend Detection study, information from historical measurements is included in model simulations to determine when trends might be statistically detected.  Climate OSSE experiments can be conducted with simulated observations at varying accuracy, frequency, and spatial resolution to understand how new observations may affect the time required to measure trends. 

Example: How long before we can detect a trend in the recovery of ozone based on current Observing Systems?  Here, a Climate OSSE experiment is conducted by running a model with interactive stratospheric chemistry, adding observational variability to the model output, and conducting a statistical analysis to determine how long it would take for a trend to be detected.  The top plot shows projections for the expected recovery of total column ozone at 40N.  The middle plot shows the same positive trend with real, measured variability from past measurements.  The final plot illustrates the level of accuracy and long-term drift that can be maintained with the best of current systems.  These graphics are illustrations for the actual calculations that have taken place estimating time to detect ozone recovery.  For climate trends, similar approaches can be used and the effect of different temporal and spatial sampling can be tested.

2. Sensitivity studies

Because many climate parameters are inadequately observed, simulations can be conducted with low and high estimates of a property to determine how sensitive a key climate parameter is to the uncertainty.  If the property has a significant impact on the climate parameter, this suggests the importance of improving the observations for that property.

Example: What impact does uncertainty in the level of stratospheric aerosols have on global radiative forcing? A climate OSSE experiment is conducted by running a model or series of models with high and low estimates of stratospheric aerosol and calculating the change in radiative forcing after a specified length of time. If global radiative forcing changes significantly, this suggests a need to improve observations of stratospheric aerosol.

3. Process Studies

Process studies investigate a single climate science question, identifying a specific set of observations at a specific location that may be critical to validate the hypothesis.

Example: Does an increase in upper tropospheric convection lead to an increase in stratospheric water vapor concentration? A climate OSSE experiment is conducted by running a model or a series of models for a sufficient period of time such that the relationship between two or more processes can be studied (in this case, large convective events in the upper troposphere, and variations in stratospheric water vapor). If the results of the climate OSSE suggest a strong relationship between mid-latitude convection and stratospheric water vapor, then it may support improved observations of these variables in these locations in order to improve current models and projections of future climate.

Overlap between weather OSSEs and climate OSSEs.

The goal of both Weather OSSEs and Climate OSSEs is to determine the impact of proposed observing systems on our understanding of the Earth system. The metric of success for weather OSSEs is improved weather forecasts.  For climate OSSEs, the metric of success is addressing an outstanding science question, such as the vertical resolution of warming in the oceans.  Because of the weather OSSEs’ focus, there is high reliance on a well calibrated Nature Run, to simulate a few years of atmospheric observations.  Climate OSSEs rarely work with a formal nature run due to the broad scope of climate science questions required and the long timescales typically associated with them. In some cases, a Nature Run can be used to understand the impact of synoptic systems on observations.  In other cases, a Nature Run can be used to help improve climate models.  For example, most climate models have a crude approximation of aerosol growth and size representation. A “Nature Run” comprised of a comprehensive sectional aerosol model can be used to evaluate and improve development of the aerosol module in the climate model.

One challenge for climate OSSEs is assuring that the proposed observations can work well with the natural variability that is well represented in a calibrated nature run.  For instance, a remote sensing technique may work well in clear sky conditions but would benefit from being tested in its ability to retrieve useful data in all sky conditions from a weather OSSE nature run. 

Final decisions about observing systems will likely take place with input from both climate OSSE experiments and weather OSSE experiments.  Developing both weather and climate OSSE, when appropriate, to work off of similar assumptions and testing conditions will help remove confusion and concerns from decision makers.

Why conduct Climate OSSEs?

Observational systems are typical expensive to implement. This is especially true with climate observing systems which often require high accuracy, global coverage, and long lifespan.  Additionally, it may take many years to directly verify whether an observing system is successful because climate science questions are often on long timescales (years or longer) and coupled with high variability on spatial and temporal scales. Therefore, the use of Climate OSSEs can increase the likelihood that an observing system is useful.  Climate OSSEs have existed for many years, but instrument developers and academic researchers may have different labels for them.  A coordinated Climate OSSE program may assure that evaluations using these tools will be honest, open and state of the art, resulting in unbiased answers to some of climate’s most important questions.