Skip Navigation
JCSDA Logo and banner

JCSDA News and Newsletters


SCIENCE UPDATE:
Assessment of Ensemble Forecast Sensitivity to Observation (EFSO) Quantities for Satellite Radiances Assimilated in the 4DEnVar GFS

From the Winter 2017 issue of the JCSDA Quarterly, DOI: 10.7289/V5S46Q0K

The Ensemble Forecast Sensitivity for Observation (EFSO) formulation (Kalnay et al. 2012) has been implemented at the National Centers of Environmental Protection (NCEP). For the Global Forecast System (GFS), this approach requires Ensemble Kalman Filter (EnKF) products as input, and it has been implemented within the current source code that provides EnKF functionality at NCEP (Ota et al. 2013 and Groff et al. 2017). As with the adjoint Forecast Sensitivity to Observation Impact (FSOI) approach (Langland and Baker 2004, Zhu and Gelaro 2008), EFSO capabilities effectively enable a simultaneous forecast impact estimate for any and all observations assimilated in a numerical weather prediction (NWP) system.

The Ensemble Forecast Sensitivity for Observation (EFSO) formulation (Kalnay et al. 2012) has been implemented at the National Centers of Environmental Protection (NCEP). For the Global Forecast System (GFS), this approach requires Ensemble Kalman Filter (EnKF) products as input, and it has been implemented within the current source code that provides EnKF functionality at NCEP (Ota et al. 2013 and Groff et al. 2017). As with the adjoint Forecast Sensitivity to Observation Impact (FSOI) approach (Langland and Baker 2004, Zhu and Gelaro 2008), EFSO capabilities effectively enable a simultaneous forecast impact estimate for any and all observations assimilated in a numerical weather prediction (NWP) system.

Based on EnKF output from a control low-resolution configuration of the four-dimensional ensemble-variational (4DEnVar) GFS, and an experimental low-resolution configuration of the 4DEnVar GFS in which aircraft data are thinned, two EFSO datasets are being generated for the retrospective time period December, January, and February 2014/2015. Thinning is done similar to radiances and other observation types for the EnKF. As with other standard forecast sensitivity-observation impact (FSOI) datasets, the EFSO quantities in these two datasets represent an estimate for the extent to which an assimilated observation has increased or decreased 24-hour forecast error. The forecast error metrics applied are dry total and moist total energy norms (Ehrendorfer et al. 1999). Self analyses are applied as verification for this EFSO dataset, but in general anything considered to be close to the truth relative to the relevant forecasts can be applied.

For the plots and maps in this article, note that negative EFSO quantities indicate that the assimilation of an observation (or subset of observations) decreased 24-hour forecast error; whereas positive EFSO quantities indicate that the assimilation of an observation (or subset of observations) increased 24-hour forecast error. On this basis, negative EFSO quantities will hereafter be referred to as beneficial and positive EFSO quantities will be referred to as detrimental.

Similar to the interpretation of adjoint Forecast Sensitivity to Observation (FSO) calculations (Zhu and Gelaro 2009), EFSO calculations represent an estimate of the forecast impact due to assimilating an observation in the context that all other observations have been assimilated. Therefore, the result of an observing system experiment (OSE) in which a subset of observations is removed has a different, but complementary, interpretation with respect to EFSO or adjoint FSOI approaches.

With this context in mind, EFSO total impact summary statistics are helpful in providing, among other insights, the extent to which an observing system is coincident with model forecast error sensitivity to initial conditions, approaches for specifying observation error, relative influence by observation type, and the spatial configuration of the full assimilated observing system. In a previous EFSO study for a pure EnKF configuration of the GFS (Ota et al. 2013), summary statistics were helpful in quantifying some of these overarching considerations for the GFS. Bar plots representing total EFSO suggested impact by observation type are indicative of the relative influences for an NWP system by observation type.

The extent to which EFSO/FSOI can be applied to identify observing system configurations that result in improved forecast skill is a matter of further investigation. It should be noted, however, that relative influence by observation type, and an observation type’s value as it relates to forecast skill, are two distinct topics. Total impact bar plots by observation type for a one-week sample of the EFSO dataset corresponding to the control 4DEnVar GFS configuration, Fig. 1, show a much larger relative influence from aircraft data with respect to satellite radiances than was reported in Ota et al. 2013. This change in the influence of aircraft data relative to satellite radiances is due to a large increase in the number of assimilated aircraft observations that occurred between 2013 and 2015. In particular, the availability of Aircraft Communications Addressing and Reporting System (ACARS) data for assimilation in the GFS increased substantially during this timeframe.

Figure 1: EFSO dry total energy impacts Figure 1. (a) EFSO dry total energy impact by observation type for the control 4DEnVar GFS; (b) same as (a) but for a configuration of the 4DEnVar GFS in which aircraft data are thinned.

For the aforementioned experimental 4DEnVar GFS configuration that included thinning of aircraft data, the relative influence of aircraft data for the 4DEnVar GFS as suggested by EFSO calculations is more similar to that reported in Ota et al. 2013. Efforts intended to assess how the changes in the relative influence from aircraft data may impact forecast skill are underway. In conjunction with an FSOI interagency comparison study presented at the 97th AMS annual meeting (Auligne et al. 2017), further efforts to assess influence by observation type for the 4DEnVar GFS are planned.

Figure 2. Total EFSO impact per cycle versus binned innovation for
					a subset of CrIS temperature sounding channels Figure 2. Total EFSO impact per cycle versus binned innovation for a subset of CrIS temperature sounding channels.

It is relevant to note again that EFSO and FSO approaches effectively enable a simultaneous estimate of forecast impact for any and all observations assimilated. Taking advantage of the simultaneity aspect of EFSO, the suggested observation impacts can be sorted by conditional information. Based on a few-week sample of EFSO calculations from the DJF 2014/2015 dataset, fig. 2 shows plots of binned bias corrected innovation versus total EFSO for a subset of CrIS temperature sounding channels. For the CrIS temperature sounding channels, there are notable asymmetries in EFSO-suggested impact with respect to innovation. In particular, positive innovations tend to be far more beneficial than negative innovations. The extent to which this result may reflect model bias, cloud contamination, forward operator errors, or situation- dependent limitations in the application of variational bias correction is a matter of further investigation.

For the same few-week sample, fig. 3 shows 7.5 by 7.5 composite maps of mean EFSO for AMSUA channel 2. In general, EFSO suggests that these observations are relatively problematic poleward of 40°N and 40°S (figure 3). Composite mean EFSO maps by innovation sign, figs. 3b and 3c, indicate that the assimilation of negative innovations is particularly problematic over ocean surfaces poleward of 40°N and 40°S.

Figure 3. composite mean EFSO Figure 3. (a) 7.5 by 7.5 composite mean EFSO for all AMSUA channel 2 assimilated observations; (b) same as (a) but for negative innovations; (c) same as (a), but for positive innovations

For the microwave radiances assimilated in the 4DEnVar GFS, it is also relevant to sort EFSO quantities based on cloud amount. Initial results indicate that the overall per-observation EFSO-suggested benefit for AMSUA temperature sounding channels is larger for cloudaffected radiances than for clear-sky AMSUA radiances. As indicated in fig. 4, however, assimilated negative innovations for AMSUA channels (6–8) tend to be detrimental when the coincident observed and background cloud liquid water (CLW) amounts are larger than .27 kg/m^2. Further investigation is necessary to determine how to utilize this conditional EFSO information in the 4DEnVar GFS.

Figure 4. 3.75 by 3.75 composite mean EFSO for assimilated AMSUA channels (6-8) Figure 4. 3.75 by 3.75 composite mean EFSO for assimilated AMSUA channels (6-8) coincident with background and retrieved Cloud Liquid Water (CLW) greater than .27 kg/m2. The map accounts only for negative innovations assimilated.

Initial results indicate that EFSO detriment with respect to innovation sign has a strong dependence on conditional radiance biascorrection information. In the next quarter, 4DEnVar GFS experiments based on EFSO/ FSOI guidance with respect to bias correction are planned, and the traditional Verification Statistics Data Base (VSDB) software package will be applied to assess forecast skill. In addition, efforts to better understand EFSO asymmetries with respect to innovation sign, and the relationship between EFSO quantities for microwave radiances and CLW, are ongoing.

David Groff (I.M. Systems Group, Inc., [IMSG] at NOAA/NCEP/EMC)

References

Ota., Y., J. Derber, E. Kalnay, and T. Miyoshi, 2013. Ensemble- based observation impact estimates using the NCEP GFS. Tellus, 65A, 20038

Whitaker, J.S., and T.M. Hamill, 2002. Ensemble data assimilation without perturbed observations. Mon. Wea. Rev., 130, 1913–1924

Zhu., Y., and R. Gelaro, 2008. Observation sensitivity calculations using the adjoint of the Gridpoint Statistical Interpolation (GSI) analysis system. Mon. Wea. Rev., 136, 335–351

Auligne, T., R. Gelaro, R. Mahajan, D. Holdaway, W. McCarty, A. Collard, N. Baker, R. Langland, D. Hotta, T. Ishibashi, J. Eyre, W. Tennant, and D. Groff, 2017. Forecast sensitivity observation impact (FSOI) inter-comparison experiment. 97th AMS Annual Meeting, Washington State Convention Center, Seattle, WA.

Groff, D., K. Ide, Y. Zhu, and R. Mahajan, 2017, Assessment of ensemble forecast sensitivity to observation (EFSO) quantities for satellite radiances in the 4DEnVar GFS. 97th AMS Annual Meeting, Washington State Convention Center, Seattle, WA.

Gelaro, R. and Y. Zhu, 2009. Examination of observation impacts derived from observing system experiments (OSEs) and adjoint models. Tellus, 61A, 179–193.

Langland, R.H. and N.L. Baker. 2004. Estimation of observation impact using the NRL atmospheric variational data assimilation adjoint system. Tellus, 56A, 189–201 Kalnay, E., Y. Ota., T. Miyoshi, J. Liu, 2012. A simpler formulation of forecast sensitivity to observations: application to ensemble Kalman filters. Tellus, 64A, 18462

Ehrendorfer, M., R.M. Errico, and K.D. Raeder, 1999. Singular-vector perturbation growth in a primitive equation model with moist physics. J. Atmos. Sci., 56, 1627–1648.

If you are a contributing author, please follow these guidelines for your JCSDA Newsletter submission.



Newsletters supplied as PDFs


 
 
Modified February 14, 2017 7:02 PM
NOAA | NWS | NASA | Navy Weather
Air Force Weather | Website owner: JCSDA
Level A conformance icon, W3C-WAI Web Content Accessibility Guidelines 1.0 and Valid HTML 4.01 Icon
icon: valid HTML 4.01 transitional. Level A conformance icon, W3C-WAI Web Content Accessibility Guidelines 1.0