JCSDA Logo and banner

Calculation of the NCEP Performance Index

by Perry Shafran of NCEP

The NCEP Performance Index (PI, or sometimes called the Decision Index, DI) is based on the Numerical Weather Prediction (NWP) Index used by the UK Meteorological Office to determine the skill of their global index versus persistence for 5 days worldwide. The PI is compiled using a selected number of parameters, over certain regions of the globe, and over certain forecast ranges. The skill scores are computed for each parameter, and are then combined to a single value using weights for each parameter.

Currently those parameters are:

NCEP Performance Index Parameters
Parameters Areas Forecast Hours
  • Mean sea level pressure
  • 500-mb height
  • 850-mb winds
  • 250-mb winds
  • The Northern Hemisphere
    (90°N - 20°N)
  • The Tropics
    (20°S - 20°N)
  • The Southern Hemisphere
    (20°S - 90°S)
  • 24 hr
  • 48 hr
  • 72 hr
  • 96 hr
  • 120 hr

Currently, the PI is calculated using the precise parameters used by the UK Met Office in their NWP Index (called here the UK Met Index). It should be noted that the PI is still under development, so this index in the future may deviate from what the UK Met Index calculates (in terms of variables, forecast hours, weights, etc.), but the current version calculates exactly what the UK Met Index calculates for their model.

The model used here is NCEP's Global Forecast System (GFS). The operational GFS 1-degree grid forecast output is compared to the GFS persistence forecast on the same grid. Persistence is created by taking the 00-hr analysis and using the very same output for comparison to the various forecast hours. For example, the 48-hr forecast today is compared to the 00-hr analysis 48 hours ago.

First, the root-mean-square (RMS) errors are calculated. The errors are generated for each individual variable (e.g. 500-mb height, sea-level pressure, etc.), at each individual forecast hour (e.g. 24 hr, 48 hr, etc.), and averaged over a particular region (e.g. NH, TR, SH). 00 UTC and 12 UTC runs are computed separately.

Then, a skill score is calculated. The skill score is called the Reduction of Variance, and is calculated as follows:

calculation of the Reduction of Variance.

Here rf is the RMS error of the forecast, and rp is the RMS error of the persistence.

This particular skill score is then used to calculate a weighted sum, where the weights are shown in the Table below.

Skill Score Weighting
Forecast hours: 24 48 72 96 120
Northern Hemisphere Sea level pressure 10 8 6 4 4
500-mb Z 6 4 2 0 0
250-mb wind 12 0 0 0 0
Tropical 850-mb wind 5 3 3 0 0
250-mb wind 6 0 0 0 0
Southern Hemisphere Sea level pressure 5 4 3 2 2
500-mb Z 3 2 1 0 0
250-mb wind 6 0 0 0 0

Note that the sum of the weights in the chart sum up to 100. The formula for calculating the weighted average is:

calculation of the weighted averageWhere w is the weight of the ith component, and SS is the skill score of the ith component. Finally, the PI is calculated using this weighted sum here:

calculation of the weighted sumAt this point the PI for the 00 UTC forecast is averaged with the PI with the 12 UTC forecast to make one daily PI for the entire day, and that value of PI is presented on the graph. The PI for a full year is presented on each graph daily. A 30-day running mean is applied to the graph to smooth out the noise.

Modified September 2, 2010 7:09 PM
NOAA | NWS | NASA | Navy Weather
Air Force Weather | Website owner: JCSDA
Level A conformance icon, W3C-WAI Web Content Accessibility Guidelines 1.0 and Valid HTML 4.01 Icon
icon: valid HTML 4.01 transitional. Level A conformance icon, W3C-WAI Web Content Accessibility Guidelines 1.0