U. Pflüger & U. Damrath, COSMO GM, Athens 2007
Current state of COSMO-EU and COSMO-DE verification using Analysis, SYNOP- and
upper air observations
Ulrich Pflüger & Ulrich Damrath
U. Pflüger & U. Damrath, COSMO GM, Athens 2007
Outline
• Verification against analysis• Verification against SYNOPs
Traditional verification - known problems Conditional verification
• Verification against upper air observations Some time series A problem using observations
• Verification against SYNOP-observations
Warning: Be prepared to see a lot of red and blue lines!
U. Pflüger & U. Damrath, COSMO GM, Athens 2007
Verification against analysis
• All levels• All forecast times with an increment of 12
hours• All elements on pressure levels• Comparison with GME
U. Pflüger & U. Damrath, COSMO GM, Athens 2007
Verification against analysis: mslp
U. Pflüger & U. Damrath, COSMO GM, Athens 2007
Verification against analysis: RH 850 hPa
U. Pflüger & U. Damrath, COSMO GM, Athens 2007
Verification against SYNOPstemperature 06 UTC
U. Pflüger & U. Damrath, COSMO GM, Athens 2007
Verification against SYNOPstemperature 12 UTC
U. Pflüger & U. Damrath, COSMO GM, Athens 2007
Verification against SYNOPstemperature 18 UTC
U. Pflüger & U. Damrath, COSMO GM, Athens 2007
Verification against SYNOPsmaximum temperature 06-18 UTC
U. Pflüger & U. Damrath, COSMO GM, Athens 2007
Verification against SYNOPspressure bias 06 UTC
U. Pflüger & U. Damrath, COSMO GM, Athens 2007
Example for conditional verificationForecasted and observed values of surface level pressure over the region of Germany during DJF 2005/2006(RMSE and STDV)
Forecasted and observed values of surface level pressure over the region of Germany during DJF 2005/2006 observed and forecasted values lower than 1020 hPa(RMSE and STDV)
Forecasted and observed values of surface level pressure over the region of Germany during DJF 2005/2006 observed and forecasted values higher than 1020 hPa(RMSE and STDV)
U. Pflüger & U. Damrath, COSMO GM, Athens 2007
Conditional verification
• Surface weather elements• Conditions concerning
cloud cover pressure temperature
• Criteria observation below/above mean observation forecast below/above mean observation
U. Pflüger & U. Damrath, COSMO GM, Athens 2007
Conditional verification (no condition, winter)
U. Pflüger & U. Damrath, COSMO GM, Athens 2007
Conditional verification (high cloud covers, winter)
U. Pflüger & U. Damrath, COSMO GM, Athens 2007
Conditional verification (low cloud covers, winter)
U. Pflüger & U. Damrath, COSMO GM, Athens 2007
Conditional verification (high temperatures, winter)
U. Pflüger & U. Damrath, COSMO GM, Athens 2007
Conditional verification (low temperatures, winter)
U. Pflüger & U. Damrath, COSMO GM, Athens 2007
Conditional verification (no condition, summer)
U. Pflüger & U. Damrath, COSMO GM, Athens 2007
Conditional verification (high temperatures, summer)
U. Pflüger & U. Damrath, COSMO GM, Athens 2007
Conditional verification (low temperatures, summer)
U. Pflüger & U. Damrath, COSMO GM, Athens 2007
RMSE Rel.Hum./Temperature (LM Area)
2006 2007
2006 2007
2005
2005
U. Pflüger & U. Damrath, COSMO GM, Athens 2007
RMSE Wind Speed/Geopotential (LM Area)
2006 2007
2006 2007
2005
2005
U. Pflüger & U. Damrath, COSMO GM, Athens 2007
BIAS Rel.Humidity/Temperature (LM Area)
2006 2007
2006 20072005
2005
U. Pflüger & U. Damrath, COSMO GM, Athens 2007
Comparison: Temperature BIAS/RMSE Summer 2006/2007 in LM Area
Summer2006
Summer2007
BIAS
LM Area
LM Area
BIASBIAS
BIASBIAS
RMSERMSE
RMSERMSE
U. Pflüger & U. Damrath, COSMO GM, Athens 2007
Comparison: Temperature BIAS/RMSE Summer 2007 in COSMO-DE/LM Area
Summer2007
Summer2007
BIAS
COSMO-DE Area
LM Area
BIASBIAS
BIASBIAS
RMSERMSE
RMSERMSE
U. Pflüger & U. Damrath, COSMO GM, Athens 2007
Comparison: Temperature BIAS/RMSE June 2007 COSMO-DE/EU
BIASBIAS RMSERMSE
U. Pflüger & U. Damrath, COSMO GM, Athens 2007
Comparison: Temperature BIAS/RMSE July 2007 COSMO-DE/EU
BIASBIAS RMSERMSE
U. Pflüger & U. Damrath, COSMO GM, Athens 2007
Comparison: Temperature BIAS/RMSE August 2007 COSMO-DE/EU
BIASBIAS RMSERMSE
U. Pflüger & U. Damrath, COSMO GM, Athens 2007
Comparison: BIAS/RMSE of COSMO-EU Temperature, Winddirection/speedagainst Radiosonde and Windprofiler Data from Lindenberg
BIASBIAS
BIASBIAS
BIASBIAS
BIASBIAS
BIASBIAS
BIASBIAS
RMSERMSE
RMSERMSE
RMSERMSE
RMSERMSE
RMSERMSE
RMSERMSE
00 UTC00 UTC
12 UTC12 UTC
U. Pflüger & U. Damrath, COSMO GM, Athens 2007
Summary• Verification against analysis confirms some problems
that can be shown with verification against SYNOPs
• Verification against SYNOPs Problems with pressure bias indicate the need of a SSO-scheme Systematic errors of temperature forecasts can be partial explained by
conditional verification
• Verification against upper air observations Effects of pseudotemps and introduction of GME/40km can be seen
clearly Cold bias of temperature within the boundary layer is a COSMO-law,
especially during winter Some systematic differences between COSMO-EU and -DE exist for the
boundary layer Be careful when using new observation systems to verify a model!