The verification results for Hirlam have given some cause for concern. In particular, it has been noticed that the rms scores for sea-level pressure are worse than for other models, and they tend to grow more rapidly with forecast range. It was also noted, in a comparison of scores for the first three months of this year, that the scores for the INM implementation of the Hirlam model were better than the other implementations (see note on verification, this issue, page 88). The most obvious differences between the Spanish configuration and others are that the forecast domain is further south and that a non-rotated latitude/longitude grid is used. As the reference HIRLAM verification is used, with an agreed list of observations (the EWGLAM list), suspicion points to the latter difference.

Could there be some problem (either a coding error or a more fundamental difficulty) associated with the rotated grid, leading to the poorer performance? To investigate this problem, José Antonio García-Moya carried out parallel tests with two grids, a regular non-rotated grid (REG) and a rotated grid (ROT). The forecast domains were not the same (they cannot be for such grids) but covered approximately the same area. The reference version of Hirlam at ECMWF was used and identical observation files were used for verification of the two parallel runs. The runs were for a period of 5 days in February, 1997 (during FASTEX). José found that the scores for the REG run were markedly better than for the ROT run. This suggests some deficiency, possibly a coding error, with the rotated grid.

To investigate the problem further, several tests have been done in Dublin. In all cases, ECMWF analyses are used on the boundaries. The HIRLAM model at Met Éireann (version 2.7) was used. Verification was done using ECMWF analyses as the `truth'. In a preliminary investigation, the processing of boundary data was examined, and it was found that the values input as boundary conditions for the model (on a rotated grid) were consistent with those acquired from the ECMWF operational dissemination (on a regular lat/long grid). Visual inspection of the analysis fields before and after initialization, and examination of the changes induced by this process, showed up no apparent anomalies and gave no reason for suspicion of this component.

To corroborate the findings of José, a series of 25 forecasts over a six-day period (during FASTEX) was run with approximately coincident REG and ROT grids. Full analyses were performed each six hours. The results were completely consistent with those found in Madrid: the scores for the REG run were markedly better than for the rotated grid.

To eliminate the analysis as a factor, another (short) parallel run was done, using ECMWF analyses as initial data as well as boundary values. The results were very much as before. This effectively eliminates the HIRLAM analysis as a likely source of the problem.

Next to be considered was a speculation that the treatment of orography might be problematical. This arose because the spatial pattern of error often appeared to be linked to orography, especially around Greenland. An area in the southern Indian Ocean was selected and two grids, one rotated and one not, were defined, having coincident eastern boundaries. (The few islands in the region were eliminated by artificallly setting orographic height to zero everywhere.) A series of forecasts over six days was run, and verifications (against analyses) were made over two sub-areas. For a verification region adjacent to the (common) eastern boundary, the REG and ROT scores were very close; both forecasts are strongly confined by the boundary conditions. For a central verification region, the REG run scored better. The western boundary of the REG grid was much closer to the verification region than that of the ROT grid. The differences between the runs were comparable to those found in the European area. Thus, the phenomenon appears not to be due to orography, which was now tentatively eliminated as a factor. (However, the similarity of the spatial error pattern to the orography remains to be studied separately).

A further series of forecasts were done for the Northern Hemisphere, now using a regular grid covering a region larger that the rotated grid. Here we will compare three parallel runs, verified over two different areas. In Fig. 1 we show the two unrotated grids, a smaller inner region (REG1) and a larger outer region (REG2). Sandwiched between them is a region defined by a rotated grid (ROT). Also shown are two verification areas, a central one (AREA 1) and an area displaced towards the west (AREA 2). For each parallel run, we show the rms errors of the 500 hPa geopotential height, verified against analysis. The scores are presented as averages over the full suite of 25 forecasts, as a function of forecast length. (Note: For the REG1 run the HIRLAM analysis was used as initial data; for the REG2 and ROT runs, ECMWF analysis were used; independent experiments indicate that this difference is unimportant in the present context).

Fig. 2 shows the scores for AREA 1. The forecast REG1, using the region whose boundary is nearest to the verification area, gives the best scores. The ROT and REG2 runs give successively poorer scores. The difference between the scores increase steadily with forecast length. Recall that, in all cases, analyses are used on the boundaries. There is a strong indication that the boundary conditions, which strongly constrain the model solution, have an increasingly dominant effect on the scores as the forecast range increases. To check this, we calculated the scores for the same three runs, now using a verification area (AREA 2 in Fig. 1) moved closer to the western boundaries. The results are shown in Fig. 3. Now the differences between the scores for the three forecast regions are more pronounced. This is consistent with the relative distances from the verification area to the boundaries being greater. Moreover, the overall error level is much lower, especially for the REG1 forecasts. This is consistent with the REG1 region boundary being closest to AREA 2. These results support the hypothesis that the analysed boundary values dominate the scores at longer forecast ranges, and explain the discrepancies between the three forecast series.

Conclusions

·

All our results can be explained in terms of the position of the boundary relative to the verification domain. As analysed boundary conditions were used, the error is necessarily zero on the boundaries.

·

If the boundary of the regular grid is closer than that of the rotated grid to the verification region, then the REG scores are better, and the difference increases with forecast range.

·

If the boundary of the rotated grid is closer than that of the regular grid to the verification region, then the opposite is the case: the ROT scores are better, and more so with increasing forecast range.

·

There is no justification for suspecting a coding error in the rotated model, or any more fundamental shortcoming with the rotation procedure.

·

There is no evidence that orography explains the differences between the REG and ROT scores.

·

There is no reason to suspect problems in the pre-processing of boundary data, the analysis or the initialization.

All runs used analysed boundary conditions. Although the discrepancies found between the REG and ROT forecasts could be explained, we have not yet accounted for the differences found in an operational context. In that case, forecast boundary conditions are used, so that much of the above reasoning does not apply. We are also aware of serious bias errors in the forecasts. These questions remain to be investigated, as a matter of urgency.