Errors and uncertainties introduced by a regional climate model in climate impact assessments: example of crop yield simulations in West Africa (2025)

Two sites of approximately one squared-degree (figure 1) are retained for this study; they are characterized by a rain-fed agriculture with low input on poor soils. The Niger site (orange box) is typical of Central Sahel conditions. There, the rainy season lasts approximately from June to September and provides roughly 450–600 mm of rainfall. The Benin site (yellow box) is in the Sudanian zone. There, the rainy season lasts up to six months from May to October providing 1200–1300 mm of rainfall per year.

For each site, the observed meteorological data are taken from the AMMA-CATCH observation system (African Monsoon Multidisciplinary Analysis—Coupling the Tropical Atmosphere and the Hydrological Cycle; www.amma-catch.org). The precipitation is interpolated by kriging over each site using more than 50 rain-gauge measurements for the 2003–2009 period (for a detailed description of the method used, see Kirstetter et al 2013) while the other variables (radiation, temperature, relative humidity and wind) are taken from synoptic stations. These data are considered as reference climate data and they are used to simulate the reference yields.

The ERA-Interim dataset (ERA-I; Simmons et al 2007, Dee et al 2011) consists of a set of global gridded analyses describing the state of the atmosphere, land and ocean wave conditions from 1979 to date. Climate variables from this dataset are used at a resolution of 1.5° to force the (i) WRF and (ii) crop models.

All of the climate downscaling experiments are performed using the non-hydrostatic Advanced Research WRF ARW/WRF model, version 3.3.1 (WRF; Skamarock et al 2008). This model has been used in a large number of studies, some of which focus on West Africa (e.g. Vigaud et al 2009, Flaounas et al 2010, Klein et al 2015), and includes a large choice of physical schemes. Simulations are carried out for a domain extending from 10° S to 30° N and 45° W to 45° E, covering the large West African region, at a resolution of 80 km. Lateral forcings are provided by six hourly ERA-I re-analyses and the integrations were carried out between 1 January 1989 and 31 December 2010 (including one year of spin-up). Only runs from 2003 to 2009 have been retained to be compared with the observed meteorological data and to drive the crop models.

Although our goal is to assess both the influence of the physical parameterizations on the simulated yield bias and the uncertainties linked to the choice of one scheme over another. For computational cost reasons, it was not possible to test all of the combinations. Three sets of experiments were thus designed to address the sensitivity to the model settings and internal variability. As the agriculture of the Sudano–Sahelian zone is highly dependent on rainfall, we focus on parameters that have been previously identified to exert the largest influence on rainfall in Africa (Pohl et al 2011): convective and shortwave (SW) radiation schemes (Set #1), the land surface model (LSM) and land use (LU) categories (Set #2), forced versus stochastic components of the regional climate variability (Set #3, through one ensemble simulation). All of the other parameters are fixed: the Yonsei University planetary boundary layer (Hong et al 2006), WRF Single-Moment 6-Class cloud microphysics (Hong and Lim 2006), the rapid radiative transfer model longwave radiation scheme (Mlawer et al 1997) and the Monin–Obukhov surface layer. More information about the model settings described below is given in table 1 and in the supplementary material.

Set #1. SW radiation and atmospheric convection

SW radiation schemes can be used to estimate the amount of energy available at different altitudes, e.g. the energy of an air parcel at the surface, while convective parameterization schemes (CPSs) are of primary importance for rainfall, especially in regions receiving predominantly convective rainfall, as is the case in West Africa. Set #1 addresses the sensitivity to these two types of settings. Three SW schemes and four CPSs are tested in this paper; their main characteristics are detailed in table 1 (and the supplementary material). For this set of experiments, the LSM is set to the Noah LSM and LU is set to the USGS (United States Geological Survey) LU map.

Set #2. LSM and LU

Atmospheric models are coupled to LSMs to resolve the fluxes at the interface between the continents and the atmosphere. They are forced by atmospheric fields and use global-scale soil and surface LU datasets. LU maps consist of typologies (e.g. 20 or 24 categories such as agriculture, deciduous forests, grasslands, etc.), where each typology is characterized by specific albedo, soil moisture, surface roughness and emissivity values. In this set of simulations, four distinct LSMs and two LUs are tested. Due to the results inherited from Set #1, the SW scheme is set to Dudhia and the CPS is set to Kain–Fritsch with the modified trigger function (KF-t, table 1). This is necessary in order to isolate the specific effects of the LSM and the LU data, when all other settings are constant.

Set #3. Ensemble simulation

In this Set #3, we quantify the amplitude of the uncertainties associated with the chaotic component of the regional atmosphere in our West African domain (hereafter referred to as the 'internal variability' of our model, IV). We then compare it to the changes induced by the settings modified in Sets #1-2. A 10-member ensemble simulation is therefore performed by initializing the integrations on January 1st at 0 h UTC, and then every 6 h, thus providing 10 different initial conditions. This simulation was performed using a unique physical configuration: the Dudhia SW scheme, the KF-t CPS, the Noah LSM and the MODIS LU data.

In order to span some of the uncertainties in crop modeling, which have been shown to be an important contributor to the overall uncertainty in climate impacts (e.g. Asseng et al 2013), we use two different crop models, SARRA-H and EPIC, to simulate Benin maize yields and Niger pearl millet yields.

2.4.1.The SARRA-H crop model

The SARRA-H v32 (Système d'Analyse Régionale des Risques Agroclimatiques-Habillée; http://sarra-h.teledetection.fr/SARRAH_Home.html) crop model (Dingkuhn et al 2003) was developed by CIRAD (Centre International de Recherche Agronomique). Based on a water balance model, it calculates the attainable yield under water-limited conditions by simulating the soil water balance, potential and actual evapotranspiration, phenology, potential and water-limited assimilation, and biomass partitioning (e.g. Kouressy et al 2008, the supplementary material in Sultan et al 2013). Soil nitrogen balance processes are not simulated. The SARRA-H model is particularly suited for the analysis of climate impacts on cereal growth and yield in dry tropical environments (e.g. Sultan et al 2013, Baron et al 2005, Sultan et al 2005). Trial and on-farm data were used to calibrate and validate the model for local varieties of Niger millet (Traoré et al 2011, Sultan et al 2013) and Benin maize (Allé et al 2014). Both varieties have a growth-cycle length of approximately 90 days.

2.4.2.The EPIC crop model

In this paper, we adopt the three-tier hierarchical approach described by Morse et al (2005; figure 2). A comparison of the ERA-I and WRF simulations with the corresponding observed variables is carried out in a tier-1 evaluation. Since Oettli et al (2011) showed that, among the climate variables derived from the RCM, precipitation and radiation are the most likely to induce errors in the simulated crop yields, this paper only concentrates on the effects of these two variables; the other climate variables (temperature, relative humidity and wind) are taken from in situ observations. In a tier-2 evaluation, the essential reference data are precipitation and radiation (as in tier-1), but the data are integrated into the two crop models to produce an estimate of the crop yields. However, the tier-2 evaluation does not validate the crop model per se since the objective of the tier-2 evaluation is not to produce an accurate yield prediction, as is the case for the tier-3 evaluation. By using a tier-2 evaluation, we will be validating the ERA-I and WRF output against the observed meteorological in situ data, but in a situation where the precipitation and radiation fields have been integrated and synthesized to produce crop yields. For both crop models, in situ climate data (our reference climate) are first used to simulate the reference yields for Niger millet and Benin maize over the 2003–2009 period. Then, the crop models are forced using ERA-I and climate variables simulated by the WRF model (three sets of experiments, see paragraph 2.3). The deviation of these alternative simulated yields from the reference yields gives an estimate of the errors that result from forcing the climate variables.

In this paper, we consider the climate inputs from the WRF or ERA-I models as the most suitable for crop yield simulations as they minimize the distance between the simulated yields and the reference yields obtained with in situ meteorological data. We were also able to validate the resulting simulated yields against the observed yield data (tier-3 evaluation); however, since the objective of this paper is to assess the sensitivity of the simulated crop yield to the RCMs biases rather than predict accurate yields in the two sites, the results reported in this paper are mainly dedicated to the tier-1 and tier-2 evaluations.

We used the same metric for the tier-1 and tier-2 evaluations to measure the distance to the observed in situ data. We calculated the mean bias error (%MBE) of the climate inputs and simulated yields relative to the observed seasonal climate variables (tier-1 evaluation) and simulated yields using the observed in situ climate variables (tier-2 evaluation). The %MBE is defined as

where x_0i is the value of the baseline and x_i, is the value to be tested.

The uncertainties are estimated by calculating the coefficient of variation (CV) of the variables derived from the different runs (climate variables or yields calculated using the different WRF configurations or the different members of the ensemble simulation). The CV is defined as follows:

where σ is the standard deviation of the values of the considered variable and μ is the averaged value of the considered variable.

Simulated yields using ERA-I data directly show biases with an amplitude higher than 20%, reaching up to −77.82% for the SARRA-H Niger millet yields (figure 3: gray dots, and table 2). The use of the WRF model to downscale the ERA-I climate data can reduce the bias in the simulated yields at both sites and for both crop models; for example, the amplitude of the mean bias error (%MBE) for the ensemble mean (in dark green) is lower than 15%. Yet, the use of the WRF model can also introduce large uncertainties due to the choice of parameterization schemes (in red): the %MBE for the simulated yield using different physical configurations ranges from −100% (crop failure) to +84.48% (table 2) and the CV for its %MBE is higher than 39% (table 2). The effect of the choice of each parameterization will be detailed later in this paper. The internal variability of the WRF model, even for a moderately sized domain (see figure 1), notably compared to the much larger CORDEX-Africa domain (Giorgi et al 2009), also induces non-negligible uncertainties in the simulated crop yields (green dots). The largest difference of yield %MBE between two members of the ensemble simulation ranges between roughly 14% (18.08%–4.36%) for the EPIC Benin maize yields and 34% (25.63%+8.32%) for the SARRA-H Niger millet yields (table 2). However, the uncertainties in the crop yield induced by the internal variability of the WRF model remain lower than those induced by the model parameterizations with an inter-member yield %MBE CV lower than 10% (table 2).

Depending on the crop model and the considered crop and site, the simulated yields are not sensitive to the same climate variable characteristics and the yield %MBE differs (figure 3). In this section, the main climate characteristics leading the simulated crop yields are identified.

Precipitation and radiation errors result in errors in the simulated yields (figure 4). When considering similar variations of the climate variables, the climate variable with the most marked influence on the %MBE of the simulated crop yields depends on the location and crop model considered.

Water is a limiting factor at the first order, especially for the Niger millet yields which are highly sensitive to a change in the total rainfall (figure 4, top). This is the case even for the Niger millet cultivar simulated with the SARRA-H model, which requires less water to reach its attainable yield. Once the water demand (approximately 600 mm) is met, the simulated SARRA-H Niger millet yields no longer depend on the total water amount. The total rainfall is not a limiting factor for Benin maize growth. When using the EPIC model, high rainfall intensity leads to nitrogen leaching, leading in turn to nitrogen stress which then restricts crop development and grain formation (figure 4(d)). In the SARRA-H crop model, the rainfall intensity has no influence on soil fertility and radiation is the most limiting climate variable for the Benin maize yield (figure 4(c)).

Ranking of the climate variables relative to the explained variance of simulated yield when included in a multiple linear regression. The climate variables taken into account are the annual cumulative rainfall, mean daily rainfall and mean daily radiation. The climate variables on the left explain the largest fraction of the variance whereas the variables on the right explain the lowest fraction for the simulated SARRA-H Niger millet, the EPIC Niger millet, the SARRA-H Benin maize, the EPIC Benin maize and the simulated SARRA-H Niger millet when the annual rainfall amount remains below 600 mm.

Large biases in the simulated Niger millet yields (up to −77% for SARRA-H) using the ERA-I model come from the strong biases in the ERA-I annual rainfall and rainfall intensity (roughly −70% and −74%, respectively; table 2). The underestimation of the rainfall intensity in the ERA-I model results in an overestimation of the EPIC simulated Benin maize yields, while the underestimation of radiation in the ERA-I model results in the underestimation of the yield by roughly −34%.

Using the WRF model to downscale the ERA-I climate data, the model-output climate variables and then the simulated crop yields generally show larger biases than the ERA-I climate variables. These biases are strongly dependent on the chosen physical configuration of the WRF model (CV, table 3). Yet, these biases are strongly reduced for some configurations. The following section investigates the influence of each tested physical parameterization on the simulated climate variables and yields.

The choice of the WRF physical parameterizations strongly modulates the climate variables relevant to crop yield simulations, thus resulting in high uncertainties in the crop yields (table 2 and figure 3). Among the different parameterizations, the choice of a LSM that takes the moisture dynamics into account is of primary importance for the simulation of realistic rainfall: when there is no soil moisture prediction ('No LSM' or 'Thermal diffusion' scheme), there is no moisture recycling and then rainfall and the simulated crop yields are generally dramatically underestimated, if not null (table 2 and figure 5(a)), especially in Niger where a larger part of the humidity comes from recycling. Climate variables derived using the other LSMs lead to a %MBE amplitude for the Niger millet yields that is lower than 15% (table 3), except for the Pleim–Xiu scheme, for which the overestimation of the precipitation leads to an overestimation of the EPIC Niger millet yields by roughly 29%. The Pleim–Xiu scheme results in simulated Benin maize yields that are significantly lower than in the other schemes; the underestimation of the EPIC Benin maize yields is due to the high daily rainfall overestimation in the Pleim–Xiu scheme while the underestimation of the SARRA-H Benin maize yield is due to an underestimation of the incoming radiation (table 3). The Pleim–Xiu LSM tends to favor larger rainfall amounts than the RUC and Noah LSMs, as already highlighted over East Africa by Pohl et al (2011), who linked it to a larger simulated surface evapotranspiration in the Pleim–Xiu model. These authors suggest that the simplified vertical profiles of soil moisture in the Pleim–Xiu LSM (two layers measuring 1 and 100 cm deep) may accelerate evapotranspiration processes. In turn, this might increase soil moisture recycling and cloud cover and decrease incoming radiation.

The second most important parameterization for the crop yield simulation is the CPS. CPSs have a strong influence on both rainfall and radiation characteristics and can result in significant uncertainties in the simulated crop yields (figure 5(b)). The %MBE of the SARRA-H simulated Benin maize yields goes from less than −50% to over +100% and it is driven by radiation. The %MBE of the EPIC simulated Benin maize yields has its smallest value when the BMJ CPS is used. This is explained by the rainfall distribution. In West Africa, the BMJ scheme leads to a more realistic distribution of the precipitation than the Kain–Fritsch and Tiedtke schemes (figure S2 and table 3). The latter two schemes systematically overestimate the rainfall intensities, and the KF-t scheme results in an underestimation of the number of rainfall events of less than 25 mm, compensated by an overestimation of the number of daily rainfall events higher than this threshold. However, this good performance obtained with the BMJ scheme in terms of the daily rainfall simulation cannot be generalized to other tropical regions, as shown for instance by Crétat et al (2011b) over Southern Africa. Using the EPIC model, the simulated Niger millet yields are more influenced by the CPS scheme (with the %MBE ranging from 16% to over 84%) than the yields simulated with the SARRA-H model (table 2). When using the SARRA-H crop model, a large overestimation of the rainfall does not result in comparable overestimations of the simulated Niger millet yields (figure 4(a)).

Among the tested radiation schemes, the Dudhia SW scheme is the one that minimizes the overestimation of both total rainfall and rainy day intensity (table 3); this tendency to favor drier conditions is consistent with the results presented in Pohl et al (2011). For the EPIC yields, this leads to lower biases than the other SW schemes. The SARRA-H yields are not very sensitive to overestimations of the rainfall amounts (for Niger millet) or intensity (for Benin maize). The RRTMG and Goddard schemes do not lead to high biases in the SARRA-H yields (figure 5(c)); for both sites, radiation is the main driver of the SARRA-H yields. The simulated radiation shows relatively low uncertainties (the radiation biases are of the same order for all SW schemes), leading to low uncertainties in the SARRA-H yields. Thus the SW schemes, mainly influencing simulated rainfall, induce high uncertainties in the EPIC yields but have a relatively lower influence on the crop yields simulated with the SARRA-H model.

The LU maps tested in this study do not produce large differences in the simulated yields (figure 5(d) and table 3).

Finally, within a crop-modeling framework and over the tested parameterizations, it is very important to choose an appropriate LSM, given the high biases in the rainfall induced by using a LSM that is too simplistic. Then, as the CPS has a significant influence on both rainfall and radiation, this parameterization should be carefully set. The SW radiation scheme mainly drives precipitation and can be important as well. In contrast, the choice of the LU scheme seems to only have a moderate influence on rainfall and radiation and thus on the yields.

In this section, we run the same experiment 10 times by modifying the initial conditions at the first model time step. The physical package and all other settings are constant in all of the simulations (members) of the ensemble. This allows disentangling the forced (reproducible) and stochastic (internal, or irreproducible) fractions of the simulated climate variability. Our idea here is to compare and rank the magnitude of the uncertainties due to the chaotic component of the regional climate, materialized by the internal variability, and those related to modifications of the physics of the model. At the first order, rainfall is generally the main driver of simulated crop yields; the WRF model was then set to the configuration that minimizes the rainfall biases. This configuration combines the Dudhia SW scheme, the KF-t convective scheme, the NOAH LSM model and MODIS LU data.

We previously showed that when the WRF model is used with an appropriate configuration to downscale the ERA-Interim climate data, the bias in the simulated yields is considerably reduced (figure 3). This can be linked to the dramatic decrease in the simulated rainfall amount and the intensity biases (table 2, ensemble set). However, the internal variability of the WRF model also introduces non-negligible uncertainties in the simulated crop yields. The 10-member simulation leads to uncertainties in rainfall, for which the annual rainfall %MBE ranges from 0% to 36% in Niger and from −2.58% to 15% in Benin (table 2). These uncertainties in the simulated yields can be linked to those in the mean states of the radiation and rainfall characteristics as well as to intra-seasonal variations (e.g. Ramarohetra et al 2013). Following Crétat et al (2011a), the reproducible fraction of day-to-day rainfall and radiation is calculated for the June–July–August–September period of each year as the ratio between V(X), the daily variance of all members (i.e. 122 days duplicated 10 times) and the daily variance of the ensemble mean (122 days): .

On average, over the seven years, the reproducible fraction of the day-to-day rainfall is approximately 30% for both sites (29% in Niger and 31% in Benin). The inter-annual variability of these scores remains quite low: it ranges from 21% to 35% in Niger and from 25% to 37% in Benin. Therefore, even for a moderate-sized domain (figure 1), the rainfall intra-seasonal variability is mainly driven by the stochastic component of the WRF model rather than by ERA-I forcing. Although the scores are a little higher, the day-to-day radiation variations are also strongly led by the model's internal variability, with averaged reproducibility values of 75% and 58% for the Niger and Benin sites, respectively.

In order to link the intra-seasonal precipitation descriptors to the simulated yield variability in the ensemble experiment, the reproducibility of these descriptors has been investigated for the year 2006 (table 5). In Niger, the least reproducible descriptors are the mean intensity of precipitation, the length of the rainy season and the number of dry spells with coefficients of variation (CV) across the ensemble members of 0.54, 0.47 and 0.40, respectively. In Benin, the least reproducible descriptors are the number of dry spells (CV=0.23), the mean total rainfall (0.14) and the mean intensity of precipitation (0.13).

CV statistics are preferred to standard deviations to better compare the various descriptors considered here, since they have different units. Large CV values denote strong differences from one ensemble member to another, and are thus indicative of a strong internal variability (i.e. chaotic or local component) and a weak to moderate large-scale forcing prescribed by the re-analyses.

Intra-seasonal precipitation descriptors controlling the simulated yield variability within the ensemble experiment were identified using a multiple regression analysis. For the SARRA-H Niger millet and the EPIC Benin maize, the simulated yield variability is mainly driven by the number of dry spells with coefficients of determination (R²) of 0.52 and 0.31, respectively (not shown). For the EPIC Niger millet, the total rainfall during the rainy season explains 0.47 of the yield variability while the SARRA-H Benin maize yield variability is mainly explained by the number of rainy days (R²=0.4).

The variability of the mean intra-seasonal radiation and precipitation descriptors explains 0.58–0.99 of the yield variability (not shown). However, the timing of events such as dry spells or heavy rainfall explains up to 0.42 of the yield variability.

Therefore, internal variability of the WRF climate model has a non-negligible impact on the simulated rainfall and radiation mean state and intra-seasonal variability, and this translates into uncertainties in the simulated crop yields that should be taken into account.