Deep Learning for Predicting Winter Temperature in North China

Abstract

It is difficult to improve the seasonal prediction skill of winter temperature over North China, owing to the complex dynamics of East Asian winter and the relatively low prediction skill level of current climate models. Deep learning (DL) may be an informative and promising tool to enhance seasonal prediction, particularly in regions where the underlying mechanisms are not clear. Here, using a DL model based on the Convolutional Neural Network (CNN), we have found that the prediction skill for North China winter temperature (NCWT) can be extended up to five months by considering the remote impact of the Northeast Pacific sea-surface temperature (SST) on North China. Based on historical simulations of winter temperatures in North China, we selected six CMIP5 models with relatively small deviations for training the CNN, and the period chosen for training was 1852–1991. The ERA5 data during 1995–2017 were utilized to evaluate the performance of the CNN. Our CNN shows the best performance in a recent 10-year period (2008–2017), showing a significantly improved level of NCWT prediction skill with a correlation skill of 0.65 at a 5-month lead time, which is much better than the forecast skill of the state-of-the-art dynamic seasonal prediction system. Heat map analysis was used to explore the possible physical mechanisms associated with the NCWT anomaly from the perspective of the CNN; the results showed that the SST over the Northeast Pacific is highly relevant to NCWT prediction. The Northeast Pacific warming in the boreal summer is related to the development of the El Niño event in the coming winter, which may induce NCWT anomalies by atmospheric teleconnection. Climate model experiments support the role of Northeast Pacific warming in the boreal summer on NCWT. The improved capability for prediction from using the CNN may help to establish the energy policy for the coming winter and reduce the economic losses from extremely cold in North China.

1. Introduction

North China has experienced several severe winter seasons during the last 10 years, with an anomalous cold winter in 2011–2012 [1], a 30-year-record cold period in the early winter of 2012–2013 [2], and severe cooling events caused by cold surges in the early winter during 2020–2021 [3]. Mild temperatures ensure that winter wheat—planted mainly in North China—is able to grow safely. The outbreak of the global pandemic COVID-19 has aroused public attention to the early warning of infectious diseases, as a negative winter temperature anomaly may be favorable for the spread of the virus [4]. Improved seasonal prediction of winter temperatures, due to its positive impact on the economy, society, public health, and energy deployment, is important [5,6].

Previous studies revealed that the winter temperature over North China may be affected by remote sea-surface temperature (SST) forcing. Serving as the strongest signal of interannual climate variability in the tropical Pacific, El Niño and Southern Oscillation (ENSO) plays an important role in the winter temperature anomaly of China. An El Niño event can lead to a warm winter climate in China, while a La Niña can lead to a cold winter [7]. On the other hand, interannual SST anomalies in the tropical Indian Ocean can induce a Pacific–North American (PNA) type of winter teleconnection in the mid and high latitudes, which has a significant influence on the winter temperature anomaly in China [8]. Other studies suggested that the Kuroshio region is the key area affecting winter temperature in China [9,10]. The Atlantic Multidecadal Oscillation (AMO) and Pacific Decadal Oscillation (PDO) also provide interdecadal background for the long-term variation of winter temperature in China and affect winter temperature through the modulation of the interannual factors. The intensity of the East Asian Winter Monsoon (EAWM) and ENSO tends to be weaker than normal when the AMO is in its warming phase; the Aleutian Low and Mongolia High (east Siberian High) tends to be strengthened (weakened) when the PDO is in its warming phase [11,12].

Although previous studies emphasized the importance of the remote impact from various climate variabilities, such remote factors are not used for the seasonal prediction of winter temperature over North China. Two dominant modes of air temperature variability over the EAWM region were identified [13], namely, the northern and southern modes. The two dominant modes can explain a significant amount of temperature variability over the entire Asia region [14]. A subtropical North Pacific–Gulf of Alaska dipole signal of SST is a significant precursor of the northern mode, which reaches its peak in the prior summer [15]. Four precursors in the prior boreal summer were selected to predict the winter temperature over China by using the Ensemble Canonical Correlation (ECC), with an unsatisfactory result occurring in North China, showing a sharp contrast with the other regions [16]. A lack of forecasting skill over this region also appears in the SEAS5, which is the state-of-the-art dynamic seasonal prediction system developed by the European Centre for Medium Range Weather Forecasts (ECMWF) [17].

In recent years, breakthrough successes in deep learning (DL) have been achieved in several disciplines [18,19,20]. Applications of DL in the geosciences are in their infancy [21], but DL has shown to be a promising tool in extreme weather detection and classification [22,23], and in atmospheric conditions prediction [24]. A high forecast skill of ENSO with an 18-month lead time based on DL was presented [25], which is regarded as a groundbreaking study for applying DL to climate prediction. Though relatively little research has been done on the seasonal predicting of temperature by DL, DL is still a promising tool in land surface temperature prediction [26,27].

In this study, we investigated seasonal prediction by using the DL method on wintertime temperature over North China. As one of the most representative DL algorithms, Convolutional Neural Network (CNN) shows outstanding performance in processing the data aspects of multi-dimensional arrays with spatial structure [28]. We constructed a CNN for North China December t2m (NCDT) seasonal prediction, with t2m and sea-level pressure (SLP) over the Northern Hemisphere as predictors and NCDT as predictand. The CNN was trained by the fifth phase of the Coupled Model Intercomparison Project (CMIP5) historical data during 1852–1991 [29]; then, the data of the fifth generation ECMWF atmospheric reanalysis (ERA5) during 1995–2017 are used to evaluate the performance of the CNN. A heat map analysis is applied to illustrate the mechanism between predictors and predictand. The explanation for the CNN and the data used in this study is described in Section 2. We estimate the prediction skill of the CNN and discuss a possible physical interpretation for the CNN results using a climate model in Section 3. We summarize our findings in Section 4.

2. Data and Methods

2.1. Prediction System and Data

We used the CNN method for the seasonal prediction of North China December t2m (averaged t2m anomaly of 110°–120° E and of 35°–45° N in December, NCDT hereafter). The architecture of the CNN used in this study is depicted in Figure 1. Three convolutional layers (denoted as Conv1, Conv2, and Conv3) are set up, and between them are two max-pooling layers (denoted as MaxPool1 and MaxPool2). The last convolutional layer (Conv3) is linked to the neurons in the fully connected layer (denoted as FC), and the FC layer is linked to the final output. The parameters of the CNN were set as follows: the number of convolutional filters and neurons of a fully connected layer was set to 50; the size of the mini-batch for each epoch was set to 140; the number of epochs was set to 8000 (an epoch refers to one cycle through the full training dataset) and the learning rate was set to 0.005; the mean squared difference between the predicted and true distributions was defined as the cost function.

The mapping process from the predictors to NCDT in the CNN can be described as follows: After preprocessing, the predictors’ data (input of the CNN) are input into the CNN; then the features of the input are extracted by the filters in Conv1 (here the output of the Conv1 layer was feature maps of the input); then the largest value of each 2 × 2 grid in the feature maps is extracted in MaxPool1 for dimensionality reduction (see Figure 1, showing the size of feature maps reduced after MaxPool1); the features of these feature maps outputted from MaxPool1 are further extracted and compressed in the next two convolutional layers and one max-pooling layer (Conv2, MaxPool2, Conv3); then the fully connected layer (FC) flattens the feature maps into one dimension and finally outputs the NCDT. The weights of the convolutional filters are determined automatically by the process of iteration.

The establishment of the CNN is driven by the training dataset and validation dataset. Each dataset consists of input data (predictors) and labels data (predictand). Based on the historical simulations of NCDT during 1995–2004 (Figure 2), we selected six CMIP5 models (MPI-ESM-MR, MIROC5, GISS-E2-R-CC, FIO-ESM, CMCC-CMS, and ACCESS1-0) with relatively small deviations for training the CNN; the training period was 1852–1991. Note that most models showed relatively poor correlation coefficients, but we assumed that a few CMIP5 models would be able to capture the response of the tropical (or subtropical) SST forcing on East Asia [30,31,32,33]. The ERA5 reanalysis data (observation) during 1995–2017 were utilized as the validation dataset of the CNN. The ERA5 reanalysis data covers the period from 1950 to the present and has good evaluation applicability, especially in terms of the mean SLP and t2m [34]. Considering the high impacts of the SST and EAWM, which can be characterized by the Siberian-Mongolian high on NCDT, we selected monthly t2m and sea-level pressure (SLP) over the Northern Hemisphere (0°–360° E, 25° S–90° N) as predictors. Before inputting the predictors to CNN, it was necessary to preprocess the original data. In this process, because different datasets and different members of CMIP5 have inconsistent levels of resolution, the original predictors’ data were first gridded to a 5 × 5 grid, and then normalized by the standard deviation to unify different atmospheric elements to the same order of magnitude. Such processed data were eventually combined as input data. There is a four-year gap between the training dataset and validation dataset to prevent the atmospheric and oceanic memories in the training period from interfering with the validation period. There is no weighting among the six historical simulations of the CMIP5, and the total sample size of the training dataset is 840 (140 for each CMIP5 model). The models were trained with lead times of 1, 2, 3, …, and 12 months separately. For each month of lead time, we also trained the model for 4 times and obtained 4 ensemble members; the average of the output of the 4 ensemble members was calculated as prediction and for the final analysis.

In addition, to compare the CNN prediction skill, a typical traditional statistical model was set as a baseline: a Multiple Linear Regression model (MLR) was driven by the same training dataset. In the MLR, the dependent variable y can be calculated by the equation in which b_i denotes the weight of each independent variable x_i:

$$y={{\displaystyle \sum }}_1^n{b}_i{x}_i+b_0$$

(1)

where n denotes the number of independent variables and is calculated based on the longitude and latitude numbers of predictors. In this study, the MLR shared the same input and output used in the CNN. The value x_i denotes the t2m and SLP standardized maps over 0°–360° and 25° S–90° N. The value y denotes the NCDT.

Finally, we compared the results of the CNN to the SEAS5 and the MLR model results. The forecast skill was computed as the correlation coefficient between the outputs of these models and the ERA5 Reanalysis.

To understand how the CNN can successfully predict NCDT for a specific lead time, we utilized a heat map [25] to analyze the CNN. The contributions of the predictors at each point to the predictand can be quantified by a heat map. The heat map value for the neuron of the output layer at grid point $\left(x, y\right)$ (indicated as $h^{x,y}$) was calculated by the following equation:

$$h^{x,y}={{\displaystyle \sum }}_{n=1}^N\left\{\tanh \left[{{\displaystyle \sum }}_{m=1}^{M_L}\left(W_{F,m,n}^{x,y}{v}_{L,m}^{x,y}\right)+\frac{b_{F,n}}{X_L{Y}_L}\right]W_{O,n}\right\}+\frac{b_O}{X_L{Y}_L}$$

(2)

where $X_L$ and $Y_L$ denote the dimensions of the feature map in the third convolutional layer. $N$ (50 here) denotes the number of neurons in the fully connected layer, $M_L$ (50 here) denotes the number of feature maps in the last convolutional layer, $W_{F,m,n}^{x,y}$ denotes the weight at the grid point $\left(x, y\right)$ (used to link the mth feature map in the last convolutional layer L to the nth neuron in the fully connected layer F), $v_{L,m}^{x,y}$ denotes the value of the $m\mathrm{th}$ feature map of the last convolutional layer L at grid point $\left(x, y\right)$, $b_{F,n}$ denotes the bias of the nth neuron in the fully connected layer F, $W_{O,n}$ denotes the weight (used to link the nth neuron in the fully connected layer F to the output layer O), and $b_O$ denotes the bias of the output layer O.

After the heat map analysis, a regression analysis was used to investigate whether the relationship between the NCDT and the predictors shown in the heat map was consistent with that in observations and to discuss the possible physical mechanism between the high-value area in the heat map and the NCDT. The regression equation can be calculated as:

$${\tilde{y}}_{\left(i, j\right)}=b_0+b_{1}x$$

(3)

where ${\tilde{y}}_{\left(i, j\right)}$ is the regressed value of $y_{\left(i,j\right)}$ onto $x$ at grid $\left(i, j\right)$; $b_0$, $b_1$ can be calculated as follows, where $m_x$ ($m_{y_{\left(i,j\right)}}$) denotes the mean of the time series of $x$ ($y_{\left(i,j\right)}$), and $n$ denotes the length of the time series.

$$\{\begin{array}{c}{b}_1=\frac{\frac{1}{n}{{\displaystyle \sum }}_{t=1}^n{x}_t{y}_{t, \left(i,j\right)}-m_x·m_{y_{\left(i,j\right)}}}{\frac{1}{n}{{\displaystyle \sum }}_{t=1}^n{x}_t^2-m_x^2}\\ b_0=m_{y_{\left(i,j\right)}}-b_1{m}_x\end{array}$$

(4)

2.2. Skill Metrics

The performances of the CNN, SEAS5, and MLR models were mainly evaluated by the Pearson correlation coefficient (CC), which can be calculated as follows:

$$\mathrm{CC}\left(y,\tilde{y}\right)=\frac{{{\displaystyle \sum }}_{i=0}^n\left(y_i-m_y\right)\left({\tilde{y}}_i-m_{\tilde{y}}\right)}{\sqrt{{{\displaystyle \sum }}_{i=0}^n{\left(y_i-m_y\right)}^2{\left({\tilde{y}}_i-m_{\tilde{y}}\right)}^2}}$$

(5)

where ${\tilde{y}}_i$ is the predicted value of ith sample and $y_i$ is the corresponding observation; $n$ is the number of samples (the length of the time series); $m_y$ is the mean of the time series $y$, and $m_{\tilde{y}}$ is the mean of the time series $\tilde{y}$. Additionally, the root mean square error (RMSE), mean absolute error (MAE), and mean absolute percentage error (MAPE) are calculated as evaluation metrics:

$$\mathrm{RMSE}\left(y,\tilde{y}\right)=\sqrt{\frac{1}{n}{{\displaystyle \sum }}_{i=0}^{n-1}{\left(y_i-{\tilde{y}}_i\right)}^2}$$

(6)

$$\mathrm{MAE}\left(y,\tilde{y}\right)=\frac{1}{n}{{\displaystyle \sum }}_{i=0}^{n-1}\left|y_i-{\tilde{y}}_i\right|$$

(7)

$$\mathrm{MAPE}\left(y,\tilde{y}\right)=\frac{1}{n}{{\displaystyle \sum }}_{i=0}^{n-1}\frac{\left|y_i-{\tilde{y}}_i\right|}{\max \left(\epsilon ,\left|y_i\right|\right)}$$

(8)

where $\epsilon$ is an arbitrary, small, yet strictly positive number in order to avoid undefined results when $y$ is zero.

2.3. Earth System Model

The third version of the Nanjing University of Information Science and Technology Earth System Model (NESM3.0) was used in this study [35,36,37]. The NESM3.0 includes atmosphere, ocean, sea ice, and land surface component models that are fully coupled by an explicit coupler. The atmosphere model has the resolution of T63L47. The resolution of the ocean model is 1°, with the meridional resolution refined to 1/3° over the equatorial region; the vertical resolution is 46 vertical layers, with the first 15 layers in the top 100 m. The NESM3.0 simulates not only reasonable climatology and East Asia winter monsoons, but also the key characteristics of decadal and multi-decadal variabilities [33,38,39,40]. Two experiments were conducted to examine the impact of the Northeast Pacific SST on North China winter temperature. The first model simulation was run using the observed SST anomalies (NESM_NEP), the second, using the climatological SST (NESM_CLIM) specified over the Northeast Pacific domain (170°–130° W, 10°–30° N) with a 1-day nudging timescale. For all the experiments, the model was integrated from 1950 to 2018 using external forces (greenhouse gases, solar constant, aerosol concentration, ozone, etc.) based on the sixth phase of the Coupled Model Intercomparison Project (CMIP6) protocols and data from the past 40 years (1979–2018). The initial conditions for integration were obtained from a historical run based on the CMIP6 protocols.

3. Results

3.1. Performance of CNN

After training on a dataset based on CMIP5 historical simulations, the CNN showed improved prediction skill for the whole validation period (1995–2017) compared to that of the SEAS5 model (Figure 3). The SEAS5 is one of the best seasonal forecast systems in the world, providing forecasts within a 6-month lead time. The CNN showed a high level of prediction skill at a 1-month lead time with a correlation skill of 0.92 and maintained relatively high skill up to a 6-month lead time. The average prediction skill of CNN at 1–6 months was 0.33, outperforming the SEAS5 (−0.053).

The CNN obtained its best performance level in the most recent 10 years (2008–2017, Figure 4a), showing high prediction skill at a 1-month lead time with a correlation skill of 0.95, and maintaining a relatively high level of prediction skill (except for a slight drop at the 2-month lead time) up to a lead time of 5 months. The prediction skill decreased from a 6-month lead time, and suddenly increased at a 10- or 12-month lead time. The high prediction skill at the 1-month lead time may be due to an autocorrelation between the November and December temperatures. The reason for the rapid increase in prediction skill at the 12-month lead time is not clear to us. We suspect that it may result from long-term climate variabilities (e.g., the ENSO, PDO, and AMO). In our comparison with SEAS5, we focused on the forecasting skills from 1- to 6-month lead times in this study. The prediction skill of the SEAS5 was relatively low (correlation skill of 0.53) for a 1-month lead time, with high correlation skills (up to 0.75) at the 2- and 3-month lead times. The prediction skill largely reduced to −0.13 at the 5-month lead time, implying no prediction skill after the 4-month lead time. The CNN has an advantage for a long lead time (>4 months) over the SEAS5. The average prediction skill at a 1–6-month lead time for the CNN was 0.57, outperforming the time of the SEAS5 (0.38). On the other hand, the MLR showed very low prediction skill for all lead times.

Notice that the sudden decrease in prediction skill of the SEAS5 and the difference in the prediction skill between the SEAS5 and the CNN reached its maximum at a 5-month lead time. Hence, we focused on the details of the results from a 5-month lead time.

The NCDT index for the 5-month-lead forecast demonstrated that the CNN model reasonably predicted the NCDT amplitude (Figure 4b). The SEAS5 failed to capture the interannual variability of the NCDT, showing an inverse variation in contrast to the observation, and its magnitude was much smaller than the observed records. Additionally, a phase transition from relatively cold (2008–2013) to relatively warm (2013–2017) shown in observational NCDT was captured by the CNN. The MLR showed false amplitude for most years, and the amplitude of variance was relatively larger than the observation. The results show that the CNN model has the potential to make up for the weakness of the state-of-the-art dynamical models and traditional statistical models and to predict the NCDT with precision to some extent.

3.2. Possible Physical Interpretation

To understand how the CNN model could improve prediction skill for the NCDT with relatively long lead times, we conducted a heat map analysis for the 5-month-lead forecast during the validation periods (Figure 5). The heat map shows how much the predictors (SLP and t2m) at each grid point contributed to the NCDT; dark red (blue) over a specific region represents more positive (negative) contribution to the predictors. The heat map shows that the t2m or SLP anomalies over the subtropical Northeast Pacific, North Pacific, and eastern Beaufort Sea in July mainly contributed to the improved prediction of the NCDT. The SST signal over the Northeast Pacific (170°–130° W, 10°–30° N) highlighted by the CNN may be closely linked to the NCDT. We examined the relationship between the NCDT and July t2m and surface wind from the observations. Figure 6 shows the regressed July t2m onto the NCDT. The results show that the warming over North China in the winter season is related to the Northeast Pacific SST warming and to the cooling of the North Pacific and western America. Compared to the results of the heat map analysis, the Northeast Pacific warming in the boreal summer could be linked to the warming over North China in the boreal winter. We noted that it is difficult to find factors with long-term memory over the continental US. Thus, this topic will be discussed in a further study.

We explore how the Northeast Pacific SST in the boreal summer affects the t2m in December over the North China region using observed and reanalysis data next in Figure 7a, which shows the regressed July t2m and surface wind onto the July Northeast Pacific SST. There is a broad warming in the central astern Pacific. The warming in the North (South) Pacific extends to the Bering Sea (a region south of Chile). Corresponding westerly anomalies are seen in the western and central tropical Pacific, and cyclonic flows occur in the North Pacific. The poleward anomalies may further generate westerlies up to December over the western and central Pacific, inducing El Niño development (Figure 7c). The warming in the eastern Pacific generates an upward motion there and a sinking motion in the western Pacific (Figure 8). The sinking motion may induce anticyclonic flow and increase northward flow, generating warming in the East China Sea and North China (Figure 7c).

To investigate the relationship between the Northeast Pacific and the NCDT, we conduct two additional experiments: the NESM3.0 was integrated for 1950–2017 with the observed SST (NESM_NEP) or climatological SST (NESM_CLIM) over the Northeast Pacific. The simulation with the observed SST was able to capture the observed horizontal pattern (Figure 9); strong warming (cooling) occurs in the central and eastern tropical Pacific (North Pacific), and the corresponding westerlies in the equatorial Pacific and the cyclonic flows in the North Pacific are well simulated. The NESM3.0 simulation with climatological SST failed to reproduce the observed pattern (Figure 10). There is no warming over the Northeast Pacific; and strong warming does appear in the North Pacific, which is not seen in the observation. The equatorial westerlies are relatively weak; the corresponding El Niño signal is weak and shifted southeastward, inducing cooling (warming) in North (South) China. These results suggest that warming in the Northeast Pacific in the boreal summer could induce warming over North China in the boreal winter by developing an El Niño event.

4. Summary and Conclusions

In this study, we developed a CNN (Convolutional Neural Network)-based DL (Deep Learning) model for the seasonal forecast of the NCDT (North China December t2m). We used t2m and SLP (sea-level pressure) as predictors; the model was trained by six CMIP5 (the fifth phase of the Coupled Model Intercomparison Project) historical simulations. Forecasts of SEAS5 (ECMWF’s fifth generation seasonal forecast system) and MLR (Multi Linear Regression model) were utilized for comparison with the CNN. The CNN produces significantly improved prediction skill over the SEA5 and MLR, particularly in the range of a 5-month to 12-month lead time. In addition, the forecast of the CNN with the 5-month lead time depicted the fluctuations of the NCDT during the La Niña event in 2011 and the El Niño event in 2015. The Northeast Pacific SST anomalies in July were shown by a heat map analysis to make a strong contribution to the NCDT anomalies; further analysis supported this result. In July, warming in the Northeast Pacific may be favorable to an El Niño event in the coming winter by generating westerlies in the western and central Pacific. The warming by El Niño generates sinking (rising) motion over the western Pacific (eastern Pacific) and induces anticyclonic flows and northward transport in the Northwest Pacific, inducing warming in the North China region. The skill of the CNN with a 5-month lead time can be explained by such a reasonable mechanism, implying DL is a reliable tool in North China winter temperature seasonal forecast.

Though the results shown here are promising, the following points need further study. Only two predictors were selected here; it is necessary to input more different atmospheric and oceanic elements from different levels to the CNN, which may result in different results. Heat map analysis suggested a significant contribution from the North Pacific and Beaufort Sea; these regions may be linked to the NCDT, a claim which needs to be analyzed in further studies. The prediction skill of the CNN model depends on the validation period. When we increased the validation period from 1995 to 2017, though it still outperformed the SEAS5, the CNN skill score decreased (Figure 3), which may be attributed to a different background mean state. The decadal or multi-decadal variability can induce a different background climate, which may affect the model’s prediction skill. Original data from CMIP5 historical simulations were used to train the CNN; however, the quality of the CMIP5 data is quite low. Correcting the bias of the CMIP5 data may be a way to consider for bettering the performance of the DL model, and the DL has been proved to be a powerful tool for correcting the bias of numerical weather prediction models [41]. These issues will be discussed in our further study.