A Global Deep Learning Perspective on Australia-Wide Monthly Precipitation Prediction ^†

Abstract

Gaining a deep understanding of precipitation patterns is beneficial for enhancing Australia’s adaptability to climate change. Driven by this motivation, we present a specific spatiotemporal deep learning model that well integrates matrix factorization and temporal convolutional networks, along with essential year-month covariates and key climatic drivers, to analyze and forecast monthly precipitation in Australia. We name this the spatiotemporal TCN-MF method. Our approach employs the precipitation profiler-observation fusion and estimation (PPrOFusE) method for data input, synthesizing monthly precipitation readings from the gauge measurement of the Bureau of Meteorology (BoM), the JAXA Global Satellite Mapping of Precipitation (GSMaP), and the NOAA Climate Prediction Center Morphing (CMORPH) technique. The input dataset spans from April 2000 to March 2021 and covers 1391 Australian grid locations. To evaluate the model’s effectiveness, particularly in regions prone to severe flooding, we employ the empirical dynamic quantiles (EDQ) technique. This method ranks cumulative rainfall levels, enabling focused analysis on areas most affected by extreme weather events. Our assessment from April 2021 to March 2022 highlights the model’s proficiency in identifying significant rainfall, especially in flood-impacted locations. Through the analysis across various climatic zones, the spatiotemporal TCN-MF model contributes to the field of continent-wide precipitation forecasting, providing valuable insights that may enhance climate change adaptability strategies in Australia.

1. Introduction

Accurate precipitation prediction is of paramount importance due to its vast implications for agriculture, water resource management, and disaster risk reduction [1,2,3]. In Australia, recognized for its marked climatic variations, the need for precise prediction tools is especially pronounced. The country’s history, dotted with prolonged droughts [4,5] and intense rainfall events [6,7,8], coupled with more recent disasters such as the 2019 Queensland floods [9] and the 2022 New South Wales deluge [10], underscores this urgency.

Obtaining and utilizing precipitation data is challenging due to their high-dimensional spatiotemporal characteristics. Australia’s precipitation patterns exhibit considerable variability, further complicating data fusion, especially when integrating high-precision but spatiotemporally uneven rain gauge network data with uniformly distributed, albeit less precise, satellite data. To address this, we employ the precipitation profiler-observation fusion and estimation (PPrOFusE) method [11] that merges the precision of the gauge data with the broad coverage of multi-satellite data to better approximate the ground truth. Central to our analysis, we fuse data from three primary sources: the Australian Bureau of Meteorology (BOM), the Japan Aerospace Exploration Agency (JAXA) Global Satellite Mapping of Precipitation (GSMaP), and the National Oceanic and Atmospheric Administration (NOAA) Climate Prediction Center Morphing technique (CMORPH). We then process and analyze the fused dataset, specifically targeting Australia’s monthly precipitation spanning from April 2000 to March 2022, by leveraging our model.

The Australian Community Climate and Earth-System Simulator–Seasonal (ACCESS-S) is a cornerstone of the Australian Bureau of Meteorology’s seasonal forecasting system [12,13]. Efforts to merge rainfall forecasts using various statistical models have underscored the importance of predictive measures. However, forecast accuracy from such models often remains inconsistent across different seasons and locations, highlighting the inherent limitations of traditional statistical models [14]. Given these issues, researchers have looked beyond traditional approaches to enhance the forecasting capability. It becomes clear that integrating more advanced models may offer a more effective solution for improving forecast power.

Deep learning has emerged as a powerful tool in advancing precipitation prediction, particularly for complex spatiotemporal datasets such as Australia’s rainfall patterns. While various neural network-based models have been employed for monthly rainfall forecasting in Australia [15,16,17], these studies often focus on specific cities or regions, lacking a comprehensive continent-wide perspective.

Addressing this gap, our research introduces the spatiotemporal TCN-MF model, a framework designed to offer a holistic prediction for Australia’s rainfall. This model, inspired by the deep global local forecaster (DeepGLO, [18]), adapts its architecture to encompass a vast and evenly distributed dataset of 1391 locations across Australia. Our adaptation includes integrating matrix factorization (MF) [19] with temporal convolutional networks (TCNs) [20], along with crucial seasonal covariates and key climatic drivers such as the Southern Oscillation Index (SOI), Dipole Mode Index (DMI), and Southern Annular Mode (SAM). This integration aims to refine predictive accuracy and offer valuable insights for ACCESS-S precipitation forecasts, capturing both global patterns and local nuances in precipitation data.

This comprehensive approach enhances the granularity of our predictions and ensures a representative model that accounts for the diverse climatic conditions across different parts of Australia. To demonstrate the model’s efficacy, particularly in predicting higher rainfall levels, we first employ the recently developed empirical dynamic quantiles (EDQ) technique [21]. This method allows us to rank the cumulative rainfall levels across the 1391 Australian locations in our dataset. Prioritizing our analysis, we focus on locations within the top 5% of these rankings, often areas prone to severe flooding and regions that have recently experienced major flood events. This targeted approach enables us to closely examine and compare the observed and predicted rainfall patterns in these high-priority areas. Furthermore, our analysis also encompasses areas with lower and median rainfall levels. This inclusion provides a comprehensive perspective on precipitation across different climatic zones. Our study, encompassing a wide range of rainfall scenarios, marks a significant advancement in the field of continent-wide precipitation forecasting.

2. Data Description

2.1. Australian Precipitation Data Fusion

The study utilizes an integrated dataset derived from three distinct sources to address the limitations inherent in each: the Australian Bureau of Meteorology (BOM), the Japan Aerospace Exploration Agency’s Global Satellite Mapping of Precipitation (the JAXA GSMaP), and the National Oceanic and Atmospheric Administration’s Climate Prediction Center Morphing technique (the NOAA CMORPH). BOM data, while highly accurate, are location-dependent and lack a gridded format, limiting their coverage. Conversely, GSMaP and CMORPH offer gridded formats and complete coverage, utilizing precipitation retrievals from orbiting satellites and estimates derived from passive microwave satellite scans, respectively. However, each satellite-based method has its limitations in terms of accuracy during certain rainfall conditions, particularly in regions with intricate topography or near coastlines.

To overcome these challenges and create a robust and comprehensive dataset, we employ the precipitation profiler-observation fusion and estimation (PPrOFusE) method, details of which can be found in [11]. This approach combines the precision of ground-based gauge measurements from BOM with the extensive coverage provided by satellite-derived estimates from GSMaP and CMORPH. The PPrOFusE method involves Linear Modeling and Spatial Modeling. The fused dataset covers 1391 grid locations uniformly distributed across Australia, spanning from April 2000 to March 2022, and boasts a spatial resolution of $1^{\circ }\left(\mathrm{lon}\right)\times 0.5^{\circ }\left(\mathrm{lat}\right)$. This integrated approach offers 22 years of monthly temporal resolution data, enhancing our ability to predict and analyze precipitation dynamics, continent wide. A detailed representation of the precipitation data from the three original sources and the fused data for March 2022 is illustrated in Figure 1.

2.2. Climate Drivers

To fully comprehend precipitation patterns in Australia, one needs to go beyond historical data. It is crucial to consider major climate drivers such as the Southern Oscillation Index, the Dipole Mode Index, and the Southern Annular Mode. These indices significantly influence Australia’s climate, affecting rainfall and temperature variations [13,22]. SOI (Southern Oscillation Index): The SOI measures air pressure differences between Tahiti and Darwin, indicating El Niño and La Niña events. Positive SOI values typically correspond to La Niña, leading to higher rainfall in Australia, while negative SOI is indicative of El Niño, associated with reduced rainfall [23,24].

DMI (Dipole Mode Index): Representing the temperature gradient across the tropical Indian Ocean, the DMI is crucial for forecasting rainfall in Southern and Eastern Australia, especially during winter and spring. Positive DMI values are linked to reduced rainfall, whereas negative values are associated with increased rainfall [25].

SAM (Southern Annular Mode): SAM describes the north-south movement of the westerly wind belt that encircles Antarctica. When SAM is positive, the belt of strong westerly winds contracts towards Antarctica, typically resulting in reduced rainfall in parts of southern Australia during the cooler months. In contrast, a negative SAM pushes the westerly winds further north, increasing the likelihood of higher rainfall in the region.

3. Method

Our study provides a global deep learning perspective on Australia-wide monthly precipitation forecasts. Figure 2 presents an overview of our forecasting framework.

3.1. Temporal Convolutional Networks (TCNs)

In this study, we integrate temporal convolutional networks (TCNs) with matrix factorization (MF) to enhance precipitation forecasting. This integration combines the strength of TCNs in capturing temporal patterns with the dimensional reduction capabilities of MF, aiming to improve both local and global prediction accuracy.

TCNs evolve from 1D Convolutional Neural Networks (CNNs), which lay the groundwork for extracting spatial features. In 1D CNNs, an input sequence is represented as $X\in {\mathbb{R}}^{T\times C}$, where T and C denote the length of the sequence and the number of channels, respectively. Convolution layers apply filters to the input, generating feature maps as described below:

$$G_n(i,m)=\sum _{j=0}^{k-1}\sum _{c=1}^C{W}_{n,m}(j,c)·x(i-j,c)$$

(1)

Here, k denotes filter size, $W_{n,m}$ represents the weights of filter m in layer n, and $G_n$ denotes the output feature maps. In TCNs, this concept is extended to specifically handle sequential data, incorporating causal and dilated convolutions, alongside residual blocks [20].

Causal and Dilated Convolutions: TCNs use causal convolutions to ensure outputs depend only on current and past inputs, achieved by padding inputs on the left. Dilated convolutions, which introduce gaps between inputs in the convolution process, expand the receptive field without added computational cost. The dilation factor d increases the model’s temporal coverage. The operation is defined as:

$$G_n(i,m)=\sum _{j=0}^{k-1}\sum _{c=1}^C{W}_{n,m}(j,c)·x(i-d·j,c)$$

(2)

Residual Blocks: To facilitate the training of deeper networks, TCNs utilize residual blocks. Each block comprises two dilated causal convolutions, followed by weight normalization, ReLU activation, and then dropout. The output of the block is then obtained by adding the transformed input, $\Phi \left(x\right)$, to the original input x, optionally via a $1\times 1$ convolution when necessary. This addition is passed through a ReLU activation to form the final output of the block, $z=ReLU(x+\Phi (x\left)\right)$.

3.2. Matrix Factorization

Following the exploration of TCNs for capturing temporal patterns, we integrate matrix factorization (MF) techniques to further enhance our model’s capability in handling high-dimensional datasets. MF is instrumental in approximating a time series matrix $Y\in {\mathbb{R}}^{n\times T}$ by breaking it down into lower-dimensional embeddings $F\in {\mathbb{R}}^{n\times k^{*}}$ and $X\in {\mathbb{R}}^{k^{*}\times T}$ [19]. Here, F and X represent series and time latent factors, respectively. The factorization aims to minimize the error between the observed and estimated data while including penalty terms to control model complexity:

$$\underset{F,X}{min}{\parallel Y-FX\parallel }_F^2+{\lambda }_f{\parallel F\parallel }_F^2+{\lambda }_x{\parallel X\parallel }_F^2$$

(3)

where ${\parallel ·\parallel }_F$ denotes the Frobenius norm, and ${\lambda }_f,{\lambda }_x$ are regularization coefficients. Alternating least squares is typically used for optimization, iteratively refining F and X.

Temporal Regularized Matrix Factorization (TRMF) extends this by integrating temporal dependencies [27]. It introduces a temporal regularization term $R_x\left(X\right)$ to the optimization problem, promoting consistency with a temporal model $M_{\Theta }$:

$$\underset{F,X,\Theta }{min}{\parallel Y-FX\parallel }_F^2+{\lambda }_f{\parallel F\parallel }_F^2+{\lambda }_x{R}_x\left(X\right|\Theta )+{\lambda }_{\theta }{\parallel \Theta \parallel }_F^2$$

(4)

where $\Theta$ represents model parameters capturing temporal dynamics, and ${\lambda }_{\theta }$ is the regularization coefficient for $\Theta$. This formulation is optimized through an iterative process, adjusting $F, X$, and $\Theta$ alternately.

Further advancing this concept, temporal convolution networks matrix factorization, as introduced by Sen et al. [18], unites the decompositional power of MF with the nonlinear modeling capabilities of TCNs. Here, a TCN is trained alongside the factorization process, incorporating a temporal regularization term:

$$R_x\left(X\right|\mathcal{N}):=\frac{1}{\left|T\right|}\sum _{t\in \{1,..,T\}}{\parallel X[:,t]-\mathcal{N}\left(X[:,t-1]\right)\parallel }_2^2$$

(5)

Here, $\mathcal{N}$ symbolizes the TCN, and $\{1,...,T\}$ is the set of time indices in the training set. The optimization aims to balance the factorization’s approximation of training data with the temporal dynamics captured by the TCN:

$$\underset{F,X,\mathcal{N}}{min}{\parallel Y_{\mathrm{tr}}-F{X}_{\mathrm{tr}}\parallel }_F^2+{\lambda }_T{R}_x\left(X_{\mathrm{tr}}\right|\mathcal{N})$$

(6)

where $Y_{\mathrm{tr}}$ and $X_{\mathrm{tr}}$ represent the training matrices, and ${\lambda }_T$ is the regularization parameter for the temporal term.

4. Analysis of Fused Precipitation Data

In this study, we utilize fused precipitation data for Australia, collated from April 2000 through March 2021, to build a multi-month precipitation forecast platform for the entire continent. The development of this platform integrates the EDQ method, enhancing the ability to detect significant rainfall events, particularly in flood-impacted locations. Our methodologies is visually represented in Figure 2.

The procedure is summarized as follows:

• The time series data under study consist of fused Australian monthly precipitation data, denoted by $Z\left(t\right)$ for $t=1,\cdots ,252$. We apply a logarithmic transformation to the raw data, resulting in $Y\left(t\right)$ for $t=1,\cdots ,252$, to reduce data variability. This dataset spans $n=1391$ dimensions, each representing a unique observation location.
• Fit the global matrix factorization model to $\left\{Y\right(t),t=1,\cdots ,252\}$.
  Step 1: Initialization

  (a)

  Firstly, initialize the TCN to ${\mathcal{N}}^{\left(0\right)}(·)$.

  (b)

  Then, the factors F and $X^{(tr)}$ are trained using ${\mathcal{N}}^{\left(0\right)}(·)$.

  Step 2: Alternate training cycles for training

  $${\mathcal{L}}_G\left(F,X^{\left(\mathrm{tr}\right)},\mathcal{N}\right)=\underset{F,X,\mathcal{N}(·)}{min}\sum _{(i,t)\in \Omega }{\left(Y_{it}-{\mathit{f}}_i^{\top }{\mathit{x}}_t\right)}^2+{\lambda }_{T}R\left(X^{(tr)}\mid \mathcal{N}(·)\right)$$
  The training of low-rank factors $F,X^{(tr)}$ alongside the temporal network $\mathcal{N}(·)$ is executed in an alternating way to minimize the loss function. This is accomplished via stochastic gradient descent in two main alternating phases:

  (a)

  With the $\mathcal{N}(·)$ fixed, the function ${\mathcal{L}}_G\left(F,X^{\left(\mathrm{tr}\right)},\mathcal{N}\right)$ is minimized in relation to the factors $F,X^{\left(\mathrm{tr}\right)}$:

  (b)

  Given $F,X^{\left(\mathrm{tr}\right)}$, train the network $\mathcal{N}(·)$.

  Step 3: Forecast the global prediction

  (a)

  Utilize the historical data-points from the fundamental time-series $X_{T-s+1:T}$, where $s=(k-1)2^l$ and l represents the number of layers, to predict the subsequent value ${\widehat{\mathit{x}}}_{T+1}$, using $\mathcal{N}\left(X_{T-s+1:T}\right)$.

  (b)

  Extend this by appending the next-step forecast ${\widehat{\mathit{x}}}_{T+1}$ to the prior sequence, creating ${\tilde{X}}_{T-s+2:T+1}$. This process is iteratively conducted h times, with each iteration feeding the extended series back into the network to project one additional future step. We generate h successive forecasts, yielding the predicted time-series ${\widehat{X}}^{\left(te\right)}$, where $h=12$.

  (c)

  The final global predictions are then given by $Y^{\left(g\right)}=F{\widehat{X}}^{\left(te\right)}$.

• Fitting the hybrid model: This model merges the global framework with local features, utilizing the training series $Y\left(t\right)$ and covariates $C\in {\mathbb{R}}^{n\times b\times (T+h)}$. Here, $b=5$, representing two temporal covariates (year, month) and climatic indices SOI, DMI, and SAM from April 2000 to March 2022.

  (a)

  The hybrid model integrates the initial input $Y_{T-s+1:T}$, global model output ${\tilde{\mathit{y}}}_{T-s+1:T}^{\left(g\right)}$, and the b-dimensional covariates ${\mathit{r}}_{T-s+1:T}$ for next-step prediction ${\widehat{y}}_{T+1}$:

  $$T_Y\left({\mathit{y}}_{T-s+1:T},{\tilde{\mathit{y}}}_{T-s+1:T}^{\left(g\right)},{\mathit{r}}_{T-s+1:T}\mid {\Theta }_Y\right)$$

  Thus, we can obtain the predicted time-series ${\widehat{Y}}^{\left(te\right)}$.

  (b)

  Finally, we use an exponential transformation on log-precipitation forecasts to predict precipitation.

• The model, tested on the integrated Australian monthly precipitation data (April 2021–March 2022).

  (a)

  Continent-wide precipitation prediction from April 2021 to March 2022.

  (b)

  We then refine our analysis with the EDQ method, which employs the following formula to generate the p level EDQ series $\left\{q_t^{\left(p\right)}\right\}$ as detailed in [21]:

  $$\begin{array}{cc}& \left\{q_t^{\left(p\right)}\right\}:=\underset{\left\{z_1,\cdots ,z_T\right\}\in \left\{\mathit{Z}\left(t\right)\right\}}{argmin}\left[\sum _{t=1}^T\left(p\sum _{x_{it}\ge z_t}\left|x_{it}-z_t\right|+(1-p)\sum _{x_{it}\le z_t}\left|x_{it}-z_t\right|\right)\right]\hfill \end{array}$$

  Here, $\left\{q_t^{\left(p\right)}\right\}$ must be one of the series in $\left\{Z\right(t),t=1,\cdots ,252\}$. Through this method, we can identify extreme rainfall areas (for $p>0.95$) and moderate to low rainfall areas (at p around $0.001$ and $0.5$).

5. Results and Discussion

In this study, we present a specific spatiotemporal method to understand Australia’s precipitation patterns, leveraging diverse data sources and advanced analytical techniques. The process includes data fusion, dimensionality reduction, empirical dynamic quantile ranking, and predictive analysis.

The dual-layer architecture of the spatiotemporal TCN-MF model integrates both global-level and local-level spatiotemporal patterns. We incorporate climatic drivers such as SOI, DMI, and SAM alongside seasonal covariates to refine the model’s predictions. For both the local and hybrid models, the configuration of channels is [32, 32, 32, 1], with a kernel size of 7. Furthermore, the rank of the global model is specified as 64.

The predictive prowess of our developed system is illuminated in Figure 3. From this, we calculate the mean absolute error (MAE) and the root mean square error (RMSE) to be 18.664 and 29.542, respectively. The bottom panel of this figure showcases forecasts of monthly precipitation for all 1391 locations from April 2021 to March 2022, which show a close correspondence with the observed precipitation data depicted in the upper panel.

To further assess the model’s performance across different rainfall scenarios, we apply the empirical dynamic quantiles method. We strategically select eight locations across a series of quantiles: $\{0.001,0.503,0.953,0.965,0.971,0.983,0.991,0.996\}$. This selection method is informed by the need to examine the model’s performance in varied rainfall intensities and geographic distributions, ensuring a comprehensive analysis.

Extreme Rainfall Areas: Locations within the top 5% quantiles are emphasized, particularly those experiencing severe flooding. Notably, we include Lismore (quantile 0.965) and Darwin (quantile 0.996) as specific case studies. Lismore is selected due to the major floods recorded in early 2021 in Eastern Australia, while Darwin is chosen for its status as one of the cities with the highest cumulative rainfall, offering a valuable perspective on the model’s ability to predict in regions of maximum rainfall.

Moderate and Low Rainfall Areas: We also analyze locations around the median and the lowest quantiles to validate the model’s performance under average and minimal rainfall conditions.

The line graphs in Figure 4 demonstrate the model’s robustness in predicting rainfall across extreme, moderate, and low rainfall regions. Further, the use of a seasonal VAR model for comparison underscores the reliability of our predictions in a range of climatic conditions. The spatiotemporal TCN-MF model, through its extensive analysis and focused case studies, significantly contributes to understanding precipitation patterns in Australia, offering valuable insights for climate adaptability and forecasting.

While the spatiotemporal TCN-MF model demonstrates good performance in predicting rainfall across Australia, there remains room for further development. A key area for future exploration involves delving deeper into the dynamics driving precipitation patterns, with the aim of enhancing our understanding of rainfall variability and intensity. Future work will focus on developing an expanded model based on the current framework, incorporating a more sophisticated analysis of rainfall dynamics.

6. Conclusions

Our study presents a comprehensive approach to continent-wide precipitation forecasting in Australia. The spatiotemporal TCN-MF model, incorporating a comprehensive dataset from gauge measurements and satellite estimations, accurately predicted monthly rainfall across 1391 diverse locations. Incorporating climatic drivers and seasonal covariates alongside this dataset, the model’s effectiveness was pronounced in regions experiencing extreme rainfall as evidenced by our focused analysis in the top 5% quantile locations like Lismore and Darwin. These areas, important for understanding and managing extreme rainfall events, were accurately captured by the model, demonstrating its potential as a vital tool in climate adaptability and forecasting. Future developments in this model will focus on enhancing its dynamical prediction capabilities and interpretability, further contributing to our understanding of complex weather patterns in Australia.