A Regional Industrial Economic Forecasting Model Based on a Deep Convolutional Neural Network and Big Data

Abstract

To accurately predict the economic development of each industry in different types of regions, a deep convolutional neural network model was designed for predicting the annual GDP; GDP growth index; and primary, secondary and tertiary industry growth values of each. In the model, raw industrial data are preprocessed by a normalization operation and subsequently transformed by the BoxCox method to approach the normal distribution. Panel data of consecutive years are constructed and used as input to the deep convolutional neural network, and industrial data of year t + 1 are used as the output of the network. Simulation experiments were conducted to analyze 23 years of industrial economic data from 31 provinces, municipalities, and autonomous regions in China. The experimental results show that R-squared value is larger than 0.91 for all 31 provinces and root mean squared log errors (RMSLE) of all regions are less than 0.1, which demonstrate that the proposed method achieves high prediction accuracy with generalization capability and can accurately predict the economic growth trends of different types of regions.

1. Introduction

In recent years, political issues, such as trade protectionism, racism, and populism, have brought great uncertainty to economic and social development worldwide. In particular, the trade dispute between China and the United States has produced major challenges to world economic development. In the current complex international environment, China, as the world’s largest developing country, must urgently revise its thinking on economic development by expanding domestic demand, adjusting its industrial structure, and establishing a new strategy for regional economic development with adaptive capabilities. In the 19th Party Congress report, Xi Jinping clearly pointed out that China has entered a new era. He also wants to build China into a harmonious and beautiful modern socialist country, and to achieve sustainable, coordinated development of China’s domestic economy by the middle of this century [1]. In 2020, China proposed the “Double Cycle: A New Pattern of National Economic Development”, which featured a major domestic economic cycle and dual domestic and international economic cycle promoting each other. The proposal focuses on strengthening the coordinated development of the domestic economy and enhancing the sustainable development of China’s economic grand cycle by enhancing the sustainable development of China’s regional industrial economies [2]. One of the main reasons for the establishment of the European Union (EU) is to strengthen economic relations between European countries, aiming at enhancing the external economic competitiveness of the EU and the influence of the EU countries in the world through the cooperation of the EU economies. In 2021, President Biden signed a trillion-dollar stimulus package to strengthen U.S. infrastructure and manufacturing, designed to promote circular and rapid growth in the U.S. domestic economy.

However, due to current complex international political relations and dynamic macroeconomic development, methods to accurately predict the future development of regional industrial economies based on current economic status and understand the interactions between different industrial systems and their impact on regional economic development have all become important issues in selecting regional pillar industries, enhancing the competitiveness of cities and the synergistic development of regional economies. By mining historical data on the economic development of each region in the country, this study investigates the key economic industries developed in each region and analyzes and predicts the growth potential of regional industrial economic development to establish a theoretical basis and value reference for the selection of advantageous industries in the region and the coordinated development of each region.

At present, regional and industrial economy research is rich and wide-ranging and involves relatively mature research methods. Scholars carrying out multilevel and multi-perspective studies on different research topics using various research methods have produced a steady stream of research results. In terms of research content, most of the literature focuses on specific industries, such as agriculture [3], manufacturing [4,5], real estate [6], logistics [7], finance [8], productive services [9], cultural industries [10], information industries [11], and other economic industries. The association of regional industries and their ripple effects [12] have also become the focus of research in recent years, and several rationalized policy recommendations have been proposed to promote industrial structure upgrading [13]. We introduced a multiregional economic correlation analysis method based on big data [14]. These recommendations are powerful tools for judging the economic contribution of industries and selecting leading regional industries. Tianren Yang et al. investigated the residential mobility patterns of households living in low-income neighborhoods and studied the neighborhood characteristics that influence their mobility [15]. Lingqiang Kong et al. summarized the big data-based urban environment, society, and sustainability (UESS) research using a systematic review approach in combination with bibliometric and thematic analyses [16]. Zaheer Allam and Zaynah A. Dhunny proposed a new framework binding AI technology and cities while ensuring the integration of key dimensions of culture, metabolism, and governance [17].

In terms of research methods, various economic forecasting models have been proposed to predict the future development of relevant industries based on the current state of the industrial economy. Tingting Zhang et al. proposed an FWA-SVR forecasting model to forecast the tourism economy [18]. Cheng Mao-Lin et al. used the gray model GM (1,1) to predict the growth trends of China’s real estate economy [19]. In another study [20], China’s macroeconomic system was analyzed by constructing a BMF-VAR model to forecast China’s macroeconomy. In [21], a deep-learning LSTM model used to forecast and analyze the inflation rate in China achieved good performance. Ji Yao used commodity price big data to construct a MIDAS model for analyzing and forecasting CPI and PPI [19,22]. Xia Maosen used a deep neural network CNN-LSTM model to analyze and forecast the Chinese consumer confidence index [23]. Pichayakone Rakpho et al. proposed a Bayesian vector autoregressive (BVAR) model to forecast energy demand and supply more accurately [24]. Yifei Lyu et al. constructed a dataset of 77 countries representing over 90 percent of global GDP and forecasted US economic growth in downturns using cross-country data [25]. Oscar Claveria et al. presented a machine-learning method for sentiment indicators construction and economic forecasting with evolved confidence indicators [26]. To enhance the prediction accuracy of COVID-19 and strengthen the economic management and control, Xuan Tang et al. introduced a self-correcting intelligent pandemic prediction model [27]. To accurately forecast business failure prediction, Soo Young Kim et al. developed three prediction models (entire period model, economic downturn model, and economic expansion model) using WEKA3.9 [28].

The above forecasting models have achieved sound forecasting results in related economic fields, but the current research remains focused on macroeconomic forecasting and analysis of a single economic industry. Since regional economic development is influenced by multiple factors, multiregional, multifactor, and related industry correlation analysis forecasting is obviously insufficient. To consider the correlation of multiple industries and factors simultaneously, we construct a multilayer convolutional neural network (CNN) to build panel economic data with temporal and continuous effects based on historical data from multiple economic indicators in each region. We use the convolutional kernel of this neural network to mine correlations in the panel data to accurately predict the future economic development trends of the regions. The contributions of our work are summarized as follows:

(1)

A novel multilayer CNN is constructed for predicting the economic development of primary, secondary and tertiary industries of multi-regions. In the CNN, the GDP of a region is separated from the other indicators of the region before full connection layers, which aims to enhance the predict accuracy according to the GDP value.

(2)

To reduce the data heterogeneity among the economic indicators of each regions and the differences in the economic development levels of each region, the data are first normalized to balance the deviations among the indicators, and then the normalized data were transformed exponentially to balance the deviations among the regions.

2. Prediction Models Based on Deep Convolutional Neural Networks

2.1. Model Architecture

As shown in Figure 1, our deep convolutional neural network [29] inputs the gross regional product (GDP), GDP growth index, primary industry growth index, secondary industry growth value, and tertiary industry growth value for n consecutive years in a region as panel data into the network to predict the output primary, secondary, and tertiary industry growth values for the region in year n + 1.

The model divides the input data into two parts.

(1) Regional GDP network: the regional GDP data are independent of other indicator data before the fully connected layer of the neural network, and feature information is extracted using convolutional neural networks (CNNs), batch normalization, and the ReLU activation function. This design reflects that the regional GDP can mirror the overall economic level of the region, which is equivalent to gauging the regional economic development level by regional GDP to divide the regions (similar to the role of the panel threshold model) and using different forecasting strategies for regions with different development levels in the fully connected layer.

(2) Regional panel data network: The regional GDP growth index and the primary, secondary, and tertiary industry growth values are combined to form panel data to identify correlations between industries and the influence of industry data on later growth values within n consecutive years using neural network convolution kernels. First, three CNN layers (including convolution, batch normalization, ReLU activation function, and pooling operations) are used to extract time series data features and potential industry connections.

The inputs of the fully connected layers are the output values of (1) and (2), and the two fully connected layers integrate the outputs of the GDP and panel data networks to further optimize the learning of the multichannel outputs of the two networks.

2.2. Model Analysis

(1) Convolution: The main purpose of the convolution layer is to extract features from the input panel and sequence data using the following expression.

$$z^{l+1}\left(i,j\right)={\displaystyle \sum _{c=1}^C{\displaystyle \sum _{x=1}^{k_x}{\displaystyle \sum _{y=1}^{k_y}\left(Z^l\left(s_{x}i+x,s_{y}j+y\right)w_c^{l+1}\left(x+y\right)\right)+b}}}$$

(1)

where $z^l\left(i,j\right)$ denotes the value of the i-th row and j-th column data of the feature image (panel or sequence) in the l-th layer, C represents the number of channels, ($k_x$, $k_y$) indicates the size of the convolutional kernel, ($s_x$, $s_y$) is the convolution step size, $w_c^{l+1}$ represents the weight matrix of channel c of l + 1, and b is a bias value.

(2) Batch Normalization (BA): BA can stabilize the training process of the network, improve training speed, and minimize problems (such as gradient disappearance and overfitting [30] using the following expression).

$$y_{(b)}^l\left(i,j\right)=BN\left(z_{(b)}^l(i,j)\right)=\gamma \cdot \frac{z_{(b)}^l(i,j)-u(i,j)}{\sigma (i,j)}+\beta$$

(2)

where $z_{(b)}^l(i,j)$ denotes the input value of the b-th sample (Row i, Column j) in the current batch. Here, $z_{(b)}^l(i,j)$ denotes the output value of the convolution operation; $u(i,j)$ and $\sigma (i,j)$ denote the mean and standard deviation of the batch sample at that position (Row i, Column j), respectively; and parameters $\gamma$ and $\beta$ are used to control the mean and variance of $y_{(b)}^l$, respectively.

(3) Relu: Relu is an activation function used to recalculate and activate the data after normalization as follows.

$$y^l=\{\begin{array}{c}x\\ \alpha \left(e^x-1\right)\end{array}\begin{array}{c},\\ ,\end{array}\begin{array}{c}x>0\\ x\le 0\end{array}$$

(3)

where $y^l$ is the output value of the l-th layer after activation by the Relu function.

(4) Pooling: Down sampling, dimensionality reduction, and feature compression are conducted on the output data of the convolutional operation. Maximum pooling is used in the first and second convolutional layers, and the mean pooling operation is used in the third convolutional layer.

(5) Loss function: The model uses the Huber loss function (as shown in Equation (4)) to optimize the prediction results as follows.

$$\begin{array}{l}loss(y,y^{\prime })=\frac{1}{n}{\displaystyle \sum _{i=1}^n{z}_i}\\ z_i=\{\begin{array}{c}0.5{\left(y_i-{\stackrel{⌢}{y}}_i\right)}^2,if\left|y_i-{\stackrel{⌢}{y}}_i\right|<1\\ \left|y_i-{\stackrel{⌢}{y}}_i\right|-0.5, \mathrm{otherwise} \end{array}\end{array}$$

(4)

where $y$ and $\stackrel{⌢}{y}$ denote the real and predicted values of the network, respectively.

(6) Optimization algorithm: For the loss function, the distance between the predicted network output and true values is minimized by optimizing the network parameters. The Adam optimization approach, which is solved iteratively by performing the gradient derivative of the loss function [31], is adopted for the optimization of the network parameters. The Adam algorithm iterative formula is written as follows.

$${\theta }_t={\theta }_{t-1}-\alpha \frac{m_t}{\sqrt{v_t+\xi }}$$

(5)

$$\begin{array}{l}{\tilde{m}}_t={\beta }_1\cdot {\tilde{m}}_{t-1}+(1-{\beta }_1)\cdot g_t\\ m_t=\frac{{\tilde{m}}_t}{1-{\beta }_1^{t+1}}\\ {\tilde{v}}_t={\beta }_2\cdot {\tilde{v}}_{t-1}+(1-{\beta }_2)\cdot g_t{}^2\\ v_t=\frac{{\tilde{v}}_t}{1-{\beta }_2^{t+1}}\end{array}$$

where ${\theta }_t$ is the parameter value of the network at the t-th iteration, $m_t$ denotes the first moment estimate, $v_t$ denotes the second moment estimate, and $g_t$ denotes the gradient value at time t. $\alpha$ represents the learning rate during the iteration (0.01 in this paper); ${\beta }_1$ and ${\beta }_2$ both represent the exponential decay rate (${\beta }_1$ is used to control the momentum of the exponential shift and the weight of the gradient at generation t (0.9), respectively; and ${\beta }_2$ is used to control the weight of the squared gradient at generation t − 1 (0.999)). $\xi$ is a very small positive number (1 × 10 − 8) designed to avoid the denominator of Equation (5) being 0.

3. Analysis and Forecast of the Industrial Economy of 31 Provinces and Municipalities in China

3.1. Dataset

In this paper, we use national macro annual data from the CEE statistical database (https://db.cei.cn/, accessed on 20 May 2021), including the annual GDP; GDP growth index; and growth values of primary, secondary, and tertiary industry economies for 31 provinces, municipalities directly under the central government, and autonomous regions of China, from 1997 to 2020.

Data preprocessing: To enable uniform network training, all the data were first normalized as follows.

$${\overline{x}}_i^j=\frac{x_i^j-\min (x_i)}{\max (x_i)-\min (x_i)}$$

(6)

where $x_i$ denotes the data corresponding to the i-th indicator; $x_i^j$ is the data value of the j-th row in the i-th indicator; and $\max (x_i)$ and $\min (x_i)$ denote the maximum and minimum values of the data in the i-th indicator, respectively.

Due to unbalanced economic development in the 31 given provinces in China, the economic levels of the developed eastern regions and the northwestern provinces differ greatly. Figure 2 (statistical histogram and QQ plot) shows the distribution of the data in three industries (after normalization) for 31 provinces, municipalities, and autonomous regions in China over 20 years. As seen in Figure 2, the data distribution of the three industries has a very large deviation, which gravely affects the accuracy of the prediction model. To eliminate the large variability of the data and convert it to an approximate normal distribution, the normalized data are transformed using the following method.

$${\stackrel{⌢}{x}}_i^j=\sqrt[5]{{\overline{x}}_i^j}$$

(7)

The transformed data follow the approximate normal distribution of Figure 3, which basically satisfies the conditions of regression analysis.

In this paper, which is based on regional economic data in China from 1997 to 2020, the regional data of five consecutive years are used as the network input, X, and the data of the sixth year are used as the output, Y. Each region generates 19 sample data (each sample is a matrix panel data with 5 rows and 5 columns), yielding a total of 19 × 31 = 589 sample data. Due to the small number of samples, the network is easily overfitted with respect to prediction learning for the 31 regions. To improve generalization capability and enhance the prediction performance of the network, the data with 589 samples are randomly fine-tuned to generate new sample data, resulting in a total of 2565 sample data points. During training, the samples are randomly shuffled, and the network is trained in batches, with 100 samples randomly selected for each batch.

3.2. Network Parameters

The network parameters are summarized as

Table 1. Deep CNN parameters.

Subnetwork 1 for Panel Data						Subnetwork 2 for GDP Data
Layers	Type	Number of Channels	Kernel Size	Stride Size	Padding Size	Layers	Type	Number of Channels	Kernel Size	Padding Size
Convolution layer 1	Convolution	64	(3,1)	(1,1)	(2,1)	Convolution layer 1	Convolution	16	(3,1)	(1,0)
	Batch normalization	64	/	/	/	/	Batch normalization	16	/	/
	Relu	64	/	/	/	/	Relu	16	/	/
	Pooling	64	(3,1)	/	/	/	Pooling	16	(3,1)	/
Convolution layer 2	Convolution	32	(1,3)	(1,1)	(1,2)	Full connection layer 0_0	Input size: 16; output size: 32
	Batch normalization	32	/	/	/	Full connection layer 0_1	Input size: 32; output size: 16
	Relu	32	/	/	/
	Pooling	32	(1,2)	/	/
Convolution layer 3	Convolution	16	(2,2)	(1,1)	(0,0)
	Batch normalization	16	/	/	/
	Relu	16	/	/	/
	Pooling	16	(2,2)	/	/
Merge the output of subnetwork 1 and subnetwork 2
Full connection layer 1_0			Input size: 16, output size:32
Full connection layer 1_0			Input size: 32, output size:3 The output is the economic growth values of the three industries in the 6th year.

.

We implemented the network model using PyTorch, including data normalization, normalization, network training, prediction, and evaluation. The source code of the algorithm is available at https://github.com/shouhengtuo/Cnn_economy (accessed on 20 May 2021).

3.3. Results and Analysis

For network training, 10-fold cross-validation was used, and the value of the loss function was essentially stable after 300 iterations. The trained network was used to predict the last three years of data for three industries in 31 provinces, municipalities that are directly under the central government, and autonomous regions of China. The following two evaluation indices were used to evaluate the prediction accuracy of the network.

(1)

R-Squared

$$R^2=1-\frac{S{S}_{residual}}{S{S}_{total}}=1-\frac{{\displaystyle \sum _{i=1}^n{\left({\widehat{y}}_i-y_i\right)}^2}}{{\displaystyle \sum _{i=1}^n{\left(\overline{y}-y_i\right)}^2}}$$

(8)

where $y_i$ and ${\widehat{y}}_i$ represent the true output value and network prediction of the ith sample, respectively; and ${\overline{y}}_i$ represents the mean value of the true value in the region.

(2)

Root Mean Squared Log Error RMSLE

$$RMSLE=\sqrt{\frac{1}{n}{\displaystyle \sum _{i=1}^n{\left(\log \left({\widehat{y}}_i+1\right)-\log \left(y_i+1\right)\right)}^{{}_2}}}$$

(9)

where $y_i$ and ${\widehat{y}}_i$ represent the true output value and network prediction of the ith sample, respectively.

The value of R-squared is in the interval [0,1], and the closer its value is to 1, the more accurate the prediction result is. In contrast, the traditional mean squared error (MSE) value is influenced by the range of values of the original data, which is not sizeable enough to determine the degree of accuracy. The value of RMSLE is not greatly influenced by the size of the original data compared to the root mean square error RMSE; thus, it is easier to discern the prediction accuracy with RMSLE.

In Table S1 (see Supplementary), the prediction accuracies of the three industries in 31 provinces are summarized by the deep multilayer convolutional neural network proposed in this paper. Figure 4 shows plots of the predicted and true values for Beijing, Shanghai, Shaanxi, Xinjiang, Heilongjiang, Guizhou, Tibet, Guangdong, Shanxi, and Zhejiang Provinces from 2002 to 2020. From Figure 4 and Table S1, it can be seen that the deep convolutional neural network prediction model proposed in this paper can accurately predict the economic growth values of the three industries in all regions. The R2 values of the network model for the 31 regions are all greater than 0.9. With the exception of R2 = 0.915671 for the secondary industry in the Shaanxi Province, R2 = 0.972776 for the primary industry in Tibet, and R2 = 0.938831 for the primary industry in the Zhejiang Province, the R2 values for the three industries are all greater than 0.98. In addition, only the RMSLE of the secondary industry in the Shanxi Province is 0.114741 (which has a relatively large error); the prediction errors of all the other regions are less than 0.1. This indicates that the method in this paper achieves high prediction accuracy and can be used to forecast the economic growth of regions with large variability.

The above experimental results and analysis indicate that the proposed forecasting model has high-precision for predicting multi-industries and multiple economic indicators of a region. The model can be easily used to the future economic development trends of the regions. However, it has some limitations, as listed below.

(1)

The multilayer CNN requires a large sample size to train the network, but the historical economic data of each region is general not sufficient due to the lack and missing of historical data.

(2)

The CNN has high prediction and recognition ability when it has seen similar sample data; however, its prediction ability is deficient for the data that has never been seen before.

(3)

The predication model proposed in this work only considers the GDP; GDP grow index; and the growth value of primary, secondary, and tertiary industry as inputs, and it can also predict the future growth values of primary, secondary, and tertiary industries in a region. However, regional economic forecasting should consider the development trends of more precise industrial economies (e.g., financial industry, tourism, ecological environment, population structure, etc.).

Future works:

(1)

Construction of a multiregional and multi-industrial structure economic index system. The indicators of regional population structure, ecological environment, tourism, logistics, service industry, agriculture and industry are integrated to build a more scientific and complete multiregional and multi-industry structure indicator system.

(2)

Construction of correlation relationship model for the multiregional and multi-industry structure.

(3)

Prediction model construction for multiregional and multi-industrial structure economic indexes.

4. Conclusions

The accurate prediction of regional industrial economic development trends is crucial for future economic decisions and sustainable coordinated development. To accurately predict and analyze the economic development of three industries in each province, municipalities directly under the central government, and autonomous region of China, this paper proposes a multilayer convolutional neural network prediction model that uses regional historical economic data to train and learn a neural network. First, the historical data of the 31 provinces are integrated, and the historical data of each of the three industries are normalized and preprocessed. Due to large differences in the economic development levels of each region, the data contain obvious bias characteristics that affected the prediction accuracy of the neural network. For this reason, the data are first normalized to balance the deviations among the indicators, and then the normalized data are transformed using Equation (7) to balance the deviations among the regions. Second, by constructing panel data and generating sample data, the sample size is increased to improve the generalization capability of the network. The experimental results show that the proposed multilayer convolutional neural network can accurately predict the economic development of each region. One disadvantage of our method is that we only analyze and predict from the combined data of GDP of primary, secondary, and tertiary industries, which cannot reflect the development trends of each industry. Since the industries in the regions tend to influence each other and have complex correlations with each other, it would be more useful to analyze and forecast from the perspective of industry big data. As future research, we intend to collect the historical data of each industry in the regions and form more complete regional economic big data by analyzing and integrating the data of each industry, i.e., population structure, education, agriculture, industry, service industry, finance, science and technology, ecological environment, traffic conditions, water resource distribution, and energy conditions, and then mine their correlations to construct a deep neural network model. In addition, we will build a deep neural network model to forecast the development of the all industries taken together.