PT-ESM: A Parameter-Testing and Integration Framework for Earth System Models Oriented towards High-Performance Computing

Abstract

High-performance computing (HPC) plays a crucial role in scientific computing, and the efficient utilization of HPC to accomplish computational tasks remains a focal point of research. This study addresses the issue of parameter tuning for Earth system models by proposing a comprehensive solution based on the concept of scientific workflows. This solution encompasses detailed methods from sensitivity analysis to parameter tuning and incorporates various approaches to enhance result accuracy. We validated the reliability of our methods using five cases in the Single Column Atmosphere Model (SCAM). Specifically, we investigated the influence of fluctuations of 11 typical parameters on 10 output variables. The experimental results show that the magnitude of the impact on the results varies significantly when different parameters are perturbed. These findings will help researchers develop more reasonable parameterization schemes for different regions and seasons.

1. Introduction

High-performance computing (HPC) plays an important role in many fields, including climate simulation [1], molecular dynamics [2,3], earthquake simulation [4], and quantum computing [5]. Naturally, it has extensive and profound applications in the field of Earth sciences, providing scientists with powerful computational capabilities to simulate, analyze, and predict the complex behavior of the Earth system. Climate modeling is one of the important applications of HPC in the field of Earth sciences. By simulating the interactions of complex systems such as the atmosphere, oceans, and land, scientists can predict future climate change trends. HPC can handle large-scale climate datasets, simulate complex physical and chemical processes, and provide high-resolution climate change predictions. This is of great significance for formulating strategies to address climate change, assessing its potential impacts, and predicting the occurrence of natural disasters [6]. In the field of geological research, HPC is widely used for simulating earthquakes, plate movements, magma flows, and more. With HPC, scientists can model complex geological processes such as crustal movement and rock deformation, leading to a deeper understanding of Earth’s structure and evolution. Additionally, HPC can be employed for data processing in oil and gas exploration, helping scientists more accurately determine the location and scale of oil and gas reservoirs [7]. Ocean science research involves extensive data processing and simulation work. HPC offers powerful computational capabilities to support complex studies, including ocean circulation, wave dynamics, and marine ecosystem simulations. By leveraging HPC, scientists can gain deeper insights into the dynamic changes occurring in the ocean, providing critical scientific evidence for marine resource development and ocean environmental protection [8].

HPC also plays a crucial role in the field of environmental science. By simulating the dispersion processes of pollutants in the atmosphere, water bodies, and soil, scientists can assess the impact of pollution on the environment and develop effective pollution control measures [9]. Additionally, HPC can be used to simulate and analyze the dynamic changes of ecosystems, providing scientific evidence for ecological protection. In addition, there is another field that cannot be overlooked. Geographic Information Systems (GIS) [10] and remote sensing technology [11] are important tools in Earth science research. HPC can efficiently process large-scale geographic information and remote sensing data, providing fast and accurate data analysis and visualization results. This is of great significance for land resource management, urban planning, disaster monitoring, and other areas [12].

Earth system models (ESMs) in high-performance computing (HPC) are used to simulate and understand the complex interactions within the Earth’s climate system, including the atmosphere, oceans, land surface, and ice. These models are essential for predicting future climate scenarios, studying environmental changes, and guiding better policy decisions. However, there is considerable uncertainty in the parameters used in ESMs, which can affect the accuracy of simulations. Parameter tuning is an ongoing challenge that requires extensive validation and calibration against observational data. Although existing research has explored parameter uncertainty in models, there is often a lack of a comprehensive methodology for large-scale parameter experiments, which is insufficient for the computational power of HPC.

Therefore, in this paper, we propose PT-ESM, a high-performance computing (HPC)-oriented framework for parameter testing and integration in Earth system models. This framework effectively harnesses the powerful computational capabilities of HPC to assist in various experiments related to Earth system science. Building on this, we present practical scenarios of parameter testing using the Single Column Atmosphere Model (SCAM) and validate the usability of this framework.

2. Related Work

Scientific workflows have become essential tools in managing complex and large-scale computational tasks in various scientific domains [13]. These workflows automate and streamline the sequence of data processing steps, from data collection and preprocessing to analysis and visualization [14]. They enhance reproducibility, scalability, and efficiency in handling intricate computational tasks, particularly in high-performance computing (HPC) environments. Researchers are developing sophisticated tools and frameworks to automate the orchestration of scientific workflows. Tools like Apache Airflow [15], Pegasus [16], and Kepler [17] enable the automation of data processing pipelines, ensuring that tasks are executed in the correct order and managing dependencies efficiently.

Scientific workflows are extensively used to manage the simulation processes in climate modeling. This includes automating the setup of model runs, data assimilation, and post-processing tasks. The Community Earth System Model (CESM) [18] and the European Earth System Model (EC-Earth) [19] are prominent examples that benefit from workflow management systems like Cylc and Apache Airflow.

Workflows facilitate the integration of various data sources, model initialization, and the execution of ensemble forecasts in weather prediction. The Weather Research and Forecasting (WRF) model, for instance, employs workflows to streamline the end-to-end forecasting process. Ocean circulation and wave models, such as the Regional Ocean Modeling System (ROMS) [20] and the Simulating WAves Nearshore (SWAN) model [21], use workflows to handle data inputs, simulation runs, and output analysis. This ensures efficient processing and accuracy in oceanographic studies.

Workflows support hydrological models like the Soil and Water Assessment Tool (SWAT) [22] and the Hydrological Predictions for the Environment (HYPE) [23] model by automating tasks related to watershed simulations and water resource management. Ecological models that simulate ecosystem dynamics, such as the Community Land Model (CLM) [24] and the Dynamic Global Vegetation Model (DGVM) [25], leverage workflows to manage simulations, data integration, and sensitivity analysis.

Parameter tuning in atmospheric models is a critical area of research aimed at improving the accuracy and reliability of climate and weather predictions [26]. This involves adjusting model parameters to better represent physical processes and align model outputs with observed data. Several approaches and techniques are currently being employed and developed for parameter tuning [27].

For example, sensitivity analysis identifies which parameters have the most significant impact on model outputs. This step is crucial for reducing the dimensionality of the parameter space and focusing tuning efforts on the most influential parameters [28]. Machine learning (ML) approaches, including neural networks and ensemble learning methods [29], are being integrated into parameter tuning processes. These techniques can learn complex relationships between parameters and model outputs, providing new ways to improve model accuracy. Data assimilation techniques, which integrate observational data into models during the simulation process, are increasingly used for parameter estimation and tuning. Methods like the Ensemble Kalman Filter (EnKF) [30] and 4D-Var (Four-dimensional Variational Data Assimilation) [31] help refine model parameters in real time. Multiscale modeling and tuning approaches account for different spatial and temporal scales within atmospheric models. By tuning parameters at various scales, models can better capture local and global phenomena. Large-scale projects like the Coupled Model Intercomparison Project (CMIP) [32] and the North American Regional Climate Change Assessment Program (NARCCAP) [33] provide standardized frameworks for parameter tuning and model evaluation. These projects facilitate the comparison of different models and parameter sets to identify best practices.

3. Methodology

A well-structured workflow ensures that resources are allocated efficiently, tasks are executed in a logical and timely manner, and potential challenges or bottlenecks are identified and addressed promptly. Based on the concept of scientific workflow, we propose a novel solution to enhance the utilization efficiency of computational resources for parameter tuning in Earth system models within HPC environments. The overall framework of this structure is shown in Figure 1.

As illustrated in Figure 1, users can access computing resources via terminals through a VPN provided by the computing center. This VPN is deployed on the public internet, enabling users from around the world to connect. Building on this, we can deploy a computing service platform within the computing center. This platform offers users a user-friendly interface and supports extended, efficient auxiliary applications.

This solution integrates advanced workflow automation, optimization algorithms, and data management techniques to streamline the tuning process, reduce computational overhead, and produce more accurate and reliable model outputs.

In our preliminary research, we found that the results of sensitivity analyses obtained by different methods may be different. This also means that the sensitivity analysis results obtained by a single method may not be accurate [34]. Therefore, we propose a fusion method for screening sensitive parameters, which can take into account multiple sensitivity analysis methods, so that the most sensitive parameters can be screened out as much as possible.

Suppose there are a total of n parameters, when applying method m, the sensitivity of parameter i is $S_{mi}$, then the normalized sensitivity of i under this method is

$$S_{mi}^{\prime }={\displaystyle \frac{S_{mi}}{{\sum }_{j=1}^n{S}_{mj}}}$$

(1)

In the case of fusion and considering M methods, the total sensitivity of parameter i is

$$S_i^{\prime }=\sum _{m=1}^M{S}_{mi}^{\prime }$$

(2)

The fusion sensitivity is sorted to compare the degree of influence each parameter in the model has on a particular output.

In our sensitivity analysis of SCAM, the main computational overhead arises from running the sampling phase model. Since there is no dependency between the sample points during sampling, these tasks can be executed simultaneously.

The number of SCAM instances to be submitted in this process is shown in Equation (3). N represents the number of simulations at this stage.

$$N_{SPP}=C\times M\times L$$

(3)

In Equation (3), C represents the number of cases to be tuned by the user, M represents the total number of parameters used for the study, and L represents the number of values of each parameter within the value range.

In this part of the diagnostic analysis, Model Absolute Baseline Bias ($MABB$) and Model Relative Baseline Bias ($MRBB$) were proposed as metrics for the output results. The former metric describes the difference between the maximum and minimum values of the model output for a set of trials. The latter metric describes the ratio of this difference to the output of a baseline trial (i.e., a control trial with default parameters). Combining these metrics allows for the comparison of the magnitude of the effect on the model output caused by parameter adjustments, both in absolute and relative terms. These metrics are described by Equations (4)–(7).

$$MAB{B}_Y=Y_{max}-Y_{min}$$

(4)

$$MAB{B}_Y^{+}=Y_{max}-Y_{base}$$

(5)

$$MAB{B}_Y^{-}=Y_{min}-Y_{base}$$

(6)

$$MRB{B}_Y={\displaystyle \frac{Y_{max}-Y_{min}}{Y_{base}}}\times 100\%$$

(7)

In the above equation, $Y_{max}$ represents the maximum value of the model simulation output, $Y_{min}$ represents the minimum value of the model simulation output, and $Y_{base}$ represents the value of the variable Y in the baseline trial.

To validate the effectiveness of the methods proposed in this paper, we conducted exemplary application validation experiments. The Single Column Atmosphere Model (SCAM) is a simplified atmospheric model that focuses on a single vertical column of the atmosphere, isolating it from horizontal interactions [35]. This model is used to test and develop parameterizations of physical processes such as convection, radiation, and cloud formation. SCAM allows researchers to study the detailed behavior of these processes in a controlled environment, providing valuable insights and validation for more complex three-dimensional climate models. In the five typical cases mentioned in

Table 1. The five typical SCAM cases studied in this paper.

Case	Full Name	Lat	Lon	Date	Type
ARM95	ARM Southern Great Plains	36	−97	July 1995	Land convection
ARM97	ARM Southern Great Plains	36	−97	June 1997	Land convection
GATEIII	GATE Phase III	9	−24	August 1974	Tropical convection
TOGAII	Tropical W. Pacific Convection	−12	131	December 1992	Tropical convection
TWP06	Tropical Ocean Global Atmosphere	−2	154	January 2006	Tropical convection

, the six sensitivity analysis methods in

Table 2. Sensitivity analysis method integrated in the proposed framework.

Name of Method	Abbr.	Reference
Morris sensitivity analysis	Morris	Morris [38]
Delta moment-independent measure	Delta	Plischke et al. [39]
Sobol’ sensitivity analysis	Sobol	Saltelli et al. [36]
PAWN sensitivity analysis	PAWN	Pianosi and Wagener [40]
High-dimensional model representation	HDMR	Li et al. [41]
Random balance designs Fourier amplitude sensitivity test	RBD-FAST	Goffart et al. [37]

are simultaneously used to analyze the sensitivity of the 11 parameters in

Table 3. The 11 parameters involved in the exemplary study.

Abbr.	Name	Description	Low Range	Default	High Range	Category
pz1	c0_lnd	Deep convection precipitation efficiency over land	0.00295	0.0059	0.00885	zmconv_
pz2	c0_ocn	Deep convection precipitation efficiency over ocean	0.0225	0.045	0.0675	zmconv_
pz3	dmpdz	Parcel fractional mass entrainment rate	−0.002	−0.001	−0.0002	zmconv_
pz4	tau	Time scale for consumption rate deep CAPE	1800	3600	5400	zmconv_
pz5	capelmt	Threshold value for CAPE	35	70	105	zmconv_
pz6	alfa	Maximum cloud downdraft mass flux fraction	0.05	0.1	0.15	zmconv_
pu1	rpen	Penetrative updraft entrainment efficiency	2.5	5.0	7.5	uwschu_
pu2	kevp	Evaporative efficiency	1 × 10${\hspace{0.17em}}^{-6}$	2 × 10${\hspace{0.17em}}^{-6}$	3 × 10${\hspace{0.17em}}^{-6}$	uwschu_
pu3	rkm	Updraft lateral mixing efficiency	7	14	21	uwschu_
pc1	rhminh	Threshold relative humidity for stratiform high clouds	0.7	0.8	0.9	cldfrc_
pc2	rhminl	Threshold relative humidity for stratiform low clouds	0.7975	0.8975	0.9975	cldfrc_

to the model simulation output. Sobol [36] and RBD-FAST [37] are more suited for detailed variance-based sensitivity analysis and can handle interactions between parameters. Morris [38] is ideal for preliminary screening when dealing with many parameters due to its lower computational cost. Delta [39] and PAWN [40] provide more comprehensive measures of sensitivity by focusing on output distributions, making them suitable for models with non-normal outputs. RBD-FAST [37] combines the efficiency of random designs with the analytical power of Fourier transforms, suitable for complex and large models. HDMR [41] is particularly useful when dealing with models that have a large number of input parameters. These parameters are highly typical and representative, originating from three different parameterization schemes. They were refined based on previous work to ensure the selection of these parameters.

Subsequently, Equations (1) and (2) were used to calculate the sensitivity of each parameter to the fusion of each output variable in

Table 4. List of output variables used for sensitivity analysis.

Name	Description	Unit
PRECT	Total precipitation rate	m/s
PRECZ	Deep convective precipitation rate	m/s
PRECSH	Shallow convective precipitation rate	m/s
PRECL	Large-scale precipitation rate	m/s
CLDTOT	Vertically-integrated total cloud	Percentage
CLDLOW	Vertically-integrated low cloud	Percentage
CLDMED	Vertically-integrated mid-level cloud	Percentage
CLDHGH	Vertically-integrated high cloud	Percentage
SHFLX	Surface sensible heat flux	W/m2
LHFLX	Surface latent heat flux	W/m2

. The single parameter perturbation test was performed on 11 parameters, each time a parameter was disturbed within 10% of the value range, and the remaining parameters kept the default values. After running SCAM5, the $MRBB$ was used to represent its sensitivity. Each output variable in Table 4 was calculated separately for the magnitude of its change. We will present and analyze the experimental results in the next section.

4. Experiments and Results

The hardware platform for our research is the Sunway TaihuLight supercomputer [42], hosted at the National Supercomputing Center in Wuxi (NSCCWuxi), which is the world’s first supercomputer with a peak performance of more than 100Pflops [43]. The GCC version 5.3.0 compiler adapted for the SW26010 CPU [44] was used to compile our expanded SCAM5. This model comes from the CAM5 component in the 1.2.2 version of CESM.

The sensitivity analysis described in Table 2 was implemented with the assistance of the SALib [45], a Python library. The heat map was used to characterize the fusion sensitivity of each parameter. Seaborn is a Python data visualization library based on Matplotlib, specifically designed for creating statistical graphics. It is a powerful and user-friendly tool for various data analysis and presentation tasks. Therefore, we used the Seaborn [46] library to generate heat maps that illustrate the experimental results.

Heat maps allow the representation of 3D information in a 2D plane, increasing the readability of the picture. In these sets of plots, color is used to indicate the magnitude of the effect caused by the parameter change on the results. Among the five variables related to precipitation, their sensitivities represented by fusion are presented in Figure 2. Furthermore, in order to compare the differences between the relative and absolute biases of the different output variables, the magnitude of the change in the absolute value of the MABB representation is shown in Figure 3. Considering the non-linearity of the model and the variability of the cases, the positive and negative MABB values are also shown in Figure 4 and Figure 5, respectively. To facilitate reader comprehension, positive deviations and the absolute value of total deviations are represented in green, while negative deviations are shown in blue. The deeper the color, the greater the deviation. Positive and negative deviations are based on the default output, aiming to verify whether the magnitude of the impact on the results is the same when the parameter adjustment ranges are identical. In other words, this is to determine whether the impact is linear.

For clouds as well as heat fluxes, the results of the sensitivity analysis of their relevant output variables are shown in Figure 6. The meaning of these is the same as in Figure 2. Similarly, Figure 6 shows their MRBB, Figure 7 their MABB, while Figure 8 and Figure 9 show the MABB for the case of considering positive and negative variations, respectively.

With regard to an SCAM case, the impact of different parameters on the output variable is inconsistent and can vary significantly (by more than 10 times). Even for the same parameter, its effects in different cases are not consistent. For example, pz1 (c0_lnd) has very little effect when the observation point is located in the tropical ocean, while pz2 affects land. Parameter pu3 has a significant impact on shallow convective precipitation (PRECSH), with an influence hundreds of times greater than other parameters.

In the same parameter space, the range of changes in different output variables varies. For example, the range of change for PRECSH is notably large, likely due to its typically small values (usually only a few millimeters per day).

For CLDTOT and CLDHGH, the two land convection cases exhibit a greater range of change, particularly evident in c0_lnd, tau, and alfa. For CLDLOW, rhminl has a more significant impact on GATEIII and TOGAII. Additionally, rkm greatly influences the sensible heat flux of TWP06.

The results of the aforementioned experiments fully demonstrate the significant practical value of the proposed solution in this paper. These findings indicate that with the powerful computational capabilities of HPC, it is possible to conduct large-scale parameter testing in a relatively short period of time.

5. Discussion

Previous studies have often focused on global models (such as CAM) [47] or on a limited number of specific cases [48], typically examining the models themselves. In contrast, this paper emphasizes the overall methodology for parameter testing and aims to develop an integrated approach for large-scale parameter testing suitable for HPC environments. To verify the efficiency and effectiveness of this approach, we analyzed the different impacts of 11 parameters on 10 output variables using SCAM as an example in the previous section’s experiments. From the results, we can observe several interesting conclusions.

First, regarding precipitation output, although PRECT is the sum of various precipitation components, it is clear that the sensitivity of PRECT to different parameters is not a simple sum of the sensitivities of its individual components. There are both commonalities and individual characteristics across different cases. For example, the output of PRECSH shows high consistency across different cases, indicating two points. On the one hand, PRECSH is less affected by the specific characteristics of the cases, such as location and season. On the other hand, its absolute value is relatively small, thus contributing less to the total precipitation. The parameter pu3, or rkm, has a significant impact on the results, confirming the role of the UW scheme in shallow convective precipitation and the importance of rkm relative to other parameters. The heatmaps of PRECT and PRECZ are more similar, reflecting that deep convective precipitation remains the main component of total precipitation in the five cases studied. Additionally, for total precipitation and deep convective precipitation, the impact of pz4, or tau, is significant and should be a focus for future research. Conversely, pc2, or rhminl, has a major impact on large-scale precipitation, especially in GATEIII and TOGAII, both of which are fully oceanic cases.

For cloud-related variables, similar conclusions can be drawn. For all five cases studied, pc2, or rhminl, has a greater impact on CLDLOW, but not on the other three variables. Overall, ARM95/97 is more influenced by the ZM deep convection scheme, making these two cases exhibit different parameter sensitivities compared to the other three tropical convective cases. This phenomenon is also evident in the two heat flux variables, where pz1, or c0_lnd, overwhelmingly impacts the output, which is closely related to the fact that these two cases are located in land regions. Conversely, pz4 and pz6 have a greater impact on tropical ocean cases. Interestingly, TWP06 shows different sensitivities in SHFLX, which warrants further investigation.

When we separate the positive and negative impacts of MABB, we also find some interesting conclusions. For example, although we adjust each parameter in a symmetric space relative to the default value, their effects on precipitation are not uniform. This is particularly evident in PRECZ and PRECL. In ARM95, PRECZ is more likely to be positively influenced, while PRECL is more likely to be negatively influenced. However, for TWP06, the opposite is true. For cloud and heat flux variables, the differences between positive and negative impacts are not very significant. Although there are some quantitative differences, they do not affect the qualitative conclusions.

In summary, the method and framework proposed in this paper provide the possibility for large-scale parameter testing based on HPC, helping researchers quickly identify the most impactful parameters. This allows for more targeted and detailed studies of different parameters in various application scenarios. Of course, the method proposed in this paper is not perfect. The study has not yet delved into further optimization of the model itself, indicating room for future improvement.

6. Conclusions

High-performance computing (HPC) is critically important in scientific computing, providing the computational power required to tackle complex problems and perform large-scale simulations. In Earth numerical modeling, HPC enables detailed simulations of atmospheric, oceanic, and land processes, offering valuable insights into Earth system dynamics. The method proposed in this paper further extends the scope of HPC applications by optimizing parameter analysis and tuning processes. Experimental results show that leveraging HPC’s computational capabilities makes large-scale parameter analysis experiments feasible, thereby enhancing the performance of corresponding models in simulations.

From the experimental results of the test cases, it can be seen that the 11 parameters have varying degrees of impact on the 10 output variables in the five typical SCAM cases. Quantitatively, the difference in their impact can be more than tenfold, and even several dozen times in some instances. This clearly demonstrates that different parameters from various parameterization schemes play different roles depending on the location and time of the outputs. This insight will inspire further research into parameterization schemes, emphasizing the need to set different parameter values for different application scenarios to improve model performance. It also shows that single-column models are indeed powerful tools for studying parameterization schemes.

Climate change concerns the future of human development, and improvements in methodology are significant from scientific research to policy-making. This study shows that it is necessary to optimize different parameters for different outputs in various locations. Extending this idea, it becomes clear that policymakers in different countries and regions also need to adopt appropriate strategies tailored to their specific conditions. This collective effort will help maintain a healthy climate and ecological environment, achieving sustainable development of the natural environment for the benefit of all humanity.