Application of Artificial Neural Networks for Catalysis: A Review

Abstract

Machine learning has proven to be a powerful technique during the past decades. Artificial neural network (ANN), as one of the most popular machine learning algorithms, has been widely applied to various areas. However, their applications for catalysis were not well-studied until recent decades. In this review, we aim to summarize the applications of ANNs for catalysis research reported in the literature. We show how this powerful technique helps people address the highly complicated problems and accelerate the progress of the catalysis community. From the perspectives of both experiment and theory, this review shows how ANNs can be effectively applied for catalysis prediction, the design of new catalysts, and the understanding of catalytic structures.

1. Introduction

Machine learning, a powerful technique of artificial intelligence (AI), has been widely used for numerical prediction [1,2], classification [3,4], and pattern recognition [5,6]. As its name implies, machine learning is able to “learn” the highly complicated relationships between the independent and dependent variables via non-linear “black box” data processing. During the past decades, it has been widely used in many scientific and industrial areas, such as biology [7,8,9], medicine [10,11,12], energy [13,14,15,16,17,18,19], environment [20,21,22], engineering [23,24,25], and information technology (IT) [26,27]. These application studies indicate that machine learning techniques have dramatically boosted the development of many different areas. With the developing concept of big data and data-mining [1], there is an increasing demand for knowledge-based machine learning models in practical applications. Artificial neural network (ANN) [28], as a non-linear fitting algorithm, has become one of the most popular machine learning techniques due to its advantages of easy-training, adaptive structure, and tunable training parameters [13]. With the development of algorithms, there are currently a large number of ANN methods, such as the back-propagation neural network (BPNN) [29], general regression neural network (GRNN) [30], and extreme learning machine (ELM) [31]. More recently, the deep neural network (DNN) has raised broad interests due to its strong learning capacity and the popular concept of deep learning techniques [32,33]. Previous studies have shown that different neural network algorithms have different advantages for practical applications [17,18,34,35].

However, compared to other areas, machine learning has not been well-studied in the field of catalysis during the past decades, due to the following two reasons: (i) Lacking of a sufficiently large database: both experimental and theoretical studies on catalytic reactions require a large amount of time and resources; (ii) Too many input variables: to predict the reaction properties, there are usually a large number of independent variables that correlate (or partially correlate) with the predicted properties (e.g., experimental conditions, reaction coordinates, electronic structures, and orbital properties). These problems significantly hinder the development and application of machine learning for the catalysis community. For the catalysis using nano-size catalysts, things become even more complicated: experimental characterizations of their catalysts are complicated and there are many nano-size effects that are difficult to observe. Theoretically, though there are some state-of-the-art computational techniques (e.g., quantum mechanical calculations [36]) that can precisely calculate the electronic structure and energy properties of a catalytic system (e.g., nanoparticles and nanorods), the high computational cost is still a huge challenge. Fortunately, during the recent decade, with the rapid development of both experimental and computational techniques, scientists have been able to more effectively and precisely acquire a large number of properties with a shorter timescale. Meanwhile, the quick developments of computer and machine learning algorithms also ensure that even with lots of independent variables, some machine learning techniques can still perform fast training with a sufficiently large database.

Nevertheless, to the best of our knowledge, there are very few review studies that summarize the machine learning applications for catalysis research. Therefore, detailed discussions on the potential applications of ANNs for both experimental and theoretical catalysis are also necessary. Here, we aim to summarize the recent progress of ANN applications for catalysis, from the perspectives of both experiments and theories. The content of this review consists of three parts: (i) Algorithmic introduction of an ANN; (ii) Applications of ANNs for experimental catalysis; and (iii) Applications of ANNs for theoretical catalysis.

2. Principle of ANN

2.1. Schematic Structure of an ANN

A complete algorithmic structure of a conventional ANN consists of at least three different layers: the input, hidden, and output layers (Figure 1) [37]. Each layer consists of a certain number of neurons. Each neuron inter-connects with all the neurons in the following layer. Each connection represents a weight that contributes to the fitting. With a proper activation function, a combination of optimized weights can generate the prediction of the dependent variable:

$$NET={{\displaystyle \sum }}_{i,j}^n{w}_{ij}{x}_i+b,$$

(1)

where $w_{ij}$ represents the weight value of a connection, $x_i$ represents an inputted independent variable, and $b$ represents a bias. For the activation function ($f\left(NET\right)$), the sigmoid function is one of the most popular forms that can introduce a smooth non-linear fitting to the training of an ANN (Equation (2)). The training of an ANN is essentially the optimization of each weight contribution based on the data groups in the training set. The most commonly used weight optimization method is the back-propagation algorithm, which iteratively analyzes the errors and optimizes each weight value based on the errors generated by the next layer. As we have mentioned above, there are also some other types of networks like GRNN and ELM. Though there are some differences in the weight training and algorithmic structures, the basic principles, as well as the training and testing processes, are very similar. More details about their principles can be found in References [28,30,38].

$$f\left(NET\right)=\frac{1}{1+e^{-NET}}$$

(2)

2.2. Model Development

The rational development of a knowledge-based ANN model consists of two parts: (i) training and (ii) testing. The training process is the so-called “learning” process from the database, while the testing process is the validation of the trained model using the data groups that are not previously involved in the training process. Detailed discussions are shown in Section 2.2.1 and Section 2.2.2. It should be noted that the training and testing processes are not only suitable for the topic discussed in this review, but are also suitable for ANN model development in many other areas.

2.2.1. Model Training

The training of an ANN consists of database preparation and the selection of variables. The size of the database should be sufficiently large to avoid over-fitting. For each variable (especially the dependent variable), the data range should be wide enough to ensure good training. If the data range is too narrow, the trained model might only have a good prediction capacity in a very local region. For the numerical prediction, the dependent variables are usually the properties that are hard to acquire from regular measurements or calculations. The independent variables, on the other hand, should be easily-measured and have potential relationships with the selected dependent variable. More details about the training criterion can be found in Reference [37].

2.2.2. Model Testing

To validate the trained ANN, a testing process is necessary. The testing of a model should use the data groups that are not used for the training process. With the inputs of the testing set, the outputted data can be compared with the actual data in the testing set, with the root mean square error (RMSE) calculated by Equation (3):

$$RMSE=\sqrt{\frac{{{\displaystyle \sum }}_i^n{\left(P_i-A_i\right)}^2}{n}},$$

(3)

where $P_i$ represents the predicted value outputted by the ANN, $A_i$ is the actual value, and $n$ represents the total number of samples. If the calculated RMSE from the testing set is relatively small, it means that the ANN is well-trained. It should be noted that for the training and testing of an ANN, a cross-validation [39] process should be performed using different components of the training and testing datasets. If the database is relatively large, a sensitivity test can be performed to replace the cross-validation, in order to avoid a high computational cost.

It should be noted that, for a typical ANN algorithm (e.g., BPNN), it is necessary to optimize the overall ANN structure before deciding the final numbers of the hidden layer and hidden nodes. Repeated training and testing should be performed with different ANN structures. On the one hand, if the numbers of hidden layers and/or hidden neurons are too high, there is a risk of over-fitting [40]; on the other hand, if their numbers are too low, this leads to under-fitting. Usually, the best ANN algorithmic configuration can be defined by comparing the average RMSEs from the testing sets during a cross-validation or sensitivity test.

3. Applications of ANN for Catalysis: Experiment

3.1. Prediction of Catalytic Activity

The most straightforward application of ANN is the numerical prediction. One of the earliest works for the catalytic application was done by Kito et al. in 1994 [41]. In this work, they predicted the product distribution of ethylbenzene oxidative hydrogenation, with the product components of styrene, benzaldehyde, benzene + toluene, CO, and CO₂ as the outputs of a network. In terms of the inputs of the ANN, they used nine different independent variables that had potential relationships with the productivity and selectivity of the catalytic reaction, including: unusual valence, surface area of the catalyst, amount of catalyst, typical valence, ionic radius, coordination number, electronegativity, partial charge of oxygen ion, and standard heat of formation of oxides. Their results found that with a good experimental database, a single hidden layer ANN could perform precise predictions for the product selectivity. The schematic structure of the ANN they used for the prediction is reconstructed in Figure 2a. In the work done by Sasaki et al. [42], they first proposed that ANNs could be used for catalytic activity predictions, as well as experimental condition optimizations. Setting the experimental conditions such as compositional quantity, temperature, and pressure, they showed that the yield and byproducts of NO decomposition over the Cu/ZSM-5 zeolite catalyst could be precisely predicted by a well-trained ANN [42]. For other more complicated reactions, such as 1-hexene epoxidation catalyzed by polymer supported Mo(VI) complexes, Mohammeda et al. showed that the ANN had a powerful predictive capacity for forecasting its catalytic activities, in excellent agreement with their experimental conclusions [43]. For photocatalysis, the ANN also shows its strong predictive capacity: Frontistis et al. studied the photocatalytic degradation of 17-ethynylestradiol (EE2) using TiO₂ catalysts with varying concentrations [44]. With the inputs of reaction time, TiO₂ concentration, EE2 initial concentration, matrix dissolved organic carbon (DOC), and matrix conductivity as the inputs, they found that a single hidden layer ANN could perform the minimized average RMSE during the testing processes (Figure 2b) [44]. In terms of the biotechnical catalysis, Rahman et al. also found that using the temperature, reaction time, substrate molar ratio, and enzyme amount as the inputs, an optimized ANN structure could be used for the yield prediction of lipase-catalyzed dioctyl adipate synthesis (Figure 2c) [45]. Recently, with the developing concept of data mining, Günay and Yildirim successfully used 1337 data points from 20 studies of selective CO oxidation over Cu-based catalysts. They concluded that ANN modeling could be used to extract valuable experimental results from previous literature data and provides powerful guidance for future experimental designs [46]. In addition to catalysis, Raccuglia et al. further found that a similar concept could even help assist the materials discovery from failed experimental data [47]. From these typical studies, it can be seen that with different types of reaction systems, catalysis, and datasets, the optimal ANN structures for prediction are significantly different. As can be seen in Figure 2, different reaction types have very different input variables and output indicators. This means that each prediction task should be predicted by a specific model with an optimal weight contribution and network structure.

3.2. Optimization of Catalysis

In addition to the activity prediction, scientists started to think about a more practical question: how can we cost-effectively design novel catalysts using the predictive power of ANN? Now that we know that ANNs can precisely predict the catalytic performances of various catalytic systems, we may want to design and generate new inputs of new expected catalysts, and acquire their predicted performances from a well-trained ANN. A general algorithmic flow chart of catalyst optimization summarized by Maldonado and Rothenberg is reconstructed in Figure 3 [48]. This is actually more challenging than a straightforward prediction. For modeling and optimizing the catalysis, one of the primary works was done by Corma et al., who first applied ANNs for the optimization of potential catalyst compositions for the oxidative dehydrogenation of ethane (ODHE) [49]. The scheme of the optimization strategy in their study is shown in Figure 4, with the inputs of various combinatorial compositions and the output of catalytic results. It should be noted that in this catalysis optimization, a genetic algorithm (GA) was introduced as the promoter of design generation [50]. Omata and Yamada developed an ANN to predict the effective additives to a Ni/active carbon (AC) catalyst for methanol vapor-phase carbonylation [51]. Using a well-trained network, they found that Sn was an effective element that could optimize this catalytic process. Hou et al. first proposed an ANN-based computer-aided framework for catalyst design [52]. They found that such a method could be effective for the design of promising propane ammoxidation catalysts. In a similar catalytic system for propane ammoxidation, Cundari et al. further combined the ANN with a GA method for a quick catalyst selection [53]. With the use of the GA method, people can more rationally design the catalysts by optimizing the inputs of ANNs. Similarly, Umegaki et al. combined the GA and ANN with a parallel activity test for optimizing a Cu−Zn−Al−Sc oxide catalyst for methanol synthesis [54]. Rodemerck et al. generalized the GA-assisted ANN method and proposed a general framework for new solid catalytic materials screening, in good agreement with their experimental data [55]. Based on the previous developments of the GA-assisted ANN methods, Baumes et al. further developed an “ANN fliter” for the high-throughput screening (HTS) of heterogeneous catalysis discovery [56]. Using the water-gas shift (WGS) reaction as an example, they showed that though the optimization method previously developed by Corma et al. was successful for ODHE (as mentioned at the beginning of this Subsection) [49], it failed to precisely estimate the WGS reaction activities. However, with a well-trained ANN classifier as a filter that could help define the “good” and “bad” catalysts, WGS catalysts could be rationally designed with a GA-assisted HTS method. Being similar to Reference [56], their framework is summarized and reconstructed in Figure 5.

In addition to the GA-assisted ANN method, there are other strategies that could also help speed up the design and/or screening of catalysts. Kasiri et al. modeled and optimized the heterogeneous photo-Fenton process using both response surface methodology (RSM) and ANN [57], with the experimental measured H₂O₂ concentration, catalyst concentration, initial pH, and initial dye concentration as the inputs of the models. They found that for a catalyst design process, ANN is as powerful as RSM. Basri et al. also found that, for optimizing the lipase-catalyzed synthesis of a palm-based wax ester, ANN superiorly outperformed RSM [58]. More comparative studies using RSM and ANN can be found in References [59,60,61,62,63,64,65].

4. Applications of ANN for Catalysis: Theory

4.1. Prediction of Reaction Descriptors

To theoretically assess the catalytic activity of a reaction, density functional theory (DFT) [66,67] calculations and molecular dynamic (MD) [68,69] simulations have been widely used to calculate the energy barriers of a catalytic reaction. With the applications of the transition state theory (TST), people can calculate the theoretical reaction rates with the knowledge of an energy barrier or reaction energy [70]. However, both DFT and MD are limited by the high computational cost with a relatively large catalytic system, which dramatically hinders the further developments of the computer-aided catalyst design and its relevant mechanism study. To address this problem, Nørskov and his co-workers successfully developed a “volcano activity plot” method to estimate the theoretical activities of monometallic heterogeneous catalysts [71,72,73]. By estimating the energy barriers using the Brønsted–Evans–Polanyi (BEP) relationship (the linear or nearly-linear relationships between the energy barrier and reaction energy under the same reaction mechanism [74]) [75,76], and predicting the binding energies of reaction intermediate species using the scaling relationship [77,78], they modeled the general trends of the catalytic activities of heterogeneous catalysts, which significantly boosted the subsequent industrial community for new catalyst design. Interestingly, they found that the heterogeneous catalytic activities always correlate well with one or two binding energies of the reaction species (e.g., CO and O for CO oxidation [73], CO for CO₂ electroduction [79], O and OH for oxygen reduction [71], and H for hydrogen evolution [72]). Using these binding energies as the reaction descriptors, a large number of new mono- and multimetallic catalysts have been discovered [80,81]. However, with the increase of industrial demand, using DFT calculations for a screening of the binding energies still requires a large number of computational resources. Especially for the design of alloy catalysts, the computational cost would further increase due to the atomic ensemble effects (specific arrangement of group of atoms [82]). To develop a knowledge-based alternative, scientists started to use ANNs to help acquire the binding energies of adsorbates on the catalytic surface. Davran-Candan et al. employed an ANN to predict the CO and O₂ binding energies over Au₂ to Au₁₀, with excellent modeling results [83]. Their training and testing results for CO and O₂ binding predictions are shown in Figure 6. To predict the more complicated system, the alloy catalysts, Ma et al. used an ANN model to screen the CO binding energies on various bimetallic models. Based on the ANN-predicted results, they found that Cu₃Y−Ni@Cu and Cu₃Sc−Ni@Cu had the desired CO binding energies for CO₂ electroreduction [79]. Their screening results are shown in Figure 7. As a further step for the CO₂ electroreduction catalyst search, Ulissi and Nørskov et al. developed a framework that combined ANN potential energy fitting and the binding energy prediction for bimetallic alloy design [84]. They concluded that a machine learning technique can enable an exhaustive screening on both bimetallic facets and active atomic ensembles for a given catalysis. Figure 8 illustrates their machine learning-assisted screening framework and their screening results shown in a volcano activity plot of CO₂ electroreduction. With the increasing demand of catalyst design and screening, DFT calculations alone may no longer fulfill the experimental requirements. With these machine learning-assisted methods, it has been shown that some state-of-the-art data mining techniques (e.g., ANNs) are able to perform ultra-fast and precise predictions of catalytic descriptors based on a sufficiently large DFT-calculated database. The research done by Ulissi and Nørskov et al. [84] can be a good milestone that illustrates how DFT can work with machine learning and perform a quick and exhaustive catalysts screening and optimization.

4.2. Prediction of Potential Energy Surface

In addition to the study of catalytic descriptors, knowledge about the configurations of catalytic systems is also vital for catalysis study. Usually, even with some state-of-the-art experimental techniques, it is still hard to detect the most favorable configurations of a catalyst system, especially for nano catalytic systems. Fortunately, Monte Carlo (MC) [86] methods are able to provide simulations that perform global optimizations for a specific catalytic system (e.g., looking for the global minima of a specific metallic cluster catalysts). However, these methods strongly rely on the energy potential for acquiring the total energy. To use an alternative potential energy, Sumpter and Noid first systemically showed that neural networks are able to map the vibrational motion determined from spectra onto a fully coupled potential energy surface (PES) with a high accuracy [87]. Subsequently, people started to use ANN and other machine learning techniques for mapping the PES of some simple chemical reactions (e.g., H₂ dissociation [88]) and molecular interactions (e.g., water dimer [89]). In terms of the global optimization of catalytic systems, Ouyang et al. used an ANN with 56 independent variables in the input layer to fit the PES of Au₅₈ clusters [90]. With a well-trained ANN, they performed a basin-hopping (BH) [91] configuration search using the network as the “black-box” potential (Figure 9). Validated by DFT calculations, they found a new global minima that was more energetically stable than all the previously studied Au₅₈ clusters.

However, most of the previous ANN-potential constructions used molecular coordinates or bond lengths as the inputs, which is seriously limited by the size of the system. If a cluster is large, there will be a large number of input variables. This would lead to an extremely long time-consumption for model training. To address this problem, reducing the number of inputs for each single network training is particularly important. To propose a good solution, Behler and Parrinello first suggested a generalized neural-network representation of high-dimensional PES (Figure 10) [92]. Using the radial symmetry function, the atomic environment of each atom can be described by the set of $\left\{G_i^{\mu }\right\}$, which only includes the independent variables of the pair interaction $\left\{G_i^1\right\}$ (Equation (4)) and angular terms $\left\{G_i^2\right\}$ (Equation (5)):

$$G_i^1={{\displaystyle \sum }}_{j\ne i}^{all}{e}^{-\eta {\left(R_{ij}-R_s\right)}^2}{f}_c\left(R_{ij}\right),$$

(4)

$$G_i^2=2^{1-\zeta }{{\displaystyle \sum }}_{j,k\ne i}^{all}{\left(1+\lambda cos{\theta }_{ijk}\right)}^{\zeta }\times e^{-\eta \left(R_{ij}^2+R_{ik}^2+R_{jk}^2\right)}{f}_c\left(R_{ij}\right)f_c\left(R_{ik}\right)f_c\left(R_{jk}\right),$$

(5)

where $\eta$ and $\zeta$ are tunable parameters and $f_c$ is the cutoff function of the interatomic distance $R_{ij}$ (Equation (6)) with a defined cutoff distance $R_c$. With this representation, ANN training of the PES can be described as the training of separate ANN models with only two inputs. Many subsequent applications of this representation method have shown that it is effective to precisely acquire the PES with an acceptable modeling time [93,94,95,96,97]. Chiriki et al. studied the dynamics and thermodynamics of Au_n nanocatalysts ($17\le n\le 58$) using ANN potentials [98]. Similar studies on varying Au configurations were also reported by Boes et al. [99]. With a similar method, Zhai and Alexandrova developed a GPU-accelerated deep neural network (DNN) [32] for the global optimization of Pt₁₃ clusters [100]. To provide a general package for PES fitting, Khorshidi and Peterson developed an atomistic machine-learning package (Amp) combined with the new neural network representation shown above [101]. Based on this package, some catalytic structural information studies were published (e.g., oxygen interaction on Pd [102]). To expand this method to multi-element systems, Boes and Kitchin studied the surface segregation effects of PdAu random alloy models. They found that together with the network representation proposed by Behler and Parrinello, using two separate models, they were able to acquire a quick and precise potential for the atomic segregation studies of PdAu (Figure 11) [103]. Combined with the MC method, they discovered a moderate surface segregation of Pd under high temperature, which might lead to different catalytic activities.

$$f_c\left(R_{ij}\right)=\{\begin{array}{cc}0.5\times \left[cos\left(\frac{\pi R_{ij}}{R_c}\right)+1\right],& for R_{ij}\le R_c\\ 0,& for R_{ij}>R_c\end{array}$$

(6)

5. Remarks and Prospects

In this review, we have shown that as a powerful predictive tool, ANN has been widely used in catalytic studies for both experiment and theory. It can be clearly seen that though the applications of ANN for experimental studies began in the 1990s, its application for theoretical catalytic studies was not popular until the last decade. The remarks can be summarized as follows:

(1)

As the most straightforward application, ANN has been widely used for the prediction of catalytic performance during the past two decades. Though there are various relevant studies, the motifs are quite similar: setting the experimental conditions and/or the properties of the catalytic system as the inputs, and the catalytic activities as the output of the model. Figure 2 summarizes three typical examples of such an application. It shows that the number of output variables can be more than one. That means an ANN with a sufficiently large database is able to perform multiple outputs to predict the product distribution and reaction selectivity.

(2)

In the catalysis community, the optimization and design of catalysts are usually more important. In addition to predicting the catalytic activities, some studies generated new input combinations for a well-trained ANN model, and acquired the predicted output activities. For the generations of new input combinations, GA is the most popular and (so far) the most successful strategy for input generation with less time-consumption. It is expected that in addition to the GA method, a machine learning-assisted HTS can be more sufficient for the generation of inputs in future study [37,104].

(3)

In terms of the theoretical catalysis study, ANN has proven to be a good tool for catalytic descriptor prediction (e.g., binding energy of adsorbate on a catalytic surface). Based on a DFT-calculated database with proper independent variables as the inputs, ANN is able to “learn” the highly-complicated intrinsic properties via a non-linear fitting process. Several successful studies such as the research done by Ulissi and Nørskov et al. [84] have shown that machine learning can be a good choice to reduce the computational cost of theoretical catalysis study, and meanwhile, provides precise and ultra-fast screening for new catalyst discovery.

(4)

To theoretically study the PES for a catalytic system and perform global optimizations, ANN has shown its capacity for mapping the PES for a specific reaction and/or a specific type of catalyst, combined with the MC methods. To reduce the inputs, Behler and Parrinello [92] developed a new ANN representation for atomic systems, which dramatically reduces the number of inputs for network training and saves much time and manpower. It is expected that this method would be more widely used for discovering the favorable structures of the catalytic systems (especially the metallic catalytic systems) and screening good potential catalysts.

(5)

It should be noted that though machine learning development has boosted the ANN applications to the catalysis study of both experiment and theory, many previous studies failed to use an appropriate training-and-testing method. Many of the studies did not perform optimization on ANN structures before they used the model for further applications. This is clearly not doing it in the correct way and could be risky for real applications. To define the optimal structure of ANN, different numbers of hidden neurons and (even) hidden layers should be tried for multiple training and testing. Failing to do this would lead to the severe potential risks of under- or over-fitting.

Though there is a huge success of ANNs that could facilitate the progress of the catalysis community, there are still some common challenges that should be addressed in future studies:

(1)

So far, most of the relevant studies have been done by a conventional ANN (e.g., BPNN). However, with the development of machine learning, conventional ANNs are sometimes no longer the best choice. For example, for fitting a PES of a metallic cluster, a BPNN with regular activation functions sometimes cannot provide smooth fitting, leading to incorrect forces. Also, the required training time of a conventional ANN is another challenge: with a larger database and higher number of hidden neurons and layers, the required training time would become much longer. At this moment, finding out the optimal ANN structure would become harder. Actually, with the algorithm developments, there are many machine learning methods that are sometimes more precise and much faster than the conventional ANN (e.g., GRNN, support vector machine (SVM) [105], and ELM [31]). Several comparative studies have been performed to compare the speed and accuracy of different algorithms [18,34,35]. It is expected that an increasing number of machine learning algorithms will be applied to catalysis studies in the future.

(2)

Similarly, with the rapid development of big data analysis and deep learning techniques [32], it is expected that they could be widely applied for catalytic activity predictions and global structural optimizations of catalytic systems. Though, so far, only a few relevant studies have focused on deep learning techniques (e.g., Zhai et al. [100]), more applicable studies should emerge in the near future. It is also expected that some of the current challenges, such as CO₂ electroreduction selectivity and machine learning-assisted MD simulations, could be well-addressed and understood by state-of-the-art data-mining analysis and deep learning techniques.

(3)

Compared to other research areas, the applications of machine learning for the catalysis community are still not popular and not well-studied. The main reasons include: (i) acquiring the original database for model training is expensive; (ii) too many input variables have to be considered for modeling training; and (iii) there is a lack of user-friendly platforms. The first two points can be addressed by the developments of experimental and computational techniques and devices. The development of good atomic representations can also help reduce the input variables. In terms of the third point, despite there being some chemical packages that could help speed up the machine learning fitting, due to the complexity of chemical systems, very few of them are effective and user-friendly enough. It is expected that future inter-disciplinary study would help address this issue and more well-developed software platforms can be provided for more complicated catalytic studies.