Energy Potentials of Agricultural Biomass and the Possibility of Modelling Using RFR and SVM Models

Abstract

Agricultural biomass is one of the most important renewable energy sources. As a byproduct of corn, soybean and sunflower production, large amounts of biomass are produced that can be used as an energy source through conversion. In order to assess the quality and the possibility of the use of biomass, its composition and calorific value must be determined. The use of nonlinear models allows for an easier estimation of the energy properties of biomass concerning certain input and output parameters. In this paper, RFR (Random Forest Regression) and SVM (Support Vector Machine) models were developed to determine their capabilities in estimating the HHV (higher heating value) of biomass based on input parameters of ultimate analysis. The developed models showed good performance in terms of HHV estimation, confirmed by the coefficient of determination for the RFR (R² = 0.79) and SVM (R² = 0.93) models. The developed models have shown promising results in accurately predicting the HHV of biomass from various sources. The use of these algorithms for biomass energy prediction has the potential for further development.

1. Introduction

With increasing population growth and negative climate change trends, there is a need to create sustainable systems of energy production and bioeconomy [1]. The use of green technologies, i.e., biofuels, is among the most effective ways for reducing greenhouse gasses that directly affect global warming. Biomass is a renewable energy source and has high potential in energy production [2]. Large amounts of lignocellulosic biomass in the world allow its utilization and conversion into an alternative fuel source [3]. Calorific value is the most important parameter in assessing the possibility of using biomass as a fuel [4]. Higher heating value (HHV) is an important factor in describing the quality of fuel and the possibility of using biomass for energy conversion. The use of biomass as an alternative fuel source is considered environmentally and economically viable and offers the possibility of replacing current fossil fuels. One of the fundamental characteristics of biomass is its chemical composition, i.e., the ultimate analysis, which includes a percentage of carbon (C), hydrogen (H), nitrogen (N), sulfur (S) and oxygen (O) [5]. Large amounts of biomass are produced as a by-product in agricultural production and most of it is unused. Biomass from agricultural products such as corn residues, straw and sunflower stalks represents an easily available energy source [6]. The authors also state that corn cobs are an economically viable and environmentally accessible source of biomass that can be used for agricultural energy production. Soybean is a legume that has economic value in terms of seed, straw and biomass yield. In the study by Krisnawati and Adie [7], the soybean was mentioned for its great prospect as a biomass energy source and is especially recognized as a biomass source for fuel production due to its favorable energy properties [8]. The sunflower is an extremely important feedstock grown on a large area that can produce a significant amount of biomass per hectare [9]. The authors also note that biomass from sunflower cultivation is a suitable feedstock for the production of second-generation biofuels.

In order to assess the possibility of using biomass as a fuel, its composition must be determined by various laboratory analyses. The elements of organic matter in biomass are carbon (C), hydrogen (H), oxygen (O), nitrogen (N) and sulfur (S) which can be measured by the ultimate analysis [10,11]. Higher heating value (HHV) is a specific characteristic of biomass that can be used to evaluate the possibility of using biomass for energy through conversion (in the form of heat, fuel, etc.) [12]. In the study of Roman et al. [13], the investigation aimed to evaluate the mechanical and energetic properties of shredded pine forest residues during the briquetting process. The shredded fragments of the forest residues were compacted by the principal stresses with a determination of the energy value consumed during the briquetting process.

In the conducted research by Ibikunle et al. [14], a model was developed to predict the HHV of municipal waste using the input data of the ultimate analysis. The models used to estimate HHV were either in linear or quadratic form. The best-fitting model in the study showed good performance in terms of prediction and the coefficient of determination (R² = 0.97) was used as the main evaluation parameter. SVM models for estimating HHV are applicable to different types of biomass, thus providing a good solution to the problem of estimating HHV [15]. A machine learning (ML) model created to estimate HHV of biomass was based on the input parameters of proximate analysis data (percentage of fixed carbon, ash and volatile matter). An extreme learning machine (ELM) method proved to be very practical in estimating the HHV, as evidenced by the high coefficient of determination for the input parameters of fixed carbon (0.972), volatiles (0.989) and ash (0.968) [12]. Bychkov et al. [16] investigated developed models for predicting the HHV of plant biomass from ultimate analysis data. In the conducted study, authors used 150 models of which 8 were selected for model testing, with 3 showing good performance in estimating the HHV of biomass with small deviations from actual values.

The aim of this paper is to determine the possibility of the mathematical modelling of HHV for corn, soybean and sunflower biomass using machine learning techniques, such as the support vector machine (SVM) and random forest regression (RFR) models for regression, where all data are divided into two parts, such that one part is for training and one part for testing the model in a ratio of 70–30(%). The statistical test “goodness of fit” is used as the main evaluation method for the model performance. A comparison of the SVM and RFR models will show which model is better for predicting HHV biomass based on the input parameters of the ultimate analysis.

2. Materials and Methods

2.1. Data Collection

The data for creating the SVM and RFR models were obtained from the literature [17,18,19,20,21,22,23,24,25,26,27,28]. The data of the ultimate analysis and HHV for 51 biomass samples, including 27 samples for corn, 15 for soybean and 19 for sunflower biomass, including the biomass and biomass products, were collected for model development.

2.2. Nonlinear Modelling

After the collection dataset, the data were divided into a part for training and a part for testing the model, in the ratio of 70% and 30%, for SVM and RFR models. Statistical analysis was performed using TIBCO STATISTICA 13.3.0 software (StatSoft TIBCO Software Inc., Palo Alto, CA, USA). The analyzed data were presented in the form of mean and standard deviation. Analysis of variance (ANOVA) and Tukey’s HSD (honestly significant difference) test were used to compare the samples to show the variability of the observed data.

SVM, as a supervised learning model, is based on statistical theories and can be used for clustering and regression [15]. The SVM model created is based on the input data from the ultimate analysis, as type 1 regression models with Kelner type (Radial Basis Function—RDF) and 9 support vectors. The total number of iterations of the SVM model is 10,000.

As nonlinear models, RFRs are suitable for predictions with medium and large data sets [28]. The RFR models were also based on the input data from the final analysis with the data split into 70% for training and 30% for testing the model. The models were built using 10,000 random trees. For each internal node within the decision trees, entropy is calculated using the formula (Equation (1)) [29]:

$$E=-{\displaystyle \sum _{i=1}^c{p}_i\times \log (}{p}_i)$$

(1)

where c represents the number of unique classes and p_i prior probability of each given class. In order to solve the nonlinear problem, the Kelner function is used to map the input vectors into a multidimensional vector proctor that is used to find the hyperplane [30]. The equation used to create the SVM model is shown in the following equation (Equation (2)) [31]:

$$\gamma ={\mathsf{\omega }}^T\theta (\chi )+b$$

(2)

where $\gamma$ is target value, w is the weight vector, b is the threshold, θ is the nonlinear function of the model and $\chi$ is input vector.

2.3. Models Verification

To show the performance of the developed SVM and RFR models with respect to the input variables of the ultimate analysis, the following statistical parameters must be calculated: x² (reduced chi-square) from Equation (3), RMSE (root mean square error) from Equation (4), MBE (mean bias error) from Equation (5), MPE (mean percentage error) from Equation (6), and SSE (sum of squared estimate error) from Equation (7). “Goodness of fit” is calculated using the above statistical parameters to find the model with the lowest error, and they are represented by the following equations [32]:

$$x^2=\frac{{\displaystyle \sum _{i=1}^N(x_{pre,i}}-x_{\exp ,i}{)}^2}{N-n}$$

(3)

$$RMSE={\left[\frac{1}{N}\cdot {\displaystyle \sum _{i=1}^N{(x_{pre,i}-x_{\exp ,i})}^2}\right]}^{1/2}$$

(4)

$$MBE=\frac{1}{N}\cdot {\displaystyle \sum _{i=1}^N(x_{pre,i}-x_{\exp ,i})}$$

(5)

$$MPE=\frac{100}{N}\cdot {\displaystyle \sum _{i=1}^N\left(\frac{\left|x_{pre,i}-x_{\exp ,i}\right|}{x_{\exp ,i}}\right)}$$

(6)

$$SSE={\displaystyle \sum _{i=1}^N{(x_{pre,i}-x_{\exp ,i})}^2}$$

(7)

3. Results

Table 1. Ultimate analysis of corn, soybean and sunflower biomass.

Sample	C (%)	H (%)	N (%)	S (%)	O (%)	HHV (MJ/kg)
Corn biomass	48.45 ± 7.3 a	5.29 ± 1.11 b	0.70 ± 0.50 a	0.08 ± 0.09 a	38.11 ± 11.22 a	19.42 ± 2.72 a
Soybean biomass	47.37 ± 3.48 a	4.53 ± 0.73 a	1.38 ± 2.32 b	0.22 ± 0.3 b	46.5 ± 6.83 b	17.93 ± 1.04 a
Sunflower biomass	54.16 ± 8.9 b	6.51 ± 0.9 c	2.22 ± 1.64 c	0.06 ± 0.13 a	35.62 ± 9.25 a	22.46 ± 4.41 b
Significance	**	*	*	**	*	*
Minimum	47.37	4.53	0.70	0.06	35.62	17.93
Maximum	54.16	6.51	2.22	0.22	46.50	22.46
Average	49.99	5.44	1.44	0.12	40.08	19.94

C—concentration of carbon; H—concentration of hydrogen; N—concentration of nitrogen; S—concentration of sulfur; O—concentration of oxygen; HHV—higher heating value. The means in the same column (various samples) with different lowercase superscripts are statistically different (p < 0.05), according to Tukey’s HSD test. Statistical significance; * p ≤ 0.01; ** p ≤ 0.05.

shows the average values of the ultimate analysis and the HHV of the observed biomass.

Table 1 shows the mean and standard deviation of the variables of the ultimate analysis of corn, soybean and sunflower biomass. Sunflower biomass has the highest value for C (average 54.16%), H (average 6.51%), N (2.22%) and HHV (average 22.46 MJ/kg), and the lowest value for S (average 0.06%) and O (35.62%). Higher proportions of C and H influenced the increase in the HHV value [33]. Soybean biomass has the lowest average value of C (47.37%), H (4.53%) and HHV (17.93 MJ/kg), while it has the highest average value of S (0.22%). The corn biomass sample has an average value of C (45.45%), H (5.29%), N (0.70%), S (0.08%), O (38.11%) and HHV (19.42 MJ/kg). All observed samples showed a significant difference. Thus, the variables C and S are statistically significant at a significance level of α = 0.05, while the observed variables H, N, O and HHV are statistically significant at a significance level of α = 0.01.

Figure 1 shows the correlation diagram of the observed variables of the ultimate analysis and HHV of the average values of corn, soybean and sunflower biomass. The correlation of the observed values is shown in the range −1 to 1, which corresponds to the color intensity. Variables C (r = 0.98), H (r = 0.99) and N (r = 0.70) are positively correlated with the HHV value, while variables S (r = −0.83) and O (r = −0.88) are negatively correlated with the HHV value.

Figure 2 shows the PCA of the observed biomass samples in relation to the variables of the ultimate analysis and the HHV. The PCA method is used to simplify a complex array of data into more understandable groups for presentation by grouping all data with minimal losses into meaningful units [34,35]. The sunflower biomass group has the highest value for H, C, N and HHV, while the soybean biomass group has the lowest value for C, H and HHV. The observed corn biomass group has the lowest value for the variable N.

4. Discussion

4.1. Support Vector Machine (SVM)

SVM models can be used for prediction in the form of regression due to their ability to generalize to different sample sizes and the possibility of nonlinear modeling [36]. The standard form of SVM belongs to the supervised form of learning and offers numerous advantages in terms of optimization and solution finding in nonlinear modeling [30].

Table 2. Vector values of the developed SVM model.

Vector No.	Weights	Support Vector					Decision Constant
		C	H	N	S	O
1	9.00	0.22	0.63	0.07	0.07	0.73	−0.09
2	−7.34	0.20	0.68	0.07	0.12	0.70
3	−0.21	0.95	0.00	0.21	0.02	0.00
4	−1.01	0.35	0.49	0.00	0.32	0.96
5	0.34	0.25	0.71	0.11	0.47	0.83
6	−9.00	0.28	0.67	0.17	0.09	0.72
7	6.14	0.51	0.70	0.16	0.00	0.64
8	−1.95	0.77	0.96	0.57	0.00	0.36
9	4.02	1.00	0.97	0.59	0.00	0.20

shows the values of the vector, weighting coefficients and decision constant with respect to the input variables in the SVM model.

Table 2 presents vector values of the developed SVM model.

4.2. Random Forest Regression (RFR)

Random Forest Regression (RFR) can easily adapt to nonlinear relationships between data and shows better predictive ability than linear regression models. RFRs are considered a reliable tool for predicting performance [29,37].

Figure 3 shows the importance of the predictor in the value 0 to 1 on the output value of HHV in the RFR model. In modeling, the highest predictor value is for O (1.00), followed by C (0.99), N (0.95), S (0.78) and H (0.57). With regard to the presented Figure 3, it can be concluded that the input parameters O, C and N have the greatest influence on the output value of HHV.

4.3. Goodness of Fit

Table 3. “Goodness of fit” for developed SVM and RFR models.

								Residual Analysis
Model	x2	RMSE	MBE	MPE	SSE	AARD	R2	Skewness	Kurtosis	SD	Variance
SVM model	0.82	0.90	−0.03	3.07	49.28	44.12	0.93	−3.04	14.32	0.91	0.82
RFR model	5.99	2.43	−0.01	8.32	359.53	103.30	0.79	0.94	2.08	2.45	5.99

shows the statistical analysis, “goodness of fit”, which presents the ability of the developed models to predict the HHV values.

Table 3 shows the statistical test, “Goodness of fit”, which shows the performance of the developed SVM and RFR models. The calculated values of x² (0.82), RMSE (0.90), MBE (−0.03), SSE (49.28), AARD (44.12) and R² (0.93), and the residual analysis skewness (−3.04), kurtosis (14.32), SD (0.91) and Var (0.82) show the low level of error of the SVM model. The values of x² (5.99), RMSE (2.43), MBE (−0.01), MPE (8.32), SSE (359.53), AARD (103.30), and R² (0.79) were calculated for the RFR model in the table. The skewness (0.94), kurtosis (2.08), SD (2.45) and variance (5.99) parameters were determined by the residual analysis. Both developed models showed satisfactory performance in modeling the HHV values. Considering that, R² is used as the main indicator of the model’s ability for estimation.

The scatterplot visualization technique was used to analyze correlations of variables on the x and y axes and to detect associations and anomalies in a multidimensional dataset [38]. Figure 4 shows the parity plot of the predicted and targeted HHV of the developed SVM and RFR models for training and test data. Both models show an extensive overlap of the data, with a coefficient of determination of R² = 0.79 for the RFR model, while the overlap values in the SVM model have a higher coefficient of determination (R² = 0.93). In the study conducted by Xing et al. [38], machine learning models were built to estimate the HHV value of biomass based on the input parameters of ultimate and proximate analysis. The RFR model in the study shows a great fit for prediction (R² > 0.94), while the SVM model also shows good performance (R²~0.90). The RFR models give good results in terms of performance. The parameters R², MAPE and RMSE are calculated at 0.94, 0.57 and 2.56, respectively, while for the SVM model the parameters R², MAPE and RMSE are 0.90, 0.76 and 3.53, respectively. The study also showed the relative importance of the input parameters of ultimate analysis in the models for C (61.6%), H (20%), O (9.6%) and N (8.8%). Considering everything, it can be concluded that the developed models are suitable for estimating the HHV based on the input parameters of the ultimate analysis. Using the performed statistical test “Goodness of Fit”, the parameters showed a low level of error in estimating the HHV, while the SVM model shows a higher level of performance in modeling.

The developed SVM and RFR models show good ability in estimating the HHV of corn, soybean and sunflower biomass. In developing the model, the input parameters of the ultimate analysis (percent concentration of C, H, N, S and O) were used to estimate HHV. In the case of the RFR model, the output value is most influenced by the variables in the following order: O (1.00), C (0.99), N (0.94), S (0.78) and H (0.57). The SVM model with a number of five independent input parameters, as a regression model, shows the best performance with nine support vectors using RDF (Radial Basis Function) as a Kelner type. Data from 51 different biomass samples were used for the study (27 for corn biomass, 15 for soybean biomass and 19 for sunflower biomass). The SVM model showed better performance than the RFR model because the mentioned model generalizes and covers the data better for a small and medium data set, while the RFR model shows better estimation performance for a medium and large data set [39,40].

5. Conclusions

Agricultural biomass generated from corn, sunflower and soybean production has a great potential as a feedstock for energy production. Large amounts of biomass are produced as a byproduct during agricultural production and are often unused. In order to assess the possibility of using biomass as a fuel, its energy properties and composition must be determined. Nonlinear models offer the possibility to estimate the HHV of biomass with high accuracy. The developed nonlinear models in the form of Random Forest Regression (RFR) and Support Vector Machine (SVM) were determined as successful tools in estimating biomass HHV. Thus, this work shows the satisfactory ability of the SVM (R² = 0.93) and RFR models (R² = 0.79) in estimating HHV based on the input parameters of the ultimate analysis of observed agricultural biomass.