Next Article in Journal
Key-Recycling Strategies in Quantum-Key-Distribution Networks
Next Article in Special Issue
Battery Management System for Future Electric Vehicles
Previous Article in Journal
Multiple Criteria Evaluation of the EU Country Sustainable Construction Industry Lifecycles
Previous Article in Special Issue
Torque and Battery Distribution Strategy for Saving Energy of an Electric Vehicle with Three Traction Motors
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Empirical Thermal Performance Investigation of a Compact Lithium Ion Battery Module under Forced Convection Cooling

by
Akinlabi A. A. Hakeem
1 and
Davut Solyali
2,*
1
Department of Engineering, Faculty of Mechanical Engineering–Eastern Mediterranean University, Famagusta, North Cyprus 99628, Turkey
2
Director, Electric Vehicle Development Center (EVDC), Eastern Mediterranean University, Famagusta, North Cyprus 99628, Turkey
*
Author to whom correspondence should be addressed.
Appl. Sci. 2020, 10(11), 3732; https://doi.org/10.3390/app10113732
Submission received: 10 April 2020 / Revised: 8 May 2020 / Accepted: 16 May 2020 / Published: 28 May 2020
(This article belongs to the Special Issue Battery Management System for Future Electric Vehicles)

Abstract

:
Lithium ion batteries (LiBs) are considered one of the most suitable power options for electric vehicle (EV) drivetrains, known for having low self-discharging properties which hence provide a long life-cycle operation. To obtain maximum power output from LiBs, it is necessary to critically monitor operating conditions which affect their performance and life span. This paper investigates the thermal performance of a battery thermal management system (BTMS) for a battery pack housing 100 NCR18650 lithium ion cells. Maximum cell temperature (Tmax) and maximum temperature difference (ΔTmax) between cells were the performance criteria for the battery pack. The battery pack is investigated for three levels of air flow rate combined with two current rate using a full factorial Design of Experiment (DoE) method. A worst case scenario of cell Tmax averaged at 36.1 °C was recorded during a 0.75 C charge experiment and 37.5 °C during a 0.75 C discharge under a 1.4 m/s flow rate. While a 54.28% reduction in ΔTmax between the cells was achieved by increasing the air flow rate in the 0.75 C charge experiment from 1.4 m/s to 3.4 m/s. Conclusively, increasing BTMS performance with increasing air flow rate was a common trend observed in the experimental data after analyzing various experiment results.

1. Introduction

There is a growing global concern of the causes and effects of climate change which has led to stricter environmental regulations on carbon-based machines [1,2] coupled with huge advancements in portable battery technology—specifically, lithium ion electric vehicles (EVs) and hybrid electric vehicles, which are starting to disrupt the automobile industry markets by presenting themselves as the vehicle choice of the future [3,4]. Some major hindrances to electric vehicle mass adaptation are the range anxiety of EVs, the lack of super-fast charging and the lack of performance driving, etc. [1,5]. The performance driving and fast charging problems of EVs are due to the limitation of the lithium ion batteries in performing outside tight operating temperature ranges [6]. The range anxiety problem of electric vehicles is also attributed to the gravimetric density of lithium ion batteries (LiBs). When compared to traditional gasoline-powered vehicles, the average energy-to-weight ratio of lithium ion batteries is 0.3 MJ/kg and it is over 30 MJ/kg for gasoline-powered vehicles [7].
While the current gravimetric property limitation of LiBs may be a design constraint on EV performance, EV manufacturers have the freedom to design robust battery thermal management systems (BTMS) for EV battery packs (BP) to efficiently limit the amount of heat generated by the LiBs during their operating cycles (charge/discharge). One technique; a sub-classification (see Figure 1) of air-cooled BTMS employed by various researchers used in improving the cooling performance of a BTMS, is reviewed and investigated in this paper.

1.1. Literature Review

In recent years there have been many studies performed with the aim of improving the cooling performance of air-cooled BTMS by employing optimizing techniques as illustrated in Figure 1. The research studies conducted regarding the performance improvement of BTMS which deal primarily with design variables pertaining to manners of arranging cells inside a BP and the placement of cooling-air intake and exhaust vents to obtain the best performance are discussed in the following paragraphs.
Chen et al. (2017) performed a configuration optimization on prismatic lithium ion cells for a parallel air-cooled system. In this model, the BTMS is optimized through arranging the spacing among the battery cells to obtain the best cooling performance. The optimization strategy is applied several times on a developed flow resistance model and a heat transfer model until the appropriate cell spacing is obtained. Their results exhibited a 42% reduction in the maximum cell temperature over the design variable optimization iterations on the developed model [9].
By comparing an aligned versus staggered cylindrical cell arrangement for a BTMS (see Figure 2A,B), N. Yang et al. (2015) in [10] investigated the effects of transverse and longitudinal spacing between cylindrical cells in a BP with a forced-air cooling system. N. Yang et al. (2015) developed a numerical and thermal model for this BP, which was used to simulate the effects of various design variables on their BP model. The model with the best performance results was validated by physical experiment. N. Yang et al. (2018) reported that under a specific cooling-air flow rate, the maximum cell temperature rise in a BP is proportional to the longitudinal interval for staggered arrays, whereas the inverse holds for aligned cell arrays. Finally, they obtained a better performing BTMS model, by optimizing the longitudinal and transverse space between the cells, coupled with optimizing an air inlet duct width for a BP with aligned arrangements [10].
Lu (2018) provided a parametric study of forced-air cooling for lithium ion batteries with staggered arrangements. They designed a three-dimensional simulation model (Gambit 2.4.6 CFD) of the BP that investigated the effects of cooling channel size and air supply strategy for the model. The CFD model was solved using the semi-implicit method for pressure-linked equations (SIMPLE) algorithm. They deduced that a cooling channel size of 1 mm was appropriate for BPs with Panasonic 18,650 cells. Upon further investigation, they reported the best cooling performance was achieved when placing the cooling-air flow inlet and outlet on the top of the BP. Finally, they reported that the efficiency factor of a BTMS deceases with the number of cells in the horizontal direction; hence, they recommended a maximum of 10 cylindrical cells along the air flow direction for a BP [11].
In a more recent study, Chen et al. (2020) in their paper numerically studied five (5) BTMS battery pack configurations and verified simulation results by conducting physical experiments. They developed a simple method to achieve symmetrical air flow inside each of the five battery packs by repositioning inlet and outlet vents on each original battery pack design (I, II, III, IV and V) to get newly optimized BPs; (I1, II1, III1, IV1 and V1) as depicted in Figure 3 below. Further parametric optimization of cell spacing revealed that uneven cell spacing in the improved battery packs resulted in better BTMS cooling performance just as BPs with a symmetrical air flow path did over their original counterparts [12]

1.2. Current Study

This paper investigates the performance of a battery thermal management system (BTMS) for a proposed battery pack model with “H” symmetrical air vents housing 100 NCR18650 lithium ion cells. The battery module is built in a 10P10S configuration with aligned cells. Maximum cell temperature (Tmax) and maximum temperature difference (ΔTmax) between cells were the performance criteria for the BTMS. The battery pack is investigated for three levels (1.4 m/s, 2.4 m/s and 3.4 m/s) of air flow rate combined using a full factorial experiment design method with two current rates (0.5 C and 0.75 C) under charging and discharging power cycles.

2. Investigated Battery Module

The cells in the battery module (Figure 4 above) are connected in a 10S10P configuration to provide a minimum rated power of 1.024 KWh minimum and 1.344 KWh maximum. Nominal data specification value of a single NCR18650 cell used in the battery module being tested are presented in Table 1 below.
An experimental setup up was designed to test the battery module for real-life scenarios while varying optimizing parameters—cooling-air velocity and current flow rate of the battery pack.
The experimental setup in Figure 5 consists of a battery pack with four switch-mode power supply (SMPS) cooling fans, a 3D-printed part connecting a flexible vent pipe to connect the fans to the atmosphere, a CPX400D (Aim TTi, Huntingdon, United Kingdom) power supply, a 16-channel temperature data acquisition device (Applent, China), a 4000 W EA-EL 9200 electronic load (Electro Automatik, San Diego, CA, USA) and a 200 V EA-PSI 9200 (Electro Automatic, San Diego, CA, USA) power supply.

2.1. Experimental Setup

2.1.1. Battery Pack

The battery pack in the experimental setup was designed and built from medium fiberboard (MDF) of 18 cm thickness with four vent holes of 0.07 m diameter (Figure 6a). The pack had dimensions (30 × 25 × 10 cm) and was built slightly larger than the exact volume of the battery module (24 × 24 × 10 cm). This extra volume in the MDF box was designed to create a partition that would accommodate the excess length of the thermocouple sensor wires attached to the cells in the battery module (see Figure 6b). Glass fiber, (a nonconductive and nonflammable material) was placed between the battery module top cover of the MDF board as a protective measure to prevent possible electrical and fire hazards present during the experiment and most importantly prevent a low-resistance path for air flowing into the battery pack (Figure 6c).

2.1.2. Temperature Data Acquisitions Device

In order to monitor and record the temperature of cells in the battery module during testing, a 16-channel Applent temperature data acquisition device in Figure 7 was employed.
The T-DAQ used in the experimental setup was powered by an ARM microprocessor capable of measuring temperatures from a variety of thermocouple types (T, J, K, E, etc.) at three sampling rates (fast, medium and slow) and had a resolution of 0.1 °C. It allowed for multiple recording of temperature values from 16 channels simultaneously which it stored onto a USB stick or directly onto a computer via a USB-serial connection.
K-Type thermocouples sensors with lower and upper limits of 0 °C and 200 °C temperature ranges well within the limits of the expected temperature rise of the cells in the battery module to be tested were employed during the experiment to measure temperature profiles of selected cells.
The positions of cells to be monitored was systematically selected based on the assumption that the temperature of each selected cell would represent the local temperature of other cells in its surrounding. Factors such as the number of channels the T-DAQ is limited to impacted the decision to monitor only 16 cells out of 100 cells in the battery module, as well as the preexisting compact nature of the battery module (see Figure 8 below).
The following measurements were taken to ensure proper contact between the sensor and the cells:
  • In order to ensure proper cell-sensor connection and improve conduction between the two entities, a thermal paste compound was applied around the sensor and the battery body in contact after which strips of strong adhesive tape were used to secure the sensor to the body of the cell (see Figure 9).
  • Silicone glue was applied to hold the protruding sensor wire at the attachment point on top of the cell terminal to provide for extra attachment strength (see Figure 9).
  • Finally, the majority of the sensor wire length was kept folded in a separate partitioned section inside the MDF box to prevent accidental tension that might threaten or sever the connection between the sensor-cell attachment.

2.1.3. CPX400D Power Supply

To achieve cooling on the battery module in the battery pack, four (4) SMPS fans attached to the vents of the MDF box were connected in parallel and routed via two connecting cables on the MDF box to be powered simultaneously using the CPX400D power supply Figure 10 below. This method of connection ensured all the cooling fans operated at the same speed at any preset voltage.
Operating the SMPS fans in constant current mode, controlling the voltage input to the fans via the power supply allowed for the adjustment of power delivered to the fans hence controlling air flow into the battery pack.
Figure 11 shows the base air-flow configuration of the battery pack investigated in this thesis.

2.1.4. Electronic Power Supply/Load

The electronic load and power supply used in this study were the Electro-Automatik (EA) heavy duty laboratory Direct Current (DC) load and power supply. The versions used in conducting the experiments—the EA-EL 9200 electronic load capable of an output of 4000 W and the EA-PSI 9200—operated at efficiency of up to 95.5% (see Figure 12).
To test the battery pack module for charge and discharge cycles during the experiments, 50 Amps-rated cables were used in connection between the battery pack, the electronic load and power supply.
As testing for two current ratings on the battery pack module (0.5 C and 0.75 C) were to be investigated, the load and power supply were programmed to a constant current rating of 0.5 C, and the maximum and minimum voltages set to fit the specification of the battery pack maximum and minimum cut off voltages (see Table 2). The Graphic User Interface (GUI) of the load/supply provided real-time voltage and current readings of the battery pack for monitoring purposes during the charge/discharge cycles.

2.2. Designing of Experiments

This study aimed to investigate the thermal performance of a hundred (100) NCR18650 lithium cells battery modules in a battery pack with four vents. The test process used in this study aimed to simulate as closely as possible near real-time application scenarios, hence the ambient conditions such as room temperature are considered an uncontrollable parameter so that little or no action is taken to control or alter ambient conditions during the test period.
After defining factors and their levels to be tested for experiment in this study, various design of experiment (DOE) methods such as Plackett–Burman, Taguchi, latin square and full factorial posed as viable methods to be used in planning and designing the experiments to be carried out. After assessing various strength and features of each method, the “full factorial experiment design” was settled for as it allowed for a study of the main and interacting factor (air and current flow rate) effects on the battery module and also allowed for the development of a response surface of the design space tested. The full factorial method employed also provided the maximum number of experiments be performed for selected factors and levels. A total number of thirty-six (36) experiments were carried out (due to repetition) and twelve (12) unique experiments analyzed after the results of experiments with similar combinations of factors were averaged.
Table 3 below shows the various factors and levels tested during the experiment and the experiment design development code using MATLAB (R2016B, Mathworks, Inc., Natick, MA, USA, 2016).

2.3. Experiment Procedure

The charge/discharge method employed while testing the battery pack performance followed the constant current–constant voltage (CC–CV) or Galvanostatic method. Employing this method of charging, the battery module is initially charged at a specified current rate of 0.5 C (16 Amps) from its minimum cutoff voltage of 32 V until it barely reaches its maximum cutoff voltage—typically 41.99 V. At this stage in the charging process, the voltage is held at constant until the current flow rate reaches 0.3 − 0.2 Amps.
Before each cycle of the experiment, the cooling fans were inspected and set to the required level of air flow rate and measured with an anemometer. As the battery pack design employs two inlet cooling fans, the area of the vents calculated in Table 4, was doubled to determine the total volume of air being pushed into the battery pack at every set cooling fan speed.
After each charge/discharge cycle was completed, time was allowed for the cells in the battery module to rest in order to ensure electrochemical stability [14] before a new cycle commenced. This waiting period also allowed for the entire battery module to reach a uniform cooled temperature. Figure 13 illustrates the systematic steps carried out to perform each experiment cycle.

2.4. Objective Functions Investigated

Temperature profiles of sixteen (16) cells from the battery module were monitored and recorded by the K-type thermocouples and the temperature DAQ as the output variables of the experiment. An average of 172 temperature data points (corresponds to 172 min) were generally recorded during the 0.5 C level charge experiment and 150 data points during the charging experiment with 0.75 C current rate. Generally, lower times of discharging period were observed for the 0.5 C and 0.75 C experiments.
The temperature values recorded during each charge/discharge cycle experiment were stored in a generic created file by the T-DAQ which was retrieved for data processing for data analysis.
For results analysis of the measured temperature profile of selected cells in the battery module, the nomenclature depicted in Figure 14 was adopted to address various individual cells (e.g., Cell#01, Cell#07, etc.), or a group of cells in a row as Row 1, Row 2, Row 3 and Row 4.

2.4.1. Maximum Temperature (TMAX)

The performance and longevity of a cell operating under any given charge or discharge cycle greatly relies on its operating temperature not exceeding 40 °C [13,15,16]. The maximum temperature (TMAX) of any individual cell in a battery pack cooled under any battery thermal management system is therefore indicative of the overall performance of the BTMS system.

2.4.2. Temperature Increase (TINC)

The Temperature increase (TINC) represents the temperature difference between the initial temperature (TI) and the highest temperature (TH) of each measured cell in the battery pack. This measure, similar to the maximum temperature of a cell in the battery pack, is indicative of the performance of a BTMS but takes into consideration the temperature profile of each measured cell in relation to its neighboring cells. It also allows for the measure of temperature uniformity between cells in similar rows—one (1) through four (4)—or submodules. For a better BTMS performance, temperature uniformity between cells improves charging uniformity.

2.4.3. Temperature Difference (ΔTMAX)

The maximum allowable temperature between cells in a BP of 5 °C has been reported in several studies including [16,17,18,19], to promote battery balancing and uniform charging and discharging during the LiB’s operating cycle. The average temperature of cells in each row was determined to determine the temperature difference among cells in the battery module for each experiment performed.
Typically, an experiment with a combination of design levels which yields a temperature difference among various cells in a battery pack above 5 °C would be considered to have performed poorly.

3. Results and Discussion

This section provides a concise and precise description of the experimental results, their interpretation as well as how they are interpreted within the perspective of previous studies.

3.1. Thermal Performance of BTMS

The maximum temperature experienced by monitored cells in the battery pack during the experiment performed in this study were obtained by averaging the temperature values of experiments performed under similar combinations of design parameters after repetition. Figure 15 presents a capture of the results of all the tests performed in this study.
The data points are plotted based on the arrangements of cells in the battery pack with respect to the cooling-air flow channel; so that maximum temperature (TMAX) of cells in Row 1 (Cells: #01, #05, #9, #13) which are closest to the inlet vents are plotted first following the systematic pattern through to cells in Row 4 based on the illustration presented in Figure 14, page 11 above.
From the graph of results presented in Figure 15 a common trend in the thermal behavior of cells in Row 1, irrespective of the current rate, charging cycle or the cooling-air speed, is that they recorded the least maximum temperature. This can be associated to the fact that they naturally experience the effects of cooling air pumped into the battery pack at its ambient state in terms of temperature and speed. Another factor that plays greatly to this observed trend is the absence of accumulated heat generated by collective cells in the battery pack at the inlet vent area.
Another common trend observed in the results across all the graphs in Figure 15 is the relatively similar maximum temperatures in the battery packs during the discharge cycle and the charge cycle experiments. Quantitatively, the absolute peak temperature obtained by the cell in channel 12 during the 0.5 C charge cycle is 30.7 °C, and 29.3 °C during the discharge cycle under a 1.4 m/s cooling rate. Table 5 presents further comparisons during the charge and discharge cycle for cell #06 observed during tests carried out in this study.
In the data presented in Figure 15 there is a relatively bigger difference in the thermal behavior of the battery pack during its operation under a cooling-air flow rate of 1.4 m/s as compared to the performance between the air flow rates of 2.4 m/s and 3.4 m/s. The closeness in performance of the battery pack for air flow rate 2.4 m/s and 3.4 m/s is observed for charging and discharging under 0.5 C and for discharging under 0.75 C while the trend does not hold true for charging under 0.75 C. This break in trend can be associated with the tendency of lithium ion cells to generate significantly heat at a higher current rate.
The general increase in the trend of the maximum temperature registered in the cells as they move further away from the inlet vents is observed for cells 1 through 16 in the entire test performed and can be associated with the effects of heat accumulation and increase in resistance of the flow of the cooling-air path. Similar effects have been reported in [18,20] and measurements such as bidirectional air flow have been proposed and investigated for battery packs minimizing cell maximum temperature and inter cell temperature difference hence improving temperature uniformity in the cells of a battery pack [18,20,21].
A critical study of the thermal result of the battery module presented in Figure 15, cellmonitored by channel 12 of the T-DAQ is noticed to always record slightly lower temperatures than cells in its locality (Row 4). Upon investigating this behavior, it was observed that Cell 12 happens to be directly in front of an exhaust vent hence it is hypothesized that Cell 12 experienced slightly better cooling as the heated air in the battery pack was constantly vented from its position. This phenomenon would also prevent heat accumulation at that locality.
Finally, comparing results obtained for the battery pack performance presented in this study, an averaged maximum temperature of 36.1 °C was obtained for the 0.75 C charge experiment at a 1.4 m/s flow rate and a temperature value of 37.5 °C after three repetitions of discharging at 0.75 C and 1.4 m/s. In a real-life application, charging under such conditions (0.75 C & 1.4 m/s) would not be recommended as the optimal temperature for operating lithium ion cells is 40 °C [13,22]: a value which the worst-obtained results in this study (4 to 5 °C) is just shy of.

3.1.1. Effects of Air Flow Rate on Maximum Temperature

From previous research conducted in the literature review stage of this study, the general trend observed in many published research under BTMS studies is that a higher cooling-air flow rate yields better BTMS performance in terms of the objective functions: minimization of maximum cell temperature, increases temperature uniformity amongst cells and minimizes temperature differences between cells.
A similar trend has been observed for the battery pack model presented and tested in this study. As shown in Figure 16. The test results presented in Figure 16 just as in Figure 15 are obtained after averaging the temperature data recorded for the unique experiments after repetition.

3.1.2. Effects of Air Flow Rate on Temperature Difference

Investigating the performance of the battery pack for temperature difference between cells, the average temperature of cells in each row (one through four) is determined for all the individual unique experiments. This measure taken reduces the measured temperature output from sixteen cells to four cells classified by their positions in the battery pack (see Figure 14, page 11 above). After classification of all sixteen cells into four rows, the maximum temperature in each row is determined. Lastly, the absolute difference in TMAX between all the combinations of rows is presented in Figure 17 and Figure 18.
In Figure 17, it is observed that for the lower current rate of 0.5 C tested, the majority of temperature differences between rows measured in the battery pack is below 5 °C. The maximum temperature difference measured during experiments conducted under the 0.5 C current rate was recorded to be 8 °C during a charge cycle under 1.4 m/s and 6.37 °C for a discharge cycle under the same flow rate which occurred between cells in Row 1 and Row 4.
For flow rates of 2.4 and 3.4 m/s tests under 0.5 C, the temperature difference between all interacting rows in the BP was kept well below 5 °C.
In Figure 18, which plots the cell temperature difference for experiments conducted under a 0.75 C current rate, the temperature differences between Row 1 and Row 2, Row 1 and Row 3, Row 1 and Row 4 and Row 2 and Row 4 were observed to be above the 5 °C threshold of under air flow rate of 1.4 m/s during the charge and discharge experiments.
As the air flow rate increases to 2.4 m/s, the temperature difference which occurred under the charge cycle is reduced from three to two interactions (Rows 1 and 3, Rows 1 and 4) similar to results obtained for the 0.5 C charge experiment under 1.4 m/s.
The maximum temperature differences between Rows 1 and 4 during the charge and discharge cycle under 0.75 C were 13.81 °C and 12.42 °C respectively for 1.4 m/s flow rate. These values reduced to 6.0 °C and 5.25 °C under an air flow rate of 3.4 m/s.

3.2. Battery Pack Model Development

In this section, two model development techniques are applied to the data set applied and obtained from the experiments in this study. Raw input data as described in Table 6 and maximum temperature data obtained from the 16 monitored cells are used as input and output training data for an artificial neural network (ANN) algorithm to develop a model for the battery pack investigated in this study. All 12 unique current rate and air flow rate combination experiments developed by the full factorial DOE method are used as input data (X,Y) with the averaged absolute maximum temperature of each unique experiment used as an output (Z) to develop a surface regression model.

3.2.1. Artificial Neural Network

An artificial neural network (ANN) is a set of interconnected neurons that mimic information processing, similar to humans. It provides a function for creating, training, visualizing and simulating neural networks capable of performing classification, regression, clustering time-series forecasting and dynamic modeling [23,24]. A neural network consists of a two layer feed-forward network with sigmoid hidden neuron and linear output neurons that can fit multidimensional mapping problems arbitrarily well, given consistent data and enough neurons.
Several types of neural network and how they are applied to solve the specific problems they are suited for have been demonstrated in literature. Peculiar to research on lithium ion batteries, X. Qian, et al. in [25] applied neural networks in optimization of his design parameters for a proposed battery pack model and H. Sassi et al. applied ANN to an empirical data set to develop a model to predict the State of Charge (SOC) level of lithium ion cells studied in their work [23].
In this study, an ANN architecture model (Figure 19) is trained offline with the input data set from all experiments conducted in this study. Input data included the charge cycle (represented as 1), discharge cycle (represented as 0), the ambient temperature during the experiment, the current rate and the air flow rate (Table 6) while the maximum temperature of all monitored cells were fed as output data to the neural network.
In total, a matrix of 4 by 37 and 16 by 37 data size were used as input and output data sets respectively. Seventy percent of input data was allocated for the neural network for model training while 15% each was allocated for the testing and validation phase of the ANN model development.
After several iterations of training, a model obtained with an R value (correlation value) of 0.99139 was obtained between the fitted model and the given training data, 0.9812 correlation R value between the fitted model and the given data for testing and an R value of 0.98869 for an overall correlation between the actual outputs and the targets was obtained (see Figure 20). In Figure 21, the error histogram is plotted showing the difference between the target and actual output of the ANN training, which revealed that among the total samples considered, the majority of the error lies in the range 0.1034 to 2.08.
The training algorithm selected for training the ANN module in this thesis study was the ‘Bayesian Regularization’ algorithm for its performance with difficult, small and noisy data sets [24]—a critical feature of the data output obtained during the conducted experiments in this study.

3.2.2. ANN Model Validation/Error Analysis

After the ANN model was developed and trained with experimental data, a simulink model illustrated in Figure 22 and a MATLAB function code was generated as a representation of the model. In this chapter, the accuracy of the model developed was tested in a closed loop fashion by comparing the model output value to a set of experimentally obtained output values and finding the absolute percentage error between the two values for a given set of input data. A lower percentage error value between a predicted and measured value is desired for a good model and implies such a model will be good for studying the physical model applying optimization if need be.
Figure 23 shows the maximum temperature comparison between empirically obtained data for a discharge cycle experiment conducted with a 0.75 C current rate and 2.4 m/s air flow rate versus the obtained maximum temperature values predicted by the trained model. Figure 23 shows a close relation between the experiment and predicted model output with an absolute maximum percentage error of 1.84%. As Figure 23 compares and presents the relative error between the outputs of the trained ANN model and actual experimental output values for a given input data set, Figure 24 illustrates the error percentage graph between the predicted and actual maximum temperature values for the entire experiment input data. The data in Figure 24 showed that 93% of the entire input-data-predicted output by the ANN model when compared to their counterpart experimentally obtained output had an absolute percentage error of less than 4%. A maximum percentage error of 10.38% between a predicted and an actual output for a given set of input was measured.

3.2.3. Response Surface Model (RSM) Development

An RSM comprises regression surface fitting over a bounded design space to predict approximated responses for input variable combinations not accounted for during a physical or simulation experiment [26]. Usually, a design of experiment method is used to obtain the minimum number of experiments needed to develop an RSM. In the RSM method implemented in this study, the approximation function used in developing the response surface illustrated in Figure 25 is a second degree polynomial. Most response surfaces functions are generated with polynomials depending on the number of data points provided from an experiment.
The RSM developed for the battery model was done using a MATLAB curve fitting tool box. Fitting parameters R2 adjusted were used to determine the goodness of fit, and sum of square error (SSE) and mean square error are used to determine the predictability of the model. The R2 adjusted values lie between 0 and 1 for which a good response surface has a value closest to 1 [26,27]. On the contrary, an SSE and Mean Square Error (MSE) value closer to zero is desired for a good response surface. An SSE value closer to zero indicates a model has smaller random error components hence higher prediction accuracy [27]. Just as for SSE, an MSE value closest to zero is desired for a good response model.
After various variations of the fit method applied on the variables were tested to obtain the best adjusted R2 and SSE values, the model was developed with the polynomial function with robustness “off” as this trail produced the most suitable fitness parameters results. Table 7 provides a comparison of various selection criteria values obtained during the iterative training process.
The model equation of the RSM developed for the charge and discharge experiments performed for the battery pack and module in this study is presented as:
f ( x , y ) = P 0 + P 1 x + P 2 y + P 3 x 2 + P 4 x y
where x and y are designed variables air flow rate and current rate respectively, P 0 P 4 are coefficient constants and f ( x , y ) is the design objective function. Table 8 presents coefficient values for the developed model equations.

3.2.4. RSM Model Validation/Error Analysis

In an attempt to validate the regression model developed in this study, equations of the developed RSM’s were tested for experimental inputs to compare the output results. Figure 26 plots and compares the predicted absolute maximum temperatures against the actual maximum temperatures for the charge experiment at 0.5 C and the discharge experiment at 0.75 C.
Table 9 below shows the absolute relative error calculations between the predicted model and the actual outputs for absolute maximum temperature in a battery pack for all twelve of the unique experiments performed. The results showed the developed model predicts accurately with an absolute maximum error of approximately 3.00%.

4. Conclusions

In this study, a battery thermal management system (BTMS) for a battery pack housing a battery module consisting of a hundred NCR18650 lithium ion cells was designed and tested in relatively cold ambient conditions.
The BTMS performance was tested for two major objective function criteria which are:
  • maximum temperature recorded by a cell in the battery pack which at any instant should be kept below 40 °C and
  • maximum temperature difference between any cells in the battery pack during testing which should not exceed a threshold of 5 °C.
The temperature limit threshold for operating LiBs was optimally obtained from various literature and battery data specification documents.
Design variables—three levels of cooling-air flow rate and two levels of current rate were combined using a full factorial experiment design method to develop a full array of experiments performed on the BTMS for the battery pack after experimentation. General BTMS performance trends were observed in the obtained results such as higher current rate experiments produced relatively higher maximum temperature amongst the cells and significant changes in maximum cell temperature and temperature difference are observed as air flow rate is increased.
Upon investigating the effects of increasing air flow rate on maximum cell temperature, a 15.09% reduction in maximum temperature recorded by a cell was achieved during a charging experiment with a 0.5 C current rate by increasing the air flow rate from 1.4 m/s to 2.4 m/s. For the same charge experiment, there was no significant improvement in the maximum temperature recorded at a higher air flow rate of 3.4 m/s. On the contrary, at a higher current rate of 0.75 C charging, a 13.20% reduction in the maximum temperature of a cell was achieved by increasing the air flow rate from 1.4 m/s to 2.4 m/s. Further increment of the air flow rate to 3.4 m/s produced a 22.81% reduction in the maximum cell temperature.
The results summarized above aids in drawing a hypothesis that for the investigated battery pack design, for each current rate, there exists an optimal cooling-air flow rate that if exceeded, will yield little or no improvement in an operating BTMS performance at a specific range of ambient conditions. This hypothesis will prove vital in scenarios where power consumption of an operating BTMS is a critical objective function to be minimized. This will be applicable and vital to the development of an intelligent/dynamic BTMS where cooling operation of a BTMS will operate in a dynamic mode depending on the current profile the battery module.
When assessing the BTMS performance based on temperature difference between cells, for a cooling-air flow rate of 1.4 m/s for every cycle experiment, there was always an instant where the maximum temperature difference between monitored cells exceeded 5 °C, reducing the performance of the BTMS. However, at higher speeds of 2.4 m/s and 3.4 m/s, in experiments under 0.5 C, the temperature difference among cells in the battery pack was found to be always below the 5 °C threshold.
Lastly, for 0.75 C experiments under 1.4 m/s, 100% of the interactions between cells in Rows 2, 3 and 4 with Row 1 exceeded 5 °C. However, after increasing the air flow rate temperature difference between cells in Rows 2, 3 and 4 with Row 1 which exceeded 5 °C, is reduced by 33.33%. The highest temperature recorded under 0.75 C with 2.4 m/s was found to be 7.71 °C (a 44.14% reduction from operation under 1.4/s) and 6.3 °C (a 54.38% reduction from operation under 1.4/s) for 3.4 m/s between cells in Row 4 and Row 1.
The observations made aides in affirming the conclusion based on literature that a higher cooling-air flow rate reduces maximum cell temperature and temperature difference between cells in a battery pack due to the increased convective heat transfer coefficient of air at higher velocities. However, the gradient in temperature difference amongst cells remains the same with at least a single case of interaction between cells at different positions in the battery module exceeding 5 °C for the investigated battery module.
To fully obtain an air-cooled BTMS performance with the possibility of no interacting cells in the battery module having ΔTmax exceeding 5 °C, a bidirectional cooling flow path scheme must be implemented.

Author Contributions

The work carried out and presented in this manuscript was carried out as part of A.A.A.H. Master’s program completion requirement under the close supervision of D.S. as his thesis project supervisor. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Acknowledgments

The authors would like to acknowledge the entire EMU EVDC [28] and the director Davut Solyali for the support provided in the successful completion of this project in terms of experimental equipment setup and a conducive laboratory working environment.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Kim, J.; Oh, J.; Lee, H. Review on battery thermal management system for electric vehicles. Appl. Therm. Eng. 2019, 149, 192–212. [Google Scholar] [CrossRef]
  2. IEA. Global EV Outlook 2017 Together Secure Sustainable Global EV Outlook 2017; International Energy Agency: Paris, France, 2017; pp. 5–55. [Google Scholar]
  3. Wu, W.; Wang, S.; Wu, W.; Chen, K.; Hong, S.; Lai, Y. A critical review of battery thermal performance and liquid based battery thermal management. Energy Convers. Manag. 2019, 182, 262–281. [Google Scholar] [CrossRef]
  4. Akinlabi, A.A.H.; Solyali, D. Configuration, design, and optimization of air-cooled battery thermal management system for electric vehicles: A review. In Renewable and Sustainable Energy Reviews; Elsevier Ltd.: Amsterdam, The Netherlands, 2020; p. 109815. [Google Scholar]
  5. Budde-Meiwes, H.; Drillkens, J.; Lunz, B.; Muennix, J.; Rothgang, S.; Kowal, J.; Sauer, D. A review of current automotive battery technology and future prospects. Proc. Inst. Mech. Eng. Part D J. Automob. Eng. 2013, 227, 761–776. [Google Scholar] [CrossRef]
  6. Li, L.; Dababneh, F.; Zhao, J. Cost-effective supply chain for electric vehicle battery remanufacturing. Appl. Energy 2018, 226, 277–286. [Google Scholar] [CrossRef]
  7. Adolf, J.; Balzer, C.H.; Louis, J.; Schabla, U.; Fischedick, M.; Arnold, K.; Pastowski, A.; Schüwer, D. Shell Hydrogen Study Energy of the Future? Sustainable Mobility through Fuel Cells and H2; Technical Report; Shell Deutschland Oil GmbH: Hamburg, Germany, 2017. [Google Scholar]
  8. Arora, S. Selection of thermal management system for modular battery packs of electric vehicles: A review of existing and emerging technologies. J. Power Sources 2018, 400, 621–640. [Google Scholar] [CrossRef]
  9. Chen, K.; Wang, S.; Song, M.; Chen, L. Configuration optimization of battery pack in parallel air-cooled battery thermal management system using an optimization strategy. Appl. Therm. Eng. 2017, 123, 177–186. [Google Scholar] [CrossRef]
  10. Yang, N.; Zhang, X.; Li, G.; Hua, D. Assessment of the forced air-cooling performance for cylindrical lithium-ion battery packs: A comparative analysis between aligned and staggered cell arrangements. Appl. Therm. Eng. 2015, 80, 55–65. [Google Scholar] [CrossRef]
  11. Lu, Z.; Yu, X.; Wei, L.; Qiu, Y.; Zhang, L.; Meng, X.; Jin, L.W. Parametric study of forced air cooling strategy for lithium-ion battery pack with staggered arrangement. Appl. Therm. Eng. 2018, 136, 28–40. [Google Scholar] [CrossRef]
  12. Chen, K.; Chen, Y.; She, Y.; Song, M.; Wang, S.; Chen, L. Construction of effective symmetrical air-cooled system for battery thermal management. Appl. Therm. Eng. 2020, 166, 114679. [Google Scholar] [CrossRef]
  13. Panasonic. Datasheet: Specifications Powercell NCR18650B; SANYO Energy Corporation: San Diego, CA, USA, 2012; No. 469; p. 1. [Google Scholar]
  14. Gümüşsu, E. Thermal Modeling of Lithium Ion Batteries Lityum Iyon Pillerin Isi L Modellemesi. Master’s Thesis, Hacettepe University, Ankara, Turkey, 2017. [Google Scholar]
  15. Al-Zareer, M.; Dincer, I.; Rosen, M.A. Performance assessment of a new hydrogen cooled prismatic battery pack arrangement for hydrogen hybrid electric vehicles. Energy Convers. Manag. 2018, 173, 303–319. [Google Scholar] [CrossRef]
  16. Li, K.; Yan, J.; Chen, H.; Wang, Q. Water cooling based strategy for lithium ion battery pack dynamic cycling for thermal management system. Appl. Therm. Eng. 2018, 132, 575–585. [Google Scholar] [CrossRef]
  17. E, J.; Han, D.; Qiu, A.; Zhu, H.; Deng, Y.; Chen, J.; Zhao, X.; Zuo, W.; Wang, H.; Chen, J.; et al. Orthogonal experimental design of liquid-cooling structure on the cooling effect of a liquid-cooled battery thermal management system. Appl. Therm. Eng. 2018, 132, 508–520. [Google Scholar] [CrossRef]
  18. Wang, S.; Li, K.; Tian, Y.; Wang, J.; Wu, Y.; Ji, S. Improved thermal performance of a large laminated lithium-ion power battery by reciprocating air flow. Appl. Therm. Eng. 2019, 152, 445–454. [Google Scholar] [CrossRef]
  19. Pesaran, A. Battery thermal models for hybrid vehicle simulations. J. Power Sources 2002, 110, 377–382. [Google Scholar] [CrossRef]
  20. Yu, K.; Yang, X.; Cheng, Y.; Li, C. Thermal analysis and two-directional air flow thermal management for lithium-ion battery pack. J. Power Sources 2014, 270, 193–200. [Google Scholar] [CrossRef]
  21. Lu, Z.; Meng, X.; Wei, L.; Hu, W.; Zhang, L.; Jin, L.W. Thermal Management of Densely-packed EV Battery with Forced Air Cooling Strategies. Energy Procedia 2016, 88, 682–688. [Google Scholar] [CrossRef] [Green Version]
  22. Liu, X.; Chen, Z.; Zhang, C.; Wu, J. A novel temperature-compensated model for power Li-ion batteries with dual-particle-filter state of charge estimation. Appl. Energy 2014, 123, 263–272. [Google Scholar] [CrossRef]
  23. Sassi, H.B.; Errahimi, F.; Es-Sbai, N.; Alaoui, C. A comparative study of ANN and Kalman Filtering-based observer for SOC estimation. IOP Conf. Ser. Earth Environ. Sci. 2018, 161, 12022. [Google Scholar] [CrossRef] [Green Version]
  24. View Neural Network—MATLAB View. Available online: https://www.mathworks.com/help/deeplearning/ref/view.html?s_tid=doc_ta (accessed on 28 December 2019).
  25. Qian, X.; Xuan, D.; Zhao, X.; Shi, Z. Heat dissipation optimization of lithium-ion battery pack based on neural networks. Appl. Therm. Eng. 2019, 162, 114289. [Google Scholar] [CrossRef]
  26. Li, Z.-Z.; Cheng, T.-H.; Xuan, D.-J.; Ren, M.; Shen, G.-Y.; Shen, Y.-D. Optimal design for cooling system of batteries using DOE and RSM. Int. J. Precis. Eng. Manuf. 2012, 13, 1641–1645. [Google Scholar] [CrossRef]
  27. Goodness of Fit Statistics. Available online: https://web.maths.unsw.edu.au/~adelle/Garvan/Assays/GoodnessOfFit.html (accessed on 21 August 2019).
  28. EVDC—Electric Vehicle Development Center—EMU. Available online: https://evdc.emu.edu.tr/en (accessed on 21 August 2019).
Figure 1. Classification of air-cooled battery thermal management systems (BTMS) and optimization parameters adapted from [1,4,8].
Figure 1. Classification of air-cooled battery thermal management systems (BTMS) and optimization parameters adapted from [1,4,8].
Applsci 10 03732 g001
Figure 2. Schematic of aligned (A) and staggered (B) cell layout optimization adapted from [4,10].
Figure 2. Schematic of aligned (A) and staggered (B) cell layout optimization adapted from [4,10].
Applsci 10 03732 g002
Figure 3. Asymmetrical vs. symmetrical BTMS battery packs adapted from [4,12].
Figure 3. Asymmetrical vs. symmetrical BTMS battery packs adapted from [4,12].
Applsci 10 03732 g003
Figure 4. Battery module investigated.
Figure 4. Battery module investigated.
Applsci 10 03732 g004
Figure 5. Experimental setup.
Figure 5. Experimental setup.
Applsci 10 03732 g005
Figure 6. (a) Built battery pack from fiberboard, (b) Placement of thermocouples to the cells. (c) Protective wool placed on top of the battery cells.
Figure 6. (a) Built battery pack from fiberboard, (b) Placement of thermocouples to the cells. (c) Protective wool placed on top of the battery cells.
Applsci 10 03732 g006
Figure 7. Temperature data acquisition device (T-DAQ).
Figure 7. Temperature data acquisition device (T-DAQ).
Applsci 10 03732 g007
Figure 8. Thermocouple position selection and attachment.
Figure 8. Thermocouple position selection and attachment.
Applsci 10 03732 g008
Figure 9. Sensor attachment measures.
Figure 9. Sensor attachment measures.
Applsci 10 03732 g009
Figure 10. Controlling fan speed with the CPX400D power supply.
Figure 10. Controlling fan speed with the CPX400D power supply.
Applsci 10 03732 g010
Figure 11. Battery vent configuration, inlet Fans I and II, outlet fans III and IV.
Figure 11. Battery vent configuration, inlet Fans I and II, outlet fans III and IV.
Applsci 10 03732 g011
Figure 12. EA electronic power supply and load.
Figure 12. EA electronic power supply and load.
Applsci 10 03732 g012
Figure 13. Experiment cycle procedures. CC-CV: constant current–constant voltage.
Figure 13. Experiment cycle procedures. CC-CV: constant current–constant voltage.
Applsci 10 03732 g013
Figure 14. Sensor position nomenclature.
Figure 14. Sensor position nomenclature.
Applsci 10 03732 g014
Figure 15. Thermal characteristics of the battery pack.
Figure 15. Thermal characteristics of the battery pack.
Applsci 10 03732 g015
Figure 16. Effects of air increasing air flow rate on maximum temperature.
Figure 16. Effects of air increasing air flow rate on maximum temperature.
Applsci 10 03732 g016
Figure 17. Effects of air flow rate on temperature differences (0.5 C).
Figure 17. Effects of air flow rate on temperature differences (0.5 C).
Applsci 10 03732 g017
Figure 18. Effects of air flow rate on temperature differences (0.75 C).
Figure 18. Effects of air flow rate on temperature differences (0.75 C).
Applsci 10 03732 g018
Figure 19. ANN architecture.
Figure 19. ANN architecture.
Applsci 10 03732 g019
Figure 20. ANN regression plot.
Figure 20. ANN regression plot.
Applsci 10 03732 g020
Figure 21. Error histogram.
Figure 21. Error histogram.
Applsci 10 03732 g021
Figure 22. ANN simulink model.
Figure 22. ANN simulink model.
Applsci 10 03732 g022
Figure 23. Predicted maximum cell temperature (Tmax) versus actual value.
Figure 23. Predicted maximum cell temperature (Tmax) versus actual value.
Applsci 10 03732 g023
Figure 24. Percentage error between real output and predicted output values for the entire experiments.
Figure 24. Percentage error between real output and predicted output values for the entire experiments.
Applsci 10 03732 g024
Figure 25. Response surface model (RSM) developed for charge and discharge experiment.
Figure 25. Response surface model (RSM) developed for charge and discharge experiment.
Applsci 10 03732 g025
Figure 26. Regression model predicted output vs. actual output for a sample input.
Figure 26. Regression model predicted output vs. actual output for a sample input.
Applsci 10 03732 g026
Table 1. NCR18650B (green) Specification [13].
Table 1. NCR18650B (green) Specification [13].
SpecificationsValue
Nominal Voltage3.6 V
Cutoff Voltage4.2 V
Minimum Rated Capacity3.2 Ah
Table 2. Battery Module Specification.
Table 2. Battery Module Specification.
Ten (10) Cells in SeriesMaximum Cutoff Voltage (V): 4.2 × 10 = 42 VoltsMinimum Cutoff Voltage (V): 3.2 × 10 = 32 Volts
Ten (10) Cells in ParallelNominal battery module capacity (Amp): 3.2 × 10 = 32 Amps
Tested Capacity0.75 C24 Amps
0.5 C16 Amps
Table 3. Design of Experiment Process.
Table 3. Design of Experiment Process.
FactorsLevels
Current Rate (C)0.5 [1]
0.75 [2]
Air flow Rate (m/s)1.4 [1]
2.4 [2]
3.4 [3]
% full factorial([2 levels of current rate, 3 levels of flow rate])
>> DOE = full fact([2,3])
DOE=
 1 1
 2 1
 1 2
 2 2
 1 3
 2 3
Experiment Map
Repetition0.5 C0.75 C
1.4 m/s2.4 m/s3.4 m/s1.4 m/s2.4 m/s3.4 m/s
1ChargeChargeChargeChargeChargeCharge
DischargeDischargeDischargeDischargeDischargeDischarge
2ChargeChargeChargeChargeChargeCharge
DischargeDischargeDischargeDischargeDischargeDischarge
3ChargeChargeChargeChargeChargeCharge
DischargeDischargeDischargeDischargeDischargeDischarge
Table 4. Battery Pack Parameters.
Table 4. Battery Pack Parameters.
Volume Parameters (m3)
Total0.0075
Partitioned0.00125
Actual0.00625
Vent Dimensions
Diameter of Vent (m)0.035
Area of vent (m2)0.003848
Two inlet vents (m2)0.007697
Volume Flow rate at various fan Speeds (m3/s)
1.40.010776
2.40.018473
3.40.026169
Table 5. Disparities in Maximum Temperature of Cell 12 during Charge/Discharge Cycle.
Table 5. Disparities in Maximum Temperature of Cell 12 during Charge/Discharge Cycle.
0.5 C0.75 C
ChargeDischargeChargeDischarge
1.4 m/s30.7 °C29.3 °C36.1 °C34.6 °C
2.4 m/s26.07 °C26.8 °C31.1 °C29.3 °C
3.4 m/s26.0 °C25.72 °C27.7 °C27.1 °C
Table 6. Artificial Neural Network (ANN) Input Training Data Sample.
Table 6. Artificial Neural Network (ANN) Input Training Data Sample.
ExperimentCycleFlow Rate (m/s)Current Rate (C)Ambient Temp °C
0.5DS1.301.40.517.10
0.75DS2.102.40.7518.76
0.75CS1.111.40.7519.08
Table 7. Comparison of Various Fitness Parameters for the Developed Regression Model.
Table 7. Comparison of Various Fitness Parameters for the Developed Regression Model.
2nd Degree Polynomial Fit
Robustness:Off
Fit TypeSSE* RMSER2Adjusted R2
Charge** poly210.90750.95260.98750.9373
Dischargepoly211.61331.27020.98240.9121
Robustness:Least Absolute Residual (LAR)
Chargepoly210.90750.95260.98750.9373
Dischargepoly211.61331.27020.98240.9121
Robustness:Bi-Square
Chargepoly211.34911.16150.98140.9068
Dischargepoly212.39841.54870.97390.8693
* Root Mean Square Error. ** poly21: A second degree polynomial fit with two degrees of X and one degree of Y. SSE: sum of square error.
Table 8. Model Equation Coefficients.
Table 8. Model Equation Coefficients.
ChargeDischarge
P 0 21.9216.48
P 1 −5.06−3.84
P 2 37.648.75
P 3 1.5751.8
P 4 −9−13.2
Goodness of Fit
Charge:SSE—0.9075, R2—0.9875, Adjusted R2—0.9373
Discharge:SSE—1.6133, R2—0.9824, Adjusted R2—0.9121
Table 9. Absolute Relative Error between Regression Model and Actual Experiment.
Table 9. Absolute Relative Error between Regression Model and Actual Experiment.
Charge at 0.5 CDischarge at 0.5 C
1.4 (m/s)2.4 (m/s)3.4 (m/s)1.4 (m/s)2.4 (m/s)3.4 (m/s)
Actual (°C)30.7026.3026.7029.4026.9025.80
Predicted (°C)30.4226.8526.4229.7726.1726.17
% Error0.91%2.08%1.04%1.25%2.72%1.42%
Charge at 0.75 CDischarge at 0.75 C
1.4 (m/s)2.4 (m/s)3.4 (m/s)1.4 (m/s)2.4 (m/s)3.4 (m/s)
Actual (°C)36.4031.4027.9037.7029.7027.50
Predicted (°C)36.6730.8528.1737.3330.4327.13
% Error0.75%1.76%0.98%0.97%2.47%1.33%

Share and Cite

MDPI and ACS Style

Hakeem, A.A.A.; Solyali, D. Empirical Thermal Performance Investigation of a Compact Lithium Ion Battery Module under Forced Convection Cooling. Appl. Sci. 2020, 10, 3732. https://doi.org/10.3390/app10113732

AMA Style

Hakeem AAA, Solyali D. Empirical Thermal Performance Investigation of a Compact Lithium Ion Battery Module under Forced Convection Cooling. Applied Sciences. 2020; 10(11):3732. https://doi.org/10.3390/app10113732

Chicago/Turabian Style

Hakeem, Akinlabi A. A., and Davut Solyali. 2020. "Empirical Thermal Performance Investigation of a Compact Lithium Ion Battery Module under Forced Convection Cooling" Applied Sciences 10, no. 11: 3732. https://doi.org/10.3390/app10113732

APA Style

Hakeem, A. A. A., & Solyali, D. (2020). Empirical Thermal Performance Investigation of a Compact Lithium Ion Battery Module under Forced Convection Cooling. Applied Sciences, 10(11), 3732. https://doi.org/10.3390/app10113732

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop