Practical Implications of the Availability of Multiple Measurements to Classify Agricultural Soil Compaction: A Case-Study in The Netherlands

Abstract

Soil compaction is a severe threat to agricultural productivity, as it can lead to yield losses ranging from 5% to 40%. Quantification of the state of compaction can help farmers and land managers to determine the optimal management to avoid these losses. Bulk density is often used as an indicator for compaction. It is a costly and time-consuming measurement, making it less suitable for farmers and land managers. Alternatively, measurements of penetration resistance can be used. These measurements are cheaper and quicker but are prone to uncertainty due to the existence of a wide array of thresholds. Classifications using either measurement may provide different outcomes when used in the same location, as they approximate soil compaction using different mechanisms. In this research, we assessed the level of agreement between soil compaction classifications using bulk density and penetration resistance for an agricultural field in Flevoland, the Netherlands. Additionally, we assessed the possible financial implications of misclassification. Balanced accuracy results indicate that most thresholds from the literature show around 70% agreement between both methods, with a maximum level of agreement of 76% at 1.8 and 1.9 MPa. The expected cost of misclassification shows a dip between 1.0 and 3.0 MPa, with an effect of crop value on the shape of the cost function. Although these results are specific to our study area, we believe they show that there is a substantial effect of the choice of measurement on the outcome of soil compaction studies.

1. Introduction

Soil compaction is internationally recognised as a threat to agricultural soil quality and productivity [1,2,3]. It is characterised by a reduction in pore space, increase in soil strength and a disruption of pore continuity [4]. The process of compaction can occur naturally during the ripening of the soil or through vertical movement of soil particles in vertisols [5,6]. In agricultural soils, it is often created by the exertion of an external force on the soil, which happens during the tillage or wheeling of the soil when the soil moisture content is too high and by using heavy machinery [5,7]. The term “soil compaction” is often also used to describe the state of soil degradation, rather than the process itself. Soil compaction can lead to reductions in permeability, pore connectivity, water retention, root growth and nutrient uptake [4,8,9,10,11,12]. Estimated yield losses due to compaction and its effects can reach as high as 40% of the potential yield [13], with more conservative estimates at 5% to 10% [14].

For the Netherlands, studies have indicated that 43% of the Dutch subsoils may be compacted below the annual plough layer [15], and large parts of the country are likely susceptible and vulnerable to soil compaction [16]. This is partially a result of the inherent properties of Dutch soils, many of which are naturally susceptible [16]. Additionally, the machines used in the Netherlands have become heavier [17], and field operations more often take place under less suitable conditions such as late autumn. This vulnerability makes it crucial for farmers and other land managers to take action to prevent, alleviate or remediate the negative effects of soil compaction.

Taking action requires insight into the current state of compaction in the soil on a scale which is relevant to farmers, including the field and sub-field scales. A method to gain this insight, which is also used in scientific research [4,18,19,20,21], is measuring the dry bulk density of the soil (BD_d). As soil compaction leads to the densification of the soil, it logically results in an increase of BD_d. The measured density can be compared to a threshold value which is representative for the value above which the soil is considered “too compacted”. Texture-specific BD_d thresholds have been determined and are generally accepted in both the scientific literature [15,22,23,24] and (inter)national quality standards [25,26].

Using BD_d to find and classify soil compaction seems suitable in principle, but the measurements and analysis are expensive and laborious. Sampling usually takes place in an excavated soil pit, rather than directly from the surface. It can take considerable time to take a core sample properly, as it needs to be extracted from the soil and cut to size without disturbing the sample. Multiple samples are often needed per location, and the analysis needs to occur in a laboratory [25,26]. This can become time-consuming and costly if many locations need to be sampled to capture spatial variation. The Dutch national soil information database (BIS) [27] does include measurements of BD_d, but these data can be outdated, and measurement locations are too sparse to provide field-specific information which Dutch farmers and land managers may require.

Soil compaction can also be assessed using penetrologgers that measure the penetration resistance (PR) of the soil. This soil property is linked to the resistance plant roots experience whilst growing through the soil [28,29]. It is a therefore a measure of soil strength rather than the density. In compacted soils, the densely packed soil material makes it more difficult for roots to penetrate into the soil due to the increase in soil strength. This results in shallow root growth, raising the vulnerability to drought and decreasing the effective depth over which nutrients are plant-available [4]. PR can be measured from the soil surface using a hand-driven cone penetrologger, requiring neither soil excavation nor laboratory analyses. It can be compared to threshold values from literature in order to evaluate the state of compaction (

Table 1. The use of soil penetration resistance thresholds in existing literature, along with any sources cited in each study.

Threshold Value for Soil Compaction	Authors	Cited Source
Using a single threshold
2.0 MPa	Agostini et al. [30]	Gupta & Allmaras [31]
	De Lima et al. [32]	Da Silva et al. [33]
	Fabrizzi et al. [34]	Gupta & Allmaras [31]; Hamblin [35]
	Fernández et al. [36]	-
	Hernandez-Ramirez et al. [37]	Da Silva et al. [33]
	Lapen et al. [38]	Bengough & Mullins [28]; Greacen [39]
	Martins et al. [40]	Barbosa et al. [41]; Otto et al. [42]; Salem et al. [43]
	Özgöz et al. [44]	Zaman [45]
	Tebebu et al. [46]	Hamza & Anderson [47]; Taylor & Gardner [48]
	Van Leeuwen et al. [49]	-
	Wilson et al. [50]	Wilson et al. [51]
2.5 MPa	Alvarez & Steinbach [52]	-
3.0 MPa	Coelho et al. [53]	Taylor & Gardner [48]
4.2 MPa	Stock et al. [54]	-
Using a range of thresholds
1.5 MPa to 3.0 MPa	Laboski et al. [55]	Boone et al. [56]
2.0 MPa to 3.0 MPa	Çelik et al. [57]	Bengough et al. [58]; De Moraes Sá et al. [59]
	Müller et al. [60]	Bengough & Mullins [28]
	Tekeste et al. [61]	Bradfort [62]; Taylor & Gardner [48]

). Above these thresholds, the measured PR is linked to detrimental effects on plant growth.

Using PR as a measure of the state of soil compaction is not without challenges. Measurements are sensitive to soil moisture conditions [63,64] and operator error [65], besides bulk density and soil texture [66,67]. The measured PR typically decreases with an increase in soil moisture content, meaning PR will be higher if the soil is dry and lower if it is wet. Operator errors can occur if the device is not pushed straight into the soil at a constant and moderate pace, resulting in improper depth assessment and unreliable measurements.

These challenges can be partly overcome by sampling under set conditions, taking multiple measurements to average out irregularities, proper instructions for the surveyor or using an automated hydraulic system [65]. However, the method remains sensitive to errors. Table 1 shows a large number of studies in which PR was used to capture compaction, using threshold values to determine the point where detrimental effects occur. Multiple different thresholds exist in the range of 2.0 to 4.2 MPa, most of which were determined for specific crops [48]. This is in contrast to the characterisation of compaction using BD_d, where a single crop-independent threshold is used after correction for clay content.

BD_d and PR are two measurements that can both be used to characterise soil compaction, but they inherently measure different soil properties (i.e., density and soil strength, respectively). Previous studies have shown that the correlation between these two measurements is far from perfect [66,67,68,69], and they did not investigate the effects of using a threshold value for classification. It is possible that a straightforward classification of the state of soil compaction with BD_d does not provide the same result as classification using PR, leading to misclassifications. If farmers and land managers use these measurements to support management choices and reduce the detrimental impact of soil compaction, this can potentially result in a situation where management would be different if another measurement tool were used. It is therefore crucial to determine both the level of agreement between both methods and its (financial) implications when used in a straightforward classification of the state of soil compaction.

Farmers and land managers will likely prefer the use of PR over BD_d due to its simplicity, ease-of-use and relatively low cost per measurement location. However, multiple thresholds are proposed in the literature, and there are no clear guidelines on which threshold is suitable in which situation (Table 1). This issue does not affect BD_d classification as much, as only a single standardised threshold exists for each soil. It is therefore useful to assess the performance of PR over a wide range of thresholds, compared to BD_d classification. The objective of this research was to assess the level of agreement between the classification of the state of soil compaction using BD_d and PR, where BD_d is used as the reference. We used a case study in Flevoland, the Netherlands, to illustrate the level of agreement between both methods in a realistic situation where either method could have been used by the farmer. The aim was to provide a clear indication of the performance of various PR thresholds, as these are the most likely measurements to be used by farmers and land managers. We also aimed to illustrate the possible financial implications from misclassification, if these measurements were to be used to steer management. Our research also functions as a general reflection on the wide array of thresholds proposed in literature and their effects when used in the field.

2. Methods and Materials

2.1. Study Area

Data for this study were collected on an agricultural field in the province of Flevoland, the Netherlands (Figure 1). The region consists of a former seabed that was reclaimed in the 1940s and has since been in use mainly as agricultural land [70]. Crop rotations are based around potatoes, sugar beets, onions and grains. Soils in Flevoland are relatively young and consist mostly of fluvisols and cambisols, using the World Reference Base for Soil Resources [71]. Topsoils in Flevoland range from loamy to clayey, with subsoils of sandy or peaty materials. The topography is linked to its history as a former seabed, being mostly flat with minor natural undulations due to a former coastal system of creeks and tidal channels. The study area is located in the eastern parts of Flevoland, in the Noordoostpolder.

The study site is located at the edge of the town of Emmeloord. In 2017, the study site was used for a large public event on potato cultivation technologies. This event included demonstrations of, e.g., harvesting equipment, tractors, and other heavy machinery. These demonstrations were conducted under rainy conditions and in moist soils, showing visual compaction signs such as the deep rutting and smearing of the soil. Satellite images from 2018 showed within-field heterogeneity, visible as slowly drying wet spots within the field (Figure 2). In 2021, two years after field sampling for this study took place, some of these patterns were still visible (Figure 3). We selected this study site because of its expected heterogeneity in degree of compaction due the heavy compaction event in 2017.

2.2. Data Collection and Pre-Processing

Fieldwork for data collection took place in the autumn of 2018, after maize harvest when the soil moisture was approximately at field capacity. The fieldwork was spread over multiple weeks and was dependent on weather conditions. Sampling was suspended for two days if it rained, allowing the topsoil to dry out enough to avoid sampling errors. The farmer left the soil bare and unploughed before sampling. However, in some places, the topsoil was loosened to a depth of 20 cm using a shallow subsoiler. In order to avoid unwanted influence by this shallow loosening, samples were not taken in loosened soil. The local soil profiles consisted of a loamy topsoil of 30 to 50 cm deep, followed by a subsoil of loamy sand (Figure 4). Visual and tactile observations indicated little vertical or horizontal heterogeneity.

Data collection during fieldwork included measurements of BD_d, volumetric soil moisture and a general description of the soil texture at 5, 15, 25, 35, 45 and 55 cm depth. PR measurements were also available from 0 until 80 cm depth, in increments of 1 cm. These data were gathered at 50 sample locations, but only a subset of 83 samples from 25 locations was suitable for this research due to laboratory handling error-related data loss. This subset included only samples in the loamy topsoil, as previous experience in this field showed that compaction was mostly present in this topsoil layer.

The original 50 sample locations were determined using the conditioned Latin hypercube sample (cLHS) method as used by Minasny and McBratney [72]. Geophysical data with a resolution of 1 m × 1 m from ground penetrating radar (GPR), electromagnetic inductance (EMI) and gamma ray detection were used as covariates in this cLHS. Additionally, the study site was stratified into 50 equal areas using k-means [73]. These strata were also added to the cLHS as a covariate, to allow for a geographic spread of the sample locations. Due to the use of a subset as a result of data-loss, the data used in this research can be considered a convenience sample based on data quality and availability rather than a cLHS.

2.3. Penetration Resistance (PR)

PR measurements were conducted from the surface using a Penetrologger [74], a cone penetrometer with data logging capabilities. In accordance with the field sampling protocol, these measurements were taken before the soil around the sample location was disturbed or trampled. This was done to avoid unwanted influence on the measurements.

PR was measured six times at each measurement location, allowing us to filter out short-distance variation and reduce measurement uncertainty during data processing. All measurements were taken within 1–2 m of the pit centre, where soil samples for measurements of bulk density were collected (Figure 5). We used a cone with a surface area of 1 cm² and an angle of 60 degrees to measure to 80 cm depth. The penetration speed was manually kept at approximately 2 cm s⁻¹. The Penetrologger measured with a vertical resolution of 1 cm and provided PR at a resolution of 0.01 MPa.

2.4. Dry Bulk Density (BD_d)

Soil pits with a surface area of 100 by 100 cm and a depth of 70 cm were excavated (Figure 5 and Figure 6). Although samples were not collected deeper than 55 cm, the extra depth was required as manoeuvring space for sampling equipment. One side of the pit was chosen as the “clean” side for soil sampling. We did not stand or lean on this side of the pit to avoid unwanted compaction or disruption of the soil during the sampling process. Samples for BD_d were taken following Dutch guidelines [26], which are identical to international guidelines [75]. We sampled the soil at 5, 15, 25, 35, 45 and 55 cm depth (Figure 5) using Kopecky cores with an internal volume of 100 cm³ and a diameter of approximately 5 cm. These cores were taken horizontally in the clean side of the pit, using a hammer to gently tap them into the soil. We took three cores at each depth in order to reduce the effect of short-distance spatial variation.

The undisturbed samples were capped to avoid losing soil, taken to the laboratory and dried in an oven for 48 h at 105 °C following the international guidelines [26,75]. The oven-dry samples were weighed using a scale with a resolution of 0.1 g. The arithmetic mean of the triplets of soil samples was determined for each depth, providing the measured BD_d at 0.001 Mg m⁻³ resolution. As previously stated, only samples from the loamy topsoil were used for further analysis in this research. The boundary between the topsoil and subsoil was clearly visible in the field (Figure 4), and it was noted down which samples were taken in the topsoil. This allowed for easy selection of topsoil samples for analysis.

2.5. Soil Moisture and Clay Content Estimation

Volumetric soil moisture content (θ_v) was measured three times at each sampling depth (Figure 5), using a TDR-based W.E.T. sensor [76]. The arithmetic mean of these three measurements was used to assess the soil moisture conditions at the time of sampling.

We estimated the topsoil clay content for all samples using a single existing auger location from the Dutch national database BIS [27]. These auger locations can be found throughout the Netherlands and include a full soil profile description, including an estimation of the clay content for all soil horizons. In our study, the BIS auger location was approximately 600 m removed from the study site, and it was measured in 2017. This is at a considerable distance from the study site, and it is not the most reliable estimate, as it provides no data on short-distance variation. We decided to still use these data because it is a realistic data source for farmers and other land managers, and the purpose of this research is to evaluate the use of BD_d and PR under realistic conditions. Taking samples for laboratory soil texture analysis at every soil pit would almost certainly have provided more reliable estimates for clay content. However, these detailed analyses would be unlikely to occur in practice due to the high cost.

2.6. Data Pre-Processing

Data pre-processing for BD_d samples and θ_v was done in Microsoft Excel. The arithmetic mean was calculated for all 83 suitable topsoil samples, using the triplets of BD_d cores and θ_v measurements at each sampling depth. Soil moisture conditions are known to have an effect on PR measurements, as moist soils have lower PR values than dry soils [77]. Samples that were taken in either too moist or dry conditions were therefore excluded from the analysis in order to avoid unwanted effects on PR. We determined the mean moisture content of all samples, which was roughly 24%. All samples and measurements that were obtained at soil moisture levels under 19% or over 29% were excluded from further analysis. Using this method, we removed 14 BD_d and PR measurements, leaving a total of 69 measurements for further analysis. The soil moisture data were approximately normally distributed, and removed samples were located at both tails of this distribution.

All PR data were analysed using the R programming language [78]. The mean PR profile from 0 to 80 cm in depth was calculated at each of the 25 sample locations, by taking a measurement depth-based weighted average of the six local replica measurements. This was done to reduce the effects of outliers and short-distance variation. This weighted mean profile was calculated using local polynomial kernel smoothing, available in the R-Package KernSmooth [79,80]. We used a Gaussian kernel with a bandwidth of 1.5. This value was chosen to align the vertical support of the BD_d sample cores with that of the averaged PR measurements, as it meant that the Gaussian function had a width that roughly equalled the soil core diameter (5 cm). This means that the pre-processed measurements of BD_d and PR are both representative for a 5 cm depth interval, centred around the measurement depth.

The BD_d measurements were obtained at specific depths, divided over the 25 sample locations. In order to match the correct PR value to each of the 69 BD_d measurements, we extracted the values at the corresponding depths from the averaged PR profiles. This resulted in 69 paired BD_d and PR measurements.

2.7. State of Soil Compaction Using BD_d and PR

The main analysis involved a comparison of the state of soil compaction estimated from the 69 paired BD_d and PR measurements (Figure 6). Within this context, it was assumed that the BD_d measurements could function as the reference that best approximated the “true” state of compaction.

Each measured BD_d value was converted to a comprehensive classification with two states: (1) compacted or (2) not compacted, using Equation (1) [25,26]. This equation uses the clay content to provide a BD_d threshold above which compaction is deemed harmful to plants. We estimated the topsoil clay content at 20%, using the available BIS data [27]. After applying Equation (1), the BD_d threshold for soil compaction was set at 1.57 Mg m⁻³. Any BD_d values equal to or greater than 1.57 Mg m⁻³ were classified as compacted, whereas samples below the threshold were considered not compacted.

$${\mathrm{Threshold}\mathrm{BD}}_{\mathrm{d}}=1.75-0.009\times \mathrm{percentage}\mathrm{clay}\left[{\mathrm{Mg}\mathrm{m}}^{-3}\right]$$

(1)

Soil compaction was classified for PR using a similar approach, where all 69 measured PR values were classified as either compacted or not compacted. Rather than using a single threshold, as was done for BD_d, multiple candidate thresholds were investigated in order to determine their performance. All values from 0.0 MPa to 4.3 MPa, in increments of 0.1 MPa, were considered as candidates. The classification using PR was therefore run 44 times—once using each candidate threshold. We purposely chose to include more candidates than given in the list of thresholds from the literature (Table 1) as this reduced the risk of missing the true optimum. The upper limit of 4.3 MPa was chosen because it was the highest PR value found in the dataset. It is theoretically possible to measure up to roughly 8.0 MPa using a hand-driven Penetrologger, but such high values were not found in the dataset.

2.8. Confusion Matrices

In order to determine the level of agreement between BD_d and PR soil compaction classifications, all paired classifications were compared to each other using BD_d as the “true” reference. As there are 44 different sets of classifications for the candidate PR thresholds, the comparison to the reference BD_d classification was done 44 times. The results were summarised in 44 separate confusion matrices, which assigned measurement pairs to one of four categories: (1) true positive, (2) true negative, (3) false positive and (4) false negative (Figure 5). This method allows for a straightforward assessment of the performance of each candidate PR threshold.

The first two “true” categories contain all pairs for which both methods are in agreement, with true positives containing compacted pairs and true negatives containing not-compacted pairs. In essence, this represents the instances for which PR provided the same classification as BD_d. The latter two “false” categories contain pairs for which PR did not provide the same classification. The category of false positives corresponds to the statistical type I error. This occurs when PR classification indicated soil compaction was present, but the BD_d reference states there is none. The false negatives correspond to the type II error, where the PR classification wrongly concludes the opposite—that there is no soil compaction.

2.9. Accuracy Metrics

The overall performance of each candidate PR threshold was also obtained from the confusion matrices through the use of accuracy metrics. For this research, we calculated the accuracy, sensitivity, specificity and balanced accuracy for all candidate PR thresholds (defined in

Table 2. Accuracy measures used to evaluate the performance of PR thresholds. N_TP = number of cases of true positives, N_TN = number of cases of true negatives, N_FP = number of cases of false positives and N_FN = number of cases of false negatives.

Metric	Formula	Interpretation
Accuracy	$\mathrm{Acc}=\frac{{\mathrm{N}}_{\mathrm{TP}}+{\mathrm{N}}_{\mathrm{TN}}}{{\mathrm{N}}_{\mathrm{TP}}+{\mathrm{N}}_{\mathrm{TN}}+{\mathrm{N}}_{\mathrm{FP}}+{\mathrm{N}}_{\mathrm{FN}}}$	Ratio between number of correctly classified samples and total number of samples
Sensitivity	$\mathrm{Sens}=\frac{{\mathrm{N}}_{\mathrm{TP}}}{{\mathrm{N}}_{\mathrm{TP}}+{\mathrm{N}}_{\mathrm{FN}}}$	Ratio of number of correctly classified positive samples to total number of positive samples
Specificity	$\mathrm{Spec}=\frac{{\mathrm{N}}_{\mathrm{TN}}}{{\mathrm{N}}_{\mathrm{TN}}+{\mathrm{N}}_{\mathrm{FP}}}$	Ratio of number of correctly classified negative samples to total number of negative samples
Balanced accuracy	$\mathrm{BA}=\frac{\mathrm{Sens}+\mathrm{Spec}}{2}$	Accuracy measure corrected for imbalance in the dataset

). This metric is also used under the assumption that BD_d can be used as a true reference. The accuracy provides a direct indication of the overall “correctness” of PR as a measure to classify soil compaction, compared to the reference classification using BD_d. It is calculated by taking the ratio of true positives and true negatives to the total number of sample pairs (Table 2).

The sensitivity and specificity provide more in-depth insights. They were calculated using the corresponding formulas from Table 2. Within the context of this research, sensitivity provides the success rate for correctly classifying compacted samples. In essence, it determines how well PR performs at finding the soils which are compacted from all measurement pairs. Specificity does the opposite, as it provides the success rate for samples that are not compacted. The added benefit of calculating the sensitivity and specificity is that it allows for an assessment of bias.

Although the accuracy is a straightforward metric, it is sensitive to data imbalance. This is especially the case in datasets where one class or classification is more abundant than another. We therefore also calculated the balanced accuracy, which is the arithmetic mean of the sensitivity and specificity (Table 2). This metric is unaffected by data imbalance. A more elaborate discussion on the use of accuracy metrics and their use given data balance is given by Tharwat [81].

2.10. Misclassification Costs: Formula

Expected misclassification costs were calculated in order to show the trends in financial effects of misclassification, which could be directly experienced by farmers. These include costs related to yield losses as a result of false negatives and tillage costs as a result of false positives. The key concept of misclassification cost is that measurements such as PR are used to inform choices regarding land management. Misclassification can then lead to unnecessary costs that would not have occurred if the measurements were perfect.

The expected misclassification costs were determined under the previously used assumption that BD_d can be used as a reference of the true state of compaction. There were three further main assumptions: (1) exceeding the BD_d threshold always leads to yield losses due to (over)compaction, (2) there are no yield losses due to soil compaction when the BD_d threshold is not exceeded, and (3) alleviating soil compaction has an associated cost. Thus, the misclassification cost of a false negative is equal to the expected yield loss. False positives result in unneeded alleviation, which results in associated costs without yield gain.

A straightforward method was used to calculate the expected misclassification costs for each candidate PR threshold value. The total expected cost was dependent on the probability of misclassification and the expected cost per misclassification (Equation (2)). The probabilities for false negatives (P_FN) and false positives (P_FP) were calculated from the 44 confusion matrices by taking the ratios of false negatives and false positives to the total number of sample pairs.

$$\mathrm{Expected}\mathrm{misclass}.\mathrm{cost}={\mathrm{P}}_{\mathrm{FN}}\times {\mathrm{Cos}\mathrm{t}}_{\mathrm{FN}}+{\mathrm{P}}_{\mathrm{FP}}\times {\mathrm{Cos}\mathrm{t}}_{\mathrm{FP}}\left[{€\mathrm{ha}}^{-1}{\mathrm{y}}^{-1}\right]$$

(2)

2.11. False Negative Costs

False negative costs (Cost_FN) were estimated from data on Dutch agricultural yields and estimated percentual revenue loss due to soil compaction. Crop yields and prices for seed potatoes, onions, sugar beets and wheat were obtained in the from the Dutch BINternet-database [82]. From this, the mean national crop revenues (€ ha⁻¹ y⁻¹) over the period from 2010 to 2019 were calculated (

Table 3. Estimated revenue and revenue loss for four crops that are commonly grown in the study area (calculated using data from: Wageningen Economic Research [82]).

Crop	Revenue from Crop (€ ha−1 y−1) Mean over 2010–2019	Estimated Revenue Loss of 10% in Case of Soil Compaction (€ ha−1 y−1)
Seed potatoes	10.355	1036
Onions	6.744	674
Sugar beets	3.337	334
Wheat	1.641	164

). The used crops were chosen because they are prevalent in the region of the study area. Additionally, they cover a wide range of revenues, allowing us to evaluate the trends for low, medium and high-value crops.

The percentual yield loss was estimated from the literature. Yield losses can range from of 5% to 40% [13,14], depending on the severity of the compaction. Therefore, we estimate that a reasonable yield loss is 10%. This is high enough that farmers will experience effects but low enough that it might not be evident without measurements. It also represents a situation where soil compaction might already be present long-term and alleviation is required. The false negative cost was therefore set at 10% of the mean crop revenue (

Table 4. Yearly false positive and false negative cost, per occurrence of a misclassification.

Crop	Cost of a False Positive (€ ha−1 y−1)	Cost of a False Negative (€ ha−1 y−1)
Seed potatoes	380	1036
Onions	260	674
Sugar beets	146	334
Wheat	90	164

). The cost for yield loss was kept the same for all crops.

2.12. False Positive Costs

The costs for a false positive are mainly driven by the costs for alleviation. These costs are difficult to assess accurately, as they depend on the type of alleviation and market dynamics (e.g., cost of fuel, cost of labour, availability of machinery). We focused on the general costs of subsoiling tillage, such as ripping and ploughing, as these are commonly used in the Netherlands. For the Netherlands, we estimated the cost of subsoiling at roughly 105 EUR/ha⁻¹. This value is an average of the estimated costs for ripping and ploughing from De Wolf et al. [83].

Although subsoiling tillage is meant to alleviate soil compaction and its effects, it also leaves the loose soil vulnerable to re-compaction. Studies show that the positive effects of subsoiling in compacted soils are negated after 2–2.5 years [84,85]. This means that it is likely that subsoiling due to a false positive measurement will actually result in soil compaction, provided the soil is used intensively. This re-emergence of soil compaction will likely lead to the re-introduction of yield losses in the years following alleviation. A common approach is to subsoil periodically in order to alleviate re-emerging soil compaction.

The yearly estimated false positive costs were therefore calculated as a combination of the subsoiling tillage costs and the yield losses due to the re-emergence of soil compaction. We assumed that subsoiling would be required every three years, meaning the yearly costs for the subsoiling itself amounted to 35 EUR/ha⁻¹. For the re-emergence of soil compaction, we assumed a linear trend over the course of three years up to the full 10% of yield loss just before periodic subsoiling. This results in a mean expected yearly yield loss of 3.33% as a result of “unnecessary subsoiling”. The total yearly false positive misclassification cost therefore amounted to the subsoiling costs of 35 EUR/ha⁻¹, plus a yearly yield loss of 3.33% of the potential mean crop revenue (Table 4). For example, for seed potatoes, this amounts to 35 + 0.0333 × 10,355 = 380 EUR/ha⁻¹ y⁻¹.

3. Results

3.1. Confusion Matrices

Figure 7 shows the results of the comparison between soil compaction classification using PR and BD_d. Whereas a confusion matrix is usually visualised using a table with four quadrants, this figure represents all 44 confusion matrices using a stacked bar plot.

The confusion matrices show a clear difference between different threshold values. At low thresholds, the matrices are dominated by true positives and false negatives. There is a slow decline of true positive cases from 0.4 MPa until 4.2 MPa and a corresponding increase in false negatives. This implies soil compaction was present at PR values ranging from 0.4 MPa to 4.2 MPa, following the comparison to the BD_d measurements.

The true negative cases show a similar trend, as they increase in number between 0.5 MPa and 4.1 MPa. False positives decrease in number over this range. However, we see that the rate of change is much higher between 0.5 MPa and 2.0 MPa. This implies that a large proportion of the not-compacted samples have a PR value below 2.0 MPa, with a smaller proportion having higher PR values.

Overall, these data show a logical pattern. As the PR threshold increases, more uncompacted sites are correctly classified (true negatives) at the cost of an increase in misclassified compacted sites (false negatives). There is some data imbalance at the extremes, with close to 20% true positives at 0.0 MPa and over 80% true negatives at 4.3 MPa.

3.2. Accuracy Metrics

Accuracy metrics for all 44 confusion matrices are provided in Figure 7 and Figure 8, and the optima from these figures are summarised in

Table 5. Summary of the accuracy metrics from Figure 8, showing optima and the corresponding (range of) PR threshold values.

Metric	Optimum Value (%)	Threshold at Which the Optimum Is Reached (MPa)
Accuracy	84.1	3.0
Sensitivity	100	0.0–0.4
Specificity	100	4.3
Balanced accuracy	76.0	1.8–1.9
Sensitivity (at max. balanced accuracy)	76.9	1.8–1.9
Specificity (at max. balanced accuracy)	75.0	1.8–1.9

. The maximum accuracy is 84.1%, which occurs at 3.0 MPa. The accuracy is lowest at low PR thresholds and rises quickly between 0.5 and 1.0 MPa. We see a continued increase from 1.0 to 2.6 MPa, albeit at a slower rate. The peak at 3.0 MPa is part of a relative plateau in accuracy, showing similar performance from 2.6 all the way to 4.3 MPa. At 4.3 MPa, the classification using PR considers all samples to be not compacted.

Sensitivity and specificity confirm the observations from the previous paragraph. Sensitivity always decreaseswith higher PR thresholds, whereas specificity always increases. Setting the threshold at zero classifies all samples as compacted, whereas a threshold at 4.3 MPa results in only not-compacted classifications. Sensitivity shows a jagged decrease from 0.5 MPa up to 4.3 MPa, indicating that there are compacted samples in that range. This jagged behaviour is likely due to the small sample size. Specificity rises quickly between 0.4 MPa and 1.0 MPa and continues to rise at a slower rate until 4.2 MPa.

The balanced accuracy provides the optimal point where trade-offs between sensitivity and specificity result in the highest mean accuracy. In Figure 9, the balanced accuracy shows a plateau roughly between 1.0 and 3.0 MPa at around 70% accuracy. The highest balanced accuracy is found at 1.8 and 1.9 MPa, at 76.0%. Sensitivity and specificity are at approximately similar levels, namely 76.9% and 75.0% respectively (Table 5). These results show that there is a relatively wide range in which balanced accuracy is relatively close to the optimum, although the ratio of sensitivity to specificity changes depends on the threshold. The optimum at 1.8 and 1.9 MPa seems to be an unbiased optimum, since both sensitivity and specificity are at similar levels. This is lower than the biased optimum using the unbalanced accuracy. Figure 10 shows a scatter plot of the raw BD_d and PR data, classified according to the confusion matrix using the PR threshold of 1.9 MPa.

3.3. Cost Functions

The expected misclassification costs for seed potatoes, onions, sugar beets and wheat are given in Figure 11 and

Table 6. Summary of misclassification costs from Figure 9, showing optima and values at the extremes.

Crop	Cost at 0.0 MPa (€ ha−1 y−1)	Cost at 4.3 MPa (€ ha−1 y−1)	Optimal Cost (€ ha−1 y−1)	Threshold at Optimal Cost (MPa)
Seed potatoes	308	195	122	1.8–1.9
Onions	211	127	82	1.8–1.9
Sugar beets	118	63	44	2.6
Wheat	73	31	23	2.6

. These show that the expected misclassification cost is highest for seed potatoes and lowest for wheat. This is a logical result, as seed potatoes are the more valuable crop. All crops show the same general pattern, with the highest expected misclassification cost at very low PR thresholds. Costs drop with increasing PR thresholds, reaching a minimum at 1.8 and 1.9 MPa for seed potatoes and onions, and at 2.6 MPa for sugar beets and wheat. When the threshold is increased further, the expected costs rise again until the maximum measured PR is reached at 4.3 MPa.

It is also visible that the cost function flattens for lower revenue crops such as wheat. Where the seed potato cost function still shows a clear dip between 0.5 and 3.0 MPa, wheat shows a relatively flat line from 1.0 MPa onwards. There is still an optimum at 2.6 MPa, but the expected costs do not increase much when the PR threshold is increased again. We also see that the cost near 0.0 MPa, where there are many false positives, is always higher than towards 8.0 MPa, where false negatives dominate.

4. Discussion

4.1. Interpretation of Results

Using a confusion matrix and accuracy metrics can be a simple yet effective way to evaluate the level of agreement between two measurement methods. When we look at the accuracy results in Figure 8 and Table 5, we see a maximum attainable accuracy of 84.1% at 3.0 MPa for the data set used in this study. There also appears to be a plateau of high accuracy extending from roughly 2.5 to 4.3 MPa. The highest attainable accuracy at 3.0 MPa would statistically result in misclassification in around only one out of every nine instances if PR were used instead of BD_d. This threshold also fits with the use of PR thresholds in a number of other studies [53,55,57,60,61]. However, these results are misleading due to the effects of data imbalance.

The confusion matrices can be used to provide context for our accuracy results and to illustrate the effects of data imbalance (Figure 7). Ideally, the accuracy of PR classification is always high regardless of whether soil compaction is actually present or not. The comparison to BD_d can become unreliable if the accuracy is very high for one class (e.g., compacted) and low for another (e.g., not compacted). The results in Figure 7 show that the plateau of high accuracy from 2.5 to 4.3 MPa is dominated by true negatives and false negatives. True or false positives are virtually absent. In other words, nearly all datapoints are uniformly classified as not compacted using PR, regardless of their BD_d classification.

This uniform classification can be problematic and is the main reason for the misleading nature of the accuracy results. It stems from imbalance in the data themselves. Figure 10 shows that over 80% of samples are below the BD_d threshold of 1.57 Mg m⁻³, meaning there is very little soil compaction present in the dataset. In this situation, high accuracy and a high level of agreement between BD_d and PR classifications can be achieved simply by labelling all PR samples as not compacted. This results in a dataset of PR classifications that completely misses all samples that were in fact compacted, but these do not carry as much weight since they only make up 20% of the total dataset. This is a cheap and easy way to achieve over 80% accuracy for this specific study area, but it does not provide an informative general assessment of the performance of PR.

The balanced accuracy (Figure 9) provides an estimation of the expected level of agreement, which is more informative as it compensates for data imbalance. The optimum of 76.0% at 1.8 and 1.9 MPa is unbiased, as the sensitivity and specificity are nearly equal to each other. These optimal thresholds are very close to 2.0 MPa—the threshold value that is most commonly used in literature [30,32,34,36,37,38,40,44,46,49,50]. We also see a plateau from 1.0 to 3.0 MPa, where the balanced accuracy is around 70% and close to the optimum. This suggests that deviating slightly from the true optimum does not result in a drastic or quick decrease in balanced accuracy. It also suggests that using either BD_d or PR will statistically result in a difference in classification result in just under 3 out of every 10 cases, which can improve to around 1 out of every 4 cases at the optimum.

Figure 9 indicates there is a clear trade-off between sensitivity and specificity. Lower thresholds of PR tend to create better agreement with BD_d classifications if locations are compacted, but at the cost of worse agreement in those locations where compaction is absent. Conversely, a higher PR threshold results in better agreement if there is no compaction and worse agreement if there is compaction.

The results from our misclassification cost analysis are in line with those for balanced accuracy, as the optima are located at 1.8, 1.9 and 2.6 MPa (Figure 11). In this analysis, the lower value crops seem to favour a higher optimum PR threshold. The misclassification cost curves also seem to flatten out with a lower crop value. This is likely a result of the ratio between the false positive and false negative costs used in Equation (2). As crop value decreases, the absolute expected yield loss as a result of a false negative also decreases. This is only partly the case for false positive cost, as it always contains a base level of 35 EUR/ha⁻¹ y⁻¹ in subsoiling costs, which does not change. When the misclassification costs converge to the same value, the potential gains of finding the optimal threshold become less pronounced and the curve flattens.

From Figure 11, it quickly becomes evident that the expected misclassification cost of setting the threshold very low (i.e., near 0.0 MPa) is much higher than that of setting it too high (i.e., at 4.3 MPa). This is quite unintuitive when taking into account the misclassification costs in Table 4, which show that the per occurrence cost of a false negative should be more than twice as expensive compared to that of a false positive. Since the costs near 0.0 MPa are mostly due to false positives, one would expect the costs to actually be lower compared to the costs at 4.3 MPa.

Looking at Equation (2), the cause for this must lie in the probabilities obtained from the data. Apparently, the probability of having a false positive is sufficiently larger than that of having a false negative. This is likely a result of the data imbalance shown in Figure 10, which already showed that over 80% of the samples in the dataset lie below the 1.57 Mg m³ BD_d threshold. Since BD_d is used as the “true” reference, this has an effect on the probability of having soil compaction in the first place. There is a higher a priori probability of “drawing” an uncompacted site simply because there are so many in the dataset. On the other hand, the a priori probability for a compacted site is low because there are so few. The expected costs near 0.0 MPa are so high because the probability of having a misclassification is high, which overpowers the lower costs per occurrence. It is therefore likely a very dataset and study site-specific phenomenon.

It is also possible to illustrate the effect on the farm-level using our results, as farmers usually have multiple hectares of arable land. The average farm size in Flevoland is 50 ha [86]. If farmers and land managers use the PR threshold with the highest level of agreement, they can get the lowest expected misclassification costs. These calculations are indicative only, but they can show the potential effects of picking a suboptimal PR threshold. For seed potatoes, the maximum potential yield is 518,000 EUR, and the optimal expected misclassification costs are 6100 EUR. If PR is used in the least optimal way by setting it close to 0.0 MPa, the misclassification costs could be as high as 15,400 EUR. This is not a logical threshold to use, but it does show the potential for cost savings at the extremes. A more logical example is the use of 1.5 MPa, 2.5 MPa or 3.0 MPa as a threshold, as these are mentioned in scientific literature (Table 1). For a farmer in our study area, these thresholds would result in costs of 6300 EUR, 6500 EUR and 7300 EUR, respectively. This implies that a seed potato farmer in Flevoland can save 200 to 1200 EUR by using the optimal PR threshold value. These savings will reduce with lower yield crops, of course.

4.2. Broader Perspective

Using either BD_d or PR to approximate the level of soil compaction is not new, as is evident from Table 1 and the literature cited in the introduction. Attempts have been made to determine the relation of the measurements themselves to each other, with varying results [66,67,68,69]. Our research did not aim to repeat these studies, focusing more on classifications rather than the numerical values of the measurements themselves. It also aimed to acknowledge the fact that both methods are often used in isolation, where the outcome can sometimes depend partially on the used method rather than the actual state of the soil.

It is cheaper and quicker to use a Penetrologger to measure the soil strength (PR) and derive the state of soil compaction from that rather than measuring the density (BD_d). The results for our study show that in the most optimal scenario, nearly one in five measurements is likely classified differently if PR is used instead of BD_d. This can lead to noticeable differences between results of both methods. Because PR is cheaper and quicker, it can allow for more measurements to be conducted within the same time frame, including duplicate measurements. This could allow farmers and other land managers to have a better spread of measurements over fields, at the cost of having a proportion of misclassifications compared to BD_d. Therefore, there may be a trade-off between accuracy, cost and practical usability.

PR is often used to approximate soil compaction, as is evident from Table 1. These thresholds can be assessed within the context of the results from our study, as any of them could be chosen by the owner of the field in our study area. As previously mentioned, the optimum level of agreement that we found is very close to the most commonly used PR threshold of 2.0 MPa (Table 1). We also see that nearly all of the studies from Table 1 use thresholds that are within our plateau of good agreement between PR and BD_d, with 4.2 MPa by Stock et al. [54] being the only exception. This means that many of the thresholds in literature would have resulted in a level of agreement that was very close to the optimum. Interestingly, thresholds between 1.0 and 2.0 MPa would still have performed reasonably well in our study area but were not found in the literature.

In this research, we used PR to make a classification into two categories: compacted and not compacted. This is the most commonly used method in the literature, and it matches the way in which BD_d classifications are made. However, some studies also use a non-binary system [55,57,60,61]. In such a system, there are also intermediate classes such as “lightly compacted” or “moderately compacted”. This is also the preferred method for the use of PR in some Dutch studies [87], who consider the range of 2.0 MPa to 3.0 MPa to be “somewhat compacted”. We were unable to directly assess this non-binary method, as it was incompatible with the binary BD_d classification. It is notable that this system of intermediate classes does overlap with our plateau of high balanced accuracy (Figure 9). We also think that using more than two classes might approximate the true state of the soil better, as soil properties can also change gradually.

It is possible that using more than two classes may result in lower expected misclassification costs, as it allows farmers to tailor their soil management more towards the gradual nature of soil properties. This can mean doing only light remediation on lightly compacted areas, such as less intensive tillage and using more deep-rooting crops. Disruptive and expensive land management options are then only used on heavily compacted areas. This may reduce the added management cost on light soil compaction and therefore also the expected misclassification cost for those areas. The effect of this depends on the exact difference in costs for soil management, an extensive investigation of which was beyond the scope of this study. The results from this study are therefore representative for a situation in which “binary” land management takes place—either there is full alleviation, or no alleviation.

Our analyses were done on the point measurement-level. However, farmers and other land managers will realistically make management decisions for larger areas of land such as individual fields. This requires a translation from point measurements to areas and subsequently to management. The advent of precision and smart farming aims to facilitate in this decision-making process, but technical and social hurdles still exist that inhibit its full use [88,89]. This means that the gap between point measurements and management policy for a field needs to be bridged by the farmers or land managers themselves. This is often done based on experience, personal conviction, advice from experts, practical considerations and many other factors [90]. Some farmers may also decide to specify management for sections of a field, where others apply do not make this within-field distinction. As a result, the way in which measurements are used is likely to be highly personal and variable. The results presented in this study can therefore be used to provide guidelines and insight for the measurements themselves but become more uncertain when the spatial heterogeneity needs to be taken into account.

4.3. Limitations and Future Research

The results in this study were obtained at our study area in Flevoland, the Netherlands. It must be stressed that these specific results are likely to be strongly tied to the conditions in the study area. The concept of misclassification and associated costs is likely to also have an effect in other locations, but the attainable level of agreement and relevant PR threshold range may deviate from those found in this study. There is likely to be an influence of soil texture, which can contribute to the location-specific nature of the relationship between BD_d and PR. This influence is hinted to in studies where pedotransfer functions are used to describe the relationship between BD_d and PR, which include clay or sand content as a covariate [66,67]. Bengough and Mullins [28] also hint at a combined soil moisture and texture effect. This is especially relevant taking into account the approach in this study, where soil texture and clay content were estimated from a single auger site and both horizontal and vertical heterogeneity were not taken into account. It is possible to include data on soil texture heterogeneity and determine its effects on the BD_d threshold. However, doing so would introduce data that farmers and other land managers will not always have. Using simplifications that are in line with common practices therefore strengthened the practical usability of this research. Future research should be used to verify the results from our study in similar soils. Studies on different soils can be used to determine the effect of differences in soil texture on the level of agreement and relevant PR thresholds.

We also mentioned that data imbalance has an effect on the (general) accuracy and the expected misclassification cost. Studies have shown that roughly half of the arable land in the Netherlands is at risk of being compacted and that this risk depends on soil type [16,91]. A follow-up study by Brus and Van den Akker [15] indicates that 43% of the country is actually likely to be compacted, although the data density in Flevoland is low. Fields in low-risk areas may have a strong data imbalance towards uncompacted sites, whereas fields in high-risk areas might show a dominance of compacted sites. Due to the nature of using a single case study, we could not explore the effects of all possible data compositions.

It is possible to hypothesize what the effects of data imbalance and composition may be. We expect that the balanced accuracy is least affected, as it is designed to compensate for imbalance. The general accuracy is sensitive to data imbalance, but its effects can easily be assessed by comparing it to the balanced accuracy. We expect that the misclassification costs are also strongly affected, as it tends to assign a higher expected cost to the more probable class. It is possible to analyse these effects by artificially creating different data balances, either through resampling of the data or modification of the cost function. Our small dataset size prevented us from doing this analysis, as it would be too prone to overfitting. The robustness of our methods and results can also be assessed by performing multiple other case studies, which should naturally have different data balances, soil types and local conditions. We recommend that these case-studies are done in future research.

Finally, a critical reflection on the underlying assumptions for the analysis is also required. In order to determine the expected misclassification costs, we implicitly assumed that the BD_d thresholds are directly linked to yield loss and that these thresholds are well-defined. Under that assumption, yield loss would always occur if the BD_d were above the calculated threshold and never if it were below the threshold. This is a very practical assumption, but it is likely not fully true. The BD_d threshold is sensitive to clay content, meaning it depends on the accuracy of the soil texture estimation or measurement. Soil compaction is also related to soil properties which are not captured in BD_d, such as pore connectivity and pore size distribution. It is a gradual phenomenon, meaning that soil compaction and its negative effects do not instantly occur at a certain threshold. This is not captured in a single threshold, as it assumes a clearly definable division between classes. More research into the possibility of using a gradual system of thresholds is therefore needed.

Similar assumptions were made for yield losses and crop value, for which we provide set values from the literature. In reality, yield losses can vary from roughly 5% to 40% depending on the level of soil compaction and the crops being grown [14]. Crop value changes based on location, time and other market dynamics. These two factors can have an effect on the misclassification function and may lead to a shift of the optima. We strongly recommend research and an evaluation of the effects of these assumptions.

5. Conclusions

The analysis in this research shows that, in our study area, classifications of soil compaction on the same soil using BD_d and PR result in a maximum level of agreement of 76%. This effectively means that around one in every four classifications differs between both methods. In a wider range of thresholds from 1.0 to 3.0 MPa, the level of agreement lies around 70%. Most thresholds from the literature are found in this range and would therefore have led to just under one misclassification in every three classifications, which is very close to the optimum.

If the farmer or land manager in the study area for this research opts for measurements using Penetrologgers instead of soil cores, they can expect between around one in four or five of the performed measurements to provide a different classification. This is a substantial number, which shows that choosing a measurement tool can have implications for a soil survey and the subsequent management. The added advantage of being able to conduct more measurements and add duplicate measurements was not investigated, but may theoretically compensate slightly for the misclassification chance.

The effects of misclassification on the expected cost to a farmer show a clear relation to the level of agreement, as the expected cost is lower in the range of 1.0 to 3.0 MPa. There is a clear effect of crop value on the cost function, showing that lower value crops such as wheat and sugar beets are less sensitive to the exact PR threshold if the aim is to keep costs low. Higher value crops such as potatoes and onions do show a clearer effect of deviations in the threshold.

These results are sensitive to data imbalance and are therefore very study area-specific. In order to create a more general overview of the level of agreement between BD_d and PR-based classifications of soil compaction, a larger number of study sites in a wider range of soil types and compositions should be investigated. We also hypothesise that there are many ways in which a farmer or land manager can improve the accuracy of soil compaction classification using PR. This includes sampling to have sufficient data, capturing the spatial soil compaction patterns and tailoring soil management to those patterns and using additional information from observations and experience to evaluate PR measurements.