Data Analytics in Agriculture: Enhancing Decision-Making for Crop Yield Optimization and Sustainable Practices

Abstract

Collaboration across the agriculture supply chain is essential to address the high-yield demand and sustainable practices amid global overpopulation. Limited resources, such as soil and water, are compromised by excessive chemical agents and nutrient use. The Internet of Things (IoT) and smart farming offer solutions by optimizing agent applications, data analysis, and farm monitoring. Evidence from numerous studies indicates that collaboration in the supply chain, including farmers, can improve efficiency and productivity, reduce costs, and enhance crop quality. This paper investigates the transformation of traditional agriculture into smart farming through the integration of IoT technology and community partnerships. It presents a case study focused on educating farm owners about advanced technologies to enhance decision-making, improve crop yields, and promote sustainability. Additionally, the paper highlights the role of data analytics in agriculture. Farmers in the southern region of Zagreb, Croatia, were trained on the use of sensors and yield monitoring. Small farms in that region face challenges in improving yields due to limited capacity and lack of entrepreneurial experience. The DMAIC methodology was employed to address these issues and measure relevant parameters. The paper also discusses consistent patterns between electrical conductivity (EC) measurements and potassium levels in soil. It explains the potential of estimating potassium concentrations based on EC readings, or vice versa. Leveraging EC as a proxy for potassium levels could offer a cost-effective means of assessing soil fertility and nutrient dynamics. Additionally, Principal Component Analysis (PCA) biplot analysis is presented, showing that pH values behaved independently. Understanding these dynamics enhances knowledge of soil variability and informs sustainable soil management practices.

1. Introduction

The digital revolution in agriculture, often referred to as “smart farming” or “smart agriculture”, integrates advanced digital tools and technologies into traditional farming practices. The precise origin of the terms “smart farming” and “smart agriculture” is challenging to trace, as they likely emerged from the collective contributions of researchers and industry experts. However, the literature suggests that precision farming technologies began to be used in the late 1980s in the United States and Australia [1].

This innovative approach leverages Artificial Intelligence (AI), automation, and the Internet of Things (IoT) to optimize crop production and enhance sustainability. AI algorithms process data from various sources to assist farmers in making informed decisions, while automated devices streamline planting, irrigation, and harvesting. IoT devices collect real-time data from sensors, drones, and other connected equipment, boosting operational efficiency [2,3,4,5,6].

Sensor technology is the backbone of smart farming, enabling real-time monitoring of farming parameters. Adamchuk et al. [7] addressed the role of soil sensors in measuring soil moisture, pH, temperature, and nutrient levels. These sensors provide data that farmers can use to make decisions about irrigation, fertilization, and other management practices, thus enhancing crop yields and sustainability.

Shafi et al. [8] proposed transforming the agriculture sector through IoT-based automation for crop monitoring. Their solution includes a wireless sensor network (WSN) for real-time monitoring of crop health and a remote sensing platform for obtaining multispectral imagery to classify healthy and unhealthy crops. Their results demonstrated enhanced agricultural practices through automation and data-driven decision-making, leading to increased production with reduced human effort. Singh et al. [9] highlighted the importance of utilizing remote sensing for efficient irrigation to improve water use efficiency in agriculture. Unmanned aerial vehicles (UAVs), as remote sensing tools, provide aerial images that can detect issues early and apply precise interventions, optimizing resources and improving crop management [10].

To monitor and optimize key environmental and soil parameters, the most important parameters are nitrogen (N), phosphorus (P), potassium (K), humidity, and temperature. N is essential for plant growth, encouraging vegetation development and leaf growth. P plays a crucial role in the energy level of plants, impacting root development, flowering, and fruiting. K enhances the resilience of plants to adapt to changes by controlling photosynthesis, protein synthesis, and nutrient uptake. Balanced NPK values greatly impact healthy growth, resilience to stress, and crop yield [11]. Weddell [12] discussed a precision management strategy focusing on the variable application of nitrogen fertilizer.

Chen et al. [13] investigated integrating soil-crop system management in China to enhance grain yield for rice, wheat, and maize while reducing environmental impacts. Their study revealed the possibility of meeting food demands sustainably by adopting precision management systems widely, ensuring food security and reducing agriculture’s environmental footprint. Ragul et al. [14] proposed an automated irrigation system for efficient water management and intruder detection, incorporating sensors for pH, soil moisture, and humidity measurements. This system empowers farmers to remotely access field information, reducing manual labor and time while improving productivity and resource utilization. Communication with the farmer is facilitated through a Global System for Mobile communication (GSM) module, enabling real-time updates on field conditions via Short Message Service (SMS).

Recently, Murali et al. [15] implemented an IoT-enabled soil nutrient system to assist farmers in making informed decisions throughout the farming process. Their model incorporates sensors, cloud computing, and machine learning (ML) tools to collect real-time data on soil conditions and analyze nutrient levels. The ML models could recommend suitable crops while minimizing fertilizer usage for enhanced productivity. According to the authors, the ML proposed model achieved high accuracy in nutrient classification and crop recommendation, providing farmers with a cost-effective and efficient solution for soil management and crop selection.

Another study [16] found that the use of digital platforms and tools can help to improve communication and coordination among supply chain partners, leading to increased efficiency, effectiveness, and productivity. Overall, the literature suggests that collaboration among various stakeholders in the farming supply chain can lead to many benefits, including increased efficiency and productivity, reduced costs, and improved product quality.

By 2050, it is urgent to produce 85% more food to feed an estimated global population of eight billion people, despite the challenges posed by scarce natural resources, environmental degradation, and climate change impacts [17]. Traditional farming, especially on small family-owned farms, must adopt the above-mentioned innovative technologies to meet these demands [18]. By optimizing resource usage, farmers reduce wastage and minimize environmental impacts [19]. Data-driven insights help farmers understand the relationship between natural systems, which is crucial for achieving both sustainability and food security [20]. Building data science capacity among agricultural producers is also essential, requiring training programs and educational initiatives to empower stakeholders in utilizing data and collaborating effectively to improve their farming practices [21]. In the context of decision-making, it is essential to evaluate the accuracy of the data and understand the factors that influence it. Therefore, it is insufficient to merely collect data; it must be collected, cleaned, structured, and prepared for analysis. “The information that crops offer is turned into profitable decisions only when efficiently managed” [2].

This paper investigates the outcomes of collaborative efforts to transform traditional agriculture practices through the integration of IoT technology and community partnerships. The primary objective is to present a case study that educates farmers on smart farming to optimize decision-making for crop yield enhancement and sustainable farming practices. Additionally, the paper demonstrates the application of data analytics in agriculture. Ultimately, this research aims to empower farmers who are managers and sole decision makers in their operations [22] to become proficient in administering data collection and monitoring. Specifically, this project focuses on the collaboration between small family farms in the southern region of Zagreb, Croatia, and academic institutions. This research involves five farms, four of which produce organic food, while one farmer specializes in cacti. All farms employ traditional methods but face challenges such as low production quantities, irregular supply chain operations, and a lack of entrepreneurial experience. Climate change aggravates these issues, leading to increased drought effects and water scarcity, further obstructing productivity. Through this partnership, one farm owner was trained to use sensors for yield monitoring, while other farmers gained insights for future implementation. By fostering collaboration among stakeholders in the agricultural supply chain, from farmers to distributors, this paper illustrates the potential of smart farming to enhance efficiency, cost-effectiveness, and sustainability. The interdisciplinary approach and industrial partnerships enhance this research’s impact, scalability, and educational benefits.

2. Materials and Methods

The central focus of this research is the deployment of advanced sensor technology to monitor critical environmental and soil parameters that affect crop yields. Furthermore, the researchers successfully raised awareness of smart farming practices among small family farms in the region. This section details the materials utilized and the methodologies employed to achieve the research objectives.

2.1. Sensors Selection and Implementations

To facilitate the farmer’s interaction with the technology, a single piece of equipment integrating multiple sensors was selected. This approach precluded the need for the farmer to implement four distinct sensors. The sensor selected is IP68 ABS Soil NPK Sensor Taidacent, B08MXXSP59, Guangzhou, China [23], High Precision Soil Nutrient Intelligent Fertilizer Detector Tester Meter NPK Sensor. Its specifications are as follows. Measuring Range: 0–1999 mg/kg—Working Humidity: 5 to 95% (relative humidity), no condensation—Measurement Accuracy: ±2%F. s—Resolution: 1 mg/kg (mg/L)—Working Temperature: 5 to 45° C—Power Supply: 12 V–24 V DC. Figure 1a presents a detailed close-up view of the sensor. The sensor was installed appropriately within the field of the small family farm in the southern region of Zagreb, Croatia, to prevent interaction with other plants or their data (Figure 1b).

The sensor system was developed using the ESP32 microcontroller (SIMAC Electronics HANDEL GmbH, Neukirchen-Vluyn, Germany), which serves as the central processing unit controlling the system. It incorporates the NPK 5-pin sensor to measure soil parameters such as potassium, phosphorus, humidity, temperature, and pH levels. The system is equipped with Wi-Fi connectivity for real-time data transmission and includes local data logging onto an SD card to ensure data backup in case of Wi-Fi instability. Figure 2 presents a schematic diagram of the system circuit (Figure 2a) and the actual system installed on the farm (Figure 2b).

The methodology was tailored specifically for growing spring onions, ensuring the monitored parameters align with the optimal conditions for this crop. The following ideal conditions were considered [24]:

Temperature: 15–21 degrees Celsius, pH Level: 5.3 to 5.8, Moisture Level: Should be moist but not too wet (50–60%), Nitrogen Level: 10–50 mg/kg, Phosphorus Level: 5–25 mg/kg, Potassium Level: 10–40 mg/kg, electrical conductivity (EC): Low to moderate level of EC (120–160 mS/m).

2.2. Data Collection Process

The data collection process was designed to capture comprehensive environmental and soil parameter data across the study site. Figure 3 illustrates the flow diagram of the steps involved in this process. Sensors continuously recorded data at set intervals (every 2 min) to capture enough variations and overall trends. The data were transmitted wirelessly to a central database using IoT communication protocols. Also, data logger was used as a backup plan to prevent data loss. The collected data were logged in a cloud-based storage system, allowing for remote access and ensuring data integrity. Regular calibration of sensors was conducted to maintain accuracy. Validation of sensor data was performed by comparing sensor readings with expected values. The analysis presented in this article is based on over twenty-seven thousand collected data points.

Table A1. Sample from the data collected in April 2024.

Time	Nitrogen	Phosphorous	Potassium	pH	EC	Temperature	Moisture
2984	7	10	20	602	101	20	18
65023	7	9	20	665	100	18	18
127061	7	10	20	685	101	16	17
189100	7	9	19	687	101	19	18
251140	7	9	20	700	98	17	19
313178	7	10	20	700	101	19	19
375218	7	10	20	700	101	23	22
437257	7	10	20	700	101	23	21
499296	7	10	20	700	101	24	27
561335	7	10	20	700	98	24	25
623374	7	9	20	700	100	18	25
685413	7	10	20	700	99	22	22
747452	7	9	20	700	99	25	25
809491	7	10	20	700	100	26	28
871533	7	10	19	700	102	28	28
933570	7	10	20	700	101	24	23
995608	7	9	20	700	104	26	26
1057645	7	10	20	700	98	28	26
1119683	7	10	20	700	102	27	27
1181721	7	10	20	700	102	27	27
1243759	7	9	19	700	103	28	27
1305796	7	10	20	700	101	26	28
1367833	7	10	20	700	103	30	30
1429871	7	10	20	700	101	28	25
1491909	7	10	20	700	101	28	25
1553947	7	10	20	700	100	27	30
1615984	7	10	20	700	102	28	29
1678022	7	10	20	700	103	28	28
1740059	7	9	20	700	103	28	31

in the Appendix A exhibits a small sample from the data collected from the sensor.

The measurements started at T₀ = 0 s and continued for the subsequent 1,740,059 s, approximately 20 days. Through this period, nitrogen levels remained around 7 mg/kg, phosphorous levels ranged between 9 and 10 mg/kg, and potassium fluctuated between 19 and 20 mg/kg. Initially, the pH level was recorded at 6.02, stabilizing at 7 shortly thereafter. EC varied between 98 and 104 mS/m. Among the measured parameters, temperature and moisture exhibited the most variation, with temperatures ranging from 18 to 30 °C and moisture levels fluctuating between 17 and 31%. These fluctuations in temperature and moisture are expected, as Croatia experiences a shift from the winter months to spring weather in April.

2.3. Analytical Techniques Used

Principal Component Analysis (PCA) was employed to reduce the dimensionality of the dataset and to visualize the relationships among variables. This technique is particularly effective in transforming a large set of correlated variables into a smaller set of uncorrelated variables called principal components, which retain most of the variance present in the original dataset [25]. The biplot facilitates the interpretation of the data structure by illustrating both the principal component scores and the variable loadings in the same plot [26]. The length and direction of the vectors indicate the strength and nature of the relationships among variables, allowing for an intuitive understanding of the underlying data patterns to examine the relationships between different parameters (e.g., NPK levels, soil moisture, temperature, and humidity). This analysis helped in understanding how these factors interact and influence crop yields.

2.3.1. Principal Component Analysis (PCA) & Biplot

PCA was used in conjunction with biplot visualization to explore the multivariate structure of our dataset. PCA is a dimensionality reduction technique that identifies the underlying patterns and relationships within high-dimensional data by transforming it into a lower-dimensional space while preserving most of the original variance. The biplot representation extends PCA by simultaneously displaying both the observations (samples) and the variables (features) in a single plot, providing insight into their relationships and contributions to the principal components. In our analysis, each data point represents a sample, and its position in the biplot is determined by its scores on the principal components. Additionally, the direction and length of the variable vectors in the biplot indicate the strength and direction of their influence on the principal components, allowing for the interpretation of variable relationships and their contributions to the overall variance in the dataset. This combined approach of PCA biplot analysis enables comprehensive visualization of the complex interrelationships among the variables and facilitates the identification of underlying patterns and trends within the data.

2.3.2. Analyzing Humidity Curve

For the automated detection of watering events based on humidity data, we leveraged the functionality provided by the latest version of the SciPy library (version 1.13.0) in Python (version 3.10). Specifically, we employed the find peaks function, which offers robust peak detection capabilities. This method utilizes various parameters, such as peak prominence and peak width, to identify local maxima in the dataset corresponding to significant changes or peaks in humidity levels as shown in Figure 4. By carefully tuning these parameters to suit our dataset and experimental conditions, we were able to accurately detect peaks indicative of watering events. This automated approach not only facilitated the timely detection of watering occurrences but also eliminated the need for manual intervention, thereby streamlining the data analysis process and enhancing its efficiency.

2.4. Educational Collaboration with the Farmers

In response to the 2020 earthquake in Petrinja, Croatia, a collaborative project was launched to support small family farms affected by the disaster. This initiative provided financial aid and educational resources to help these farms recover and become sustainable enterprises. This section details the initiation process, technological implementation, and farmer engagement.

2.4.1. Project Initiation and Collaboration

The collaborative and educational project, that began in response to the earthquake, provided aid and support to five farms in the region, including two honey producers, two fruit and vegetable producers, and one cacti producer, with the goal of helping these young families rebuild and develop their businesses into sustainable enterprises.

2.4.2. Technological Implementation and Farmer Engagement

The research phase of the project commenced with the installation of sensor systems, as explained in the previous section, on one of the participating farms. The farm owner was trained in the use of sensor technology to monitor yields and enhance agricultural processes. An interview was conducted with the farm owner to gather feedback on the effectiveness of technology. The owner expressed the willingness to experiment and adapt, recognizing the advantages of technological improvements.

In addition to technological advancements, the project focused on increasing sustainability through the development of entrepreneurial skills among the farmers. The farm owner actively engaged in greenhouse activities, utilizing the Six Sigma framework, specifically the Define, Measure, Analyze, Improve, and Control (DMAIC) methodology, to systematically address inefficiencies in agricultural production. This approach started with defining the specific challenges and objectives related to crop yield and resource management. Through meticulous measurement and data collection, including the use of sensor technology, variables such as water and energy consumption were quantitatively assessed. Analysis of this data identified root causes, such as excessive greenhouse visits. Subsequent improvements were made to optimize watering schedules and reduce energy use, guided by technological integration and best practices. Rigorous control measures and continuous monitoring ensured sustained efficiency gains, supporting long-term sustainability in agricultural operations.

3. Results and Discussion

3.1. Nutrient and Environmental Data Analysis

Utilizing the 5-in-1 sensor and the data logger, the data on N, P, K, pH, and temperature were collected over several days. Figure 5 below presents a snapshot of the data plotting. As can be seen from the figure, the N, P, and K values decreased as the plant grew. This outcome aligns with expectations and is corroborated by the reviewed literature, which confirms that plants consume nutrients during growth.

3.1.1. Potassium and Electrical Conductivity Association

Our analysis revealed consistent patterns between EC measurements and potassium levels within the soil. Given the known association between potassium ions and EC [11,14], this observation suggests the potential for approximating potassium levels based on electrical conductivity readings, or vice versa, Figure 6.

This finding offers an intriguing avenue for leveraging electrical conductivity as a proxy for potassium levels, thereby providing a cost-effective and efficient means of estimating potassium concentrations in soil. Additionally, this correlation underscores the interplay between these two parameters and highlights their utility in assessing soil fertility and nutrient dynamics. Further investigation into the precise relationship between electrical conductivity and potassium levels could yield valuable insights into soil health and management practices. The observation of decreasing trends in both electrical conductivity measurements and potassium levels over time suggests a progressive depletion of soil minerals and ions. This decline in essential nutrients has significant implications for plant health and yield. The diminishing availability of key minerals and ions in the soil could adversely affect plant growth, development, and overall productivity. As nutrient reserves become depleted, plants may experience nutrient deficiencies, leading to stunted growth, reduced vigor, and diminished yield potential. Employing electrical conductivity as an indicator for potassium levels [11] allows farmers to efficiently monitor soil health and make informed fertilizer application decisions. This method reduces the chances of over-fertilization, thereby decreasing environmental impact and optimizing resource utilization. Moreover, the prolonged depletion of soil minerals and ions may exacerbate soil degradation, compromising its fertility and resilience. These findings underscore the importance of proactive soil management strategies, such as nutrient replenishment and conservation practices, to mitigate nutrient depletion and sustain optimal soil health for agricultural productivity. Further investigations into the dynamics of soil nutrient depletion and its impact on plant performance are warranted to inform targeted interventions and enhance agricultural sustainability.

3.1.2. PCA Biplot Reveals Dependencies in Data

The PCA biplot analysis unveiled distinct patterns among the various parameters under investigation. Notably, the pH value exhibited a unique and independent pattern, primarily associated with PC2. This separation from the other parameters suggests that pH variations are driven by different underlying factors compared to the remaining parameters. In contrast, the parameters including moisture, nitrogen, potassium, phosphorus, temperature, and electrical conductivity displayed similar variation patterns in the data, predominantly characterized by PC1, Figure 7.

These findings on the PCA biplot analysis are consistent with the established understanding of PCA biplots [15]. The pH value’s unique pattern primarily associated with PC2 suggests that it is influenced by factors that are distinct from those affecting the other parameters.

Furthermore, these parameters clustered together and exhibited coherent directional trends, indicating shared variance and potential interdependencies. This collective behavior suggests that changes in one of these parameters may be indicative of concurrent changes in others, reflecting underlying relationships and interconnectedness within the soil ecosystem. Overall, the PCA biplot analysis provides valuable insights into the multivariate structure of the dataset, highlighting the distinctiveness of pH dynamics and the cohesive behavior of other key parameters, thereby enhancing our understanding of soil variability and dynamics.

3.2. Empowering Farmers through Educational Collaboration

In addition to the analysis, as detailed in Section 2.4, this research successfully raised awareness about the advantages of digital transformation and smart farming for the development of entrepreneurial skills. During discussions on scaling the project, the farmers acknowledged that collaborative efforts could optimize their production and distribution processes. They are now willing to collaborate and participate in future research. The goal is to educate them on how to use technology to build a sustainable agricultural supply chain.

This iterative approach holds promise for enhancing agricultural sustainability by promoting efficient resource utilization, reducing environmental impacts, and enhancing overall crop resilience in the face of changing climatic conditions and evolving agricultural landscapes. Further investigations into the dynamics of soil nutrient depletion and its impact on plant performance are warranted to inform targeted interventions and enhance agricultural sustainability. Scaling up this research to encompass multiple plant species and diverse environmental conditions will enable a comprehensive assessment of how sensor data correlates with variations in yield quality and quantity across different agricultural contexts.

4. Conclusions

This paper explored collaborative smart agriculture within the context of technology and community partnerships. By collaborating with small family farms near Zagreb, Croatia, and academic institutions, the pragmatic benefits of smart farming were demonstrated using IoT to overcome their challenges. The findings have promising implications for agricultural practices aimed at optimizing plant yield and enhancing water use efficiency for sustainability. By leveraging the insights gained from the PCA biplot analysis, predictive models can be developed to forecast plant yield based on soil parameters and tailor watering intervals to maximize productivity while minimizing water usage. This approach offers a data-driven strategy for precision agriculture, enabling farmers to make informed decisions regarding irrigation scheduling and resource allocation, thereby promoting sustainable water management practices. Furthermore, the study lays the groundwork for future research endeavors aimed at expanding the scope and applicability of sensor-based monitoring systems in agricultural settings.

In conclusion, this study underscores the potential of sensor-based monitoring systems coupled with advanced data analytics techniques to revolutionize agricultural practices and contribute to the realization of sustainable food production systems. The importance of comprehensive data collection cannot be overstated, as it forms the backbone of informed decision-making processes. By collecting accurate and real-time data, farmers can make data-driven decisions that optimize resource use, enhance crop yields, and promote environmental sustainability. Through continued interdisciplinary research efforts and collaborative partnerships, the significant potential for impact and scalability was highlighted. The findings also demonstrate how data-driven approaches can lead to actionable insights, helping farmers adapt to changing conditions and improve their agricultural practices. By embracing smart farming, a more resilient and productive agricultural supply chain can be created—one that benefits both farmers and the environment.

Future steps should include the development of machine learning models to predict crop yields, optimize irrigation schedules, and identify potential pest infestations. By adopting smart farming practices, we can build a more resilient and productive agricultural supply chain, benefiting both farmers and the environment.