Research on the Smart Broad Bean Harvesting System and the Self-Adaptive Control Method Based on CPS Technologies

Abstract

With the rapid development of new-generation cyber–physical system (CPS) technologies, the smart operation and management of the broad bean harvesting system are enabled. This paper proposed a smart broad bean harvesting system (SBHS) and a self-adaptive control method based on CPS technologies. Firstly, the overall architecture of the SBHS is designed, which consists of four main components, namely optimal intelligent perception environment configuration, digital twin model construction, virtual simulation and real-time optimization, self-adaptive adjustment and control. Then, three key enabling technologies are discussed in detail, namely, intelligent perception environment configuration for the SBHS, digital twin model construction for the SBHS, colored Petri net (CPN)-based self-adaptive analysis and control of the harvesting system. Lastly, a proof-of-concept experiment based on a cooperative company is presented to illustrate the main work logic and advantage of the proposed SBHS. After the edge–cloud cooperative intelligent harvesting environment is configured, the CPN model for the workflow of the SBHS is created to analyze and optimize the harvesting processes. In addition, a management and control platform are developed to further illustrate the implementation of the proposed SBHS and the self-adaptive control method.

1. Introduction

The dramatic changes in global climate and the increasingly complex international situation have posed higher challenges to global food security [1]. Agricultural informatization utilizes advanced information technology to transform traditional manual agriculture, and serves as an important support for the modernization of agriculture. With the wide application of new-generation cyber–physical system (CPS) technologies [2,3], such as Internet of Things (IoT)-based sensing technologies, digital twin-based modelling and analysis technologies, cloud computing (CC) [4], edge computing (EC) [5] and artificial intelligence (AI) [6], the smart operation and management of the agricultural harvesting system are enabled [7,8]. Intelligent on-site control, wise cloud decision making, and mobile terminal monitoring and scheduling are the future direction of smart agriculture development. For example, intelligent agricultural harvesting robots are gradually replacing humans to perform time-consuming and labor-intensive tasks, which can detect, locate, and grasp the fruits by themselves [9,10].

Broad beans are widely cultivated in China and are one of the most important edible legume crops in the world, possessing significant value in various sectors such as the food processing industry, feed processing, medicinal use, and the production of functional foods [11]. The planting area of broad beans in China has been increasing year by year, and has gradually developed towards specialization and scaling. However, the harvesting of broad beans is mainly relying on manual operations, which is costly, inefficient, and hard to meet the requirement of short-term intensive harvesting. Therefore, developing a highly efficient and automated broad bean harvesting mechanism will play a crucial role in the future of the smart harvesting system.

Recently, CPS technologies are rapidly developed and widely applied in the smart system [12], which can integrate advanced environmental sensing, embedded computing, information communication, and automatic control technologies, physical systems are enabled with capabilities for computation, communication, precise control, remote collaboration, and autonomy [13,14]. Based on the application of CPS technologies, the integration between the physical harvesting system and virtual digital twin model can be enabled. Thus, the optimization of the harvesting system can be made efficiently in the virtual space. To apply the CPS technologies into the smart broad bean harvesting system, three kinds of studies are important, i.e., smart agriculture environment sensing, harvesting process modelling and self-adaptive decision making.

The sensing of the agriculture environment is the basis of the smart harvesting system. Many advanced sensing technologies are applied. Rudrakar and Rughani proposed an IoT-based agriculture framework, and the architecture, security and forensics are discussed in detail [15]. Ruan et al. presented a life cycle framework of green IoT-based agriculture, and its finance, operation, and management issues are analyzed [16]. Sheng et al. presented a multimodal data sensing and feature learning-based self-adaptive hybrid approach [17]. The vision-based accurate target recognition and three-dimensional (3D) positioning of objects are widely researched for autonomous robotic harvesting [18]. Tang et al. completed the localization of Camellia oleifera fruit by building a binocular stereo-vision system [19]. Lei et al. discussed the detection and positioning of Camellia oleifera fruit based on LBP image texture matching and binocular stereo vision [20]. Rad presented a CPS architecture model for the smart monitoring of potato crop in the field of precision agriculture [21].

After the real-life situations are monitored, the analysis of the large-amount agriculture data is critical to make harvesting decisions. Ouafiq et al. proposed an AI-based modeling and data-driven evaluation method for smart farming-oriented big-data architecture [22]. Ma et al. presented a digital twin and big data-driven sustainable smart manufacturing based on information management systems for energy-intensive industries [23]. Mitsanis et al. presented a 3D functional plant modelling framework for agricultural digital twins [24]. Kalyani et al. presented a digital twin deployment for smart agriculture in Cloud-Fog-Edge infrastructure [25]. Alves et al. discussed the development of a Digital Twin for smart farming, and the irrigation management system for water saving is taken as an example [26]. Manocha et al. presented an IoT-digital twin-inspired smart irrigation approach for optimal water utilization [27]. Huang et al. presented an agent-based simulation model of wheat market operation [28].

Based on the virtual model of agriculture system, the optimal decisions can be made based on the self-adaptive algorithms. Lin et al. established a dynamic and static knowledge model of the manufacturing cell management system, which can provide knowledge push approach for the harvesting system [29]. Zhang et al. discussed the construction and optimization of a collaborative harvesting system for multiple robotic arms and an end-picker in a trellised pear orchard environment [30]. Ren et al. presented a mobile robotics platform for strawberry sensing and harvesting within precision indoor farming systems [31]. Zamora-Izquierdo presented a smart farming IoT platform based on edge and cloud computing [32]. Zhang presented the design and experiments of the whole field path tracking algorithm for a track-based harvester [33]. Dai et al. presented a multiobjective multipicking-robot task allocation, i.e., mathematical model and discrete artificial bee colony algorithm [34].

Despite significant progress in the development of smart agriculture system, the construction and control of the broad bean harvesting system are still hard to work under natural conditions due to the various uncertainties from weather, light, wind, etc. Three main kinds of challenges need to be solved.

Firstly, the existing sensing approach for the agriculture mainly focus on the high-level framework for the advanced precision agriculture or a specific area (such as fruit detection). To make a real-time harvesting monitoring and control system, the whole easy-to-deploy and easy-to-use architecture should be further discussed based on the CPS infrastructure.

Secondly, amounts of agriculture data are collected and analyzed in a cloud center, which requires more time for data transmission and computation. The computation capacity of smart harvesting machines with edge computing ability is seldom considered. Further, reliable information exchange on the field side is hard to ensure. How to efficiently and accurately making decisions at the practical harvesting site needs to be researched.

To solve the above problems, this paper proposes a smart broad bean harvesting system (SBHS) and a self-adaptive control method based on CPS technologies. Firstly, the overall architecture of the SBHS is designed, which consists of four main components, namely optimal configuration of smart sensing environment for bean harvesting, digital twin model construction for the smart bean harvesting system, virtual simulation and real-time optimization of the harvesting system, self-adaptive operation and management of the bean harvesting system. Then, three key enabling technologies are discussed in detail, namely, optimal configuration of sensing environment for s the mart bean harvesting system, digital twin model construction for the bean harvesting system, colored Petri net (CPN)-based self-adaptive operation and control of the bean harvesting system. Lastly, an operation and control platform for the smart bean harvesting system is developed to demonstrate the main work-logic and advantages of the proposed smart bean harvesting architecture and the self-adaptive control method.

The rest of this paper is organized as follows. Section 2 proposes the overall framework of the smart broad bean harvesting system and the self-adaptive control method. Section 3 describes the three main key technologies of the SBHS. A case study is discussed in Section 4. Conclusions are drawn in Section 5.

2. Overall Framework

The objective of the smart broad bean harvesting system aims to present a real-time and comprehensive perception of the harvesting site data, accurate and dynamic modelling of the harvesting processes, and self-adaptive and optimal control of the harvesting system. The overall architecture of the proposed system is shown in Figure 1. Mainly, it consists of four modules from the bottom to the top: intelligent perception environment configuration of the SBHS, digital twin model construction of the SBHS, virtual simulation analysis and real-time optimization, and self-adaptive optimization and control of the SBHS.

The intelligent perception environment configuration of the SBHS involves the arrangement of agricultural sensors and automatic control devices to meet the on-site management requirements for broad bean harvesting. This allows the system to autonomously transmit its own status data and independently receive control instructions from higher-level systems. It can also adaptively adjust various components of the harvesting system, such as the height of the harvester cutting table, the optimal time of the harvesting process, and the optimal harvesting route. On the one hand, multisource harvesting data should be collected, including field weather, soil moisture, ambient temperature, pod height of broad beans, and the geographic location of growth. To meet the data capturing needs, various sensor should be analyzed and deployed, such as the vision-based camera, temperature and humidity sensors, and weather information obtainment devices from the meteorological observatory. On the other hand, adaptive control devices are integrated into the control end of the harvesting machinery to ensure that the harvesting commands issued from the upper-level systems are executed with precision. Concurrently, from a management standpoint, the parameters of the sensors and control devices are configured in areas such as sensor registration, invocation, and information retrieval. This configuration ensures the accuracy and reliability of information gathering, transmission, and facility control.

The digital twin model construction for the SBHS module is designed to create a virtual digital twin model of the harvesting system based on real-time agricultural field data. Firstly, after the vast and heterogeneous raw agricultural production data is obtained, data preprocessing techniques are employed to perform deduplication, noise reduction, and standardization. This prepares high-quality data for the subsequent modeling and analysis processes. Then, the Web Service Description Language (WSDL) and ontology technologies are combined to construct the virtual model of distributed devices that located at different positions and serve various functions. Lastly, by conducting an analysis of the interrelations among different harvesting resource elements, an integrated digital twin model of the SBHS is formed in the virtual space. This integration facilitates the construction of a digital twin model of the harvesting resources and environment. This model is capable of being dynamically updated and promptly adjusted based on real-time information, ensuring the model’s validity and utility.

The virtual simulation analysis and real-time optimization module is executed based on the digital twin model of the SBHS. It integrates data and knowledge from the harvesting process to derive the optimal configuration for on-site machinery, including adjustments of the machinery height, planning of the harvesting path, and identification of the most opportune harvesting times. The digital twin model of the SBHS mirrors the actual conditions of the production site, offering fundamental reference data for the harvesting process. On the one hand, critical performance information, such as patterns of weather change and indicators of edible bean maturity, can be deduced through statistical analysis of data collected from the harvesting site. On the other hand, by integrating the specialized knowledge of farmers (e.g., the optimal time for picking) and historical production data (including bean loss rates under specific picking strategies) into the digital twin model, a simulation analysis engine can be employed utilizing colored Petri nets (CPN). Finally, the optimal harvesting plan can be extracted, including suitable picking time, optimal harvesting routes, and lowest cutting table height.

The self-adaptive optimization and control module of the SBHS implements the optimal control plans derived from simulation analysis at the real-life device end, such as the harvesting machines and transport vehicles on the broad bean harvesting site. The harvesting devices then devise more refined harvesting operation plans based on the actual conditions at the harvesting site and their own operational status. This includes making minor adjustments to the harvesting path and enabling self-adjustment of the harvesting cutting table height. For example, these harvesting devices are capable of making dynamic adjustments based on the execution of the control measures, such as reducing the harvesting speed when the broad beans are densely populated. Ultimately, once the anticipated harvesting objectives have been met, they provide timely feedback to the cloud controller. This allows the cloud control platform to promptly and effectively develop the subsequent control plans for the harvesting process.

3. Key Technologies

3.1. Intelligent Perception Environment Configuration of the SBHS

The intelligent perception environment configuration of the SBHS is a prerequisite and foundation for its dynamic operation and control. It mainly serves two functions: on-site data acquisition and transmission of broad bean harvesting, and a cloud-based analysis and adaptive control platform of the harvesting system, as shown in Figure 2.

On-site data acquisition and transmission for the broad bean harvesting system aims to sense and transmit the multisource data in the harvesting process. Various aspects of the harvesting system are involved, such as the harvester status (e.g., working time and efficiency), weather condition, soil humidity, and the growth status of broad beans (e.g., maturity and pod height). To obtain these multisource data, difference sensors should be selected and configured. Due to the difference in the attributes of the sensors (i.e., price, measurement accuracy, application scenarios), the optimal selection and installation strategy can be obtained according to historical operator knowledge. Since different sensors have distinct communication methods, connection interfaces, and sensing ranges, a standardized sensor management center is set. Three main functions are fulfilled by the center, i.e., sensors registration, parameter setting, and working status monitoring. After configuring the sensors, real-time information about the soil, weather, ripeness, temperature, and humidity of the SBHS can be obtained. Then, the obtained data can be processed and transmitted to the upper-level data analysis platform through wireless transmission networks such as Bluetooth and WiFi.

In terms of the cloud-based analysis and adaptive control platform, the sensed harvesting data are further integrated and exchanged. On the one hand, the platform can analyze the massive, heterogeneous, and multisource data on the growth and harvesting of broad beans, thereby obtaining the real-time status of the broad bean harvesting site. On the other hand, the platform can optimize the control of the harvesting system by managing various resources of the SBHS.

After the configuration of the intelligent harvesting perception environment, the resources involved in the harvesting process, which are distributed in various directions, can possess a certain degree of intelligence. That is to say, they are not only capable of transmitting their own status to the outside world in real time but also have decision-making capabilities. They can actively accept the harvesting instructions issued by the higher level and, based on the current state, formulate appropriate execution plans to better complete the harvesting tasks assigned by the higher level.

3.2. Digital Twin Model Construction for the SBHS

The construction of digital twin model for the SBHS aims to extract key harvesting resource states from amounts of multisource and heterogeneous broad bean harvesting site data, form digital twin model for each broad bean harvesting resources, and integrate the individual models to create an overall digital twin model of the SBHS. The specific workflow is as shown in Figure 3.

Firstly, by configuring the intelligent harvesting environment, a vast amount of multisource and heterogeneous data are automatically collected. High-quality information from the harvesting site is then obtained through data preprocessing techniques, which mainly include four categories: deduplication, noise reduction, heterogeneous value standardization, and missing value fill.

• Deduplication involves filtering out spatially repetitive data generated by multiple sensors covering the same area, ensuring that the same event is represented by a single information entry.
• Noise reduction is the process of directly filtering out uninteresting data to retain only those that are useful or of interest to the harvesting operator.
• Heterogeneous value standardization aims to unify data from different sensors with different structures using a standardized descriptive language, such as XML, to facilitate the easy reading of information.
• Missing value fill is carried out by filling in missing numerical values through methods such as default value filling, mean value filling, mode filling, KNN imputation, and predicting missing values as a new label using a model, to ensure the continuity and effectiveness of the data.

Secondly, digital twin models for various broad bean harvesting resources are constructed based on the processed data. Initially, it is essential to model the geometry and physical dimensions of the resources, so that the external framework of the harvesting resources can be created, such as, a virtual model of the agricultural fields, the digital framework of the harvesters, and the geometric arrangement of other infrastructures. Then, the behaviors of the broad bean harvesting resources are modeled by analyzing their operational rules. This includes modeling the overall operational path of the harvester, the start time and mode for the harvesting process, and the parameter adjustment of the harvester. In addition, a binding between the virtual digital twin resources and the actual scenario must be formed. This ensures that the harvesting resources in the virtual space can adapt to changes in the physical resources within the real-world environment.

Finally, different harvesting resources models are aggregated to create an overall digital twin model of the SBHS. The upper-level cloud platform is primarily concerned with the entire operation status of the farm, and then makes production optimization decisions based on the overall situation. To achieve this, it is necessary to integrate different harvesting resource models to form a comprehensive digital twin model of the SBHS. Common methods include association, combination, integration, and mapping.

• Association analysis is conducted by using correlation mining techniques on the data from the harvesting site to identify frequent patterns, associations, correlations, or causal structures between collections or sets of harvesting resources.
• Combination involves grouping similar schemes together by assigning unified tags, which facilitates subsequent effective analysis and management of similar resources.
• Integration refers to the consolidation of different types of harvesting resources, enabling the overall broad bean harvesting system to be uniformly scheduled and managed in a coordinated manner.
• Mapping relationships ensure that there is synchronization between the physical and virtual spaces, guaranteeing that the digital twin system model can accurately depict the operational conditions within the physical space.

3.3. Colored Petri Net-Based Self-Adaptive Analysis and Optimization of the Harvesting Process

The real-time analysis and dynamic optimization of the broad bean harvesting system are achieved by analyzing their digital twin model. Two main functions are important, i.e., key performance indicators obtainment and optimization of the harvesting processes. Nowadays, Petri nets (PN) have been successfully used to model, control and analyze the dynamic system that are characterized by concurrency or parallelism, asynchrony, deadlocks, conflicts, and event-driven processes [35]. The colored Petri net (CPN) can largely reduce the complexity of the PN models and prefers better modeling ability, this paper uses the CPN model to model and analyze the workflow of the digital twin model. Figure 4 shows the work-logic of CPN-based analysis and optimization of the harvesting process.

The CPN model is capable of fully describing the relationships between resources in the digital twin model of the broad bean harvesting process. It adeptly reflects the asynchronous, concurrent, and conflicting characteristics of various events within the broad bean harvesting system. By employing the approach of colored tokens, the CPN model can efficiently mitigate the state space explosion that is common when modeling with basic Petri net models. This enables the efficient capture of critical operational performance metrics of the intelligent harvesting system, such as working time and work efficiency. Additionally, it facilitates the selection of the optimal resource allocation plan for the harvesting process.

A CPN model can be represented as an eight-tuple:

N = <P, T, C, I, O, G, D, M>

(1)

where P refers to the places in CPN model and is used to describe the status of harvesting resources (e.g., harvesters and sensors), five main kinds of subsets are contained, i.e., general place (P_g) to model the status of resources, sensing place (P_s) to exchange with real-life sensors, command place (P_s) to transfer the extracted information to resources, in/out place (P_i,o) to transmit data among different CPN models, monitor (P_m) place to record the historical harvesting events; T denotes transitions for the events in the harvesting process; C denotes the colored mapping from P or T to W, where W is a finite and nonempty set for the different status of the resources, and the colored information can be described in the tokens through the CPN models; I and O denote the input or output matrix between P and T, which can describe the information or tokens flow within the CPN model; G is the guard function of the P or T, which can describe the event triggering conditions; D refers to the time delay of the events since harvesting events often require a period of time; M refers to the marking of the tokens in the CPN model, which can be used to analyze the whole harvesting status, and M₀ is are often set as the initial status.

The CPN model can be created based on the harvesting process following six main procedures, as shown in the middle of Figure 4.

• Obtain the overall structure of the harvesting system and transform all the high-level resources into CPN elements to create a main CPN model in the cloud platform. In general, a whole harvesting system can be divided into five kinds of resources, i.e., harvester, broad bean, growth environment, bean collector and deliverer. Each kind of resource can be seen as an abstract element in the overall CPN model, and each element can be replaced by a detailed sub-model.
• Further analyze the activities of the high-level resources and create a detailed sub-CPN model for each resource or sub-systems. These sub-CPN models can be deployed on the distributed resource side and just set a connection with the high-level elements in the cloud platform.
• Repeat the second procedure until any element is described in detail. There are many elements participating in the harvesting system, and many performance indicators can be monitored, construct a CPN model that considers all the status or elements in the real-life system is impossible and useless. Thus, this article mainly considers the harvesting efficiency, broad bean loss rate, and a hierarchical structure is set, where the elements in higher-level CPN model can be replaced by a detailed sub-model.
• Set arcs, guard functions, and token delivery rules between different places and transitions, so that the CPN model can reflect the practical workflow of the harvesting system.
• Set communication elements between the virtual CPN models and the practical sensors in real-life system, and bind the intelligent tokens with real-life resources so that the virtual CPN models can change their status with the real harvesting system.
• Simulate the CPN models to find the potential deadlocks or other shut-down problems, and then set measures to prevent them. For example, if two harvesters call one broad bean collector, one deadlock happens, then one priority can be given to one harvester according to distance, and the deadlock can be removed. After all the potential problems are solved, the CPN models can work with liveness.

After the whole CPN model is established, the real-life resource status in the actual harvest system can be represented by token states within the CPN model. These token states are then linked and bound to the resources within the real physical environment. Then, the number, position, and attributes of colored tokens in the CPN model can be dynamically updated based on the following rules

$$\forall P_i\in P:M^{\prime }(P)=M(P)-O(P,T)+I(T,P)$$

(2)

where M′(P) and M(P) refer to the next state and current of the place, respectively; O(P, T) represents the output from the place to its following transitions; I(T, P) represents the input from the previous transition to the place.

According to the CPN model’s statistical analysis capabilities, the operational performance of the entire SBHS can be obtained. Furthermore, various harvesting resource configuration strategies can be analyzed within the virtual space. Based on the CPN model’s robust simulation and inferential abilities, different resource optimization and allocation plans can be assessed. By evaluating the benefits and performance outcomes associated with each scheme, an optimal resource optimization and allocation strategy can be derived. This strategy encompasses critical parameters such as the height of the harvester cutting table, the speed at which it operates, and the paths it traverses.

Based on the extracted optimal harvesting scheme, the related directives can be created and transmitted to the physical resources, and then the actual harvesting resources can autonomously adapt and perform harvesting tasks. Further, when the growth environment changes or abnormal events happens in the broad bean harvesting system, prompt feedback can be relayed to the CPN model. This feedback facilitates the dynamic adjustment and optimization of the harvesting directives, ensuring that the system remains responsive and efficient in the face of changing conditions.

4. Case Study

4.1. Case Scenerio

To verify the feasibility and effectiveness of the proposed method, a case from a cooperative broad bean farm was conducted. At present, the harvesting process of broad beans mainly relies on manual picking by farmers. Due to the low efficiency of manual picking, rapid changes in weather and intensification of the aging population of farmers, premature harvesting of broad beans and post-rain harvesting often happen, which can lead to waste during the harvesting process.

Since the implementation of the proposed SBHS to a real-life farm is a very difficult and complex task. To demonstrate the advantages of this research, a proof-of-concept experiment is designed in our lab. A management and control platform for SBHS was developed to provide a reference for the promotion of intelligent precision agriculture. As depicted in Figure 5, this system includes four main aspects, i.e., a cloud control center, edge control devices, smart broad bean harvesters, and an intelligent broad bean sensing environment.

• The intelligent broad bean sensing environment was constructed by configuring different sensors, and key information can be captured, including soil moisture, weather, the maturity of broad beans, and the height of bean pods.
• The smart broad bean harvesting machines were equipped with embedded edge–cloud control devices. Mainly three kinds of machines are considered, the harvesters for harvesting the broad bean in field, collectors for collecting beans from different harvesters, and deliverer for move the beans to the warehouse. On the one hand, the real-time status of the itself can be monitored and analyzed, such as the height of cutting table, the location, and the speed. On the other hand, adaptive adjustment of the harvesting cutting table height and autonomous planning of the harvesting path can be achieved by themselves.
• The edge control devices can sense the raw status of the harvesters, pre-integrate the raw sensed data, so that the data volume can be largely reduced and only key information can be uploaded to cloud control center. At the same time, cloud control directives can be converted to the command that can be executed by the harvester.
• The cloud control center is capable of providing optimal harvesting time information based on the key information provided by the intelligent broad bean sensing environment. It formulates optimized decision making for the harvesting path of broad beans based on the geographical information of the bean growth environment.

After the smart environment is configured, the broad bean harvesting system can be modelled in the virtual space, which can be seen as the digital twin of the physical harvesting system. Then, the CPN model is used to analyze the workflow of the harvesting system, and a management and control platform is developed for the SBHS.

4.2. CPN Model-Based Self-Adaptive Analysis and Control for the SBHS

To model the harvesting processes, a CPN model can be created in the software CPN Tools, as seen in Figure 6. There is one main model for the overall harvesting system, and it has eight macro-transitions which are connected to eight sub-CPN models. The sub-models represent the specific activities for the key harvesting resources. Similarly, each sub-CPN model of activities has several macro-transitions that can be replaced by more detailed sub models. To simplify the description, only the main CPN model and one sub-CPN model for harvester is discussed in this paper.

As shown in main CPN models in Figure 6a, there are four kinds of ‘check’ transitions, the working conditions for the harvester, collector, deliverer, and the bean growth will be evaluated firstly. Since the check processes for the resources are complex, the corresponding ‘check’ transitions in the main CPN model can be expanded by four detailed sub-CPN models, respectively. If the broad bean growth status arrives the maturity level and the four machines are ready, the harvesting activities will begin, and several harvesters can work simultaneously. After the collection box of the harvesters is full, the broad beans should be unloaded to the collector, and then the collector will upload the broad bean to the deliverer car. At last, the deliverer car will convey the harvested broad beans to the warehouse.

The harvesting process is presented in Figure 6b. When the harvesting process begins, the harvester will move to the field side firstly. Then, the machine vision device will analyze the height of the broad bean pod, and the cutting table height can be set accordingly. At the same time, the optimal harvesting path can be obtained based on the field condition by the harvester. Before the normal harvesting, a preharvest event is set to evaluate the harvesting effect. If the harvesting result can be accepted, the harvester will start the normal harvesting for all the broad beans.

The tokens in the CPN model are used to describe the resource state, which bind colored information about the resource attributes. The global color set declarations are important for the CPN model. For example, the tokens in place ‘harvester’ are set as DATA, which can record the ID, name, or other key information. The firing functions and rules are given in the CPN models based on the collected data from the case farm. The consuming time for the transitions can be represented by the value after symbol ‘@’ at the top right corner of transitions.

In this case, once all the resources are ready, the harvesting process starts to operate by themselves. Moreover, the key harvesting information can be tracked and measured, including the work-time of the harvesters, the overall progress of the harvesting, and the collected volume of broad beans. Then, the obtained high-level information can be uploaded to the management and control platform for further analysis.

4.3. The Management and Control Platform for the SBHS

The developed management and control platform for SBHS consists of three main modules: real-time perception of the intelligent broad bean growth environment, self-adaptive adjustment of the harvester, and cloud-based harvesting optimization and decision making.

The real-time perception of intelligent broad bean growth environment is equipped with a variety of sensors at each critical monitoring point within the bean cultivation farm. It can process the sensed data through an edge–cloud collaborative mechanism. This process is then visually presented to provide reliable information support for the subsequent stages of intelligent broad bean harvesting. Figure 7 illustrates the key data points related to the growth of broad beans, and the users can click on different node positions to access detailed information such as the maturity level of the broad beans and the soil moisture levels.

The self-adaptive adjustment of the harvester module aims to proactively sense its operational status and relay the information to the cloud-based decision-making center. After receiving the harvesting directives created by the cloud decision center, the module can autonomously make adjustments based on real-life situation of field site, including the cutting table height rising or falling, the harvesting speed altering, and harvesting path planning. The self-adaptive adjustment of the harvester module is composed of the four corresponding sub-modules for the above four aforementioned functions. As described in Figure 8, the harvesting path planning module can offer basic information of the harvester itself, fundamental details regarding the broad bean harvester, and the specific information for the harvesting path.

The cloud-based harvesting decision-making module aims to develop harvesting decision directives based on the growth information of broad beans. It can also assign the working areas among multiple harvesters according to their operational capabilities, and then deliver the established work instructions to the specific harvesters in a timely manner. Furthermore, the module facilitates the real-time visualization of the harvesting status of broad beans for the personnel in charge at the cloud-based centralized control center. Figure 9 shows one screen of the cloud-based harvesting decision-making module. Four main kinds of information can be shown in the platform, i.e., harvesting information, harvester information, optimization orders and historical information. As shown in Figure 9, the harvesting information is given, the basic information for the overall harvesting process is given firstly, and then a pie chart is used to represent the ratio of beans that have been harvested versus those that remain to be harvested. It also gives an estimated yield for the beans that have already been harvested.

5. Conclusions

With the wide application of smart technologies, intelligent agriculture with self-sensing and self-decision-making capacity is enabled. However, the intelligent harvesting of broad bean is hard to achieve due the high complex growth environment and the large volume of sensed data.

To address these issues, a smart broad bean harvesting system (SBHS) is designed based on CPS technologies. Firstly, the overall framework of the SBHS and self-adaptive method is presented to give a high-level structure for the application of the proposed system. Then, three key enabling technologies are discussed in detail, i.e., intelligent perception environment configuration, digital twin model construction, and CPN-based analysis and optimization of the harvesting process. At last, a proof-of-concept case study is presented to verify the main work logic of the proposed SBHS and the self-adaptive control method. After the demo case environment is constructed, a hierarchical CPN model is created for the overall SBHS. According to the simulation result of the CPN model, the key harvesting information can be tracked and measured. Then, the extracted information can be uploaded to the management and control platform for further analysis. The main parts and workflow of the developed management and control platform for SBHS is presented in detail, which can provide a useful feasible reference for the practical system.

There are three main challenges waiting to be solved. The first challenge is to incorporate the proposed SBHS to a practical harvester that can work in a real-life farm. The second challenge is to build a highly accurate and comprehensive digital-twin model for a physical broad bean harvesting system, which consists of a large amount of dynamic natural elements. The third challenge is how to combine the more intelligent and efficient deep learning algorithms for decision making in the SBHS.