1. Introduction
Environmental factors such as the consequences of climate change [
1] directly affect producers in the agricultural price setting process [
2]. Agricultural trade is expected to grow at roughly half the rate of the past ten years [
3]. Nevertheless, for most commodities, the share of total production that is being traded on global markets will remain relatively constant. Technology will also be more important in guaranteeing global food security, due to natural resource constraints in many countries [
3]. In this regard, the concept of bioeconomy is gaining importance within the European Union objectives for the 2030 Agenda and its Sustainable Development Goals, and new indicators to measure the performance of bioeconomy sectors are emerging [
4]. Globalization and the possibility of exporting and importing, as well as competing on a larger market, have led to a paradigm in which agricultural products are a financial indicator of the economy and take part in the prices of future markets. Agricultural commodity prices also tend to correlate with trends on energy markets, with oil prices experiencing unusual rises and falls in recent years [
5]. Although agricultural inputs, production, storage, and transportation have long been influenced by energy prices, the rapid growth of the biofuels sector has created new types of relationships between agricultural and energy markets [
5]. Moreover, in periods of financial crises, the volatility of the agricultural market has been very high [
6,
7]. Those fluctuations on commodity futures markets [
8] contribute to the agricultural price oscillations for the farmers. However, not all agricultural products are affected by the same factors, the agricultural market is very diverse in terms of product attributes and particularities which are subject to external factors. For instance, corn futures prices may be determined by certain factors (such as location, transportation costs, contamination, warehouses, weather conditions, etc.) in comparison to wheat prices, that are less sensitive to weather. The changes in climate are something that has been more accentuated in recent years due to the greenhouse effect [
9]. Analysis of greenhouse emissions as well as limited resource expenditures or energy consumption is becoming more widespread in precision agriculture (PA) [
10] and smart agriculture (SA) [
11].
PA is a term that has been coined in recent years. It refers to the concept of using new technologies to increase the yield and profitability of crops while reducing the resources needed for cultivation [
12,
13]. Thanks to technological advances, farms in developed and developing countries can benefit from the application of low-cost technologies. In this regard, the Internet of Things (IoT) and, more specifically, the Industrial Internet of Things (IIoT), is presented as a key enabling technology for implementing and monitoring resource management solutions in various scenarios in Industry 4.0, including smart agriculture environments [
11]. The monitoring of all these values can be carried out with sensors, using new paradigms such as edge computing which enable monitoring while reducing the cost of data analysis in the cloud [
14,
15] increase the efficiency of agricultural processes [
16]. The application of multi-agent systems to monitor agricultural processes is common, especially to achieve the efficient use of land in terms of investment and production [
17], to manage resources [
18], to increase the efficiency of irrigation systems [
19,
20], to optimize energy use [
21], or predict the prices of agricultural products [
22]. Moreover, in the field of selecting suppliers, Valluri and Croson [
23] conducted a research towards best supply selection through a game theory approach with agents, in a scenario where reward and punishment were complicated by incomplete information. Over the last years, the number of processes oriented to sustainability objectives [
24] and to developed tools [
25] is growing. For instance, it has become more popular to integrate environmental, economic, and social attributes when selecting a supplier and sourcing process [
26,
27,
28]. There are many differences in price setting in terms of requirements and the most important attributes for which each model can be built. Most models rely on historical data; however, others focus on the buyer, in which there are sales quotas, decision history polynomials, probabilities, and regressions of the potential prices that might be accepted [
29]. In the literature, other approaches have been found, such as agent-based modeling with reinforcement learning conditioned by inventory [
30,
31]. When product pricing strategies are connected to estimations of price allocation and real-time reporting is taken into account, the relationship between data and price allocation parameters can be modeled dynamically, as demonstrated in the multi-agent supply chain [
32]. In the context of a market, goods are exchanged and there are two main players. One of the basic rules in a price system is that goods can be exchanged according to the relative prices of the goods in question. Twenty years ago, Wellman and Wurman [
33] already developed the first market-based multi-agent paradigm at a theoretical level.
The above examples demonstrate that multi-agent systems are applied in cases where a single agent is not capable of carrying out all the processes, as several agents and objectives interact in real-time [
34]. Although currently there are different MAS focused on price forecast [
35] or choice of suppliers [
26,
27,
28], to this day, there is still no MAS in which a buyer can have information on the sustainability of a supplier or a product by monitoring the greenhouse emissions involved in production, the use of pesticides or the consumption of natural resources and having access to information on prices on the futures market. Considering that these needs cannot be fulfilled with a single agent, a MAS has been built [
36]. The proposed system consists of MAS that helps choose a supplier through decision-making based on certain attributes. These attributes measure the sustainability of a supplier and also have prices of agricultural futures markets. The different attributes that have been considered are agri-environmental indicators (AEIs) [
37]. These parameters consist of water consumption, greenhouse emissions, energy consumption, as well as the use of edge computing technologies that represent an improvement in terms of data transfer costs to the cloud. More concisely, the MAS consists of a preference-based multi-objective optimization problem that is open to real implementation, and therefore each user can fix the desired threshold and input requirements. Although not all products are comparable at the world level, the developed MAS is intended to give an indication of the most recent price developments. Futures markets are an important source of price information for farmers, but only a small percentage of farmers directly trade futures. The availability of high-frequency (intraday) data can help market participants make quicker decisions compared to low-frequency data, such as daily or monthly data. Having high-frequency data allows to better forecast the stock prices [
38] so that farmers can sell according to trends in agricultural futures market real prices without having to wait to learn of the effects of selling products.
The rest of this paper is structured as follows.
Section 2 consists of a revision of the state of the art of technologies involved in the system.
Section 2.1 introduces the edge computing technologies as they play a significant role in monitoring and cost reduction in agriculture and identifying the most important trends in the application of those paradigms in smart farming scenarios. Then, in
Section 2.2, state-of-the-art MAS are described, specifically those designated for agriculture scenarios, and finally, in
Section 2.3 the deep Q-learning concept is introduced.
Section 3 describes the MAS architecture and the data that have been used to conduct the experiment.
Section 4 describes the experiments that have been conducted and the results. Finally,
Section 5 discusses the solution, implementation fields, conclusions, and future work.
4. Results
The problem that the presented MAS overcomes is the difficulty of choosing a supplier according to sustainability parameters, as well as having transparent market information. The MAS presented helps potential buyers to identify a supplier according to their preferences in terms of sustainability as well as efficient use of resources. It also combines the selection of the supplier according to certain preferences, as well as seeing the actual price differences with the quotations of the products and being able to buy at the most optimal times. The designed MAS has two main results that are presented below. The first result is related to the supplier selection ranking, and the second one to price forecasting.
To achieve the first goal, different values have been assigned to the attributes, in terms of user preferences. To test the model, different cases have been tested with the following preferences as represented in
Table 3.
Then, a value between [0,10] is assigned to each supplier, where value is understood as the total sum of the weights of different attributes, assigned according to the attribute, and attribute importance. The outputs and the values of each of the parameters are represented in the
Figure 3,
Figure 4 and
Figure 5, in which three dimensions are represented; the value according to the different weights for each of the parameters.
As can be seen from
Figure 3,
Figure 4 and
Figure 5, the different suppliers’ selection would vary according to the preferences set by the user. Once the different suppliers have been ranked, the results are sent to the Q-learning agent. The deep Q-learning agent’s goal is to maximize the total amount of reward it receives. In this case, the agent’s goal is to buy at the lowest price given a certain amount of money. Therefore, the user should fix an initial amount of money, which in this case has been 20,000. To build the agent, the different parameters have been fixed as shown in
Table 4. Then, different hyperparameters have been changed to identify the best agent performance. The set hyperparameters are shown in
Table 4, and then the other hyperparameters have been compared, as represented in
Table 5.
The Q-learning agent has one hidden layer with 256 neurons and the activation function is the rectified linear unit (ReLU) [
76]. In the learning process, DL optimizes the weights,
, to minimize the error estimated by the loss function. Therefore, the error or loss are measured as the difference between the predicted result and the actual result. The loss function for the different cases which are described in
Table 5, is represented in
Figure 6.
As can be seen in
Figure 6, cases 3 and 4 are the ones achieving the best results in all
Figure 6a–c, while, for instance, case 1 has the worst results in
Figure 6a but then has better results in
Figure 6b. Moreover, in
Figure 7 and
Figure 8, all the marked values are placed in the real price variation line.
To view the comparisons of cases 1, 3, and 4 in more detail, the results are represented in
Figure 7, where all the different cases are compared and represented according to the number of iterations in
Figure 7a–c. Has been collected the different cases that were performing better according to the loss functions of
Figure 6.
The conclusion that can be drawn from
Figure 6 and
Figure 7 is that the best options can be chosen by basing the decision-making on hyperparameters, either case 1, 2, or 3, so this case’s comparisons have been compared in
Figure 8. As can be seen in
Figure 8a, case 1 presents good results either in case of 1200 iterations and 2400. Therefore, the option that included in the MAS is the one with the above-mentioned hyperparameters. Thus, the buyer is at last presented with a potential list of suppliers and the lowest market prices at a given moment, which allows them to identify the most suitable suppliers for them as well as the best times to buy the products and also to identify the trends in specific products.