Multiagent Coordination and Teamwork: A Case Study for Large-Scale Dynamic Ready-Mixed Concrete Delivery Problem

Abstract

The ready-mixed concrete delivery (RMC) problem is a scheduling problem, where multiple trucks deliver concrete to order sites abiding by hard constraints in a dynamic environment. It is an NP-hard problem, impractical to solve using exhaustive methods. Thus, it requires heuristic-based approaches for generating sub-optimal schedules. Due to its distributed nature, we address this problem using a decentralised, scalable, cooperative MAS (multiagent system) that dynamically generates schedules. We explore the impact of teamwork by trucks on schedule optimisation. This work illustrates two novel approaches that address the dynamic RMC problem; a Delegate MAS approach and a team-extended approach. We present an empirical study, comparing our novel approaches with existing ones. The evaluation is performed by classifying the RMC case study scenarios into unique stress, scale, and dynamism characteristics. With 40% to 70% improvement over different metrics, the results show that both approaches generate better schedules, and using agent teams augments the performance. Thus, such decentralized MAS with the appropriate coordination approach and teamwork can be used for solving constrained dynamic scheduling problems.

1. Introduction

In NP-hard problems, exact and optimal solutions cannot be found, and only sub-optimal solutions are relied upon. Such problems keep researchers seeking alternate approaches to finding better sub-optimal solutions. One such problem is the dynamic ready-mixed concrete delivery (RMC) problem, in which a fleet of trucks delivers concrete from production sites to the order sites. It is a challenging problem, as, apart from being NP-hard, dynamic RMC also requires handling (1) hard constraints; (2) dynamism due to unexpected events; (3) decentralisation and distribution, as trucks and production sites belong to multiple owners; and (4) large-scale operation, as numerous trucks deliver tens of deliveries.

Dynamic RMC is a transport scheduling problem under the subcategory of pick-up and delivery problems. In this problem, the concrete is prepared at a production site and delivered to an order site (i.e., construction site) by a fleet of trucks. Typically, the order site requires a series of deliveries, which must abide by hard constraints (Figure 1). Three features make dynamic RMC a highly constrained problem: (1) the perishable nature of concrete; (2) successive deliveries of trucks for a single order; and (3) real-time events that make it a dynamic scheduling problem, augmenting the challenges in scheduling this constrained problem. For instance, the time between the successive deliveries of an order may be at most 30 min (see LT in

Table 1. Abbreviationsfor RMC problem with typical values used in simulation.

Term Name	Explanation	Abbreviation	Typical Value
Lag Time	Time between two consecutive  deliveries. (minutes)	$LT$	maximum 30
Concrete Perish Time	Maximum time, concrete can remain in truck (minutes)	$CPT$	90 to 110
Unloading Rate	The rate of unloading by truck at an OS (m${}^3$/h)	$ULR$	10 to 20
Concrete Loading Time	Time required by a PS to load a truck (minutes)	$CLT$	2 to 10
Start Time delay	The delay in delivery time for first delivery, from the requested start time of an OS (minutes)	$STdelay$	depends on schedule

); the concrete should not remain in the truck for more than 100 min ($CPT$ in Table 1). If a truck breaks down during its delivery to order, a new truck must be scheduled, delivering concrete within 30 min of the previous delivery.

In general, multiple companies are involved in RMC, specifically after the emergence of 3PL (Third Party Logistics) [1], and 4PL [2]. Decentralisation in this context is significant, as a typical view in 3PL and 4PL logistics is that different administrations own production sites, truck fleets, and order sites. Each problem entity has multiple owners providing services, i.e., the truck fleet contains trucks from multiple transportation companies. A decentralised approach can facilitate security by not sharing information at a centralised location. Therefore, each organisation can protect its privacy and stakes while sharing only the necessary information [2]. However, devising a large-scale approach that can adapt to the dynamic environment and is decentralised and distributed is arduous.

Most existing approaches for the RMC problem generate a schedule at once, not adapting to the dynamics of the environment. These traditional approaches use centralised combinatorial optimisation techniques. With our problem definition (detailed in Section 2.1), RMC is an NP-Hard problem [3,4]. Although for a small scale (4 to 6 trucks making 15–20 deliveries) and with a simplified model of the problem, optimal solutions are found [4], the rest of the existing approaches are heuristic-based, yielding valid sub-optimal schedules (see Section 5.1, Related Work). Considering the main challenges of hard constraints, large scale, and dynamism, none of the current work addresses the RMC problem. Moreover, to our knowledge, the RMC problem is never addressed using a MAS approach; hence, no advantages of using MAS are ever gained.

Cooperative MASs are often studied for developing decentralised approaches that address Coordination and Control (C&C) applications involving distributed decision making. In C&C applications, agents control entities in an underlying physical environment, such as in UAV control, logistics [5], and multi-robot patrolling. In these applications, the environment is dynamic; entities may break, new entities may appear, or the topology may change. Therefore, designing coordination mechanisms for distributed decision makers (agents) that scale well, are adaptive, and are decentralised is challenging in such applications.

We address the dynamic RMC problem by considering it to be a C&C application and designing a cooperative MAS. In our MAS, each agent is mapped with the problem entities (i.e., orders and trucks). Agents generate schedules by coordinating via Delegate MAS [6]. It is a bio-inspired coordination mechanism that focuses on decentralisation and distribution. Agents make these schedules by interacting with each other while maximising their utility. The inter-dependent deliveries of orders alongside hard constraints lead us to introduce agent teams. Making temporary teams with other truck agents means more orders are fully completed, and more stable schedules are generated.

We present an empirical study of a dynamic decentralised C&C approach for the dynamic RMC problem and assess the added value of using teamwork with Delegate MAS. We designed two coordination mechanisms; one uses Delegate MAS without teams, and the other uses Delegate MAS with teams. These mechanisms are thoroughly evaluated using simulations in dynamic environments by varying the problem scale and load on a typical day. Our empirical study compares our two approaches with a centralised hyper-heuristic approach and Greedy MAS.

The main contributions of this work are the following:

• We use the cooperative MAS for the large-scale dynamic RMC problem to generate schedules dynamically. It is a novel approach in which a scheduling problem is decomposed not done in any previous works. Consequently, (a) our approaches adapt to the dynamic environment. (b) They are suitable when decentralisation/distribution is required and do not require everything to belong to a single organisation and share all the information in one place. (c) We can schedule large-scale problem scenarios.
• We develop an RMC input generator that generates the RMC input scenarios based on real-life data. It is available as open-access software [7].
• A feature-based evaluation is performed for the first time for the RMC problem. We characterise problem scenarios into stress, dynamism, and scale. It allows us to identify the strengths and weaknesses of a scheduling approach while considering the complexity, difficulty, and scale of an input scenario.
• In the context of agent coordination research, we introduce teams on top of the Delegate MAS. The agents are committed to the team, resulting in the completion of more orders.
• In the context of teamwork research, we combine MAS with decentralised coordination to dynamically generate overlapping co-existing teams. Using BDI-agents, such a composition has not been investigated before.

The rest of this paper is organised as follows. Our case study, the dynamic RMC problem, is described in Section 2. In Section 3, we report the two proposed coordination mechanisms and the two existing approaches used for the comparison. Section 4 details the ment setup and evaluation of all the coordination approaches. Related work is discussed in Section 5, followed by concluding remarks in Section 6.

2. Case Study—The Dynamic RMC Problem

Dynamic RMC is a scheduling problem similar to pick up and delivery problems (PDPs) [8,9]. It considers a set of orders that must be fulfilled by delivering concrete from several production sites (PSs) to order sites (OSs) using a fleet of concrete delivery trucks. One delivery refers to a round trip of a truck (Figure 1); it gets loaded at PS, travels to OS for unloading the delivery, and travels back to a PS to get washed and loaded again.

Typically, the ordered amount of concrete for an OS is more than the capacity of a single truck, necessitating a sequence of deliveries by multiple trucks. There is a hard constraint that concrete must not remain in a truck for more than the concrete perish time ($CPT$, Table 1). A truck must always be loaded to its total capacity; otherwise, the $CPT$ is disturbed due to the lower quantity. The problem is dynamic because orders arrive dynamically; additionally, unexpected events, like truck breakdowns, traffic jams, or accidents at OS, may occur.

Some general terms and concepts related to the dynamic RMC problem are listed with their abbreviations in Table 1. The typical values of these terms used as simulation parameters are selected based on consultation with a European concrete delivery company.

2.1. Formal Notation and Problem Description

To describe the details of our approaches and their evaluation, we need to define some notations formally. The following passages define notations and roles related to each problem entity.

2.1.1. Production Site

A production site (PS) services one truck at a time by cleaning and loading it. Because of the fully automated processes of today’s PSs, it takes as few as 2 min (see $CLT$ in Table 1) to service a truck [10].

In our notation, set P represents a set of PSs:

• $P = \{p_0,p_1,...,p_n\}$

2.1.2. Orders

An order represents information about (a) the required quantity of concrete to be delivered to an OS, (b) the unloading time for the first delivery (called start time of the order), and (c) the location of the OS. Orders are represented as follows:

• $O=\{o_0,o_1,...,o_n\}$, set of orders
• $o_i^q$ = quantity of order $o_i$
• $o_i^{st}$ = start time for order $o_i$
• $o_i^{wq}$ = wasted concrete for order $o_i$ in addition to the required concrete that is served.

Order delivery: For multiple truck deliveries required by order $o_i\in O$, the notation is as follows:

• $o_{i}D$ = $\{o_i{d}_0,o_i{d}_1,...,o_i{d}_n\}$, set of deliveries of an order $o_i$
• $o_i{d}_k^{ut}$ = Unload time of $k^{th}$ delivery of order $o_i$
• $o_i{d}_k^q$ = quantity of $k^{th}$ delivery of order $o_i$

The deliveries of an order must be scheduled subject to the following constraints:

• The sum of concrete delivered by all n deliveries for an order $o_i$ should be equal to the total quantity $o_i^q$ ordered by order $o_i$, i.e.,
  ${\displaystyle \sum _{k=0}^n}{o}_i{d}_k^q=o_i^q$,     $\forall o_i\in O$, $\forall o_i{d}_k\in o_{i}D$
• Every delivery of an order $o_i$ should be started after the start time of an order $o_i^{st}$:
  $o_i{d}_i^{st}\ge o_i^{st}$,      $\forall o_i\in O$, $\forall o_i{d}_i\in o_{i}D$
• At the order site, one truck can be unloaded at a time, and as a result, there cannot be an overlap between two consecutive deliveries of an order $o_i$:
  $o_i{d}_j^{ut}<o_i{d}_k^{ut}$ AND $o_i{d}_k^{ut}>o_i{d}_j^{ut}+(o_i{d}_j^q/ULR)$,     $j<k,\forall o_i\in O$, $\forall o_i{d}_k\in o_{i}D$
• The time between two consecutive deliveries of an order $o_i$ should not exceed LT (Table 1), i.e.,
  $o_i{d}_{i+1}^{ut}\le o_i{d}_i^{ut}+(o_i{d}_i^q/ULR)+LT$,      $\forall o_i\in O$, $\forall o_i{d}_i\in o_{i}D$

2.1.3. Delivery Trucks

The delivery trucks are represented by set $V.$ Trucks start from a PS, referred to as their start location. Once loaded, a truck cannot serve multiple OSs and must return to a PS after serving a delivery.

The notation for delivery trucks is as follows:

• $V=\{v_0,v_1,...,v_m\}$, set of trucks
• $v_i^c$ = capacity for truck $v_i$
• $v_i^{tt}$ = total distance travelled by truck $v_i$

Truck schedule: A truck’s schedule lists delivery jobs it must carry out. A job is formally represented as follows:

• $v_{i}J$ = $\{v_i{j}_0,v_i{j}_1,...,v_i{j}_m\}$, set of jobs of a truck $v_i$
• $v_i{j}_k^{st}$ = start time of $k^{th}$ job of truck $v_i$
• $v_i{j}_k^{et}$ = end time of $k^{th}$ job of truck $v_i$

We use $v_i{j}_0=o_x{d}_2$ to denote that truck $v_i$’s first delivery job is to serve order $o_x$’s third delivery. Figure 2 shows a typical schedule of a truck $v_i$.

2.2. Requirements

The main functional requirement for the dynamic RMC is to continually produce an updated schedule for delivery trucks, similar to Figure 2. The schedule of each truck is a list of delivery jobs that adapts according to the dynamic environment.

The non-functional requirements include the following: (1) Consistent performance with increasing stress and scale; stress refers to the amount of concrete required to be delivered by a specific number of trucks, while scale defines the size of the problem (details in Section 4.1). (2) Adaptive schedule in a dynamic environment; dynamism is caused by orders arriving dynamically and truck breakdowns at any time during the day. These breakdowns result in delivery failure. Despite dynamic events, we want to avoid unnecessary changes in the truck schedules (details in Section 4.1). (3) Maximising the number of fully served orders; when a truck fails to deliver, other trucks should gracefully handle the failed delivery.

Our concern for the dynamic RMC problem is to serve a maximum number of orders at minimal cost. The optimisation objectives for the dynamic RMC problem are listed below, and the assigned weights are obtained from a concrete delivery company in Belgium:

• Start time delay ($STdelay$); $w_l=10$.
• Wasted concrete; $w_w=10$.
• Truck travel time; $w_t=20$.

The scalarising of the multi-objective optimisation [11] results in minimising the following equation:

$$Objectivefunction=w_l\times \left(STdelay\right)+w_w\times \left(wastedConcrete\right)+w_t\times \left(truckTravelTime\right)$$

This implies that

$$Objectivefunction=\sum _{i=0}^n{o}_i^{st}{w}_l+\sum _{i=0}^n{o}_i^{wq}{w}_w+\sum _{k=0}^m{v}_k^{tt}{w}_t$$

(1)

where n and m represent the total number of orders and trucks.

3. Approaches Used to Address the Dynamic RMC Problem

This section explains all approaches by which we address the dynamic RMC problem. We first explain the Delegate MAS (DMAS) approach that uses only Delegate MAS as a coordination mechanism. Then, we highlight some situations in RMC where this DMAS approach could be improved using teams. We elaborate on the extension of DMAS by integrating teams and name it ‘Delegate MAS with Teams’ (DMAST). Lastly, the existing two approaches used for evaluation are discussed. These are the centralised hyper-heuristic (CHH) and Greedy MAS (GMAS). In our text, DMAS, DMAST, or GMAS refers to both the coordination mechanism and the whole C&C MAS approach.

3.1. Coordination via DMAS

In our simulation, the world is simulated as an environment where problem entities, like OS, PS and trucks, are situated and mapped to agents [12]. There are two types of agents, order agents and truck agents, representing the problem entities $o_i\in O$ and $v_i\in V$. Internally, the agents are modelled as BDI-agents [13]. Trucks are mobile, and for each truck, there is a truck agent to manoeuvre it. We consider that the truck drivers follow the truck agent. No agent exists on a PS, but a PS can communicate and store information.

Order agent: An order agent (we use the notation described in Section 2.1.2 is used for both orders and order agents) ensures that the desired quantity of concrete for its order is served without violating any of the constraints described in Section 2.1.2. According to $o_i^{st}$ of order $o_i$, at $t_0$, the order agent announces the time at which $o_i$ requires its first delivery. This announced time is called interested time as shown in Figure 3a. After the announcement, it waits for truck agent proposals for the delivery. For the best proposal based on the objective function (see Equation (1)), it replies with a $SEEMS\_OK$ message. All other proposals are replied to with a $REJECT$ message.

Further dialogue between the truck agent and the order agent, as shown in Figure 3b, is explained subsequently in this section. After scheduling delivery at $t_1$, if more concrete is required, $o_i$ will announce a new interested time and wait for truck proposals (Figure 3a). Thus, the order agent continues to schedule its deliveries until the truck agents book all the required concrete.

Truck agent: A truck agent (the notation introduced in Section 2.1.3 is used for both trucks and truck agents) is responsible for creating the schedule for its truck while taking into account the objective function defined in Section 2.2. It explores the environment to find orders to book delivery jobs for empty time slots in its truck’s schedule. Once an appropriate order’s delivery is found, it sends a proposal. The order agent replies to this proposal with a response of either $REJECT$ or $SEEMS\_OK$ as shown in Figure 3b.

Interaction between agents: In DMAS, agents coordinate using Delegate MAS, a bio-inspired coordination technique between agents. Delegate MAS enables a system to adapt to the dynamic environment without centralised control. The coordination is carried out via the environment: agents disseminate their information and collect relevant information from the environment [6]. This dissemination and collection are carried out by lightweight agents called ants. A Delegate MAS is responsible for managing ants, including their timing, frequency, initial state, and the results reported by the ants [14].

Adhering to the patterns of Delegate MAS [14], two types of Delegate MASs are used: exploration ants for collecting the relevant information from the environment and intention ants for communicating decisions taken by the agents. In both Delegate MASs, ants are sent periodically using the EXPLORATION_INTERVAL and INTENTION_INTERVAL. These are used as parameters in the simulator and defined in

Table 2. Simulation parameters.

No.	Parameter Name	Description	Value
1.	ORDER_INFORM_INTERVAL	Periodic interval for informing interested time to PSs.	90 s
2.	ORDER_INFORM_ EVAPORATION	Interval for evaporating information in Order Table at a PS.	ORDER_INFORM_ INTERVAL + 60 s
3.	EXPLORATION_INTERVAL	Periodic interval for sending exploration ants.	60 s
4.	INTENTION_INTERVAL	Periodic interval for sending Intention ants.	90 s
5.	INTENTION_EVAPORATION	Interval to evaporate intention when not refreshed by the Truck agent.	INTENTION_ INTERVAL + 60 s
6.	STDELAY_LIMIT	Time passed after which, due to not booking all concrete, the order agent decides to delay the start time.	10 min
7.	STDELAY_BY_ PERIOD	After STDELAY_LIMIT elapses, the time period by which the start time of an order is delayed.	15 min
8.	CPT	Concrete perish time (fixed by problem domain)	100 min
9.	ULR	Unloading rate per min (fixed by problem domain)	10
10.	CLT	Concrete loading time (at PS)—(fixed by problem domain)	5 min
11.	TEAM_THRESHOLD	Maximum delivery cost at which team member will make delivery	2

. We explain our coordination mechanism in three steps:

Step 1:

An order agent sends messages to those PSs from which it can receive a delivery without violating the $CPT$ limit (see Table 1) by limiting propagation beyond a specific radius. The radius is calculated based on $ULR$ (see Table 1) and travel distance. Consequently, each PS will have a list of order agents with their interested times and travel distances (to that PS), maintained in an order table. The interested time of an order agent continues to change until the order gets all its concrete booked (Figure 3a).

Step 2:

A truck agent uses Delegate MAS to explore the environment for delivery jobs. It sends exploration ants (ExpAnts) with a truck’s updated schedule to each PS. The activities conducted by each ExpAnt are shown in Figure 4a. At PS, the ExpAnt reads the order table. For each order in the order table, the ExpAnt verifies if its truck can serve the delivery job towards that order agent. If the ExpAnt finds the delivery feasible, it clones itself and moves towards the order agent. Then it inquires whether there would be any wasted concrete or lag time involved if the truck makes the delivery at the specific time called ‘proposed time’. The lag time is measured by the difference between the proposed time and the interested time, while for wasted concrete, the procedure in Figure 5b is used. The ExpAnt then adds the delivery job to the truck’s schedule and returns to a randomly selected PS. In the end, the ExpAnt returns to the truck agent.

Step 3:

The ExpAnts described in Step 2 are periodically sent by the truck agent after the EXPLORATION_INTERVAL. After each INTENTION_INTERVAL, a truck agent selects the best ExpAnt from the returned ExpAnts using the heuristic in Figure 5a. The truck agent then sends an intention ant (IntAnt) using the best ExpAnt’s schedule to propose the corresponding delivery job to the order agent. The activities performed by an IntAnt are shown in Figure 4b.

The truck agent periodically sends both types of ants. The order agent and the truck agent maintain the status of the delivery bookings. The algorithm the order agent uses to respond to IntAnts is shown in Figure 6. Suppose order agent $o_i$ receives proposals from truck agents $v_1,v_2$ and $v_3$ but selects the proposal of $v_2$. Order agent $o_i$ replies to the corresponding IntAnt of $v_2$ with a message of $SEEMS\_OK$ (Figure 6, Algorithm 2). For the IntAnts of $v_1$ and $v_3$, $o_i$ replies with a $REJECT$ message (Figure 6, Algorithm 1).

When truck agent $v_2$ receives a $SEEMS\_OK$ reply, it adds the delivery job to its schedule. On the next INTENTION_INTERVAL, $v_2$ refreshes its booking, and $o_i$ repeats Step 1 with the new $interested$ $time$ (Figure 6, Algorithm 3). After a successful refresh, the IntAnt returns to $v_2$ with $WEAK\_ACCEPT$. When all the concrete of $o_i$ is booked, the booking status at both sides becomes $STRONG\_ACCEPT$ (Figure 6, Algorithm 4). An abstract view of the interaction between an order agent and a truck agent for a single delivery is shown in Figure 3b.

Handling Dynamism:

In the case of a truck breakdown, the delivery jobs in its schedule cannot be fulfilled and are considered failed. The truck agent stops sending IntAnts. An order agent detects this failure since its booking is not refreshed. It adjusts its plan by announcing a new $interested$ $time$ and, if required, sends delivery job $REJECT$ messages to the other truck agents. Figure 7a shows that due to the failure of $o_x{d}_2$ at $t_1$, the order agent starts rebooking all deliveries that appear after the failed delivery. It announces the new interested time as $o_x{d}_2^{ut}$.

If order agent $o_x$ is unable to get all its concrete booked until $STDELAY\_LIMIT$ (Table 2), it changes its plan for $o_x^{st}$. Instead of starting at $o_x^{st}$, it starts the order at $o_x^{st}+STDELAYED\_BY\_PERIOD$. Order agent $o_x$ repeats Step 1, considering its new $o_x^{st}$. Whenever IntAnts visit the order agent to refresh the bookings related to the previous plan, they are responded to with $REJECT$ and thus are removed from the truck’s schedule.

3.2. DMAS Discussion and Challenging Situations

An initial version of DMAS with preliminary results is described in [15]. DMAS is a decentralised coordination mechanism since no entity maintains a global view of the whole system. Instead, it is decentralised: data, computation, and control are decomposed across entities. The underlying communication infrastructure used by agents is abstracted, considering that our MAS can be deployed both on a distributed infrastructure or in a single server setting.

Using DMAS, each truck agent only interacts with order agents and makes independent decisions. This fact is more considerable in the following challenging situations, where an order might remain unserviced using only truck level commitments:

1

When an order cannot be fully scheduled, the truck agents are least concerned, as they treat their delivery jobs individually. Therefore, it is not the case that a group of agents mutually believe they are performing a task (to fulfil an order) together. The delivery of an order with five deliveries booked and the final delivery remaining is considered equal to the delivery of an order with only the first delivery booked and five remaining.

2

In the case of failure of delivery $o_i{d}_k$ of order $o_i$, all of the truck agent’s other intentions for that order’s deliveries ($o_{i}D$) are in vain (illustrated in Figure 7a). Therefore, the order agent needs to re-plan the failed delivery as well as the deliveries following it.

3

If truck agent $v_x$ is made to decide based on its gain or loss to remove a delivery job $v_x{j}_2=o_i{d}_1$ from its schedule and make some other delivery, it cannot make a correct decision. This inability is because the truck agent lacks knowledge about the loss that order $o_i$ may bear because of the dropping of $v_x{j}_2=o_i{d}_1$, thus disregarding the welfare of the whole system.

These challenging situations motivated us to introduce teams among the truck agents, detailed in the next section.

3.3. Coordination via DMAST

In this section, we report on an extension of DMAS by adding teams and name it Delegate MAS with teams (DMAST).

In DMAST, teams are formed dynamically. Initially, agents coordinate like DMAS. Later, teams keep developing on top of DMAS to make truck agents more cooperative. Our teams are temporal and overlapping. Temporal means that agents are not a permanent part of a team throughout their life cycle. Overlapping means that agents are part of more than a single team simultaneously.

Using DMAST, when order $o_x$ has booked most of its deliveries, and only 20% of concrete is remaining, the order forms team $o_{x}Team$ of trucks that book its deliveries. These truck agents of the team commit to serving $o_x$ at the booked times. They agree on the team’s response if any member fails to perform their job (refer to point 3 in Section 3.2).

Consider two scenarios: the last delivery fails to get booked by any of the trucks, and the delivery fails due to truck failure (the order notices this failure, as its booking does not get refreshed). In both scenarios, $o_x$ seeks help from team $o_{x}Team$ to get itself fully served.

Handling dynamism:

Let us explain the DMAST by giving an example of a truck failure. When truck $v_i$ fails, the future delivery jobs it booked need to be reassigned to other trucks (Figure 7b). Suppose the failed truck has three jobs in its schedule with the following statuses:

• $v_i{j}_0$ = $o_2{d}_3$   $->SEEMS\_OK$
• $v_i{j}_1$ = $o_1{d}_3$   $->$ $WEAK\_ACCEPT$
• $v_i{j}_2$ = $o_4{d}_2$   $->STRONG\_ACCEPT$

Here, orders $o_2$ and $o_1$ wait for the intention refresh from $v_i$ but after waiting for a refresh interval of INTENTION_EVAPORATION (Table 2), they consider it a failure and readjust their $interested$ $time$. Order $o_4$ is fully booked, and the team of trucks $o_{4}Team$ is already formed. Therefore, $o_4$ asks for help from the team members (Figure 7b). A special smart message, called commitment ant (ComAnt), is used by $o_4$ to interrogate the cost it takes a truck $v_x$$\in o_{4}Team$ if the truck agent performs the failed delivery. This cost depends on the travel cost and delivery cost.

The travel cost depends on travel distance to a PS for loading and $o_4$ for unloading. The delivery cost depends on the job status that must be sacrificed if the failed delivery $o_4{d}_2$ is taken by $v_x$. More specifically, see the following.

Schedule is free	$=>$ Delivery cost = 0
$SEEMS\_OK$ job OR $WEAK\_ACCEPT$ job (delivery time is >2 h ahead)	$=>$ Delivery cost = 1
$STRONG\_ACCEPT$ job OR [$SEEMS\_OK$ job OR $WEAK\_ACCEPT$ job (delivery time is <2 h ahead)]	$=>$ Delivery cost = 2
$STRONG\_ACCEPT$ (job starts within hour)	$=>$ Delivery cost = 5
If two jobs	$=>$ Delivery cost = cost1 + cost2

While assigning delivery costs, DMAST differentiates if more than 2 h remain in delivery times, as if the delivery time is close, rescheduling that delivery after dropping will not benefit the whole system.

On receiving all the costs from $o_{4}Team$, $o_4$ selects a member using the heuristic given in Figure 8b. The selected truck agent $v_x\in o_{4}team$ puts the assigned job in its schedule, and this job starts to become refreshed along with the other delivery jobs of the truck agent. The dialogue between the order and team members is shown in Figure 8a. We use the $TEAM\_THRESHOLD$ value to select the maximum delivery cost value for a team member to schedule the failed delivery. If none of the $v_x\in o_{4}Team$ can take the failed delivery, then the delivery is open to all trucks for booking, similar to DMAS. That is to say, the interested time will be announced, and potentially all trucks can propose that delivery.

When a team member removes its existing job, if it is a $SEEMS\_OK$ or $WEAK\_ACCEPT$ job, the related order also notices that its booking is not refreshed and resets its interested time accordingly. If it is a $STRONG\_ACCEPT$ job, the order will consider it a failure and propose it to its team.

3.4. Insights about DMAS and DMAST

DMAS and DMAST are decentralised coordination mechanisms, and the main difference is teamwork when DMAST trucks cooperate within a team. In DMAS, failures are handled by offering the failed delivery to a wider audience of trucks, resulting in a broader search space. However, truck agents neglect the failed delivery job due to having no commitments.

Contrary to DMAS, in DMAST, the failed delivery job is only presented to the team members. Since team members are more committed and may sacrifice their current jobs, truck agents would be better able to deal with the dynamism. If the failed delivery job is infeasible for the team members, then just like DMAS, it is offered to all the trucks. Consequently, the search space is first reduced, but if necessary, it is expanded again, making the approach more adaptive in a dynamic environment.

In DMAST, when there is a dynamic event, only one truck agent’s schedule is disturbed because of the committed agents of a team. However, in DMAS, several truck agents may get disturbed as discussed in Section 3.1. Moreover, introducing the $TEAM\_THRES-HOLD$ parameter allows us to fine tune the over-commitment of trucks [16].

Handling Challenging Situations of DMAS:

The challenging situations discussed in Section 3.2 are all tackled in DMAST. For situation 1, by adding teams, if a single delivery of an order is remaining and not getting booked, it will be picked up by the team members. In the case of situation 2, the team members in DMAST are committed to serving the failed delivery and struggle to accommodate the failed delivery in their schedules. Thus, the orders having a delivery failure have more chances to become served without disturbing other deliveries.

Handling situation 3, in DMAST, the cost calculation based on the status of a delivery job gives the necessary knowledge to make the correct decision. The status messages of $SEEMS\_OK$, $WEAK\_ACCEPT$ and $STRONG\_ACCEPT$ play a role in decision making while handling time-critical tasks, increasing the system’s reliability.

3.5. Other Approaches for Evaluation

To evaluate the effectiveness and performance of our approaches, we compare DMAS and DMAST with an existing centralised optimisation approach and with a simple MAS-based approach. Both approaches are described below.

1 

Centralised hyper-heuristic: Centralised hyper-heuristic (CHH) is a combinatorial optimisation approach that solves the static version of the problem [17]. As discussed in Section 1, RMC is an NP-Hard problem at the scale we are addressing. Therefore, this problem is solved by relying on heuristics. We use this CHH approach, presented as the best hyper-heuristic approach amongst the five other approaches in [17], originally presented in [18]. Amongst all other existing approaches for RMC discussed in the related work section, only CHH uses a detailed problem definition similar to ours, which is taken from an RMC production company in Belgium. Therefore, we give it the same input instances as our approaches. However, the CHH approach does not use dynamism; it is a static approach, and its results are based on one-step execution.

2 

Greedy multiagent system: Greedy multiagent system (GMAS) is a decentralised MAS that handles the dynamic RMC problem like DMAS and DMAST. It is a basic version of MAS for RMC, on top of which we use DMAS and DMAST as coordination mechanisms. Thus, it does not follow a specific coordination approach; every truck agent books the delivery based on availability. The truck agents close to PS book the delivery on a first-come first-served basis, and order agents accept the first proposal that arrives; hence, we name it Greedy MAS. It is the current norm in the industry, as trucks are kept in abundance to handle customer requests instead of optimising the schedules of minimal trucks. When a truck fails, its deliveries are available at PS to be scheduled by the other trucks. If any other truck is available, it will book that time slot. Otherwise, the failed delivery will remain unbooked, and the order will remain partially completed.

4. Evaluation

We evaluate DMAS and DMAST by comparing them with each other and by existing approaches. We implement them using the Rinsim [19] framework and run a variety of problem instances. Figure 9 shows a screenshot of the simulation.

4.1. Input Instances and Characterizing RMC

An input scenario given to our simulator is named an input instance. An input instance contains problem details mentioned in Section 2, except the details about order deliveries and truck schedules, which are generated as output after scheduling. These input instances generated by an input instance generator are based on real-life data from an RMC production company in Belgium. The details about the input instance generator can be found in [7,20]. We generate these input instances due to the limitations of real-life data, discussed at the end of this section. They encompass all the real-life data. A regular practice (evident from hundreds of research articles) for performance evaluation of PDPs is that they are defined as a “number of jobs” or “number of vehicles”. However, we perform theoretical analysis on the RMC problem and characterise each input instance explicitly based on three dimensions.

These three dimensions are discussed below:

(a)

Scale: Scale gives a notion of the problem size, and we use the number of trucks to express scale.

(b)

Dynamism: It captures the number of events happening throughout the day during scheduling. For instance, if the scale is 20, then 20% dynamism means four trucks may fail during the day. Orders also arrive dynamically, and each order $o_i$ is announced two hours before its $o_i^{st}$.

(c)

Stress: Stress is a theoretical measure based on the average concrete a truck can deliver in a day. On average, each truck takes approximately two hours to complete a delivery. Thus, on a typical day of 16 h, a truck can serve a maximum of 8 deliveries. For instance, a scale of 6 means six trucks have the potential to serve approximately 48 delivery jobs. However, due to time constraints, it is seldom the case that trucks can use their full potential. The stress of an input instance is calculated as follows:

$$Stress=\frac{Load}{(TotalNo.OfTruck\times DeliverableConcretePerTruckPerSecond)}$$

(2)

where

$$\begin{array}{c}\hfill Load=\frac{TotalAmountOfConcreteByAllOrders}{LoadedSecondsConsideringUnloadingRate}\end{array}$$

(3)

For example, an input instance $I_x$ has two orders $o_1$ and $o_2$ with

$o_1^{st}=$ 11:00$,o_1^q$ = 20 m${}^3$ and $o_2^{st}=$ 13:00$,o_2^q$ = 10 m${}^3$.

Therefore,

$$I_x^{load}=2.8{\text{mm}}^3/\mathrm{s}$$

Consider in $I_x$ that the truck capacity is 10 m${}^3$. The concrete demand requires three deliveries between 11:00 and 14:00. Figure 10a shows plots of the delivery required compared with the deliverable concrete with a different number of trucks. Realistically, a truck can deliver concrete at the rate of $5 {\mathrm{m}}^3$ per hour, with each delivery taking approximately two hours. Therefore, the value of stress is

$$I_x^{Stress}\text{ considering }1\text{ truck}=2.0$$

$$I_x^{Stress}\text{ considering }2\text{ trucks}=1.0$$

As inferred from the horizontal line in Figure 10b, typically, stress of 1.0 places a manageable load on the trucks. Defining stress helps us to compare two input instances with different total concrete amounts required but an equal number of trucks.

The real-life data that we obtained from the RMC company have some limitations. For instance, if we want to evaluate our approaches across scale, the real-life data instances need to be consistently increasing, like 4, 6, 8, …, 20 trucks. In reality, the real-life data have 4, 12, and 14 truck fleets available. Thus, we generate more instances to fill these gaps from various dimensions to cover the whole scale range. Moreover, we have a random number of instances in real-life data for each scale–stress value pair. Thus, we make five input instances of each type for a more justified evaluation. We use the simulation approach as used in different application domains by domain experts for testing the hypothesis; one such example is given [21].

4.2. Evaluation Metrics

The metrics used for evaluation are related to measuring performance and schedule stability and are defined below:

(a)

Delivered concrete: It expresses the percentage of concrete that is delivered by the trucks from the total concrete ordered by all orders.

$$\begin{array}{l}percentageOfConcreteDelivered=(concreteDeliveredByTrucks-\\ wastedConcrete)/totalConcreteOrderedByOrders\end{array}$$

Truck breakdowns will cause less concrete to be delivered.

We calculate a single value for the objective function for each simulation run. We analyse performance differences by comparing these values at identical problem sizes (scale) for all approaches.

(b)

Completed orders: It represents the trucks’ percentage of fully completed orders. Some orders may not be served fully due to the dynamic environment and high stress.

(c)

Number of disturbances: Finally, the stability of schedules is the measure of disturbances faced by individual trucks. It depicts the sum of changes made in each truck’s schedule due to a dynamic event and is calculated as follows:

$$\begin{array}{c}\hfill NoOfDisturbances=\sum _{k=0}^m(SEEMS\_OKmessagesRemoved+\\ \hfill WEAK\_ACCEPTmessagesRemoved+\\ \hfill STRONG\_ACCEPTmessagesRemoved)\end{array}$$

where m represents the total number of trucks. This metric is important because if trucks keep changing delivery schedules, their routes keep altering. Additionally, order sites will have to keep changing their internal operations schedule.

4.3. Experiment Setup

All the approaches are tested under an identical experimental setup. An input instance is a combination of three sets O, V and P, all situated in a region of 50 × 50 km${}^2$, approximately equal to 1/12th of the area of Belgium. For all input instances, there are three PSs at fixed locations, ensuring the coverage of the whole region as shown in Figure 9. Sets O and V vary for each input instance. Table 2 lists parameters related to our approaches. The parameter sensitivity analysis is used to tune these parameter values [16].

Instead of generating a whole schedule of trucks in a single shot, the simulation runs continuously and adapts schedules throughout the day from 6 AM to 10 PM. Figure 11 shows a sample of concrete load during the day. The transported concrete exceeds the required demand due to wasted concrete.

Based on the requirements of addressing dynamic RMC and characteristics of input instances, we run two kinds of experiments.

(a)

Stress experiments: In a stress experiment, we investigate the performance of the four approaches while varying stress. The other two characteristics, namely scale and dynamism, are kept fixed. All the performance criteria (see Section 4.2) can be compared by varying stress and keeping the size of the problem constant.

(b)

Scale experiments: For the scale experiments, we investigate how all approaches cope with increasing the problem size while keeping stress and dynamism fixed. Not every aspect can be compared using the results of scale experiments. For instance, it is not very sensible to compare objective function values of two different scales with each other.

For all experiments, dynamism is fixed to 20, which means 20% of trucks are prone to fail during the day. For stress experiments, the values used for stress range from 0.4 to 1.6. For each value of this range, we use multiple scale values. However, since trends are similar, plots are shown against scale 14 only.

The range of scale values for scale experiments is 6 to 20, and for each value of scale, multiple stress values are used. Due to similar trends, the result is only shown for stress 1.0, considering 1.0 as a reasonable stress value as discussed in Section 4.1. Five input instances are used for each scale–stress value pair. Each input instance is run 20 times with a different random seed to minimise the effect of outliers. We use simulations with extensive experiments as used by many domain experts in various fields of sciences such as these [22,23,24,25].

4.4. Results

This section details the experiments conducted with input instances to evaluate and compare all four approaches. Each dot in the plots of Figure 12a, Figure 13, Figure 14, Figure 15, Figure 16 and Figure 17 indicates an average of five input instances of identical characteristics, with each instance run 20 times. The CHH experiments are carried out on Intel Core i5, 1.8 GHz with 8 GB RAM, using the JPPF platform (Java Parallel Processing Framework). Each instance is tested ten times, and the time limit used as a stopping condition is generously made to be 1 h, though in the original work it was 30 min.

Figure 12a shows the percentage of concrete successfully delivered during a day by each approach. DMAST and DMAS consistently deliver more than 82% of concrete demand during the day. Both approaches show more than 50% improvement over GMAS, which delivers close to 60% when scale is less than ten. We find that as a centralised and static approach, CHH delivers 100% concrete at each scale.

In the stress experiment of Figure 12b, we notice that as stress increases, the performance starts decreasing, as it becomes harder for the same number of trucks to deliver the increasing amount of concrete. While GMAS shows the worst performance, our approaches DMAST and DMAS manage to achieve around 80% concrete delivered, even with stress higher than 1.0. CHH showing 100% concrete delivered even at high stress hugely compromises on STdelay, which is discussed while discussing Figure 15.

Figure 13 gives the evaluation results related to the objective function obtained by the four approaches when stress increases. Although the four approaches are developed to minimise the objective function, the upward trend in the objective function is due to the additional orders that need to be managed with an increase in stress value. These additional orders result in a higher amount of delivered concrete, start time delay and travel time.

For DMAS and DMAST, after stress 1.4, the objective function values stabilise, showing that the trucks cannot meet the increasing demand for concrete. DMAST improves DMAS by 8% concerning the optimisation cost. Apparently, GMAS shows good performance with decreased values, while CHH has the toughest time managing stress. We investigate further by analysing the number of deliveries alongside the objective function. The total number of deliveries versus stress differs considerably across approaches (Figure 13b). CHH has the highest number of deliveries, even with 1.6 stress levels, which is one of the reasons for the steepest objective function curve in Figure 13a.

This finding implies that the objective function alone is unreliable for conclusive opinions. Therefore, we normalise the objective function with the number of deliveries in Figure 14, which depicts the per delivery cost incurred by each approach. We find that DMAST and DMAS show around 70% improvement over GMAS at minimal stress 0.4 and more than 50% improvement at stress 0.6 to 1.2. There is up to 380% improvement over CHH, though the objective function by CHH is unexpectedly high.

Investigating profoundly, we analyse all approaches’ elements of the objective function. We find that CHH has very high values of objective function and delivers 100% concrete because it compromises hugely on STdelay as shown in Figure 15a. STdelay is plotted against stress after normalising it with the number of deliveries. CHH delays up to 366 h at stress 1.0. In the zoomed plot of Figure 15b, we also notice that DMAST has more STdelay as compared to DMAS (Figure 15b), which is discussed in the subsequent passages. However, DMAS and DMAST make it possible to deliver on the same day while making dynamic decisions.

Figure 16a and Figure 17 are the plots which show the difference in performance between DMAS and DMAST; both seem to have equal performance otherwise. Because of the dedicated teams in DMAST, it can fully complete approximately 80 to 85% of orders. On the other hand, DMAS completes 70 to 75% as DMAS considers the deliveries individually; more orders are started instead of completing orders that have booked deliveries earlier. By examining the output data files, we find that in DMAST, more than 70% of the failures are handled by teams. So overall, the number of deliveries is identical, but in the case of DMAS, these belong to a wider range of orders. Thus, on the scale ranging from 6 to 20, DMAST consistently completes 10 to 15% more orders than DMAS. GMAS is the worst at completing orders, and barely 25% of orders are fully completed.

Similarly, in the stress experiments shown in Figure 16b, we find that DMAST can complete 8% to 15% more orders than DMAS. However, previously in Figure 15, we observe that this higher number of completed orders has a trade-off on STdelays. So DMAST delays orders but ensures that the maximum number of orders are completed.

The other area in which we notice DMAST outperforming DMAS is the number of disturbances while scheduling (Figure 17). Since DMAST tends to find the failed delivery within a team, other trucks’ schedules’ are less disturbed, as they do not receive a $REJECT$ message for already existing bookings. Consequently, using DMAST, we have up to 20% more stable and reliable schedules than DMAS.

The results in this section indicate that our two dynamic approaches can cope well with the constrained scheduling problem. The challenges faced by the simple GMAS highlight that a good coordination mechanism is required for agents to solve such a constrained scheduling problem cooperatively.

5. Related Work

We position our work from three different perspectives. At the heart of this study is the RMC problem; we briefly discuss existing approaches to address this problem. Next, since DMAS and DMAST are fundamentally two coordination approaches between agents, we state the existing work related to decentralised coordination. Lastly, we relate our work with existing teamwork approaches.

5.1. The RMC Problem

The RMC problem belongs to the category of PDPs as classified in [9,26]. A wide variety of solutions for PDP are based on combinatorial optimisation techniques [9,27]. The constrained nature of the RMC problem makes the solution computationally expensive. If the problem model is not simplified and large-scale, like in our case, it is proved NP-Hard [3,4,28]. Therefore, other approaches are used, including heuristics, meta-heuristics, and hyper-heuristics [18,29]. Some efforts for the RMC problem use stochastic models [30] or genetic algorithms [31] as well. For small-scale applications with simplified constraints, integer-programming based exact methods are also proposed [4,32].

All these approaches are useful for solving static problems which generate one-shot output but cannot effectively handle the changes that occur in a dynamic problem.

For handling dynamism, a common trend found in PDP is to manipulate data available at a certain moment in time using a mathematical model. As new data become available, instead of recomputing the mathematical model, they re-optimise the solution using meta-heuristics [33,34]. To the best of our knowledge, the only works that address partially dynamic RMC are those by Hoffman and Durbin [35], Zhang et al. [36], and Garza [37]. Hoffman and Durbin describe a planner component that plans for already known information and a runtime component capable of handling dynamic events. Zhang et al. offer a dynamic rescheduling strategy for RMC that incorporates the dynamically arriving orders. Garza recently presented an industrial case study targeting the whole business model, including staff scheduling of a specific RMC company with re-optimisation strategies. In our approach, orders arrive dynamically, and we also handle dynamic events like truck breakdowns or problems at the order site. Thus, our approaches are much more responsive and adaptive compared to existing works, as they keep updating schedules at runtime.

From the scale perspective, though several approaches address large-scale PDPs s [9,26], RMC is scarcely considered at a large scale. Maghrabi et al. solve large-scale instances using IP and MIP with simplified constraints [38]. Liu et al. have a couple of instances that deliver close to 50 deliveries to 8 sites [3]. Moreover, in another work by Maghrabi et al., more than 40 trucks make 346 deliveries, using a column generation approach solving RMC with and without time windows [32]. Even with time windows, the time lags between the deliveries remain undiscussed; thus, some constraints are relaxed. On the other hand, one of the benefits of using decentralised cooperative MAS is that we can schedule large-scale input scenarios without relaxing the constraints. In our approaches, tens of trucks make hundreds of daily deliveries in a dynamic environment while abiding by hard constraints.

We find that no existing approach for the dynamic RMC problem uses a MAS approach like ours. We address dynamic RMC without static components, centralised data or control and cope well with the dynamic events during the day. Thus, it is a responsive, decentralised and distributed approach, where schedules adapt according to the dynamic events during the day.

With recent developments, when autonomous and AI models are emerging, MAS systems are gaining more attention for creating a blend of AI and autonomous agents. A very recent work by Abderraouf Maoudj et al. uses the MAS approach for AGV scheduling in a complex environment [39]. As the RMC problem is a sub-category of PDP, we also want to mention one of the recent works that addresses PDP using the MAS approach by Johan Los et al. [40].

Additionally, in existing research, it is rarely found that input data are categorised for evaluation. A traditional characterisation of PDP is related to the distribution of orders in the geographical region, initially proposed by Solomon [41] and later extended and followed by many researchers [9]. Some others used carefully designed randomly generated input [9,27], or real-life data [42]. No scheduling approach characterises input for the load on available resources as defined by the stress or with other dimensions of scale and dynamism. This characterisation facilitates drawing meaningful conclusions as elaborated in Section 4.

5.2. Decentralised Coordination and Ant-Based Algorithms

MASs provide a modelling abstraction for building distributed and decentralised solutions without requiring a centralised mathematical model [43]. However, existing MAS based approaches for PDP or RMC also rely on centrally maintained global information [34]. Our MAS is decentralised; neither a single entity maintains the global information nor are the dynamic events updated at a centralised location. Moreover, from a control perspective, our approach is distributed; no single entity acts as a controller.

We used Delegate MAS as a foundation of our decentralised coordination related to ant-based coordination mechanisms. Therefore, here, we relate our work with other similar approaches. Over the last two decades, considerable research has been conducted using decentralised mechanisms inspired by stigmergy and swarm intelligence [44]. Examples include ant colony optimisation (ACO), particle swarm optimisation, and bee colony optimisation.

ACO slightly resembles our approach since ants in nature inspire both mechanisms. However, the behaviour of ants and the concept of pheromone is different in the ACO approach. Rizoli et al. present an online ACO algorithm incorporating dynamic change by evaporating old information and reinstating the new change in the schedule [45]. In our approach, the intention ants resemble these stigmergic mechanisms since ants drop intentions, which, if not refreshed, evaporate over time. Nevertheless, there is no probabilistic decision making involved. Our exploration ants are different and are more related to smart messages [46], having explicit behaviour and states [14]. Since our implementation of Delegate MAS is performed according to [14], where coordination is delegated to a component within an agent, we consider it rather unique compared to the other stigmergic approaches.

Santos et al. propose an ACO-based approach in a realistic domain that uses stigmergy, where real-life entities are mapped to agents [47]. However, we do not use roadside pheromones. Rather, information is stored at the PS or order agent. Moreover, Delegate MAS has recently been used in mobile robotics for route planning, AGV trajectory and schedule planning [48,49,50].

5.3. Teams in MAS

Teamwork is a major topic in research for MAS, distributed computing and game theory. A recent review discusses some teamwork frameworks application independently [51].

Research in teamwork has two dimensions: the teams related to cooperative MAS [52] and the coalition from game theory. Our work is related to teamwork and MAS. We took inspiration from the early theories and frameworks of teams [53] and used teams of BDI-agents. Many of these earlier theories and frameworks consider agents working in a single team, but in our case, multiple teams co-exist.

Although some works in teamwork use DCOP, DEC-POMDP or other approaches [52], researchers are returning to BDI-agents [13] by presenting hybrid approaches for MAS teams [54]. An example of a hybrid team involving humans for disaster management is a work by Reily et al. [55].

In our work, multiple teams co-exist and overlap, which means agents are part of multiple teams simultaneously. Close to our teamwork, various multiagent task allocation approaches use agent teams to schedule tasks. Scerrie et al. use the term extreme team depicting a large group of agents working in a dynamic environment with overlapping teams [56]. They used a DCOP-based algorithm with potential tokens to partially commit agents to tasks with other dependencies. Like our $SEEMS\_OK$ and $WEAK\_ACCEPT$, an agent uses potential tokens to commit that if inter-dependencies are sorted out, it will accomplish the task.

We form teams dynamically using a decentralised algorithm DMAST. Close to our effort is work by Ramchurn et al. [57]. They use DCOP agents in contrast to our BDI-agents. Due to BDI-agents, our teams are formed intentionally by agents, unlike Ramchurn’s work, where teams are formed implicitly without agents being aware. However, like Ramchurn, we have spatial and temporal constraints in different contexts.

6. Conclusions

This work addresses the dynamic RMC problem using the decentralised MAS by proposing two approaches, DMAS and DMAST. In both approaches, agents coordinate to generate better schedules. In DMAST, additional advantages in scheduling deliveries are achieved by extending DMAS (based on Delegate MAS, the bio-inspired coordination mechanism) with teamwork. Existing work on the RMC problem never used a MAS approach and thus could not take advantages offered by a MAS.

Additionally, we introduce the characterisation of real-life RMC data into stress, Dynamism and scale and generated input instances. Particularly, no scheduling approach is evaluated by considering problem size versus available resources as defined by stress. As a by-product, an open-source input generator is developed.

In this paper, we also contribute by comparing our work with existing approaches not done by other works on RMC. Results show 40% to 70% improvements over the greedy approach on various metrics. DMAS and DMAST can deliver more than 80% of concrete demand on the same day, while the existing hyper-heuristic approach delays orders to multiple days. Thus, both approaches are outstanding choices for large-scale, stressful environments and small-scale, less-load scenarios.

Our novel way of generating RMC schedules will contribute to the dynamic scheduling of other large-scale constrained problems. We decompose the dynamic RMC problem such that control is distributed and only necessary information is shared across multiple entities. This decomposition enables more privacy and security benefits and will assist in decomposing other NP-hard scheduling problems in new ways. Moreover, such decomposition can also benefit problems like green security [58] and emergency rescue [57].

Further studies on integrating multiple coordination mechanisms may increase our understanding of using teams in a decentralized and dynamic environment. In the future, we can improve an agent’s internal decision making. In our approach, occasionally, agents need to make an optimized decision based on their local information. Currently, we use simple heuristics, which could be substituted by an exact method.