Balancing Data Acquisition Benefits and Ordering Costs for Predictive Supplier Selection and Order Allocation

Abstract

The strategic selection of suppliers and the allocation of orders across multiple periods have long been recognized as critical aspects influencing company expenditure and resilience. Leveraging the enhanced predictive capabilities afforded by machine learning models, direct lookahead models—linear programming models that optimize future decisions based on forecasts generated by external predictive modules—have emerged as viable alternatives to traditional deterministic and stochastic programming methodologies to solve related problems. However, despite these advancements, approaches implementing direct lookahead models typically lack mechanisms for updating forecasts over time. Yet, in practice, suppliers often exhibit dynamic behaviours, and failing to update forecasts can lead to suboptimal decision-making. This study introduces a novel approach based on parametrized direct lookahead models to address this gap. The approach explicitly addresses the hidden trade-offs associated with incorporating forecast updates. Recognizing that forecasts can only be updated by acquiring new data and that the primary means of acquiring supplier-related data is through order allocation, this study investigates the trade-offs between data acquisition benefits and order allocation costs. An experimental design utilizing real-world automotive sector data is employed to assess the potential of the proposed approach against various benchmarks. These benchmarks include decision scenarios representing perfect foresight, no data acquisition benefits, and consistently positive benefits. Empirical findings demonstrate that the proposed approach achieves performance levels comparable to those of decision-makers with perfect foresight while consistently outperforming benchmarks not balancing order allocation costs and data acquisition benefits.

1. Introduction

Supplier selection and order allocation (SSOA) has always received considerable attention in the supply chain management (SCM) community [1]. Indeed, considering that an efficient selection of suppliers and a proper allocation of orders can reduce costs, improve product quality, and ensure timely delivery, its impact on the company’s performance is evident. For many companies, the cost of raw materials and components represents 50% to 80% of the final cost of the finished product [2]. Moreover, small deliveries can decrease sales by up to 10%, and delays of up to 45 days can result in an almost 40% loss in sales [3].

In addition to its relevance, SSOA has gained interest among scholars because of the challenges the problem presents. Properly assigning orders to selected suppliers in future periods requires knowing in advance future supplier performance. Uncertainty about the future is thus one of the main challenges this problem presents. Secondly, when orders need to be allocated over multiple periods, the problem presents in the form of a sequential decision-making one, where past decisions affect future periods as well. Lastly, the presence of uncertainty about future supplier forecasts and the sequential nature of the problem present a third challenge. When forecasts are related to supplier performance, there is a peculiar interdependence between the possibility of obtaining forecast updates and how orders are allocated to suppliers. Updating forecasts related to supplier performance requires sending orders to suppliers themselves, as this is the only possibility for acquiring new supplier-related data.

Due to the relevance of the problem and its challenges, researchers have started to develop mathematical formulations to solve it. First, approaches providing a solution for a single period in the future have been considered, and those dealing with multiple periods have also been generated. Similarly, deterministic models (i.e., not dealing with future uncertainty) have been investigated, while risk and uncertainty have been progressively considered. Indeed, multiperiod formulations that consider risk factors have become essential to effectively capture the dynamic nature of business. At the same time, the increased instability of supply chains has led to rethinking the SSOA not only to reduce cost but also to gain resilience and ensure business continuity. As a consequence, stochastic programming is now a widely adopted technique for building models for SSOA to cope with uncertainty.

However, recent advancements in machine learning (ML) and deep learning (DL) have opened the door to alternative solutions in the management of supply chain problems [4,5,6,7,8]. Indeed, existing approaches for multiperiod SSOA present some limitations when addressing its main challenges. First, deterministic models are not able to deal with future uncertainty about supplier performance. Secondly, according to [9], although stochastic programming presents several benefits, it is limited in its application to real-life problems due to the high computational complexity that some scenarios can generate. Lastly, even if reinforcement learning has been proposed for solving several allocation problems in the supply chain [10,11], it requires numerous interactions with the environment to learn the optimal way to allocate orders in SSOA, and, in real-world applications, this necessity is risky and expensive.

Conversely, direct lookahead models (DLMs) are a potentially effective solution for real industrial problems. They can deal with the necessity of estimating future supplier performance and not require learning how to allocate orders recurring to a trial-and-error strategy. Indeed, DLMs assume the form of linear programming models where future decisions are taken considering the forecast provided by predictive models. The possibility of integrating ML and DL models in the predictive modules of these approaches has recently led several authors to consider this approach [12,13,14,15,16,17].

However, despite the advancements made by these newly proposed approaches, the main motivation leading this work is that all these studies assume that forecasts remain stable over time. However, suppliers often change their performance due to unexpected events, making initial predictions unreliable. Adopting updating mechanisms in these forecasts thus represents a fundamental necessity to capture real-life dynamics. However, as previously mentioned, one of the challenges of the problem is that updating forecasts related to supplier performances requires sending orders to suppliers, as this is the only way to acquire new delivery data. From this perspective, new approaches have not been built to balance the trade-off between the benefits of data acquisition and the costs related to this acquisition (i.e., the cost of sending orders to suppliers). In the new AI era, where data are the fuel [18,19,20], approaches to solving the multiperiod SSOA problem thus need to be rethought explicitly to tackle this new trade-off.

Given the relevance of the topic and the lack of studies on this aspect, this study aims to offer two innovative contributions:

• The formulation of a new approach based on a parametrized DLM for solving the multiperiod SSOA problem, considering the trade-off between order emission costs and data acquisition benefits when forecasts related to the risk of non-punctual deliveries from suppliers are considered.
• An empirical evaluation of the advantages that the proposed approach can offer compared to benchmark approaches, representing, respectively, a decision-maker with perfect knowledge of the future and two decision-makers where classical DLM models are implemented without balancing the trade-off between data acquisition benefits and order emission costs.

The structure of this paper unfolds as follows: Section 2 provides an in-depth exploration of the research background, reviewing pertinent literature. Section 3 delineates the proposed approach, elucidating its constituent modules and expounding the experimental design employed to evaluate the efficacy of the approach. Section 4 presents the findings garnered from the experimental design. Subsequently, Section 5 discusses the results and encapsulates the conclusions drawn from this study.

2. Literature Review

This section presents literature on different deterministic and stochastic approaches used to solve the multiperiod SSOA problem when considering different risks. Afterwards, approaches based on DLM are reviewed to present the state of the art of these methods. Lastly, addressing the limitations identified in existing approaches, the main novelties of the proposed study are highlighted.

2.1. Deterministic and Stochastic Programming in Multiperiod SSOA

Various strategies have been explored to manage the risks within the framework of the multiperiod SSOA problem. One prevalent approach involves the development of mathematical models, often formulated as linear programming models, where risk parameters are expressed as deterministic values, such as rates or percentages. In several notable studies, including those by [21,22], risk has been quantified in terms of deterministic rates and percentages. For instance, ref. [21] assessed supplier quality risk by quantifying the expected defect rate of purchased components, while ref. [22] measured supplier delivery delays and quality risks as percentages of products delivered on time and rejection rates of products, respectively. An alternative approach demonstrated in the investigation by [23] involves representing supplier disruption risk based on the availability of suppliers per period. This approach recognizes the impact of supplier availability on the overall risk profile and decision-making process within the SSOA problem. Conversely, ref. [24] adopted a comprehensive perspective by concurrently considering multiple risks alongside quality risks. They employed a failure mode and effect analysis (FMEA) to compute a single-risk metric summarizing the identified risks. Similarly, refs. [25,26] integrated various risk factors into a single metric using different methodologies, facilitating a holistic risk assessment within the SSOA framework. Lastly, ref. [27] introduced a unified risk metric to model supplier cost risk in their approach. By consolidating diverse risk factors into a single metric, they simplified the decision-making process and enhanced the efficiency of their optimization model.

While the deterministic models adopted in these studies provide valuable insights, they must be considered limited by the assumption that the exact future value of risks is known in advance. As this assumption is not always valid, researchers have thus started to design stochastic formulations of the multiperiod SSOA problem to capture the inherent uncertainty associated with risk factors. In these formulations, different risk scenarios are formulated, and a specific probability of occurrence is usually associated with each scenario. For example, ref. [28] presented a formulation considering supplier local disruption risk, while ref. [29] extended this approach to consider stochastic supplier local disruption and capacity risks alongside deterministic quality risks. Lastly, ref. [30] incorporated stochastic supplier costs and demand risks into their formulation.

2.2. Direct Lookahead Models in Multiperiod SSOA

Despite the vast potential of stochastic programming, two main limitations need to be recognized when considering adopting this approach to solve a real-world multiperiod SSOA problem. First, stochastic programming models often assume a stationary distribution for risks, which may not accurately reflect real-world dynamics. Risk factors can indeed be subject to temporal variations and non-stationarity behaviors. Moreover, according to [9], stochastic programming can be highly computationally expensive, especially when involving integer variables in the problem formulation.

To address these limitations, researchers have proposed DLMs (i.e., approaches combining predictive models with linear programming models) to account for dynamic risks and reduce the computational effort required to solve the problem. Recent advancements in the fields of ML and DL have significantly increased the accuracy reachable by forecasting modules adopted as inputs in a DLM and have thus contributed to significantly improving the quality of decisions that can result as outputs from these approaches.

Reference [12] can be considered one of the first works suggesting this approach to solve the problem. In the study, the authors exploited ML models to forecast the capability of suppliers to deliver future orders on time, and orders were assigned to those suppliers who reported the highest probability of delivering components on time. Afterwards, several studies, such as those provided by [13,14,15], have developed ML regression models to punctually estimate demand risk to optimize order allocation over time. Moreover, predictions about future cost risks were adopted to solve the problem in the study reported by [17].

2.3. Research Gap and Novelties of the Proposed Approach

According to the literature, DLMs have only recently started to be investigated more intensively. Indeed, compared to deterministic and stochastic programming models, DLMs have received less attention, and only a few papers related to the topic can be found [12,13,14,15,16,17].

Moreover, the state of the art of the current proposed DLMs always assumes static values for the forecast and thus does not consider the uncertainty related to changes that newly acquired data can have on this forecast and subsequently on final decisions. As stochastic programming models have been introduced to deal with uncertainty that is not considered in deterministic ones, parametrized DLMs might be an under-investigated potential alternative to cope with the limitations of simple DLMs. Indeed, parametrized DLMs represent an extension of DLMs incorporating tunable parameters that can deal with the uncertainty related to the variability and sensitivity of the system to different inputs or conditions.

Considering the current literature and its gaps, the main novelty of this study is represented by the introduction of a new approach to solve the multiperiod SSOA when forecasts related to the risk of supplier non-punctual delivery are considered subject to update. Indeed, according to

Table 1. Literature summary.

Study	Model Type	Ordering Cost	Purchasing Cost	Shortage Cost	Holding Cost	Data Acquisition Benefit
[21]	DP	✔	✔		✔
[22]	DP		✔	✔	✔
[23]	DP	✔	✔	✔	✔
[24]	DP	✔	✔		✔
[25]	DP		✔	✔
[26]	DP	✔	✔		✔
[27]	DP	✔	✔		✔
[28]	SP	✔	✔	✔	✔
[29]	SP	✔	✔		✔
[30]	SP	✔	✔		✔
[12]	DLM
[13]	DLM		✔		✔
[14]	DLM	✔	✔		✔
[15]	DLM	✔	✔		✔
[16]	DLM	✔	✔		✔
[17]	DLM	✔			✔
This study	PDLM	✔	✔	✔	✔	✔

DP: deterministic programming, SP: stochastic programming, DLM: direct lookahead model, PDLM: parametrized direct lookahead model.

, approaches based on a parametrized DLM have never been adopted to solve this problem. In addition, the proposed approach, which considers the uncertain benefits of newly acquired data on forecast accuracy and thus on final decisions, optimizes the trade-off between data acquisition benefits and order emission costs for the first time. Purchasing costs, non-punctual delivery risk-related costs, and data acquisition benefits are thus balanced for the first time. Moreover, an empirical investigation of the benefit of considering this new trade-off is reported.

3. Materials and Methods

In this section, we first provide an overview of the proposed approach. We then delve into a detailed examination of the constituent components of the approach. Finally, we outline the research methodology employed to empirically evaluate the proposed approach’s performance compared to benchmarks.

3.1. Proposed Approach

Three modules have been integrated into the proposed approach to solve the multiperiod SSOA problem while considering the data acquisition and order emission trade-off: a predictive module, a prescriptive module, and a continuous training module. An illustration of how these modules are integrated is provided in Figure 1. Moreover, the pseudocode of the proposed approach and of its main module is reported in Appendix A in Figure A1, Figure A2, Figure A3 and Figure A4.

According to Figure 1, past historical data on suppliers’ delivery punctuality are provided as inputs to train the predictive module.

Predictions related to the suppliers’ future delivery punctuality provided by the predictive module will thus be adopted as inputs by the prescriptive module. Here, decisions about which supplier to select and how to allocate orders to suppliers over future periods are made. Moreover, considerations related to the advantages of data acquisition over order allocation cost are modelled to solve the problem while considering this trade-off.

Lastly, the continuous training module will adopt new collected supplier delivery punctuality data to update the forecasts released from the predictive module over time. Indeed, as time passes, order decisions translate into real purchasing orders and consequently into new supplier deliveries whose punctuality can be recorded. Suppose orders will be allocated to a specific supplier. In that case, the continuous training module will update the predictive model related to the specific supplier with the new data, and new forecasts related to the supplier’s future delivery performance will be provided as new input to the prescriptive module. On the other hand, if no order is allocated to a specific supplier, its forecast will not be updated.

3.1.1. Prescriptive Module

A parametrized DLM based on mixed-integer linear programming (MILP) is proposed to build the prescriptive module. The model is constructed based on several key assumptions, outlined as follows:

• The demand for the components allocated among suppliers is predetermined and constant for the designated future period.
• The unit purchasing cost varies among suppliers, and the total purchasing cost escalates linearly with the quantity ordered without any quantity discounts.
• The unit costs associated with untimely deliveries differ depending on whether the delivery is early or late. These costs are contingent on the components and remain consistent across suppliers. Additionally, the overall cost attributed to untimely deliveries increases linearly with the quantity delivered late and the duration of the delay.
• There is no relationship between the ordered quantity and the future value of the supplier delivery performance until orders remain within the supplier capacity limits.

Following these assumptions, the principal sets, parameters, decision variables, constraints, and components constituting the objective function to be minimized are enumerated below:

• Sets

• I = {1, …, i, …, M}: set of suppliers
• P = {1, …, t, …, T}: set of time period

• Parameters

• $C_i^{period}$: maximum period capacity of supplier i
• $c_i^{orderemission}$: unitary order emission cost of supplier i
• $c_i^{purchasing}$: unitary purchasing cost of supplier i
• $c^{delay}$: unitary cost related to late delivery of a component
• $c^{advance}$: unitary cost related to advance delivery of a component
• $d_t$: demand of the component that needs to be ordered for day t
• $f_{it}^{delay}$: number of days of delays predicted based on the predictive module for the number of components ordered from supplier i for day t
• $f_{it}^{advance}$: number of days of advance delivery predicted by the predictive module for the number of components ordered from supplier i for day t
• ${\theta }_i^{databeneifit}$: tunable parameters expressing the advantages of sending orders to supplier i to acquire new data. This parameter models the data acquisition benefit for supplier i
• $M$: positive large number

• Decision variables

• $Y_{it}$: amount of quantity ordered from supplier i for day t
• $Z_{it}$: binary variable equal to 1 if an order is sent to supplier i for day t, 0 otherwise

• Objective function

$$\mathrm{M}in{\sum }_{t=0}^T{\sum }_{i=1}^M{Y}_{it}(c_i^{purchasing}+c^{delay}{f}_{it}^{delay}+c^{advance}{f}_{it}^{advance})+Z_{it}(c_i^{orderemission}-{\theta }_i^{databeneifit})$$

(1)

In Equation (1), the objective function is aimed at finding the optimal trade-off between purchasing costs and costs related to non-punctual delivery by suppliers, with the order emission cost on one side and the data acquisition benefits on the other.

• Constraints

$${{\sum }_{t=0}^T{Y}_{it}\le \mathrm{C}}_{\mathrm{i}}^{\mathrm{p}\mathrm{e}\mathrm{r}\mathrm{i}\mathrm{o}\mathrm{d}}\forall i$$

(2)

$${\sum }_{i=1}^M{Y}_{it}=d_t\forall t$$

(3)

$$Y_{it}\le M\ast Z_{it}\forall i,\forall t$$

(4)

$$Y_{it}\ge 0,\forall i,\forall t$$

(5)

$$Z_{it}Binary\forall i,\forall t$$

(6)

Equation (2) is devised to ensure that the maximum quantity ordered from each supplier remains within acceptable limits. Equation (3) guarantees fulfilment of the daily demand for the component. Equation (4) ensures that a certain quantity of a specific component can be acquired from supplier i in period t only if an order is sent to supplier i in period t. Lastly, Equations (5) and (6) mandate, respectively, that the orders dispatched to each supplier during each period must exceed zero, and that the variable $Z_{it}$ has to be binary.

3.1.2. Predictive Module

A long short-term memory (LSTM) model is adopted to constitute the predictive module. LSTM models are DL models designed to address the challenges of learning long-term dependencies in sequential data by capturing temporal dependencies over extended sequences. These features make them well-suited for tasks such as time series forecasting, and several studies have proved their advantages over other models in solving predictive tasks in the field of SCM [31,32,33,34].

One separate LSTM model for each supplier is proposed to be adopted in the predictive module to estimate the future delivery punctuality of suppliers. The dependent variable (label) predicted by each LSTM model is represented by the number of days of delay or advance that a specific supplier will deliver the purchased components with respect to the planned delivery dates. On the other hand, the historical record of supplier delivery punctuality has been adopted as independent variables (features) to estimate the value of the label.

3.1.3. Continuous Training Module

A continuous training module is introduced in the proposed approach to ensure a continuous update of the predictive module over time as new data become available. Indeed, the continuous training paradigm has been identified as a fundamental pillar in ML applications [35]. Introducing a continuous training module ensures that the predictive module adapts dynamically to data distribution and environment changes. By continuously adapting to changing data patterns and environments, models trained using a continuous training approach can maintain a high performance and relevance over time, leading to more effective decision-making and improved outcomes in dynamic real-world scenarios.

Each time a new order is delivered by a specific supplier, the continuous training module adopts the newly recorded supplier delivery performance to trigger a full retraining of the LSTM model related to that supplier with the newly available data. After the retraining phase, new predictions of supplier behavior are formulated, updating the old forecasted values.

3.2. Research Methodology

In this section, we outline the research methodology used to assess the effectiveness of the proposed approach in contrast to benchmark methods. A case study was carefully chosen to gather authentic data, and an experimental design was developed to evaluate the performance of the approach. Section 3.2.1 provides insights into the chosen case study and the pertinent data collected, while Section 3.2.2 elaborates on the design of the experimental setup.

3.2.1. Case Study Data Collection

A real-life case study in the automotive sector was chosen to evaluate the effectiveness of the proposed approach. This sector was selected due to its substantial economic significance and widespread adoption of multiple sourcing as a risk management strategy. This underscores the critical importance of making optimal decisions in multiperiod SSOA. This study was focused on eight critical components of an important Italian automotive company for which a dual-sourcing strategy was implemented between 1 January 2021 and 31 December 2023. Detailed summaries of the key data associated with the components, the first supplier, and the second supplier under consideration are provided in

Table 2. Case study component data.

	1	2	3	4	5	6	7	8
Mean demand [pieces]	1091	329	1049	342	937	1346	1275	1595
Demand standard deviation	10.8	13.5	8.7	12.6	8.3	10.1	10.8	12.1
Unitary holding cost [%]	51.1%	176.4%	87.4%	140.7%	173.0%	134.4%	29.5%	163.5%
Unitary shortage cost [%]	184.6%	84.8%	28.7%	196.2%	119.7%	144.3%	163.3%	93.9%

,

Table 3. Case study first supplier data.

	1	2	3	4	5	6	7	8
Mean delay/advance [days]	9.8	−4.7	10.8	−31.4	−13.4	−38.7	−11.6	−22.5
Delay/advance standard deviation	22.7	21.2	22.2	22.3	23.5	27.9	47.3	28.9
Unitary purchasing cost [euro]	3079	4768	3271	2743	2872	2302	684	3928
Overall capacity [%]	95.9%	18.4%	85.8%	7.7%	85.9%	69.6%	67.6%	75.2%

and

Table 4. Case study second supplier data.

	1	2	3	4	5	6	7	8
Mean delay/advance [days]	11.4	35.9	−9.0	1.8	21.3	−6.5	5.9	−11.0
Delay/advance standard deviation	19.6	31.4	19.9	31.9	12.5	31.9	41.2	22.3
Unitary purchasing cost [%]	80.9%	97.7%	91.6%	84.6%	103.0%	94.7%	97.7%	110.2%
Overall capacity [%]	41.3%	84.1%	26.2%	98.7%	24.7%	74.1%	76.2%	87.0%

, respectively. In Table 2, for each component, the unitary holding cost and the unitary shortage cost are expressed as a percentage of the first supplier purchasing cost. Similarly, Table 3 reports the supplier capacity as a percentage of the overall components’ demand. Lastly, Table 4 reports the unitary purchasing cost for the second supplier as a percentage of the first supplier’s purchasing cost, while the second supplier’s overall capacity is expressed as a percentage of the overall demand.

3.2.2. Experimental Design

The data acquired from the case study outlined in Section 3.2.1 were adopted to set up an experimental design to thoroughly evaluate the effectiveness of the proposed approach in contrast to benchmarks. Section Benchmarks delineates the benchmarks against which the proposed approach was assessed. Subsequently, Section Evaluation Metrics enumerates the metrics chosen for the comparative analysis. Finally, Section Experiment Setup elucidates the experimental configurations employed for each evaluation.

Benchmarks

Three benchmarks were adopted to assess the performance of the approach: the Horacle approach, the Zero Exploration approach, and the Continuous Exploration approach.

In the Horacle approach, the decision-maker is assumed to know the future values of supplier delivery punctuality perfectly. The order allocation decisions resulting from the Horacle approach thus represent the optimal decision, and the resulting cost obtained when implementing this approach represents an ideal lower bound for the other approaches. While the Horacle approach aims to provide a reference point to evaluate the efficiency of the proposed approach in optimally solving the multiperiod SSOA problem, the two other approaches aim to investigate the advantages of providing a new approach to balance the trade-off between data acquisition benefits and order emission costs.

The Zero Exploration approach assumes that the value of parameter ${\theta }_i^{databeneifit}$ is equal to zero for both suppliers. A decision-maker relying on this approach supposes that no data acquisition benefits are present for either supplier. Sending orders for the same period to both suppliers is not considered in this approach (exceptions are only dictated by capacity constraints). The acquisition of new data points related to supplier delivery punctuality is assumed not to affect the value of future forecasts; therefore, orders are always allocated to the supplier reporting the best value in the initial estimates.

Contrariwise, in the Continuous Exploration approach, the value of parameter ${\theta }_i^{databeneifit}$ is assumed to be large enough to generate at least one order emitted to both suppliers in each period. A decision-maker relying on this approach assumes that sending orders to both suppliers (even if in different quantities) is always an effective strategy. Indeed, even if sending orders to only one of them can result in an immediate lower cost, acquiring new data can affect forecast values and thus lead to revised order allocation decisions.

Evaluation Metrics

Four economic metrics are considered to compare the cost when implementing order allocation decisions according to the proposed approach with those resulting from the benchmark approaches:

$${{}_{\Delta }}_{\left(proposed,benchmark\right)}^{PURCHASINGCOST}=\frac{{\sum }_{t^{\prime }=0}^T{\sum }_{t^{\prime }\ge t>t^{\prime }}^T{\sum }_{i=1}^M(Y_{{itt}^{\prime }}^{proposed}-Y_{{itt}^{\prime }}^{benchmark})c^{purchasing}}{{\sum }_{t^{\prime }=0}^T{\sum }_{t^{\prime }\ge t>t^{\prime }}^T{\sum }_{i=1}^M{Y}_{{itt}^{\prime }}^{benchmark}{c}^{purchasing}}$$

(7)

$${{}_{\Delta }}_{\left(proposed,benchmark\right)}^{DELIVERYCOST}=\frac{{\sum }_{t^{\prime }=0}^T{\sum }_{t^{\prime }\ge t>t^{\prime }}^T{\sum }_{i=1}^M(Y_{{itt}^{\prime }}^{proposed}-Y_{{itt}^{\prime }}^{benchmark})(c^{delay}{true}_{it}^{delay}+c^{advance}{true}_{it}^{advance})}{{\sum }_{t^{\prime }=0}^T{\sum }_{t^{\prime }\ge t>t^{\prime }}^T{\sum }_{i=1}^M{Y}_{{itt}^{\prime }}^{benchmark}(c^{delay}{true}_{it}^{delay}+c^{advance}{true}_{it}^{advance})}$$

(8)

$${{}_{\Delta }}_{\left(proposed,benchmark\right)}^{ORDEREMISSIONCOST}=\frac{{\sum }_{t^{\prime }=0}^T{\sum }_{t^{\prime }\ge t>t^{\prime }}^T{\sum }_{i=1}^M(Z_{itt\prime }^{proposed}-Z_{itt\prime }^{benchmark})c_i^{orderemission}}{{\sum }_{t^{\prime }=0}^T{\sum }_{t^{\prime }\ge t>t^{\prime }}^T{\sum }_{i=1}^M{Z}_{itt\prime }^{benchmark}{c}_{it}^{orderemission}}$$

(9)

$$\begin{array}{c}{{}_{\Delta }}_{\left(proposed,benchmark\right)}^{OVERALLCOST}\hfill \\ =\frac{{\sum }_{t^{\prime }=0}^T{\sum }_{t^{\prime }\ge t>t^{\prime }}^T{\sum }_{i=1}^M\left(Y_{it{t}^{\prime }}^{proposed}-Y_{it{t}^{\prime }}^{benchmark}\right)\left({c^{purchasing}+c}^{delay}{true}_{it}^{delay}+c^{advance}{true}_{it}^{advance}\right)}{{\sum }_{t^{\prime }=0}^T{\sum }_{t^{\prime }\ge t>t^{\prime }}^T{\sum }_{i=1}^M{Y}_{it{t}^{\prime }}^{benchmark}\left({c^{purchasing}+c}^{delay}{true}_{it}^{delay}+c^{advance}{true}_{it}^{advance}\right)+Z_{it{t}^{\prime }}^{benchmark}{c}_i^{orderemission}}+\hfill \\ \frac{{\sum }_{t^{\prime }=0}^T{\sum }_{t^{\prime }\ge t>t^{\prime }}^T{\sum }_{i=1}^M(Z_{{itt}^{\prime }}^{proposed}-Z_{{itt}^{\prime }}^{benchmark})c_i^{orderemission}}{{\sum }_{t^{\prime }=0}^T{\sum }_{t^{\prime }\ge t>t^{\prime }}^T{\sum }_{i=1}^M{Y}_{it{t}^{\prime }}^{benchmark}\left({c^{purchasing}+c}^{delay}{true}_{it}^{delay}+c^{advance}{true}_{it}^{advance}\right)+Z_{{itt}^{\prime }}^{benchmark}{c}_i^{orderemission}}\hfill \end{array}$$

(10)

Here, $Y_{{itt}^{\prime }}^{proposed}$ represents the optimal quantity to allocate to a specific supplier i for a specific period t based on the forecast updated at day t′ according to the proposed approach, while $Z_{{itt}^{\prime }}^{proposed}$ expresses whether an order is sent to supplier i in period t based on forecasts updated at day t′ by the proposed approach. Contrarywise, $Y_{{itt}^{\prime }}^{benchmark}$ and $Z_{{itt}^{\prime }}^{benchmark}$ represent the same optimal quantity identified when considering one of the benchmark approaches reported in Section Benchmarks, respectively. The parameters ${true}_{it}^{delay}$ and ${true}_{it}^{advance}$ represent the true delivery punctuality reported by supplier i in period t. These values can be higher or lower than the predicted values reported in the parameters $f_{it}^{delay}$ and $f_{it}^{advance}$.

According to their formulations, Equations (7)–(10) thus represent the difference between the purchasing cost, the delivery cost, the order emission cost, and the overall cost, respectively, following the decision suggested by the proposed approach and the decisions made following one of the benchmark approaches. The difference is expressed as a percentage of the cost reported by the benchmark approach. A positive value reported in Equation (11), for example, thus represents that the proposed approach led to a higher overall cost value than the benchmark.

Experiment Setup

The experimental setup remained consistent across all the eight components outlined in Section 3.2.1. The historical data of the delivery performance reported by each supplier for a specific component are divided into three consecutive subsets, as illustrated in Figure 2.

The three subsets, referred to as the training set (comprising 60% of the historical data), the validation set (comprising 20% of the historical data), and the test set (containing the remaining data), were utilized to achieve distinct objectives.

The training set facilitated the initial learning phase of the models constituting the predictive module. During this phase, the models learned the relationship between the selected features and the label to predict. The hyperparameters used for the models are detailed in

Table 5. LSTM hyperparameters.

Hyperparameter	Hyperparameter Value
Number of layers	1
Max number of epochs	100
Early stopping threshold (σ)	15 epochs with no improvements in the RMSE of the validation dataset
Loss function	MSE
Input chunk length	7

and have been obtained following a trial-and-error procedure. Although other strategies could have been adopted for the hyperparameter tuning phase, selecting a more basic tuning procedure was preferred, as the extensive optimization of the hyperparameters was assumed not to be this study’s primary goal.

The validation set is instead adopted only when considering the proposed approach to find the best value of parameter ${\theta }_i^{databeneifit}$. A Bayesian optimization (BO) strategy [36] is proposed to find the best value of the parameter ${\theta }_i^{databeneifit}$ within a prespecified research space ϴ. A BO strategy is adopted, as it represents a valuable iterative optimization technique for finding the maximum or minimum of an objective function when it is expensive to evaluate. This feature is particularly useful, considering that the optimal value of the tunable parameter ${\theta }_i^{databeneifit}$ needs to be found by looking for a value in research space ϴ that minimizes the following result of Equation (11):

$$\begin{array}{c}\hfill \mathrm{M}in{\sum }_{t^{\prime }=0}^T{\sum }_{t^{\prime }+1\ge t>t^{\prime }}^T{\sum }_{i=1}^M{\widehat{Y}}_{it{t}^{\prime }}(c_i^{purchasing}+c^{delay}{true}_{it}^{delay}+c^{advance}{true}_{it}^{advance})+\\ \hfill {\sum }_{t^{\prime }=0}^T{\sum }_{t^{\prime }+1\ge t>t^{\prime }}^T{\sum }_{i=1}^M{\widehat{Z}}_{it{t}^{\prime }}(c_i^{orderemission}-{\widehat{\theta }}_i^{databeneifit})\end{array}$$

(11)

In Equation (11), ${\widehat{Y}}_{it{t}^{\prime }}$ and ${\widehat{Z}}_{it{t}^{\prime }}$ represent the optimal decisions obtained from the prescriptive module when considering a specific value, and ${\widehat{\theta }}_i^{databeneifit}$ indicates the data acquisition benefit and forecast updated on day $t^{\prime }$. Conversely, ${true}_{it}^{advance}$ and ${true}_{it}^{delay}$ represent the ex-post real amount of advance or delay of supplier deliveries. The computation of Equation (11) cannot be considered cheap. First, it requires solving the multiperiod SSOA defined in Section 3.2.1 based on the forecast provided by the predictive module described in Section 3.2.2 to find the optimal values of ${\widehat{Y}}_{it{t}^{\prime }}$ and ${\widehat{Z}}_{it{t}^{\prime }}$. Afterwards, it requires performing an ex-post evaluation of the decisions based on the true values of delivery performance experienced by the selected suppliers in each considered period (${true}_{it}^{advance}$ and ${true}_{it}^{delay}$).

The predictive module is thus initially trained using the training set’s past historical data. Then, a value of parameter ${\theta }_i^{databeneifit}$ is selected within search space ϴ following the BO strategy, and the prescriptive module is executed considering these inputs. Order allocation decisions are thus generated, new data points are acquired, and the predictive models are updated in the validation set following the order allocation decisions. Once all the periods in the validation set have been simulated, the value of Equation (11) is computed ex-post, and the best value of parameter ${\theta }_i^{databeneifit}$ found so far is recorded. This procedure is repeated K times. A random sampling strategy was adopted to select the starting value for the BO, and five different scenarios involving different starting points were considered for each component. The value of ${\theta }_i^{databeneifit}$ resulting in the lowest value of Equation (11) is adopted in the test set by the proposed approach as the effective value for parameter ${\theta }_i^{databeneifit}$. The value of research space ϴ and the number of trials K are reported in

Table 6. Bayesian optimization parameters.

Parameter	Value
Search space (ϴ)	0–99,999
N trials	50

. The number K of trials to follow has been selected, recurring to a heuristic elbow procedure. Indeed, the trials suggested that only small increments in performance were achievable for a greater value of K. Furthermore, similar K values were observed [37] when no more than 20 trials were reported.

Finally, the test set is adopted to simulate and compare the decisions generated by the proposed approach and the benchmarks. Initially, the predictive module is trained on the historical data coming from both the training and the validation set, and the value adopted by the proposed approach of ${\theta }_i^{databeneifit}$ is the one identified in the validation set. As time evolves discretely with the arrival of new, unevenly sampled deliveries, based on the decisions resulting from the prescriptive module, new supplier performance data are collected over the test set, and the continuous training module updates the predictive ones with the newly acquired data. In conclusion, the values of the evaluation metrics described in Section Evaluation Metrics are thus computed considering all the discrete periods contained in the test set for both the proposed approach and the benchmarks.

4. Results

The results of the experimental comparison between the proposed approach and the benchmark approaches described in Section Benchmarks based on the evaluation metrics described in Section Evaluation Metrics are reported in this section. The boxplots in Figure 3, Figure 4 and Figure 5 report the distribution of the evaluation metrics computed by comparing the proposed approach with the Horacle approach, the Zero Exploration approach, and the Continuous Exploration approach, respectively, for the eight components investigated in the case study described in Section 3.2.1. The boxes in a plot represent the interquartile range, which spans from the first to the third quartile; a line inside the box represents the median of the considered values; and a triangle indicates the mean value. The whiskers extend from the edges of the box to the minimum and maximum values within 1.5 times the interquartile range from the first and third quartiles, respectively.

According to Figure 3, the proposed approach resulted, on average, in an increase of 42% in the overall cost compared to a decision-maker with perfect knowledge of the future (Horacle approach). However, this difference resulted in variance in the interquartile range from a minimum increase of 7% to a maximum increase of 56% for half of the considered components. The higher average differences between the evaluation metrics are observed in the delivery and order emission costs.

Specifically, the delivery cost produced by the proposed approach was, on average, 45% higher than that of the Horacle approach. Instead, an average increase of 15% was observed in the order emission cost. No significant differences in the purchasing cost were observed between the two approaches.

Different results were obtained by comparing the proposed approach with the Zero Exploration approach.

According to Figure 4, the proposed approach produced, on average, a decrease of 20% in terms of the overall cost compared to an approach that optimizes the order allocation decision without considering the benefit of sending orders to suppliers to acquire new data (Zero Exploration approach). This difference was observed to potentially vary between a minimum decrease of 6% to a maximum reduction of 31% for 50% of the considered components. In addition, when comparing the proposed approach with the Zero Exploration approach, the average increase of 10% in the order emission cost was effectively counterbalanced by an improved capability of reducing the delivery cost (by 21% on average). Also, in this comparison, the two approaches exhibited no significant difference in the purchasing cost.

Lastly, the results of the comparison between the proposed approach and the Continuous Exploration approach are reported in Figure 5.

Similarly to the previous comparison, Figure 5 highlights an advantage of the proposed approach over the Continuous Exploration approach in solving the multiperiod SSOA problem, corresponding on average to a decrease of 21% in the overall cost. The average decrease of 38% in order emission costs did not lead the proposed approach to produce higher delivery costs. Indeed, the delivery cost reported by the proposed approach was 20% lower on average than the one reported for the benchmark approach. Also, no significant differences in purchasing cost were seen between the two approaches.

5. Discussion and Conclusions

The multiperiod SSOA problem has always received considerable attention in the literature due to its practical relevance. Recently, DLMs (i.e., deterministic integer linear programming models solved based on values provided by forecasting modules) have started to be adopted to solve the problem due to the benefits that recent advancements in ML and DL have brought to the predictive accuracy of forecasts. However, approaches proposed in the literature often assume static forecast values over time and do not consider that in the new AI era, where data are the fuel, allocating orders to suppliers can also be seen as a way to acquire new data and thus obtain updated, more accurate forecasts.

Considering the current state of the literature, two main contributions are proposed in this study. First, a new approach implementing a parametrized DLM to solve the multiperiod SSOA problem is designed and proposed. For the first time, a parametrized DLM is explicitly designed to consider the trade-off between data acquisition benefits and order emission costs when forecasts about the future delivery performances of suppliers are considered to vary as new data are acquired over time. Moreover, empirical results describing the performances reached by the proposed approach and benchmarks in a real automotive case study are reported. These results provide a starting point for understanding the potential benefits of the proposed approach and, more generally, the advantages of adequately balancing data acquisition benefits against order emission costs when solving the multiperiod SSOA problem.

In particular, compared to a decision-maker with perfect knowledge of the future, the proposed approach led to increased overall costs that spanned from a minimum increase of 7% to a maximum increase of 56%. The comparison with a decision-maker without perfect knowledge of the future, assuming no benefit of data acquisition, indicated the advantages of the proposed approach, which, on average, led to a 20% reduction in the overall cost. Similar results were observed in the comparison between the proposed approach and one simulating a decision-maker without perfect knowledge of the future, but assuming that it is always positive to send orders to two suppliers to exploit the benefit that new data can provide to the forecasting accuracy.

The empirical results obtained from the analysis of the case study thus led to two main implications. First, the results proved that the proposed approach could lead to good supplier selection and order allocation decisions, therefore supporting adopting a parametrized DLM to solve the multiperiod SSOA problem. Secondly, they highlighted the necessity to balance data acquisition benefits and order emission costs over time. Indeed, according to the results, neither assuming zero data acquisition benefits nor assuming that data acquisition benefits are always present resulted in the best choice. Acquiring new data can allow forecast value updates; however, acquiring new supplier-related data is expensive, and updating forecast values does not always lead to changes in order allocation decisions.

The observed results are subject to limitations. First, a single case study was adopted to investigate the performance of the proposed approach, thus limiting the generalizability of the findings. Secondly, the MILP model integrated into the proposed approach relied on the assumption that no relationship exists between the delivery performance of suppliers and the ordered quantity. While this assumption can be true for certain components, it cannot hold for others. Finally, a potential limitation of the proposed approach is that supplier purchasing costs are assumed to be known, and no costs related to quality issues have been considered.

Based on these limitations, potential future research directions can point to expanding the empirical evaluation of the proposed approach to multiple case studies. Adapting the proposed approach to cases where a linear or nonlinear dependency between the ordered quantity and the variable to forecast can be seen as another interesting and practically relevant research direction. Lastly, expanding the number of forecasted risks in the proposed approach and investigating how data acquisition benefits differ between forecasts related to these risks and how to balance the data acquisition benefits and order emission costs in this case could be a further direction to improve the proposed approach.