Single-Stage Causal Incentive Design via Optimal Interventions

Abstract

We introduce Causal Incentive Design (CID), a framework that applies causal inference to canonical single-stage principal–agent problems (PAPs) characterized by bilateral private information. Within CID, the operating rules of PAPs are formalized using an additive-noise causal graphical model (CGM). Incentives are modeled as interventions on a function space variable, $\Gamma$, which correspond to policy interventions in the principal–follower causal relation. The causal inference target estimand $V(\Gamma )$ is defined as the expected value of the principal’s utility variable under a specified policy intervention in the post-intervention distribution. In the context of additive-Gaussian independent noise, the estimand $V(\Gamma )$ decomposes into a two-layer expectation: (i) an inner Gaussian smoothing of the principal’s utility regression; and (ii) an outer averaging over the conditional probability of the follower’s action given the incentive policy. A Gauss–Hermite quadrature method is employed to efficiently estimate the first layer, while a policy-local kernel reweighting approach is used for the second. For offline selection of a single incentive policy, a Functional Causal Bayesian Optimization (FCBO) algorithm is introduced. This algorithm models the objective functional $\gamma \mapsto V\left(\gamma \right)$ using a functional Gaussian process surrogate defined on a Reproducing Kernel Hilbert Space (RKHS) domain and utilizes an Upper Confidence Bound (UCB) acquisition functional. Consequently, the policy value $V\left(\gamma \right)$ becomes an interventional query that can be answered using offline observational data under standard identifiability assumptions. High-probability cumulative-regret bounds are established in terms of differential information gain for the proposed FBO algorithm. Collectively, these elements constitute the central contributions of the CID framework, which integrates causal inference through identification and estimation with policy search in principal–agent problems under private information. This approach establishes a causal decision-making pipeline that enables commitment to a high-performing incentive in a single-shot game, supported by regret guarantees. Provided that the data used for estimation is sufficient, the resulting offline pipeline is appropriate for scenarios where adaptive deployment is impractical or costly. Beyond the methodological contribution, this work introduces a novel application of causal graphical models and causal reasoning to incentive design and principal–agent problems, which are central to economics and multi-agent systems.

1. Introduction

In an incentive design (ID) problem, a central institution alters the behavior of self-interested agents to improve overall performance by implementing an incentive function that adjusts their individual payoffs. The central institution, typically represented as a principal agent, may aim to drive the system’s performance towards a more desirable behavior, such as maximizing revenue or social welfare within the multi-agent system. This work concentrates on the single-stage canonical principal–agent problem (PAP), in which an incentive designer agent (the principal) commits to an incentive function that influences the behavior of a single follower. The interaction is a one-shot hierarchical inverse Stackelberg game, in which the principal first announces an incentive function; the follower then responds optimally based on that commitment, and the outcomes are realized without any subsequent adaptation or additional rounds. We maintain the assumption of private information for both players; specifically, the principal is unaware of the follower’s utility function and vice versa.

The single-stage PAP is the canonical problem in incentive design (ID) because it includes the core structural features in a one-shot setting: the principal commits once to an incentive, the follower best-responds, and outcomes are realized; information asymmetry and incomplete knowledge persist; and strategic reasoning is driven by ex ante beliefs about utilities or types. While there is no intertemporal adaptation or belief updating between rounds, the principal–agent (leader–follower) tension remains fundamental: one party creates incentives in the face of uncertainty, while the other optimizes behavior in response to that commitment. These features remain structurally invariant when extending the single-period formulation to multiple followers and multiple principals. With private information on both sides, the design task still entails bilateral inference at design time—reasoning about the other party’s unknown objectives—which is the core difficulty that scales to richer multi-agent settings. Thus, the single-stage formulation preserves the irreducible skeleton of ID problems and the central epistemic and incentive-alignment challenges that govern their multi-agent variants [1].

Consider a lender as the principal and a borrower as the follower. The lender must choose, once, a contract rule—an incentive function that maps borrower attributes or application information to loan terms (e.g., interest rate, credit limit, collateral, or a repayment rebate). The borrower then decides whether to accept the contract and, if funded, how much effort to exert toward repayment. Both sides hold private information (e.g., the borrower’s risk and effort costs, and the lender’s risk appetite and cost structure). The outcome—acceptance, repayment, or default, and the lender’s realized payoff—occurs once; there is no opportunity to update terms after observing behavior. This one-shot contract choice is exactly a single-stage PAP: the lender designs the incentive rule under uncertainty; the borrower best-responds; the realized payoff reflects both the selection induced by the rule and the behavioral response it elicits.

We treat the principal’s single-stage choice as the endpoint of a finite-horizon sequential optimization carried out before deployment. The principal starts from historical observations of the system—past incentive rules, the follower’s responses, and realized payoffs—and uses them to estimate the principal’s expected utility under any candidate incentive function. A data-efficient search procedure (Bayesian optimization over a space of incentive functions) iteratively proposes candidate incentives, updates beliefs about expected utility using the accumulated evidence, and records the most promising options. After a fixed number of iterations, the principal selects one incentive function—the estimated best—to implement in the single-stage game. This procedure makes no restrictive assumptions about the players’ utility forms and respects the private-information setting; it is simply a principled way to use prior observations to select a high-value one-shot incentive.

1.1. Approach and Contributions

Even in the single-stage canonical principal–agent problem (PAP), it is difficult to predict how a committed incentive will affect the follower’s behavior under bilateral private information. We address this challenge by integrating causal inference with incentive design: we represent the single-stage PAP as a causal graphical model (CGM) in which the incentive function is treated as an intervention on the function space variable $\Gamma$, and we target the principal’s utility variable $J_L$ under that policy, $V\left(\gamma \right)\equiv \mathbb{E}[J_L\mid \mathrm{do}(\Gamma =\gamma )]$. This causal target is identified via the g-formula, enabling estimation of policy values from observational data without requiring controlled experiments. We then cast policy selection as a black-box sequential offline optimization over a function space of incentives, using Bayesian optimization to efficiently search among costly evaluations of $V\left(\gamma \right)$ before committing to a single policy in the one-shot game.

In the proposed framework, the leader’s policy $\Gamma$ acts as a functional intervention, the follower’s action is an explicit mediator, and the leader’s payoff $J_L$ is the outcome node. The policy value $V\left(\gamma \right){\displaystyle =}\mathbb{E}[J_L\mid do(\Gamma =\gamma )]$ constitutes an interventional and counterfactual query on this causal graphical model, which can be identified and estimated from observational data (offline logs) under the assumptions of positivity and additive noise. Accordingly, this work advances causal reasoning with graphical models and introduces a novel application domain in incentive design and principal–agent models, thereby complementing established applications in biology, medicine, and economics.

We formalize the single-stage canonical PAP (one principal and one follower) as an additive-noise CGM with endogenous variables $\{\Gamma ,{\Omega }_L,{\Omega }_F,J_F,J_L\}$ that capture the inverse-Stackelberg semantics ${\Omega }_L=\gamma \left({\Omega }_F\right)+U_L$, where incentives are interventions $\mathrm{do}(\Gamma =\gamma )$. From this model, we derive a non-parametric identification of $V\left(\gamma \right)$ by marginalizing over ${\Omega }_L$ and ${\Omega }_F$. In the additive-Gaussian setting, the estimand becomes a nested expectation with two interpretable components: an inner expectation (a Gaussian smoothing of the outcome regression ${\mu }_L({\Omega }_L,{\Omega }_F,\Gamma )$) and an outer expectation over the mediator law $p({\Omega }_F\mid \Gamma =\gamma )$ induced by the incentive. Practically, we compute the inner term by Gauss–Hermite quadrature and approximate the outer term by policy-local kernel reweighting in policy space; a linear credit-market example (affine pricing rule) illustrates a closed-form specialization.

We optimize $\gamma \mapsto V\left(\gamma \right)$ over an admissible set $\Gamma$ modeled as a Reproducing Kernel Hilbert Space (RKHS) ${\mathcal{H}}_k$. A functional Gaussian process (GP) surrogate on ${\mathcal{H}}_k$—with a functional RBF kernel driven by RKHS distances between incentive functions—provides posterior mean and uncertainty over policy values. A functional GP-UCB acquisition functional trades off exploration and exploitation to propose the next incentive to evaluate, building a data set ${\left\{({\gamma }^t,\widehat{V}\left({\gamma }^t\right))\right\}}_{t=1}^T$ from which we select the best ${\gamma }^{+}$ to deploy in the single-stage game. We provide high-probability cumulative-regret bounds for this Stackelberg Functional Causal Bayesian Optimization (FCBO) procedure in terms of differential information gain, with guarantees for both finite policy sets and infinite (non-parametric) RKHS domains.

Our pipeline builds on established ingredients: causal identification via the g-formula and policy interventions [2,3], RKHS/GP surrogates for Bayesian optimization [4,5,6], and GP-UCB-style exploration with information-gain regret analysis [7,8]. We have four new key contributions: (i) a PAP-specific CGM that treats incentives as functional interventions and makes the follower action an explicit mediator of policy value; (ii) a two-layer identification and estimation recipe for single-stage PAPs, using Gaussian inner smoothing with policy-local outer reweighting; (iii) a functional GP surrogate on an RKHS of incentive rules with a policy-space RBF and support-aware acquisition; and (iv) regret bounds stated in terms of differential information gain for the Stackelberg functional setting with a uniform sub-Gaussian envelope for estimator noise. Figure 1 sketches the full CID pipeline (identification and estimation of $V\left(\gamma \right)$ and FCBO policy search) and Figure 2 depicts one FCBO iteration; shaded boxes on Figure 1 and Figure 2 mark components introduced in this paper. Our contributions can be summarized as follows:

• CID for single-stage PAPs. We establish a generic CGM for the single-stage canonical PAP in which incentives are formalized as functional interventions, yielding a principled causal estimand for policy value $V\left(\gamma \right)=\mathbb{E}[J_L\mid \mathrm{do}(\Gamma =\gamma )]$ identified from observational data via the g-formula.
• Semi-parametric estimation strategy. We develop a practical, modular estimator for $V\left(\gamma \right)$ under additive Gaussian noise: (a) learn the principal’s action mechanism and outcome regression; (b) compute the inner Gaussian expectation by Gauss–Hermite quadrature; and (c) approximate the induced mediator law with policy-local kernel weights. A linear credit-market pricing example provides a closed form that clarifies the roles of policy parameters and the induced borrowing response.
• Functional Bayesian optimization for incentives. We propose the Stackelberg FCBO algorithm: a functional GP surrogate on ${\mathcal{H}}_k$ with a functional GP-UCB acquisition functional to sequentially (offline) optimize over incentive functions when policy evaluations are expensive and noisy.
• Theoretical guarantees. We prove high-probability cumulative-regret bounds that scale with $\sqrt{T{\beta }_T{\mathcal{I}}_T}$, where ${\mathcal{I}}_T$ is the information gain and ${\beta }_T$ is the exploration schedule. We extend the analysis from finite admissible sets to infinite RKHS domains via covering arguments and uniform approximation—quantifying the data-efficiency and reliability of the offline design that precedes the one-shot deployment.

Figure 1. Overview of the CID pipeline. Identification yields a two-layer estimand; the estimator feeds a functional GP surrogate and a support-aware GP-UCB search to select a single policy offline. Shaded boxes indicate contributions introduced in this paper.

Figure 1. Overview of the CID pipeline. Identification yields a two-layer estimand; the estimator feeds a functional GP surrogate and a support-aware GP-UCB search to select a single policy offline. Shaded boxes indicate contributions introduced in this paper.

Figure 2. One iteration of Stackelberg FCBO algorithm. Novel elements are highlighted with shaded boxes: support-aware GP-UCB selection and the nested estimator used for off-policy evaluation.

Figure 2. One iteration of Stackelberg FCBO algorithm. Novel elements are highlighted with shaded boxes: support-aware GP-UCB selection and the nested estimator used for off-policy evaluation.

1.2. Related Work

Incentive design (ID) has long been studied in economics and control through the lens of principal–agent theory [9,10]. Within this tradition, the single-stage (one-shot) PAP is a canonical formulation of contracting under asymmetric information: the principal commits to a contract, the agent best-responds, and outcomes are realized once. Classical approaches model uncertainty with priors over types and often assume a known mapping from actions to outcomes, yielding foundational results in adverse selection and moral hazard [1,9]. Control-theoretic treatments similarly emphasize leader–follower structure but typically rely on parametric models of dynamics and payoffs (e.g., early closed-loop Stackelberg formulations and multi-level control [10], and modern bilevel optimization theory and hardness results [11]).

More recently, the machine learning literature has approached ID through online learning paradigms—such as multi-armed bandits, reinforcement learning, and Bayesian optimization [5,12]—where the principal adapts incentives across rounds while observing partial feedback. This work has produced algorithms for repeated principal–agent settings, combinatorial bandits for dynamic preferences [13,14], meta-gradient methods for adaptive mechanism design [12,15], and BO-based procedures that steer multi-agent systems toward desirable equilibria [16]. While powerful, these approaches are inherently multi-stage and often treat the environment as a black box, emphasizing empirical adaptation rather than explicit reasoning about how a committed incentive causes behavioral change in a one-shot deployment. Beyond incentive design, robustness studies in multilayer transportation networks similarly demonstrate that inter-layer coupling critically governs aggregate performance—see the bilayer railway–aviation analysis with discrete cross-layer assignment [17].

A parallel line of research integrates causal reasoning into sequential decision-making [18,19,20,21]. Causal bandits, counterfactual policy evaluation, and causal reinforcement learning utilize graphical models to structure exploration and target the effects of interventions [22,23]. In Causal Bayesian Optimization (CBO) [24], the optimization routine exploits known causal structure among inputs to identify high-value interventions more efficiently. These methods, however, typically consider atomic (variable-level) interventions and multi-round interactions. In contrast, our focus is on a one-shot commitment problem: the principal must select a single incentive function before deployment and cannot adapt it afterward.

Our work connects these strands by introducing a causal, offline design pipeline tailored to the single-stage PAP. We represent the principal–agent interaction with a causal graphical model (CGM) and interpret incentives as functional (conditional) interventions and policies as interventions on the node that embodies the principal’s rule. This yields a policy value target for the principal (the expected utility under the post-intervention distribution) that can be identified from observational data via the g-formula and estimated without requiring online experimentation. To efficiently search over a space of incentive functions before deployment, we adopt a Functional Bayesian Optimization procedure [6,25]: a functional Gaussian process surrogate on an RKHS of admissible incentives, coupled with a UCB-type acquisition functional, guides a small number of costly offline evaluations of policy value. Taken together, the proposed approach complements classical single-stage contract theory by supplying a causal identification–estimation pathway for policy value under private information. Finally, our analysis provides high-probability regret guarantees in terms of differential information gain, quantifying how efficiently the offline procedure learns the principal’s objective over the function space.

Our emphasis is on the theoretical underpinnings of single-stage causal incentive design, specifically, (i) identification of policy value under functional interventions; (ii) a two-layer estimation approach with policy-local support diagnostics; and (iii) support-aware Functional Bayesian Optimization with regret control under a uniform sub-Gaussian envelope. To maintain focus and adhere to space constraints, we reserve numerical simulations and applied case studies for subsequent work that will use the present framework as a baseline to explore design choices, including policy classes, kernel metrics, and bandwidth selection across various domains.

1.3. Outline

The rest of the paper is organized as follows. The preliminary background, Section 2, first reviews the bilevel viewpoint of hierarchical Stackelberg games and inverse Stackelberg formulations. We then formalize the single-stage principal–agent problem (SS-PAP) and specify the information model with bilateral private information. The background closes with the essentials of causal graphical models (CGMs) and the notions of hard and functional (policy) interventions used later for identification and estimation. We introduce our Causal Incentive Design (CID) framework for canonical SS-PAPs in Section 3. We present an additive-noise Gaussian CGM for the canonical single-stage PAP and its assumptions, define the principal’s policy value target estimand $V\left(\gamma \right)=\mathbb{E}[J_L\mid \mathrm{do}(\Gamma =\gamma )]$, and provide a semi-parametric identification formula for $V\left(\gamma \right)$ via the g-formula. We then describe a practical offline estimation pipeline for the identification formula in general and provide an illustrative credit-market example with an affine pricing rule that demonstrates a closed-form specialization of the estimand. Section 4 details the Stackelberg FCBO procedure for offline policy selection. We model the objective $\gamma \mapsto V\left(\gamma \right)$ with a functional Gaussian-process surrogate over an RKHS of admissible incentives and employ a functional GP-UCB acquisition functional. We then present the Stackelberg FCBO algorithm and provide high-probability cumulative-regret bounds in terms of differential information gain. Section 5 reflects on the implications of CID for single-shot incentive design and outlines directions for empirical validation and broader multi-agent extensions.

2. Preliminary Background

The background is organized into three components: (i) the Stackelberg and inverse Stackelberg perspectives for incentive design (Section 2.1); (ii) the single-stage principal–agent problem (PAP) setting and the information model for the single-stage PAP with bilateral private information (Section 2.2); and (iii) the fundamentals of causal graphical models (CGMs), hard interventions, and functional (policy) interventions, which serve as the foundation for interpreting incentives as policies (Section 2.3).

2.1. Incentive Design as Inverse Stackelberg Games

Games characterized by a hierarchical decision-making structure, where one or more players, called the leaders, declare their strategy first and impose this strategy upon the other players, called the followers, are referred to as Stackelberg games. If leaders declare their strategy as a mapping from the followers’ decision space into their own decision space, we refer to this as inverse Stackelberg games. The essence of a Stackelberg game can be described considering the basic single-leader single-follower single-stage game as follows. Let us denote by ${\omega }_L\in {\mathbf{\Omega }}_L\subseteq {\mathbb{R}}^{n_L}$ and ${\omega }_F\in {\mathbf{\Omega }}_F\subseteq {\mathbb{R}}^{n_F}$ the leader and follower decision variables, respectively. The leader chooses an action or decision from its decision space ${\mathbf{\Omega }}_L$, and the follower, informed of the leader’s choice, subsequently chooses an action from its decision space ${\mathbf{\Omega }}_F$. Each player aims to maximize its utility functions $J_L$: ${\mathbf{\Omega }}_L\times {\mathbf{\Omega }}_F\to \mathbb{R}$ and $J_F$: ${\mathbf{\Omega }}_L\times {\mathbf{\Omega }}_F\to \mathbb{R}$, both functions of the decision spaces ${\mathbf{\Omega }}_L$ and ${\mathbf{\Omega }}_F$.

Since the leader acts first and announces his decision ${\omega }_L$, which is subsequently made known to the follower, the follower’s action becomes a function of the leader’s action ${\omega }_F=r_F\left({\omega }_L\right)$. The function $r_F$: ${\mathbf{\Omega }}_L\to {\mathbf{\Omega }}_F$ is called the reaction function of the follower (as it indicates how the follower will react to the leader’s decision). Knowing ${\omega }_L$, the follower chooses ${\omega }_F^{*}$, with 

$${\omega }_F^{*}\in \underset{{\omega }_F\in {\Omega }_F}{\arg \max }{J}_F({\omega }_L,{\omega }_F)=J_F({\omega }_L,r_F\left({\omega }_L\right)).$$

Taking into account the aforementioned, before the leader announces his decision ${\omega }_L$, he will realize how the follower will react and hence based on this knowledge, the leader will choose and subsequently announce the leader’s optimal decision ${\omega }_L^{*}\in {\mathbf{\Omega }}_L$, with 

$${\omega }_L^{*}\in \underset{{\omega }_L\in {\mathbf{\Omega }}_L}{\arg \max }{J}_L({\omega }_L,r_F\left({\omega }_L\right)).$$

In contrast to the original Stackelberg game discussed above, in the inverse Stackelberg game, the leader acts first by deciding and announcing an incentive function $\gamma$ to the follower, instead of ${\omega }_L$. This incentive function is a mapping of the follower’s decision space to the leader’s decision space, i.e., $\gamma$: ${\mathbf{\Omega }}_F\to {\mathbf{\Omega }}_L$, so the leader’s decision now follows directly from the follower’s decision. Note that the Stackelberg game is a special case of the inverse Stackelberg game where the incentive function maps to a constant, that is, the case in which $\gamma$: ${\mathbf{\Omega }}_F\to \left\{{\omega }_L\right\}$. In addition, the incentive function approach allows the leader to first determine a particular desired point $({\omega }_L^d,{\omega }_F^d)$ that he intends to achieve. A natural choice for this $({\omega }_L^d,{\omega }_F^d)$ would be a global optimum of the leader $({\omega }_L^{*},{\omega }_F^{*})\in \arg {\max }_{{\omega }_L\in {\mathbf{\Omega }}_L,{\omega }_F\in {\mathbf{\Omega }}_F}{J}_L({\omega }_L,{\omega }_F)$. Given the desired point, the problem can be formulated as follows:

$$\begin{array}{}\mathrm{(1)}& \hfill \mathrm{Find}:& \gamma \in \Gamma ,\hfill \mathrm{(2)}& \hfill \mathrm{s}.\mathrm{t}.& ({\omega }_L^d,{\omega }_F^d)\in \underset{{\omega }_F\in {\mathbf{\Omega }}_F}{\arg \max }{J}_F(\gamma \left({\omega }_F\right),{\omega }_F),\hfill \end{array}$$

where $\Gamma =\{\gamma$: ${\mathbf{\Omega }}_F\to {\mathbf{\Omega }}_L\}$ is the admissible set of incentive functions $\gamma$ from which the leader can choose. Even this static inverse Stackelberg problem with a single leader and a single follower is hard to solve analytically because of the function composition, the fact that there could be more than one best answer and the fact that the follower response might not be unique. We can say for sure that this inverse Stackelberg game is at least strongly NP-hard, as it can be posted as the following program:

$$\begin{array}{}\mathrm{(3)}& \hfill ({\omega }_L^d,{\omega }_F^d)\in & \underset{{\omega }_L\in {\mathbf{\Omega }}_L,{\omega }_F\in {\mathbf{\Omega }}_F}{\arg \max }{J}_L(\gamma \left({\omega }_F\right),{\omega }_F),\hfill \mathrm{(4)}& \hfill \mathrm{s}.\mathrm{t}.{\omega }_F^d\in & \underset{{\omega }_F\in {\mathbf{\Omega }}_F}{\arg \max }{J}_F(\gamma \left({\omega }_F\right),{\omega }_F).\hfill \end{array}$$

Taking ${\omega }_L$ as a free variable for $\gamma \left({\omega }_F\right)$, this is equivalent to a bilevel optimization problem formulation shown in Equation (5) next, which is proven to be strongly NP-hard even when the objective functions are linear [11].

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill \underset{{\mathbf{x}}_U\in {\mathbf{X}}_U,{\mathbf{x}}_L\in {\mathbf{X}}_L}{\max }& F({\mathbf{x}}_U,{\mathbf{x}}_L),\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& {\mathbf{x}}_{\mathbf{L}}\in \arg \max \left\{f({\mathbf{x}}_U,{\mathbf{x}}_L)\mid g_i({\mathbf{x}}_U,{\mathbf{x}}_L)\le 0,i\in \mathbf{I}\right\},\hfill \\ \hfill & G_j({\mathbf{x}}_U,{\mathbf{x}}_L)\le 0,j\in \mathbf{J},\hfill \end{array}\end{array}$$

(5)

where ${\mathbf{X}}_U$, ${\mathbf{X}}_L$ denote the decision space of the upper and lower levels, respectively, F: ${\mathbf{X}}_U\times {\mathbf{X}}_L\to \mathbb{R}$ is the upper-level objective function, f: ${\mathbf{X}}_U\times {\mathbf{X}}_L\to \mathbb{R}$ is the lower-level objective function, and the functions $g_i$: ${\mathbf{X}}_U\times {\mathbf{X}}_L\to \mathbb{R}$, $G_i$: ${\mathbf{X}}_U\times {\mathbf{X}}_L\to \mathbb{R}$ represents the lower-level and upper-level constraints respectively, for $i\in \mathbf{I}\subseteq \mathbb{N}$, $j\in \mathbf{J}\subseteq \mathbb{N}$. This nested two-level optimization task structure requires that only optimal solutions of the lower-level task are acceptable as feasible solutions of the upper-level task.

2.2. The Single-Stage Principal–Agent Problem

In practice, the leader and follower agents base their decisions on a set of information that is readily accessible to them. Incentive design (ID), also known as contract theory, investigates inverse Stackelberg games under a variety of information models available to the leader and the follower agent and the information asymmetries between them. In this domain, the inverse Stackelberg game is called the principal–agent problem, with the leader as the principal and the follower as the agent. The way this information set is conceived and theoretically described is a major factor in how the fields of economics, control theory, and machine learning approach the principal–agent problem.

In this research, we focus on the single-stage principal–agent problem (SS-PAP), which is a hierarchical inverse Stackelberg game with a specific information model played between a single principal (leader) and a single follower. This single-stage game problem is as follows: First, the principal decides or selects an incentive function $\gamma$ from a predetermined set of functions $\Gamma =\{\gamma$: ${\mathbf{\Omega }}_F\to {\mathbf{\Omega }}_L\}$, and announces $\gamma$ to the follower, without knowledge of the follower’s utility. Then, with knowledge of this incentive function $\gamma$, the follower agent selects an action ${{\omega }_F}^{*}\in {\mathbf{\Omega }}_F$ that maximizes his utility $J_F(\gamma \left({\omega }_F\right),{\omega }_F)$. We do not impose restrictive assumptions on the functional form of utilities. Regarding the information model, we assume that both the principal and the follower agent possess private information, i.e., the utility function of the follower is unknown to the principal and vice versa. The outcome or result in this single-stage principal–agent problem are the utilities $J_F\left({\omega }_F^{*}\right),{\omega }_F^{*})$ and $J_L\left({\omega }_F^{*}\right),{\omega }_F^{*})$ after the principal decides the incentive function $\gamma$. Our focus is on the principal’s perspective of the problem, where the principal wants to select the incentive function $\gamma$ that maximizes its utility $J_L$.

We assume that the principal solves this problem as a sequential optimization problem with finite horizon T, using previously observed historical data, given as N observations ${\mathcal{D}}^{-1}=\left\{\right({\gamma }^i,{\omega }_F^i,{\omega }_L^i,J_L^i{\left)\right\}}_{i=1}^N$ of the system. That is, using ${\mathcal{D}}^{-1}=\left\{\right({\gamma }^i,{\omega }_F^i,{\omega }_L^i,J_L^i{\left)\right\}}_{i=1}^N$, to estimate $\mathbb{E}\left[J_L^t\right]$ given that an incentive function ${\gamma }^t\in \Gamma$ is applied, for rounds T, the principal decides on the best ${\gamma }^{*}\in \Gamma$ to implement in the SS-PAP. In other words, the principal (as the decision-maker), at each stage $t\in \left[T\right]$, is responsible for the development of a policy ${\pi }^t$: ${\mathcal{D}}^{t-1}\to {\gamma }^t$, where ${\mathcal{D}}^{t-1}=\left\{({\gamma }^1,\mathbb{E}\left[J_L^1\right]),\dots ,({\gamma }^{t-1},\mathbb{E}\left[J_L^{t-1}\right])\right\}$; and when the accumulated history data set ${\mathcal{D}}^T$ is complete, the principal decides the incentive function ${\gamma }^{+}$ such that

$${\gamma }^{+}\in \underset{\left({\gamma }^t,\mathbb{E}\left[J_L^t\right]\right)\in {\mathcal{D}}^T}{\arg \max }\left\{\mathbb{E}\left[J_L^1\right],\dots ,\mathbb{E}\left[J_L^T\right]\right\}.$$

(6)

Therefore, we deal with a sequential optimization problem, and we take a Bayesian approach to the optimization of the functional objective of the principal. Appendix C provides details about Bayesian optimization (BO), which specializes in sequentially optimizing objectives with the above-mentioned characteristics, relying on a statistical model of the objective function whose beliefs guide the algorithm in making the most fruitful decisions.

2.3. Causal Graphical Models and Causal Inference

This research proposes solving the single-stage principal agent problem (SS-PAP) as defined in the previous subsection by incorporating causal reasoning. In this section, we provide the essential concepts of causal inference required to understand the Causal Incentive Design (CID) framework proposed in Section 3 to solve SS-PAP.

A causal graphical model (CGM) consists of a directed acyclic graph (DAG) $\mathcal{G}$, which is called the causal structure, together with a four-tuple $\langle \mathit{U},\mathit{V},\mathit{F},p\left(\mathit{U}\right)\rangle$ where

• $\mathit{U}$ is a set ${\left\{U_i\right\}}_{i\in \left[p\right]}$ of exogenous (unobserved) variables, which are determined by factors external to the model;
• $\mathit{V}$ is a set ${\left\{V_i\right\}}_{i\in \left[p\right]}$, with $p\in \mathbb{N}$, of endogenous (observed) variables.
• $\mathit{F}=\{f_{V_1},\dots ,f_{V_p}\}$ is a set of functions known as structural equations, such that $V_i=f_{V_i}(\mathit{pa}\left(v_i\right),u_i)$, for each $V_i\in \mathit{V}$, with $\mathit{pa}\left(V_i\right)\subset \mathit{V}$ denoting the parents of $V_i$;
• $p\left(\mathit{U}\right)$ is a set of probability distributions ${\left\{p\left(U_i\right)\right\}}_{i\in \left[p\right]}$ for each $U_i$, where $U_i$ is a random disturbance distributed according to $p\left(U_i\right)$, independently of all other $U_j$ with $i\ne j$.

The set of vertices in the causal structure $\mathcal{G}$ corresponds to a set of random variables $\mathit{V}=\{V_1,\dots ,V_p\}$ and each edge of the set of edges $\mathit{E}$ represents a direct functional relationship between the corresponding variables, expressed by saying that $V_i$ is the direct cause of $V_j$ for an edge $(V_i,V_j)\in \mathit{E}$. So, the graph $\mathcal{G}$ encodes the causal relationship between the variables in $\mathit{V}$. We assume that the system $\mathit{V}$ is causally sufficient, and therefore there are no variables in $\mathit{V}$ that are common direct causes of at least two observed variables that are unmeasured. Let $\mathcal{G}$ be the causal structure of a CGM, i.e., a directed acyclic graph (DAG) with variables $\mathit{V}$. The statistical model associated to $\mathcal{G}$ consists of all distributions that factorize as

$$p\left(\mathit{V}\right)=\prod _{V\in \mathit{V}}p\left(V\mid {\mathit{pa}}_{\mathcal{G}}\left(V\right)\right),$$

(7)

where ${\mathit{pa}}_{\mathcal{G}}\left(V\right)$ denotes the parents of V in $\mathcal{G}$.

The notation $do(X=x)$ indicates an idealized experiment or intervention in which the values of X are set to x, for some $X\in \mathit{X}\subseteq \mathit{V}$. Hard interventions is the term used to describe this category of interventions. A fundamental problem in Causal Inference is to identify $p\left(y\right|do(X=x))$ from observational data. In practice, experimental data are not always accessible due to the fact that randomized controlled experiments might be impractical, costly, or even unethical. Within the causal system $\mathit{V}$, we distinguish three different types of variables: no manipulative variables $\mathit{C}$, treatment variables $\mathit{X}$ that can be set to specific values, i.e., intervene them, and a target variable Y that represents the outcome of interest. For a fully observed causal system $\mathit{V}$ with a causal structure DAG $\mathcal{G}$, the post-intervention distribution can be identified through the g-formula, also known as the truncated formula. We follow the standard g-formula identification [26]:

$$p(\mathit{V}\setminus X\mid do(X=x))=\prod _{V\in \mathit{V}\setminus X}p\left(V\mid \mathit{pa}\left(V\right)\right){|}_{X=x}.$$

(8)

So, a CGM induces joint observational and interventional probability distributions given in Equations (7) and (18). Pearl (see [26]) formalized the concept of causal effect of X on Y after manipulating X, that is, doing $X=x$ for some $x\in \mathcal{R}\left(X\right)$, where $\mathcal{R}(·)$ denotes the range of a random variable, as the expected value of the distribution $p\left(Y\right|do(X=x))$, i.e., the distribution of Y after operation $do(X=x)$, denoted as ${\mathbb{E}}_{p\left(Y\right|do(X=x))}\left[Y\right]$ or $\mathbb{E}\left[Y\mid do(X=x)\right]$. Like any ordinary probability distribution, $p\left(·\mid do(X=x)\right)$ obeys the same rules of conditioning and marginalization. So, we can specify expectations in the conventional manner, as shown in Equation (9).

$$\mathbb{E}\left[Y\mid do(X=x)\right]={\int }_{\mathcal{R}\left(Y\right)}yp\left(y\mid do(X=x)\right)dy$$

(9)

The traditional hard interventions discussed above are unconditional actions that simply force a variable X to adopt a designated value x. Functional interventions refer to a broader category of interventions in which a variable X is manipulated to respond in a predetermined manner to a set of other observable variables $\mathbf{Z}\subseteq \mathit{Pa}\left(X\right)$ in the causal system through a functional relationship $X=g\left(\mathbf{z}\right)$. Specifically, a functional intervention can be considered as a substitution of the structural equation $f_X\in \mathit{F}$, of the intervened variable X, with a deterministic function of some parents in the CGM. In other words, by a functional intervention, we substitute the structural equation $X=f_X(\mathbf{pa}\left(\mathbf{x}\right),{\mathbf{u}}_{\mathbf{x}})$ with a function $X=g\left(\mathbf{z}\right)$ for some $\mathbf{Z}\subseteq \mathit{Pa}\left(X\right)$ in the causal model. It is important to note that hard interventions are the specific instance of functional interventions in which the structural equation $f_X$ is substituted by a constant function g.

Pearl first explored functional interventions under the name of conditional actions in [2]. More recently, Correa and Bareinboim in [3] introduced a new set of inference rules that extends the do-calculus for hard interventions in [2] to derive claims about functional interventions. Following these, let $p(y\mid do(X=g\left(\mathbf{z}\right)\left)\right)$ stand for the post-intervention distribution of Y established under the policy $X=g\left(\mathbf{z}\right)$. To compute the post-intervention distribution $p(y\mid do(X=g\left(\mathbf{z}\right)\left)\right)$ for the functional intervention $do(X=g(\mathbf{z}\left)\right)$, we condition on $\mathbf{Z}$ and write

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill & p(y\mid do(X=g\left(\mathbf{z}\right)\left)\right)\hfill \\ \hfill & ={\int }_{\mathcal{R}\left(\mathbf{Z}\right)}p(y\mid do(X=g\left(\mathbf{z}\right)),\mathbf{z})p(\mathbf{z}\mid do(X=g\left(\mathbf{z}\right)))\hfill \\ \hfill & ={\int }_{\mathcal{R}\left(\mathbf{Z}\right)}p(y\mid do(X=g\left(\mathbf{z}\right)),\mathbf{z})p\left(\mathbf{z}\right)\hfill \\ \hfill & =\mathbb{E}\left[p\right(y\mid do(X=g(\mathbf{z}\left)\right),\mathbf{z}\left)\right]\mathrm{with}x=g\left(\mathbf{z}\right),\hfill \end{array}\end{array}$$

(10)

where the equality $p(\mathbf{z}\mid do(X=g\left(\mathbf{z}\right)\left)\right)=p\left(\mathbf{z}\right)$ comes from the fact that $\mathbf{Z}$ cannot be a descendant of X, and therefore any intervention in X has no effect on the distribution of $\mathbf{Z}$. Thus, the causal effect of a functional intervention $X=g\left(\mathbf{Z}\right)$ can be computed from the expression $p(y\mid do(X=x),\mathbf{z})$ by substituting x for $g\left(\mathbf{z}\right)$ and taking the expectation over $\mathbf{Z}$.

3. Causal Incentive Design for Single-Stage Canonical PAPs

In this section, we develop the proposed Causal Incentive Design (CID) framework by showing how causal inference can be incorporated into the principal–agent problem described in Section 2.2 to solve it. First in Section 3.1, we propose the general additive-noise Gaussian CGM ${\mathcal{M}}_{L_1{F}_1}$ that represents the dynamics of the single-stage canonical principal–agent problem $PA{P}_{L_1{F}_1}$ and describe its key semantic aspects and underlying assumptions. This $PA{P}_{L_1{F}_1}$ is the canonical version in which the principal–agent problem is analyzed with a single principal controlling a unique decision variable ${\omega }_L\in {\mathbf{\Omega }}_L$ and a single follower agent with a unique decision variable ${\omega }_F\in {\mathbf{\Omega }}_F$. The core concepts of CID can be elucidated through the examination of the canonical principal–agent problem $PA{P}_{L1F1}$ and the CGM ${\mathcal{M}}_{L_1{F}_1}$ that represents it.

The CID for the single-stage PAP is developed in three parts: (i) specification of the additive-noise CGM for the single-stage PAP and definition of the policy-value target $V\left(\gamma \right)=\mathbb{E}[J_L\mid do(\Gamma =\gamma )]$ (Section 3.1); (ii) derivation of a semi-parametric identification for $V\left(\gamma \right)$ via the g-formula, demonstrating its decomposition into an inner Gaussian smoothing and an outer expectation over the follower’s action (Section 3.2); and (iii) presentation of a two-layer estimator (Section 3.3), which first computes the inner Gaussian expectation (Section 3.3.3), and then approximates the outer expectation using policy-local kernel reweighting with accompanying diagnostics (Section 3.3.4).

3.1. A Causal Graphical Model for the Single-Stage Canonical Principal–Agent Problem

We consider the single-stage canonical principal–agent problem $PA{P}_{L_1{F}_1}$ as a causal system with endogenous observable random variables $\mathbf{V}=\{\Gamma ,{\Omega }_L,{\Omega }_F,J_F,J_L\}$, where $\Gamma$ is the admissible set of incentive functions with range ${\mathcal{R}}_{\Gamma }=\Gamma$. The variables ${\Omega }_L,{\Omega }_F$ represent the decision variables of the principal and the follower, respectively, with ranges ${\mathcal{R}}_{{\Omega }_L}={\mathbf{\Omega }}_L$ and ${\mathcal{R}}_{{\Omega }_F}={\mathbf{\Omega }}_F$, where ${\mathbf{\Omega }}_L$, ${\mathbf{\Omega }}_F$ are the decision spaces of the principal and the follower, respectively. The variables $J_F,J_L$ represent the values of the utility functions of the follower and the principal, respectively.

Definition 1.

The CGM ${\mathcal{M}}_{L_1{F}_1}$ for $PA{P}_{L_1{F}_1}$ has endogenous variables set $\mathbf{V}$, exogenous variables set $\mathit{U}$, and structural equations set $\mathit{F}$ given as follows:

$$\mathbf{V}=\{\Gamma ,{\Omega }_L,{\Omega }_F,J_F,J_L\},\mathit{U}=\{U_{{\Omega }_L},U_{{\Omega }_F},U_{J_F},U_{J_L}\},\mathit{F}=\{f_{\Gamma },f_{{\Omega }_L},f_{{\Omega }_F},f_{J_F},f_{J_L}\},\mathit{with}$$

$$\begin{array}{}\mathrm{(11)}& \hfill \Gamma & := \gamma \hfill \mathrm{(12)}& \hfill {\Omega }_F& := f_{{\Omega }_F}(\gamma ,{ϵ}_{{\Omega }_F})=BR\left(\gamma \right)+{ϵ}_{{\Omega }_F}\hfill \mathrm{(13)}& \hfill {\Omega }_L& := f_{{\Omega }_L}(\gamma ,{\Omega }_F,{ϵ}_{{\Omega }_L})=\gamma \left({\Omega }_F\right)+{ϵ}_{{\Omega }_L}\hfill \mathrm{(14)}& \hfill J_F& := f_{J_F}({\Omega }_L,{\Omega }_F,{ϵ}_{J_F})=J_F({\Omega }_L,{\Omega }_F)+{ϵ}_{J_F}\hfill \mathrm{(15)}& \hfill J_L& := f_{J_L}({\Omega }_L,{\Omega }_F,{ϵ}_{J_L})=J_L({\Omega }_L,{\Omega }_F)+{ϵ}_{J_L}\hfill \end{array}$$

where ${ϵ}_{{\Omega }_F}\in {\mathcal{R}}_{U_{{\Omega }_F}}$, ${ϵ}_{{\Omega }_L}\in {\mathcal{R}}_{U_{{\Omega }_L}}$, ${ϵ}_{{\Omega }_{J_F}}\in {\mathcal{R}}_{U_{J_F}}$, ${ϵ}_{{\Omega }_{J_L}}\in {\mathcal{R}}_{U_{J_L}}$; with $p\left(U_i\right)=\mathcal{N}(0,{\sigma }_i^2)$ for all $U_i\in \mathit{U}$ and $p\left(\mathbf{U}\right)$ is jointly independent. The function $\gamma :{\mathcal{R}}_{{\Omega }_F}\to {\mathcal{R}}_{{\Omega }_L}\in \Gamma$ is an incentive function, decided by the principal from the function space Γ. The functional $BR\left(\gamma \right)$ is the best response functional for the follower agent given γ. The functions $J_F({\omega }_L,{\omega }_F)$, $J_L({\omega }_L,{\omega }_F)$ are the utility functions of the follower and the leader, respectively, in the $PA{P}_{L1F1}$.

Assumptions Underlying the CGMs for Single-Stage Canonical PAPs

The CGM ${\mathcal{M}}_{L_1{F}_1}$ is an additive-noise Gaussian, Structural Causal Model (SCM); not inherently linear, as we do not assume that the structural equations $f_{V_i}(\mathit{pa}\left(v_i\right),u_i)$, for each $V_i\in \mathit{V}$, hold a specific functional form. We assume that the set of endogenous variables $\mathbf{V}=\{\Gamma ,{\Omega }_L,{\Omega }_F,J_F,J_L\}$ in the CGM—namely, the incentives variable $\Gamma$, the principal’s decision variable ${\Omega }_L$, the follower’s decision variable ${\Omega }_F$, and their respective utilities $J_L$ and $J_F$—is causally sufficient. That is, there are no latent common causes outside this set that jointly influence any two of these variables.

The causal structure ${\mathcal{G}}_{L_1{F}_1}$ of ${\mathcal{M}}_{L_1{F}_1}$, has directed edges from $\Gamma$ to $\{{\Omega }_L,{\Omega }_F\}$, from ${\Omega }_L$ to $\{J_F,J_L\}$, and from ${\Omega }_F$ to $\{{\Omega }_L,J_F,J_L\}$. The variable $\Gamma$ does not depend on an exogenous random variable or any other variable in the causal system; it is a deterministic root variable whose value is decided and fixed by the principal choice of $\gamma \in \Gamma$. Thus, $\gamma$ acts as a fixed input to the causal system, selected by the principal agent.

The best response functional $BR\left(\gamma \right)$ in the structural Equation (13), is the mapping from the principal’s chosen incentive function $\gamma$ to the action ${\omega }_F^{*}$ that maximizes the follower’s utility. That is,

$$BR\left(\gamma \right)={\omega }_F^{*}\in \underset{{\omega }_F\in {\Omega }_F}{\arg \max }\mathbb{E}\left[J_F\left(\gamma \left({\omega }_F\right)+{ϵ}_{{\Omega }_L},{\omega }_F\right)\right].$$

(16)

This formulation reveals how the follower’s decision responds to the incentive function chosen by the principal. The structural Equation (14), ${\Omega }_L:= f_{{\Omega }_L}(\gamma ,{\Omega }_F,{ϵ}_{{\Omega }_L})=\gamma \left({\Omega }_F\right)+{ϵ}_{{\Omega }_L}$, is central to represent the inverse Stackelberg game semantics. It expresses that after the principal proposes the incentive function $\gamma$, the principal’s action ${\Omega }_L$ is given as a function $\gamma \left({\omega }_F\right)$ of the follower’s action ${\omega }_F$. This composition in the directed acyclic triangle $\Gamma \to {\Omega }_F$, ${\Omega }_F\to {\Omega }_L$, $\Gamma \to {\Omega }_L$, with the structural Equations (13) and (14), and it highlights how the principal’s concrete action ${\Omega }_L$ responds to the follower’s action, influenced by the incentive function $\gamma$ selected by the principal. The utility variables $J_L$ and $J_F$ are defined as functions of $({\Omega }_L,{\Omega }_F)$, consistent with conventional formulations of principal–agent problems.

The generative causal model ${\mathcal{M}}_{L_1{F}_1}$ must be examined from two perspectives: the principal’s and the follower’s. From the principal’s view, the follower’s utility $J_F$ is an unobservable or hidden variable. So, for the principal, the set of hidden variables is ${\mathbf{H}}_L=\left\{J_F\right\}$, while the set of observable variables is ${\mathbf{O}}_L=\{\Gamma ,{\Omega }_L,{\Omega }_F,J_L\}$, with ${\mathbf{H}}_L\cup {\mathbf{O}}_L=\mathbf{V}$. On the other hand, from the follower’s view, ${\mathbf{H}}_F=\left\{J_L\right\}$ and ${\mathbf{O}}_F=\{\Gamma ,{\Omega }_L,{\Omega }_F,J_F\}$, with ${\mathbf{H}}_F\cup {\mathbf{O}}_F=\mathbf{V}$. These separated perspectives are just marginalizations of the same generative SCM ${\mathcal{M}}_{L_1{F}_1}$, but maintain consistency with the information model in $PA{P}_{L_1,F_1}$.

It is also important to establish that, with respect to the information model in $PA{P}_{L_1,F_1}$, the principal views the follower mechanism $f_{{\Omega }_F}(\gamma ,U_{{\Omega }_F})$ as a black box. In other words, the principal does not know the function $BR\left(\gamma \right)$ that the follower uses to decide ${\omega }_F$, but only observes the decision ${\omega }_F$. This work centers on the principal’s perspective, and for the remainder, we will concentrate exclusively on this view of the causal model ${\mathcal{M}}_{L_1{F}_1}$, assuming $\mathbf{V}={\mathbf{O}}_L\cup {\mathbf{H}}_L$ and treating $f_{{\Omega }_F}(\gamma ,U_{{\Omega }_F})$ as a black box.

3.2. A Non-Parametric Identification for the Causal Inference Target

In the Causal Incentive Design (CID) framework, with respect to the single-stage $PA{P}_{L_1{F}_1}$, the principal’s objective is to determine an incentive function ${\gamma }^{*}$ such that its utility $J_L$ is maximized, utilizing prior observations of the system in question. To accomplish this, the principal should first be able to estimate the expected value of their utility $J_L$, in the particular scenario where the incentive function $\gamma$ is applied, ideally utilizing only observational data.

In the context of causal inference, this requires determining whether the expected value of the utility of the principal $\mathbb{E}\left[J_L\right]$ in the post-intervention distribution $p\left(J_L\mid do(\Gamma =\gamma )\right)$ is identifiable. In other words, determine whether

$${\mathbb{E}}_{p\left(J_L\mid do(\Gamma =\gamma )\right)}\left[J_L\right]\mathrm{or}\mathrm{equivalently},\mathbb{E}\left[J_L\mid do(\Gamma =\gamma )\right],$$

(17)

can be estimated using exclusively observational data from variables $\left(\Gamma ,{\Omega }_L,{\Omega }_F,J_L\right)$, i.e., a data set $\mathcal{D}={\left\{{\gamma }^i,{\omega }_F^i,{\omega }_L^i,j_L^i\right\}}_{i=1}^N$ of $N\in \mathbb{N}$ observations of the system. We also employ the simpler notation ${\mathbb{E}}^{\gamma }\left[J_L\right]$ to denote $\mathbb{E}\left[J_L\mid do(\Gamma =\gamma )\right]$. Recall that the variable $J_F$ is a hidden variable from the principal’s perspective; so the principal has no knowledge or information about this variable.

Identification via the g-Formula

We first use the g-formula over the principal’s perspective to construct a non–parametric identification for the causal inference target $\mathbb{E}\left[J_L\mid do(\Gamma =\gamma )\right]$, which is then adequate to become a semi-parametric target estimand by considering the Gaussian additive independent errors on the CGM ${\mathcal{M}}_{L_1{F}_1}$. The g-formula over the principal’s perspective for the intervention $do(\Gamma =\gamma )$ is given by

$$p({\mathbf{O}}_L\setminus \Gamma \mid do(\Gamma =\gamma ))=\prod _{O_L\in {\mathbf{O}}_L\setminus \Gamma }p(O_L\mid \mathrm{pa}(O_L)){|}_{\Gamma =\gamma }.$$

(18)

That is, we obtain the post-intervention distribution $p({\mathbf{O}}_L\setminus \Gamma \mid do(\Gamma =\gamma ))=p\left({\omega }_F,{\omega }_L,j_L\mid do(\Gamma =\gamma )\right)$, removing all factors from the pre-intervention distribution $p\left({\mathbf{O}}_L\right)$ corresponding to the intervened variable and substituting $\Gamma =\gamma$ into the remaining factors. So, the post-interventional distribution $p\left({\omega }_F,{\omega }_L,j_L\mid do(\Gamma =\gamma )\right)$ is factorized as shown in Equation (19). Then, we can estimate the target post-intervention distribution $p(j_L\mid do(\Gamma =\gamma ))$ by marginalizing the variables ${\omega }_F$ and ${\omega }_L$ on the factorization for $p\left({\omega }_F,{\omega }_L,j_L\mid do(\Gamma =\gamma )\right)$, as shown in Equation (20).

$$p\left({\omega }_F,{\omega }_L,j_L\mid do(\Gamma =\gamma )\right)=p({\omega }_F\mid \Gamma =\gamma )p({\omega }_L\mid {\omega }_F,\Gamma =\gamma )p(j_L\mid {\omega }_L,{\omega }_F,\Gamma =\gamma )$$

(19)

$$p(j_L\mid do(\Gamma =\gamma ))=\int \int p(j_L\mid {\omega }_L,{\omega }_F,\Gamma =\gamma )p({\omega }_L\mid {\omega }_F,\Gamma =\gamma )p({\omega }_F\mid \Gamma =\gamma )d{\omega }_{L}d{\omega }_F.$$

(20)

The target functional value $V\left(\gamma \right)=\mathbb{E}\left[J_L\mid do(\Gamma =\gamma )\right]$ under the incentive function (policy) $\gamma$ is the expectation of $J_L$ in the post-intervention world, that is,

$$\begin{array}{cc}\hfill V\left(\gamma \right)& ={\mathbb{E}}^{\gamma }\left[J_L\right]=\int j_{L}p(j_L\mid do(\Gamma =\gamma ))d{j}_L.\hfill \end{array}$$

(21)

Therefore, substituting the factorization form (Equation (20)) of the post-intervention distribution $p(j_L\mid do(\Gamma =\gamma ))$, we obtain the following:

$$\begin{array}{}\mathrm{(22)}\hfill V\left(\gamma \right)& =\int j\left\{\int \int p(j\mid {\omega }_L,{\omega }_F,\Gamma =\gamma )p({\omega }_L\mid {\omega }_F,\Gamma =\gamma )p({\omega }_F\mid \Gamma =\gamma )d{\omega }_{L}d{\omega }_F\right\}dj.\hfill \mathrm{(23)}\hfill & =\int \int \left\{\int jp(j\mid {\omega }_L,{\omega }_F,\Gamma =\gamma )dj\right\}p({\omega }_L\mid {\omega }_F,\Gamma =\gamma )p({\omega }_F\mid \Gamma =\gamma )d{\omega }_{L}d{\omega }_F,\hfill \end{array}$$

where the exchange of integrals is allowed by the Tonelli/Fubini Theorem. Observe that the inner integral is the following conditional mean:

$$\begin{array}{cc}\hfill \int jp(j\mid {\omega }_L,{\omega }_F,\Gamma =\gamma )dj& =\mathbb{E}[Y\mid {\Omega }_L={\omega }_L,{\Omega }_F={\omega }_F,\Gamma =\gamma ]\hfill \end{array}$$

(24)

Let us denote ${\mu }_L({\omega }_L,{\omega }_F,\gamma ):= \mathbb{E}[Y\mid {\Omega }_L={\omega }_L,{\Omega }_F={\omega }_F,\Gamma =\gamma ]$. Thus, we have

$$\begin{array}{}\mathrm{(25)}\hfill V\left(\gamma \right)& =\int \int {\mu }_L({\omega }_L,{\omega }_F,\gamma )p({\omega }_L\mid {\omega }_F,\Gamma =\gamma )p({\omega }_F\mid \Gamma =\gamma )d{\omega }_{L}d{\omega }_F\hfill \mathrm{(26)}\hfill & =\int \left\{\int {\mu }_L({\omega }_L,{\omega }_F,\gamma )p({\omega }_L\mid {\omega }_F,\Gamma =\gamma )d{\omega }_L\right\}p({\omega }_F\mid \Gamma =\gamma )d{\omega }_F.\hfill \end{array}$$

From structural Equation (14), ${\Omega }_L:= f_{{\Omega }_L}(\gamma ,{\Omega }_F,{ϵ}_{{\Omega }_L})=\gamma \left({\Omega }_F\right)+{ϵ}_{{\Omega }_L}$, if $U_{{\Omega }_L}$ has density $f_{U_{{\Omega }_L}}$, then for a fixed ${\omega }_F$, we have $p({\omega }_L\mid {\omega }_F,\Gamma =\gamma )=f_{U_L}({\omega }_L-\gamma \left({\omega }_F\right))$. Hence, the inner integral of Equation (26) becomes

$$\int {\mu }_L({\omega }_L,{\omega }_F,\gamma )f_{U_L}({\omega }_L-\gamma \left({\omega }_F\right))d{\omega }_L.$$

(27)

Which, by the change of variable $r={\omega }_L-\gamma \left({\omega }_F\right)$, $d{\omega }_L=d\left(r+\gamma \left({\omega }_F\right)\right)$, becomes

$$\int {\mu }_L(\gamma \left({\omega }_F\right)+t,{\omega }_F,\gamma )f_{U_{{\Omega }_L}}\left(r\right)d\left(r+\gamma \left({\omega }_F\right)\right)={\mathbb{E}}_{U_{{\Omega }_L}}\left[{\mu }_L(\gamma \left({\omega }_F\right)+U_{{\Omega }_L},{\omega }_F,\gamma )\right],$$

(28)

since $r={\omega }_L-\gamma \left({\omega }_F\right)={ϵ}_{{\Omega }_L}\sim p\left(U_{{\Omega }_L}\right)$, as ${\Omega }_L=\gamma \left({\omega }_F\right)+U_{{\Omega }_L}$ with $U_{{\Omega }_L}\perp {\Omega }_F$. Therefore, Equation (26) can be expressed as

$$\begin{array}{cc}\hfill V\left(\gamma \right)& =\int \left\{\int {\mu }_L({\omega }_L,{\omega }_F,\gamma )p({\omega }_L\mid {\omega }_F,\Gamma =\gamma )d{\omega }_L\right\}p({\omega }_F\mid \Gamma =\gamma )d{\omega }_F\hfill \\ \hfill & =\int \left\{{\mathbb{E}}_{U_{{\Omega }_L}}\left[{\mu }_L(\gamma \left({\omega }_F\right)+U_{{\Omega }_L},{\omega }_F,\gamma )\right]\right\}p({\omega }_F\mid \Gamma =\gamma )d{\omega }_F.\hfill \end{array}$$

(29)

The outer integral of Equation (29) corresponds to averaging over the follower’s action distribution $p({\Omega }_F\mid \Gamma =\gamma )$ induced by the policy $\gamma$. Recall that from the principal’s perspective, the follower mechanism ${\Omega }_F=f_{{\Omega }_F}(\gamma ,U_{{\Omega }_F})$ is unknown and is treated as a black-box mechanism.

3.3. Estimations on the Semi-Parametric Identification Formula

We now demonstrate how to estimate $V\left(\gamma \right)$; first through a working example in the context of the credit market, and subsequently, we describe how the estimation can be conducted in general under additive, independent, mean-zero Gaussian noise, without committing to a particular parametric form for the structural functions. In this section, for simplicity and clarity on the estimation task, we rename two of the causal system variables. Here, we use $M:={\Omega }_F$ to denote the follower’s action and refer to it as the mediator, because it acts as a mediator on the causal path from a fixed incentive policy $\gamma \in \Gamma$ to the principal’s utility $J_L$. We refer here to the principal’s utility as the outcome of the system, and we denote it here as $Y:=J_L$ for convenience.

Before going into the details for the estimation on the linear parametrized PAP example and the general case estimation; we provide an introduction of the general components to learn on the identified estimand:

$$\begin{array}{}\mathrm{(30)}\hfill V\left(\gamma \right)& =\mathbb{E}\left[Y\mid do(\Gamma =\gamma )\right]\hfill \mathrm{(31)}\hfill & =\int \left\{\underset{\mathrm{Inner}\mathrm{expectation}\mathrm{for}\mathrm{the}\mathrm{outcome}\mathrm{model}.}{\underbrace{{\mathbb{E}}_{U_L}\left[{\mu }_L(\gamma \left(m\right)+U_L,m,\gamma )\right]}}\right\}\underset{\text{Black-box}\mathrm{mediator}\mathrm{conditional}.}{\underbrace{p(m\mid \Gamma =\gamma )}}dm,\hfill \end{array}$$

where ${\mu }_L({\omega }_L,m,\gamma )=\mathbb{E}[Y\mid {\Omega }_L={\omega }_L,M=m,\Gamma =\gamma ]$, and ${\Omega }_L=\gamma \left(M\right)+U_L$ holds. The two main components are inner expectation for the outcome model and the outer black-box mediator conditional, as shown in Equation (31).

For the first, we unpack the nested structure by first estimating the action mechanism ${\Omega }_L=f_{{\Omega }_L}(\gamma ,M,U_L)={\widehat{f}}_L(\gamma ,M)+{\widehat{\sigma }}_L(\gamma ,M)U_L$ by learning ${\widehat{f}}_L(\gamma ,M)$ and ${\widehat{\sigma }}_L(\gamma ,M)$ from the data; i.e., regress ${\Omega }_L$ on $(M,\Gamma )$ for the mean ${\widehat{f}}_L$ and regress squared residuals on $(M,\Gamma )$ to obtain ${\widehat{\sigma }}_L^2$. With these, we learn the conditional outcome mean ${\mu }_L({\omega }_L,m,\gamma )=\mathbb{E}[Y\mid {\Omega }_L={\omega }_L,M=m,\Gamma =\gamma ]$ as regression problem with features $({\Omega }_L,M,\Gamma )$ and outcome Y. Then, the inner expectation for the outcome model (see Equation (31)) may be computed by Gauss–Hermite quadrature or Monte Carlo using common random numbers across m to reduce variance.

The second main component is the post-intervention mediator law (what the follower does under the policy $\gamma$); the black-box mediator conditional $p(m\mid \Gamma =\gamma )$. We approximate $p(m\mid \Gamma =\gamma )$ near the target policy by a policy-local reweighting of the observation, selecting a policy-space metric $d_{\mathcal{G}}$ and kernel K with bandwidth h. Thus, the outer expectation over the mediator law under a fixed $\gamma$ is computed by policy-local kernel weights. Therefore, in general, the inner layer (first component) is a Gaussian smoothing of the outcome regression around the mean principal’s action (in linear models, the variance drops out and we recover the closed form; in general, we keep a small quadrature); and the outer layer (second component) is a weighted average over observed mediators. The kernel with bandwidth provides a principled way to compute those weights with global regularization as we show in the following sections.

Since empirical performance is determined by domain-specific design choices, such as policy parameterization, kernel selection on policy space, bandwidth h, and support criteria, we defer numerical examples and simulations to future work that will focus on applied instantiations of the present framework. We concentrate on formal properties that hold independent of the chosen application.

3.3.1. The Estimation of a Linear Parametrized Instance

Consider a financial institution that is required to establish a transparent pricing plan before evaluating a potential borrower’s response. The bank (the principal) announces an affine policy, which includes a fixed fee and an interest “slope” that is dependent on the size of the loan. The bank, thereafter, observes how much the business (the follower) actually borrows. This is an instance of a canonical $PA{P}_{L_1{F}_1}$, which we can parameterize as the following linear Gaussian Structural Causal Models (SCM).

$$\begin{array}{}\mathrm{(32)}\hfill \Gamma & := {\gamma }_{(\alpha ,\beta )}\left(m\right)=\alpha m+\beta ,\hfill \mathrm{(33)}\hfill M& := {\mu }_M(\alpha ,\beta )+U_M,U_M\sim \mathcal{N}(0,{\sigma }_M^2),\hfill \mathrm{(34)}\hfill {\Omega }_L& := \alpha M+\beta +U_{{\Omega }_L},U_{{\Omega }_L}\sim \mathcal{N}(0,{\sigma }_L^2),\hfill \mathrm{(35)}\hfill Y& := b+r{\Omega }_L+qM+U_Y,U_Y\sim \mathcal{N}(0,{\sigma }_Y^2),\hfill \end{array}$$

with $U_M,U_{{\Omega }_L},U_Y$ mutually independent and independent of $\Gamma$. We model the bank’s pricing rule as an affine policy ${\gamma }_{(\alpha ,\beta )}\left(m\right)=\alpha m+\beta$, with $\alpha \ge 0$, $\beta \ge 0$; so, we have an affine pricing as the incentive functions family. As we have assumed from the principal’s perspective, the follower’s mechanism and utility remain hidden from the principal. So, the function ${\mu }_M(\alpha ,\beta )$ is an unknown function for the principal. In this context, we can see ${\Omega }_F$ as the realized cash inflow from the borrower, i.e., the bank collections follow the mechanism ${\Omega }_L=\gamma \left(M\right)+U_L=\alpha M+\beta +U_L$; the interest or the cash inflow M plus the fee actually received at the single stage, possibly noisy. The principal utility is consider linear given as $Y=b+r{\Omega }_L+qM+U_Y$, where b represents the baseline overheads, r is the realization factor that converts collections into value (often $r\approx 1$ if a dollar collected is a dollar of revenue before costs; r < 1 if haircuts rules apply), and q is the marginal value of the loan size M. It is typically negative if it aggregates funding cost, expected credit loss, and capital operating costs per unit of balance.

$$\begin{array}{cc}\hfill \underset{\mathit{Profit}}{\underbrace{J_L}}& =\underset{\mathit{Collections}}{\underbrace{r{\Omega }_L}}+\underset{\mathit{Funding}\mathit{and}\mathit{risk}\mathit{cost}\mathit{per}\mathit{unit}}{\underbrace{qM}}+\underset{\mathit{Fixed}\mathit{margin}}{\underbrace{b}}\hfill \\ \hfill & =r\left(\alpha M+\beta \right)+qM+b.\hfill \end{array}$$

In short, profit is given as the collections minus (negative q) the funding cost per size. This aligns with the canonical $PA{P}_{L_1{F}_1}$, where the utility of the principal depends on the decisions of the leader ${\Omega }_L$ and the follower ${\Omega }_F$.

Given this outcome model, the conditional mean ${\mu }_L({\omega }_L,m,\gamma )$ used inside the g-formula identification in Equation (29) is

$$\begin{array}{cc}\hfill {\mu }_L({\omega }_L,m,\gamma )& =\mathbb{E}[Y\mid {\Omega }_L={\omega }_L,M=m,\Gamma =\gamma ]\hfill \\ \hfill & =b+r{\omega }_L+qm,\hfill \end{array}$$

since $\mathbb{E}[U_Y\mid {\Omega }_L,M,\Gamma ]=0$. Then, we reduce ${\mathbb{E}}_{U_{{\Omega }_L}}\left[{\mu }_L(\gamma \left(m\right)+U_{{\Omega }_L},m,\gamma )\right]$, the inner expectation over the principal’s action noise $U_{{\Omega }_L}$ in the identification formula, using ${\Omega }_L=\gamma \left(m\right)+U_{{\Omega }_L}$ and $\mathbb{E}[U_{{\Omega }_L}\mid M=m,\Gamma =\gamma ]=0$. If we fix m, the inner term becomes

$$\begin{array}{cc}\hfill {\mathbb{E}}_{U_{{\Omega }_L}}\left[{\mu }_L(\gamma \left(m\right)+U_{{\Omega }_L},m,\gamma )\right]& =\mathbb{E}\left[b+r(\gamma \left(m\right)+U_{{\Omega }_L})+qm\right]\hfill \\ \hfill & =b+r\gamma \left(m\right)+qm.\hfill \end{array}$$

(36)

Because ${\mu }_L$ is linear on ${\omega }_L$ and $U_{{\Omega }_L}$ is mean-zero (conditional on m), the action-noise variance do not affect the mean; only the mean of ${\Omega }_L$ matters at this step. We reduce the outer integral by plugging the reduced inner term (36) into the outer integral in (29), and take the (outer) expectation with respect to the mediator law $p(m\mid \Gamma =\gamma )$:

$$\begin{array}{cc}\hfill V\left(\gamma \right)& =\int \left(b+r\gamma \left(m\right)+qm\right)p(m\mid \Gamma =\gamma )dm\hfill \\ \hfill & =b+r\mathbb{E}\left[\gamma \left(M\right)\mid \Gamma =\gamma \right]+q\mathbb{E}[M\mid \Gamma =\gamma ].\hfill \end{array}$$

Now, substituting the affine policy ${\gamma }_{(\alpha ,\beta )}\left(m\right)=\alpha m+\beta$:

$$\begin{array}{cc}\hfill \mathbb{E}\left[\gamma \left(M\right)\mid \Gamma =(\alpha ,\beta )\right]& =\mathbb{E}[\alpha M+\beta \mid \Gamma =(\alpha ,\beta \left)\right]\hfill \\ \hfill & =\alpha {\mu }_M(\alpha ,\beta )+\beta ,\hfill \end{array}$$

with ${\mu }_M(\alpha ,\beta ):=\mathbb{E}[M\mid \Gamma =(\alpha ,\beta )]$. Hence,

$$\begin{array}{cc}\hfill V(\alpha ,\beta )& =b+r\left(\alpha {\mu }_M(\alpha ,\beta )+\beta \right)+q{\mu }_M(\alpha ,\beta )\hfill \\ \hfill & =b+r\beta +(r\alpha +q){\mu }_M(\alpha ,\beta ).\hfill \end{array}$$

(37)

Therefore, the expression in (37) for $V(\alpha ,\beta )$ is the estimand we want to approximate from the data, given as N i.i.d. observations $\left\{\right({\alpha }_i,{\beta }_i,M_i,{\Omega }_{L,i},Y_i{\left)\right\}}_{i=1}^N$. The formula in (37) depends on (i) the outcome coefficients $(b,r,q)$, and (ii) the black-box mean of the mediator under policy ${\mu }_M(\alpha ,\beta )=\mathbb{E}[M\mid \Gamma =(\alpha ,\beta )]$. We estimate them separately in two steps and subsequently input them into the closed form. In a first step, we estimate $(\widehat{b},\widehat{r},\widehat{q})$ from a regression of Y on $({\Omega }_L,M)$; and in a second step, the estimation of ${\widehat{\mu }}_M(\alpha ,\beta )$ is performed through the regression of M with respect to the variables $(\alpha ,\beta )$. The closed form estimator is then

$$\widehat{V}(\alpha ,\beta )=\widehat{b}+\widehat{r}\beta +(\widehat{r}\alpha +\widehat{q}){\widehat{\mu }}_M(\alpha ,\beta ).$$

(38)

A natural way to estimate $(\widehat{b},\widehat{r},\widehat{q})$ is by weighting observations according to their location in relation to $(\alpha ,\beta )$. Then, a weighted least-squares (WLS) fit yields $(\widehat{b},\widehat{r},\widehat{q})$ that are tuned to the neighborhood of the target policy ${\gamma }_{(\alpha ,\beta )}$. That is, for a target incentive policy ${\gamma }_{(\alpha ,\beta )}\left(m\right)=\alpha m+\beta$, we fit a local linear outcome model $Y\approx b+r{\Omega }_L+qM+\epsilon$, by weighting observations according to their proximity to ${\gamma }_{(\alpha ,\beta )}$ in the policy space.

Let $X_i=[1,{\Omega }_{L,i},M_i]$ and $y_i=Y_i$ for $i\in \left[N\right]$, and arrange the data in the matrix $X\in {\mathbb{R}}^{n\times 3}$, and the vector $y\in {\mathbb{R}}^n$. A practical choice for a distance $d\left(({\alpha }_i,{\beta }_i),(\alpha ,\beta )\right)$ in the policy space is an isotropic distance defined as

$$\begin{array}{cc}\hfill & d\left(({\alpha }_i,{\beta }_i),(\alpha ,\beta )\right)=\sqrt{{\left({\tilde{\alpha }}_i-\tilde{\alpha }\right)}^2+{\left({\tilde{\beta }}_i-\tilde{\beta }\right)}^2},\hfill \end{array}$$

with standardized coordinates $\tilde{\alpha }=(\alpha -\overline{\alpha })/\widehat{\mathrm{sd}}\left(\alpha \right)$ (and analogously for $\beta$).

Let $K(·)$ be a radial kernel (Gaussian or Epanechnikov) and $h>0$ a scalar bandwidth. We then define the unnormalized weights as follows:

$${\tilde{w}}_i\left(h\right)=K\left({\displaystyle \frac{d(({\alpha }_i,{\beta }_i),(\alpha ,\beta ))}{h}}\right),w_i\left(h\right)={\displaystyle \frac{{\tilde{w}}_i\left(h\right)}{{\sum }_{j=1}^n{\tilde{w}}_j\left(h\right)}},\sum _i{w}_i\left(h\right)=1,$$

and $W=\mathrm{diag}\left(w_1\left(h\right),\dots ,w_n\left(h\right)\right)$. Then, the WLS estimator $\widehat{\theta }={(\widehat{b},\widehat{r},\widehat{q})}^{\top }$ is

$$\widehat{\theta }=\left[\begin{array}{c}\widehat{b}\\ \widehat{r}\\ \widehat{q}\end{array}\right]={\left(X^{\top }WX\right)}^{-1}{X}^{\top }Wy.$$

We estimate ${\mu }_M(\alpha ,\beta )=\mathbb{E}[M\mid \Gamma =(\alpha ,\beta )]$ for a fixed policy $(\alpha ,\beta )$, regressing the realized mediator M on the policy coordinates $(\alpha ,\beta )$, using observed tuples ${\left\{({\alpha }_i,{\beta }_i,M_i)\right\}}_{i=1}^N$. We proceed similarly in this regression with the idea of weighting observations according to their proximity to ${\gamma }_{(\alpha ,\beta )}$. Let ${\theta }_i=({\alpha }_i,{\beta }_i)$ and $\theta =(\alpha ,\beta )$ denote observed and target incentive policies. We use the same policy-space isotropic distance and unnormalized weights as in the previous regression.

We can approximate ${\mu }_M$ by a local plane around $\theta =(\alpha ,\beta )$, given as the linear combination ${\theta }_0+{\theta }_1({\alpha }_i-\alpha )+{\theta }_2({\beta }_i-\beta )$ of the variables $({\alpha }_i-\alpha )$ and $({\beta }_i-\beta )$. Thus, using the approximation $M_i\approx {\theta }_0+{\theta }_1({\alpha }_i-\alpha )+{\theta }_2({\beta }_i-\beta )$, the WLS problem is given by

$$\underset{{\theta }_0,{\theta }_1,{\theta }_2}{min}\sum _{i=1}^N{w}_i\left(h\right){\left[M_i-\left\{{\theta }_0+{\theta }_1({\alpha }_i-\alpha )+{\theta }_2({\beta }_i-\beta )\right\}\right]}^2,$$

(39)

with the same kernel weights $w_i\left(h\right)$. Then, the estimate of the mean at the target is the intercept in this linear model. Therefore, $\widehat{{\theta }_0}$ represents the estimated mean value of the plane at the specific target policy $(\alpha ,\beta )$ and ${\widehat{\mu }}_M(\alpha ,\beta )=\widehat{{\theta }_0}$. In matrix form, let the local design row be $x_i=[1,{\alpha }_i-\alpha ,{\beta }_i-\beta ]$, arranging in the matrix $X\in {\mathbb{R}}^{n\times 3}$ the $x_i$, and $W=\mathrm{diag}\left(w_1\left(h\right),\dots ,w_n\left(h\right)\right)$. A local linear WLS estimator for ${\mu }_M(\alpha ,\beta )=\mathbb{E}[M\mid \Gamma =(\alpha ,\beta )]$ is

$$\widehat{\mathit{\theta }}={\left(X^{\top }WX\right)}^{-1}{X}^{\top }W\mathit{M},{\widehat{\mu }}_M(\alpha ,\beta )={\widehat{\theta }}_0,\mathrm{with}\mathit{M}={(M_1,\dots ,M_n)}^{\top }$$

3.3.2. The Estimation in the General Gaussian Additive-Noise Case

We model the conditional distribution of the leader’s realized action ${\Omega }_L$ given the mediator M and policy $\gamma$ as the following Gaussian:

$$p\left({\Omega }_L\mid M=m,\Gamma =\gamma \right)\sim \mathcal{N}\left({\widehat{f}}_L(m,\gamma ),{\sigma }_L^2(m,\gamma )\right)$$

(40)

We learn non-parametrically the mean map ${\widehat{f}}_L$: $M\times \mathcal{A}\to \mathbb{R}$ and the variance map ${\sigma }_L^2$: $M\times \mathcal{A}\to (0,\infty )$ from data ${\left\{(M_i,A_i,{\Omega }_{L,i})\right\}}_{i=1}^N$ using RKHS regression kernel, i.e., Kernel Ridge Regression (KRR) on $(M,\Gamma )$ [4,27]. Therefore, to learn the mean function ${\widehat{f}}_L$, we solve the KRR problem, where the goal is to find a function f that minimizes a regularized loss function.

$${\widehat{f}}_L(m,\gamma )\in \arg \underset{f\in {\mathcal{H}}_{MA}}{min}{\left({\omega }_{Li}-f(m_i,{\gamma }_i)\right)}^2+{\lambda }_f{\parallel f\parallel }_{{\mathcal{H}}_{MA}}^2.$$

(41)

The first term measures how well the function fits the training data. The second term is a regularization penalty, which penalizes the complexity of the function as measured by its norm in the RKHS, using the regularization hyperparameter ${\lambda }_f$ that balances the trade-off. By the representer theorem, the solution of problem (41) can be expressed as a linear combination of kernel functions centered at the training data points $f(·)={\sum }_{i\in \left[N\right]}{\alpha }_{fi}{k}_{MA}(x_i,·)$. In matrix form, ${\widehat{f}}_L(m,a)=k_{(m,a)}^{\top }{\mathit{\alpha }}_f$, where $k_{(m,a)}={\left(k_{MA}(x_1,(m,a)),\dots ,k_{MA}(x_n,(m,a))\right)}^{\top }$; and ${\mathit{\alpha }}_f={\left(K+{\lambda }_{f}I\right)}^{-1}\mathit{y}$, where $K\in {\mathbb{R}}^{N\times N}$ with $K_{ij}=k_{MA}(x_i,x_j)$, $\mathit{y}={(y_1,\dots ,y_N)}^{\top }$.

A product kernel on $(m,a)$ can capture interactions as in our case, where the effect of m on ${\Omega }_L$ depends on policy $\gamma$. So, we can establish a product kernel on $(m,a)$ as follows:

$$k_{MA}^{\mathrm{prod}}\left((m,a),(m^{\prime },a^{\prime })\right)=\underset{k_M(m,m^{\prime })}{\underbrace{exp\left(-{\displaystyle \frac{{(m-m^{\prime })}^2}{2{\mathcal{l}}_M^2}}\right)}}\times \underset{k_{\Gamma }(a,a^{\prime })}{\underbrace{exp\left(-{\displaystyle \frac{\parallel a-a^{\prime }{\parallel }^2}{2{\mathcal{l}}_{\Gamma }^2}}\right)}}.$$

Likewise, for the variance function ${\sigma }_L^2(m,\gamma )$, we compute the residuals $r_i={\omega }_{Li}-{\widehat{f}}_L(m_i,{\gamma }_i)$; and we can define pseudo-responses $z_i=\log (r_i^2+ϵ)$ with small $ϵ>0$, for numerical stability. Modeling $\log {\sigma }_L^2(m,\gamma )=v(m,\gamma )$ in an RKHS, guarantees strict positivity after exponentiation, turns multiplicative scale effects into additive structure, and improves residual behavior under Gaussian assumptions. Therefore, we fit KRR on $(m,\gamma )\to z$ and solve

$${\widehat{v}}_L(m,\gamma )\in \arg \underset{v\in {\mathcal{H}}_{MA}}{min}{\left(z_i-v(m_i,{\gamma }_i)\right)}^2+{\lambda }_v{\parallel v\parallel }_{{\mathcal{H}}_{m\gamma }}^2.$$

(42)

As before, $v(·)={\sum }_{i\in \left[N\right]}{\alpha }_{vi}{k}_{MA}(x_j,·)$ with ${\mathit{\alpha }}_v={(K+{\lambda }_{v}I)}^{-1}\mathit{z}$ and $\mathit{z}={(z_1,\dots ,z_n)}^{\top }$. We predict $\widehat{v}(m,a)$ and set ${\widehat{\sigma }}_L^2(m,a)=exp\left(\widehat{v}(m,a)\right)>0$.

Using the above, we can now estimate ${\mu }_L(\omega ,m,a)=\mathbb{E}[Y\mid {\Omega }_L=\omega , M=m, \Gamma =a]$. We can use a weighted KRR formulation to compute ${\widehat{\mu }}_L(\omega ,m,a)$ with observations $x_i=({\omega }_i,m_i,a_i)$, and output $y_i=Y_i$, $i=1,\dots ,n$. Let $k_X$ be a positive-definite kernel on $x=(\omega ,m,a)$ with RKHS ${\mathcal{H}}_X$. Then, we solve the following weighted regularized least-squares problem:

$${\widehat{\mu }}_L\left(x\right)\in \arg \underset{\mu \in {\mathcal{H}}_X}{min}\sum _{i=1}^n{w}_i\left(\gamma \right){\left(y_i-\mu \left(x_i\right)\right)}^2+{\lambda }_{\mu }{\parallel \mu \parallel }_{{\mathcal{H}}_X}^2,$$

(43)

employing $\mu (·)={\sum }_{i\in \left[N\right]}{\alpha }_{\mu i}{k}_X(x_i,·)$, ${\mathit{\alpha }}_{\mu }={\left(KW+\lambda I_n\right)}^{-1}W\mathit{y}$; thus, ${\widehat{\mu }}_L\left(x\right)=k_x^{\top }\mathit{\alpha }$, where $K_{ij}=k_X(x_i,x_j)$, $K\in {\mathbb{R}}^{n\times n}$, $W=\mathrm{diag}\left(w_1\left(\gamma \right),\dots ,w_n\left(\gamma \right)\right)$, $\mathit{y}={(y_1,\dots ,y_n)}^{\top }$ and $k_x={[k_X(x_1,x),\dots ,k_X(x_n,x)]}^{\top }$. The policy weights $w_i\left(\gamma \right)\ge 0$ with ${\sum }_i{w}_i\left(\gamma \right)=1$, prioritize fidelity near $\gamma$. More details about these weights are given in Section 3.3.4.

3.3.3. Computing the Inner Gaussian Expectation

So far, we have estimated what appears within the curly brackets before integration, in the identification Formula (29), shown again below for clarity:

$$\begin{array}{cc}\hfill V\left(\gamma \right)& =\int \left\{\int {\mu }_L({\omega }_L,m,\gamma )p({\omega }_L\mid m,\Gamma =\gamma )d{\omega }_L\right\}p(m\mid \Gamma =\gamma )dm\hfill \\ \hfill & =\int \left\{{\mathbb{E}}_{U_{{\Omega }_L}}\left[{\mu }_L(\gamma \left(m\right)+U_{{\Omega }_L},m,\gamma )\right]\right\}p(m\mid \Gamma =\gamma )dm,\hfill \end{array}$$

where ${\mu }_L({\omega }_L,m,\gamma ):= \mathbb{E}[Y\mid {\Omega }_L={\omega }_L,M=m,\Gamma =\gamma ]$. That is, we show before how to estimate ${\mu }_L({\omega }_L,m,\gamma )$ and $p({\omega }_L\mid m,\Gamma =\gamma )$ and now we show how to approximate

$$\begin{array}{cc}\hfill \int {\mu }_L({\omega }_L,m,\gamma )p({\omega }_L\mid m,\Gamma =\gamma )d{\omega }_L& ={\mathbb{E}}_{U_{{\Omega }_L}}\left[{\mu }_L(\gamma \left(m\right)+U_{{\Omega }_L},m,\gamma )\right]\hfill \\ \hfill & ={\mathbb{E}}_{U_{{\Omega }_L}}\left[{\mu }_L\left({\Omega }_L,m,\gamma \right)|M=m,\Gamma =\gamma \right]\hfill \\ \hfill & =g_{U_{{\Omega }_L}}(m;\gamma ).\hfill \end{array}$$

An efficient and natural way to compute the inner Gaussian expectation ${\widehat{g}}_{U_{{\Omega }_L}}(m;\gamma )$ is by the Gauss–Hermite (GH) quadrature method. The integral for $g_{U_{{\Omega }_L}}(m;\gamma )$ can be expressed as

$$g_{U_{{\Omega }_L}}(m;\gamma )={\int }_{-\infty }^{\infty }{\mu }_L({\omega }_L,m,\gamma ){\displaystyle \frac{1}{\sqrt{2\pi }{\sigma }_{{\Omega }_L}}}exp\left(-{\displaystyle \frac{{({\omega }_L-{\mu }_{{\Omega }_L})}^2}{2{\sigma }_{{\Omega }_L}^2}}\right)d{\omega }_L,$$

where ${\mu }_{{\Omega }_L}:={\widehat{f}}_L(\gamma ,m)$, ${\sigma }_{{\Omega }_L}:={\sigma }_L(\gamma ,m)>0$, which we previously estimate and $p\left({\Omega }_L\mid (M=m,\Gamma =\gamma )\right)\sim \mathcal{N}\left({\mu }_{{\Omega }_L},{\sigma }_{{\Omega }_L}^2\right)$. Writing ${\Omega }_L={\mu }_{{\Omega }_L}+U_L$ with $U_L\sim \mathcal{N}(0,{\sigma }_{{\Omega }_L}^2)$, the change of variables $U_L={\sigma }_{{\Omega }_L}Z$, i.e., $Z=U_L/{\sigma }_{{\Omega }_L}$, transforms the expectation to the standard-normal scale. No information is lost; it is the same integral expressed in a convenient form for the GH method. Therefore, with a standard normal $Z\sim \mathcal{N}(0,1)$, equivalently, we have

$$g_Z(m;\gamma )={\mathbb{E}}_Z\left[{\mu }_L\left({\mu }_{{\Omega }_L}+{\sigma }_{{\Omega }_L}Z,m,\gamma \right)\right]={\int }_{-\infty }^{\infty }{\mu }_L\left({\mu }_{{\Omega }_L}+{\sigma }_{{\Omega }_L}z,m,\gamma \right){\displaystyle \frac{e^{-z^2/2}}{\sqrt{2\pi }}}dz.$$

(44)

The GH method approximates integrals of the form ${\int }_{-\infty }^{\infty }{e}^{-x^2}f\left(x\right)dx\approx {\sum }_{m=1}^M{w}_m^{GH}f\left(x_m^{GH}\right)$, where ${\{x_m^{GH},w_m^{GH}\}}_{m=1}^M$ are Hermite nodes and weights. To bring the standard-normal expectation (44) to this form, set $z=\sqrt{2}x$ ($dz=\sqrt{2}dx$), yielding

$$\begin{array}{}\mathrm{(45)}\hfill g_Z(m;\gamma )& ={\displaystyle \frac{1}{\sqrt{\pi }}}{\int }_{-\infty }^{\infty }\underset{f\left(x^{GH}\right)}{\underbrace{{\mu }_L\left({\mu }_{{\Omega }_L}+{\sigma }_{{\Omega }_L}\sqrt{2}x,m,\gamma \right)}}\underset{\mathrm{GH}\mathrm{weights}{w}^{GH}}{\underbrace{e^{-x^2}}}dx.\hfill \mathrm{(46)}\hfill & \approx {\displaystyle \frac{1}{\sqrt{\pi }}}\sum _{m=1}^M{w}_m^{GH}{\mu }_L\left({\mu }_{\Omega }+{\sigma }_{\Omega }\sqrt{2}{x}_j,m,\gamma \right),\hfill \end{array}$$

where the computable approximation in (46) for the definite integral in (45) is given by applying a M-point rule GH. The sum of all the weights for the GH rule is always ${\sum }_{m=1}^M{w}_m^{GH}={\int }_{-\infty }^{\infty }{e}^{-x^2}dx=\sqrt{\pi }$ and the Golub–Welsch Algorithm is a standard numerical method for computing the nodes and weights of any Gaussian quadrature rule, including the Gauss–Hermite quadrature. Unlike Monte Carlo, GH has no sampling variance; this stability is valuable when comparing many candidate policies $\gamma$. The inner integral is always one-dimensional (action noise), so GH is computationally light, and the nodes ${\{x_m^{GH},w_m^{GH}\}}_{m=1}^M$ are pre-computable and reusable.

3.3.4. Policy-Local Empirical Measure Construction in the Outer Integral

Finally, the identified value of a fixed policy $\gamma$ can be written as a conditional expectation $\mathbb{E}\left[g_{\gamma }\left(M\right)\mid \Gamma =\gamma \right]$, since:

$$\begin{array}{cc}\hfill V\left(\gamma \right)& =\int \left\{{\mathbb{E}}_{U_{{\Omega }_L}}\left[{\mu }_L(\gamma \left(m\right)+U_{{\Omega }_L},m,\gamma )\right]\right\}p(m\mid \Gamma =\gamma )dm,\hfill \\ \hfill & =\int g_{U_{{\Omega }_L}}(m;\gamma )p(m\mid \Gamma =\gamma )dm,\hfill \\ \hfill & =\mathbb{E}\left[g_{\gamma }\left(M\right)\mid \Gamma =\gamma \right]\hfill \end{array}$$

So, the task is to estimate the expectation over the mediator distribution under policy $\gamma$. We estimate this by a local kernel smoother in policy space. We observe i.i.d. ${\left\{(m_i,{\gamma }_i,\dots )\right\}}_{i=1}^N$ with realized policies ${\gamma }_i$ (e.g., ${\gamma }_i=({\alpha }_i,{\beta }_i)$). Because $\Gamma$ is continuous, the event $\{\Gamma =\gamma \}$ has probability zero; therefore, we approximate the conditional expectation by localizing to policies ${\gamma }_i$ that are close to $\gamma$ and form a policy-local empirical distribution over the observed mediators $\left\{M_i\right\}$. For that policy-local empirical distribution, we may define a distance $d({\gamma }_i,\gamma )$ in the policy space. Since ${\gamma }_i,\gamma$ lie in an RKHS $H_k$, we use $d({\gamma }_i,\gamma )={\parallel {\gamma }_i-\gamma \parallel }_{H_k}$ to reflect the policy geometry. Additionally, we select a non-negative, radial kernel $K\left(u\right)\ge 0$ with $u=d/h$ and a bandwidth $h>0$ as the Gaussian kernel, also known as the Radial Basis Function (RBF) kernel $K\left(u\right)=exp(-u^2/2)$. So, as in the linear parametrized instance before, we define the unnormalized weights:

$${\tilde{w}}_i(\gamma ;h)=K\left({\displaystyle \frac{d({\gamma }_i,\gamma )}{h}}\right),w_i(\gamma ;h)={\displaystyle \frac{{\tilde{w}}_i(\gamma ;h)}{{\sum }_{j=1}^n{\tilde{w}}_j(\gamma ;h)}}\ge 0,\mathrm{with}\sum _{i\in \left[N\right]}{w}_i=1,$$

(47)

Thus, we approximate $\mathbb{E}\left[g_{\gamma }\left(M\right)\mid \Gamma =\gamma \right]$ using a policy-local empirical measure supported on the observed mediators for the conditional $p(m\mid \Gamma =\gamma )$. The policy-local empirical distribution for approximating $p(m\mid \Gamma =\gamma )$ is defined as

$${\widehat{P}}_{\gamma }^{\left(h\right)}\left(dm\right)=\sum _{i=1}^n{w}_i(\gamma ;h){\delta }_{m_i}\left(dm\right),$$

(48)

where ${\delta }_{M{m}_i}$ is the unit mass at $m_i$. Intuitively, ${\widehat{P}}_{\gamma }^{\left(h\right)}$ is the law of M one would obtain by resampling historical mediators with probability proportional to how close their policies ${\gamma }_i$ are to the target $\gamma$. This approximation for $p(m\mid \Gamma =\gamma )$ turns the outer integral into an interpretable, positivity–respecting kernel average. Integrating the test function ${\widehat{g}}_{U_{{\Omega }_L}}(·;\gamma )$ against this measure yields

$$\begin{array}{}\mathrm{(49)}\hfill \widehat{V}\left(\gamma \right)& =\widehat{\mathbb{E}}[g_{\gamma }\left(M\right)\mid \Gamma =\gamma ]\hfill \mathrm{(50)}\hfill & =\int {\widehat{g}}_{U_{{\Omega }_L}}(m;\gamma ){\widehat{P}}_{\gamma }^{\left(h\right)}\left(dm\right)\hfill \mathrm{(51)}\hfill & =\sum _{i=1}^n{w}_i(\gamma ;h)\widehat{g}(M_i;\gamma ),\hfill \end{array}$$

which is precisely the outer half of the nested g-computation estimator $\widehat{V}\left(\gamma \right)$. The policy-local reweighting in (49)–(51) implicitly requires a positivity (overlap) condition: for any target policy $\gamma$, there must be sufficient probability mass of logged policies in a neighborhood of $\gamma$ so that the conditional law $p(M\mid \Gamma =\gamma )$ can be well approximated by the empirical measure ${\widehat{\mathbb{P}}}_{\gamma }^{\left(h\right)}$. In finite samples, this induces an empirical support constraint: $\widehat{V}\left(\gamma \right)$, which is reliable only if the weights $w_i(\gamma ;h)$ do not degenerate. We quantify local support via the effective sample size (ESS):

$$\mathrm{ESS}(\gamma ;h):={\displaystyle \frac{{\left({\sum }_{i=1}^N{w}_i(\gamma ;h)\right)}^2}{{\sum }_{i=1}^N{w}_i{(\gamma ;h)}^2}}.$$

The ESS is a measure of how many independent samples the weighted sample set is equivalent to. A low ESS indicates high variance and instability in the estimate $\widehat{V}\left(\gamma \right)$ because the estimate is dominated by a small number of data points. We also define the support set:

$${\mathcal{S}}_{\mathrm{support}}(h,\tau ,w_{\max }):=\left\{\gamma \in \Gamma :\mathrm{ESS}(\gamma ;h)\ge \tau ,\underset{i}{\max }{w}_i(\gamma ;h)\le w_{\max }\right\}.$$

The support set ${\mathcal{S}}_{\mathrm{support}}$ defines the region in the policy space $\Gamma$ where the estimate $\widehat{V}\left(\gamma \right)$ is considered empirically reliable. Policies $\gamma$ outside this set are not trustworthy due to a lack of data overlap. The $\mathrm{ESS}(\gamma ;h)\ge \tau$ condition ensures that the evaluation is based on a sufficiently large effective sample size, preventing the estimate from being dominated by just a few data points. By requiring $\mathrm{ESS}(\gamma ;h)\ge \tau$, we enforce a degree of overlap between the target policy $\gamma$ and the logged policies. Higher $\tau$ means we demand better overlap and a more stable estimate, which shrinks the support set ${\mathcal{S}}_{\mathrm{support}}$. The ${\max }_i{w}_i(\gamma ;h)\le w_{\max }$ condition directly guards against extreme importance weights, which are a hallmark of poor overlap or model mismatch in off-policy evaluation. This constraint helps bound the influence of rare, yet highly weighted, events. Policies $\gamma$ that require assigning an enormous weight to any single data point are excluded from the support set. Lower $w_{\max }$ means we tolerate less weight variance and demand better weight balance, which also shrinks the support set ${\mathcal{S}}_{\mathrm{support}}$. The effective sample size (ESS) serves as a proxy for uncertainty in the outer expectation. A small ESS suggests high estimator variance and low information, which necessitates a trust-region restriction in policy space. Conversely, a large ESS indicates data-supported proposals and reduced uncertainty.

4. A Functional Bayesian Optimization Algorithm for Single-Stage Canonical PAPs

Functional Bayesian Optimization (FBO) is the version of Bayesian Optimization (BO) (see Appendix C) in which the domain $\mathbf{D}$ of the objective function to maximize $f_{obj}$: $\mathbf{D}\to \mathbb{R}$ is a space of functions. We leverage the sequential optimization process of Bayesian optimization and the use of the Gaussian Process Upper Confidence Bound (GP-UCB) acquisition function (see Appendix C.2) to solve the single-stage canonical $PA{P}_{L_1{F}_1}$. Specifically, we propose an FBO algorithm to sequentially solve the following optimization problem.

$${\gamma }^{*}\in \underset{{\gamma }^t\in \Gamma }{\arg \max }\mathbb{E}\left[J_L\mid do\left(\Gamma ={\gamma }^t\right)\right].$$

(52)

This is an objective functional, since its domain is a space of incentive functions $\Gamma$. We employ a GP-UCB acquisition functional (see Appendix C.2 and Section 4.2) in the proposed FBO algorithm as the strategy to guide the selection of the best next incentive function evaluation ${\gamma }^t$ in the sequential optimization of Equation (52). Additionally, we use a GP-UCB acquisition functional to establish upper bounds on the cumulative regret $R^T$ for sequentially optimizing the objective functional in Equation (52) with horizon T when the FBO algorithm described in the following subsections is employed.

In order to describe the proposed FBO algorithm to solve Equation (52), we begin by establishing a functional Gaussian process as a surrogate function model for the objective functional in Section 4.1. For this, we set up a functional space for the incentive functions space $\Gamma$, which allows the definition of a Gaussian process kernel over this space of functions in Section 4.1.1, and in light of this, we show how to compute the posterior distribution for this functional Gaussian process in Section 4.1.2. Later, the GP-UCB acquisition functional for functional search is described in Section 4.2. Having these elements available, we present, in Section 4.3, Algorithm 1 to solve Equation (52). This algorithm is referred to as the Stackelberg Functional Causal Bayesian Optimization (FCBO) algorithm for $PA{P}_{L1F1}$. In Section 4.4, we show cumulative-regret bounds results in terms of differential information gain for the Stackelberg FCBO algorithm.

Algorithm 1: Stackelberg FCBO for single-stage $PA{P}_{L1F1}$
	Input    : ${\mathcal{D}}^{-1}=\left\{\right({\gamma }^i,{\omega }_F^i,{\omega }_L^i,J_L^i{\left)\right\}}_{i=1}^N$; the ${\mathcal{H}}_k$ specification; the horizon T;                 the functional GP kernel K on $\Gamma \subset {\mathcal{H}}_k$; the exploration schedule ${\left\{{\beta }_t\right\}}_{t=1}^T$;                 and the CGM ${\mathcal{M}}_{L1F1}$ for a $PA{P}_{L1F1}$ from the principal’s perspective.
	Output : ${\mathcal{D}}^T={\left\{({\gamma }^t,{\mathbb{E}}^{{\gamma }^t}\left[J_L^t\right])\right\}}_{t\in \left[T\right]}$, with ${\gamma }^t\in {\mathcal{H}}_k^d$ or ${\gamma }^t\in {\mathcal{H}}_k$
1	.
2	Initialize functional GP $\mathcal{GP}\left({\mu }^0\left(\gamma \right),K^0(\gamma ,{\gamma }^{\prime })\right)$;
3	for $t=1,\dots ,T$ do
4		Select ${\gamma }^t\in \underset{\gamma \in {\mathcal{H}}_k}{\arg \max }{\mu }^t\left(\gamma \right)+{\beta }^t{\sigma }^t\left(\gamma \right)$ (see Section 4.2);
5		Estimate ${\mathbb{E}}^{{\gamma }^t}\left[J_L\right]$ as $\widehat{V}\left({\gamma }^t\right)=\widehat{\mathbb{E}}\left[J_L\mid do\left(\Gamma ={\gamma }^t\right)\right]$ (see Section 3.3);
6		Set ${\mathcal{D}}^t\leftarrow {\mathcal{D}}^{t-1}\cup \left\{({\gamma }^t,{\mathbb{E}}^{{\gamma }^t}\left[J_L^t\right])\right\}$;
7		Update functional GP $\mathcal{GP}\left({\mu }^t\left(\gamma \right),K^t(\gamma ,{\gamma }^{\prime });{\mathcal{D}}^t\right)$ (see Equation (55));
8	end
9	return ${\mathcal{D}}^T={\left\{({\gamma }^t,{\mathbb{E}}^{{\gamma }^t}\left[J_L^t\right])\right\}}_{t\in \left[T\right]}$

4.1. Functional Gaussian Process Surrogate Model

We employ a Reproducing Kernel Hilbert Space (RKHS) as a space of functions for the domain of the objective functional ${\mathbb{E}}^{{\gamma }^t}\left[J_L\right]$: $\Gamma \to \mathbb{R}$ in Equation (52). Taking $\Gamma$ as ${\mathcal{H}}_k$, that is, ${\mathbb{E}}^{{\gamma }^t}\left[J_L\right]$: ${\mathcal{H}}_k\to \mathbb{R}$, where ${\mathcal{H}}_k$ is an RKHS with reproducing kernel k: $\mathbf{X}\times \mathbf{X}\to \mathbb{R}$ and $\mathbf{X}\subseteq \mathbb{R}$ (see Appendix B), allows us to directly define a Gaussian process kernel over functions in ${\mathcal{H}}_k$ (see Section 4.1.1). In each round t, an incentive function ${\gamma }^t\in {\mathcal{H}}_k$ is selected by means of a GP-UCB acquisition functional (see Section 4.2) and a noisy evaluation ${\mathbb{E}}^{{\gamma }^t}\left[J_L\right]$ is returned.

We use a Gaussian process (GP) as a surrogate model for the objective functional ${\mathbb{E}}^{{\gamma }^t}\left[J_L\right]$: ${\mathcal{H}}_k\to \mathbb{R}$. We assume specifically that the input function space is an RKHS ${\mathcal{H}}_k$, which allows us to define a GP kernel over functions in a RKHS function space in Section 4.1.1. Therefore, the GP we use for this objective functional with domain ${\mathcal{H}}_k$ is given as a stochastic process $F_{\gamma }=\{f\left(\gamma \right)\mid \gamma \in {\mathcal{H}}_k\}$, in which every finite collection of random variables $f\left(\gamma \right)$ has a multivariate normal distribution. So, we use this GP $F_{\gamma }=\{f\left(\gamma \right)\mid \gamma \in {\mathcal{H}}_k\}$, with $F_{\gamma }\sim \mathcal{GP}(\mu \left(\gamma \right),K(\gamma ,{\gamma }^{\prime }))$ as a surrogate model for ${\mathbb{E}}^{{\gamma }^t}\left[J_L\right]$. Next, based on [6], we formulate a GP kernel $K(\gamma ,{\gamma }^{\prime })=cov[f\left(\gamma \right),f\left({\gamma }^{\prime }\right)]$ with $\gamma ,{\gamma }^{\prime }\in {\mathcal{H}}_k$. See [4] for GP fundamentals and [6,25] for functional BO surrogates.

4.1.1. Functional Gaussian Process Kernel

Based on the RKHS ${\mathcal{H}}_k$ kernel k, the GP kernel $K(\gamma ,{\gamma }^{\prime })=cov[f\left(\gamma \right),f\left({\gamma }^{\prime }\right)]$ can be constructed. We focus here on the construction of a functional version of the standard Radial Basis Function (RBF) kernel, which we assume in the regret analysis of this functional Bayesian framework.

The functional RBF can be stated as in Equation (53), where ${\parallel ·\parallel }_{{\mathcal{H}}_k}$ is the norm in the RKHS ${\mathcal{H}}_k$, so ${\parallel \gamma ,{\gamma }^{\prime }\parallel }_{{\mathcal{H}}_k}^2$ is the squared distance between the functions $\gamma ,{\gamma }^{\prime }\in {\mathcal{H}}_k$, which can be calculated as the internal product ${\langle \gamma -{\gamma }^{\prime },\gamma -{\gamma }^{\prime }\rangle }_{{\mathcal{H}}_k}$ in the RKHS ${\mathcal{H}}_k$.

$$K(\gamma ,{\gamma }^{\prime })=exp\left(-{\displaystyle \frac{{\parallel \gamma ,{\gamma }^{\prime }\parallel }_{{\mathcal{H}}_k}^2}{2{\sigma }^2}}\right).$$

(53)

Thus, the squared distance ${\parallel \gamma ,{\gamma }^{\prime }\parallel }_{{\mathcal{H}}_k}^2$ in a functional RBF between two incentive functions $\gamma$ and ${\gamma }^{\prime }$ is given by Equation (54), with ${\upsilon }_i,{\upsilon }_j,{\xi }_i,{\xi }_j\in \mathbb{R}$, $n,m\in \mathbb{N}$, $x_i,x_j,x_i^{\prime },x_j^{\prime }\in \mathbf{X}$.

$${\langle \gamma -{\gamma }^{\prime },\gamma -{\gamma }^{\prime }\rangle }_{{\mathcal{H}}_k}=\sum _{i\in \left[n\right]}\sum _{j\in \left[n\right]}{\upsilon }_i{\upsilon }_{j}k(x_i,x_j)-2\sum _{i\in \left[n\right]}\sum _{j\in \left[m\right]}{\upsilon }_i{\xi }_{j}k(x_i,x_j^{\prime })+\sum _{i\in \left[m\right]}\sum _{j\in \left[m\right]}{\xi }_i{\xi }_{j}k(x_i^{\prime },x_j^{\prime }).$$

(54)

We reserve lower-case $k(x,x^{\prime })$ for the reproducing kernel of the input RKHS ${\mathcal{H}}_k$ over X, and upper-case $K(\gamma ,{\gamma }^{\prime })$ for the Gaussian-process kernel over incentive functions $\Gamma ={\mathcal{H}}_k$. Thus, k appears only inside ${\parallel ·\parallel }_{{\mathcal{H}}_k}$ and expansions like Equation (54), while K and its posterior $K^t$ govern GP means, covariances, and Gram matrices.

4.1.2. Functional Gaussian Process Posterior Distribution

After building the functional RBF kernel primarily by finding out how to measure the distance between the elements $\gamma$ of the function space ${\mathcal{H}}_k$, the posterior distribution of the functional GP $F_{\gamma }\sim \mathcal{GP}(\mu (·),K(·,·))$ may be updated in a way very similar to a scalar GP. The posterior distribution is again a GP $F_{\gamma }\sim \mathcal{GP}(\mu (·),K(·,·))$, with functional mean ${\mu }^t(·)$ and covariance kernel $K^t(·,·)$ given as follows:

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill {\mu }^t\left(\gamma \right)=& {\mathbf{k}}^t{\left(\gamma \right)}^{\top }{(G^t+{\sigma }^{2}I)}^{-1}{\mathbf{y}}^t,\hfill \\ \hfill K^t(\gamma ,{\gamma }^{\prime })=& K(\gamma ,{\gamma }^{\prime })-{\mathbf{k}}^t{\left(\gamma \right)}^{\top }{(G^t+{\sigma }^{2}I)}^{-1}{\mathbf{k}}^t\left({\gamma }^{\prime }\right),\hfill \end{array}\end{array}$$

(55)

where ${\mathbf{k}}^t\left(\gamma \right)={[K(\gamma ,{\gamma }^1),K(\gamma ,{\gamma }^2),\dots ,K(\gamma ,{\gamma }^t)]}^{\top }$, $G^t$ is the $t\times t$ Gram matrix with entries $G_t[i,j]=K({\gamma }_i,{\gamma }_j)$, for $i,j\in \left[t\right]$ (see Appendix B), and ${\mathbf{y}}^t={[y^1,\dots ,y^t]}^{\top }$ is the vector of estimations $y^1={\mathbb{E}}^{{\gamma }^1}\left[J_L\right],\dots ,y^t={\mathbb{E}}^{{\gamma }^t}\left[J_L\right]$.

4.2. Upper Confidence Bound Acquisition Functional

The functional version of the GP-UCB acquisition function (see Appendix C.2, Equation (A20)) is given in Equation (56):

$${\alpha }_{\text{fGP-UCB}}(\gamma ;\mathcal{D},{\beta }^t)={\mu }^t\left(\gamma \right)+{\beta }^t{\sigma }^t\left(\gamma \right),$$

(56)

where ${\sigma }^t\left(\gamma \right)=\sqrt{K^t(\gamma ,\gamma )}$ (see Equation (55)). This acquisition functional can be interpreted as the strategy to choose an incentive function as follows:

$${\gamma }^{t+1}=\underset{\gamma \in {\mathcal{H}}_k}{\arg \max }{\mu }^t\left(\gamma \right)+{\beta }^t{\sigma }^t\left(\gamma \right),$$

(57)

which if we remove the second term ${\beta }^t{\sigma }^t\left(\gamma \right)$ and optimize just for ${\mu }^t\left(\gamma \right)$, i.e., maximizing the expected reward based on the posterior distribution so far, this rule would be too greedy too soon and would lead to getting stuck in shallow local optima. Instead, the full objective ${\mu }^t\left(\gamma \right)+{\beta }^t{\sigma }^t\left(\gamma \right)$ in Equation (57) prefers both incentive functions $\gamma$ where $f\left(\gamma \right)$ is uncertain, that is, with large ${\sigma }^t\left(\gamma \right)$, and where we expect to achieve high rewards, i.e., with large ${\mu }^t\left(\gamma \right)$. Thus, it implicitly deals with the trade-off between exploration and exploitation. An interpretation of this intervention rule is that it greedily selects the intervention functions $\gamma$ such that $f\left(\gamma \right)$ should be a reasonable upper bound on $f\left({\gamma }^{*}\right)$, where ${\gamma }^{*}$ is the optimal incentive function (see Appendix C.2; this follows the GP-UCB principle [7]). The UCB term ${\beta }_t^{1/2}{\sigma }^{t-1}\left(\gamma \right)$ promotes policies with high expected information gain about the functional value. As shown in the regret analysis in Section 4.4, this connects the functional acquisition directly to mutual-information control. Our analysis offers theoretical guidance for acquisition, specifically through trust-region methods using ESS and weight caps. Empirical tuning and benchmarking are intentionally deferred to a companion, application-driven study.

4.3. The Stackelberg FCBO Algorithm for Single-Stage Canonical PAPs

We present a Functional Causal Bayesian Optimization (FCBO) algorithm for the single-stage $PA{P}_{L1F1}$ by integrating all components of Section 4.1 and Section 4.2. We refer to this algorithm, shown in Algorithm 1, as the Stackelberg FCBO algorithm for the single-stage $PA{P}_{L1F1}$. Recall that Algorithm 1 addresses the optimization problem of Equation (52).

Algorithm 1 gets as input a data set of past observations of the system, ${\mathcal{D}}^{-1}=\left\{\right({\gamma }^i,{\omega }_F^i,{\omega }_L^i,J_L^i{\left)\right\}}_{i=1}^N$; the specification of the incentive function space given as an RKHS ${\mathcal{H}}_k$, where ${\gamma }^t\in {\mathcal{H}}_k^d$ (finite function space) or ${\gamma }^t\in {\mathcal{H}}_k$ (infinite function space); the number of rounds T contemplated in the sequential optimization; the kernel K of the functional GP; the exploration schedule ${\left\{{\beta }_t\right\}}_{t=1}^T$; and the CGM ${\mathcal{M}}_{L1F1}$ for a $PA{P}_{L1F1}$ from the principal’s perspective. The functional space ${\mathcal{H}}_k$ can be specified by stating the reproducing kernel k of the RKHS ${\mathcal{H}}_k$ (see Appendix B). In addition, the functional GP kernel K over $\Gamma \subset {\mathcal{H}}_k$ must be specified (by default, we can use the functional RBF kernel of Equation (53)). In Algorithm 1, we use a time-indexed exploration coefficient ${\beta }^t$ to match the high-probability guarantees in Section 4.4. For these results, we instantiate

$${\beta }^t=2\log \left({\displaystyle \frac{t^2{\pi }^2|\Gamma |}{6\delta }}\right),$$

so, the acquisition is ${\mu }^t\left(\gamma \right)+{\beta }^t{\sigma }^t\left(\gamma \right)$. In practice, one may instead tune a constant confidence level $\rho$ and set $\beta ={\Phi }^{-1}\left(\rho \right)$ (where ${\Phi }^{-1}$ is the quantile function, i.e., the inverse cumulative distribution function (CDF); see Appendix C.2). This is equivalent to using a constant exploration coefficient ${\beta }^t\equiv \beta$. Our regret bounds in Section 4.4 hold whenever the run uses the exploration schedule ${\left\{{\beta }^t\right\}}_{t=1}^T$ as ${\beta }^t=2\log \left(\frac{t^2{\pi }^2|\Gamma |}{6\delta }\right)$ or any schedule that dominates it pointwise. We can set the exploration schedule in this way for default in Algorithm 1.

The ${\mathcal{M}}_{L1F1}$ from the principal’s perspective have the induced DAG, from the set of observable variables ${\mathbf{O}}_L=\{\Gamma ,{\Omega }_L,{\Omega }_F,J_L\}$, with arrows $\Gamma \to {\Omega }_L$, $\Gamma \to {\Omega }_F$, ${\Omega }_F\to {\Omega }_L$, ${\Omega }_L\to J_L$ and ${\Omega }_F\to J_L$, as causal structure. Additionally, the follower utility $J_F$ is an unobservable or hidden variable, and the follower mechanism $f_{{\Omega }_F}(\gamma ,U_{{\Omega }_F})$ is a black box function. Algorithm 1 returns the accumulated history data set ${\mathcal{D}}^T$ after T rounds in the sequential optimization:

$${\mathcal{D}}^T=\left\{\left({\gamma }^1,{\mathbb{E}}^{{\gamma }^1}\left[J_L\right]\right),\dots ,\left({\gamma }^T,{\mathbb{E}}^{{\gamma }^T}\left[J_L\right]\right)\right\}.$$

(58)

Modeling $f\left(\gamma \right)$ as a sample from $F_{\gamma }=\{f\left(\gamma \right)\mid \gamma \in {\mathcal{H}}_k\}$ with $F_{\gamma }\sim \mathcal{GP}(\mu \left(\gamma \right),K(\gamma ,{\gamma }^{\prime }))$, as a surrogate model for the objective functional ${\mathbb{E}}^{{\gamma }^t}\left[J_L\right]$: ${\mathcal{H}}_k\to \mathbb{R}$ (see Section 4.1). We completely specified this GP by its mean functional $\mu \left(\gamma \right)=\mathbb{E}\left[f\right(\gamma \left)\right]$ and kernel functional $K(\gamma ,{\gamma }^{\prime })=\mathbb{E}\left[\left(f\left(\gamma \right)-\mu \left(\gamma \right)\right)\left(f\left({\gamma }^{\prime }\right)-\mu \left({\gamma }^{\prime }\right)\right)\right]$. We assume, without loss of generality (see [4]), a GP with mean function zero, i.e., $\mu \equiv 0$, as prior distribution over $F_{\gamma }$. So, we initialize the functional GP in line 2 with ${\mu }^0\left(\gamma \right)$ as the zero function and $K^0(\gamma ,{\gamma }^{\prime })$ as the functional RBF in Equation (53), to begin the inductive process of Algorithm 1.

From Line 3 to Line 8, the sequential optimization is performed, going through T rounds. The decision on the incentive function ${\gamma }^t$ is made employing the GP-UCB acquisition functional (see Section 4.2) as a policy ${\pi }^t$: ${\mathcal{D}}^{t-1}\to {\gamma }^t$. The expected value of $J_L$ on the post-intervention distribution $p\left(J_L\mid do(\Gamma ={\gamma }^t)\right)$ is estimated by computing $\widehat{V}\left({\gamma }^t\right)=\widehat{\mathbb{E}}\left[J_L\mid do\left(\Gamma ={\gamma }^t\right)\right]$, using the techniques developed in Section 3.3. The estimation of ${\mathbb{E}}^{{\gamma }^t}\left[J_L\right]$, is the approach by which Algorithm 1 gets information about the objective functional of Equation (52) to guide the search for ${\gamma }^{*}$. To complete a round t in the sequential optimization process outlined in Algorithm 1, first the data set ${\mathcal{D}}^t$ is updated in Line 6 by adding the new observation pair $({\gamma }^t,{\mathbb{E}}^{{\gamma }^t}\left[J_L\right])$. Then, in Line 7, the surrogate functional, i.e., the functional GP $\mathcal{GP}$, for the objective functional of the optimization problem in Equation (52), is updated using ${\mathcal{D}}^t$. Figure 2 complements Algorithm 1 with a flow-chart of one FCBO round. The support-aware GP-UCB step and the two-layer estimator are highlighted as novel elements.

Once the T rounds are completed, Algorithm 1 returns ${\mathcal{D}}^T$. Observe that, despite the fact that the algorithm returns the cumulative history data set in Equation (58), it implicitly addresses the optimization problem of Equation (52), since the following holds for large T:

$${\gamma }^{*}\approx {\gamma }^{+}\in \underset{\left({\gamma }^t,{\mathbb{E}}^{{\gamma }^t}\left[J_L\right]\right)\in {\mathcal{D}}^T}{\arg \max }\left\{{\mathbb{E}}^{{\gamma }^1}\left[J_L\right],\dots ,{\mathbb{E}}^{{\gamma }^T}\left[J_L\right]\right\}.$$

(59)

That is, due to the convergence result in Section 4.4 for Algorithm 1, we maintain the best incentive function ${\gamma }^{+}$ from $\left({\gamma }^{+},{\mathbb{E}}^{{\gamma }^{+}}\left[J_L\right]\right)\in {\mathcal{D}}^T$ to solve Equation (52), as ${\gamma }^{+}$ approximates the optimum ${\gamma }^{*}$ of the functional optimization problem in Equation (52). Furthermore, we can bound the expected cumulative regret $\mathbb{E}\left[R_{{\mathcal{D}}^T}^T\right]$ using Algorithm 1 to address the singe stage canonical $PA{P}_{L1F1}$. This result is presented in the following subsection.

4.4. Information-Theoretic Regret Bounds on the Stackelberg FCBO Algorithm

The cumulative-regret bounds for Algorithm 1 are derived by adapting the regret bounds results in terms of differential information gain (see Section 4.4.1) from [7] to our problem setting. In our setting, each observation $y_t=\widehat{V}\left({\gamma }_t\right)$ aggregates information about an interventional query $V\left({\gamma }_t\right)$ on the underlying CGM. The resulting regret bounds therefore quantify how quickly an information-guided sequence of interventions $do(\Gamma ={\gamma }_t)$ converges, in policy value, to the best available intervention in the considered class. First, in Section 4.4.2, we deal with the setting in which the set of admissible incentive functions $\Gamma$ is finite. Specifically, we assume that $\Gamma$ is represented as the finite RKHS ${\mathcal{H}}_k^d$ provided in Appendix B, so $|\Gamma |=d\in \mathbb{N}$. Then, we show how to extend the results for the finite case from Section 4.4.2 to the general case where $\Gamma$ is an infinite RKHS ${\mathcal{H}}_k$ in Section 4.4.3.

Recall from Section 4.1 that the objective functional in Equation (52) is viewed as a sample path f of the functional GP $F_{\gamma }=\{f\left(\gamma \right)\mid \gamma \in {\mathcal{H}}_k\}$ with $F_{\gamma }\sim \mathcal{GP}(\mu \left(\gamma \right),K(\gamma ,{\gamma }^{\prime }))$. Algorithm 1 uses a centered GP $\mathcal{GP}\left(\mu \equiv 0,K(\gamma ,{\gamma }^{\prime })\right)$ as a prior distribution $\mathcal{GP}\left({\mu }^0\left(\gamma \right),K^0(\gamma ,{\gamma }^{\prime })\right)$ over $F_{\gamma }$. So, we are assuming a priori that the objective functional in Equation (52) is a sample path f from a centered GP $\mathcal{GP}\left(\mu \equiv 0,K(\gamma ,{\gamma }^{\prime })\right)$. The initialization of the functional GP in Algorithm 1 also specified $K^0(\gamma ,{\gamma }^{\prime })$ as the functional RBF in Equation (53). However, following the analysis in [7], we do not assume a specific functional kernel.

At round t, the observation is $y_t=\widehat{V}\left({\gamma }_t\right)$ with estimation error ${\epsilon }_t$: $=y_t-V\left({\gamma }_t\right)$. We assume that ${\left\{{\epsilon }_t\right\}}_{t\ge 1}$ forms a conditionally sub-Gaussian martingale-difference sequence with respect to the algorithm’s filtration $\left\{{\mathcal{F}}_t\right\}$, i.e., $\mathbb{E}[{\epsilon }_t\mid {\mathcal{F}}_{t-1}]=0$ and $\mathbb{E}[exp\left(\lambda {\epsilon }_t\right)\mid {\mathcal{F}}_{t-1}]\le exp({\lambda }^2{R}^2/2)$ for all $\lambda \in \mathbb{R}$ and some envelope $R>0$ (cf. kernelized bandit analyses with conditionally sub-Gaussian noise [8]). The variance may depend on ${\gamma }_t$ (heteroskedasticity). For the analysis, we further assume a uniform bound $\mathrm{Var}({\epsilon }_t\mid {\mathcal{F}}_{t-1})\le {\sigma }_{*}^2$ on the support set where the Stackelberg FCBO algorithm proposes candidates. Under this uniform envelope, the standard GP-UCB high-probability confidence sets and regret bounds apply, with ${\sigma }_{*}^2$ entering the information-gain and ${\beta }_t$ expressions (Section 4.4.1).

4.4.1. Differential Information Gain

A primary concern in the regret analysis of Algorithm 1 is to measure the efficiency with which $\tau$ noisy observations ${\mathcal{D}}^{\tau }=\left\{\left({\gamma }^1,{\mathbb{E}}^{{\gamma }^1}\left[J_L\right]\right),\dots ,\left({\gamma }^{\tau },{\mathbb{E}}^{{\gamma }^{\tau }}\left[J_L\right]\right)\right\}$, and acquire knowledge about the objective functional $\mathbb{E}\left[J_L\mid do(\Gamma ={\gamma }^t)\right]$ in Equation (52). This learning process becomes apparent through the improvement of the GP surrogate model $\mathcal{GP}(\mu \left(\gamma \right),K(\gamma ,{\gamma }^{\prime }))$ for $\mathbb{E}\left[J_L\mid do(\Gamma ={\gamma }^t)\right]$, after the observation process in which the Algorithm 1 decides on ${\gamma }_t$, by the GP-UCB acquisition functional policy ${\pi }^t$: ${\mathcal{D}}^{t-1}\to {\gamma }^t$, and then, after the follower response ${\omega }_F$, computes $\widehat{V}\left({\gamma }^t\right)=\widehat{\mathbb{E}}\left[J_L\mid do(\Gamma ={\gamma }^t)\right]$ to obtain the observed pair $\left({\gamma }^t,{\mathbb{E}}^{{\gamma }^t}\left[J_L\right]\right)$. The informativeness of this observation process, as a function of the number of observations $\tau$, provides a way to bound the ability to learn about the objective functional $\mathbb{E}\left[J_L\mid do(\Gamma ={\gamma }^t)\right]$.

Here, we use the notation ${\widehat{\mathbb{E}}}^{{\gamma }^t}\left[J_L\right]={\mathbb{E}}^{{\gamma }^t}\left[J_L\right]+{ϵ}^t$, to explicitly distinguish between the assumed noisy observations ${\widehat{\mathbb{E}}}^{{\gamma }^t}\left[J_L\right]$ and exact observations ${\mathbb{E}}^{{\gamma }^t}\left[J_L\right]$ on the objective functional $\mathbb{E}\left[J_L\mid do(\Gamma ={\gamma }^t)\right]$. The information capacity of an arbitrary set of sampling points ${\Gamma }^{\tau }=\{{\gamma }^1,\dots ,{\gamma }^{\tau }\}\subset \Gamma$ regarding the objective functional $\mathbb{E}\left[J_L\mid do(\Gamma ={\gamma }^t)\right]$ can be measured by the mutual information $I^{\tau }$, also known as information gain, between ${\mathbf{f}}^{\tau }=\{{\mathbb{E}}^{{\gamma }^1}\left[J_L\right],\dots ,{\mathbb{E}}^{{\gamma }^{\tau }}\left[J_L^{\tau }\right]\}$ and the noisy observations ${\mathbf{y}}^{\tau }=\{{\widehat{\mathbb{E}}}^{{\gamma }^1}\left[J_L\right],\dots ,{\widehat{\mathbb{E}}}^{{\gamma }^{\tau }}\left[J_L\right]\}$ at these points. That is, measuring the reduction in uncertainty over $\mathbb{E}\left[J_L\mid do(\Gamma ={\gamma }^t)\right]$ resulting from the disclosure of ${\mathbf{y}}^{\tau }$, which is calculated as in Equation (60).

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill I^{\tau }({\mathbf{f}}^{\tau };{\mathbf{y}}^{\tau })& =H\left({\mathbf{f}}^{\tau }\right)-H({\mathbf{f}}^{\tau }\mid {\mathbf{y}}^{\tau })\hfill \\ \hfill & =H\left({\mathbf{y}}^{\tau }\right)-H({\mathbf{y}}^{\tau }\mid {\mathbf{f}}^{\tau })=I^{\tau }({\mathbf{y}}^{\tau };{\mathbf{f}}^{\tau }),\hfill \end{array}\end{array}$$

(60)

as mutual information is symmetric, and where $H\left({\mathbf{f}}^{\tau }\right)$ denotes the differential entropy of ${\mathbf{f}}^{\tau }$ and $H({\mathbf{y}}^{\tau }\mid {\mathbf{f}}^{\tau })$ denotes the conditional differential entropy of ${\mathbf{y}}^{\tau }$ given ${\mathbf{f}}^{\tau }$. The differential entropy of a set of multivariate Gaussian distributed random variables is calculated as $H(X_1,\dots ,X_n)=H\left({\mathcal{N}}_n(\mathit{\mu },K)\right)={\displaystyle \frac{1}{2}}\log [{\left(2\pi e\right)}^n\left|K\right|]$, where $\left|K\right|$ denotes the determinant of the covariance matrix K (see [28]). Thus, we can write its covariance as $K^{\tau }+{\sigma }^{2}I$, that is, separating the additive noise into a diagonal matrix ${\sigma }^{2}I$, where $K^{\tau }=[K({\widehat{\mathbb{E}}}^{\gamma }\left[J_L\right],{\widehat{\mathbb{E}}}^{{\gamma }^{\prime }}\left[J_L\right]$ for every ${\widehat{\mathbb{E}}}^{\gamma }\left[J_L\right],{\widehat{\mathbb{E}}}^{{\gamma }^{\prime }}\left[J_L\right]\in {\mathbf{y}}^{\tau }$. Then, we can compute $H\left({\mathbf{y}}^{\tau }\right)$ as follows:

$$H\left({\mathbf{y}}^{\tau }\right)={\displaystyle \frac{1}{2}}\log [{\left(2\pi e\right)}^{\tau }|K^{\tau }+{\sigma }^{2}I\left|\right].$$

(61)

To compute $H({\mathbf{y}}^{\tau }\mid {\mathbf{f}}^{\tau })$, observe that given ${\mathbf{f}}^{\tau }$, the only randomness left is the noise, which has covariance ${\sigma }^{2}I$, so

$$H({\mathbf{y}}^{\tau }\mid {\mathbf{f}}^{\tau })={\displaystyle \frac{1}{2}}\log [{\left(2\pi e\right)}^{\tau }|{\sigma }^{2}I\left|\right]={\displaystyle \frac{1}{2}}\log \left[{\left(2\pi e\right)}^{\tau }{\sigma }^{2\tau }\right].$$

(62)

Putting all together, the information gain $I^{\tau }({\mathbf{f}}^{\tau };{\mathbf{y}}^{\tau })$ is given by ${\displaystyle \frac{1}{2}}\log \left[|I+K^{\tau }{\sigma }^{-2}|\right]$ as shown next in Equation (63):

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill I^{\tau }({\mathbf{f}}^{\tau };{\mathbf{y}}^{\tau })& =H\left({\mathbf{y}}^{\tau }\right)-H({\mathbf{y}}^{\tau }\mid {\mathbf{f}}^{\tau })\hfill \\ \hfill & ={\displaystyle \frac{1}{2}}{\log [\left(2\pi e\right)}^{\tau }|K^{\tau }+{\sigma }^{2}I\left|\right]-{\displaystyle \frac{1}{2}}{\log [\left(2\pi e\right)}^{\tau }|{\sigma }^{2}I\left|\right]\hfill \\ \hfill & ={\displaystyle \frac{1}{2}}\log \left[{\displaystyle \frac{|K^{\tau }+{\sigma }^{2}I|}{|{\sigma }^{2}I|}}\right]={\displaystyle \frac{1}{2}}\log \left[|I+K^{\tau }{\sigma }^{-2}|\right]\hfill \end{array}\end{array}$$

(63)

$$I({\mathbf{f}}_t;{\mathbf{y}}_t)=H\left({\mathbf{y}}_t\right)-H({\mathbf{y}}_t\mid {\mathbf{f}}_t)={\displaystyle \frac{1}{2}}\log \left|\mathbf{I}+{\sigma }^{-2}{\mathbf{K}}_t\right|$$

In our setting, we define the maximum information gain ${\mathcal{I}}^T$ after T rounds as

$${\mathcal{I}}^T={\displaystyle \frac{1}{2}}\log \left|\mathbf{I}+{\Sigma }_T^{-1}{\mathbf{G}}^T\right|,{\Sigma }_T:=\mathrm{diag}({\sigma }_1^2,\dots ,{\sigma }_T^2),$$

(64)

where $\mathbf{I}$ is the identity matrix and ${\mathbf{G}}^T$ is the covariance matrix. For notational simplicity in the stated bounds we adopt the conservative specialization ${\Sigma }_T⪯{\sigma }_{*}^2\mathbf{I}$, which yields $I_T\le \frac{1}{2}\log |\mathbf{I}+{\sigma }_{*}^{-2}{G}_T|$ and recovers the expressions in our regret constants.

After presenting the concept of mutual information on differential entropy to assess the informativeness with respect to the objective functional $\mathbb{E}\left[J_L\mid do(\Gamma ={\gamma }^t)\right]$ of a noisy set of observations ${\mathbf{y}}^{\tau }=\{{\widehat{\mathbb{E}}}^{{\gamma }^1}\left[J_L\right],\dots ,{\widehat{\mathbb{E}}}^{{\gamma }^{\tau }}\left[J_L\right]\}$ at a certain set of sampling points ${\Gamma }^{\tau }=\{{\gamma }^1,\dots ,{\gamma }^{\tau }\}\subset \Gamma$. We are now able to present the regret analysis for Algorithm 1 in terms of information gain ${\mathcal{I}}^{\tau }({\mathbf{f}}^{\tau };{\mathbf{y}}^{\tau })$.

4.4.2. Regret Bound for a Finite Incentive Function Space

The cumulative regret in a sequential optimization process is the loss in reward resulting from the inability to predict the points that maximize the objective function, in advance. For Algorithm 1, this is the error between the decided incentive functions ${\gamma }^1,\dots ,{\gamma }^T$ and those that maximize the objective functional $\mathbb{E}\left[J_L\mid do(\Gamma =\gamma )\right]$. That is, letting ${\gamma }^{*}\in \arg {\max }_{{\gamma }^t\in \Gamma }f\left(\gamma \right)$, with $f\left(\gamma \right)=\mathbb{E}\left[J_L\mid do(\Gamma ={\gamma }^t)\right]$, for a incentive function choice ${\gamma }^t$ in round t, we incur instantaneous regret $r^t=f\left({\gamma }^{*}\right)-f\left({\gamma }^t\right)$, and the cumulative regret $R_T$ is defined as $R^T={\sum }_{t=1}^T{r}^t$.

Theorem 1 establishes a probabilistic bound for the cumulative regret of Algorithm 1, ensuring its convergence and efficiency with high probability. Considering a confidence bound $\delta \in (0,1)$, the result of Theorem 1 is that the cumulative regret $R^T$ of Algorithm 1 is bound by the square root of a linear expression of the information gain ${\mathcal{I}}^T({\mathbf{f}}^T;{\mathbf{y}}^T)$, with probability of at least $1-\delta$. Recall from previous Section 4.4.1 that information gain ${\mathcal{I}}^T({\mathbf{f}}^T;{\mathbf{y}}^T)$ is a measure of the reduction in uncertainty over the objective functional, resulting from the disclosure of observations ${\mathbf{y}}^T=\{{\widehat{\mathbb{E}}}^{{\gamma }^1}\left[J_L\right],\dots ,{\widehat{\mathbb{E}}}^{{\gamma }^T}\left[J_L\right]\}$, generated by the decision of the incentive function, i.e., sampling points, ${\Gamma }^{\tau }=\{{\gamma }^1,\dots ,{\gamma }^{\tau }\}\subset \Gamma$. The linear expression for the information gain in the specified upper bound includes the low coefficients ${\beta }^T$ and $C_1$. The coefficient ${\beta }^t$ is the exploration coefficient in the acquisition functional (see Section 4.2) on the strategy to choose the next incentive function in Equation (57), which for the purpose of the bound we use ${\beta }^t=2\log \left({\displaystyle \frac{t^2{\pi }^2|\Gamma |}{6\delta }}\right)$, for all $t\in \left[T\right]$. This choice reconciles the exploration coefficient used in Algorithm 1 with the regret analysis: any implementation that uses ${\beta }^t$ above (or a pointwise larger schedule) satisfies the bound of Theorem 1. For the auxiliary coefficient $C_1$, we use $C_1={\displaystyle \frac{4{\sigma }^2}{\log (1+{\sigma }^{-2})}}$. Theorem 1 given below assumes the finite functional space ${\mathcal{H}}_k^d$, which was established in the Appendix B, as the finite set of allowed incentive functions, i.e., $\Gamma ={\mathcal{H}}_k^d$ so $|\Gamma |=d$ for some $d\in \mathbb{N}$. Recall from Section 4.1 that the objective functional in Equation (52) is viewed as a sample path f of the functional GP $F_{\gamma }=\{f\left(\gamma \right)\mid \gamma \in {\mathcal{H}}_k\}$ with $F_{\gamma }\sim \mathcal{GP}(\mu \left(\gamma \right),K(\gamma ,{\gamma }^{\prime }))$. Assume further that the observation errors $\left\{{\epsilon }_t\right\}$ are conditionally R-sub-Gaussian and form a martingale-difference sequence with a uniform variance envelope $\mathrm{Var}({\epsilon }_t\mid {\mathcal{F}}_{t-1})\le {\sigma }_{*}^2$ over the proposal support.

Theorem 1.

Let $\delta \in (0,1)$ and assume that Γ is finite, with $\Gamma ={\mathcal{H}}_k^d$ for some $d\in \mathbb{N}$. Running Algorithm 1 using ${\beta }^t=2\log \left({\displaystyle \frac{t^2{\pi }^2|\Gamma |}{6\delta }}\right)$ for a sample path $f\left(\gamma \right)=\mathbb{E}\left[J_L\mid do(\Gamma =\gamma )\right]$ of a functional GP $F_{\gamma }=\{f\left(\gamma \right)\mid \gamma \in {\mathcal{H}}_k^d\}\sim \mathcal{GP}(\mu \left(\gamma \right),K(\gamma ,{\gamma }^{\prime }))$ with mean function ${\mu }^t\left({\gamma }^t\right)=0$ and covariance function $K^t(\gamma ,{\gamma }^{\prime })$, we get a cumulative-regret bound of ${\mathcal{O}}^{*}\sqrt{T{\mathcal{I}}^{\tau }({\mathbf{f}}^{\tau };{\mathbf{y}}^{\tau })\log |\Gamma |}$ with high probability. Specifically,

$$\mathrm{Pr}\left\{R_{{\mathcal{D}}^T}^T\le \sqrt{C_{1}T{\beta }^T{\mathcal{I}}^T({\mathbf{f}}^T;{\mathbf{y}}^T)}\right\}\ge 1-\delta ,$$

for all $T\ge 1$, where $C_1={\displaystyle \frac{4{\sigma }^2}{\log (1+{\sigma }^{-2})}}$.

The proof of Theorem 1 is provided in Appendix A by demonstrating a series of partial results given as Propositions A1–A4.

4.4.3. Regret Bound on an Infinite Incentive Function Space

The regret bound of Theorem 1 is established under the assumption that the set of admissible incentive functions $\Gamma$ is finite, specifically $\Gamma ={\mathcal{H}}_k^d$. In practice, however, the Stackelberg FCBO algorithm can operate over a continuous, potentially infinite-dimensional admissible set of incentive functions $\Gamma \subset {\mathcal{H}}_k$, where $H_k$ is a bounded RKHS. In this subsection, we extend the cumulative-regret analysis of the Stackelberg FCBO algorithm to an infinite admissible space of incentive functions $\Gamma \subset {\mathcal{H}}_k$ where ${\mathcal{H}}_k$ is a bounded separable RKHS. This corresponds to the setting where incentive functions ${\gamma }^t\in \Gamma ={\mathcal{H}}_k$ are sampled and optimized over a non-parametric space of continuous functions. In accordance with the infinite domain analysis in [7], we assume that the functional $f\left(\gamma \right)$, with $f\left(\gamma \right)=\mathbb{E}\left[J_L\mid do(\Gamma =\gamma )\right]$ is Lipschitz continuous and is a member of the RKHS that is induced by the functional kernel.

Applying the regret bounds of Theorem 1 to a finite subset of $H_k$ obtained by discretizing $H_k$ does not ensure that the regret bound is applicable to the original continuous infinite domain $H_k$. The issue is that the objective functional f: $H_k\to \mathbb{R}$, even if continuous, may not be well approximated on all of the function space $H_k$ by its restriction to a finite set, and this would only control regret over the discretized surrogate problem, not the full functional optimization problem:

$${\gamma }^{*}\in \underset{{\gamma }^t\in \Gamma \subset {\mathcal{H}}_k}{\arg \max }\mathbb{E}\left[J_L\mid do\left(\Gamma ={\gamma }^t\right)\right].$$

Furthermore, regret is accumulated over the true sequence of evaluations, which may lie outside any fixed discretization. So, to rigorously lift the regret guarantees from a finite discretization to the full RKHS domain $H_k$, we require uniform approximation of both (i) the infinite set of admissible incentive functions and (ii) the objective functional f: $H_k\to \mathbb{R}$ itself. Therefore, the key step is to approximate the infinite-dimensional domain $H_k$ and the functional $f\left(\gamma \right)$ in a way that allows the application of the previously developed finite-set regret analysis from Theorem 1.

In order to overcome this, we approximate the infinite function space by constructing a finite-dimensional subspace by means of two applications of the Stone–Weierstrass theorem (see Appendix D). In the first application, the objective functional f: $\Gamma \to \mathbb{R}$, with $\Gamma \subset {\mathcal{H}}_k$ compact, is uniformly approximated on $\Gamma$ by finite linear combinations of kernel sections $K(·,{\gamma }_i)$ associated with a carefully chosen finite set of basis incentive functions $\{{\gamma }_1,\dots ,{\gamma }_d\}\subset {\mathcal{H}}_k$. As a consequence, for every ${\epsilon }_1>0$, there exists such a finite set and coefficients ${\alpha }_1,\dots ,{\alpha }_d\in \mathbb{R}$ satisfying

$$\underset{\gamma \in \Gamma }{sup}\left|f\left(\gamma \right)-\sum _{i=1}^d{\alpha }_{i}K({\gamma }_i,\gamma )\right|<{\epsilon }_1,$$

and the set ${\left\{{\gamma }_i\right\}}_{i=1}^d\subset \Gamma$ then acts as a basis of reference incentive functions, and the approximant ${\sum }_{i=1}^d{\alpha }_{i}K({\gamma }_i,\gamma )$ providing a finite-dimensional parametric model for f. Details of this construction are given in Appendix D, where an explanation of two formal methods for selecting such basis incentive functions (one based on minimal $\epsilon$-covers and one based on spectral decomposition of the kernel operator) is also provided in Appendix D.2.

From the first application of the Stone–Weierstrass theorem, we obtained a finite set of basis incentive functions $\{{\gamma }_1,\dots ,{\gamma }_d\}\subset {\mathcal{H}}_k$ and coefficients $\left\{{\alpha }_i\right\}$ such that $f\left(\gamma \right)$ is uniformly approximated, to within an error ${\epsilon }_1>0$, by

$$\sum _{i=1}^d{\alpha }_{i}K({\gamma }_i,\gamma ).$$

In the second application, each basis function ${\gamma }_i$ is itself uniformly approximated on the compact input domain ${\Omega }_F$, the input space of the follower’s decision variable, by a polynomial $p_i^{\left(n\right)}$ of degree at most n, so that the entire approximation to f is parameterized by finitely many coefficients. That is, we now apply the Stone–Weierstrass theorem a second time, but this time to each basis function ${\gamma }_i$ individually, treating it as a continuous scalar-valued function on the compact domain ${\Omega }_F$. Let ${\mathcal{P}}_n$ denote the algebra of real polynomials on ${\Omega }_F$ of degree at most n. The Stone–Weierstrass theorem on ${\Omega }_F$, compact in the Euclidean topology, ensures that ${\mathcal{P}}_n$ is uniformly dense in $C({\Omega }_F,\mathbb{R})$. Therefore, for any ${\epsilon }_2>0$ and for each $i\in \{1,\dots ,d\}$, there exists a polynomial $p_i^{\left(n\right)}\in {\mathcal{P}}_n$ such that

$$\underset{x\in {\Omega }_F}{sup}|{\gamma }_i\left(x\right)-p_i^{\left(n\right)}\left(x\right)|<{\epsilon }_2.$$

Therefore, each ${\gamma }_i$ can be replaced by a polynomial $p_i^{\left(n\right)}$ of degree $\le n$ with uniform approximation error ${\epsilon }_2$ on ${\Omega }_F$. So, the resulting functional approximation becomes

$$f\left(\gamma \right)\approx \sum _{i=1}^d{\alpha }_{i}K\left(p_i^{\left(n\right)},\gamma \right),$$

where K is evaluated between the polynomial surrogate $p_i^{\left(n\right)}$ and the input $\gamma$.

Replacing each ${\gamma }_i$ with $p_i^{\left(n\right)}$ parameterises the basis incentive functions by their polynomial coefficients. So, the full parameterisation of the approximant to f now consists of (1) the d coefficients $\left\{{\alpha }_i\right\}$ from the first approximation step, and (2) the coefficients of the d polynomials $\left\{p_i^{\left(n\right)}\right\}$, each with finitely many coefficients determined by its degree n. This yields a finite-dimensional cross-parameter space:

$${\Theta }_{d,n}=\left\{({\alpha }_1,\dots ,{\alpha }_d,\mathrm{coeffs}\left(p_1^{\left(n\right)}\right),\dots ,\mathrm{coeffs}\left(p_d^{\left(n\right)}\right))\right\}$$

that fully describes the approximant to f. Hence, the second Stone–Weierstrass application replaces infinite-dimensional basis functions ${\gamma }_i\in {\mathcal{H}}_k$ with finite-dimensional polynomial surrogates $p_i^{\left(n\right)}$, thus turning the problem of learning f over $\Gamma$ into an optimization over the finite-dimensional parameter set ${\Theta }_{d,n}$.

This two-step reduction maps the original optimization problem over the infinite-dimensional domain $\Gamma \subset {\mathcal{H}}_k$ to an optimization over a finite-dimensional parameter space, while controlling the approximation error. The uniform error ${\epsilon }_2$ in approximating each ${\gamma }_i$ propagates to the approximation of f in a way controlled by the Lipschitz constant of f with respect to $\gamma$, ensuring that the total error from both applications remains bounded when d and n are chosen large enough. So, by Lipschitz continuity of f on $\Gamma$ with respect to ${\parallel ·\parallel }_{{\mathcal{H}}_k}$, with constant $L>0$, this perturbation changes the output of f by at most $L{\epsilon }_2$. Consequently, the regret bound in the infinite-dimensional case satisfies

$$R_{\inf }^T\le R_{\mathrm{finit}}^T+T\left({\epsilon }_1+L{\epsilon }_2\right),$$

where $R_T^{\mathrm{finit}}$ is the cumulative-regret bound from the finite-basis case. By choosing the basis size d and polynomial degree n large enough, both ${\epsilon }_1$ and ${\epsilon }_2$ can be made arbitrarily small, so that the additive term $T({\epsilon }_1+L{\epsilon }_2)$ remains negligible compared to the leading $\sqrt{T}$-scale term in $R_T^{\inf }$. See Theorem 2 below for explicit assumptions and a sublinear cumulative-regret guarantee in the infinite-domain setting.

This reduction method explicitly connects the choice of basis to the covering number $N\left(\Gamma ,{\parallel ·\parallel }_{{\mathcal{H}}_k},\epsilon \right)$, which determines the discretization size $|{\Gamma }_{\epsilon }|$, where ${\Gamma }_{\epsilon }=\{{\gamma }_1,\dots ,{\gamma }_d\}$ and

$$N\left(\Gamma ,{\parallel ·\parallel }_{{\mathcal{H}}_k},\epsilon \right)=min\left\{|{\Gamma }_{\epsilon }|:{\Gamma }_{\epsilon }\subset \Gamma \mathrm{is}\mathrm{an}\epsilon \text{-cover}\mathrm{of}\Gamma ,\mathrm{for}{\parallel ·\parallel }_{{\mathcal{H}}_k}\right\}.$$

In the finite function space regret analysis of Theorem 1 (See Equation (A4)), the confidence parameter has the form ${\beta }^T=2\log ({\displaystyle \frac{{\pi }^2{T}^2}{6\delta }}|\Gamma |)$, which leads to following:

$${\beta }^T=2\log \left({\displaystyle \frac{{\pi }^2{T}^2}{6\delta }}N\left(\Gamma ,{\parallel ·\parallel }_{{\mathcal{H}}_k},\epsilon \right)\right)\le 2\log \left({\displaystyle \frac{{\pi }^2{T}^2}{6\delta }}\left|{\Gamma }_{\epsilon }\right|\right);$$

(65)

as in practice, the set may come from a near-minimal cover so $d=|{\Gamma }_{\epsilon }|\ge N\left(\Gamma ,{\parallel ·\parallel }_{{\mathcal{H}}_k},\epsilon \right)$, i.e., the selected set forms a valid but not necessarily minimal $\epsilon$-cover; typically bounded by a small factor (often at most logarithmic in N for standard greedy covering procedures on compact metric spaces (See Appendix D.2). This relationship in Equation (65) shows that ${\beta }^T$ grows logarithmically with T but also depends logarithmically on the covering number of $\Gamma$, so reducing d toward the minimal covering number directly leads to a smaller ${\beta }_T$ and, therefore, a tighter regret bound. Nevertheless, even for a valid, but not necessarily minimal, $\epsilon$-cover, $|{\Gamma }_{\epsilon }|$ is constant for a fixed $\epsilon$, so ${\beta }_T\in \mathcal{O}\left(\log T\right)$.

Considering all the elements mentioned above and following the same reasoning as in the proof of Theorem 1 (See Appendix A), the regret analysis for $R_{finit}^T$ can be extended to the infinite-dimensional domain $\Gamma \subset H_k$ using a discretization argument as follows. We first construct an ${ϵ}_T$-cover ${\Gamma }_{{ϵ}_T}\subset \Gamma$ in the ${\mathcal{H}}_k$ norm, with size $|{\Gamma }_{{ϵ}_T}|=d_{{ϵ}_T}$, so that every $\gamma \in \Gamma$ is within ${ϵ}_T$ of some $\tilde{\gamma }\in {\Gamma }_{{ϵ}_T}$. A union bound over all $\gamma \in {\Gamma }_{{ϵ}_T}$ and all $t\le T$ yields, with probability at least $1-\delta$, a uniform confidence interval of the form

$$|f\left(\gamma \right)-{\mu }^{t-1}\left(\gamma \right)|\le \sqrt{{\beta }^T}{\sigma }^{t-1}\left(\gamma \right)+L{ϵ}_T,$$

where ${\beta }^T=2\log \left({\displaystyle \frac{T^2{\pi }^2}{6\delta }}{d}_{{ϵ}_T}\right)$ and L is the Lipschitz constant of f in the ${\mathcal{H}}_k$ norm, ${\parallel ·\parallel }_{{\mathcal{H}}_k}$.

Applying this bound to both the optimal ${\gamma }^{*}$ and the chosen ${\gamma }^t$ in the round t shows that the instantaneous regret satisfies $r^t=f\left({\gamma }^{*}\right)-f\left({\gamma }^t\right)\le 2\sqrt{{\beta }^T}{\sigma }^{t-1}\left({\gamma }^t\right)+2L{ϵ}_T$ (Proposition A2). Summing over $t=1,\dots ,T$ and using Cauchy–Schwarz together with the information gain bound ${\sum }_{t=1}^T{\sigma }_{t-1}^2\left({\gamma }_t\right)\le C_1{I}^{\tau }({\mathbf{f}}^{\tau };{\mathbf{y}}^{\tau })$ (Proposition A3), we obtain

$$R_{\inf }^T=\sum _{t=1}^T\left[f\left({\gamma }^{*}\right)-f\left({\gamma }^t\right)\right]\le 2\sqrt{C_{1}T{\beta }^T{\mathcal{I}}^T({\mathbf{f}}^T;{\mathbf{y}}^T)}+2TL{ϵ}_T.$$

(66)

Choosing ${ϵ}_T={\displaystyle \frac{1}{T}}$ makes the additive term $2TL{ϵ}_T$ a constant, which can be upper-bounded by ${\displaystyle \frac{{\pi }^2}{6}}$, using ${\sum }_{t=1}^{\infty }{\displaystyle \frac{1}{t^2}}={\displaystyle \frac{{\pi }^2}{6}}$. The final result is the high-probability bound.

$$R_{\inf }^T\le \sqrt{C_{1}T{\beta }^T{\mathcal{I}}^T({\mathbf{f}}^T;{\mathbf{y}}^T)}+{\displaystyle \frac{{\pi }^2}{6}}.$$

(67)

Theorem 2.

Let $\Gamma \subset {\mathcal{H}}_k$ be compact in ${\parallel ·\parallel }_{{\mathcal{H}}_k}$, and let K: $\Gamma \times \Gamma \to \mathbb{R}$ be bounded and continuous. Suppose observations follow $y^t=f\left({\gamma }^t\right)+{\epsilon }^t$ with σ-sub-Gaussian noise ${\epsilon }^t$, and f: $\Gamma \to \mathbb{R}$ is a sample path of the centered functional GP $\mathrm{GP}(0,K)$ and is L-Lipschitz with respect to ${\parallel ·\parallel }_{{\mathcal{H}}_k}$. For analysis, fix any discretization schedule ${\left\{{\epsilon }^t\right\}}_{t=1}^T$ and an ${\epsilon }^t$-cover ${\Gamma }_{{\epsilon }^t}\subset \Gamma$ with size $N(\Gamma ,{\parallel ·\parallel }_{{\mathcal{H}}_k},{\epsilon }^t)$. Run Algorithm 1 with GP-UCB and the exploration schedule

$${\beta }^t=2\log \left({\displaystyle \frac{{\pi }^2{t}^{2}N(\Gamma ,{\parallel ·\parallel }_{{\mathcal{H}}_k},{\epsilon }^t)}{6\delta }}\right),\delta \in (0,1).$$

Let ${\mathcal{I}}_T$ denote the (maximum) information gain after T rounds. Then, with probability at least $1-\delta$,

$$R_{inf}^T\le \sqrt{C_{1}T{\beta }^t{\mathcal{I}}_T}+2L\sum _{t=1}^T{\epsilon }^t,C_1=4{\sigma }^2\log \left(1+{\sigma }^{-2}\right).$$

In particular, if ${\sum }_{t=1}^{\infty }{\epsilon }^t<\infty$ (e.g., ${\epsilon }^t=c/t^2$), then $R_{inf}^T=\tilde{O}\left(\sqrt{T{\mathcal{I}}_T}\right)$ is sublinear; if moreover ${\mathcal{I}}_T=o\left(T\right)$, then $R_{inf}^T=o\left(T\right)$.

Proof. 

Combine the finite-$|\Gamma |$ GP-UCB bound with a union bound over the ${\epsilon }^t$-covers ${\Gamma }_{{\epsilon }^t}$ to obtain uniform confidence bands on ${\Gamma }_{{\epsilon }^t}$, which yields

$${\beta }^t=2\log ({\pi }^2{t}^{2}N(\Gamma ,{\parallel ·\parallel }_{{\mathcal{H}}_k},{\epsilon }^t)/\left(6\delta \right)).$$

Lipschitz continuity lifts these bands from ${\Gamma }_{{\epsilon }^t}$ to all $\Gamma$, adding $2L{\epsilon }^t$ to per-round regret; summing over t produces the stated bound. The two-step uniform-approximation argument (via discretization) recovers $R_{\inf }^T\le R_{\mathrm{finit}}^T+T({\epsilon }_1+L{\epsilon }_2)$, absorbed by ${\sum }_t{\epsilon }^t$ when the schedule is summable. □

Therefore, schedule ${\beta }^t$ mirrors the finite-set case with $|\Gamma |$ replaced by the covering number, and the bound extends Theorem 1 by adding a discretization penalty $2L{\sum }_t{\epsilon }^t$. Contemplating the following assumptions for the infinite-domain analysis: (i) $f\sim \mathrm{GP}(0,K)$ on $\Gamma$ with K bounded and continuous. (ii) f is L-Lipschitz in ${\parallel ·\parallel }_{{\mathcal{H}}_k}$ on $\Gamma$. (iii) Noise ${\epsilon }^t$ is $\sigma$-sub-Gaussian and Algorithm 1 uses ${\beta }^t$ as above. (iv) $\Gamma \subset {\mathcal{H}}_k$ is compact and covers ${\Gamma }_{{\epsilon }^t}$ exist with size $N(\Gamma ,{\parallel ·\parallel }_{{\mathcal{H}}_k},{\epsilon }^t)$. (v) The information gain ${\mathcal{I}}_T$ is finite.

4.4.4. Practical Implications of the Regret Bounds

The regret bounds for Stackelberg FCBO turn our theory into concrete guarantees for one-shot incentive design. They quantify, with high probability, how far the principal’s expected utility under the selected incentive can be from that of the best admissible incentive, before deployment. This matters in practice because the single-stage PAP requires a single commitment—there is no opportunity to “learn on the fly.”

The bounds certify a maximum difference in actual versus expected performance at the time of decision-making, offering an up-front measure of acceptable risk for using an incentive in important decisions (such as setting credit terms, granting subsidies, or allocating resources). Because the bound scales with differential information gain—that is, the knowledge added by each experiment—each offline evaluation reflects its informational value. This allows practitioners to prioritize experiments that reduce uncertainty the most, rather than running large, costly test batteries.

The bounds yield practical stopping rules. For example, stop when the acquisition-driven Upper Confidence Bound (a statistical estimate indicating the highest likely value) on the best candidate falls below a target gap (such as a minimum acceptable return on investment, ROI, or a maximum loss threshold). It also helps determine how many offline evaluations are needed to reach a desired near-optimality level, meaning how close the chosen candidate is to the optimal option.

In summary, the regret bounds provide actionable guarantees for offline policy selection: they quantify residual risk, guide the allocation of evaluation effort, and ensure that the final one-shot incentive is provably close to optimal under the given causal model and data.

4.5. On Extending CID to Multi-Follower Single-Stage Principal–Agent Problems

This subsection addresses the extension of the CID framework for single-stage principal–agent problems (SS–PAPs) to account for multiple followers. We illustrate this extension by presenting two contrasting CGMs for the base case of a PAP consisting of one principal and two followers. We use the notation ${\mathcal{M}}_{L_v^m{F}_u^n}$, where the superscripts m, n represent the number of principal agents and follower agents in the CGM, respectively (we omit the superscript when it is equal to one, as in the canonical CGM ${\mathcal{M}}_{L_1{F}_1}$), and the subscripts v, u the number of variables controlled by the principals, followers, respectively. To show how causal inference on canonical SS–PAPs can be extended to multi-follower SS–PAPs, we employ two representative models: the CGM ${\mathcal{M}}_{L_2{F}_1^2}$ with individualized incentives and independent follower utility functions, and the CGM ${\mathcal{M}}_{L_1{F}_1^2}$ with universal incentives and joint utility functions. For these CGMs, the endogenous variables are incentive function space variable $\Gamma$; follower actions ${\Omega }_F^1,{\Omega }_F^2$; principal action ${\Omega }_{L_1},{\Omega }_{L_2}$ (individualized) or action ${\Omega }_L$ (universal); utilities variables $J_F^1,J_F^2,J_L$. We assume as before that the exogenous variables are mutually independent, mean-zero; and Gaussian when needed for analytic smoothing: $U_{A^1},U_{A^2}$ (follower action noises), $U_{L_1},U_{L_2}$ or $U_L$ (leader-action noises) and $U_{J_F^1},U_{J_F^2},U_{J_L}$ (utility noises). In this subsection, for clarity in the notation, we sometimes use $a^1$, $a^2$ or $a^i$, $a^{-i}$ for the followers actions ${\omega }_F^1$ and ${\omega }_F^2$, and ${\mathcal{A}}_1$, ${\mathcal{A}}_2$ for ${\Omega }_F^1$, ${\Omega }_F^2$. As has been consistently acknowledged in this research, we assume the principal’s perceptive; i.e., the principal does not have knowledge about the followers utilities nor their best-response maps.

4.5.1. CGMs with Individualized Incentives and Independent Follower Utilities

In ${\mathcal{M}}_{L_2{F}_1^2}$, the principal has two decision variables ${\Omega }_{L_1}$ and ${\Omega }_{L_2}$, i.e., it must decides on two distinct incentive functions ${\gamma }_1\in \Gamma$ and ${\gamma }_2\in \Gamma$, one for each follower. This type of CGM for SS–PAPs with multiple followers allows the selection of distinct incentives for different followers, which is useful in multi-agent systems where it is important to break symmetries between followers. Furthermore, in the ${\mathcal{M}}_{L_2{F}_1^2}$ CGM, the utility function of each follower does not depend on the decisions of the other followers, i.e., there is no interference between the follower utilities: $J_F^i$ depends only on its own channel (though principal utility $J_L$ may couple channels). ${\Gamma }_1\to {\Omega }_F^1$ and ${\Gamma }_2\to {\Omega }_F^2$ via best responses (68), as shown in the structural equations $\mathbf{F}$ given below:

$$\begin{array}{}\mathrm{(68)}\hfill {\Omega }_F^1& ={\mathrm{BR}}_1({\Gamma }_1,U_{A^1}),\hfill & \hfill {\Omega }_F^2& ={\mathrm{BR}}_2({\Gamma }_2,U_{A^2}),\hfill \mathrm{(69)}\hfill {\Omega }_{L_1}& ={\Gamma }_1\left({\Omega }_F^1\right)+U_{L_1},\hfill & \hfill {\Omega }_{L_2}& ={\Gamma }_2\left({\Omega }_F^2\right)+U_{L_2},\hfill \mathrm{(70)}\hfill J_F^1& =g_F^1({\Omega }_{L_1},{\Omega }_F^1)+U_{J_F^1},\hfill & \hfill J_F^2& =g_F^2({\Omega }_{L_2},{\Omega }_F^2)+U_{J_F^2},\hfill \mathrm{(71)}\hfill J_L& =g_L({\Omega }_{L_1},{\Omega }_{L_2},{\Omega }_F^1,{\Omega }_F^2)+U_{J_L}.\hfill & \hfill {\Gamma }_1& ={\gamma }_1;{\Gamma }_2={\gamma }_2\hfill \end{array}$$

Therefore, choosing policies ${\Gamma }_1={\gamma }_1,{\Gamma }_2={\gamma }_2$, the principal wants to estimate the causal estimand $V({\gamma }_1,{\gamma }_2)=\mathbb{E}\left[J_L|\mathrm{do}({\Gamma }_1={\gamma }_1,{\Gamma }_2={\gamma }_2)\right]$ and targets incentive policies ${\gamma }_1^{*}$ and ${\gamma }_2^{*}$ such that

$${\gamma }_1^{*},{\gamma }_2^{*}\in \underset{{\gamma }_1\in {\Gamma }_1,{\gamma }_2\in {\Gamma }_2}{\arg \max }\mathbb{E}\left[J_L\mid do\left({\Gamma }_1={\gamma }_1,{\Gamma }_2={\gamma }_2\right)\right].$$

(72)

Using the g-formula, the identification formula for $V({\gamma }_1,{\gamma }_2)$ admits the familiar nested expectation form:

$$\begin{array}{}\mathrm{(73)}\hfill V({\gamma }_1,{\gamma }_2)& =\int \int \underset{\mathrm{inner}\mathrm{expectation}(\mathrm{Gaussian}\mathrm{smoothing})}{\underbrace{{\mathbb{E}}_{U_{L_1},U_{L_2}}\left[g_L\left({\gamma }_1\left(a^1\right)+U_{L_1},{\gamma }_2\left(a^2\right)+U_{L_2},a^1,a^2\right)\right]}}\hfill \mathrm{(74)}\hfill & \times p(a^1\mid {\Gamma }_1={\gamma }_1)p(a^2\mid {\Gamma }_2={\gamma }_2)d{a}^{1}d{a}^2,\hfill \end{array}$$

4.5.2. CGMs with Universal Incentive and Joint Follower Utilities

In contrast, in ${\mathcal{M}}_{L_1{F}_1^2}$, incentives are said to be universal, that is, the incentive function $\gamma$ is applied equally to all followers, and here the utility function of each follower depends on the decisions of the other followers. The structural equations $\mathbf{F}$ for ${\mathcal{M}}_{L_1{F}_1^2}$ are given in Equations (76)–(79).

$$\begin{array}{}\mathrm{(75)}\hfill \Gamma & =\gamma \hfill \mathrm{(76)}\hfill ({\Omega }_F^1,{\Omega }_F^2)& =\Phi (\Gamma ,U_{A^1},U_{A^2})(\mathrm{Joint}\text{best-response}\mathrm{equilibrium}\mathrm{map}),\hfill \mathrm{(77)}\hfill {\Omega }_L& =\Gamma ({\Omega }_F^1,{\Omega }_F^2)+U_L,\hfill \mathrm{(78)}\hfill J_F^i& =g_F^i({\Omega }_L,{\Omega }_F^1,{\Omega }_F^2)+U_{J_F^i},i\in \{1,2\},\hfill \mathrm{(79)}\hfill J_L& =g_L({\Omega }_L,{\Omega }_F^1,{\Omega }_F^2)+U_{J_L}.\hfill \end{array}$$

The direct causal relations from arrows $\Gamma \to ({\Omega }_F^1,{\Omega }_F^2)$ are via the joint best-response equilibrium map $\Phi$; as shown in the structural Equation (76). One can write ${\Omega }_F^1={\mathrm{BR}}_1({\Gamma }_1,U_{A^1})$, ${\Omega }_F^2={\mathrm{BR}}_2({\Gamma }_2,U_{A^2})$, and define $\Phi$ as the fixed-point solver. That is, each follower i has payoff $J_F^i(a^i,a^{-i};\gamma )$ with the best-response correspondence:

$${\mathrm{BR}}_i(\gamma ,a^{-i})\in \arg \underset{a^i\in {\mathcal{A}}_i}{\max }{u}_i(a^i,a^{-i};\gamma ),i\in \{1,2\},$$

and the joint best-response correspondence is

$${\mathcal{B}}_{\gamma }(a^1,a^2)={\mathrm{BR}}_1(\gamma ,a^2)\times {\mathrm{BR}}_2(\gamma ,a^1)\subseteq {\mathcal{A}}_1\times {\mathcal{A}}_2,$$

for compact action sets ${\mathcal{A}}_i\subset {\mathbb{R}}^{d_i}$. So, ${\mathcal{B}}_{\gamma }(a^1,a^2)$ takes an action profile $(a^1,a^2)$ as input and returns the set of all possible joint actions where each player is best responding to the input action of the other player. A Nash equilibrium under $\gamma$ is then any fixed point $(a^{1★},a^{2★})\in {\mathcal{B}}_{\gamma }(a^{1★},a^{2★})$, as $a^{1★}\in {\mathrm{BR}}_1(\gamma ,a^{2★})$, i.e., follower 1’s action is a best response to follower 2’s action; and $a^{2★}\in {\mathrm{BR}}_1(\gamma ,a^{1★})$, i.e., follower 2’s action is a best response to follower 1’s action. However, in scenarios where the followers game presents multiple Nash Equilibria (NE), the conventional method of defining NE as a fixed point fails to provide clarity on the actual outcome that players will select, resulting in a model that lacks predictive capability. Therefore, it is important to consider some additional assumptions for this joint best-response equilibrium map $\Phi$, such as the existence of the equilibrium and the specification of an equilibrium selection rule, where the goal is to move from the Nash equilibrium correspondence (a set of possible outcomes) to a unique joint action law (a single predicted outcome).

Under ${\mathcal{M}}_{L_1{F}_1^2}$ the principal wants to estimate the causal estimand $V\left(\gamma \right)=\mathbb{E}\left[J_L\left|\mathrm{do}\right(\Gamma =\gamma \right]$ and targets the incentive function ${\gamma }^{*}$ such that:

$${\gamma }^{*}\in \underset{\gamma \in \Gamma }{\arg \max }\mathbb{E}\left[J_L\mid do\left(\Gamma =\gamma \right)\right].$$

(80)

By the g-formula, the identification for $V\left(\gamma \right)$ also admits the nested expectation form:

$$\begin{array}{cc}\hfill V\left(\gamma \right)& =\int \int \underset{\mathrm{inner}\mathrm{expectation}(\mathrm{Gaussian}\mathrm{smoothing})}{\underbrace{{\mathbb{E}}_{U_L}\left[g_L\left(\gamma (a^1,a^2)+U_L,a^1,a^2\right)\right]}}p(a^1,a^2\mid \Gamma =\gamma )d{a}^{1}d{a}^2.\hfill \end{array}$$

(81)

In the identification formula for $V\left(\gamma \right)$, we can observe the importance of having a joint best-response equilibrium map $\Phi$ with an equilibrium selection rule. In multiple equilibria cases, $p(a^1,a^2\mid \Gamma =\gamma )$ becomes non-unique, ambiguous, and very hard to estimate from data; so it is important to impose or learn an equilibrium selection rule that makes the joint action law unique or as unambiguous as possible.

5. Discussion

This work formalizes the Causal Incentive Design (CID) framework for the canonical single-stage principal–agent problem (SS–PAP) by treating incentives as interventions in a causal graphical model (CGM). The causal target $V\left(\gamma \right)=\mathbb{E}[J_L\mid do(\Gamma =\gamma )]$ is the expected value of the principal’s utility variable $J_L$ under a specified policy intervention $\Gamma =\gamma$ in the post-intervention distribution $p\left(J_L\mid do(\Gamma =\gamma )\right)$. The construction of an estimand by the identification of $V\left(\gamma \right)$ is via the g-formula, and the selection of a single deployment incentive policy is through a Functional Bayesian Optimization algorithm. The process pipeline on CID for SS–PAPs is explicitly offline: historical data records from the system are mapped into estimates of $V\left(\gamma \right)$, which are then optimized prior to a one-shot commitment. This connection between causal identification and sample-efficient search yields high-probability regret guarantees for the offline selection step.

A central contribution is the anatomy of the estimand $V\left(\gamma \right)$ as a nested expectation with two interpretable layers: (i) an inner Gaussian smoothing of the outcome regression around the principal’s induced action, and (ii) an outer averaging with respect to the follower’s induced action law $p(M\mid \Gamma =\gamma )$; which extends to multi-follower setting. The inner layer admits quadrature-based evaluation (e.g., Gauss–Hermite), while the outer layer is handled by a policy-local reweighting scheme that respects positivity and interpretability. In the affine credit-market specialization (Section 3.3.1), this decomposition collapses to a closed form, clarifying how pricing slope and fees interact with induced borrowing behavior.

The CID framework demonstrate a recent advance in causal reasoning grounded in causal graphical models (CGMs): policies are formalized as functional interventions on a policy node, policy value is represented as an interventional or counterfactual query, and identification and estimation are performed directly on the CGM using observational logs. Additionally, the single-stage principal–agent model examined here introduces a novel application domain for CGMs in incentive design, complementing established applications in biology, medicine, and economics, and facilitating further exploration of multi-agent scenarios.

The remainder of the Discussion is structured as follows. Section 5.1 revisits the Stackelberg FCBO procedure and interprets the information–gain regret bounds as causal decision guarantees for one-shot policy selection in single-stage PAPs. Section 5.2 reviews the primary modeling, identification, and learning assumptions of the CID framework, including the CGM structure, positivity/overlap, the additive-noise model, and the uniform sub-Gaussian noise envelope used in the regret analysis. Section 5.3 discusses the practical scope of the current single-stage, offline analysis, emphasizing limitations related to policy-local support, potential model misspecification such as heavy-tailed noise or unmodeled confounding, and robustness considerations, as well as the importance of effective-sample-size diagnostics. Section 5.4 analyzes the computational demands of the CID pipeline and the FCBO algorithm, specifying how time and memory requirements scale with the number of logged units, the number of policy evaluations, and GP updates, and indicating regimes in which value estimation or surrogate modeling is the dominant computational cost. Section 5.5 describes extensions of the CID framework and nested estimand beyond the single-follower case, considering individualized versus joint incentives and the impact of interactions among multiple followers on the CGM, identification, and the design of learning algorithms.

5.1. Offline Functional BO and Causal Decision Guarantees

The Stackelberg FCBO algorithm models $\gamma \mapsto V\left(\gamma \right)$ as a black-box functional objective on a Reproducing Kernel Hilbert Space (RKHS) and uses a functional GP surrogate with a GP-UCB-style acquisition functional. The resulting cumulative-regret bounds for the Stackelberg FCBO algorithm scale with $\sqrt{T{\beta }^t{\mathcal{I}}_T}$, where ${\mathcal{I}}_T$ is an information gain term. This offers concrete guidance on evaluation budgets (sampling costs), stopping rules (such as UCB gaps), and the exploration schedule prior to the one-shot deployment; i.e., determining when enough data have been collected to make a final decision.

In contrast to online-learning methods such as bandits and reinforcement learning over actions, CID for SS–PAPs is distinctly one-shot and offline, utilizing intrinsic causal structure in PAPs and causal reasoning to assess policies without the need for deployment. In relation to causal BO methods, we argue that interventions $do(\Gamma =\gamma )$ in the function-valued random variable $\Gamma$, which involve functional (conditional) interventions $do\left({\Omega }_L=\gamma \left({\omega }_L\right)\right)$, since

$$\mathbb{E}\left[J_L\mid do(\Gamma =\gamma )\right]\approx \mathbb{E}\left[J_L\mid do(\Gamma =\gamma ),do\left({\Omega }_L=\gamma \left({\omega }_L\right)\right)\right],$$

are the right abstraction for causal reasoning about incentive functions in PAPs. In ${\mathcal{M}}_{L_1{F}_1}$, the node $\Gamma$ explicitly represents the incentive policy and is connected to both ${\Omega }_L$ and ${\Omega }_F$. These connections $\Gamma \to {\Omega }_L$ and $\Gamma \to {\Omega }_F$, together with ${\Omega }_F\to {\Omega }_L$, ${\Omega }_L\to J_L$, and ${\Omega }_F\to J_L$, constitute the minimal policy-aware motif that captures how incentives $\gamma \in \Gamma$ shape the leader’s realized decision ${\Omega }_L$ and the follower’s decision ${\Omega }_F$, to produce the principal’s utility variable $J_L$ in the system. In structural terms, the leader’s action is generated by the policy applied to the follower context, ${\Omega }_L=\gamma \left({\Omega }_F\right)$, while the adoption of a policy also reshapes the distribution of follower variables through $\Gamma \to {\Omega }_F$.

This inverse Stackelberg-aware approach makes the hierarchical semantics explicit: the dependence between principal and follower decisions is encoded both by ${\Omega }_F\to {\Omega }_L$ (the policy takes follower context as input) and through behavior modifications driven by incentive policies $\Gamma \to {\Omega }_L$ and $\Gamma \to {\Omega }_F$ (the policy governs how actions are produced and how follower behavior is distributed under intervention). Moreover, the follower decision ${\Omega }_F$ acts as a mediator M in the causal path from $\Gamma$ to $J_L$ in the principal’s perspective of ${\mathcal{M}}_{L_1{F}_1}$, in agreement with the intrinsic semantics of PAPs.

The CID framework for SS–PAPs illustrates the fundamental difference between considering the causal relations between the input variables ($\Gamma$, ${\Omega }_F$ in SS–PAPs ) and not considering them in an optimization problem. This variation is the central motivation in the causal decision-making framework of Causal Bayesian Optimization (CBO) [24] over classical Bayesian optimization. In this sense, the CID framework for SS–PAPs exhibits PAPs, inverse Stackelberg games, and bilevel optimization problems, as a fundamental family of problems that elucidates this important distinctness (between BO and CBO), given that the hierarchical semantics of an inverse Stackelberg game rely on the causal relationships $\Gamma \to {\Omega }_L$, $\Gamma \to {\Omega }_F$ and ${\Omega }_F\to {\Omega }_L$. Disregarding the causal relationships between these three variables $\Gamma$, ${\Omega }_F$, and ${\Omega }_L$ merely distorts the semantics of the functional optimization problem that solves a PAP.

5.2. Assumptions

The identification and estimation of CID are based and rely on the following assumptions: (i) Consistency: the recorded outcome equals the counterfactual under the realized policy. (ii) Causal sufficiency: for variables $\{\Gamma ,{\Omega }_L,{\Omega }_F,J_L,J_F\}$ in the generative CGM ${\mathcal{M}}_{L_1{F}_1}$; (iii) Black-box follower’s best-response: we work over the principal’s perspective where the observed variables are $\{\Gamma ,{\Omega }_F,{\Omega }_L,J_L\}$ and the follower’s best-response is considered a black-box function. (iv) Positivity (overlap): in a policy-local sense, we assume that there exists a neighborhood ${\mathcal{N}}_{\rho }\left(\gamma \right)$ such that $p({\Omega }_F\mid \Gamma \in {\mathcal{N}}_{\rho }\left(\gamma \right))>0$ wherever the estimator needs support to approximate $p({\Omega }_F\mid \Gamma =\gamma )$. (v) Policy compliance: The action implemented by the principal is consistent with the distribution of ${\Omega }_L=\gamma \left({\Omega }_F\right)+U_L$ in practical deployment, as it is presupposed. Otherwise, if this alignment is not achieved, the action $do(\Gamma =\gamma )$ and the behavior observed in practice will differ significantly. (vi) Additive, zero-mean Gaussian noise SCM: featuring mutually independent exogenous terms; in the leader’s action, in particular, allowing inner Gaussian smoothing on the identification formula. We also assume i.i.d. logged units and no interference across units, i.e., the Stable Unit Treatment Value Assumption (SUTVA). Violations or weak forms of these conditions, such as latent confounding, measurement error, or insufficient policy-local overlap, can bias $V\left(\gamma \right)$ and compromise the calibration of uncertainty. (vii) Noise envelope for regret analysis: While the outer estimator is heteroskedastic across $\gamma$, we assume a uniform sub-Gaussian envelope on ${\epsilon }_t=\widehat{V}\left({\gamma }_t\right)-V\left({\gamma }_t\right)$ over the proposal support. This implies a diagonal noise model in Section 4.4.1 with ${\Sigma }_T⪯{\sigma }_{*}^2\mathbf{I}$, under which the stated GP-UCB bounds hold (see [8,29]).

5.3. Scope and Limitations

The present CID analysis is single-stage and offline: it commits once to a policy and cannot adapt after deployment. Dynamics (carry-over effects), learning-by-doing, and path-dependent constraints are beyond the scope of this paper. The follower mechanism is treated as a black box (we observe ${\Omega }_F,{\Omega }_L,J_L$, but not the follower utilities); this pushes strategic dependence into the observed mediator law and increases sensitivity to support gaps in ${\Omega }_F$ near the target policy $\gamma$. So, in the absence of extensive historical policy support, the sensitivity in the outer layer increases. The additive Gaussian assumption simplifies the inner layer but may under-represent heavy tails or heteroskedasticity; when noise departs from Gaussian, it can lead to inaccuracies in inner smoothing and variance estimates.

For diagnostics and protection against weak overlap: (i) Monitor weight distributions, to look for outlier weights pointing to weak overlap and high variance. (ii) Monitor the effective sample size (ESS), which measures the effective number of independent samples you have after applying the weights.

$$\mathrm{ESS}(\gamma ;h):={\displaystyle \frac{{\left({\sum }_{i=1}^N{w}_i(\gamma ;h)\right)}^2}{{\sum }_{i=1}^N{w}_i{(\gamma ;h)}^2}}.$$

A low ESS relative to the total sample size (N) indicates that a few heavily weighted units dominate the estimation, signaling a high variance problem. (iii) Monitor sensitivity to the policy-space bandwidth, by re-running the evaluation using different values for this bandwidth parameter h. If the value of the new policy changes significantly when you slightly adjust the bandwidth, the estimate is unstable and highly dependent on a tuning parameter, which reduces confidence in the result. (iv) Reduce overfitting bias in the outer layer employing sample-splitting, cross-fitting, and orthogonalization.

In practice, Algorithm 1 can constrain candidate policies to a trust region ${\mathcal{S}}_{\mathrm{support}}$ (see Section 3.3.4 for the definition of ${\mathcal{S}}_{\mathrm{support}}$), adapt the bandwidth h to balance bias–variance and/or inflate posterior uncertainty as $\gamma$ approaches the boundary of ${\mathcal{S}}_{\mathrm{support}}$. Crucially, kernel reweighting does not create support: if $\gamma$ is far from the historical policies, ${\widehat{\mathbb{P}}}_{\gamma }^{\left(h\right)}$ collapses, and $\widehat{V}\left(\gamma \right)$ becomes high-variance (and possibly biased). Therefore, the practical search space of Algorithm 1 is data-dependent and effectively limited to neighborhoods of previously observed policies. Because the outer estimator relies on policy-local overlap, we can incorporate a policy-support score $s\left(\gamma \right):=\mathrm{ESS}(\gamma ;h)/N$ into the selection. Concretely, we can either constrain GP-UCB to $\gamma \in {\mathcal{S}}_{\mathrm{support}}(h,\tau ,w_{\max })$ (hard trust region) or use a soft penalty ${\tilde{A}}_t\left(\gamma \right)=A_t\left(\gamma \right)·\phi \left(s\left(\gamma \right)\right)$, where $A_t\left(\gamma \right)$ is the acquisition function, with $\phi \left(s\right)\downarrow 0$ as $s\downarrow 0$, i.e., this penalty function must approach zero as the policy-support score $s\left(\gamma \right)$ approaches zero. Conversely, when $s\left(\gamma \right)$ is high (close to 1), $\phi \left(s\right)$ should be close to 1 to minimally impact the acquisition function $A_t\left(\gamma \right)$. This prevents proposals in regions where $p(M\mid \Gamma =\gamma )$ cannot be estimated from logs and makes the practical search space explicit and data-dependent.

5.4. Time–Space Complexity over a Full FBO Run

Estimating $V\left(\gamma \right)$ once via the semi-parametric identification formula (assuming additive, zero-mean Gaussian noise) costs $\mathcal{O}\left(N(P+C_{\gamma }+Q_{\mathrm{eff}}{C}_{\mu })\right)$, under policy-local reweighting over N recorded tuples, and where P is the policy-space representation size, $C_{\gamma }$ is the cost to compute $\gamma \left({\Omega }_F\right)$ once, $Q_{\mathrm{eff}}$ is the inner integration budget (e.g., Gauss–Hermite nodes or MC points), and $C_{\mu }$ is the cost to evaluate the fitted outcome regression once. Across a BO horizon of T iterations (one evaluation per step), the total cost to produce these off-line evaluations is $\mathcal{O}\left(TN(P+C_{\gamma }+Q_{\mathrm{eff}}{C}_{\mu })\right)$. With local reweighting over only the top $L\ll N$ nearest policies, this becomes $\mathcal{O}\left(T(P\log N+L(P+C_{\gamma }+Q_{\mathrm{eff}}{C}_{\mu }))\right)$. The one-time training of the outcome model contributes an upfront $C_{\mathrm{train}}^{\mu }$ that is then amortized. The memory requirement for the estimator is $\mathcal{O}\left(N\right)$ (or $\mathcal{O}\left(L\right)$ when local weighting is applied), in addition to $\mathcal{O}\left(NP\right)$ if the policy embeddings are stored in cache. The quadrature nodes are $\mathcal{O}\left(Q_{\mathrm{eff}}\right)$ and can be considered negligible.

The implementation of an exact functional Gaussian Process surrogate with incremental updates throughout the execution adds $\mathcal{O}\left((1+B)T^3+(1+B)T^{2}P\right)$ for Cholesky updates and acquisition scoring of B candidates per iteration. Since, across the full horizon T, kernel updates are ${\sum }_{t=1}^T\mathcal{O}\left(N_{t}P\right)=\mathcal{O}\left(T^{2}P\right)$; Cholesky factorizations are ${\sum }_{t=1}^T\mathcal{O}\left(N_t^2\right)=\mathcal{O}\left(T^3\right)$; and total UCB functional acquisition scoring work (each needs mean and variance) ${\sum }_{t=1}^T\mathcal{O}\left(B(N_{t}P+N_t^2)\right)=\mathcal{O}\left(B(T^{2}P+T^3)\right)$, where $N_t$ is the number of observed policy evaluations at iteration t (so $N_t=t$). Merging the estimation cost and exact GP updates plus functional acquisition work cost, the total end-to-end time is

$$\mathrm{Total}\mathrm{time}\approx C_{\mathrm{train}}^{\mu }+\underset{\mathrm{For}\mathrm{all}\widehat{V}\left(\gamma \right)\mathrm{evaluations}}{\underbrace{\mathcal{O}\left(TN(P+C_{\gamma }+Q_{\mathrm{eff}}{C}_{\mu })\right)}}+\underset{\mathrm{For}\mathrm{the}\mathrm{exact}\mathrm{functional}\mathrm{GP}}{\underbrace{\mathcal{O}\left((1+B)T^3+(1+B)T^{2}P\right)}}.$$

Replace N by L and add $TP\log N$ under local weighting. Comparing the estimation cost $TN(P+C_{\gamma }+Q_{\mathrm{eff}}{C}_{\mu })$ of $V\left(\gamma \right)$ with the dominant cubic GP term $(1+B)T^3$, we can establish that GP dominates when

$$T\gg \sqrt{N(P+C_{\gamma }+Q_{\mathrm{eff}}{C}_{\mu })/(1+B)},$$

while the estimation cost of $V\left(\gamma \right)$ dominates below that scale. In terms of space, the exact GP stores dense Gram/Cholesky factors of size $T\times T$, i.e., $\mathcal{O}\left(T^2\right)$.

5.5. Toward Multi-Follower Settings

We outline a multi-follower generalization of CID that preserves the CGM semantics of the canonical case. Let followers be indexed by $i\in \left[n\right]$ with contexts ${\Omega }_F^{\left(i\right)}$, and let the leader’s (possibly vector-valued) action be ${\Omega }_L$. The policy node $\Gamma$ represents the incentive rule(s) chosen by the principal and, in the policy-augmented CGM, carries arrows $\Gamma \to {\Omega }_L$ and $\Gamma \to {\Omega }_F^{\left(i\right)}$ for all i. As in the single-follower case, contexts feed into the leader’s realized action via ${\Omega }_F^{\left(i\right)}\to {\Omega }_L$, but now the post-intervention law involves the joint mediator $M=({\Omega }_F^{\left(1\right)},\dots ,{\Omega }_F^{\left(n\right)})$, since both ${\Omega }_L$ and $\left\{{\Omega }_F^{\left(i\right)}\right\}$ affect the principal’s payoff $J_L$. Two instructive extensions clarify how identification and learning must adapt beyond the canonical case: (i) the regime IIIU: individualized incentives with conditionally independent follower utilities (factorized mediator law), and (ii) the regime UIJU: universal incentives with joint follower utilities (coupled actions via a joint best-response map).

In the IIIU regime, the principal deploys a collection of incentive functions (e.g., one per channel, segment, or product), and the followers’ utilities are conditionally independent given their own contexts and the relevant part of the policy. Graphically, the mediator law factorizes under interventions as follows:

$$p\left(M|do\left(\{{\Gamma }^{\left(i\right)}={\gamma }^{\left(i\right)}\}\right)\right)=\prod _{i=1}^{n}p\left({\Omega }_F^{\left(i\right)}|do({\Gamma }^{\left(i\right)}={\gamma }^{\left(i\right)})\right).$$

(82)

Estimation and optimization inherit a separable structure: outer-layer reweighting and inner smoothing decompose over i, enabling parallel computation and straightforward uncertainty aggregation. Surrogate modeling should mirror this factorization (e.g., with parallel GPs, modeling the functional objective as a sum or collection of independent functional GPs or using block-diagonal kernels over policy components, where the overall kernel is constructed by combining separate, independent kernels for each component), and identifiability reduces to policy-local overlap for each channel. This setting suits applications where incentives are personalized or targeted and cross-follower externalities are negligible.

In the UIJU regime, a single policy $\gamma$ applies to all followers whose actions are strategically coupled; so, we use a joint best-response map $A=\Phi \left(\gamma ,M,U_F\right)$, with exogenous terms $U_F$, where we use A to denote the actions carried out by followers $\{{\omega }_F^{\left(1\right)},\dots ,{\omega }_F^{\left(n\right)}\}$. The identification requires an equilibrium-selection rule to make the post-intervention law; i.e., $p\left(A|M,do(\Gamma =\gamma )\right)$ is well defined through a selection mapping S: $(\gamma ,M)\mapsto A$. Three practical approaches are (i) Uniqueness: Assume that $do(\Gamma =\gamma )$ almost surely results in a unique equilibrium outcome A. (ii) Parametric selection: Encode tie-breaking or stability criteria in S and learn its parameters from observed play. (iii) Stochastic selection: Models the choice among equilibria as a stochastic process, where each equilibrium is chosen with a specific probability.

Surrogate modeling for the UIJU regime should reflect cross-follower dependence (e.g., using multi-task GPs with coregionalization and structured kernels over policy components), and outer-layer estimation must use joint reweighting over M (and, if modeled, A) rather than product weights. Positivity must hold in a joint sense: support near $\gamma$ is needed for the relevant regions of the $(M,A)$-space, a stricter requirement than in the individualized case.

6. Conclusions

This study introduces a principled, offline framework for single-stage incentive design using causal inference, grounded in causal graphical models (CGMs) and information-theoretic analysis. The single-stage principal–agent problem (SS–PAP) is formalized as a CGM, where the principal’s incentive rule is modeled as a functional intervention on a policy node $\Gamma$, the follower’s action serves as an explicit causal mediator, and the principal’s payoff $J_L$ is the outcome. The causal target estimand is defined as the principal’s expected utility under an incentive policy intervention, $V\left(\gamma \right)=\mathbb{E}[J_L\mid do(\Gamma =\gamma )]$, which quantifies how a committed incentive causally influences the follower’s behavior and the principal’s payoff under private information.

For the identification of the estimand, a semi-parametric expression for $V\left(\gamma \right)$ is derived using the g-formula, which decomposes into an inner Gaussian smoothing over noise and an outer expectation with respect to the post-intervention distribution of the mediator. Building on this structure, a two-layer estimator is proposed for offline logs: the inner layer computes a Gaussian expectation, implemented numerically via Gauss–Hermite quadrature, while the outer layer approximates the interventional mediator law using policy-local kernel reweighting with effective-sample-size and weight-cap diagnostics. This design explicitly addresses the bias–variance trade-off and ensures that off-policy evaluation maintains overlap in policy space.

For policy selection, the value functional $\gamma \mapsto V\left(\gamma \right)$ is embedded in a functional Gaussian process surrogate over a Reproducing Kernel Hilbert Space (RKHS) of admissible incentives, and a support-aware GP-UCB search is performed. The resulting regret bounds are characterized in terms of differential information gain, directly connecting the exploration–exploitation trade-off to information-theoretic quantities under a uniform sub-Gaussian envelope for estimator noise. Collectively, these components establish a unified pipeline for causal reasoning and optimal intervention design in single-stage principal–agent settings using only offline observational logs.

The proposed offline approach is particularly suitable in scenarios where adaptive experimentation is infeasible, ethically constrained, or operationally costly. For instance, this applies when contracts, tariffs, or credit terms must be fixed prior to deployment and cannot be iteratively adjusted. Within this regime, the analysis clarifies the conditions under which causal identification is possible, the extent to which extrapolation in policy space is feasible while maintaining overlap, and methods for quantifying uncertainty through effective sample size and information gain.

The CID framework extends beyond the canonical single-follower case. In a multi-follower generalization, the nested form of the estimand is preserved, but a strong dependence emerges based on the interaction of follower utilities. In regimes with individualized incentives and conditionally independent utilities, identification, estimation, and surrogate modeling decompose across followers, enabling parallelization and simpler diagnostics. Conversely, in regimes with universal incentives and jointly coupled utilities, identification depends on equilibrium-selection assumptions, and outer-layer estimation must address joint mediator laws and stricter positivity requirements. These extensions demonstrate both the robustness of the two-layer CID structure and the additional challenges encountered in more complex multi-agent systems.

Overall, the presented CID framework advances the field of causal graphical models and their applications by integrating a CGM-based representation of principal–agent interactions with explicit causal reasoning about policy interventions and information-theoretic guarantees for offline policy search. This framework establishes a foundational link between causal inference and policy search in principal–agent problems, supporting reliable single-stage incentive design in complex systems and providing a basis for future research involving empirical studies, richer policy classes, and more complex multi-follower environments.

Future Work

The proposed approach enables the integration of causal reasoning into incentive design, especially for canonical single-stage PAPs. It also reveals opportunities to strengthen robustness in causal inference estimation, improve computational scalability, validate the framework under realistic constraints, and expand to multi-follower and multi-stage PAPs. Future work will address these limitations and explore extensions to make CID dependable at scale and deployable in practice.

Beyond the scope of this work, an important future extension is a robust identification toolkit that tolerates limited policy-local overlap in data logs and mild unmeasured confounding. Specifically, we plan to incorporate partial-identification estimation bounds, i.e., reporting intervals for the causal target estimand $V\left(\gamma \right)$ rather than a single point estimation, with techniques based on [30] tailored to policy interventions, or proximal/negative-control strategies when credible proxy variables are available. In addition to targeted sensitivity analyses using weight clipping and bandwidth sweeps with effective sample size diagnostics. By addressing this limitation, the practical utility of CID improves in data regimes where perfect overlap and strict sufficiency are unrealistic.

Other aspects for strengthening robustness involve exploring beyond additive-Gaussian noise and augmenting policy-local reweighting with structural conditional density models to learn the mediator law given an incentive policy $p(M\mid \Gamma )$. Regarding the first, we plan to replace the inner Gaussian smoothing with robust or heavy-tailed quadrature methods, such as Student-t distributions or mixture models, which can better account for outliers and extreme data points. In addition, to explore heteroskedastic noise models and variance-adaptive smoothing, to propagate these refinements into uncertainty quantification. Regarding conditional density models for $p(M\mid \Gamma )$, the plan is to utilize conditional normalizing flows or score-based estimators while preserving the identification logic and benchmarking bias–variance trade-offs. These improvements are essential to reduce bias in heavy-tailed regimes and to maintain calibrated uncertainty when noise scale depends on context.

The forthcoming work will involve the complete extension of CID to multi-follower over the distinct configurations. This will require adaptations to surrogate modeling tailored to specific regimes and adjusting the regret bounds. Many systems are situated between regimes: individualized incentives with conditionally independent follower utilities and universal incentives with joint follower utilities. A pragmatic approach is to adopt clustered dependence: partition followers into groups with strong in-group coupling and weak cross-group effects. The CGM should then mix block-factorized mediator laws with group-level equilibrium selection. For a large amount of followers, exchangeability and mean-field approximations can effectively reduce dimensionality. These techniques summarize follower interactions through aggregate statistics (such as empirical means or occupancy measures), which can be used as input to the system’s laws and particularly on the joint best-response map, yielding scalable identification and surrogate models with kernels over aggregates. This implies measuring similarity between two policies based on the similarity of their predicted aggregate statistics, making the optimization process efficient for massive populations.

Beyond the existing framework, a promising direction is constraint-aware CID. The core motivation is that incentive policies deployed in real-world settings (such as credit, pricing, or subsidies) must reliably satisfy constraints, and not just on average. This means mitigating the risk of rare and catastrophic failures. Essential constraints include safety (avoiding dangerous or unintended side effects), budget (ensuring costs remain within defined limits), and fairness (ensuring similar treatment or impact across groups, minimizing disparities in approval rates or resource allocation). The planned efforts on this matter involve implementing three interconnected components in the search process for policy interventions: (i) constrained acquisitions that enforce high-probability feasibility; (ii) risk-sensitive objectives and chance constraints; and (iii) fairness controls. The analysis of this extension must delineate clear Pareto trade-offs, enabling decision-makers to transparently assess trade-offs between maximizing utility and satisfying constraints such as safety, fairness, and budget.

A paramount direction is to generalize from the single-stage setting to multi-stage environments, where incentives are applied over time. Specifically, we plan to formalize a dynamic CGM with history-dependent policies $\Pi$, identify the causal target $V(\Pi )$ with a sequential (stage-wise) analogue of our two-layer estimator, and lift the optimization to a Dynamic Functional Causal Bayesian Optimization algorithm that accounts for temporal causal relations and stage-wise uncertainty. This expansion is essential to capture learning-by-doing, carryover effects, and path-dependent constraints—features central to many real-world deployments.

Finally, translating these theoretical gains into robust real-world value is a key long-term goal. This involves curated benchmarks (credit pricing, platform subsidies, and market mechanisms), rigorous baselines (contextual bandits with off-policy evaluation (OPE), policy-gradient RL with OPE, and non-causal BO), and comprehensive reporting of offline policy regret, uncertainty coverage, and constraint satisfaction—with open, reproducible artifacts. Future work will include detailed simulations and case studies to demonstrate (a) calibration of the two-layer estimator under policy dispersion and overlap constraints; (b) separation of outer policy-local reweighting and inner Gaussian smoothing through ablation studies; and (c) performance of end-to-end Functional Causal Bayesian Optimization (FCBO) with support-aware acquisition across representative policy classes.

Additionally, it is essential to monitor run-time and memory usage as policy classes and evaluation budgets expand. The primary bottleneck in Bayesian Optimization (BO) is the Gaussian Process (GP) surrogate model, which often has cubic time complexity over the BO horizon. This can be overcome by replacing costly exact computations with approximations, such as random-feature surrogates chosen in the policy space, and by coordinating multi-fidelity evaluations of $V\left(\gamma \right)$. That is, using “cheap samples” that can be sampled quickly at a low computational cost in the inner layer of the identification formula to gain a coarse understanding of the policy’s value, and reserve the expensive, high-accuracy estimations (like high-fidelity quadrature for integrating probabilities) for only the most promising policies. This is essential for validating scalability in resource-constrained environments and for enabling larger, more expressive policy classes without sacrificing regret guarantees.

Taken together, these directions aim not only to refine the current methodology but also to broaden CID’s theoretical and practical footprint. Specifically, the field must transition from single-stage to multi-stage decision-making; move from brittle point estimates to robust identification; progress from exact, heavy surrogates to scalable multi-fidelity optimization; expand from isolated agents to coupled, constraint-laden systems; and shift from stylized demonstrations to reproducible, high-stakes applications. Advancing along these directions will facilitate the development of reliable, deployable incentive designs grounded in causal reasoning.