Dynamic Multiagent Incentive Contracts: Existence, Uniqueness, and Implementation

Abstract

Multiagent incentive contracts are advanced techniques for solving decentralized decision-making problems with asymmetric information. The principal designs contracts aiming to incentivize non-cooperating agents to act in his or her interest. Due to the asymmetric information, the principal must balance the efficiency loss and the security for keeping the agents. We prove both the existence conditions for optimality and the uniqueness conditions for computational tractability. The coupled principal-agent problems are converted to solving a Hamilton–Jacobi–Bellman equation with equilibrium constraints. Extending the incentive contract to a multiagent setting with history-dependent terminal conditions opens the door to new applications in corporate finance, institutional design, and operations research.

1. Introduction

In this paper, we consider the problem of a single party, called the principal, creating contracts to delegate a task to a group of different agents. Incentive contracts stimulate the agents to act in the principal’s interest by compensating them for achieving two goals: (i) they accept the offered contract (i.e., the contract is subject to the individual rational (IR) constraint); and (ii) they exert the effort at a desired level determined by the compensation spelled out in the contract (i.e., the contract is subject to the incentive compatible (IC) constraint). Such incentive contracts have been used for many practical problems ranging from corporate finance to strategic behavior in politics to institutional design [1,2,3,4,5,6,7,8,9,10].

In a dynamic setting, the goal as before is to incentivize agents to exert the desired effort over the planning horizon. To achieve this, each contract defines a stream of payoff amounts, which depend on the effort exerted by the corresponding agent. In the framework we consider in this paper, the agent’s effort process is not perfectly observable, possibly due to the cost or the difficulty of monitoring it. Instead, the principal observes a noisy output process, which is a result of the effort exerted by the agent. This proxy results in information asymmetry about the agent’s effort (the agent knows it, but the principal can only infer it from a proxy). The asymmetric information can create a potential moral hazard problem in the contract design [11]. The system efficiency is degraded as the first-best contract is not admissible. Given all these considerations, the incentive contract must solve the moral hazard problem and maximize the principal’s utility.

This fundamental incentive contract problem in the case of a single agent has been explored in many settings [7,12,13,14,15]. We consider the problem as a special case of stochastic Stackelberg differential games played between a principal and an agent. The principal expects the agent to exert a targeted level of effort and knows ex ante that once the agent has accepted the contract, it has no incentive to deviate from this target level (thus bypassing any resulting moral hazard). This incentive-compatible condition can be satisfied if the agents’ actions form a subgame perfect Nash equilibrium. However, finding such a globally optimal contract over the planning horizon is not trivial. The dynamic moral hazard problem has been studied in a discrete-time setting, where the state space explodes exponentially in the size of the planning horizon (the curse of dimensionality in dynamic programming) [16]. Holmstrom and Milgrom [17] proposed a continuous-time model. In this setting, the agent’s output process is represented by a stochastic differential equation (SDE) whose drift term is controlled by the agent’s effort. As a result, the continuous-time incentive contract problem is a limit of discrete-time dynamic games whose number of stages becomes unbounded in any finite interval. Some extensions of their work include Schattler and Sung [18], Sung [19], and Muller [20]. In recent years, following the groundbreaking work of Sannikov [21], there has been a resurgence of interest in the dynamic contract theory. The main contribution of [21] was to parameterize the incentive-compatible constraint at each epoch using the Martingale representation theorem. As a consequence, we can decouple the principal’s and the agent’s problems by representing the agent’s effort as a function of a parameter. The principal’s problem can then be solved by dynamic programming (more specifically, a Hamiltonian Jacobi–Bellman equation) for the incentive contract [3,15,21,22].

A significant extension of the single-agent incentive contract is multiagent incentive contracts. For example, a company hires multiple employees to collaborate on a project. Since employees with correlated responses may have different capabilities and utility functions, designing contracts separately for each is not viable. Koo et al. [23] presented the first extension of multiagent incentive contracts that initiated a stream of literature for team incentives using the Martingale approach [24,25,26,27]. In the multiagent setting, new challenges arise due to varied interactions between agents. For example, an arbitrary agent may compare both its effort and payoff with others; such a phenomenon is called inequity aversion [28]. Goukasian and Wan showed that inequity aversion is present in multiagent incentive contracts [29], and agents’ comparisons lower their exerted effort levels.

The critical condition for the existence of effective multiagent incentive contracts is that agents’ actions at each epoch must form a Nash equilibrium. This equilibrium then incentivizes each agent to choose the principal’s desired actions and nullifies the moral hazard in the contract. The conditions for the existence of this equilibrium is still an open question. Prior work [27] assumed that the existence conditions are satisfied in their setting without verification. The agents’ optimal actions constituting a Nash equilibrium led to a circular argument. Yet, characterizing the existence of a Nash equilibrium in multiagent contracts is non-trivial [30,31,32], more so in the dynamic setting considered in this work. The following example demonstrates the importance of investigating the existing conditions in a static matrix game setting. A principal chooses to compensate $c_i$ to two agents as either low (L) or high (H) payoff, i.e., $c_i\in \{L,H\}$ for $i\in \{1,2\}$. Agents putting effort into a project generate output denoted as $X_i\in \{A,B\}$ at levels A or B.

• The principal desires to stimulate Agent 1 to exert output A and Agent 2 to exert output B. The outcomes of signing contracts are represented by the matrices in

  Table 1. Static incentive contracts with two agents.

  	Agent 1		Agent 2
  	$\mathit{A}$	$\mathit{B}$	$\mathit{A}$	$\mathit{B}$
  L	4,2	2,1	2,1	4,2
  H	3,3	1,2	1,2	3,3

  , where each entry is the principal’s and agent’s utility received from the contract. If these two contracts are signed separately, the unique equilibria are $\{L,A\}$ with Agent 1 and $\{L,B\}$ with Agent 2.
• We now assume that two agents’ outputs are aggregated in a linearly additive way. In this case, the principal’s dominant policy is $[c_1,c_2]=[L,L]$. Notice that the existence and the number of equilibria may vary with the agents’ utility functions $u_i(c_i,X_i,X_{-i})$. Three possible outcomes for the contracts are below:
  • Unique Nash equilibrium: Assume that the utility of each agent is only dependent on its payoff, i.e., $u_i(c_i,X_i,X_{-i})=c_i$. The agents’ best responses are $[X_1,X_2]=[A,B]$. With a fixed $[c_1,c_2]=[L,L]$, their utility follows

    Table 2. Agents’ output with a single Nash equilibrium.

    	A	B
    A	2,1	2,2
    B	1,1	1,2

    .
  • Multiple Nash equilibria: Assuming that the principal rewards whoever delivers B an additional unit of compensation, there exists two Nash equilibria, $[X_1,X_2]=[A,B]$ and $[X_1,X_2]=[B,B]$, for which their utility follows

    Table 3. Agents’ output with multiple Nash equilibria.

    	A	B
    A	2,1	2,3
    B	2,1	2,3

    .
  • No Nash equilibrium: Assuming that the utility of each agent is affected by the other’s action such that the principal would reward the agents when their outputs match, i.e., $u_i(c_i,X_i,X_{-i})=c_i+2$ if $X_i=X_{-i}$, then there is no Nash equilibrium, as seen in

    Table 4. Agents’ output with no Nash equilibrium.

    	A	B
    A	4,3	2,2
    B	1,1	3,4

    .

Our goal in this paper is to find conditions that guarantee the existence of a unique multiagent Nash equilibrium in incentive contracts. We see that even if the existence problem is settled [27], the uniqueness of the multiagent Nash equilibrium must still be tackled. Characterizing unique equilibrium has practical value as coordinating agents to select the optimal Nash equilibrium is improbable; it also has theoretical value as the optimal contracts with a set of equilibria is computationally intractable. Using a fixed-point theorem (specifically, the Kakutani fixed-point theorem), we prove the existence of a subgame perfect Nash equilibrium. The existence conditions include the assumption that all agents are risk-averse and the interactions of all agents’ actions on other’s output follow a concave function. With a slight strengthening of the condition on the Hessian matrix of the interaction functions and with the use of the theorem of Gale and Nikaido [33] and Kojima and Saigal [34], we prove that the equilibrium is unique. These results then enable us to develop a provably convergent iterative procedure to solve for the incentive contracts.

Unlike the infinite horizon setting of [21], we consider the problem with a finite horizon where the terminal condition may be path-dependent. Such terminal conditions are widely used in modeling options, mortgage defaults, and car leasing, thus enhancing the applicability of the methodology.

The general notation used in the rest of this paper is as follows. A set of indices $\left[n\right]=\{1,2,\cdots n\}$. Bold variables are vectors or matrices of random variables or functions. In equilibrium analysis for the $i^{th}$ agent, we denote a vector as $\mathit{x}=[x_1,\cdots ,x_i,\cdots ,x_n]=[{\mathit{x}}_{-i},x_i]$, where $x_i$ indicates the variable associated with the $i^{th}$ agent. $x_P$ indicates that the variable is associated with the principal. $\tilde{x}$ is a variable that deviates from x in the domain of x. $D_{x}F$ is the Jacobian, and $D_x^{2}F$ is the Hessian of the $C^2$ function F of x.

The remainder of this paper is organized as follows. In Section 2, we describe the setting of multiagent incentive contracts. In Section 3, we characterize the agents’ optimal responses and prove the existence of a unique Nash equilibrium. We then formulate the principal’s problem as a Hamilton–Jacobi–Bellman equation. We also give an iterative procedure to implement the optimal incentive contracts. In Section 5, we draw the final conclusion.

2. Setting

There is a single principal and n agents (indexed by $i\in \left[n\right]$) entering the contracts simultaneously at epoch $t=0$. A contract signed between the principal and each agent i specifies the payoff $c_i\left(t\right)$ that the agent will receive by outputting $X_i\left(t\right)$, a proxy for the the agent’s action $a_i\left(t\right)$ in working for the principal over the horizon $t\in [0,T]$. The vectors of n agents’ actions and compensations are denoted as $\mathit{a}\left(t\right)$ and $\mathit{c}\left(t\right)$, respectively. Since the principal’s goal is to incentivize n agents to collaborate on one project, these n contracts are correlated in many ways. The principal’s decision, the payoff $c_i\left(t\right)$ for agent i, is in a domain ${\mathcal{C}}_i\subseteq \mathbb{R}$; the agent i’s decision, the effort level $a_i\left(t\right)$, is in a domain ${\mathcal{A}}_i\subseteq \mathbb{R}$. The size of the domains may vary for each $i\in \left[n\right]$. The Cartesian products of compensations and efforts are denoted as $\mathcal{C}$ and $\mathcal{A}$, respectively.

2.1. Output Processes and Terminal Conditions

In an environment of uncertainty, the principal can only observe output processes $\mathit{X}\left(t\right)={[X_1\left(t\right),\cdots ,X_n\left(t\right)]}^T\in \mathcal{X}$, which are imperfect observations of agents’ actions. We assume that the dynamics of $X_i\left(t\right)$ follow an SDE that depends on n agents’ actions $\mathit{a}\left(t\right)$:

$$d{X}_i\left(t\right)=f_i\left(\mathit{a}\left(t\right)\right)dt+{\sigma }_{i}d{B}_i\left(t\right),\forall i\in \left[n\right],$$

(1)

which follows the following assumptions that are a general extension of the multiagent contract in [23,24,27,35].

• The drift term $f_i:\mathcal{A}\to {\mathbb{R}}_{+}$ in (1) is in an $L^2$ space such that ${\int }_0^T{f}_i^{2}ds<\infty$ for all $i\in \left[n\right]$.
• $f_i$ is partially differentiable almost everywhere with respect to $a_i\left(t\right)$ for all $i\in \left[n\right]$.
• The diffusion term ${\sigma }_i$ is a known constant for all $i\in \left[n\right]$.
• The Brownian motions $\mathit{B}\left(t\right)={[B_1\left(t\right),\cdots ,B_n\left(t\right)]}^T$ are correlated with the correlation matrix $E\left(\mathit{B}\left(t\right)\mathit{B}{\left(t\right)}^T\right)=\mathsf{\Sigma }$, strongly positive definite, i.e., ${\mathit{x}}^T\mathsf{\Sigma }\mathit{x}\ge \mathit{\alpha }{\parallel \mathit{x}\parallel }^2$ for all $\mathit{x}\in R^n$ and some constant $\mathit{\alpha }>0$.

For each agent $i\in \left[n\right]$, there is a path-dependent terminal payoff ${\mathsf{\Phi }}_i$ at the end of planning horizon $T<\infty$. In other words, $\mathsf{\Phi }$ is a vector of functions of ${\{\mathit{X}\left(t\right),\mathit{c}\left(t\right)\}}_{0\le t\le T}$. Path-dependent terminal conditions strengthen the commitments in contracts. Each agent could be charged a penalty if its cumulative outputs do not reach a specified target at termination. Similarly, the principal may rectify the payoff if the cumulative compensations do not reach a certain threshold. Let $\mathit{Z}\left(t\right)$ denote the cumulative measures along the sample paths whose dynamics $d{Z}_i$ for $i\in \left[n\right]$ follows:

$$d{Z}_i\left(t\right)={\mu }_{Z_i}(\mathit{X}\left(t\right),\mathit{c}\left(t\right))dt+{\sigma }_{Z_i}d{B}_{Z_i}\left(t\right),$$

(2)

where ${\mu }_{Z_i},{\mathit{\sigma }}_{Z_i}$ are deterministic functions of appropriate dimension and $B_{Z_i}$ are independent Brownian motions. Two sets of processes $\mathit{B}\left(t\right)$ and ${\mathit{B}}_Z\left(t\right)$ (${\mathit{B}}_Z\left(t\right)={(B_{Z_1}\left(t\right),\cdots ,B_{Z_n}\left(t\right))}^T$) are also independent.

An example of a path-dependent terminal condition is an Asian-options type, i.e., $\mathit{Z}\left(t\right)\in {\mathbb{R}}^n$ represents the total observed output from n-agents from zero to t:

$$\begin{array}{cc}\hfill \mathit{Z}\left(t\right)& ={\int }_0^t\mathit{X}\left(s\right)ds,\hfill \end{array}$$

(3)

and this can be derived from (2) by letting ${\mu }_{Z_i}(\mathit{X},\mathit{c})=X_i$ and ${\mathit{\sigma }}_{Z_i}=0$.

The two systems of SDEs, (1) and (2), are adapted to the filtration generated by the Brownian motions $B_i$ and $B_{Z_i}$ for all $i\in \left[n\right]$. It is a well-known result that the vector $\left(\mathit{X}\right(t),\mathit{Z}(t\left)\right)$ is a Markov process.

2.2. Solving Optimal Contracts

$u_i:\mathcal{A}\times {\mathcal{C}}_i\to \mathbb{R}$ is the $i^{th}$ agent’s instantaneous utility, i.e., utility in $[t,t+dt)$ and $u_P:\mathcal{X}\times \mathcal{C}\to \mathbb{R}$ is the principal’s instantaneous utility. Note that $u_i$ is possibly a function of all agents’ actions.

The principal’s and the agents’ goals are to maximize the respective expected total discounted utility over the finite horizon $[0,T]$. We denote the $i^{th}$ agent’s expected total discounted utility by $U_i$ and the principal’s expected total discounted utility from contracting with n agents by $U_P$ as follows:

$$\begin{array}{cc}\hfill U_i& ={\mathbf{E}}^{\mathit{a}}\left[r_i{\int }_0^T{e}^{-r_{i}s}{u}_i(\mathit{a}\left(s\right),c_i\left(s\right))ds+r_i{e}^{-r_{i}T}{\mathsf{\Phi }}_i\left(Z_i\left(T\right)\right)\right],\forall i\in \left[n\right],\hfill \\ \hfill U_P& ={\mathbf{E}}^{\mathit{a}}\left[r_P{\int }_0^T{e}^{-r_{P}s}{u}_P(\mathit{X}\left(s\right),\mathit{c}\left(s\right))ds-r_P{\mathbf{1}}^{⊺}·e^{-r_{P}T}\mathsf{\Phi }\left(\mathit{Z}\left(T\right)\right)\right],\hfill \end{array}$$

where $r_i\in (0,1)$ and $r_P\in (0,1)$ are the discount rate of the $i^{th}$ agent and the principal, respectively. The discount rates in front of the integral normalize the utility to annuity costs [16]. In the case that the principal is risk-neutral, i.e., $u_P$ is a linear function of $\mathit{X}$, we can reduce the principal’s problem using the following observation. After taking expectations on the integral of the $i^{th}$ agent’s output process, $\mathbf{E}[\int X_i\left(t\right)dt]=\mathbf{E}[\int f_i\left(\mathit{a}\left(t\right)\right)dt]+\mathbf{E}[\int {\sigma }_{i}d{B}_i\left(t\right)]=\mathbf{E}[\int f_i\left(\mathit{a}\left(t\right)\right)dt]$, using the fact that the expectation of Ito’s integral is zero. Thus, we can write $U_P$ in terms of $\mathit{a}$ only in this special case [24].

Optimal multiagent contracts should maximize the principal’s expected total discounted utility $U_P$ subject to (a) n individual-rational (IR) constraints at $t=0$ and (b) n incentive-compatible (IC) constraints at any $t\in [0,T]$. The IR constraints guarantee that agents would agree to enter the contracts if the expected utility exceeds certain thresholds; the IC constraints guarantee that agents would realize the target efforts at each epoch of the horizon. In the presence of the interactions between agents, we have one additional constraint that the n agents’ best responses constitute a Nash equilibrium at each $t\in [0,T]$. In summary, optimal multiagent contracts can be solved as follows:

$$\begin{array}{cc}\hfill & \underset{\mathit{c}\left(t\right),t\in [0,T]}{max}{U}_P\hfill \\ \hfill s.t.& U_i\ge {\underline{W}}_i,\forall i\in \left[n\right]\hfill & \hfill \left(\mathrm{individual}\text{-}\mathrm{rational} \mathrm{constraint}\right),\\ \hfill & a_i^{\ast }\left(t\right)\in arg \underset{a_i}{max}{U}_i,\forall i\in \left[n\right],\forall t\in [0,T]\hfill & \hfill \left(\mathrm{incentive}\text{-}\mathrm{compatible} \mathrm{constraint}\right).\end{array}$$

(4)

3. Incentive-Compatible Constraints

In this section, we characterize an individual agent’s optimum action within given multiagent contracts.

3.1. Parametrization of the Individual Agent’s Problem

We analyze an arbitrary $i^{th}$ agent’s optimum action given the other agents’ optimum actions. Without loss of generality, we reformulate the analysis of the prior work [3,21] under a new multiagent contracts setting.

In dynamic Stackelberg games, one commonly defines the continuation value $W_i\left(t\right)$ (the value function in dynamic programming) when the optimal actions $\mathit{a}$ are taken by all agents in $[t,T]$, i.e., the agent i’s conditional expected optimal discounted utility received from t to T, as follows,

$$\begin{array}{cc}\hfill W_i\left(t\right)& ={\mathbf{E}}^{\mathit{a}}\left[{\int }_t^T{r}_i{e}^{-r_i(s-t)}{u}_i(\mathit{a}\left(s\right),c_i\left(s\right))ds+r_i{e}^{-r_i(T-t)}{\mathsf{\Phi }}_i\left(Z_i\left(T\right)\right)|{\mathcal{F}}_t^{\mathit{B},{\mathit{B}}_z}\right].\hfill \end{array}$$

(5)

where ${\mathcal{F}}_t^{\mathit{B},{\mathit{B}}_z}$ is the filtration generated by the Brownian Motions $\mathit{B}$ and ${\mathit{B}}_z$.

We now describe the dynamics of $W_i\left(t\right)$ for a single agent with a path-dependent terminal condition as follows:

Proposition 1.

There exists an ${\mathcal{F}}_t^{\mathit{B},{\mathit{B}}_z}$ adapted process ${\mathit{Y}}_i\left(t\right)={(Y_{i1}\left(t\right),Y_{i2}\left(t\right))}^T$ such that the continuation value $W_i\left(t\right)$ of the $i^{th}$ agent is represented by the process:

$$\begin{array}{c}\hfill d{W}_i\left(t\right)=r_i\left[W_i\left(t\right)-u_i\left(\mathit{a}\left(t\right),c_i\left(t\right)\right)\right]dt+r_i{Y}_{i1}\left(t\right){\sigma }_{i}d{B}_i\left(t\right)+r_i{Y}_{i2}\left(t\right){\sigma }_{Z_i}d{B}_{Z_i}\left(t\right),\end{array}$$

Conversely, a process $W_i\left(t\right)$ satisfying the SDE is the $i^{th}$ agent’s continuation value.

Proof. 

Given fixed and optimal n-agents’ efforts $\left\{\mathit{a}\right(t):t\ge 0\}$ and the filtration ${\mathcal{F}}_t={\mathcal{F}}_t^{\mathit{B},{\mathit{B}}_z}$, we have:

$$\begin{array}{cc}\hfill U_i\left(t\right)& ={\mathbf{E}}^{\mathit{a}}\left[{\int }_0^T{r}_i{e}^{-r_{i}s}{u}_i(\mathit{a}\left(s\right),c_i\left(s\right))ds+r_i{e}^{-r_{i}T}{\mathsf{\Phi }}_i\left(Z_i\left(T\right)\right)|{\mathcal{F}}_t\right],\hfill \end{array}$$

(6)

$U_i\left(t\right)$ is an ${\mathcal{F}}_t$-Martingale, i.e., for any $s<t$, using (6) and the iterated conditional expectation, it is readily seen that ${\mathbf{E}}^{\mathit{a}}\left(U_i\left(t\right)\right|{\mathcal{F}}_s)=U_i\left(s\right)$. From the Martingale representation theorem [3], we obtain the existence of adapted processes $Y_{i1}\left(t\right)$ and $Y_{i2}\left(t\right)$ such that:

$$\begin{array}{c}\hfill d{U}_i\left(t\right)=r_i{e}^{-r_{i}t}{Y}_{i1}\left(t\right){\sigma }_{i}d{B}_i\left(t\right)+r_i{e}^{-r_{i}t}{Y}_{i2}\left(t\right){\sigma }_{Z_i}d{B}_{Z_i}\left(t\right).\end{array}$$

From (5), it is easily seen that (6) can be rewritten as:

$$\begin{array}{c}\hfill U_i\left(t\right)={\int }_0^t{r}_i{e}^{-r_{i}s}{u}_i(\mathit{a}\left(s\right),c_i\left(s\right))ds+e^{-r_{i}t}{W}_i\left(t\right),\end{array}$$

and using Ito’s lemma, we obtain the dynamics:

$$d{U}_i\left(t\right)=r_i{e}^{-r_{i}t}{u}_i(\mathit{a}\left(t\right),c_i\left(t\right))dt+e^{-r_{i}t}d{W}_i\left(t\right)-r_i{e}^{-r_{i}t}{W}_i\left(t\right).$$

Equating the above two dynamics of $d{U}_i\left(t\right)$ gives the result. □

The expansion of the state space (when compared to [21]) is needed to accommodate the path-dependent terminal condition, requiring the vector ${(\mathit{X}\left(t\right),\mathit{Z}\left(t\right))}^T$ to be a part of the state space. Dynamic contracts between the principal and the $i^{th}$ agent must specify: (a) the instantaneous compensations $c_i\left(t\right)$ and (b) two processes $Y_{i1}\left(t\right)$ and $Y_{i2}\left(t\right)$ as the sensitivity of the agent’s continuation value $W_i\left(t\right)$ to the output $X_i\left(t\right)$ and terminal process $Z_i\left(t\right)$, respectively.

Given a contract ${\{c_i\left(t\right),{\mathit{Y}}_i\left(t\right)\}}_{t\in [0,T]}$, we use the one-shot deviation principle to derive the necessary condition for the optimality of the effort ${\left\{a_i\left(t\right)\right\}}_{0\le t\le T}$ with given ${\left\{{\mathit{Y}}_i\left(t\right)\right\}}_{0\le t\le T}$. This optimality condition is equivalent to the IC constraint in (4). Such an optimality condition holds for an arbitrary $i^{th}$ agent’s $a_i\left(t\right)$ given ${\mathit{a}}_{-i}$.

Proposition 2.

For any fixed ${\mathit{a}}_{-i}\left(t\right)$, the contracted compensation $c_i\left(t\right)$ for the agent i is implementable if and only if $\left\{a_i\left(t\right)\right\}$ satisfies:

$$\begin{array}{c}\hfill a_i\left(t\right)\in arg\underset{{\tilde{a}}_i\left(t\right)\in {\mathcal{A}}_i}{max}\left[Y_{i1}\left(t\right)f_i({\mathit{a}}_{-i}\left(t\right),{\tilde{a}}_i\left(t\right))+u_i({\mathit{a}}_{-i}\left(t\right),{\tilde{a}}_i\left(t\right),c_i\left(t\right))\right],\end{array}$$

(7)

for all $t\in [0,T]$.

Proof. 

Let $\mathit{a}\left(t\right)$ be the optimal effort vector, and let the effort of the $i^{th}$ agent, for a fixed $t>0$, be:

$${\tilde{a}}_i\left(s\right)=\left\{\begin{array}{cc}{\tilde{a}}_i\left(s\right)& \mathrm{if} s<t\\ a_i\left(s\right)& \mathrm{if} s\ge t.\end{array}\right.$$

We denote $\tilde{\mathit{a}}=({\mathit{a}}_{-i},{\tilde{a}}_i)$. Choosing actions $\tilde{\mathit{a}}$ will change the dynamics of $X_i$ and $W_i$. To obtain the new dynamics, we apply Girsanov’s theorem with the kernel $\varphi \left(t\right)=f_i\left(\tilde{\mathit{a}}\left(t\right)\right)-f_i\left(\mathit{a}\left(t\right)\right)$. The new dynamics adapted to Brownian motions ${\tilde{B}}_i$ and ${\tilde{B}}_{Z_i}$ on the space $(\mathsf{\Omega },\mathcal{A},\tilde{P})$ are given by:

$$\begin{array}{c}\hfill \left\{\begin{array}{cc}{\sigma }_{i}d{B}_i\left(t\right)\hfill & ={\sigma }_{i}d{\tilde{B}}_i\left(t\right)+\varphi \left(t\right)dt,\hfill \\ {\sigma }_{Z_i}d{B}_{Z_i}\left(t\right)\hfill & ={\sigma }_{Z_i}d{\tilde{B}}_{Z_i}\left(t\right).\hfill \end{array}\right.\end{array}$$

Substituting in (1) and Proposition 1 under $\tilde{\mathit{a}}$, the dynamics of $U_i\left(t\right)$ become:

$$\begin{array}{cc}\hfill d{\tilde{U}}_i\left(t\right)=& r_i{e}^{-r_{i}t}\left(u_i(\tilde{\mathit{a}}\left(t\right),c_i\left(t\right))-u_i(\mathit{a}\left(t\right),c_i\left(t\right))+Y_{i1}\left(t\right)(f_i\left(\tilde{\mathit{a}}\left(t\right)\right)-f_i\left(\mathit{a}\left(t\right)\right)\right)dt+\hfill \\ \hfill & Y_{i1}\left(t\right){\sigma }_{i}d\tilde{B}\left(t\right)+Y_{i2}{\sigma }_{Z_i}d{\tilde{B}}_{Z_i}\left(t\right).\hfill \end{array}$$

Since $a_i$ is optimal, the drift of this SDE must be non-positive. This completes the proof. □

These two propositions decouple the principal’s and an arbitrary $i^{th}$ agent’s problem. To specify the target efforts that are not observable, the principal can incentivize the agent by recommending a sensitivity level $r_i{\mathit{Y}}_i\left(t\right)$. With n agents, the Nash equilibrium is equivalent to finding the optimal $\mathit{Y}\left(t\right)={[{\mathit{Y}}_1\left(t\right),\cdots {\mathit{Y}}_n\left(t\right)]}^{⊺}$ jointly. The principal can create a contract with: (a) functions for ${\left\{c_i(\mathit{W}\left(t\right),\mathit{X}\left(t\right),\mathit{Z}\left(t\right))\right\}}_{i\in \left[n\right]}$ for each agent i; and (b) functions of the sensitivity ${\left\{r_i{\mathit{Y}}_i\left(t\right)\right\}}_{i\in \left[n\right]}$ that specify the target effort processes. Hence, we create multiagent contracts that provide consistent information for all agents over the planning horizon, which are thus implementable.

Characterizing implementable multiagent contracts require that the actions of the agents $\mathit{a}\left(t\right)$ form a multiagent Nash equilibrium at each epoch $t\in [0,T]$. We note that in our formulation, there are interactions among n-agents both in the instantaneous utility $u_i$ and drift term of output processes $f_i$ for all $i\in \left[n\right]$. The principal thus chooses a target effort level $\mathit{a}\left(t\right)$, which form a Nash equilibrium among agents, so that each agent $i\in \left[n\right]$ is disincentivized to deviate from the target $a_i\left(t\right)$ when the other agents do not, i.e., implementing the targeted $a_i\left(t\right)$.

3.2. Multiagent Nash Equilibrium

We now prove the existence of a Nash equilibrium among n-agents’ best responses (7) at a fixed epoch t. Bellman’s principle of optimality guarantees that it is sufficient to show the existence of a Nash equilibrium within the Hamiltonian of the IC constraint to prove the existence of a subgame perfect Nash equilibrium.

We need the following assumptions on the functions $u_i$ and $f_i$ for all $i\in \left[n\right]$:

• $u_i:\mathcal{A}\times {\mathcal{C}}_i\to \mathbb{R}$ is twice continuously differentiable, decreasing in $c_i$, and concave in $a_i$.
• $f_i:\mathcal{A}\to {\mathbb{R}}_{+}$ is twice continuously differentiable, increasing and concave in $a_i$.
• For each i and $\mathit{a}$, $\frac{\partial f_i({\mathit{a}}_i,0)}{\partial a_i}\ne 0$ and $f_i\left(\mathit{a}\right)\to \infty$ while $\frac{\partial f\left(\mathit{a}\right)}{\partial a_i}\to 0$ as $a_i\to \infty$.
• The set ${\cap }_i\{(\mathit{a},\mathit{c}):u_i(\mathit{a},c_i)\ge 0 \mathrm{for} \mathrm{all} i\}$ is nonempty and compact.
• There exists an $m>0$ such that $m<{sup}_x{u}_i({\mathit{a}}_{-i},x,c_i)$, and $u_i\to -\infty$ as $x\to \infty$, for all i and ${\mathit{a}}_{-i},c_i$.
• $u_i({\mathit{a}}_{-i},0,c_i)\ge 0$ for each ${\mathit{a}}_{-i},c_i$.

The single-agent contract in [21] and the multiagent contracts in [24] are special cases of the functions above with u separable in $a\left(t\right)$ and $c\left(t\right)$ and $f\left(a\right(t\left)\right)=a\left(t\right)$. Assumption 4 is satisfied because an arbitrary agent can choose effort $a_i\left(t\right)=0$ to have zero utility. Assumption 6 is valid because $a_i\left(t\right)\notin {\mathcal{A}}_i$ if $u_i<0$. With these assumptions, we can show the following lemmas.

Lemma 1.

Let ${\mathit{\alpha }}_i=({\mathit{a}}_{-i},c_i)$, and we define:

$$g_i^{{\mathit{\alpha }}_i}\left(x\right)=\frac{-u_i^{\prime }({\mathit{a}}_{-i}\left(t\right),x,c_i\left(t\right))}{f_i^{\prime }({\mathit{a}}_{-i}\left(t\right),x)}.$$

$g_i^{{\mathit{\alpha }}_i}$ is continuously differentiable and monotonically increasing as a function of x in the domain ${\mathcal{A}}_i$. Furthermore, there exist $0\le {\beta }_i<{\gamma }_i$ such that for each ${\beta }_i<y<{\gamma }_i$ and ${\mathit{\alpha }}_i\in {\mathbb{R}}^n$, $g_i^{{\mathit{\alpha }}_i}\left(x\right)=y$ has a solution.

Proof. 

$g_i^{{\mathit{\alpha }}_i}$ is well defined from Assumption 3 on $f_i^{\prime }$, i.e., it is nonzero, and its monotonicity follows from the concavity of $u_i$ and $f_i$. We define:

$${\widehat{g}}_i\left(x\right)={\inf }_{\mathit{\alpha }\in {\mathbb{R}}^n}{g}_i^{\mathit{\alpha }}\left(x\right),{\beta }_i=\max \{0,{\sup }_{\mathit{\alpha }\in {\mathbb{R}}^n}{g}_i^{\mathit{\alpha }}\left(0\right)\}.$$

Let ${\theta }_i$ be the $i^{th}$ agent’s greatest effort, i.e., ${\theta }_i=sup{\mathcal{A}}_i$. Define ${\gamma }_i=\widehat{g}\left({\theta }_i\right)$ and ${\theta }_i$ sufficiently large so that $[{\beta }_i,{\gamma }_i]$ is nonempty. This exists as ${\widehat{g}}_i$ is an increasing function in Figure 1.

For arbitrary $y\in [{\beta }_i,{\gamma }_i]$, we define ${\widehat{g}}_i\left(\widehat{x}\right)=y$. Such an $\widehat{x}\in {\mathcal{A}}_i$ exists because the function ${\widehat{g}}_i$ is monotonically increasing. Now, for any ${\mathit{\alpha }}_i$, the function $g_i^{{\mathit{\alpha }}_i}\left(\widehat{x}\right)\ge y$ and $g_i^{{\mathit{\alpha }}_i}\left(0\right)\le {\beta }_i$. The result follows from the continuity of $g_i^{{\mathit{\alpha }}_i}$ and the intermediate value theorem. □

Applying Lemma 1 to all agents $i\in \left[n\right]$, we define a set $\mathcal{Y}={\prod }_i[{\beta }_i,{\gamma }_i]$. We can now rigorously define the multiagent Nash equilibrium as follows.

Definition 1.

The multiple agents’ effort $\mathit{a}$ is called a Nash equilibrium if and only if an arbitrary agent’s deviation from the stipulated effort level in $\mathit{a}$ while the other agents follow their stipulated actions will result in a loss to the agent, i.e., for each $i\in \left[n\right]$,

$$\begin{array}{c}\hfill a_i\in {\mathsf{\Gamma }}_i({\mathit{a}}_{-i},c_i,y_{i1})=\left\{\widehat{x}:\widehat{x}=arg\underset{x}{max}\left[y_{i1}{f}_i({\mathit{a}}_{-i},x)+u_i({\mathit{a}}_{-i},x,c_i)\right]\right\}.\end{array}$$

(8)

Note that the multiagent equilibrium is independent of $Y_{i2}$. We now prove a simple lemma to characterize the equilibrium:

Lemma 2.

For all $t\in [0,T]$ and each $\mathit{y}\left(t\right)\in \mathcal{Y}$ and $\mathit{c}\left(t\right)\in \mathcal{C}$, if $\mathit{a}\left(t\right)$ satisfying (8) exists, it lies in the set ${\bigcap }_i\{(\mathit{a},c_i):u_i(\mathit{a},c_i)\ge 0\}$.

Proof. 

For any given contract $\mathit{y}\left(t\right),\mathit{c}\left(t\right)$, let $\mathit{a}\left(t\right)$ be a Nash equilibrium for each $t\in [0,T]$, and let $u_i(\mathit{a}\left(s\right),c_i\left(s\right))<0$ for some $i\in \left[n\right]$ and $s\in (t_1,t_2)$. Thus, ${\int }_{t_1}^{t_2}{u}_i({\mathit{a}}_{-i}\left(s\right),a_i\left(s\right),c_i\left(s\right))ds<0$. However, from Property 6,

$${\int }_{t_1}^{t_2}{u}_i(({\mathit{a}}_{-i}(s),0,c_i(s))ds\ge 0.$$

Thus, $\mathit{a}\left(t\right)$ is not a Nash equilibrium for $t\in (t_1,t_2)$, a contradiction. The result follows from the fact that as $u_i({\mathit{a}}_{-i}\left(t\right),a_i\left(t\right),c_i\left(t\right))$ is continuous, thus it cannot be strictly negative on a set of measure zero in $[0,T]$. □

The following corollary shows that agents continue to abide by the conditions of the contracts until the termination epoch.

Corollary 1.

A consequence of the implementation of the Nash equilibrium is that no agent has an incentive to leave the contracts before the terminal epoch T.

Proof. 

As is seen in the proof of Lemma 2, when agents’ actions form a multiagent Nash equilibrium, each agent receives a positive utility in any finite interval, thus making each agent’s total utility an increasing function of its continuation value. Therefore, no agent is motivated to deviate from the target action before the termination epoch T. □

The theorem below establishes the existence of such an equilibrium in each given epoch t.

Theorem 1.

For each given $\mathit{y}\left(t\right)\in \mathcal{Y}$ and $\mathit{c}\left(t\right)\in \mathcal{C}$, there exists a subgame perfect Nash equilibrium $\mathit{a}\left(t\right)\in \mathcal{A}$ for every $t\in [0,T]$.

Proof. 

For a fixed agent $i\in \left[n\right]$, given the concavity of the functions in Proposition 1, a necessary and sufficient condition for $\widehat{x}$ to solve the optimization problem is that $g_i^{\mathit{a}\left(t\right),c_i\left(t\right)}\left(\widehat{x}\right)=y_{i1}\left(t\right)$. We note that as defined in (8), ${\mathsf{\Gamma }}_i(\mathit{a}\left(t\right),c_i\left(t\right),{\mathit{y}}_i\left(t\right))=\{x:g_i^{\mathit{a}\left(t\right),c_i\left(t\right)}\left(x\right)=y_{i1}\left(t\right)\}$.

We now define a point-to-set map,

$$\mathsf{\Gamma }\left(\mathit{a}\left(t\right)\right):={\mathsf{\Gamma }}^{\mathit{c}\left(t\right),\mathit{y}\left(t\right)}\left(\mathit{a}\left(t\right)\right)=\left[{\mathsf{\Gamma }}_1(\mathit{a}\left(t\right),c_1\left(t\right),{\mathit{y}}_1\left(t\right))\right),\cdots ,{\mathsf{\Gamma }}_n(\mathit{a}\left(t\right),c_n\left(t\right),{\mathit{y}}_n\left(t\right))\left)\right].$$

Note that $\mathsf{\Gamma }:\mathcal{A}\to {\mathcal{A}}^{\ast }$, where ${\mathcal{A}}^{\ast }$ is the set of all compact and convex subsets of $\mathcal{A}$. To see that $\mathsf{\Gamma }$ is an upper hemicontinuous point to set map, let ${\mathit{a}}^k$ be a sequence in $\mathcal{A}$ that converges to $\mathit{a}$. Furthermore, let $x^k\in \mathsf{\Gamma }\left({\mathit{a}}^k\right)$ for each k such that ${\mathit{x}}^k$ converges to $\mathit{x}$. To see that $\mathit{x}$ is in $\mathsf{\Gamma }\left(\mathit{a}\right)$, we note that $x_i^k$ is such that $g_i^{{\mathit{\alpha }}^k\left(t\right)}\left(x_i^k\right)=y_{i1}\left(t\right)$. From the definition of $g_i$ in Lemma 1, it is a continuous function of $\mathit{\alpha }$ and x, thus $y_{i1}\left(t\right)={lim}_{k\to \infty }{g}_i^{{\mathit{\alpha }}^{\mathit{k}}\left(t\right)}\left(x^k\right)=g_i^{\mathit{\alpha }\left(t\right)}\left(x\right)$ for each i. The existence of the Nash equilibrium now follows from Lemma 2, Property 4, in the assumptions, and the Kakutani fixed-point theorem [36]. □

3.3. On the Uniqueness of the Nash Equilibrium in Multiagent Contracts

The individual incentive contract assumes that, if multiple subgame perfect Nash equilibria exist, the principal has the power to choose her or hiss preferred one. However, if multiple equilibria exist in the multiagent contracts, first all equilibria must be found, and then, we look for plausible selection criteria to convince the agents to implement a specific chosen equilibrium. To avoid this computational problem at each epoch t, we impose reasonable and mild additional conditions to guarantee a unique Nash equilibrium. We now state these conditions:

• $u_i$ is strictly concave in $a_i$ and $u_i^{\prime }({\mathit{a}}_{-i},a_i,c_i):=\frac{\partial u_i({\mathit{a}}_{-i},a_i,c_i)}{\partial a_i}<0$ for each $i\in \left[n\right]$ and each ${\mathit{a}}_{-i}$.
• Let for each $i\in \left[n\right]$, $u_{ij}^{{}^{″}}:=\frac{{\partial }^2{u}_i}{\partial a_i\partial a_j}$ for each $i,j$ and, similarly, $f_{ij}^{{}^{″}}$. The matrix $D^2{u}_i$ ($D^2{f}_i$) is such that its ith row is strictly diagonally dominant (diagonally dominant) in variables $\mathit{a}$, i.e.,

  $$-u_{ii}^{{}^{″}}>\sum _{i\ne j}|u_{ij}^{{}^{″}}\left|\right(-f_{ii}^{{}^{″}}\ge \sum _{i\ne j}|f_{ij}^{{}^{″}}\left|\right).$$

Remark 1.

Comments on the uniqueness conditions of the Nash equilibrium of agents:

1.

Condition 1 stipulates that the optimal effort the agents exert is unique and also has a negative effect on their instantaneous utility, i.e., the marginal utility as a function of the agent i’s effort $a_i$ is negative.

2.

Condition 2 states that agent i’s particular decision mostly affects the decrease in his or her marginal utility. In contrast, the other agents’ efforts have a minor effect (note the strict concavity implies that $u_{ii}^{{}^{″}}$ is negative).

3.

The signs of $u_{ij}^{″}$ are related to whether $a_i$ is a strategic complement or a strategic substitute [36]. Diagonal dominance thus assumes that the magnitude of the effect of any agent’s actions exceeds the magnitude of the combined strategic effects of all the other agents’ actions.

We now prove a result:

Lemma 3.

Let $u_i$ and $f_i$ satisfy Conditions 1 and 2 above and $g_i^{{\mathit{\alpha }}_i}$ be as defined in Lemma 1, and $g\left(\mathit{a}\right)={[g_1^{{\mathit{\alpha }}_1}\left(\mathit{a}\right),\cdots ,g_n^{{\mathit{\alpha }}_n}\left(\mathit{a}\right)]}^T$. The Jacobian matrix of g, $D_{\mathit{a}}g\left(\mathit{a}\right)$ is then a P-matrix, i.e., has all principal minors positive.

Proof. 

We first show that $D_{\mathit{a}}g\left(\mathit{a}\right)$ is a strictly row diagonally dominant Jacobian matrix. Note that, suppressing the argument $\mathit{a}$, $\mathit{c}$, we obtain:

$$\begin{array}{ccc}\hfill \frac{\partial g_i}{\partial a_i}& =& \frac{1}{f_i^{\prime }}\{-u_{ii}^{{}^{″}}-\frac{-u_i^{\prime }}{f_i^{\prime }}{f}_{ii}^{{}^{″}}\},\hfill \\ \hfill \frac{\partial g_i}{\partial a_j}& =& \frac{1}{f_i^{\prime }}\{-u_{ij}^{{}^{″}}-\frac{-u_i^{\prime }}{f_i^{\prime }}{f}_{ij}^{{}^{″}}\}.\hfill \end{array}$$

The row dominance now follows from Condition 2 and the observation that $f_i>0$, $-u_{ii}^{{}^{″}}>0$, $-f_{ii}^{{}^{″}}\ge 0$, and $g_i^{{\mathit{\alpha }}_i}=\frac{-u_i^{\prime }}{f_i^{\prime }}>0$ in the domain $g^{-1}\left(\mathcal{Y}\right)\subset \mathcal{A}$. Furthermore, it is easy to see that each principal submatrix of $D_{\mathit{a}}g$ is also strictly row diagonally dominant. Using Gershgorin’s theorem [37], it follows that all the principal submatrices of $D_{\mathit{a}}g$ are nonsingular. We now let B be any such principal submatrix and let $I_B$ be the diagonal matrix of its diagonal elements and $A_B$ the matrix of its off-diagonal elements. Define $B\left(t\right)=I_B+t{A}_B$ for each $t\in [0,1]$. $A\left(t\right)$ is strictly row diagonally dominant for each t, and since $det\left(B\right(0\left)\right)>0$, $det\left(B\right(1\left)\right)$ is also positive. Thus, $D_{\mathit{a}}g$ is a P-matrix. □

Theorem 2.

Assume Conditions 1 and 2 above hold. Then, for each epoch $t\in [0,T]$, the Nash equilibrium is unique.

Proof. 

From the strict concavity of $u_i$ and (8), we see that for given $\mathit{y}$ and $\mathit{c}$, $\mathit{a}$ is a Nash equilibrium if and only if:

$$g\left(\mathit{a}\right)=\mathit{y}.$$

Let ${\theta }_i$ be the largest effort agent i can put, as found in Lemma 1; define $\widehat{\mathcal{A}}={\mathsf{\Pi }}_i[0,{\theta }_i]$, and consider the set $g\left(\widehat{\mathcal{A}}\right)=\{\mathit{y}:g\left(\mathit{a}\right)=\mathit{y},\mathit{a}\in \widehat{\mathcal{A}}\}$. Using the P-matrix property of $D_{\mathit{a}}g$, the fact that $\widehat{A}$ is a hypercube and the Gale–Nikaido theorem [33] (or [34]), we see that g maps $\widehat{\mathcal{A}}$ homeomorphically onto $g\left(\widehat{\mathcal{A}}\right)$. The uniqueness follows as $\mathcal{Y}\subset g\left(\widehat{\mathcal{A}}\right)$. □

4. The Optimal Multiagent Contracts

In this section, we solve the optimal multiagent contracts given that n-agents put effort at equilibrium in Section 3. We denote the principal’ controls as $\mathit{v}\left(t\right)=\left(\mathit{c}\right(t),\mathit{y}(t\left)\right)$. Define:

$$\begin{array}{c}\hfill U_P^{\mathit{v}}={\mathbf{E}}^{\mathit{v}}\left[{\int }_0^T{r}_P{e}^{-r_{P}s}\left(u_P({\mathit{X}}^{\mathit{v}}\left(t\right),\mathit{c}\left(s\right)\right)ds-r_P{e}^{-r_{P}T}{\mathbf{1}}^{⊺}·\mathsf{\Phi }\left(\mathit{Z}\left(T\right)\right)\right].\end{array}$$

(9)

With the parameterized IC constraints and a well-defined set of Nash equilibria $\mathsf{\Theta }\left(\mathit{v}\right)$ for given $\left\{\mathit{v}\right(t\left)\right\}$ for all $t\in [0,T]$, the principal’s problem is as follows:

$$\begin{array}{cc}\hfill {\mathit{v}}^{\ast }& ={\mathrm{argmax}}_{\begin{array}{c}{\{\mathit{v}:\mathit{a}\in \mathsf{\Theta }\left(\mathit{v}\right)\}}_{0\le t\le T}\end{array}}{U}_P^{\mathit{v}}\hfill \end{array}$$

(10)

$$U_P:=U_p^{{\mathit{v}}^{\ast }}.$$

(11)

We note here that, in general, getting all Nash equilibrium points is generally not possible, but if it is unique, the problem (10) can be solved. Let $R_P^{\mathit{v}}$ be the present value of the conditional expectation of the continuation value of the principal at time t when the policy $\mathit{v}\left(\xi \right)$ is followed in $\xi \in [t,T]$. Thus:

$$\begin{array}{c}\hfill R_P^{\mathit{v}}\left(t\right)={\mathbf{E}}^{\mathit{v}}\left[{\int }_t^T{r}_P{e}^{-r_p\xi }\left(u_P(\mathit{X}\left(\xi \right),\mathit{c}\left(\xi \right)\right)d\xi -r_P{e}^{-r_{P}T}{\mathbf{1}}^{⊺}·\mathsf{\Phi }\left(\mathit{Z}\left(T\right)\right)|{\mathcal{F}}_t^{\mathit{B},{\mathit{B}}_z}\right].\end{array}$$

(12)

and note that $\left\{R_P^{\mathit{v}}\left(t\right)\right\}$ is a random process. Therefore, define:

$$\begin{array}{cc}\hfill U_P^{\mathit{v}}\left(t\right)& ={\mathbf{E}}^{\mathit{v}}\left[U_P^{\mathit{v}}|{\mathcal{F}}_t^{\mathit{B},{\mathit{B}}_z}\right]\hfill \\ \hfill & ={\int }_0^t{r}_P{e}^{-r_p\xi }\left(u_P(\mathit{x}\left(\xi \right),\mathit{c}\left(\xi \right)\right)d\xi +R_P^{\mathit{v}}\left(t\right).\hfill \end{array}$$

In case the optimal solution ${\mathit{v}}^{\ast }$ exists, then $R_P^{{\mathit{v}}^{\ast }}\left(t\right)$ is a ${\mathcal{F}}_t^{\mathit{B},{\mathit{B}}_z}$ adapted Martingale and thus has a zero drift, and for other $\mathit{v}$’s, its drift is non-positive. We now make the following assumption about the principal’s continuation value:

Assumption 1.

We assume that the value $R_p^{\mathit{v}}\left(t\right)$ has the following $C^{1,2,2,2}$ functional form $F^{\mathit{v}}(t,\mathit{W}\left(t\right),\mathit{X}\left(t\right),\mathit{Z}\left(t\right))$ in variables t, the n-agents’ continuation vector $\mathit{W}\left(t\right)$, the observed output vector $\mathit{X}\left(t\right)$, and the termination value descriptor vector $\mathit{Z}\left(t\right)$.

$C^k$ represents the differentiability class regarding the scalar or the vector. In what follows, for the ease of exposition, we will shorten $F^{\mathit{v}}(t,\mathit{W}\left(t\right),\mathit{X}\left(t\right),\mathit{Z}\left(t\right))$ to $F_t^{\mathit{v}}$ whenever there is no possibility of confusion.

Remark 2.

The state space includes $\mathit{Z}\left(t\right)$ to assure that the vector process $\left(\mathit{W}\right(t),\mathit{X}(t),\mathit{Z}(t\left)\right)$ is Markov. In the special case that $T\to +\infty$ (i.e., an infinite time horizon with the transversality condition), the state space does not contain t, as in [3].

Then, the optimum utility received by the principal, following the optimal control ${\mathit{v}}^{\ast }$, can also be written as:

$$\begin{array}{c}\hfill U_P\left(t\right)={\int }_0^t{r}_P{e}^{-r_{P}s}{u}_P(\mathit{x}\left(\xi \right),{\mathit{c}}^{\ast }\left(\xi \right))d\xi +F^{{\mathit{v}}^{\ast }}(t,\mathit{W}\left(t\right),\mathit{X}\left(t\right),\mathit{Z}\left(t\right)).\end{array}$$

(13)

Note that, at epoch t, ${\left\{\mathit{x}\left(\xi \right)\right\}}_{\xi \le t}$ is realized, and ${\left\{\mathit{X}\left(\xi \right)\right\}}_{\xi >t}$ is determined by the control $\mathit{v}$. Following the argument of Proposition 1, we see that $U_P\left(t\right)$, defined by (11) and (13), is a ${\mathcal{F}}^{\mathit{B},{\mathit{B}}_{\mathit{Z}}}$-adapted Martingale and thus has drift zero. Applying Ito’s multidimensional lemma and the dynamics of $\mathit{W}\left(t\right)$, $\mathit{X}\left(t\right)$, and $\mathit{Z}\left(t\right)$, we obtain the dynamics of $U_P\left(t\right)$. Thus, we can solve for F by setting the drift of its dynamics to zero.

To obtain the drift term, we recall the dynamics of the state variables. For notational convenience, we let $\sigma =\mathrm{diag}(({\sigma }_1,\cdots ,{\sigma }_n)$, ${\mathit{Y}}_1\left(t\right)=\mathrm{diag}(r_1{\sigma }_1{\mathit{Y}}_{11}\left(t\right),\cdots ,r_n{\sigma }_n{\mathit{Y}}_{1n}\left(t\right))$, ${\mathit{Y}}_2\left(t\right)=\mathrm{diag}(r_1{\sigma }_{Z_1}{\mathit{Y}}_{12}\left(t\right),\cdots ,r_1{\sigma }_{Z_n}{\mathit{Y}}_{2n}\left(t\right))$, and $\mathit{r}=\mathrm{diag}(r_1,\cdots ,r_n)$. Let $\mathit{L}$ be the Cholesky factor of $\mathsf{\Sigma }$ (i.e., $\mathsf{\Sigma }=\mathit{L}{\mathit{L}}^T$), the covariance matrix of $\mathit{B}\left(t\right)$. There exists a process $\widehat{\mathit{B}}$, a vector of n independent Brownian motions, with:

$$\begin{array}{c}\hfill \left\{\begin{array}{c}\mathit{B}\left(t\right)=\mathit{L}\widehat{\mathit{B}}\left(t\right),\hfill \\ \mu (t,\mathit{W}(t\left)\right)=\mathit{W}\left(t\right)-\mathit{u}\left(\mathit{a}\right(\mathit{y}\left(t\right)),\mathit{c}(t\left)\right),\hfill \\ \sigma \left(\mathit{y}\left(t\right)\right)={\mathit{Y}}_1\left(t\right)\mathit{L}.\hfill \end{array}\right.\end{array}$$

Using Proposition 1, we get:

$$\begin{array}{c}\hfill d\mathit{W}\left(t\right)=\mathit{r}\left[\mathit{W}\left(t\right)-\mathit{u}\left(\mathit{a}\right(\mathit{y}\left(t\right)),\mathit{c}(t\left)\right)\right]dt+\sigma \left(\mathit{y}\left(t\right)\right)d\widehat{\mathit{B}}\left(t\right)+{\mathit{Y}}_2\left(t\right)d{\mathit{B}}_{\mathit{z}}\left(t\right),\end{array}$$

(14)

and similarly for the dynamics of $\mathit{Z}\left(t\right)$ using (2).

We define a differential operator ${\mathcal{H}}^{\mathit{v}}$ as a function of the control vector $\mathit{v}={(\mathit{y},\mathit{c})}^T$ as follows,

$$\begin{array}{ccc}\hfill {\mathcal{H}}^{\mathit{v}}{F}_t& =& \mathit{r}\left(D_{\mathit{w}}{F}_t\mu (t,\mathit{w}\left(t\right))\right)+D_{\mathit{x}}{F}_{t}f\left(\mathit{a}\left(\mathit{y}\left(t\right)\right)\right)+D_{\mathit{z}}{F}_t{\mu }_{\mathit{z}}(\mathit{x}\left(t\right),\mathit{c}\left(t\right))+\hfill \\ & & \frac{1}{2}\mathrm{trace}(\sigma {\left(\mathit{y}\left(t\right)\right)}^{⊺}{D}_{\mathit{w}}^2{F}_t\sigma \left(\mathit{y}\left(t\right)\right)+{\mathit{Y}}_2\left(t\right)D_{\mathit{w}}^2{F}_t{\mathit{Y}}_2\left(t\right)+{\mathit{L}}^T{\sigma }^{⊺}{D}_{\mathit{x}}^2{F}_t\sigma \mathit{L}+\hfill \\ & & \sigma \left(\mathit{y}\left(t\right)\right)D_{\mathit{w}\mathit{x}}^2{F}_t\sigma \mathit{L}+{\sigma }_{\mathit{z}}^{⊺}{D}_{\mathit{z}}^2{F}_t{\sigma }_{\mathit{z}}),\hfill \end{array}$$

(15)

where $D_{\mathit{x}}{F}_t$ and $D_{\mathit{x}}^2{F}_t$ are the first and second derivative matrices of $F_t$ with respect to $\mathit{x}$. We note here that in the above, we suppressed the superscript in $F_t$.

Applying the multidimensional Ito’s lemma to (13), we get the drift of the dynamics of $U_P\left(t\right)$ as:

$$\begin{array}{c}\hfill \frac{\partial }{\partial t}{F}_t^{{\mathit{v}}^{\ast }}+r_P{e}^{-r_{P}t}{u}_P(\mathit{X}\left(t\right),{\mathit{c}}^{\ast }\left(t\right))+{\mathcal{H}}^{{\mathit{v}}^{\ast }}{F}_t^{{\mathit{v}}^{\ast }}.\end{array}$$

(16)

We now prove the theorem that verifies Assumption 1 and sets up a Hamilton–Jacobi–Bellman equation that solves the problem (9):

Theorem 3.

The principal’s problem can be formulated as the Hamilton–Jacobi–Bellman equation:

$$\begin{array}{cc}\hfill & {\displaystyle \frac{\partial }{\partial t}}{F}_t+\underset{\mathit{v}=(\mathit{y},\mathit{c})}{max}\left\{r_P{e}^{-r_{P}t}{u}_P(\mathit{x}\left(t\right),\mathit{c}\left(t\right))+{\mathcal{H}}^{\mathit{v}}{F}_t\right\}=0\hfill \\ \hfill & s.t.F(T,\mathit{w},\mathit{x},\mathit{z})=-r_P{e}^{-r_{P}T}{\mathbf{1}}^{⊺}·\mathsf{\Phi }\left(\mathit{z}\right),\forall \mathit{w},\mathit{x},\mathit{z},\hfill \\ \hfill & \mathit{a}\left(\mathit{v}\right(t\left)\right)\in \mathsf{\Theta }\left(\mathit{v}\right(t\left)\right),\forall t\in [0,T].\hfill \end{array}$$

(17)

Let its solution be $G(t,\mathit{w},\mathit{x},\mathit{z})$ and the control $\widehat{\mathit{v}}(t,\mathit{w},\mathit{x},\mathit{z})$. $F=G$ and ${\mathit{v}}^{\ast }=\widehat{\mathit{v}}$ solve the optimization problem (9). Thus, Assumption 1 is verified.

Proof. 

For the ease of notation, we define $\mathit{s}={(\mathit{w},\mathit{x},\mathit{z})}^T$ and let $G_t=G(t,\mathit{s}\left(t\right))$ be the weak solution of the equation (17) under control $\widehat{\mathit{v}}$.

Now, using an arbitrary control law $\mathit{v}$, such that $\mathit{a}\left(\mathit{v}\right(t\left)\right)\in \mathsf{\Theta }\left(\mathit{v}\right(t\left)\right),\forall t\in [0,T]$ at the arbitrary time t, with the state dynamics of ${\mathit{S}}^{\mathit{v}}$ governed by the Brownian motions $\mathit{B},{\mathit{B}}_{\mathit{Z}}$ and when G solves the HJB equation, we see that:

$$\begin{array}{c}\hfill {\displaystyle \frac{\partial }{\partial t}}{G}_t+r_P{e}^{-r_{P}t}{u}_P({\mathit{x}}^{\mathit{v}}\left(t\right),\mathit{c}\left(t\right))+{\mathcal{H}}^{\mathit{v}}{G}_t\le 0,\end{array}$$

for all $\mathit{v}$. Thus, we have, for each time $\xi \in [0,T]$,

$$\begin{array}{c}\hfill {\displaystyle \frac{\partial }{\partial t}}{G}_{\xi }+{\mathcal{H}}^{\mathit{v}}{G}_{\xi }^{\mathit{v}}\le -r_P{e}^{-r_{P}t}{u}_P({\mathit{x}}^{\mathit{v}}\left(\xi \right),\mathit{c}\left(\xi \right)).\end{array}$$

(18)

Integrating the above system from t to T, using Ito’s lemma to $G(t,\mathit{S})$, and integrating (which sets the stochastic integral to zero), we see that:

$$\begin{array}{c}\hfill G_t^{\mathit{v}}={\mathbf{E}}^{\mathit{v}}\left[G_T^{\mathit{v}}-{\int }_t^T({\displaystyle \frac{\partial }{\partial t}}{G}_{\xi }^{\mathit{v}}+{\mathcal{H}}^{\mathit{v}}{G}_{\xi }^{\mathit{v}})d\xi |{\mathcal{F}}_t^{\mathit{B},{\mathit{B}}_z}\right].\end{array}$$

From the boundary condition, we also have $G_T=-r_P{e}^{-r_{P}T}{\mathbf{1}}^{⊺}\mathsf{\Phi }\left({\mathit{z}}^{\mathit{v}}\right)$. Integrating the above expression and Inequality (18), we obtain:

$$\begin{array}{c}\hfill G_t^{\mathit{v}}\ge {\mathbf{E}}^{\mathit{v}}\left[{\int }_t^T{r}_P{e}^{-r_P\xi }{u}_P({\mathit{X}}^{\mathit{v}}\left(\xi \right),{\mathit{c}}^{\mathit{v}}\left(\xi \right))d\xi -r_P{e}^{-r_{p}T}{\mathbf{1}}^{⊺}\mathsf{\Phi }\left({\mathit{Z}}^{\mathit{v}}\left(T\right)\right)|{\mathcal{F}}^{\mathit{B},{\mathit{B}}_z}\right]=R_P^{\mathit{v}}\left(t\right).\end{array}$$

Since the control $\mathit{v}$ was chosen arbitrarily, $R_P^{\mathit{v}}$ as in (12), the optimal solution to the problem (9), we have:

$$\begin{array}{c}\hfill G_t\ge \underset{\mathit{v}}{sup}{G}_t^{\mathit{v}}\ge \underset{\mathit{v}}{sup}{R}_P^{\mathit{v}}\left(t\right)=R_P\left(t\right).\end{array}$$

(19)

To see the converse, let $G_t$ and $\widehat{\mathit{v}}$ solve the HJB $\left(17\right)$. Ito’s lemma gives, as in (18) an Ito integral J:

$$\begin{array}{c}\hfill {\int }_t^T({\displaystyle \frac{\partial }{\partial t}}{G}_{\xi }+{\mathcal{H}}^{\widehat{\mathit{v}}}{G}_{\xi })d\xi +J=G_T-G_t.\end{array}$$

Using (17) and the above with minor rearrangement and taking an expectation conditioned on ${\mathcal{F}}^{\mathit{B},{\mathit{B}}_z}$, we get:

$$\begin{array}{cc}\hfill G_t& ={\mathbf{E}}^{\widehat{\mathit{v}}}\left[{\int }_t^T{r}_P{e}^{-r_P\xi }\left(u_P({\mathit{X}}^{\widehat{\mathit{v}}}\left(\xi \right),\mathit{c}\left(\xi \right)\right)d\xi -r_P{e}^{-r_{P}T}{\mathbf{1}}^{⊺}·\mathsf{\Phi }\left({\mathit{Z}}^{\widehat{\mathit{v}}}\left(T\right)\right)|{\mathcal{F}}^{\mathit{B},{\mathit{B}}_{\mathit{Z}}}\right]=R_P^{\widehat{\mathit{v}}}\left(t\right),\hfill \end{array}$$

Since $\widehat{\mathit{v}}$ is a control and since $R_P\left(t\right)$ is the optimal continuation value under the optimal control ${\mathit{v}}^{\ast }$, $R_P\left(t\right)\ge R_P^{\widehat{\mathit{v}}}\left(t\right)$. Thus, combining with (19), we get:

$$G_t\ge R_P\left(t\right)\ge R_P^{\widehat{\mathit{v}}}\left(t\right)=G_t.$$

The theorem now follows since we have from the above inequalities $G_t=R_P\left(t\right)$ for arbitrary t, and $\widehat{\mathit{v}}$ is the optimal contract. □

Iterative Algorithm for Solving Multiagent Contracts

Since adding an equilibrium constraint causes new computational issues, we propose here an iterative algorithm to obtain the optimal multiagent contracts in Theorem 3. The main idea is to integrate a numerical method for solving the HJB (i.e., Howard’s algorithm [38]) with a fixed-point algorithm (i.e., Eaves–Saigal’s algorithm [39]). For brevity, we denote the state variable at time t by a time-generic vector $\mathit{s}=(\mathit{w},\mathit{x},\mathit{z})\in {\mathbb{R}}^{3n}$ (note that the mesh width for each type of state may vary) and the control at time t by $\mathit{v}=(\mathit{c},\mathit{y})\in {\mathbb{R}}^{2n}$. We discretize the $\mathit{s}-t$ plane by choosing uniform mesh widths $\Delta \mathit{s}=(\Delta \mathit{w},\Delta \mathit{x},\Delta \mathit{z})\in {\mathbb{R}}^{3n}$ and a time step $\Delta t$ such that $T/\Delta t\in \mathbb{N}$. We define the discrete mesh points ${\mathit{s}}_{\mathit{i},\mathit{j},\mathit{k}}$ by:

$$\begin{array}{ccc}\hfill & {\mathit{s}}_{\mathit{i},\mathit{j},\mathit{k}}={(\mathit{i},\mathit{j},\mathit{k})}^{⊺}\Delta \mathit{s},\hfill & \hfill (\mathit{i},\mathit{j},\mathit{k})={(i_1,\dots ,i_n,j_1,\dots j_n,k_1,\dots ,k_n)}^{⊺}\in {\mathbb{N}}^{3n},\\ \hfill & t_{\tau }=\tau \Delta t,\hfill & \hfill \tau \in \left[\frac{T}{\Delta t}\right].\end{array}$$

Our goal is to compute an approximation $F_{\mathit{i},\mathit{j},\mathit{k}}^{\tau }$ to the solution $F(t,\mathit{w},\mathit{x},\mathit{z})$ in (17) by discretization and a finite difference method.

Now, define the approximation for the Hamiltonian operator ${\mathcal{H}}^{\mathit{v}}{F}_t$ in (15) as ${\mathcal{H}}^{\mathit{v}}{\widehat{F}}_{t_{\tau }}$ (we use a forward-in-time and central-in-space scheme) with the following approximations for gradients:

$$\begin{array}{c}\hfill \left\{\begin{array}{c}\frac{\partial {\widehat{F}}_{t_{\tau }}}{\partial t}=\frac{F_{\mathit{i},\mathit{j},\mathit{k}}^{\tau +1}-F_{\mathit{i},\mathit{j},\mathit{k}}^{\tau }}{\Delta t}\hfill \\ D_{\mathit{w}}{\widehat{F}}_{t_{\tau }}{|}_{\ell }=\frac{F_{\mathit{i}+{\mathit{e}}_{\ell },\mathit{j},\mathit{k}}^{\tau }-F_{\mathit{i}-{\mathit{e}}_{\ell },\mathit{j},\mathit{k}}^{\tau }}{2\Delta w},\hfill & \forall \ell \in \left[n\right]\hfill \\ D_{\mathit{x}}{\widehat{F}}_{t_{\tau }}{|}_{\ell }=\frac{F_{\mathit{i},\mathit{j}+{\mathit{e}}_{\ell },\mathit{k}}^{\tau }-F_{\mathit{i},\mathit{j}-{\mathit{e}}_{\ell },\mathit{k}}^{\tau }}{2\Delta x},\hfill & \forall \ell \in \left[n\right]\hfill \\ D_{\mathit{z}}{\widehat{F}}_{t_{\tau }}{|}_{\ell }=\frac{F_{\mathit{i},\mathit{j},\mathit{k}+{\mathit{e}}_{\ell }}^{\tau }-F_{\mathit{i},\mathit{j},\mathit{k}-{\mathit{e}}_{\ell }}^{\tau }}{2\Delta z},\hfill & \forall \ell \in \left[n\right]\hfill \end{array}\right.,\end{array}$$

where ${\mathit{e}}_{\ell }\in {\mathcal{R}}^n$ is a unit vector with one in the ${\ell }^{th}$ entry and zero elsewhere. The ${\ell }^{th}$ entry of the approximation for a Hessian (we only present the Hessian with respect to $\mathit{w}$) is:

$$\begin{array}{c}\hfill D_{\mathit{w}}^2{\widehat{F}}_{t_{\tau }}{|}_{\ell ,{\ell }^{\prime }}=\left\{\begin{array}{c}{\displaystyle \frac{F_{\mathit{i}+{\mathit{e}}_{\ell }+{\mathit{e}}_{{\ell }^{\prime }},\mathit{j},\mathit{k}}^{\tau }-F_{\mathit{i}+{\mathit{e}}_{\ell }-{\mathit{e}}_{{\ell }^{\prime }},\mathit{j},\mathit{k}}^{\tau }-F_{\mathit{i}-{\mathit{e}}_{\ell }+{\mathit{e}}_{{\ell }^{\prime }},\mathit{j},\mathit{k}}^{\tau }+F_{\mathit{i}-{\mathit{e}}_{\ell }-{\mathit{e}}_{{\ell }^{\prime }},\mathit{j},\mathit{k}}^{\tau }}{4\Delta w^2}}\mathrm{if}\ell \ne {\ell }^{\prime },\hfill \\ {\displaystyle \frac{F_{\mathit{i}+{\mathit{e}}_{\ell },\mathit{j},\mathit{k}}^{\tau }-2F_{\mathit{i},\mathit{j},\mathit{k}}^{\tau }+F_{\mathit{i}-{\mathit{e}}_{\ell },\mathit{j},\mathit{k}}^{\tau }}{\Delta w^2}}\mathrm{otherwise}.\hfill \end{array}\right.\end{array}$$

We define the function ${\mathsf{\Psi }}^{\mathit{v}}:=r_P{u}_P+{\mathcal{H}}^{\mathit{v}}{\widehat{F}}_t$ and the principal’s value function under optimal control at time t as $F^{\ast }:=F^{\mathit{v}\ast }(t,\mathit{s})$. We initialize with the boundary condition $F^P(T,\mathit{w},\mathit{x},\mathit{z})=-r_P{e}^{-r_{P}T}{\mathbf{1}}^{⊺}\mathsf{\Phi }\left(\mathit{z}\right)$ as the terminal conditions and the well-posed conditions for the state space. Especially, we note that, in an n-agents’ contract, when $n_1$-agents have zero continuation values w, we need to first solve an $(n-n_1)$-agents subproblem as a boundary condition. In the $m^{th}$ step in the policy iteration, policy evaluation under controls $v^m$ is conducted by solving the approximation of the PDE as ${\left(\frac{\partial {\widehat{F}}_{t_{\tau }}}{\partial t}\right)}^m+{\mathsf{\Psi }}^{{\mathit{v}}^m}=0$.

Since the PDE under arbitrary control is well-posed, we can find a weak solution to $F_t$ [40]. We then (1) solve a fixed-point problem to find the agents’ unique optimal responses ${\mathit{a}}^{\ast }\left(t\right)\in \mathsf{\Theta }\left({\mathit{v}}^m\right)$ and (2) use a greedy algorithm to improve the policy as:

$$\begin{array}{c}\hfill {\mathit{v}}^{m+1}=arg\underset{{\mathit{v}}^{\prime }\in V}{max}{\mathsf{\Psi }}^{{\mathit{v}}^{\prime }}.\end{array}$$

Summarizing the above, we can solve for the optimal multiagent contracts by adopting the following backward scheme:

• Initialize the terminal condition $F(T,\mathit{s})=-{\mathbf{1}}^{⊺}\mathsf{\Phi }\left(\mathit{z}\right)$.
• While $t=T-\tau \Delta t\ge 0$, with a fixed $ϵ>0$,

  (a)

  For each state $\mathit{w},\mathit{x},\mathit{z}$, start with an arbitrary contract ${\mathit{v}}_0=\{{\mathit{c}}_0,{\mathit{y}}_0\}$.

  (b)

  Solve a fixed point problem such that ${\mathit{a}}^{\ast }\left(t\right)\in \mathsf{\Theta }\left({\mathit{v}}_0\right)$. If the conditions in Section 3.3 are satisfied, the equilibrium is unique.

  (c)

  Solve for the boundary conditions as a single-agent contract in [21]. Then, solve a parabolic PDE within (17), i.e., with fixed contracts, to obtain $\tilde{F}(t,\mathit{s})$ [39].

  (d)

  Optimize the objective value $\tilde{F}(t,\mathit{s})$ for each state $\mathit{s}=(\mathit{w},\mathit{x},\mathit{z})$ by the gradient ascent method. The gradient is ${\nabla }_{\mathit{v}}\tilde{F}\in {\mathbb{R}}^{2n}$, and the step size $\gamma$ can be determined by a line-search method. If $\parallel {\nabla }_{\mathit{v}}\tilde{F}\parallel \ge ϵ$, go back to (b) with the new contracts ${\mathit{v}}_0\leftarrow (\mathit{c},\mathit{y})$.

  (e)

  Go to Step 3 if $\parallel {\nabla }_{\mathit{v}}\tilde{F}\parallel <ϵ$.

• Update the contracts $\left\{\mathit{c}\right(t),\mathit{y}(t\left)\right\}$ and continuation value $F(t,\mathit{s})$. Go to Step 2 with $\tau \leftarrow \tau +1$.

Lemma 4.

The iterative algorithm for multiagent incentive contracts converges to the optimal contract as $m\to \infty$.

Proof. 

The backward scheme is a generic Howard’s algorithm, which guarantees that the sequence $F^m$ converges to $F^{\ast }$ and ${\mathit{v}}^m$ converges to ${\mathit{v}}^{\ast }$ as $m\to \infty$ [38]. In addition, we need to guarantee the following three conditions are met. First, under any implementable contracts, the numerical method can evaluate the value F in (17). This is because the weak solution of a linear parabolic PDE can be computed by the finite difference method [40]. Second, for any given ${\mathit{v}}^m$, the Nash equilibrium of agents $a^{\ast }\left(t\right)$ exists, Theorem 1, and the feasible region is non-empty for each ${\mathit{v}}^m$. Finally, if there are multiple Nash equilibrium, we must consider the policy-search procedure in a vector-valued case and compare the objective values of all Nash equilibria, which is known to be difficult if not impossible. Imposing the uniqueness conditions in Theorem 2, searching for all multiagent Nash equilibria is not required [39], and the convergence of the iterative algorithm follows. With these conditions, Howard’s algorithm solves (17) to the optimum and obtains the optimal contract by Theorem 3. □

The multiagent Nash equilibrium is defined for noncooperative multiplayer concave games where each player’s objective function is concave only in his/her own decisions and not necessarily concave with respect to other players’ decisions. Alternative approaches that fully exploit the structure of concave games in searching in equilibrium were reviewed in [41]. The above procedure has been implemented to solve a multiagent incentive contract designed for the simultaneous penetration of electric vehicles and charging stations (with real-world data) in the transportation infrastructure [42].

5. Conclusions

Multiagent incentive contracts with broad applications are hard to solve in general. We characterize the sufficient conditions under which the Nash equilibrium of agents exists and additional requirements for the Nash equilibrium to be unique. We develop a backward iterative algorithm to find optimal contracts. The implication of our result is two-fold. First, compared to the single-agent setting, multiagent contracts can model either team collaborations or competitions depending on the context. Second, those conditions of existence and uniqueness contain new insights about the inertia of effective contracting in multiagent systems.

The limitations of the multiagent incentive contracts’ model include:

• The Martingale approach is restricted to the SDE output process, where the each agent’s decision only affects the drift term. An extension to controlling the diffusion of output process may cause significant technical difficulties even in the single-agent case.
• The coupled gradient-based and fixed-point optimization restricts the computational efficiency of solving the contracts. In the absence of a unique multiagent Nash equilibrium, the proposed algorithm can only compute local optimum contracts, and thus, the verification theorem in Theorem 3 fails. Developing more efficient algorithms for multiagent contracts and with multiple Nash equilibria is a meaningful future direction.

In summary, this work presents a solvable multiagent incentive contracts’ model that opens the door to implementing dynamic contracts with a wide range of applications in quantitative finance, economics, operations research, and decentralized controls.