Opponent-Aware Planning with Admissible Privacy Preserving for UGV Security Patrol under Contested Environment

Abstract

Unmanned ground vehicles (UGVs) have been widely used in security patrol. The existence of two potential opponents, the malicious teammate (cooperative) and the hostile observer (adversarial), highlights the importance of privacy-preserving planning under contested environments. In a cooperative setting, the disclosure of private information can be restricted to the malicious teammates. In adversarial setting, obfuscation can be added to control the observability of the adversarial observer. In this paper, we attempt to generate opponent-aware privacy-preserving plans, mainly focusing on two questions: what is opponent-aware privacy-preserving planning, and, how can we generate opponent-aware privacy-preserving plans? We first define the opponent-aware privacy-preserving planning problem, where the generated plans preserve admissible privacy. Then, we demonstrate how to generate opponent-aware privacy-preserving plans. The search-based planning algorithms were restricted to public information shared among the cooperators. The observation of the adversarial observer could be purposefully controlled by exploiting decoy goals and diverse paths. Finally, we model the security patrol problem, where the UGV restricts information sharing and attempts to obfuscate the goal. The simulation experiments with privacy leakage analysis and an indoor robot demonstration show the applicability of our proposed approaches.

1. Introduction

With the development of intelligent unmanned system technology, unmanned ground vehicles (UGVs) have become extremely rugged for the harshest military use, such as executing monitoring tasks in harsh and complex urban environments [1]. As our world becomes increasingly well-connected, there is an increased need to enable UGVs to cooperate in generating plans for security patrol. For example, the iRobot’s PackBot, which has played a critical role in providing situational awareness for anti-terrorist operations [2].

Several approaches have been proposed in recent years to address the conundrum of privacy preservation through controlling privacy leakage for different requirements under contested environments. One of them is differential privacy [3], which adds appropriate noise to the transmission state in order to limit the opponent to acquiring only the true state of the transmitted signal at a predetermined level of accuracy. Another approach uses cryptography for secure multi-party computation (MPC). In [4,5], the authors encrypt the messages with a public-key homomorphic cryptosystem and apply techniques (e.g., random masking and random permutation) to protect the agents’ privacy. So, the encrypted messages can be exchanged among the agents in various ways [6]. A third approach tries to guarantee privacy as a loss of observability [7,8], but it is difficult to achieve strong privacy of this form. All these mainly focus on the privacy leakage from the information perspective, while decision-related privacy preservation (e.g., privacy-preserving planning) has mostly been neglected.

Several recent pieces of research on privacy-preserving planning for multi-agent systems have captured the attention of the planning community [9,10,11]. Privacy-preserving plans represent plans that do not actively disclose sensitive private information. In fact, privacy preservation is the goal pursued by multi-agent planning, which has been a crucial concern for multi-agent systems in some contexts, such as agent negotiation [12], multi-agent reinforcement learning and policy iteration [4,5], deep learning [13], and distributed constraint optimization problems (DCOPs) [14,15,16]. Multi-agent planning (MAP) in cooperative environments aims at generating a sequence of actions to fulfill some specified goals [17]. Most multi-agent systems rely intrinsically on collaboration among agents to accomplish a joint task, in which the collaboration depends on the exchange of information among them, so the privacy preservation of the information naturally rises.

Security patrol has been widely studied among the defense and security fields in the past decade. Unmanned ground vehicle patrols have gained increased interest in recent decades mainly due to their relevance to various security applications [18]. The common mode of urban security patrol is to patrol checkpoints. As illustrated in Figure 1, the UGVs perform security patrol, one supply center, and some determined checkpoints are distributed across the security patrol environment. The UGVs are required to repeatedly visit some checkpoints to monitor the local area, but the UGVs do not share information about the task plan with the supply center. We assume that the adversary can exploit any predictable behavior of the UGVs, which means the adversary has full knowledge of the patrolling task. Since the opponent is collecting information, the objective of privacy-preserving planning is to protect private information in different situations.

Regarding urban security patrol, some checkpoints located around the urban trunk road are high risk. Thus, it is feasible to deploy UGVs to patrol such checkpoints regularly and collect information (e.g., images, video, etc.). Although UGVs have become quite ubiquitous in patrols with logging and tracking capabilities, they are mostly at risk. The hostile observer will constantly monitor the task execution and get access to the UGVs’ data and actions. The challenge of privacy preservation arises because all aspects of information are private and the UGVs are not eager to share, which drives us to compute privacy-preserving plans that can protect privacy when executed in cooperative and adversarial environments.

In this paper, we address the problem of opponent-aware privacy-preserving planning for security patrol and attempt to answer the following questions: what is opponent-aware privacy-preserving planning, and how can we generate opponent-aware privacy-preserving plans? Our contribution lies in the opponent-aware privacy-preserving planning architecture. In a cooperative setting, the search-based planning method could be restricted to obtain the public information shared by the cooperative agents. Whereas, in an adversarial setting, the observation of the adversary could be purposefully controlled by exploiting decoy goals and diverse paths. Finally, simulation experiments with privacy leakage analysis and indoor robot demonstration show the applicability of our proposed approaches.

The rest of this paper is organized as follows. In Section 2, some related works about privacy, security, and metrics are presented. In Section 3, we decompose the opponent-aware privacy-preserving planning problem into two subproblems from different perspectives. In Section 4, experimental evaluations of plan generation and information leakage analysis were conducted. In Section 5, we conclude this paper and point out further directions.

2. Background and Related Work

2.1. Privacy and Security Assumption

Privacy and Security: Many privacy models have been adopted in multi-agent planning according to three different criteria: the information model (imposed privacy [19], induced privacy [20]), the information-sharing scheme (MA-STRIPS [19], subset privacy [21]), and practical privacy guarantees (no privacy [22], weak privacy [23], object cardinality privacy [24], and strong privacy [9]). Privacy can be divided into different categories, such as agent privacy, model privacy, decision privacy, topology privacy, and constraint privacy [4,14]. Here we introduce some widely used types of privacy.

Definition 1

(Agent privacy). No agent should be able to recognize the identity or existence of another agent.

Agent privacy can be achieved by employing anonymous or coded names. Such as one agent would not want the opponents to know the identity or existence.

Definition 2

(Model privacy). No agent should be able to recognize the model of another agent, including environmental and algorithmic models which are related to states, actions, observations, transition probability, and rewards.

Model privacy is the key issue in adversarial environments; one agent will not get more information about others except for what has been revealed.

Planning algorithms for privacy preserving can be divided into weak or strong privacy [17], $\epsilon$-strong privacy [25], and provable guarantees privacy [26].

Definition 3

(Weak privacy preserving). The agent will not disclose private information of the states, private actions, and private parts of the public actions during the whole run of the algorithm.

In other words, the agent will share only the information in the public part. Even if not communicated, the adversary will deduce the existence and values of private variables, preconditions, and effects from the (public) information communicated.

Definition 4

(Strong privacy preserving). The adversary can deduce no information about the private variables, preconditions, or effect of the actions, beyond the shared public projection of actions and plans.

Privacy essentially concerns a semi-honest adversary who is interested in learning the information. Privacy is equivalent to the concept of unobservability among the control community, and it is closely related to the concept of semantic security from cryptography [27], where secure plans build on the concept of independent inputs [28]. A secure plan is always private, which imposes an additional constraint (all possible goals must result in to the same observations) to the privacy problem [29].

Security Assumption: In [30], the authors define the notion of privacy-preserving planning based on secure MPC and provide some proper analysis of privacy leakage in multi-agent planning. Many assumptions specify the properties of the agent, environment, and algorithm in some secure multi-party computation literature [10,28,31].

Assumption 1

(Adversary model). An honest but curious adversary who is passive and follows the algorithm and the protocol correctly but may glean information from the execution and communicated data to learn about the privacy. A malicious adversary, who can actively deviate from the protocol specification.

Assumption 2

(Algorithm known). The adversary has access to the algorithm and knows how the algorithm works. The agent should not rely on the privacy of the algorithmic mechanism itself.

Assumption 3

(Input independent). The adversary can rerun the algorithm by setting different goals as real goals to check the variability of the output.

Assumption 4

(FIFO). When the actor takes action to reach a corresponding state, only then does the adversary receive the corresponding observation in the order which was emitted by the plan execution.

As is usually done by cryptography, these assumptions do not take the adversary’s recognition model into consideration, which is quite different from the artificial intelligence (AI) community.

2.2. Privacy-Preserving Planning

The planning problem of privacy preserving can be modeled as a multi-agent planning (MAP) problem with a privacy-preserving requirement. MAP comes in different types, such as deterministic MAP (DMAP) [19,32], interactive partially observable Markov decision processes (I-POMDPs) [33], and decentralized POMDPs (Dec-POMDPs) [34]. Regarding privacy, there are many synonymous concepts in the recent literature, which all aim at generating obfuscated behavior, such as deception, security, and obfuscation, as shown in

Table 1. Some synonymous concepts of privacy.

Concepts	Main Contributions
Obfuscation	k-ambiguous and d-diverse [35]
	one candidate goal [36]
	secure MAFS [9]
Privacy	privacy leakage [10]
	plan set intersection [11]
	privacy-preserving policy iteration [4]
Security	equidistant states [28]
	last deceptive point [37,38]
	deceptive shortest path [39]
	equidistant states [28]
Deception	bounded deception [40]
	hide intention [41]
	$\lambda$ Deception [42]
	deceptive adversary [43]

. A secure plan is always private; a deceptive plan is always obfuscating, but may or may not be dissimulating [29]. A simple illustration of different strategies is shown in Figure 2.

In a cooperative environment, many multi-agent planners have been proposed to address privacy-preserving planning problems, such as MAFS (multi-agent forward search) [30], MADLA (multi-agent distributed and local asynchronous) [44], and PSM (planning state machine) [11]. In [11], the author proposed one secure planner for multi-agent planning, but this planner is impractical to compute all possible solutions. In [9], the authors introduce a modified version of the multi-agent forward search algorithm, Secure-MAFS [30], which is implemented based on an equivalent macro sending technique [24]. Some privacy guarantee planning algorithms have been provided in [9], but they are restricted to very special cases.

In an adversarial environment, the adversary implicitly uses the signal behavioral cues of the actors during the plan execution, and perform diagnosis on the internal information based on the resulting observations. Recently, there has been some interest in exploring privacy preservation [36], goal obfuscation [28,35], deception [37,38], intention hiding [41], etc. In [35], Kulkarni et al. attempted to make plans with k-ambiguous goals, but they were not guaranteed to be secure. In [36], Keren et al. proposed to preserve privacy by keeping their goal ambiguous for as long as possible, but there was only one candidate goal and one partially obfuscated plan. In [38], Masters et al. applied some deceptive strategies for path planning, but these do not support deception when the adversary knows the explicit model. In [28], Kulkarni et al. proposed to securely obfuscate the real goal by making all candidate goals equally likely for as long as possible, but the heuristic deployed makes the planner incomplete. All these studies employ goal or plan recognition modules.

2.3. Information Leakage Metric

Although the key motivation for privacy-preserving planning is preserving privacy, some private information will be leaked during the planning, which means it is impossible to achieve complete privacy. If one malicious teammate directly receives any of the private information, or can indirectly deduce the privacy from the communicated public information, the privacy information will be leaked. To evaluate the privacy leakage, we consider the foundations of quantitative information flow [45]. The leakage of the private information is based on the uncertainty of the adversary about the input. Here we use the min-entropy (an instance of Rényi entropy [46]) as a better measure of the privacy information leakage $\left(PIL\right)$:

$$PIL=H_{\infty }\left(H\right)-H_{\infty }\left(H\right|L),$$

(1)

where the initial uncertainty is $H_{\infty }\left(H\right)$, the residual uncertainty is $H_{\infty }\left(H\right|L)$.

Using the uniform distribution case, we denote the number of states as $t_{prio}$ and $t_{post}$, then the remaining uncertainty gives a security guarantee. The expected probability that the adversary could guess H given L decreases exponentially with $H_{\infty }\left(H\right|L)$: $2^{-H_{\infty }\left(H\right|L)}=2^{-log{t}_{\mathrm{post}}}=1/t_{\mathrm{post}}$, and we can obtain the privacy information leakage:

$$PIL=log{t}_{\mathrm{prio}}-log{t}_{\mathrm{post}}=log\frac{t_{\mathrm{prio}}}{t_{\mathrm{post}}}.$$

(2)

3. Methodology

3.1. Opponent-Aware Privacy-Preserving Planning

In privacy-preserving planning (PPP), it is important to acknowledge that two potential opponents are involved: the malicious teammate (cooperator) and the hostile observer (adversary). PPP should produce plans that reveal neither the goal nor the activities of the agents, but many planners cannot have completeness, strong privacy preserving, and efficiency together. So, it is practical for them to achieve opponent-aware privacy preservation within bounded privacy information leakage. As illustrated in Figure 3, privacy leakage will occur at the information layer and decision-making layer. At the information layer, differential privacy and homomorphic encryption are applicable techniques to protect private information.

In this paper, we mainly focus on the middle layer for decision-making. For task planning in cooperative environments, we need to restrict the information sharing to malicious teammates, and for the path planning in adversarial environments, we need to control the observability of the adversary. Here, we define the opponent-aware privacy-preserving planning problem as follows:

Definition 5

(Opponent-aware privacy-preserving planning). We define opponent-aware privacy-preserving planning as a multi-agent planning problem of protecting privacy secure to certain extent considering two opponents.

As a result, the generated plans protect privacy from two potential opponents: the malicious teammate and the hostile observer. In a cooperative setting, to cope with malicious teammates, we should restrict the disclosure of private information to malicious teammates. In adversarial settings, real combat scenarios often consist of hostile opponents, so we will add obfuscation to control the observability of the opponents.

3.2. Information Sharing Restricted Task Planning

In cooperative environments, the agents are cooperative in concurrently planning and executing their local plans to achieve a joint goal. We could model all other agents as a single adversary, who can collect the information to infer more. Information sharing restricted task planning with privacy preservation can be defined as follows [10]:

Definition 6

(Information sharing restricted task planning). For a set of agents $\mathcal{N}$, the information sharing restricted task planning problem for multi-agent $\mathcal{M}={\left\{{\mathsf{\Pi }}_i\right\}}_{i=1}^{\left|\mathcal{N}\right|}$ is a set of agent problems, where for each agent $n\in \mathcal{N}$ the problem is:

$${\mathsf{\Pi }}_i=〈{\mathcal{V}}_i={\mathcal{V}}^{pub}\cup {\mathcal{V}}_i^{priv},{\mathcal{A}}_i={\mathcal{A}}_i^{pub}\cup {\mathcal{A}}_i^{priv},\mathcal{I},\mathcal{G}〉,$$

(3)

where ${\mathcal{V}}_i$ is a set of variables, s.t. each $V\in {\mathcal{V}}_i$ has a finite domain $dom\left(V\right)$, if $\left|dom\right(V\left)\right|=2$, then all variables are binary. ${\mathcal{V}}^{pub}$ is the set of public variables common to all agents and ${\mathcal{V}}_i^{priv}$ is the set of variables private to agent $n_i\in \mathcal{N}$, s.t. ${\mathcal{V}}^{\mathrm{pub}}\cap {\mathcal{V}}_i^{\mathrm{priv}}=\varnothing$. The state $\mathcal{I}$ is the initial state and $\mathcal{G}$ is the goal.

Each action is defined as a tuple $a=\langle pre\left(a\right),eff\left(a\right),cost\left(a\right)\rangle$, where $pre\left(a\right)$ and $eff\left(a\right)$ are partial states representing the precondition and effect, respectively, $cost\left(a\right)$ is the cost of action a. So, the state transition can be defined as $\mathsf{\Gamma }(s,a)\vDash s\cup eff\left(a\right)$. We follow the formal treatment of privacy-preserving planning from [10,30], for each agent $n\in \mathcal{N}$, the private parts of the problem ${\mathsf{\Pi }}_i$ are:

• The set of private variables ${\mathcal{V}}_i^{priv}$ and the number $|{\mathcal{V}}_i^{priv}|$, the domains $dom\left(V\right)$ and the size $\left|dom\right(V\left)\right|$.
• The set of private actions ${\mathcal{A}}_i^{priv}$ and the number $|{\mathcal{A}}_i^{priv}|$, the number and values of variables in $pre\left(a\right)$ and $eff\left(a\right)$.
• The private parts of the public actions in ${\mathcal{A}}^{\mathrm{pub}}$, such as the numbers and values of private variables in $pre\left(a\right)\cap {\mathcal{V}}_i^{priv}$ and $eff\left(a\right)\cap {\mathcal{V}}_i^{priv}$ for each action $a\in {\mathcal{A}}_i^{pub}$.

3.2.1. Task Plan Generation

The multi-agent planning problem $\mathcal{M}={\left\{{\mathsf{\Pi }}_i\right\}}_{i=1}^{\left|\mathcal{N}\right|}$ can be viewed from different perspectives, called projections. The view of a single agent $n_i$ on the global problem is not the only ${\mathsf{\Pi }}_i$, projections of other agents are available as well. As for agent $n_i$, the public projection of an action $a\in {\mathcal{A}}_i^{pub}$ is $a^{⊳}=〈pre{\left(a\right)}^{⊳},eff{\left(a\right)}^{⊳}〉$, the public projection of ${\mathsf{\Pi }}_i$ can be represented as follows:

$${\mathsf{\Pi }}_i^{⊳}=〈{\mathcal{V}}^{pub},{\mathcal{A}}_i^{⊳}=\left\{a^{⊳}\right|a\in {\mathcal{A}}_i^{pub}\},{\mathcal{I}}^{⊳},{\mathcal{G}}^{⊳}〉$$

(4)

So, the task planning solution to ${\mathsf{\Pi }}_i$ is a sequence ${\pi }_i$ of actions from ${\mathcal{A}}_i\cup {\bigcup }_{j\ne i}{\mathcal{A}}_i^{⊳}$, the goal state ${\mathcal{G}}_k={\pi }_i\circ \mathcal{I}$, which means $\mathsf{\Gamma }(\mathcal{I},{\pi }_i)\vDash {\mathcal{G}}_k$. The public projection of $\pi$ is ${\pi }^{⊳}=(a_1^{⊳},...,a_k^{⊳})$ with all private actions omitted. The global solution of $\mathcal{M}$ is a set of task plans ${\left\{{\pi }_i\right\}}_{i=1}^{\left|\mathcal{N}\right|}$, s.t. each ${\pi }_i$ is a local solution to ${\mathsf{\Pi }}_i$. If ${\pi }_i^{⊳}={\pi }_j^{⊳}$ for the public action, we call these local solutions equivalent.

3.2.2. Privacy Leakage Analysis

We adopt the privacy leakage metric from [47,48], as we set the number $|{\mathcal{V}}_i^{priv}|\le p$ and the size $d={max}_{V\in {\mathcal{V}}_i^{priv}}\left|dom\left(V\right)\right|$. The prior information is a tuple:

$${\mathrm{I}}_{prio}=〈{\mathsf{\Pi }}^{⊳},{\pi }^{⊳},p,b〉.$$

(5)

The additional information obtained by the adversary is a sequence of messages exchanged between the agents $\mathcal{N}=(n_1,...n_k)$. After information exchange during the planning process, the posterior information available to the adversary is a tuple:

$${\mathrm{I}}_{post}=〈{\mathsf{\Pi }}^{⊳},{\pi }^{⊳},p,b,\mathcal{N}〉.$$

(6)

Considering the transition system of the ${\mathsf{\Pi }}_i$, we associate the prior information ${\mathrm{I}}_{prio}$ and ${\mathrm{I}}_{post}$ with variables $\tau \left({\mathrm{I}}_{prio}\right)$ and $\tau \left({\mathrm{I}}_{post}\right)$, which represent the uncertainty of the planning algorithm. So, the final information leakage is computed as:

$$PIL=log\tau \left({\mathrm{I}}_{prio}\right)-log\tau \left({\mathrm{I}}_{post}\right)=log\frac{\tau \left({\mathrm{I}}_{prio}\right)}{\tau \left({\mathrm{I}}_{post}\right)}.$$

(7)

The upper bound of all transition systems’ number is $t^0={\left(2^{d^2}-1\right)}^p$. After classifying the actions into five categories, i.e., inital -applicable (ia), not-inital-applicable (nia), privately-dependent (pd), privately-independent (pi), privately-nondeterministic (pn) [47], the final information leakage formula is as follows:

$$PIL=log\frac{{\prod }_{a^{⊳}\in {\mathcal{A}}^{⊳}}{\tau }_{prio}\left(a\right)}{{\prod }_{a^{⊳}\in {\mathcal{A}}^{⊳}}{\tau }_{post}\left(a\right)}.$$

(8)

In this paper, we mainly use MAFS algorithms for task planning, and the privacy leakage can be computed as follows: we first reconstruct the search tree, then identify the parent states and applied actions, and classify the actions into five classes (ia, nia, pd, pi, pn). Finally, we compute the information leakage (see Algorithm 1 for details). Here, the privacy leakage computation with sets of actions can be reformulated as a mixed-integer linear program (MILP) problem with disjunctive constraints.

Algorithm 1: Privacy information leakage analysis based on the MAFS algorithm.

Input: $\mathcal{M}={\left\{{\mathsf{\Pi }}_i\right\}}_{i=1}^{\left|\mathcal{N}\right|}$, number p, and size d  Output: privacy information leakage $PIL$ 1 reconstruct the search tree based on the MAFS algorithm [30]. 2  identify possible parent states. 3  identify possible applied actions. 4 classify actions into five classes (ia, nia, pd, pi, pn). 5 compute privacy information leakage using the Equation (8). 6 return $PIL$

For the possible number of the transition system, we construct the following combinatorial optimization problem, which can be solved using the off-the-shelf solver IBM CPLEX [49].

$$\max log\left(\prod _{a^{⊳}\in {\mathcal{A}}^X}{\tau }_{post}\left(a^{⊳}\right)\right)$$

(9)

$$s.t.\underset{a^{⊳}\in {\mathcal{A}}^X}{\bigvee }{\tau }_{post}\left(a^{⊳}\right)\le t^X,$$

(10)

where the $t^X\le t^0$, action type $X\in \{ia,nia,pd,pi,pn\}$, ${\mathcal{A}}^X\subseteq {\mathcal{A}}^{⊳}$.

3.3. Observability Controlled Path Planning

In adversarial environments, the observed agents try to control the observation of the adversary by obfuscating their goals. Considering the observation of the adversary in the adversarial setting (mission planning, reconnaissance, etc.), privacy immediately follows from setting with partial observation [28,35,40]. The observability controlled path planning problem is to find a path from the start location to the goal on the navigation map (discrete grid, connected graph, or continuous space representation). So, the discrete path planning problem can be defined as follows:

Definition 7

(Observability controlled path planning). For every agent $n\in \mathcal{N}$, the observability controlled path planning problem is a tuple [35]:

$$\mathsf{\Phi }=〈\mathcal{D},\mathcal{I},\mathcal{G},\mathcal{P},\mathsf{\Omega },\mathcal{O}〉$$

(11)

• $\mathcal{D}=〈\mathcal{S},\mathcal{A},\mathrm{c}〉$ is the path planning domain, $\mathcal{S}$ is a non-empty set of location nodes, $\mathcal{A}\subseteq \mathcal{S}\times \mathcal{S}$ is a set of action-related edges, $\mathrm{c}:\mathcal{E}\mapsto R_0^{+}$ returns the cost of traversing each edge.
• $\mathcal{I}\in \mathcal{S}$ is the start location and $g_r\in \mathcal{G}$ is the real goal;
• $\mathcal{G}=\{g_r\cup g_0\cup g_1\cdots \}$ is a set of candidate goals, where $g_r$ is the real goal
• $\mathsf{\Omega }=\left\{o_i|i=1,\cdots \right\}$ is a set of m observations that can be emitted as a result of the action taken and the state transition.
• $\mathcal{O}:(\mathcal{A}\times \mathcal{S})\to \mathsf{\Omega }$ is a many-to-one observation function which maps the taken action and the next state reached to an observation in Ω.

In adversarial environments, the adversary will receive the observation sequence associated the actions performed by the observed agent. We could model this process as a one-sensor model, where the adversary maintains one belief space according to the observations. Following the definition of belief space from [35], we take the belief space of the adversary into account in path planning, so as to control the observability of the adversary.

Definition 8.

A belief $b_n$ is induced by observation ${\mathcal{O}}_i$, emitted by action $a_i$, resulting in state ${\widehat{s}}_i$. The belief state and belief update are defined as:

$$b_0=\left\{{\widehat{s}}_0|\mathcal{O}\left(\varnothing ,\mathcal{I}\right)=o_0\wedge \mathcal{O}\left(\varnothing ,{\widehat{s}}_0\right)=o_0\right\},$$

(12)

$$b_{i+1}=update\left(b_i,o_{i+1}\right)=\left\{{\widehat{s}}_{i+1}\right|\exists \widehat{a},\mathsf{\Gamma }\left({\widehat{s}}_i,\widehat{a}\right)\vDash {\widehat{s}}_{i+1}\wedge {\widehat{s}}_i\in b_i\wedge \mathcal{O}\left(\widehat{a},{\widehat{s}}_{i+1}\right)=o_{i+1}\}.$$

(13)

3.3.1. Path Plan Generation

Decoy Goals: If the adversary is aware of the actor’s candidate goals but not the real goal. An observability controlled path plan is to hide the real goal with decoy goals, where ${\mathcal{G}}_n=\{g_0\cup \dots \cup g_{n-1}\}$ is the set of decoy goals, the observation sequence should not be biased by any goal in ${\mathcal{G}}_n$. The objective here is to minimize the privacy leakage by making the adversary’s belief space consistent with the decoy goals.

Definition 9.

An observability controlled path plan with decoy goals:

$$\mathsf{\Gamma }(\mathcal{I},{\pi }_k)\vDash g_r\mathrm{and}\left|G\in \mathcal{G}:\exists s\in b_n,s\right|=G|⩾m$$

(14)

where $m<n$, for ease of computation, in this paper, we set $m=2$, as in the final stage, path generation will only depend on two goals.

Observability Controlled Path: Predictability and obfuscation are a pair of incompatible concepts. With the decoy goals selected, patrolling on an observability controlled plan, the observed agent will start with obfuscated steps, but the agent will adopt predictable steps when approaching the goal in the end. So, one observability controlled path is the one with steps that are obfuscated for as long as possible. There is one obfuscated turning point, where all subsequent steps are predictable.

We will employ one probabilistic goal recognition model as the adversary’s sensor model.

Definition 10.

An obfuscated turning point is the final state in the observation sequence ${\mathcal{O}}_n=\{o_1,...o_n\}$, s.t. the posterior probability of the real goal does not exceed any selected decoy goals, otherwise, the point is predictable to the adversary.

$$\mathcal{P}\left(g_r\right|{\mathcal{O}}_n)\le \mathcal{P}\left(g\right|{\mathcal{O}}_n),\forall g\in {\mathcal{G}}_n\backslash \left\{g_r\right\}$$

(15)

Definition 11.

A last obfuscated turning point is the last state ${\pi }^i$ of one given path π, which all subsequent states, ${\pi }^j,\forall j\in \{i+1,...|\pi \left|\right\}$ are predictable to the adversary.

Here, we mainly focus on the last obfuscated turning point. The observability controlled path plan will cover two parts. As shown in Figure 4, one part of the obfuscated path from the start point to the last obfuscated turning point, and one part of the predictable path from the last obfuscated turning point $\left(LOTP\right)$ to the real goal. We can get the strong goal obfuscated path $\pi$ with continually obfuscated steps to the $LOTP$. Using the cost-difference-based probabilistic goal recognition model introduced in [50], we can get the $LOTP$ after selecting the decoy goals:

$$optc\left(LOTP,g_r\right)\approx \frac{optc\left(g_r,g_d\right)+optc\left(s,g_r\right)-optc\left(s,g_d\right)}{2},$$

(16)

where $g_d$ is the selected decoy goal, and $optc(a,b)$ is the optimal cost from the state a to b. If we adopt discrete grid or graph-based discrete domain representations for path planning, we will approximate the $LOTP$ to the closet state.

Diverse Path: When the adversary knows the observed agent’s goal, in order to control the adversary’s observability, we need diverse paths. We can compute the diversity between all the pairs of plans using one plan distance metric mentioned in Appendix A.1. Two plans are a $\delta$ distant pair with respect to distance metric d, if $d(p_1,p_2)=\delta$. A path plan set ($PPS$) induced by plan p starting at $\mathcal{I}$ is minimally $\delta$ distant if $\delta ={min}_{p1,p2\in PPS}d(p1,p2)$.

Definition 12.

A plan ${\pi }_k$ is k diverse path plan $(k\ge 2)$:

$$d_{min}\left(PPS(\mathcal{I},{\pi }_k)\right)\ge \delta \mathrm{and}\left|PPS(\mathcal{I},{\pi }_k)\right|\ge k.$$

(17)

As a result, if the adversary does not know the real goal, the first part of the path is done by performing a two-decoy-goals path planning. After getting the $LOPT$, we can compute the whole path plan. If the adversary does know the real goal, we need to generate diverse path plans. The details of an observability controlled path plan are given in Algorithm 2.

Algorithm 2: An observability controlled path plan generation algorithm.

3.3.2. Privacy Leakage Analysis

Planning with obfuscated goals involves preserving privacy with minimized information leakage. Under the requirement of privacy preservation, the observed agent will deliberately choose misleading actions to obfuscate the goal. We can quantify the information leakage of the states and actions as follows:

Definition 13

(S-PI). The privacy information leakage based state privacy information metric is defined as:

$${\mathcal{I}}_{S-PI}\left(s_j\right)=H(\underset{g_i\in G\setminus \left\{g_r\right\}}{max}P\left(g_i|s_j\right)-P\left(g_r|s_j\right))=-log(\underset{g_i\in G\setminus \left\{g_r\right\}}{max}P\left(g_i|s_j\right)-P\left(g_r|s_j\right)).$$

(18)

Definition 14

(A-PI). As for $a_i\in \mathcal{E}\left(s\right)$, $a_j\in {\mathcal{E}}^{\prime }\left(s\right)$, and ${\mathcal{E}}^{\prime }\left(s\right)=\mathcal{E}\left(s\right)\backslash a_i$. The information leakage based action privacy information metric is defined as:

$${\mathcal{I}}_{A-PI}\left(a_i\right)=\frac{{\sum }_{a_j\in {\mathcal{E}}^{\prime }\left(s\right)}{I}_{S-PI}\left(s\right)}{\left|{\mathcal{E}}^{\prime }\left(s\right)\right|}.$$

(19)

Using the action privacy information metric ${\mathcal{I}}_{S-PI}\left(s\right)$ as additional action cost, we can analyze the privacy leakage of the observability controlled path plan.

4. Experiments

In this section, experiments were conducted for opponent-aware privacy-preserving planning. All the experiments were executed on one Alienware running Ubuntu 16.04 with 4 CPU cores and 8 GB of RAM. We used the MAFS algorithm [30] for information sharing restricted task plan generation. The algorithms for privacy leakage analysis and observability controlled path planning were coded with Python.

4.1. Plan Generation and Privacy Leakage Analysis

Here, we first generate task plans for a robot in the urban security patrol scenario. Then we present three different goal configuration scenarios for path planning. Besides, we analyze the privacy leakage for the task plan and path plan. Finally, we present an indoor robot demonstration using the TurtleBot3 Burger [51].

4.1.1. Task Plan Generation and Privacy Leakage Analysis

As shown in Figure 1, we now define some variables for security patrol scenario. The interaction of the simplified security patrol scenario can be modeled between four agents: two UGVs, one supply center (the malicious teammate), and the hostile observer (opponent).

Variables Definition: For task planning under a cooperative environment, $\mathcal{N}=\{UGV1,UGV2,SC\}$. After patrolling any candidate checkpoint in zone 1, UGV will return to the supply center to charge and transmit collected data, and the task will be completed after patrolling the two zones.

As shown in

Table 2. Variables for task planning.

Variable in	Description	Variable	Values	$\mathcal{I}$	$\mathcal{G}$
	UGV1 is charged	cg1	T/F	T	-
${\mathcal{V}}^{pub}$	UGV2 is charged	cg2	T/F	T	-
	task1 is complete	tc1	T/F	F	T
	task2 is complete	tc2	T/F	F	T
	checkpoint 1 is patrolled	cp1	T/F	F	-
	checkpoint 2 is patrolled	cp2	T/F	F	-
${\mathcal{V}}_{UGV1}^{priv}$	checkpoint 3 is patrolled	cp3	T/F	F	-
	checkpoint 4 is patrolled	cp4	T/F	F	-
	zone 1 is patrolled	zn1	T/F	F	T
	checkpoint 5 is patrolled	cp5	T/F	F	-
	checkpoint 6 is patrolled	cp6	T/F	F	-
${\mathcal{V}}_{UGV2}^{priv}$	checkpoint 7 is patrolled	cp7	T/F	F	-
	checkpoint 8 is patrolled	cp8	T/F	F	-
	zone 2 is patrolled	zn2	T/F	F	T
${\mathcal{V}}_{SC}^{priv}$	supply center can provide support	sc	T/F	T	-

, we set the binary variables with $T/F$ values. In the initial state, the supply center has enough supplies, the UGV is charged. In the goal state, the task of the UGV is complete. The following set of variables can be used to describe the task planning problem.

Information Sharing Restricted Task Plan: Here, we simply set UGV1 for zone 1 and UGV2 for zone2. Each UGV will choose two checkpoints to patrol (e.g., checkpoint 1 and 3). The actions of ${\mathcal{A}}_{UGV1}$ and ${\mathcal{A}}_{SC}$ can be formulated as shown in

Table 3. Actions for UGV and supply center.

Action	Description	Label	pre(a)	eff(a)
	patrol checkpoint 1	PC1	{cg1 = T}	{cp1 = T, cg1 = F}
${\mathcal{A}}_{UGV1}^{pub}$	patrol checkpoint 3	PC3	{cg1 = T}	{cp3 = T, cg1 = F}
	task1 is complete	TC	{cp1 = T, cp3 = T, tc1 = F}	{zn1 = T, tc = T}
${\mathcal{A}}_{SC}^{pub}$	recharge	RC	{sc = T, cg1 = F}	{sc = F, cg1 = T}
	recharge and resupply	RR	{sc = F, cg1 = F}	{sc = T, cg1 = T}

. We provide the action descriptions for UGVs and the supply center in Figure 5.

In the following, we will compute the task plan of UGV1 for zone 1 security patrol. The projection of public actions and related transition system are shown in Figure 6. The projection results of the public actions are shown in

Table 4. Projection the public actions of the UGV and the supply center.

Action	pre(a)	eff(a)
$P{C}^{⊳}$	{cg1 = T}	{cg1 = F}
$T{C}^{⊳}$	{tc = F}	{tc = T}
$R^{⊳}$	{cg1 = F}	{cg1 = T}

. The actions $PC{1}^{⊳},PC{3}^{⊳}\in {\mathcal{A}}_{UGV}$, $R{C}^{⊳},R{R}^{⊳}\in {\mathcal{A}}_{SC}$ both have an equal projection. We denote them simply as $P{C}^{⊳}$ and $R^{⊳}$.

We chose MAFS and Secure-MAFS algorithms for task plan generation. The solution of UGV1 to the security patrol scenario is ${\pi }_{UGV1}^{⊳}=\{R,PC,R,PC,TC\}$, which is public to the supply center.

Privacy Leakage Analysis: Then, complete transition is shown in Figure 7. In MAFS, if the state of the UGV is expanded using public action, the resulting public projection state will be sent to the supply center. We analyzed the privacy leakage based on the sent and received states from the UGV.

An upper bound of the UGV transition system is $t^0={15}^p$, where $p=|{\mathcal{V}}_{UGV1}^{priv}|$. After classifying the action types, the $P{C}^{⊳}$ belongs to $\{ia,pi,pn\}$, $T{C}^{⊳}$ belongs to $\left\{pd\right\}$. ${\tau }_{PC}^{ia}={12}^p$, ${\tau }_{PC}^{pi}={15}^p-6^p$, ${\tau }_{PC}^{pn}={15}^p-8^p$, ${\tau }_{PC}^{ia\times pi}={12}^p-3^p$, ${\tau }_{PC}^{ia\times pn}={12}^p-6^p$, ${\tau }_{TC}^{ia\times pi}={15}^p-3^p$. Using Algorithm 1 with Equation (8), we could compute the privacy information leakage for UGV1: $PIL=log\tau \left({\mathrm{I}}_{prio}\right)-log\tau \left({\mathrm{I}}_{post}\right)=10.4-9.7\approx 0.7$.

4.1.2. Path Plan Generation and Privacy Leakage Analysis

Observability Controlled Path Plan: As shown in Figure 8, we used a $13\times 13$ discrete grids based simulation environment with different configurations (line, circular, and triangular) for experimental evaluation. We simply set $m=2$, $k=2$, and the UGV patrolled one checkpoint through one observability controlled path and chose one diverse path back to the supply center. For any checkpoint, after choosing the candidate decoy checkpoints, we used Algorithm 2 to generate an observability controlled path.

Privacy Leakage Analysis: Following the “single-observation” cost difference based probabilistic goal recognition model from [50], we could pre-compute the cost difference for each state offline to calculate the likelihood that each goal will be the selected checkpoint. As shown in Figure 9, we could create heatmaps for the discrete grids domain, showing the posterior goal probability of each goal at each state. Armed with heatmaps, we could use the state/action privacy information metrics (Equations (13) and (14)) for privacy leakage analysis. The results of the privacy leakage of the paths to each checkpoint under different configurations are shown in

Table 5. The privacy leakage of the paths to each checkpoint under different configurations.

Configuration	Checkpoint1	Checkpoint2	Checkpoint3
Line	13.5	14.9	13.5
Circular	20.7	10.5	10.5
Triangular	12.3	12.2	14.7

.

4.2. Indoor Robot Demonstration

To simulate the security patrol scenario with an internal robot and an external human, we used the TurtleBot3 Burger for an indoor robot demonstration. The TurtleBot3 Burger is a mobile robot platform established on ROS (robot operation system).

Table 6. The configuration of TurtleBot3 Burger.

Items	Configuration
Lidar	360-degree laser Lidar LDS-01 (HLS-LFCD2)
SBC	Raspberry PI 3 and Intel Joule 570x
Battery	Lithium polymer 11.1V 1800 mAh
IMU	Gyroscope 3 Axis, Accelerometer 3 Axis, Magnetometer 3 Axis
MCU	OpenCR (32-bit ARM Cortex M7)
Motor	DYNAMIXEL(XL430)

shows the configuration. As shown in Figure 10, the TurtleBot3 Burger contains several modules, and we designed the ROS nodes for the software framework and built an experimental scene with four checkpoints. The initial state of the robot is in the middle of the scene.

As shown in Figure 11, after generating the information sharing restricted task plan, the robot should generate an observability controlled path plan for checkpoint patrol. As for any checkpoint, the robot will follow the generated path to visit. The trajectories of the robot and the objects in the scene were visualized through RVIZ, and the environment map was built through the Lidar LDS-01.

5. Conclusions and Future Work

In this paper, the opponent-aware privacy-preserving planning problem in a complex environment is addressed and two questions are answered. Owing to the explosion of privacy preservation in planning, we first define opponent-aware privacy-preserving planning. Then, we present approaches for information sharing restricted task plan generation and observability controlled path plan generation. The final experiments with privacy leakage analysis and the indoor robot demonstration show the applicability of the proposed approaches to generating plans. In fact, many pieces of research have modeled the interaction between patrol UGVs and adversary with Stackelberg or stochastic games, in which agents pursue utility maximization. Additionally, many robust and online goal recognition approaches have been proposed, such as the self-modulating model proposed in [38] for rational and irrational agents. In the future, we will use a stochastic game model with active adversaries to model this problem.