Bisimulation for Secure Information Flow Analysis of Multi-Threaded Programs

Abstract

Preserving the confidentiality of information is a growing concern in software development. Secure information flow is intended to maintain the confidentiality of sensitive information by preventing them from flowing to attackers. This paper discusses how to ensure confidentiality for multi-threaded programs through a property called observational determinism. Operational semantics of multi-threaded programs are modeled using Kripke structures. Observational determinism is formalized in terms of divergence weak low-bisimulation. Bisimulation is an equivalence relation associating executions that simulate each other. The new property is called bisimulation-based observational determinism. Furthermore, a model checking method is proposed to verify the new property and ensure that secure information flow holds in a multi-threaded program. The model checking method successively refines the Kripke model of the program until the quotient of the model with respect to divergence weak low-bisimulation is reached. Then, bisimulation-based observational determinism is checked on the quotient, which is a minimized model of the concrete Kripke model. The time complexity of the proposed method is polynomial in the size of the Kripke model. The proposed approach has been implemented on top of PRISM, a probabilistic model checking tool. Finally, a case study is discussed to show the applicability of the proposed approach.

1. Introduction

The increase in number and variety of security attacks on computing systems amplifies the need for improvement in protection mechanisms against the security attacks. Most of these attacks target the confidentiality of sensitive information.

Cryptography, access control and firewalls are common protection mechanisms against the attacks on confidentiality. These mechanisms allow access (read or write) to the secret information for authorized users only. However, once access to secret information is granted for an authorized user, they do not have control over how the information is used. They have no technique to verify and prevent the flow of secret information to unauthorized users. For example, consider an android game with in-app purchase capability. While installing this app, it asks for permission to access the Internet and credit card information (for in-app purchases, like buying additional coins). Once an android user allows these permissions, they have no way to ensure that the credit card information is used in a legal manner and are not sent to an unauthorized server. This is an example of information flows, which potentially result in dangerous types of attacks, such as code injection or sensitive information leakage [1].

To avoid information flows and stop information leakages, secure information flow has been introduced. Secure information flow is a security mechanism for verifying and ensuring that secret information in a program does not flow to unauthorized users. It assigns two security levels to program variables: some variables are assigned public (low) security level and some are assigned secret (high) security level. The attacker, in secure information flow context, is supposed to know the source code of the program and is able to execute the program and observe values of the public variables. Secure information flow seeks to detect information flows from secret variables to public ones [2,3].

For example, consider the program l:=h, where h is a secret variable and l is a public variable. This program has a direct flow, since the attacker can infer secret information (h) by observing the public variable (l). As another example, consider the program if h>0 then l:=-5 else l:=5, which has an indirect flow. The attacker can infer some secret information (h being positive or not) by observing the value of l. Verifying confidentiality of multi-threaded programs and ensuring secure information flows is the main motivation of this paper.

To ensure secure information flow in multi-threaded programs, a confidentiality property needs to be formalized and a verification method is needed to check whether the program satisfies the property or not.

A commonly used confidentiality property to ensure secure information flow for concurrent programs is Observational determinism [4,5,6]. It requires the program to appear deterministic to an attacker capable of observing values of the public variables during the program execution. There are various definitions of observational determinism [6,7,8,9,10,11]. However, the time complexity of verifying most of these properties is exponential in the size of the state space of the program [10]. Furthermore, they are not restrictive enough in some security contexts to thoroughly avoid information leakages.

In this paper, an automatic approach is proposed to analyze secure information flow for multi-threaded programs. The proposed approach consists of two main parts: (1) a new formalization, Bisimulation-based Observational Determinism (BOD), for specifying secure information flow for multi-threaded programs (Section 4.1) and (2) a polynomial-time algorithm for verifying BOD to ensure that BOD holds in a multi-threaded program (Section 4.2).

Kripke structures are used to model program behavior. Any path in the Kripke structure corresponds to an execution of the program that the Kripke structure is modeling. The sequence of public values of each path denotes a public behavior that is observable by the attacker. We define an equivalence relation, called divergence weak low-bisimulation, between paths of the Kripke structure. Divergence weak low-bisimulation requires paths of the program to affect the public variables in the same way. We formalize BOD using divergence weak low-bisimulation and show that a program satisfies BOD, if and only if all paths that have the same initial public value are divergence weak low-bisimilar. Thus, paths that have the same initial public value pass through the same equivalence classes of states but not at the same time, that is, BOD allows some paths to run slower than others. BOD is defined language- and scheduler-independent.

To verify BOD, a sound model checking method is proposed. The method chooses an arbitrary path for all paths having low-equivalent initial states. BOD requires divergence weak low-bisimilarity between each arbitrary path and the corresponding paths in the program. The method computes a divergence weak low-bisimulation quotient of the program, which is an abstract model of the program and consists of equivalence classes under divergence weak low-bisimulation. We show that BOD is satisfied if and only if initial states of each arbitrary path and the corresponding paths in the program are in the same equivalence classes. The time complexity of computing the quotient and hence verifying BOD is polynomial in the size of the program model.

In summary, our contributions include (1) formalizing observational determinism using a bisimulation foundation, (2) a verification algorithm for checking observational determinism.

The remainder of the paper is structured as follows. Section 2 discusses preliminaries and assumptions made throughout the paper. Section 3 discusses previous work. In Section 4, the proposed approach is explained. In this section bisimulation-based confidentiality property is formally defined using low-bisimulation and then the verification is explained in detail. Finally, Section 5 concludes by a discussion on future work.

2. Preliminaries and Assumptions

A security analysis should include a model for program, attacker and property [12]. Program and attacker models along with preliminaries for formalizing the property are explained in this section. The confidentiality property for specifying requirements is defined in Section 4.1.

2.1. Program Model

We model a program as a set of concurrently running threads with a memory shared across the threads. Kripke structures are utilized to model multi-threaded programs and paths are used to model program behavior (executions).

Definition 1

(Kripke structure). A Kripke structure $\mathcal{K}$ is a tuple $(S,\to ,I,AP,V)$ where S is a set of states, $\to \subseteq S\times S$ is a transition relation, $I\subseteq S$ is a set of initial states, $AP$ is the set of possible values of the public variables and $V:S\to 2^{AP}$ is a labeling function.

$\mathcal{K}$ is called finite if S and $AP$ are finite. The set of successors of a state s is defined as $Post\left(s\right)=\{s^{\prime }\in S|s\to s^{\prime }\}$. A state s is called terminal if $Post\left(s\right)=\varnothing$. For a Kripke structure modeling a sequential program, terminal states represent the termination of the program.

Definition 2

(Path). An path fragment $\widehat{\pi }$ of $\mathcal{K}$ is a finite state sequence $s_0{s}_1\dots s_n$ such that $s_i\in Post\left(s_{i-1}\right)$ for all $0<i\le n$ or an infinite state sequence $s_0{s}_1{s}_2\dots$ such that $s_i\in Post\left(s_{i-1}\right)$ for all $0<i$. An path of $\mathcal{K}$ is a path fragment that starts in an initial state and is either finite, ending in a terminal state or infinite.

The first state of the path $\pi =s_0{s}_1{s}_2\dots$ is extracted by $\pi \left[0\right]$, that is, $\pi \left[0\right]=s_0$. $Paths\left(s\right)$ denotes the set of paths starting in s and $Paths\left(\mathcal{K}\right)$ the set of paths of the initial states of $\mathcal{K}$: $Paths\left(\mathcal{K}\right)={\cup }_{s\in I}Paths\left(s\right)$. Suppose $I^{\prime }$ is a subset of I; Then, $Paths\left(I^{\prime }\right)={\cup }_{s\in I^{\prime }}Paths\left(s\right)$.

A trace of a path $\pi =s_0{s}_1\cdots$ is defined as $T=trace\left(\pi \right)=V\left(s_0\right)V\left(s_1\right)\cdots$. Two traces $T_1$ and $T_2$ over $2^{AP}$ are stutter equivalent, denoted $T_1\triangleq T_2$, if they are both of the form $A_0^{+}{A}_1^{+}{A}_2^{+}\cdots$ for $A_0,A_1,A_2,\cdots \subseteq AP$ where $A_i^{+}$ is the Kleene plus operation on $A_i$ and is defined as $A_i^{+}=\{x_1{x}_2\cdots x_k|k>0\mathrm{and}\mathrm{each}{x}_i=A_i\}$. A finite trace $t_1$ is called a prefix of t, if there exists another infinite trace $t_2$ such that $t_1{t}_2=t$. Two traces are prefix & stutter equivalent, if one is stutter-equivalent to a prefix of another.

Two Kripke structure can be combined in a single composite Kripke structure.

Definition 3

(Composite Kripke structure ${\mathcal{K}}_1$ ⊕ ${\mathcal{K}}_2$). For ${\mathcal{K}}_i=(S_i,{\to }_i,I_i,AP,V_i)$, $i=1,2$: ${\mathcal{K}}_1\oplus {\mathcal{K}}_2=(S_1\uplus S_2,{\to }_1\uplus {\to }_2,I_1\uplus I_2,AP,V)$ where ⊎ stands for disjoint union and $V\left(s\right)=V_i\left(s\right)$ if $s\in S_i$.

  Let us assume $\mathcal{K}=(S,\to ,I,AP,V)$ is a Kripke structure that models executions of a multi-threaded program, with all possible interleavings of the threads. A state of $\mathcal{K}$ indicates the current values of all variables together with the current value of the program counter that indicates the next program statement to be executed. Execution steps of the program are modeled by the transition relation. In case a state has more than one outgoing transition, the next execution step (transition) is chosen in a purely non-deterministic fashion. In consequence, executions result from the resolution of the possible non-determinism in the program. This resolution is performed by a scheduler. A scheduler chooses in any state s one of the enabled transitions according to a scheduling policy. Probabilistic choices are not considered, that is, the scheduler is possibilistic. $AP$ is the set of values of the public variables and the function V labels each state with these values. If $\mathcal{K}$ has a terminal state $s_n$, we include a transition $s_n\to s_n$, that is, a self-loop, ensuring that the Kripke structure has no terminal state. Therefore, all paths of $\mathcal{K}$ are infinite. It is assumed that the state space of the Kripke structure of the program and the shared memory used by the threads are finite.

As an example of program modeling, consider the following program which consists of two threads

 l:=0;
 l:=1  ||  if l=1 then l:=h                          (P1)

Suppose that h is a one-bit secret variable, l is a one-bit public variable and || is the parallel operator. Two threads are secure if they are executed separately but concurrent execution of the two threads might leak the value of h into l. The Kripke structure of the program is depicted in Figure 1.

In this Kripke structure, the state space is $S=\{s_0,s_1,\dots ,s_8\}$. The set of initial states consists of two states, that is, $I=\{s_0,s_5\}$. The set of transitions is $\{s_0\to s_1,s_1\to s_3,\dots ,s_7\to s_8,s_8\to s_8\}$. The set of possible values of the public variable is $AP=\{0,1\}$. States $s_0$, $s_1$, $s_4$, $s_5$ and $s_6$ are labeled with $\left\{0\right\}$ and states $s_2$, $s_3$, $s_7$ and $s_8$ are labeled with $\left\{1\right\}$. The set of paths of the program is $\{s_0{s}_1{s}_3^{\omega },s_0{s}_2{s}_4^{\omega },s_5{s}_6{s}_8^{\omega },s_5{s}_7{s}_8^{\omega }\}$, where $\omega$ denotes infinite iteration.

2.2. Attacker Model

Secure information flow is a security mechanism for establishing program confidentiality. The sufficiency of information flow depends on the attacker model. This model defines the capabilities of the attacker, such as being able to observe program output, read program code or even inject code in the program. These capabilities may lead to information flow channels that transfer secret information to the attacker. Examples of such channels are direct, indirect, termination behavior, probabilistic, internally observable timing, externally observable timing [13], injection and power [14].

Direct flows are caused when the value of a secret variable is directly assigned to a public variable. Indirect flows are caused by the control structure of the program. Termination flows leak information through the termination or non-termination of a program execution [3]. Probabilistic channels signal secret information through the probabilistic behavior of the program. Internally observable timing flows are created when secret information influences the timing behavior of a thread, which, through the scheduler, influences the execution order of assignments to public variables [15]. Externally observable timing flows are caused when secret information influences the timing behavior of the program. Injection flows signal information when an attacker is able to inject her desired code in the program [14]. Power channels leak information via the power consumed to execute an action dependent on secret information, assuming the attacker can measure this consumption [3].

Which of these channels are a concern depends strongly on the specific context and on the power of the attacker, that is, on the observations she is able to make. We suppose that the attacker has the knowledge of the program’s source code, is able to choose a scheduling policy, run the program and observe program traces.

We assume a shared memory for the multi-threaded program, where public values are stored and the attacker has read access to it. Stating in terms of Kripke structures, the attacker can only observe state labels. We also assume that the attacker can not access other memory areas, including secret variables (i.e., access control and memory protection works correctly). We aim to detect direct, indirect and internal timing leaks.

2.3. Low-Bisimulation

In this section, low-bisimulation and variations of it are formally defined. These definitions will be used in the next section to formalize observational determinism. Execution (path) indistinguishability is a key requirement for preventing information leaks in concurrent programs [6]. Bisimulation is a widely-used foundation for execution indistinguishability and thus for characterizing security for multi-threaded programs (e.g., References [13,16,17,18]).

We assume the attacker can observe the state labels and cannot distinguish those states that have the same label but might have different secret values. This observational capability of the attacker is formalized by the following relation.

Definition 4

(Low-equivalent states). A state s is low-equivalent to another state $s^{\prime }$, written $s{=}_L{s}^{\prime }$, if $V\left(s\right)=V\left(s^{\prime }\right)$.

We believe a program is secure if all executions (paths) having low-equivalent initial states are divergence weak low-bisimilar. Thus, we define observational determinism in terms of divergence weak low-bisimulation. For further clarity on the concept of divergence weak low-bisimulation, we first define weak low-bisimulation. Weak low-bisimulation is defined as a relation between states of a Kripke structure and requires for low-equivalent states that each transition is matched by a (suitable) path fragment.

Definition 5

(Weak low-bisimulation ≈_L). Let $\mathcal{K}=(S,\to ,I,AP,V)$ be a Kripke structure. A weak low-bisimulation for $\mathcal{K}$ is a binary relation R on S such that for all $s_{1}R{s}_2$, the following three conditions hold:

1.

$s_1{=}_L{s}_2$.

2.

If $s_1^{\prime }\in Post\left(s_1\right)$ with $(s_1^{\prime },s_2)\notin R$, then there exists a finite path fragment $s_2{u}_1\dots u_n{s}_2^{\prime }$ with $n\ge 0$ and $s_{1}R{u}_i$, $i=1,\dots ,n$ and $s_1^{\prime }R{s}_2^{\prime }$.

3.

If $s_2^{\prime }\in Post\left(s_2\right)$ with $(s_1,s_2^{\prime })\notin R$, then there exists a finite path fragment $s_1{v}_1\dots v_n{s}_1^{\prime }$ with $n\ge 0$ and $V_{i}R{s}_2$, $i=1,\dots ,n$ and $s_1^{\prime }R{s}_2^{\prime }$.

States $s_1$ and $s_2$ are weak low-bisimilar, denoted $s_1{\approx }_L{s}_2$, if there exists a weak low-bisimulation R for $\mathcal{K}$ with $s_{1}R{s}_2$.

Weak low-bisimulation is defined as a relation between states within a single Kripke structure. An alternative perspective is to consider low-bisimulation as a relation between two Kripke structures. This enables comparing different Kripke structures. Take Kripke structures ${\mathcal{K}}_1$ and ${\mathcal{K}}_2$ and combine them in a single composite Kripke structure ${\mathcal{K}}_1\oplus {\mathcal{K}}_2$ (see Definition 3). Then, ${\mathcal{K}}_1{\approx }_L{\mathcal{K}}_2$ if and only if, for every initial state $s_1$ of ${\mathcal{K}}_1$, there exists a weak low-bisimilar initial state $s_2$ of ${\mathcal{K}}_2$, and vice versa.

Two Kripke structures can be weak low-bisimilar (initial states can be weak low-bisimilar) but may not produce indistinguishable paths. This is caused by stutter paths, that is, paths that stay forever in an equivalence class without performing any visible step. This behavior is called divergent. We would like to adapt weak low-bisimulation such that states may only be related if they both exhibit divergent paths or none of them has a divergent path. This yields a variant of weak low-bisimulation called divergence weak low-bisimulation.

Definition 6

(Divergence weak low-bisimulation ${\mathbf{\approx }}_{\mathit{L}}^{\mathit{d}\mathit{i}\mathit{v}}$ [19]). Let R be an equivalence relation on S. A state $s\in S$ is R-divergent if there exists an infinite path fragment $\pi =s{s}_1{s}_2\dots \in Paths\left(s\right)$ such that $sR{s}_j$ for all $j>0$. Stated in words, a state s is R-divergent if there is an infinite path starting in s that only visits states in ${\left[s\right]}_R$. ${\left[s\right]}_R$ is the equivalence class of s under the equivalence relation R. R is divergence-sensitive if for any $s_{1}R{s}_2$: if $s_1$ is R-divergent, then $s_2$ is R-divergent. States $s_1$, $s_2$ are divergence weak low-bisimilar, denoted $s_1{\approx }_L^{div}{s}_2$, if there exists a divergence sensitive weak low-bisimulation R such that $s_{1}R{s}_2$.

We defined ${\approx }_L^{div}$ between two states. It can also be defined between two paths.

Definition 7

(Divergence weak low-bisimilar paths [19]). For infinite path fragments ${\pi }_i=s_{0,i}{s}_{1,i}{s}_{2,i}\dots$, $i=1,2$ in $\mathcal{K}$, ${\pi }_1$ is divergence weak low-bisimilar to ${\pi }_2$, denoted ${\pi }_1{\approx }_L^{div}{\pi }_2$ if and only if there exists an infinite sequence of indexes $0=j_0<j_1<j_2<\dots$ and $0=k_0<k_1<k_2<\dots$ with:

$$s_{j,1}{\approx }_L^{div}{s}_{k,2}forall{j}_{r-1}\le j<j_{r}and{k}_{r-1}\le k<k_{r}withr=1,2,\dots$$

The following lemma asserts that divergence stutter low-bisimilar states produce paths that are divergence stutter low-bisimilar. It also asserts that initial states of divergence stutter low-bisimilar paths are themselves divergence stutter low-bisimilar. The fact that ${\approx }_L^{div}$ can be lifted from states to paths and vice versa will used in proving the correctness of the verification algorithm in Section 4.2.

Lemma 1.

Divergence weak low-bisimilar states have divergence weak low-bisimilar paths and vice versa:

$$s_1{\approx }_L^{div}{s}_{2}iff\forall {\pi }_1\in Paths\left(s_1\right)(\exists {\pi }_2\in Paths\left(s_2\right).{\pi }_1{\approx }_L^{div}{\pi }_2)$$

Proof. 

See Baier and Katoen [19], page 550. □

Divergence weak low-bisimulation is an equivalence and partitions the set of states of a Kripke structure into equivalence classes. The result is called the quotient Kripke structure. For a state set S and an equivalence R, ${\left[s\right]}_R=\{s^{\prime }\in S|(s,s^{\prime })\in R\}$ defines the equivalence class of the state $s\in S$ under R. The quotient Kripke structure with respect to ${\approx }_L^{div}$ is defined as follows.

Definition 8 

(Divergence weak low-bisimulation quotient $\mathcal{K}/{\approx }_L^{div}$). The divergence weak low-bisimulation quotient of a Kripke structure $\mathcal{K}$ is defined by $\mathcal{K}/{\approx }_L^{div}=(S/{\approx }_L^{div},{\to }^{\prime },I^{\prime },AP,V^{\prime })$, where $S/{\approx }_L^{div}=\left\{{\left[s\right]}_{{\approx }_L^{div}}\right|s\in S\}$, $V^{\prime }\left({\left[s\right]}_{{\approx }_L^{div}}\right)=V\left(s\right)$, $I^{\prime }=\left\{{\left[s\right]}_{{\approx }_L^{div}}\right|s\in I\}$ and ${\to }^{\prime }$ is defined by

$${\displaystyle \frac{s\to s^{\prime }\wedge s\neg {\approx }_L^{div}{s}^{\prime }}{{\left[s\right]}_{{\approx }_L^{div}}{\to }^{\prime }{\left[s^{\prime }\right]}_{{\approx }_L^{div}}}}and{\displaystyle \frac{sis{\approx }_L^{div}-divergent}{{\left[s\right]}_{{\approx }_L^{div}}{\to }^{\prime }{\left[s\right]}_{{\approx }_L^{div}}}}$$

3. Related Work

In the literature, there are various definitions of observational determinism. These definitions formalize determinism in terms of various forms of stutter equivalence of the program executions, such as prefix and stutter equivalence on traces of each public variable [6], stutter equivalence on traces of each public variable [7], prefix and stutter equivalence on traces of all public variables together [8], stutter equivalence on traces of all public variables [9,20], scheduler-specific stutter equivalence on traces of each public variable and also traces of all public variables together [10] and equivalence to public operations [11].

For most of these definitions, model checking algorithms have been presented to verify them [7,9,10]. Huisman and Blondeel [9] specify observational determinism in modal $\mu$-calculus. They use Concurrency Workbench (CWB) as the model checking tool. They encode the self-composed program and modal $\mu$-calculus formulation of observational determinism in CWB’s specification language. Dabaghchian and Abdollahi [20] specify observational determinism in Linear Temporal Logic. They encode the self-composed program and LTL formula in the PROMELA specification language, using the SPIN model checker. Ngo et al. [10] present two bisimulation-based algorithms to verify observational determinism. These model checking methods have exponential time complexity, while BOD requires polynomial time complexity. BOD is a little more restrictive than these definitions, as it uses divergence weak low-bisimulation (Divergence weak low-bisimulation implies stutter equivalence but the reverse does not necessarily hold (See Baier and Katoen [19], page 549).). This restriction is not bad, as most applications require strict confidentiality properties in order to avoid leakage of sensitive information.

Giffhorn and Snelting [21] define observational determinism in terms of low-equivalence on public operations and present a program analysis algorithm for verifying it. The algorithm models concurrent programs as dependence graphs. In a program dependence graph, nodes represent program statements and edges represent data and control dependencies. Giffhorn and Snelting’s definition of observational determinism is based on traces which consist of read/write operations and memory values and are enriched with control and data dependencies. This is different from our definition of BOD and many other definitions of observational determinism, in which traces are defined using public variable values.

Another widely-used scheduler-independent property is strong security, introduced by Sabelfeld and Sands [13]. They define a partial equivalence relation, called strong low-bisimulation, that relates two multi-threaded programs, with the same number of threads, only if they execute in lock-step and affect the public variables in the same manner. Then, a multi-threaded program satisfies strong security if it is related to itself. Strong security is too strong and requires step-by-step indistinguishability, which implies, many intuitively secure programs are rejected. Compared to strong security, BOD is much more permissive on harmless programs.

Mantel and Sudbrock [16] propose Flexible Scheduler-Independent (FSI) Security for multi-threaded programs. They partition threads of the program into high threads that definitely do not modify public variables and public threads that potentially modify public variables. An order-preserving bijection, called low matching, is defined to map the positions of public threads in two multi-threaded programs. Then, they define a partial equivalence relation, called low-bisimulation modulo low matching, which relates two multi-threaded programs, with the same number of threads, if each step of a public thread in one execution is matched by a step of the matching public thread in the other execution. Thus, a multi-threaded program is FSI-secure if it is related to itself. FSI-security is less restrictive than strong security but is language-dependent. Note that BOD is language-independent.

Type systems have been widely used for verifying secure information flow properties [8,13,16,18,22,23,24]. They are language-dependent, in the sense that a simple language is considered and the confidentiality property is defined using the semantics of this language. Type systems are useful because they support automated, compositional verification. However, they are not extensible, as they are language-dependent and for each change in the program language or the security property, a new type system needs to be defined and proven sound [25]. On the other hand, security requirements are subject to dynamic changes. Accordingly, we use algorithmic verification and model checking, instead of type systems to verify secure information flow.

Another verification approach for information flow properties is to extend temporal logics and introduce new logics to specify these properties. HyperLTL [26] is a recently introduced temporal logic, which is an extension of Linear-time Temporal Logic (LTL) [27]. Runtime verification of information flow properties, including observational determinism, using HyperLTL is discussed in [28,29]. Verifying BOD using HyperLTL would be an interesting future work.

4. The Proposed Approach

The proposed approach for verifying observational determinism consists of two main parts: (1) a new formalization, Bisimulation-based Observational Determinism (BOD), for specifying secure information and (2) an algorithm for verifying BOD. BOD is discussed in Section 4.1 and the verification is explained in Section 4.2.

4.1. Bisimulation-Based Observational Determinism

Observational determinism needs to ensure that as secret inputs change, public behavior of the program remains unchanged. Thus, it requires paths with low-equivalent initial states to be indistinguishable [6]. We characterize indistinguishability of paths by divergence weak low-bisimulation.

Definition 9 

(BOD). A multi-threaded program $MT$ satisfies BOD with respect to all public variables, if and only if

$$\forall \pi ,{\pi }^{\prime }\in Paths\left(\mathcal{K}\right).\pi \left[0\right]{=}_L{\pi }^{\prime }\left[0\right]⟹\pi {\approx }_L^{div}{\pi }^{\prime }$$

where $\mathcal{K}$ denotes the Kripke structure of the program $MT$, ${=}_L$ is state low-equivalence relation, and ${\approx }_L^{div}$ is divergence weak low-bisimulation relation. BOD is acronym for Bisimulation-based Observational determinism.

Stated in words, BOD requires two initial states that have the same public data but different secret data, to produce paths that visit the same sequence of ${\approx }_L^{div}$-equivalence classes and affect the public variables in the same sequence. This implies weak stepwise indistinguishability, by which direct, indirect and internally observable timing flows are detected. Termination behavior flows are eliminated by adding self-loops to terminal states. As discussed earlier, paths of the program result from the resolution of the possible non-determinism in the program and the probability of a path is not considered. This research hence does not consider probabilistic flows. In order to handle externally observable timing flows, the confidentiality property must be too restrictive, ensuring that the execution time does not depend on secret inputs. Requiring a high degree of restriction causes rejection of many intuitively secure programs and harms the precision of the property. That is why BOD does not impose a high degree of restriction. A confidentiality property should be as restrictive as possible in the sense that it does not accept leaky programs and should be permissive enough, such that it does not reject intuitively secure programs.

As an example of information flow analysis using BOD, consider the following program

 l:=0; h:=l+3; l:=l+1                                      (P2)

The program is intuitively secure because h is not read. The Kripke structure of the program is depicted in Figure 2.

It has just one path $s_0{s}_1{s}_2^{\omega }$. Thus, P2 satisfies BOD. As another example, consider the following secure program in which l is not updated

 if l>0 then h:=h+1
         else h:=0                                     (P3)

This program is clearly BOD-secure, because for each value of l, the Kripke structure has one state with a self-loop.

For insecure examples, consider the program P (Figure 1). Two threads are secure if they are executed separately. However, concurrent execution of the two threads might leak the value of h into l. The paths of this program are not divergence weak low-bisimilar and hence not BOD-secure. As another example, consider the following program

 l:=0; while h>0 do {l++; h--}                                 (P4)

This program is insecure and leaks the value of h into l. The Kripke structure of the program is shown in Figure 3.

The set of paths is $\{s_0^{\omega },s_1{s}_2^{\omega },s_3{s}_4{s}_5^{\omega },\dots \}$. These paths are not divergence weak low-bisimilar and thus BOD correctly recognizes the program as insecure. For an example of externally observable timing flow, consider the following program

 l:=0; if h>0 then l:=1
          else {sleep 100; l:=1}                             (P5)

where sleep 100 means 100 consecutive skip commands. The Kripke structure of the program is depicted in Figure 4.

Paths produced are $s_0{s}_1^{\omega }$ and $s_2{s}_3$ …$s_{102}{s}_{103}^{\omega }$, which are divergence weak low-bisimilar. Therefore, BOD (incorrectly) classifies the program as secure, while the program has external timing leak. As discussed earlier, BOD (and all other definitions of observational determinism) do not try to detect external timing leaks and avoid them.

For an example of internally observable timing flow, consider the following insecure program

 l:=0
 l:=2  ||  (if h=1 then sleep 100); l:=1                        (P6)

A part of the Kripke structure is depicted in Figure 5.

Due to the fact that, for example, two paths $s_0{s}_1{s}_2{s}_3^{\omega }$ and $s_0{s}_4{s}_5{s}_6^{\omega }$ are not divergence weak bisimilar, BOD correctly recognizes the program as insecure.

4.2. Verifying BOD

Here we discuss the proposed model checking algorithm for BOD verification. First, an arbitrary path ${\pi }_i$ is extracted for each set of low-equivalent initial states and ${\approx }_L^{div}$ is checked between the arbitrary path ${\pi }_i$ and each path starting in these low-equivalent initial states.

A main step of the verification algorithm is to construct the quotient structure of the program w.r.t ${\approx }_L^{div}$. Divergence weak low-bisimulation quotient is an abstraction of the Kripke structure and by considering it, enormous state-space reductions may be obtained [19]. We use an abstraction refinement technique to compute the quotient structure.

4.2.1. The Algorithm

The main steps of the verification algorithm are outlined in Algorithm 1.

Algorithm 1 Verification of BOD

Input: finite Kripke structure $\mathcal{K}$ modeling the program Output:true if the program satisfies BOD; Otherwise, false Partition I into initial state blocks $IS{C}_0$,…,$IS{C}_m$; Define $ISC=\{IS{C}_0,\dots ,IS{C}_m\}$; Take an arbitrary path ${\pi }_i\in Paths\left(IS{C}_i\right)$ for each $IS{C}_i(i=0,\dots ,|ISC|-1)$; Construct Kripke structure ${\mathcal{KP}}_i$ from ${\pi }_i$; Construct Kripke structure ${\mathcal{KP}}_{min}={\mathcal{KP}}_0\oplus {\mathcal{KP}}_1\dots \oplus {\mathcal{KP}}_{\left|ISC\right|-1}$; Construct Kripke structure ${\mathcal{K}}^{\prime }=\mathcal{K}\oplus {\mathcal{KP}}_{min}$; Compute divergence weak low-bisimulation quotient ${\mathcal{K}}^{\prime }/{\approx }_L^{div}$; for each initial state block $IS{C}_i$ and the corresponding path ${\pi }_i$ do     $clas{s}_i={\left[{\pi }_i\left[0\right]\right]}_{{\approx }_L^{div}}$; /* equivalence class of ${\pi }_i\left[0\right]$ w.r.t. ${\approx }_L^{div}$ */     if $IS{C}_i\neg \subseteq clas{s}_i$ then         return false     end if end for return true

The input of the algorithm is a finite Kripke structure $\mathcal{K}=(S,\to ,I,AP,V)$ modeling the program and the output is true or false. I is partitioned into sets of low-equivalent initial states called initial state blocks$IS{C}_0$,…,$IS{C}_m$ and $ISC=\{IS{C}_0,\dots ,IS{C}_m\}$ is defined. An arbitrary path ${\pi }_i\in Paths\left(IS{C}_i\right)$ is extracted from $\mathcal{K}$ for each $IS{C}_i$ and a Kripke structure ${\mathcal{KP}}_i$ is built from ${\pi }_i$. Kripke structures ${\mathcal{KP}}_i(i=0,\dots ,|ISC|-1)$ are combined, forming a single Kripke structure ${\mathcal{KP}}_{min}=(S_{{\mathcal{KP}}_{min}},{\to }_{{\mathcal{KP}}_{min}},I_{{\mathcal{KP}}_{min}},A{P}_{{\mathcal{KP}}_{min}},V_{{\mathcal{KP}}_{min}})$, where ${\mathcal{KP}}_{min}={\mathcal{KP}}_0\oplus {\mathcal{KP}}_1\oplus \dots \oplus {\mathcal{KP}}_{\left|ISC\right|-1}$. Kripke structure ${\mathcal{K}}^{\prime }=(S^{\prime },{\to }^{\prime },I^{\prime },AP,V^{\prime })$ is also constructed, where ${\mathcal{K}}^{\prime }=\mathcal{K}\oplus {\mathcal{KP}}_{min}$ and quotient of ${\mathcal{K}}^{\prime }$ w.r.t. ${\approx }_L^{div}$ is computed. Then, $\mathcal{K}$ satisfies BOD iff

$$\forall i\in \{0,\dots ,|ISC|-1\}.IS{C}_i\subseteq {\left[{\pi }_i\left[0\right]\right]}_{{\approx }_L^{div}}$$

where ${\left[{\pi }_i\left[0\right]\right]}_{{\approx }_L^{div}}$ denotes the equivalence class of ${\pi }_i\left[0\right]$ in ${\mathcal{K}}^{\prime }/{\approx }_L^{div}$. Stated in words, $\mathcal{K}$ satisfies BOD if after computing the divergence weak low-bisimulation quotient of ${\mathcal{K}}^{\prime }=\mathcal{K}\oplus {\mathcal{KP}}_{min}$, all initial states of an initial state block and the initial state of the corresponding arbitrary path belong to the same equivalence class.

Now we discuss the main steps of Algorithm 1.

Taking an arbitrary path. To take an arbitrary path from $\mathcal{K}$, a depth-first search is done. The search starts from an initial state and stops at a terminal state (a state with a self-loop).

Computing the divergence weak low-bisimulation quotient. Before explaining how to compute the quotient with respect to ${\approx }_L^{div}$, some definitions are provided.

Definition 10 

(Stutter cycle). A stutter cycle is a cycle $s_0{s}_1\dots s_n$ in $\mathcal{K}$ such that $s_0{=}_L{s}_i$ for $i=1,\dots ,n$.

Definition 11 

(Divergence-sensitive Expansion $\overline{\mathcal{K}}$). The divergence-sensitive expansion of finite Kripke structure $\mathcal{K}=(S,\to ,I,AP,V)$ is $\overline{\mathcal{K}}=(S\cup \left\{s_{div}\right\},\to ,I,AP\cup \left\{div\right\},\overline{V})$, where $s_{div}\notin S$, → extends the transition relation of $\mathcal{K}$ by the transitions $s_{div}\to s_{div}$ and $s\to s_{div}$ for every state $s\in S$ on a stutter cycle in $\mathcal{K}$ and $\overline{V}\left(s\right)=V\left(s\right)$ if $s\in S$ and $\overline{V}\left(s_{div}\right)=\left\{div\right\}$.

In order to compute the quotient with respect to ${\approx }_L^{div}$, ${\mathcal{K}}^{\prime }$ is transformed into divergence-sensitive expansion $\overline{{\mathcal{K}}^{\prime }}$ such that the equivalence classes under ${\approx }_L$ in $\overline{{\mathcal{K}}^{\prime }}$ coincide with the equivalence classes under ${\approx }_L^{div}$ in ${\mathcal{K}}^{\prime }$. To construct divergence-sensitive expansion, all states on a stutter cycle must be determined, as a transition from each of these states to $s_{div}$ will be added. This is done by finding the strongly connected components (SCCs) in ${\mathcal{K}}^{\prime }$ that only contain stutter steps (Transition $s\to s^{\prime }$ is a stutter step if $s{=}_L{s}^{\prime }$.). The latter can be carried out using a depth-first search algorithm. Then, computation of the quotient with respect to ${\approx }_L^{div}$ is reduced to the problem of computing the quotient with respect to ${\approx }_L$. The algorithm proposed by Groote and Vaandrager [30] is used to compute the quotient of $\overline{{\mathcal{K}}^{\prime }}$ with respect to ${\approx }_L$. The algorithm starts by partitioning the state space based on the low-equivalence relation. Thus, the first partition contains blocks of low-equivalent states. Then, each block is refined based on the set of reachable states from the block. Refinement continues until no refinement is possible and the blocks are stable. For more details of the algorithm please refer to Groote and Vaandrager [30] and Baier and Katoen [19]. Finally, ${\mathcal{K}}^{\prime }/{\approx }_L^{div}$ is obtained from $\overline{{\mathcal{K}}^{\prime }}/{\approx }_L$ by replacing the transitions $s\to s_{div}$ in $\overline{{\mathcal{K}}^{\prime }}$ with self-loops ${\left[s\right]}_{{\approx }_L^{div}}\to {\left[s\right]}_{{\approx }_L^{div}}$ and removing the state $s_{div}$.

4.2.2. Correctness of the Algorithm

Correctness of Algorithm 1 is proven by the following theorem. Algorithm 1 is correct when it returns true if and only if the input Kripke structure satisfies BOD, otherwise returns false.

Theorem 1.

A Kripke structure $\mathcal{K}$ satisfies BOD iff

$$\forall i\in \{0,\dots ,|ISC|-1\}.IS{C}_i\subseteq {\left[{\pi }_i\left[0\right]\right]}_{{\approx }_L^{div}}$$

Proof. 

Given $\mathcal{K}=(S,\to ,I,AP,V)$, BOD requires that all path starting in low-equivalent initial states are divergence weak low-bisimilar. To prove BOD, for each $IS{C}_i$ ($i=0,\dots ,\left|ISC\right|-1$) an arbitrary path ${\pi }_i\in Paths\left(IS{C}_i\right)$ is extracted and divergence weak low-bisimilarity between ${\pi }_i$ and each path $\sigma \in Paths\left(IS{C}_i\right)$ is checked. ${\pi }_i$ is divergence weak low-bisimilar to $\sigma$, if and only if ${\pi }_i\left[0\right]$ is divergence weak low-bisimilar to $\sigma \left[0\right]$. The latter fact is developed as a result of Lemma 1. Then, ${\pi }_i\left[0\right]$ is divergence weak low-bisimilar to each $\sigma \left[0\right]$, if and only if

$$\forall C\in S^{\prime }/{\approx }_L^{div}.{\pi }_i\left[0\right]\in C⟺\sigma \left[0\right]\in C$$

where $S^{\prime }/{\approx }_L^{div}$ is the quotient of $S^{\prime }$ w.r.t ${\approx }_L^{div}$, which is an equivalence class. Note that all states of an initial state cluster should have divergence weak low-bisimilar paths and hence belong to the same equivalence class. The above condition ensures the latter too.

Thus, BOD requires that all initial states of an initial state block $IS{C}_i$ and the initial state of the corresponding arbitrary path ${\pi }_i$ belong to the same block in ${\mathcal{K}}^{\prime }/{\approx }_L^{div}$. Then, $\mathcal{K}$ satisfies BOD, iff

$$\forall i\in \{0,\dots ,|ISC|-1\}.IS{C}_i\subseteq {\left[{\pi }_i\left[0\right]\right]}_{{\approx }_L^{div}}$$

where ${\left[{\pi }_i\left[0\right]\right]}_{{\approx }_L^{div}}$ denotes the equivalence class of ${\pi }_i\left[0\right]$ in ${\mathcal{K}}^{\prime }/{\approx }_L^{div}$. □

4.2.3. Complexity of the Algorithm

Assume t is the number of transitions of $\mathcal{K}$. The time complexity of taking an arbitrary path is $O(t+|S\left|\right)$. Thus, the time complexity of constructing ${\mathcal{KP}}_{min}$ is $O\left(\right|I|\ast (t+\left|S\right|\left)\right)$. The time complexity of determining SCCs in ${\mathcal{K}}^{\prime }$ is $O(t^{\prime }+|S^{\prime }\left|\right)$, where $t^{\prime }$ denotes the number of transitions of ${\mathcal{K}}^{\prime }$. The quotient space of ${\mathcal{K}}^{\prime }$ under ${\approx }_L^{div}$ can be computed in time $O\left(\right(|S^{\prime }|+t^{\prime })+|S^{\prime }|\ast (\left|AP\right|+t^{\prime }\left)\right)$ under the assumption that $t^{\prime }\ge \left|S^{\prime }\right|$. Thus, the costs of verifying BOD are dominated by the costs of finding the SCCs and computing the quotient space under ${\approx }_L^{div}$, which are both polynomial-time.

4.2.4. Implementation and Case Study

We have implemented the proposed approach upon the PRISM model checker [31]. PRISM is a tool for modeling and analyzing concurrent and probabilistic systems [31]. It includes a language, called the PRISM language, which is a state-based language for specifying systems and programs. The PRISM tool uses binary decision diagrams to build a state model of a PRISM program and store states and transitions of the model. A PRISM program can contain a set of modules which run in parallel. The overall model of the program contains all possible transitions and interleavings of the modules. A PRISM program can also contain global variables, which can be accessed and modified by all modules of the program.

We use PRISM modules to represent threads of a multi-threaded program. Global variables represent the shared memory of the multi-threaded program. Then, the state model (Kripke structure) is built as the parallel composition of the modules containing all possible interleavings. We changed parser of PRISM and added two reserved keywords observable and secret to indicate public (observable) and secret variables.

For evaluation, we take a case study (described below) and specify it in the PRISM language. We run PRISM to build the Kripke structure of the program. We extract the set of reachable states and create a sparse matrix containing the transitions. We then traverse the Kripke, take an arbitrary path for each initial state cluster and add the path to the Kripke with new state numbers. The divergence weak low-bisimulation quotient of the new model is computed and BOD is checked.

As a case study, consider the following program, which is borrowed from [23,32]. The program—which we call the SmithVolpano program—consists of three threads.

 Thread α:
     while mask != 0 do
          while trigger0 = 0 do; /* busy waiting */
          result := result | mask ; /* bitwise ‘or’ */
          trigger0 := 0;
          maintrigger := maintrigger + 1;
          if maintrigger = 1 then trigger1 := 1
		   
 Thread β:
     while mask != 0 do
          while trigger1 = 0 do; /* busy waiting */
          result := result & ~ mask ; /* bitwise ‘and’ with the complement of mask */
          trigger1 := 0;
          maintrigger := maintrigger + 1;
          if maintrigger = 1 then trigger0 := 1
		   
 Thread γ:
     while mask != 0 do
          maintrigger := 0;
          if (PIN & mask ) = 0 then trigger0 := 1
                     else trigger1 := 1;
          while maintrigger != 2 do; /* busy waiting */
          mask  := mask / 2;
	  trigger0 := 1;
          trigger1 := 1

Assume that PIN is a secret variable and result is a public variable. If the scheduling of the threads is fair, that is, each thread gets its turn infinitely often and the program starts in an initial state where maintrigger=0, trigger0=0, trigger1=0 and result=0, the program leaks some bits of PIN into result. If PIN is n bits long and the initial value of mask is equal to $2^k$ ($k<n$), then SmithVolpano leaks $k+1$ bits of PIN into result.

The PRISM description for the SmithVolpano program is given in Appendix A. It is composed of three modules Alpha, Beta and Gamma, where each module models a thread. For the sake of brevity, we only allow modules to change turn when they are in the busy waiting state. The global variables result, mask, pin, trigger0, trigger1, maintrigger and turn encode the shared variables (memory) of the program. States of the model are represented by the values of its variables, that is, (result, mask, pin, trigger0, trigger1, maintrigger, turn, c1, c2, c3).

For $n=2$, the Kripke structure of the SmithVolpano program has 228 states and 236 transitions. It contains 4 initial states, in which result (the public variable) is 0 and PIN (the secret variable) varies between 0 and 3. Thus, there is one initial state cluster and a path, containing 46 states, is extracted and added to the Kripke structure. Then, the quotient is computed. It contains 16 blocks. Since the initial state of the extracted path and all initial states of the Kripke do not belong to the same block, then the approach labels the program as BOD-insecure.

Each increase in n (i.e., the bit size of PIN) doubles the number of states and transitions of the Kripke structure. However, in all cases the computed quotient contains 16 blocks and the program is labeled as BOD-insecure.

Since the approach uses binary decision diagrams to construct the Kripke model and a sparse matrix is utilized to access the Kripke’s transitions, it computes the quotient and verifies BOD in seconds.

To our knowledge, no other evaluations of BOD have been published and we cannot perform a quantitative comparison of our implementation to other algorithms. However, we compare our approach to a closely related work that has been done by Ngo [32]. Ngo discusses the SmithVolpano program as a case study for his work on observational determinism. He formalizes observational determinism by stutter equivalence on traces of each public variable and also traces of all public variables together. Then, he presents two algorithmic verification methods for checking observational determinism. He has implemented his methods on the LTSmin-check tool [33]. As a case study, he specifies the SmithVolpano program in the PRISM language and uses the PRISM tool to export its state machine into three text files. His method then reads the state machine from the text files and uses the LTSmin-check tool to verify observational determinism. The verification algorithms have exponential time complexity, while our algorithm is polynomial. Furthermore, Ngo’s method reads the input model from three text files, while our method builds the model on-the-fly using binary decision diagrams. This makes the time complexity of our method far faster than Ngo’s methods.

5. Conclusions and Future Work

Aiming at a widely-applicable scheduler-independent analysis for secure information flow, a bisimulation-based foundation was proposed in terms of the semantics of state transition systems. Concretely, BOD was formalized to specify secure information flow by requiring indistinguishability of executions of the program. Indistinguishability was characterized in terms of divergence weak low-bisimulation between paths of the transition system of the program. Then, a model checking algorithm was proposed to verify BOD. The algorithm constructs an abstraction of the input model and then checks BOD on it. We proved the correctness of the algorithm and discussed its time complexity. Finally, the implementation and a case study was discussed. We showed how the proposed approach can be used to model and analyze secure information flow of multi-threaded programs.

As future work, we plan to develop a compositional analysis for multi-threaded programs, as compositionality can be important in making the analysis scale. We believe thread-modular verification [34] is one good candidate for such an analysis. One can also use temporal logics, such as HyperLTL [26], to logically specify BOD and then use model checking methods and tools to check BOD.