A Differentially Private (Random) Decision Tree without Noise from k-Anonymity ^†

Abstract

This paper focuses on the relationship between decision trees, a typical machine learning method, and data anonymization. It is known that information leaked from trained decision trees can be evaluated using well-studied data anonymization techniques and that decision trees can be strengthened using k-anonymity and ℓ-diversity; unfortunately, however, this does not seem sufficient for differential privacy. In this paper, we show how one might apply k-anonymity to a (random) decision tree, which is a variant of the decision tree. Surprisingly, this results in differential privacy, which means that security is amplified from k-anonymity to differential privacy without the addition of noise.

1. Introduction

Recently, with the rapid evolution of machine learning technology and the expansion of data due to developments in information technology, it has become increasingly important that companies determine how they might utilize big data effectively and efficiently. However, big data often include personal and private information; thus, careless utilization of such sensitive information may lead to unexpected sanctions.

To overcome this problem, many privacy-preserving technologies have been proposed for the utilization of data that nevertheless maintains privacy. Typical privacy-preserving technologies include data anonymization (e.g., [1,2,3]) and secure computation (e.g., [4]). This paper focuses on the relationship between data anonymization and decision trees, a typical machine learning method. Historically, data anonymization research has progressed from pseudonymization to k-anonymity [1], ℓ-diversity [2,5], and t-closeness [3] and is continually growing. Currently, many researchers are focused on membership privacy and differential privacy [6].

In [7,8], the authors pointed out that the decision tree is not robust to homogeneity attacks and background knowledge attacks; they then demonstrated the application of k-anonymity and ℓ-diversity in order to amplify security. However, their proposals could not satisfy the requirements of differential privacy. In this paper, we discuss how we might prevent the leakage of private information via differential privacy provided by a learned decision tree using data anonymization techniques such as k-anonymity and ℓ-diversity.

To prevent leakage of private information, we propose the application of k-anonymity and sampling to a random decision tree, which is a variation of the expanded decision tree proposed by Fan et al. [9]. Interestingly, we show in this paper that this modification results in differential privacy. The essential idea is that instead of adding Laplace noise, as in [10,11] (please see [12] for a survey of a differentially private (random) decision tree), we propose a method of enhancing the security of a random decision tree by sampling and then removing the leaf containing fewer data than some threshold k, which applies to the other leaves of the tree. The basic concept is outlined in [13]. Our proposed model, in which k-anonymity is achieved after sampling, provides differential privacy, as in [13].

As mentioned above, researchers have shifted their attention to differential privacy rather than k-anonymization and ℓ-diversity. In fact, building upon the work outlined in [14], decision trees that satisfy differential privacy use techniques that are typical of differential privacy, such as the exponential, Gaussian, and Laplace mechanisms [10,11,15]. That is, all of these algorithms achieve differential privacy by adding some kind of noise. Our approach is very different from those of others. That said, the basic technique involves applying k-anonymity to each leaf in the random decision tree; this is similar to pruning, which is a widely accepted technique used to avoid overfitting.

The remainder of this paper is organized as follows. Section 2 introduces relevant preliminary information, e.g., anonymization methods and decision trees, and demonstrates how strategies for attacking data anonymization can be converted into attacks targeting decision trees. In Section 3, we demonstrate how much security and accuracy can be achieved in practice when the random decision tree is strengthened using a method that is similar to k-anonymity. In Section 4, the potential advantages of our proposal are discussed. Finally, the paper is concluded in Section 5, which includes a brief discussion of potential future research topics.

2. Preliminaries

2.1. Data Anonymization

When providing personal data to a third party, it is necessary to modify data to preserve user privacy. Here, modifying the user’s data (i.e., a particular record) such that an adversary cannot re-identify a specific individual is referred to as data anonymization. As a basic technique and to prevent re-identification, an identifier, e.g., a person’s name or employee number, is deleted or replaced with a pseudonym ID by the data holder. This process is referred to as pseudonymization. However, simply modifying identifiers does not ensure the preservation of privacy. In some cases, individuals can be re-identified by a combination of features (i.e., a quasi-identifier); thus, it is necessary to modify both the identifier and the quasi-identifier to reduce the risk of re-identification. In most cases, the identifiers themselves are not used for data analysis; thus, removing identifiers does not significantly sacrifice the quality of the dataset. However, if we modify quasi-identifiers in the same manner, although the data may become anonymous, they will also become useless. A typical anonymization technique for quasi-identifiers is “roughening” the numerical values.

2.1.1. Attacks Targeting Pseudonymization

A simple attack is possible against pseudonymized data from which identifiers, e.g., names, have been removed. In this attack, the attacker uses the quasi-identifier of a user u. If this attacker obtains the pseudonymized data, by searching for user u’s quasi-identifier in pseudonymized data, the attacker can obtain sensitive information about u. For example, if the attacker obtains the dataset shown in

Table 1. Example of a dataset *.

Zip	Age	Nationality	Disease
13053	28	Russian	Heart
13068	29	American	Heart
13068	21	Japanese	Flu
13053	23	American	Flu
14853	50	Indian	Cancer
14853	55	Russian	Heart
14850	47	American	Flu
14850	59	American	Flu
13053	31	American	Cancer
13053	37	Indian	Cancer
13068	36	Japanese	Cancer
13068	32	American	Cancer

* Excerpt from [16].

and knows friend u’s zip code is 13068, their age is 29, and their nationality is American, then, by searching the dataset, the attacker can identify that user u is suffering from some heart-related disease. This attack is referred to as the uniqueness attack.

2.1.2. k-Anonymity

k-anonymity is a countermeasure used to prevent uniqueness attacks.

In k-anonymity, features are divided into quasi-identifiers and sensitive information, and the same quasi-identifier is modified such that it does not become less than k users.

Table 2. k-anonymity ($k=4$) of the dataset shown in Table 1.

Zip	Age	Nationality	Disease
130 **	<30	*	Heart
130 **	<30	*	Heart
130 **	<30	*	Flu
130 **	<30	*	Flu
1485 *	<30	*	Cancer
1485 *	>40	*	Heart
1485 *	>40	*	Flu
1485 *	>40	*	Flu
130 **	30–40	*	Cancer
130 **	30–40	*	Cancer
130 **	30–40	*	Cancer
130 **	30–40	*	Cancer

The * and ** in the Zip column mean that the last one and two digits of the original data are hidden, respectively; while the * in the Nationality column means no partitioned for the corresponding attribute.

shows anonymized data that has been k-anonymized ($k=4$) using quasi-identifiers, e.g., zip code, age, and nationality.

2.1.3. Homogeneity Attack

At a cursory glance, k-anonymity appears to be secure; however, even if k-anonymity is employed, a homogeneity attack is still feasible. This attack becomes possible if the sensitive information is the same. Taking the k-anonymized dataset shown on the right side of Figure 1 as an example and assuming the presence of the attacker on the left side of Figure 1, we can make the following statements. Here, the attacker has the necessary information (zip, age) = (13053, 37); all of this sensitive information in the records corresponds with cancer. The attacker can therefore deduce that Bob has cancer.

2.1.4. Background Knowledge Attack

Homogeneous attacks suggest a problem when records with the same quasi-identifier have the same sensitive information; however, a previous study [3] also argued that there was a problem even in cases in which records were not the same. The k-anonymized dataset on the right side of Figure 2 shows four records with quasi-identifiers (130, <30, *), and two types of sensitive information, i.e., (heart, flu). Here, we can assume that the attacker has background knowledge of the data similar to that shown on the left side of Figure 2. In this case, there are certainly possibilities of heart disease and flu; however, if the probability of Japanese individuals experiencing heart disease is extremely low, Umeko is estimated to instead have the flu. Thus, it must be acknowledged that k-anonymity does not provide a high degree of security.

ℓ-diversity: ℓ-diversity is a measure used to counteract homogeneity attacks. A k-anonymization table is denoted ℓ-diverse if each similar class of quasi-identifiers has at least ℓ “well-represented” values for sensitive information. There are several different interpretations of the term “well-represented” [2,3]. In this paper, we adopt distinct ℓ-diversity. Distinct ℓ-diversity means that there are at least ℓ distinct values for the sensitive information in each similar class of quasi-identifiers.

Table 3. ℓ-diversity (ℓ = 2) after k-anonymity of the dataset shown in Table 1.

Zip	Age	Nationality	Disease
130 **	<30	*	Heart
130 **	<30	*	Heart
130 **	<30	*	Flu
130 **	<30	*	Flu
1485 *	<30	*	Cancer
1485 *	>40	*	Heart
1485 *	>40	*	Flu
1485 *	>40	*	Flu
130 **	30–40	*	Cancer
130 **	30–40	*	Cancer
130 **	30–40	*	Cancer
130 **	30–40	*	Flu

The * and ** in the Zip column mean that the last one and two digits of the original data are hidden, respectively; while the * in the Nationality column means no partitioned for the corresponding attribute.

shows anonymized data that are two-diverse ($\ell =2$).

2.2. Decision Trees

2.2.1. Basic Decision Trees

Decision trees are supervised learning methods that are primarily used for classification tasks, and a tree structure is created while learning from data (Figure 3). When predicting the label y of $\mathbf{x}$, the process begins from the root of the tree, and the corresponding leaf is searched for while referring to each feature of $\mathbf{x}$. Finally, through this referral process, y is predicted.

The label determined by the leaf is derived from the dataset D used to generate the tree structure. In other words, after the tree structure is created, for each element $({\mathbf{x}}_i,y_i)$ in dataset D, the corresponding leaf $\mathtt{leaf}$ is found, and the value of $y_i$ is stored. If $y_i\in Y=\{0,1\}$, then in each $\mathtt{leaf}$, the number of $y_i$ labeled 0 and the number of $y_i$ labeled 1 are preserved. More precisely, $[n\left({\mathtt{leaf}}^{\left(1\right)}\right),n\left({\mathtt{leaf}}^{\left(0\right)}\right)]$ are preserved for each leaf $\mathtt{leaf}$, where $n\left({\mathtt{leaf}}^{\left(0\right)}\right)$ and $n\left({\mathtt{leaf}}^{\left(1\right)}\right)$ represent the numbers of data points with label y that have values of 0 and 1, respectively.

Table 4. The notations used in this paper.

Notation	Description
k	Parameter for k-anonymity
ℓ	Parameter for ℓ-diversity
D	Dataset $\left\{({\mathbf{x}}_i,y_i)\right\}$, for $i\in \{1,\dots ,n\}$
$\mathbf{X}$	Data space with m features $F_1,\dots ,F_m$
$\mathbf{x}$	Data $(x_1,\dots ,x_m)\in \mathbf{X}=(F_1,\dots ,F_m)$
y	Label of data $\mathbf{x}$
Y	Label space
$\mathtt{leaf}$, ${\mathtt{leaf}}_i$	Leaf or leaves
$n\left({\mathtt{leaf}}_i^{\left(y\right)}\right)$	The number of data points classed as ${\mathtt{leaf}}_i$ with
	label $y\in Y$.
$n\left({\mathtt{leaf}}_i\right)$	The number of data points classed as ${\mathtt{leaf}}_i$, i.e.,
	$n\left({\mathtt{leaf}}_i\right)={\sum }_{y\in Y}n\left({\mathtt{leaf}}_i^{\left(y\right)}\right)$
$n_j\left({\mathtt{leaf}}_i^{\left(y\right)}\right)$	The number of data points with label $y\in Y$ classed as
	${\mathtt{leaf}}_i$ of the j-th tree.
	This is used for the random decision tree.
${\mathcal{H}}_u$	Total number of users who can be identified by
	a homogeneous attack
${\mathcal{H}}_{\mathtt{leaf}}$	Number of leaves that can be identified by
	a homogeneity attack
$N_t$	Number of random decision trees
ACC	Accuracy

shows the notations used in the paper.

For a given prediction $\mathbf{x}$, we first search for the corresponding leaf, and it may be denoted as 1 if ${\displaystyle \frac{n\left({\mathtt{leaf}}^{\left(1\right)}\right)}{n\left({\mathtt{leaf}}^{\left(0\right)}\right)+n\left({\mathtt{leaf}}^{\left(1\right)}\right)}}>{\displaystyle \frac{1}{2}}$, and 0 otherwise. Here, the threshold $1/2$ can be set flexibly depending on where the decision tree is applied, and when providing the learned decision tree to a third party, it is possible to pass $n\left({\mathtt{leaf}}^{\left(0\right)}\right)$ and $n\left({\mathtt{leaf}}^{\left(1\right)}\right)$ together for each leaf $\mathtt{leaf}$. In this paper, we considered the security of decision trees in such situations.

Generally, the deeper the tree structure, the more likely it is to overfit; thus, we frequently prune the tree, and this technique was employed to preserve data privacy within this paper.

2.2.2. Random Decision Tree

The random decision tree was proposed by Fan et al. [9]. Notably, in their approach, the tree is randomly generated without depending on the data. Furthermore, sufficient performance can be ensured through the appropriate selection of parameters.

The shape of a normal (not random) decision tree depends on the data used; the eventual shape may cause private information to be leaked from the tree. However, the random decision tree avoids this leakage due to its random generation. Therefore, its performance is expected to match the performance of other proposed security methods.

A random decision tree is described in Algorithms 1 and 2. Algorithm 1 shows that the generated tree does not depend on dataset D, except for $n_i\left({\mathtt{leaf}}^{\left(y\right)}\right)$ created by $\mathtt{UpdateStatistics}$. Here, $n_i\left({\mathtt{leaf}}^{\left(y\right)}\right)$ denotes the 2D array, which represents the number of feature vectors reaching each leaf $\mathtt{leaf}$. However, the preservation of privacy is necessary because $n_i\left({\mathtt{leaf}}^{\left(y\right)}\right)$ depends on D.

In the measurement of each parameter, the depth of the tree being (${\displaystyle \frac{\mathrm{dimension}\mathrm{of}\mathrm{feature}\mathrm{vectors}}{2}}$) and the number of trees being 10 are general rules.

2.3. Security Definitions

Part of this work adopts differential privacy to evaluate the security and efficiency of the model.

Definition 1.

A randomized algorithm A satisfies $(ϵ,\delta )$-DP, if for any pair of neighboring datasets, D and $D^{\prime }$, and any $O\subseteq \mathcal{O}$:

$$Pr\left[A\left(D\right)\in O\right]\le e^{ϵ}·Pr[A\left(D^{\prime }\right)\in O]+\delta ,$$

(1)

where $\mathcal{O}$ is a range of A.

When studying differential privacy, it is assumed that the attacker knows all the elements in D. However, such an assumption may not be realistic. This is taken into consideration, and the following definition is given in [13]:

Definition 2

(DP undersampling, $(\beta ,ϵ,\delta )$-DPS). An algorithm A satisfies $(\beta ,ϵ,\delta )$-DPS if and only if the algorithm $A^{\beta }$ satisfies $(ϵ,\delta )$-DP, where $A^{\beta }$ denotes the algorithm used to initially sample each tuple in the input dataset A with probability β.

In other words, the definition describes the output of A by inputting $D^{\prime }$, which is a result of sampling dataset D. Hence, the attacker may know D but not $D^{\prime }$.

Algorithm 1 Training the random decision tree [9]

Input: Training data $D=\{({\mathbf{x}}_1,y_1),\dots ,({\mathbf{x}}_n,y_n)\}$, the set of features $X=\{F_1,\dots ,F_m\}$, number of random decision trees to be generated $N_t$ Output: Random decision trees $T_1,\dots ,T_{N_t}$ 1: for $i\in \{1,\dots ,N_t\}$ do 2:    $T_i=\mathtt{BuildTreeStructure}(\mathrm{root},\mathbf{x})$ 3: end for 4: for $i\in \{1,\dots ,N_t\}$ do 5:    $T_i=\mathtt{UpdateStatics}(T_i,D)$ 6: end for 7: return $T_1,\dots ,T_{N_t}$ 8:   9: $\mathtt{BuildTreeStructure}(\mathrm{node},X)$ 10: if ($X\ne \varnothing$) then 11:    Set the node as a leaf 12: else 13:    $F\leftarrow X$ 14:    /* Set F as the feature of the node. Also, assume $\left|F\right|=c$ */ 15:    for $i\in \{1,\dots ,c\}$ do 16:      $T=\mathtt{BuildTreeStructure}({\mathrm{node}}_i,X-F)$ 17:    end for 18: end if 19:   20: $\mathtt{UpdateStatistics}(T_i,D)$ 21: Set $n_i\left({\mathtt{leaf}}^{\left(y\right)}\right)=0$ for all leaves, $\mathtt{leaf}$, and all labels, y. 22: for $(\mathbf{x},y)\in D$ do 23:    Find the leaf $\mathtt{leaf}$ corresponding to $\mathbf{x}$, and set $n_i\left({\mathtt{leaf}}^{\left(y\right)}\right)=n_i\left({\mathtt{leaf}}^{\left(y\right)}\right)+1$. 24: end for

Algorithm 2 Classifying the random decision tree [9]

Require: $\{T_1,\dots ,T_{N_t}\},\mathbf{x}$   1: return ${\sum }_{i=1}^{N_t}{n}_i\left({\mathtt{leaf}}^{\left(y\right)}\right)/\left({\sum }_{y^{\prime }}{\sum }_{i=1}^{N_t}{n}_i\left({\mathtt{leaf}}^{\left(y^{\prime }\right)}\right)\right)$ for each $y\in Y$, where $\mathtt{leaf}$ denotes the leaf corresponding to $\mathbf{x}$.

2.4. Attacks Targeting the Decision Tree

Generally, a decision tree is constructed from a given dataset; however, it is also possible to partially reconstruct the dataset using the decision tree. This kind of attack is considered in [7]. In this section, we explain the workings of such attacks.

Figure 4 shows the reconstruction of a dataset from the decision tree shown in Figure 3. As shown, it is impossible to reconstruct the original data completely from a binary tree model; however, it is possible to extract some of the data. By exploiting this essential property, it is possible to mount some attacks against reconstructed data, as discussed in Section 2.1. In the following, taking Figure 4 as an example, we present a uniqueness attack, a homogeneous attack, and a background knowledge attack.

• Uniqueness attack: in the dataset (Figure 4) recovered from the model, there is one user whose height is greater than 170 and who is under 15 years of age (as shown in the sixth row (see the 2nd red box shown on the figure), where the relevant leaf is converted via $[n_3\left({\mathtt{leaf}}^{\left(1\right)}\right),n_3\left({\mathtt{leaf}}^{\left(0\right)}\right)]=[1,0]$); thus, it is possible to target this user with a uniqueness attack.
• Homogeneous attack: similarly, in the fourth and fifth rows (see the 1st red box shown on the figure), which are converted from the relevant leaf via $[n_2\left({\mathtt{leaf}}^{\left(1\right)}\right),n_2\left({\mathtt{leaf}}^{\left(0\right)}\right)]=[0,2]$, the height $<170$, weight $\ge 60$, and health status are the same (i.e., “unhealthy”); therefore, a homogeneous attack is possible.
• Background knowledge attack: Similarly, in the seventh, eighth, and ninth rows, there are three users whose data match in both height $\ge 170$ and age $\ge 15$. Among these users, one is healthy (yes) and two are unhealthy (no). As an attacker, we can consider the following:
  • (Background knowledge of user A) height: 173, age: 33, healthy;
  • (Background knowledge of target user B) height: 171, age: 19.
  In this case, if the adversary knows that user A is healthy, he/she can identify that user B is unhealthy.

3. Proposal: Applying $\mathit{k}$-Anonymity to a (Random) Decision Tree

3.1. Construction of $k-\mathtt{AnonToRdt}$

In this section, we demonstrate how one might achieve differential privacy from k-anonymity. More specifically, we present a proposal based on a random decision tree, which is a variant of the decision tree outlined in Section 2.2.2. Said proposal is shown as Algorithm 3. It differs from the original random decision tree in the following ways:

• (Pruning): for some threshold k, if there exists a tree $T_i$, a leaf $\mathtt{leaf}$, and a label y, satisfying $n_i\left({\mathtt{leaf}}^{\left(y\right)}\right)<k$, then let $n_i\left({\mathtt{leaf}}^{\left(y\right)}\right)$ equal 0.
• (Sampling): training $T_i$ using $D_i$, which is the result obtained after sampling dataset D of each tree $T_i$ with probability $\beta$.

3.2. Security: Strongly Safe k-Anonymization Ensures Differential Privacy

In the field of data anonymization, in study [13], the authors demonstrated that performing k-anonymity after sampling achieved differential privacy; our proposal is a development upon this core principle. Below, we outline the items necessary to evaluate data security.

Definition 3

(Strongly Safe k-anonymization algorithm [13]). Suppose that a function g has $\mathcal{D}\to \mathcal{T}$, where $\mathcal{D}$ and $\mathcal{T}$ are the domain and range of g, respectively. Suppose that g does not depend on $D\subseteq \mathcal{D}$, i.e., g is constant. The strongly safe k-anonymization algorithm A with input $D\subseteq \mathcal{D}$ is defined as follows:

• Compute $Y_1=\left\{g\left(\mathbf{x}\right)\right|\mathbf{x}\in D\}$.
• $Y_2=\left\{\right(y,\left|\{\mathbf{x}\in D:g\left(\mathbf{x}\right)=y\}\right|):y\in Y_1\}$.
• For each element in $Y_2=\left\{(y,c)\right\}$, if $c<k$, then the element is set to $(y,0)$, and the result is set to $Y_3$.

Algorithm 3 Proposed training process

Input: Training data $D=\{({\mathbf{x}}_1,y_1),\dots ,({\mathbf{x}}_n,y_n)\}$, the set of features $X=\{F_1,\dots ,F_m\}$,   number of random decision trees to be generated $N_t$ Output: Random decision trees $T_1,\dots ,T_{N_t}$ 1: for $i\in \{1,\dots ,N_t\}$ do 2:    $T_i=\mathtt{BuildTreeStructure}(\mathrm{root},X)$ 3: end for 4: for $i\in \{1,\dots ,N_t\}$ do 5:    $D_i\leftarrow \varnothing$ 6:    Regarding each $(\mathbf{x},y)\in D$, $D_i\leftarrow D_i\cup (\mathbf{x},y)$ with probability $\beta$ 7:    $T_i=\mathtt{UpdateStatics}-k-\mathtt{anon}(T_i,D_i)$ 8: end for 9: return $T_1,\dots ,T_{N_t}$ 10:   11: $\mathtt{UpdateStatistics}-k-\mathtt{anon}(T_i,D_i)$ 12: Set $n_i\left({\mathtt{leaf}}^{\left(y\right)}\right)=0$ for all leaves $\mathtt{leaf}$ and labels y. 13: for $(\mathbf{x},y)\in D$ do 14:    Find $\mathtt{leaf}$ corresponding to $\mathbf{x}$, and set $n_i\left({\mathtt{leaf}}^{\left(y\right)}\right)=n_i\left({\mathtt{leaf}}^{\left(y\right)}\right)+1$. 15: end for 16: for All pairs of leaf and label $(\mathtt{leaf},y)$ do 17:    if $n_i\left({\mathtt{leaf}}^{\left(y\right)}\right)<k$ then 18:       $n_i\left({\mathtt{leaf}}^{\left(y\right)}\right)\leftarrow 0$/*Removing leaves with fewer data*/ 19:    end if 20: end for

Assume that $f(j;n,\beta )$ denotes the probability mass function: the probability of succeeding j times after n attempts, where the probability of success after one attempt is $\beta$. Furthermore, the cumulative distribution function is expressed as follows in Equation (2):

$$f^{\prime }(j;n,\beta )=\sum _{i=0}^{j}f(i;n,\beta ).$$

(2)

Theorem 1

(Theorem 5 in [13]). Any strongly safe k-anonymization algorithm satisfies $(\beta ,ϵ,\delta )$-DPS for any $0<\beta <1$, $ϵ\ge -ln(1-\beta )$, and

$$\delta =d(k,\beta ,ϵ)={\max }_{n:n\ge \lceil {\displaystyle \frac{k}{\gamma }}-1\rceil }\sum _{j>\gamma n}^{n}f(j;n,\beta ),$$

(3)

where $\gamma ={\displaystyle \frac{e^{ϵ}-1+\beta }{e^{ϵ}}}$.

Equation (3) shows the relationship between $\beta$ and $\delta$ in determining the value of $ϵ$ when k is fixed.

Let us consider the case where record $(\mathbf{x},y)\in D$ is applied to a random decision tree $T_i$, and the leaf reached is denoted by $\mathtt{leaf}$. If a function $g_i$ is defined as

$$g_i\left((\mathbf{x},y)\right)=(\mathtt{leaf},y),$$

(4)

then $g_i$ in Equation (4) is apparently constant, that is, it does not depend on D. Therefore, $n_i\left({\mathtt{leaf}}^{\left(y\right)}\right)$, which is generated using $g_i$, can be regarded as an example of strongly safe k-anonymization; consequently, Theorem 1 can be applied.

However, Theorem 1 above can be applied in its original form when there is one $T_i$, i.e., when the number of trees $N_t=1$. Theorem 2 can be applied when $N_t>1$.

Theorem 2

(Theorem 3.16 in [17]). Assume $A_i$ is an $({ϵ}_i,{\delta }_i)$-DP algorithm for $1\le i\le N_t$. Then, the algorithm

$$A\left(D\right):=(A_1\left(D\right),A_2\left(D\right),\dots ,A_{N_t}\left(D\right))$$

(5)

satisfies $({\sum }_{i=1}^{N_t}{ϵ}_i,{\sum }_{i=1}^{N_t}{\delta }_i)$-DP.

In Algorithm 3, each $T_i$ is selected randomly, and sampling is performed for each tree. Hence, the following conclusion can be reached.

Corollary 1.

The proposed algorithm satisfies $(\beta ,N_tϵ,N_t\delta )$-DPS, for any $0<\beta <1$, $ϵ\ge -ln(1-\beta )$, and

$$\delta =d(k,\beta ,ϵ)={\max }_{n:n\ge \lceil {\displaystyle \frac{k}{\gamma }}-1\rceil }\sum _{j>\gamma n}^{n}f(j;n,\beta ),$$

(6)

where $\gamma ={\displaystyle \frac{e^{ϵ}-1+\beta }{e^{ϵ}}}.$

Table 5. Approximate value of $N_t\delta$ for $k=5,10,20$ and $N_t=10$.

${\mathit{N}}_{\mathit{t}}\mathit{ϵ}\setminus \mathit{\beta }$	$\mathit{k}=5$			$\mathit{k}=10$			$\mathit{k}=20$
	$\mathbf{0.01}$	$\mathbf{0.1}$	$\mathbf{0.4}$	$\mathbf{0.01}$	$\mathbf{0.1}$	$\mathbf{0.4}$	$\mathbf{0.01}$	$\mathbf{0.1}$	$\mathbf{0.4}$
$1.0$	$0.001$	-	-	$4.66$ × ${10}^{-7}$	$0.355$	-	$1.20\times {10}^{-13}$	$0.045$	-
$2.0$	$5.52\times {10}^{-5}$	0.352	-	$1.08\times {10}^{-9}$	$0.034$	-	$7.00\times {10}^{-19}$	$0.000$	-
$3.0$	$7.68\times {10}^{-6}$	0.127	-	$2.72\times {10}^{-11}$	$0.005$	-	$4.75\times {10}^{-22}$	$7.82\times {10}^{-6}$	$0.426$
$4.0$	$1.86\times {10}^{-6}$	0.043	-	$1.68\times {10}^{-12}$	$0.001$	$0.583$	$1.92\times {10}^{-24}$	$2.37\times {10}^{-7}$	$0.134$
$5.0$	$7.47\times {10}^{-7}$	0.028	0.994	$2.85\times {10}^{-13}$	$0.000$	$0.348$	$3.54\times {10}^{-26}$	$1.61\times {10}^{-8}$	$0.051$
$6.0$	$2.417\times {10}^{-7}$	0.009	0.963	$3.19\times {10}^{-14}$	$3.93\times {10}^{-5}$	$0.191$	$7.71\times {10}^{-28}$	$1.03\times {10}^{-9}$	$0.016$
$7.0$	$1.22\times {10}^{-7}$	0.009	0.963	$8.51\times {10}^{-15}$	$2.05\times {10}^{-5}$	$0.175$	$5.75\times {10}^{-29}$	$1.48\times {10}^{-10}$	$0.008$
$8.0$	$1.22\times {10}^{-7}$	0.004	0.498	$4.07\times {10}^{-15}$	$4.53\times {10}^{-6}$	$0.093$	$6.27\times {10}^{-30}$	$1.56\times {10}^{-11}$	$0.003$
$9.0$	$5.47\times {10}^{-8}$	0.002	0.410	$7.58\times {10}^{-16}$	$1.87\times {10}^{-6}$	$0.078$	$5.06\times {10}^{-31}$	$2.82\times {10}^{-12}$	$0.001$

shows the relationship, derived from Equation (6), between $\beta$ and $N_t\delta$ in determining the value of $N_tϵ$ when k and $N_t$ are fixed. The cells in the table represent the approximate value of $N_t\delta$. For k and $N_t$, we chose $(k,N_t)=(5,10),(10,10)$, and $(20,10)$, as shown in Table 5.

3.3. Experiments on k-$\mathtt{AnonToRdt}$

The efficiency of the proposal was verified using the Nursery dataset [18], the Adult dataset [19], the Mushroom dataset [20], and the Loan dataset [21]. The characteristics of each dataset are as follows.

• The Nursery dataset contains 12,960 records with eight features, with a maximum of five values for each feature;
• The Adult dataset contains 48,842 records with 14 features. Here, each feature has more possible values and more records than in the Nursery datasets;
• The Mushroom dataset contains 8124 records with 22 features. Compared to the above two datasets, there are more features, but the number of records is small. In general, applying k-anonymity to this kind of dataset is challenging.
• The Loan dataset [21] contains 9578 records with 13 features. We used the “not.fully.paid” feature to label classes. There are four binary attributes and seven numerical attributes in this dataset.

Appendix A contains the evaluation of the basic decision tree with each dataset. Firstly, $\delta ,N_t,k$ were set to $\delta =0.4$, $N_t\in \{8,\dots ,11\}$, and $k\in \{5,10\}$.

The results from the Nursery dataset obtained via $k-\mathtt{AnonToRdt}$ with these parameters are as follows:

• The accuracy of the original decision tree was 0.933, as shown in

  Table A1. Number of users (${\mathcal{H}}_u$) and leaves (${\mathcal{H}}_{\mathtt{leaf}}$) that can be identified by homogeneity attacks.

  Tree	Nursery			Adult			Mushroom
  Depth	$\mathtt{ACC}$	${\mathcal{H}}_{\mathbf{u}}$	${\mathcal{H}}_{\mathcal{leaf}}$	$\mathtt{ACC}$	${\mathcal{H}}_{\mathbf{u}}$	${\mathcal{H}}_{\mathtt{leaf}}$	$\mathtt{ACC}$	${\mathcal{H}}_{\mathbf{u}}$	${\mathcal{H}}_{\mathtt{leaf}}$
  4	0.863	3965.5	2	0.843	215.2	2.4	0.979	3293.6	9
  5	0.880	5217	4.9	0.851	829.5	7.1	0.980	3568.4	12
  6	0.888	5747.1	9.9	0.853	1621.7	18.3	0.995	6122.6	16
  7	0.921	6837.4	19.5	0.855	1957.4	35.2	1.000	6499	20
  8	0.933	7912.7	35.7	0.855	2316.1	60.8	1.000	6499	20

  .
• As shown in

  Table 6. Accuracy of $k-\mathtt{AnonToRdt}$.

  		(a)			(b)
  k	#Trees	Nursery Dataset (with 5 class labels)			Adult Dataset
  	8	$d=3$	$d=4$	$d=5$	$d=7$	$d=8$	$d=9$
  0 *		0.748	0.832	0.854	0.781	0.806	0.814
  5		0.788	0.818	0.783	0.781	0.807	0.811
  10		0.749	0.782	0.598	0.794	0.801	0.814
  0	9	0.807	0.819	0.867	0.789	0.813	0.817
  5		0.798	0.843	0.816	0.789	0.813	0.817
  10		0.808	0.775	0.626	0.804	0.812	0.813
  0	10	0.815	0.841	0.860	0.795	0.791	0.816
  5		0.808	0.841	0.818	0.795	0.791	0.816
  10		0.816	0.801	0.673	0.786	0.803	0.813
  0	11	0.832	0.847	0.868	0.786	0.807	0.818
  5		0.836	0.847	0.823	0.786	0.807	0.817
  10		0.830	0.818	0.663	0.793	0.797	0.813
  		(c)			(d)
  k	#Trees	Mushroom Dataset			Loan Dataset
  	8	$d=3$	$d=4$	$d=5$	$d=4$	$d=5$	$d=6$
  0 *		0.935	0.964	0.979	0.838	0.843	0.838
  5		0.935	0.964	0.979	0.843	0.843	0.840
  10		0.903	0.964	0.974	0.838	0.845	0.838
  0	9	0.940	0.967	0.980	0.839	0.839	0.842
  5		0.940	0.967	0.980	0.839	0.839	0.842
  10		0.854	0.963	0.978	0.839	0.839	0.84
  0	10	0.957	0.972	0.978	0.841	0.846	0.835
  5		0.957	0.972	0.978	0.836	0.841	0.839
  10		0.957	0.949	0.975	0.841	0.846	0.835
  0	11	0.944	0.973	0.978	0.844	0.841	0.840
  5		0.944	0.973	0.978	0.839	0.841	0.840
  10		0.921	0.968	0.977	0.844	0.841	0.840

  * $k=0$ means original decision tree without pruning. #Trees denote the number of trees.

  (a), for a tree depth equal to four, the accuracy obtained was 0.84, which was inferior to that of the original decision tree.
• As shown in Table 6 (a), for a tree depth equal to five, the accuracy decreased drastically as k increased.

The results from the Adult dataset obtained via $k-\mathtt{AnonToRdt}$ with the same $\delta ,N_t,k$ were as follows: (there were numerical values in this dataset; to handle this, a threshold t was chosen randomly from its domain, and two children for ≤t and >t were produced by the tree)

• The accuracy of the original decision tree was 0.855, as shown in Table A1.
• As shown in Table 6 (b), the achieved accuracy when $k=5$ was 0.817.

The results from the Mushroom dataset obtained via $k-\mathtt{AnonToRdt}$ with the same $\delta ,N_t,k$ were as follows:

• The accuracy of the original decision tree was 0.995, as shown in Table A1.
• As shown in Table 6 (c), the achieved accuracy when $k=5$ was 0.98.

The results from the Loan dataset obtained via $k-\mathtt{AnonToRdt}$ with the same $\delta ,N_t,k$ were as follows:

• The accuracy of the original decision tree was 0.738 in our experiment.
• As shown in Table 6 (d), the achieved accuracy was around 0.84.

In summary, the accuracy achieved by $k-\mathtt{AnonToRdt}$ was slightly inferior to that of the original decision tree.

Changing sampling rate β: To achieve secure differential privacy, the sampling rate should remain small. Maintaining values of $N_t=10$, $N_tϵ=2.0$, which were small enough for our practical application,

Table 7. Accuracy and approximate $N_t\delta$ of $k-\mathtt{AnonToRdt}$ when $N_tϵ=2.0$.

k	Nursery Dataset (with 3 Class Labels)						Mushroom Dataset (with 2 Class Labels)
	$\mathit{\beta }\mathbf{=}\mathbf{0.01}$		$\mathit{\beta }\mathbf{=}\mathbf{0.1}$		$\mathit{\beta }\mathbf{=}\mathbf{0.4}$		$\mathit{\beta }\mathbf{=}\mathbf{0.01}$		$\mathit{\beta }\mathbf{=}\mathbf{0.1}$		$\mathit{\beta }\mathbf{=}\mathbf{0.4}$
	${\mathit{N}}_{\mathit{t}}\mathit{\delta }$	$\mathbf{ACC}$	${\mathit{N}}_{\mathit{t}}\mathit{\delta }$	$\mathbf{ACC}$	${\mathit{N}}_{\mathit{t}}\mathit{\delta }$	$\mathbf{ACC}$	${\mathit{N}}_{\mathit{t}}\mathit{\delta }$	$\mathbf{ACC}$	${\mathit{N}}_{\mathit{t}}\mathit{\delta }$	$\mathbf{ACC}$	${\mathit{N}}_{\mathit{t}}\mathit{\delta }$	$\mathbf{ACC}$
0	-	0.971	-	0.971	-	0.971	-	0.945	-	0.945	-	0.945
5	$5.52\times {10}^{-5}$	0.942	0.352	0.958	-	0.972	$5.52\times {10}^{-5}$	0.900	0.352	0.942	-	0.914
10	$1.08\times {10}^{-9}$	0.774	0.034	0.969	-	0.971	$1.08\times {10}^{-9}$	0.833	0.034	0.930	-	0.933
20	$7.00\times {10}^{-19}$	0.383	0	0.965	-	0.963	$7.00\times {10}^{-19}$	0.631	0	0.913	-	0.932

shows how the values of $N_t\delta$ changed according to the sampling rate $\beta$. As shown, for some parameters, the accuracy of the proposed method was relatively good even when $N_tϵ=2.0$ and $N_t\delta$ were small.

4. Discussion

  In another highly relevant study [10], Jagannathan et al. proposed a variant of a random decision tree that achieved differential privacy. The accuracy of their proposal is shown in Figures 1 and 2 of [10] for the same datasets with the same class labels; (in [10], instead of five class labels, three were used for the Nursery dataset, i.e., some of the similar labels were merged) their method resulted in similar precision. Because our proposal employs sampling, it is limited by the size of the dataset being utilized; the smaller the dataset (e.g., the Mushroom dataset), the less pronounced the accuracy. However, it must be noted that their approach was very different from ours: Laplace noise was added instead of pruning and sampling. Notably, within their proposal,

$$n_i\left({\mathtt{leaf}}^{\left(y\right)}\right)+\mathrm{Laplace}\mathrm{Noise}$$

(7)

for all trees $T_i$, all leaves $\mathtt{leaf}$, and all labels y. Even in this context, if $n_i\left({\mathtt{leaf}}^{\left(y\right)}\right)$ is small for a certain $T_i$, $\mathtt{leaf}$, and y, it may be regarded almost as a personal record. A good general approach to handling such cases is to remove the rare records, i.e., to “remove the leaves containing fewer records”. This is a broadly accepted data anonymization technique [22] that is commonly used to avoid legal difficulties. Our proposal shows that pruning and sampling can be combined to ensure differential privacy. If rare sensitive records need to be removed, our method may therefore represent an excellent option.

5. Conclusions

  In this paper, we aimed to show the close relationship between the security of data anonymity and decision trees. Specifically, we show how to obtain differentially private (random) decision tree from k-anonymity. Our proposal consists of applying sampling and k-anonymity to the original random decision tree method, which results in differential privacy. Compared to existing schemes, the advantage of our proposal is its ability to implement differential privacy without adding Laplace or Gaussian noise, which provides trained decision trees with a new route to differential privacy. We believe that in addition to random decision trees, other similar algorithms can be augmented to achieve differential privacy for general decision trees. In addition, in future studies, we will explore the differential privacy of federated learning decision trees by extending the proposed method.