Accounting for Measurement Error and Untruthfulness in Binary RRT Models

Abstract

This study examines the effect of measurement error on binary Randomized Response Technique models. We discuss a method for estimating and accounting for measurement error and untruthfulness in two basic models and one comprehensive model. Both theoretical and empirical results show that not accounting for measurement error leads to inaccurate estimates. We introduce estimators that account for the effect of measurement error. Furthermore, we introduce a new measure of model privacy using an odds ratio statistic, which offers better interpretability than traditional methods.

1. Introduction

Examining controversial attitudes and behaviors via survey often necessitates a careful approach. Rather than affirmatively responding to questions about sensitive or self-incriminating behavior, survey respondents may instead withhold information or provide inaccurate answers. To mitigate the bias that these actions introduce into survey data, Ref. [1] developed the Randomized Response Technique (RRT) to restore the assurance of confidentiality. Through the deliberate introduction of randomness into survey procedures, randomized response models safeguard respondent privacy and yield more accurate data.

The first binary RRT model was proposed by [1]. The Warner model prompts participants to answer sensitive questions in direct or indirect form. This method utilizes binary sensitive variables that preserve respondent privacy. Since then, many other models have been proposed, including the model by [2]. The Greenberg model asks respondents to answer either a direct sensitive question or an unrelated question. In 2023, Ref. [3] proposed a mixture model that combines the Warner and Greenberg models, treating the two models as special cases of the larger model. This paper also investigated the impact of untruthfulness on binary RRT models, but these models have not accounted for the effect of measurement error. In this paper, we aim to investigate the impact of measurement error and untruthfulness on binary RRT models.

The effect of measurement error has been studied in quantitative RRT models by many authors including [4,5]. Ref. [6] explored the effect of measurement error on the [1] binary model. This model and the mixture model proposed by [3] are discussed in detail in Section 2. We apply the effect of measurement error explored by [6] onto the Lovig mixture model in Section 3. In this paper, we aim to study the impact of measurement error in the Lovig mixture model in comparison to the models of [1] and the [2].

Certain models may be better suited against the effects of measurement error. In Section 4, we test our mixture model that accounts for measurement error by introducing measurement error in a numerical study. This study computes several levels of the error that is introduced into the model by the measurement error, $\mathrm{MSE}\left[\widehat{m}\right]$, for different choices of p and q. This design compares the mixture model to the Warner and Greenberg models since these two are considered special cases of the Lovig mixture model ($q=0$ and $q=1-p$, respectively). This investigation relies on our estimation of measurement error m, which uses a secondary question described in Section 3.3.2.

In the discussion of privacy in Section 5, we propose a new measure of privacy for binary RRT models offered by the model. This method uses binary logistic regression to compute the odds ratio (OR) between true participant responses and recorded responses from the model as a measure of predictability. A higher OR value indicates higher individual predictability of the presence of the sensitive trait as the recorded response changes from “No” to “Yes”, thereby offering lower privacy. This privacy measure is estimated in the presence of measurement error and participant untruthfulness.

2. Previous Models

Several RRT models form the building blocks for the proposed model in Section 3. The Warner model was the first RRT model introduced in [1]. In this model, participants are presented with either the sensitive question or an indirect version of the sensitive question using a randomization device with known probabilities. The key aspect of RRT models is that the interviewer remains unaware of whether the respondent is answering the direct or the indirect question, ensuring confidentiality. Similarly, the Greenberg model involves respondents responding to either a sensitive question or an entirely unrelated question.

Prior to [7], all binary RRT models assumed that respondents provide truthful responses. In this work, the authors accounted for a lack of trust and demonstrated that not accounting for untruthfulness leads to poor estimates in the Greenberg model. Later, Ref. [3] demonstrated the negative impact that untruthfulness has on the efficiency of the Greenberg, Warner, and Lovig mixture binary models. The method of accounting for untruthfulness proposed by [7] added a node to switch the respondent’s answer with probability $1-A$ only when the respondent is in the sensitive group with probability ${\pi }_x$. This design ensures that the respondent’s mistrust of the model will only occur at questions that the respondent finds sensitive. This conceptual framework of accounting for lack of trust was utilized to develop a more efficient and private model.

The Warner model has been shown to offer the highest privacy protection to respondents, and the best efficiency is found in the Greenberg model [3]. The mixture model proposed in [3] that combines these two models asks participants an indirect sensitive question, a direct sensitive question, or an unrelated question, each with known likelihoods. This model is discussed and depicted in Section 2. The approach in [3] offered an opportunity to compare privacy and efficiency between the Warner model, the Greenberg model, and a mixture of the two accounting for lack of trust. To fairly compare the three models, this work uses a modified version of the unified measure proposed by [8], which proposed unified measure $\mathbb{M}$. This novel model that accounts for untruthfulness may be improved to also account for measurement error.

Accounting for the effect of measurement error in the model of [1] was first proposed in [6]. This model represents the first work exploring measurement error in binary RRT models. This work developed a unique method of defining measurement error in the context of binary RRT models and examined the estimation of the prevalence of sensitive characteristics under measurement error in some cases. The authors found that, in most cases, the measurement error introduced a non-negligible bias into the efficiency offered by the Warner model. This work includes estimators for measurement error and the sensitive trait accounting for measurement error. The approach in this paper examines the same effect of measurement error in the more comprehensive mixture model by [3].

Mixture Model Accounting for Lack of Trust by [3]

The [3] mixture model offers a trichotomy that randomly chooses from the direct question, the indirect question, or the unrelated question. This model is best shown by the flow diagram given in Figure 1 below:

This work found that respondent lack of trust decreases model efficiency and increases privacy when truthfulness remains unaccounted for. Note that the Warner and Greenberg models may be considered as special cases of the Lovig mixture model as pictured above. Comparing the three models, the Greenberg model $(q=0)$ outperforms other models in terms of efficiency, the Warner model $(q=1-p)$ excels in privacy protection, and the Lovig mixture model $(q\ne 1-p)$ emerges as the best model in terms of the unified measure $\mathbb{M}$.

The Lovig mixture model outperforms both the Greenberg and the Warner models. It exhibits a lower MSE than the Warner model, better privacy protection than the Greenberg model, and significantly better unified measure $\mathbb{M}$ than both of the basic models in most recommended cases. This model was groundbreaking in investigating and accounting for the effect of untruthfulness in binary RRT models. The proposed model in this work is an extension of the Lovig model in Figure 1 to account for measurement error.

3. Proposed Model

We hypothesize that measurement error imposes a significant bias on model efficiency. This section proposes a model that accounts for such effects by estimating it using a secondary question. We acknowledge that this will not account for every type of measurement error. The appropriate methodological guidelines should still be followed to reduce the risk of measurement error confounding study results. These guidelines may include types of survey modes, response formats, respondent training, and response biases.

This paper proposes a model that introduces the effect of measurement error as in [6] onto the binary RRT mixture model that accounts for lack of trust proposed by [3]. Following the work of [7] and utilizing the model proposed by [3], a respondent in the sensitive group (with probability ${\pi }_x$) will switch his/her answer due to mistrust with probability $1-A$. So far, this design constitutes the Lovig mixture model shown in Figure 1. To account for measurement error, the model is designed such that each recorded response is switched with probability m. Nodes that switch the recorded response with rate m are added on each terminal node because we assume that the measurement error discussed has an equal chance of occurring at each branch.

After demonstrating that poor estimates occur when measurement error is not accounted for in Section 3.2, we outline a method for estimating the prevalence of measurement error m and trust rates A in Section 3.3 using secondary questions. To begin this discussion, our proposed mixture model is best shown by Figure 2 below:

For this model, we use the following notation:

• $n=\mathrm{size}\mathrm{of}\mathrm{random}\mathrm{sample}\mathrm{with}\mathrm{replacement}$;
• $p=\mathrm{probability}\mathrm{that}\mathrm{the}\mathrm{respondent}\mathrm{was}\mathrm{in}\mathrm{the}\mathrm{direct}\mathrm{question}\mathrm{group}$;
• $q=\mathrm{probability}\mathrm{that}\mathrm{the}\mathrm{respondent}\mathrm{was}\mathrm{in}\mathrm{the}\mathrm{indirect}\mathrm{question}\mathrm{group}$;
• $1-p-q=\mathrm{probability}\mathrm{that}\mathrm{the}\mathrm{respondent}\mathrm{was}\mathrm{in}\mathrm{the}\mathrm{unrelated}\mathrm{question}\mathrm{group}$;
• ${\pi }_x=\mathrm{proportion}\mathrm{of}\mathrm{the}\mathrm{sensitive}\mathrm{trait}$;
• ${\pi }_y=\mathrm{proportion}\mathrm{of}\mathrm{the}\mathrm{unrelated}\mathrm{trait}$;
• $A=\mathrm{proportion}\mathrm{of}\mathrm{people}\mathrm{who}\mathrm{trust}\mathrm{the}\mathrm{underlying}\mathrm{RRT}\mathrm{model}$;
• m = probability that the participant’s recorded response was switched due to measurement error;
• $P_y=\mathrm{probability}\mathrm{of}\mathrm{the}\mathrm{respondent}\mathrm{entering}\mathrm{a}“\mathrm{Yes}”\mathrm{response}.$

3.1. Estimating Trust Parameter A with a Greenberg Model

Before estimating the proportion of the sensitive trait, we must estimate the trust parameter A. This study uses an initial question to estimate A using a Greenberg model similar to the approach in [3]. Since the Greenberg model is the most efficient model and privacy is not prioritized in this initial question, the Greenberg model is the ideal choice to estimate untruthfulness. For this question, let there be the following:

• $p_g$ = proportion of the direct question used in a Greenberg model to estimate truthfulness;
• $P_{yg}$ = probability of a “Yes” response in this Greenberg model;
• ${\pi }_{yg}$ = proportion of people who would answer “Yes” to the unrelated question in the Greenberg model.

Following the approach from [3], we use the question, “Do you trust the model?”, for Question 1 with probability $p_g$ and an unrelated question with probability $1-p_g$. This leads to the equations:

$$\begin{array}{cc}\hfill P_{yg}& =p_{g}A+(1-p_g){\pi }_{yg}\hfill \\ \hfill \widehat{A}& =\frac{\widehat{P_{yg}}-(1-p_g){\pi }_{yg}}{p_g}\hfill \end{array}$$

where $\widehat{P_{yg}}$ is the proportion of “Yes” responses in the sample of Greenberg responses. Then,

$$\begin{array}{cc}\hfill \mathbb{E}\left[\widehat{P_{yg}}\right]& =P_{yg};\mathrm{Var}\left[\widehat{P_{yg}}\right]=\frac{P_{yg}(1-P_{yg})}{n}\hfill \\ \hfill \mathbb{E}\left[\widehat{A}\right]& =A;\mathrm{Var}\left[\widehat{A}\right]=\frac{P_{yg}(1-P_{yg})}{n{p}_g^2}.\hfill \end{array}$$

(1)

Since $\mathbb{E}\left[\widehat{A}\right]=A$, this question design provides an unbiased estimator for respondent trust prevalence A.

Now, we discuss the efficiency of the proposed model using the estimator $\widehat{A}$. In Section 3.2, we compute the efficiency of the proposed model that does not account for measurement error to test the behavior of the bias. After demonstrating its impact, this initial question that estimates untruthfulness is used as Question 1 in the proposed model in Section 3.3.

3.2. Proposed Model: Not Accounting for Measurement Error

3.2.1. Efficiency

In this section, we build an estimator for estimator $\widehat{{\pi }_{xu}}$ for the sensitive trait that does not account for measurement error using the probability of a “Yes” response. The probability of a “Yes” response is given by

$$\begin{array}{cc}{P}_y=\hfill & q\left\{(1-{\pi }_x)(1-m)+{\pi }_x\left[(1-A)(1-m)+Am\right]\right\}+p\left[{\pi }_x\left[A(1-m)+(1-A)m\right]\right]\hfill \\ & +(1-{\pi }_x)m]+(1-p-q)\left[{\pi }_y(1-m)+(1-{\pi }_y)m\right]\hfill \\ & =\left(A{\pi }_x\left(p-q\right)+{\pi }_y\left(1-p-q\right)\right)\left(1-2m\right)+m\left(1-2q\right)+q.\hfill \end{array}$$

(2)

If the researcher ignores measurement error and erroneously assumes $m=0$, our model becomes naive to measurement error. Let $P_{yu}$ be the probability of a “Yes” response in this naive case. Then, we may find $P_{yu}$ using (2) as follows:

$$P_{yu}=A{\pi }_x\left(p-q\right)+{\pi }_y\left(1-p-q\right)+q.$$

From this, we have the estimator for the sensitive trait under the naive case:

$$\begin{array}{cc}\hfill \widehat{{\pi }_{xu}}& =\frac{\widehat{P_y}-q-(1-p-q){\pi }_y}{\widehat{A}\left(p-q\right)};p\ne q\hfill \end{array}$$

where $\widehat{P_y}$ is the proportion of “Yes” responses in the sample assuming random sampling with replacement is given by

$$\mathbb{E}\left[\widehat{P_y}\right]=P_y;\mathrm{Var}\left[\widehat{P_y}\right]=\frac{P_y(1-P_y)}{n}.$$

(3)

Since $\widehat{{\pi }_{xu}}$ is a function of random variables $\widehat{P_y}$ and $\widehat{A}$, we use a first-order Taylor’s expansion for $\widehat{{\pi }_{xu}}$ given by

$$f(x,y)=f(x_0,y_0)+(x-x_0)f_x(x_0,y_0)+(y-y_0)f_y(x_0,y_0)$$

where $x=\widehat{P_y}$, $x_0=P_y$, $y=\widehat{A}$, and $y_0=A$. Then, we have the expansion:

$$\widehat{{\pi }_{xu}}\approx \frac{P_y-q-(1-p-q){\pi }_y}{A\left(p-q\right)}+\frac{\widehat{P_y}-P_y}{A\left(p-q\right)}-(\widehat{A}-A)\left[\frac{P_y-q-(1-p-q){\pi }_y}{A^2\left(p-q\right)}\right];p\ne q.$$

(4)

From (4), we can easily verify

$$\begin{array}{cc}\mathbb{E}\left[\widehat{{\pi }_{xu}}\right]\hfill & \approx \frac{P_y-q-(1-p-q){\pi }_y}{A\left(p-q\right)}\hfill \\ & \approx \frac{A{\pi }_x\left(p-q\right)\left(1-2m\right)-2m{\pi }_y\left(1-p-q\right)+m\left(1-2q\right)}{A\left(p-q\right)};p\ne q\hfill \\ \mathrm{Var}\left[\widehat{{\pi }_{xu}}\right]\hfill & \approx \frac{\mathrm{Var}\left[\widehat{P_y}\right]}{A^2{\left(p-q\right)}^2}+\mathrm{Var}\left[\widehat{A}\right]{\left[\frac{P_y-q-(1-p-q){\pi }_y}{A^2\left(p-q\right)}\right]}^2;p\ne q\hfill \end{array}$$

(5)

where $\mathrm{Var}\left[\widehat{A}\right]$ is given by (1) and $\mathrm{Var}\left[\widehat{P_y}\right]$ is given by (3). From (5), we may observe the bias that is introduced when measurement error is not accounted for. Bias for this naive approach is given by:

$$\begin{array}{cc}\mathrm{Bias}\left[\widehat{{\pi }_{xu}}\right]\hfill & =\mathbb{E}\left[\widehat{{\pi }_{xu}}\right]-{\pi }_x\hfill \\ & =\frac{m\left(1-2A{\pi }_x\left(p-q\right)-2{\pi }_y\left(1-p-q\right)-2q\right)}{A\left(p-q\right)};p-q\ne 0\hfill \\ \mathrm{MSE}\left[\widehat{{\pi }_{xu}}\right]\hfill & =\mathrm{Var}\left[\widehat{{\pi }_{xu}}\right]+\mathrm{Bias}{\left[\widehat{{\pi }_{xu}}\right]}^2.\hfill \end{array}$$

(6)

3.2.2. Simulation Results

To simulate randomized response trials, we used the NumPy module in Python to randomly generate survey results based on specified parameters for ${\widehat{\pi }}_x$, ${\widehat{\pi }}_y$, p, q, A, and m. Estimators were then calculated using these generated data. We tabulated the data across trials via the Pandas software library in Python and created visualizations with the Matplotlib plotting library.

The simulations in

Table 1. Estimates ${\widehat{\pi }}_{xu}$ and $\widehat{\mathrm{MSE}\left[{\widehat{\pi }}_{xu}\right]}$ when measurement error is not accounted for. Results aggregated over $10,000$ trials with $n=500$, $p=0.7$, $q=0.15$, ${\pi }_x=0.4$, and ${\pi }_y=\frac{1}{12}$.

p	q	m	A	$\widehat{\mathit{A}}$	${\mathit{\pi }}_{\mathit{x}}$	${\widehat{\mathit{\pi }}}_{\mathit{xu}}$	$\mathbf{MSE}\left[{\widehat{\mathit{\pi }}}_{\mathit{xu}}\right]$	$\widehat{\mathbf{MSE}\left[{\widehat{\mathit{\pi }}}_{\mathit{xu}}\right]}$
0.7	0.15	0.00	1.00	1.0001	0.4	0.3998	0.0017	0.0016
0.7	0.15	0.00	0.95	0.9504	0.4	0.3991	0.0019	0.0017
0.7	0.15	0.00	0.90	0.8999	0.4	0.4002	0.0021	0.0019
0.7	0.15	0.01	1.00	1.0001	0.4	0.4041	0.0017	0.0016
0.7	0.15	0.01	0.95	0.9503	0.4	0.4048	0.0019	0.0017
0.7	0.15	0.01	0.90	0.8999	0.4	0.4061	0.0021	0.0019
0.7	0.15	0.05	1.00	1.0005	0.4	0.4208	0.0021	0.0020
0.7	0.15	0.05	0.95	0.9502	0.4	0.4240	0.0025	0.0023
0.7	0.15	0.05	0.9	0.8996	0.4	0.4284	0.0029	0.0027
0.7	0.15	0.10	1.00	0.9999	0.4	0.4432	0.0035	0.0035
0.7	0.15	0.10	0.95	0.9502	0.4	0.4489	0.0043	0.0042
0.7	0.15	0.10	0.90	0.9002	0.4	0.4563	0.0052	0.0051

demonstrate the simulated effect of the bias from measurement error using the Lovig model in Section 3.2.1. We used MSEs to compare different models. The MSE has been used for this purpose in all major RRT papers including the seminal papers [1,2]. Observing the $\mathrm{MSE}\left[{\widehat{\pi }}_{xu}\right]$, which encompasses the variance and bias of ${\widehat{\pi }}_{xu}$ as in (6), the error rates increase in this estimator of ${\pi }_x$ as significant levels of A and m are introduced. This positive bias causing poor estimates is especially prevalent when measurement error is introduced. Low levels $(m=0.01)$ do not cause noticeable changes, but a 10% rate of measurement error causes approximately a 5% error rate in the sensitive trait. A similar positive bias may be observed when untruthfulness is introduced, but this is not as severe.

3.3. Proposed Model: Accounting for Measurement Error

Following [3], we first estimate A, then the probability of measurement error (m), and finally, the sensitive trait ${\pi }_x$.

3.3.1. Approach

Let Question 1 in this approach be the question, “Do you trust the model?”, using the Greenberg model outlined in Section 3.1. This provides an estimator for A. This is followed by Questions 2 and 3 to estimate measurement error m in the model and the prevalence of the sensitive trait ${\pi }_x$, respectively, using the proposed mixture model.

3.3.2. Estimating Measurement Error Using a Secondary Question

Section 3.2 demonstrates that not accounting for measurement error leads to inaccurate estimates. Following [6], we estimate the parameter for measurement error $\widehat{m}$ using our model with an additional modified question that ensures a known sensitivity probability.

For example, this test question could be, “Are you a robot?” Such a question ensures a known sensitive probability of ${\pi }_x=0$ for human respondents. Note that truthfulness A does not have to be accounted for in this question since sensitivity is always zero. Researchers must design a question for this estimator $\widehat{m}$ that satisfies these two requirements. For this secondary design, we may derive the probability of a “Yes” response in this rigged question $P_{y0}$ using (2), where ${\pi }_x=0$.

Question 2: (with secondary question) “Are you a robot?”

$$P_{y0}=m\left(1-2q-2{\pi }_y\left(1-p-q\right)\right)+q+\left(1-p-q\right){\pi }_y.$$

(7)

Using (7), we can set

$$\begin{array}{ccc}\hfill \widehat{m}& =\frac{\widehat{P_{y0}}-q-{\pi }_y(1-p-q)}{1-2q-2{\pi }_y(1-p-q)};\hfill & \hfill p+q\ne 1\\ \hfill \mathbb{E}\left[\widehat{P_{y0}}\right]& =P_{y0};\mathrm{Var}\left[\widehat{P_{y0}}\right]=\frac{P_{y0}(1-P_{y0})}{n}\hfill \\ \hfill \mathbb{E}\left[\widehat{m}\right]& =\frac{\mathbb{E}\left[\widehat{P_{y0}}\right]-q-{\pi }_y(1-p-q)}{1-2q-2{\pi }_y(1-p-q)}=m;\hfill & \hfill p+q\ne 1.\end{array}$$

(8)

Since $\mathbb{E}\left[\widehat{m}\right]=m$, (8) provides an unbiased estimator of m. We may then find the MSE given by

$$\begin{array}{cc}\hfill \mathrm{Var}\left[\widehat{m}\right]& =\frac{\mathrm{Var}\left[\widehat{P_{y0}}\right]}{{\left(1-2q-2{\pi }_y(1-p-q)\right)}^2}=\frac{P_{y0}(1-P_{y0})}{n{\left(1-2q-2{\pi }_y(1-p-q)\right)}^2}=\mathrm{MSE}\left[\widehat{m}\right];p+q\ne 1.\hfill \end{array}$$

(9)

3.3.3. Estimating Proportion of Sensitive Trait ${\pi }_x$

Now that we have estimates for m and A, Question 3 uses the proposed mixture model from Figure 2 to ask a direct sensitive, indirect sensitive, or unrelated question. We use the full probability of a “Yes” response $P_y$ to this question to derive the estimator $\widehat{{\pi }_x}$.

Question 3: (with mixture model) “Do you have the sensitive trait?”

$$\begin{array}{cc}\hfill P_y& =\left(A{\pi }_x\left(p-q\right)+{\pi }_y\left(1-p-q\right)\right)\left(1-2m\right)+m\left(1-2q\right)+q\hfill \\ \hfill \widehat{{\pi }_x}& =\frac{\widehat{P_y}-\widehat{m}\left(1-2q\right)-q-{\pi }_y\left(1-p-q\right)\left(1-2\widehat{m}\right)}{A\left(1-2\widehat{m}\right)\left(p-q\right)};p\ne 0,\widehat{m}\ne \frac{1}{2}\hfill \end{array}$$

(10)

We use the following Taylor’s approximation to make use of the estimator $\widehat{{\pi }_x}$:

$$f(x,y,z)\approx f(x_0,y_0,z_0)+(x-x_0)f_x(x_0,y_0,z_0)+(y-y_0)f_y(x_0,y_0,z_0)+(z-z_0)f_z(x_0,y_0,z_0)$$

where $x=\widehat{P_y}$, $y=\widehat{m}$, $z=\widehat{A}$, $x_0=P_y$, $y_0=m$, and $z_0=A$. Then, $\widehat{{\pi }_x}$ is approximated by:

$$\begin{array}{cc}\hfill \widehat{{\pi }_x}\approx & \frac{P_y-m\left(1-2q\right)-q-{\pi }_y\left(1-p-q\right)\left(1-2m\right)}{A\left(1-2m\right)\left(p-q\right)}+\frac{\widehat{P_y}-P_y}{A\left(1-2m\right)\left(p-q\right)}\hfill \\ & +(\widehat{m}-m)\left[\frac{2P_y-1}{A(p-q){(1-2m)}^2}\right]\hfill \\ & -(\widehat{A}-A)\left[\frac{P_y-m\left(1-2q\right)-q-{\pi }_y\left(1-p-q\right)\left(1-2m\right)}{A^2\left(1-2m\right)\left(p-q\right)}\right];p\ne 0,\widehat{m}\ne \frac{1}{2}.\hfill \end{array}$$

(11)

From (11), we may note that

$$\begin{array}{cc}\hfill \mathbb{E}\left[\widehat{{\pi }_x}\right]& \approx {\pi }_x\hfill \\ \hfill \mathrm{Var}\left[\widehat{{\pi }_x}\right]& \approx \frac{\mathrm{Var}\left[\widehat{P_y}\right]}{{\left[A\left(1-2m\right)\left(p-q\right)\right]}^2}+\mathrm{Var}\left[\widehat{m}\right]{\left[\frac{2P_y-1}{A(p-q){(1-2m)}^2}\right]}^2\hfill \\ & +\mathrm{Var}\left[\widehat{A}\right]{\left[\frac{P_y-m\left(1-2q\right)-q-{\pi }_y\left(1-p-q\right)\left(1-2m\right)}{A^2\left(1-2m\right)\left(p-q\right)}\right]}^2;p\ne 0,\widehat{m}\ne \frac{1}{2}\hfill \end{array}$$

where $\mathrm{Var}\left[\widehat{P_y}\right]$ is given by (3), $\mathrm{Var}\left[\widehat{m}\right]$ is given by (9), and $\mathrm{Var}\left[\widehat{A}\right]$ is given by (1).

3.3.4. Simulation Results

We observed that the estimates for ${\pi }_x$ were more accurate when we accounted for measurement error. The estimates in

Table 2. Estimates ${\widehat{\pi }}_x$ and $\widehat{\mathrm{MSE}\left[{\widehat{\pi }}_x\right]}$ when we account for measurement error. Results aggregated over $10,000$ trials, with $n=500$, $p=0.7$, ${\pi }_x=0.4$, and ${\pi }_y=\frac{1}{12}$.

q	m	$\widehat{\mathit{m}}$	A	$\widehat{\mathit{A}}$	${\mathit{\pi }}_{\mathit{x}}$	${\widehat{\mathit{\pi }}}_{\mathit{xu}}$	$\mathbf{MSE}\left[{\widehat{\mathit{\pi }}}_{\mathit{xu}}\right]$	$\widehat{\mathbf{MSE}\left[{\widehat{\mathit{\pi }}}_{\mathit{xu}}\right]}$
Greenberg model
0.00	0.01	0.0099	1.00	0.9997	0.4	0.4001	0.0010	0.0009
0.00	0.01	0.0101	0.95	0.9503	0.4	0.3997	0.0011	0.0010
0.00	0.01	0.0101	0.90	0.9002	0.4	0.3996	0.0012	0.0011
0.00	0.05	0.0499	1.00	1.0002	0.4	0.3997	0.0012	0.0011
0.00	0.05	0.0501	0.95	0.9497	0.4	0.4000	0.0014	0.0012
0.00	0.05	0.0502	0.90	0.9001	0.4	0.4005	0.0015	0.0013
0.00	0.10	0.1003	1.00	0.9995	0.4	0.3994	0.0016	0.0014
0.00	0.10	0.0999	0.95	0.9499	0.4	0.4001	0.0017	0.0016
0.00	0.10	0.0999	0.90	0.9002	0.4	0.3992	0.0019	0.0017
Lovig model
0.15	0.01	0.0100	1.00	1.0000	0.4	0.3996	0.0018	0.0016
0.15	0.01	0.0103	0.95	0.9500	0.4	0.3995	0.0019	0.0018
0.15	0.01	0.0102	0.90	0.8995	0.4	0.3993	0.0021	0.0020
0.15	0.05	0.0501	1.00	0.9993	0.4	0.4003	0.0021	0.0019
0.15	0.05	0.0499	0.95	0.9501	0.4	0.3995	0.0023	0.0021
0.15	0.05	0.0502	0.90	0.9002	0.4	0.3996	0.0025	0.0023
0.15	0.10	0.0996	1.00	0.9998	0.4	0.4014	0.0026	0.0025
0.15	0.10	0.1001	0.95	0.9499	0.4	0.3997	0.0029	0.0028
0.15	0.10	0.0996	0.90	0.8999	0.4	0.4004	0.0032	0.0030
Warner model
0.3	0.01	0.0104	1.00	1.0000	0.4	0.3993	0.0034	0.0032
0.3	0.01	0.0107	0.95	0.9500	0.4	0.4012	0.0037	0.0036
0.3	0.01	0.0099	0.90	0.9002	0.4	0.3987	0.0042	0.0040
0.3	0.05	0.0520	1.00	1.0003	0.4	0.4001	0.0040	0.0038
0.3	0.05	0.0510	0.95	0.9498	0.4	0.4006	0.0044	0.0042
0.3	0.05	0.0507	0.90	0.8998	0.4	0.4006	0.0049	0.0046
0.3	0.10	0.0995	1.00	1.0003	0.4	0.3998	0.0050	0.0048
0.3	0.10	0.1001	0.95	0.9503	0.4	0.3996	0.0055	0.0054
0.3	0.10	0.1000	0.90	0.8996	0.4	0.3992	0.0062	0.0060

contrast the poor results shown in Table 1, where measurement error was ignored. It is now clear that neglecting to account for measurement error in binary RRT models results in inaccurate estimators. Table 2 simulates the estimators discussed.

The simulation in Table 2 offers several insights. Our estimators $\widehat{m}$, $\widehat{A}$, ${\widehat{\pi }}_x$, and $\mathrm{MSE}\left[{\widehat{\pi }}_x\right]$ are statistically close to their theoretical values, indicating that these are good estimators. Most significantly, the ${\widehat{\pi }}_x$ column confirms that our estimator of ${\pi }_x$ using the proposed model is accurate as untruthfulness and measurement error are introduced. The Greenberg model offers the greatest efficiency indicated by the lowest $\mathrm{MSE}\left[{\widehat{\pi }}_x\right]$ rates, and the Warner model offers the worst efficiency. The Lovig mixture model is between these values for each level of $m,A$. We will note in Section 5 that, when privacy protection is also factored in, the mixture model will offer the best performance.

4. Comparison of Measurement Error between Models

Observing the effect of measurement error across models will aid in the researcher’s choice of the model design. Since it is always desired to minimize the impact of measurement error in the chosen model, we discuss the MSE of m for different choices of p and q. To do this, we compare the effect of measurement error between the Lovig, Warner, and Greenberg models.

Each model is differentially affected by measurement error since it is estimated using a secondary question outlined in Section 3.3.2. Recall that the Warner and Greenberg models are special cases of the Lovig mixture model, where $q=0$ and $q=1-p$, respectively. The estimator for m in (8) is unbiased for all models, so we will compare the measurement error between the models using the respective MSEs.

Numerical Comparison

The simulation results so far have fixed the level of p and parameterized q, but the choice of p has an effect on the model’s performance. The numerical discussion provided in Figure 3 provides four levels of p to observe the behavior of $\mathrm{MSE}\left[\widehat{m}\right]$.

Asymptotic behavior is expected around $q=0.5$ when $p<0.5$ since we have $p-q$ terms in the denominators of these estimators for $\mathrm{MSE}\left[\widehat{m}\right]$. This indicates that $p,q$ should never be chosen close to 0.5 to avoid poor estimators.

When $p>0.5$, the Warner model introduces the highest error in terms of measurement error, while the Greenberg model offers the least error due to measurement error for all considered values of m. Since this error function is monotonically increasing, the Lovig model offers moderate error as the bold values in

Table 3. Theoretical and estimated values of MSE of measurement error estimators in Section 3.3.2. Estimates aggregated over $10,000$ trials, with $n=500$, ${\pi }_x=0.4$, $p=0.7$, and ${\pi }_y=\frac{1}{12}$.

p	q	m	$\widehat{\mathit{m}}$	$\mathbf{MSE}\left[\widehat{\mathit{m}}\right]$	$\widehat{\mathbf{MSE}\left[\widehat{\mathit{m}}\right]}$
Greenberg model
0.7	0.00	0.01	0.0099	0.0001	0.0001
0.7	0.00	0.05	0.0499	0.0001	0.0001
0.7	0.00	0.10	0.1003	0.0002	0.0002
Lovig model
0.7	0.15	0.01	0.0100	0.0006	0.0006
0.7	0.15	0.05	0.0501	0.0007	0.0007
0.7	0.15	0.10	0.0996	0.0008	0.0008
Warner model
0.7	0.3	0.01	0.0104	0.0026	0.0027
0.7	0.3	0.05	0.0520	0.0027	0.0028
0.7	0.3	0.10	0.0995	0.0028	0.0027

demonstrate. When $p<0.5$, it is imperative that the parameter q be carefully chosen for the Lovig model to avoid large values of $\mathrm{MSE}\left[\widehat{m}\right]$. In this case, the comparative rank of error due to measurement error between models is similar to when $p>0.5$. This is seen in Table 3 with the $\mathrm{MSE}\left[\widehat{m}\right]$ values provided below. This table also shows that $\widehat{\mathrm{MSE}\left[\widehat{m}\right]}$ is a good estimator.

Both Figure 3 and Table 3 show that the choice of p and q has a more significant impact than the level of m on the amount of error introduced into the model from $\mathrm{MSE}\left[\widehat{m}\right]$. This fact underscores the importance of a proper choice of model parameters since certain models are better suited for accounting for measurement error. To provide a complete recommendation to a researcher designing an RRT study, a discussion on privacy and a unified measure is required. The choice of $p,q$ involves considering trade-offs between efficiency, privacy protection, and measurement error. This is provided in Section 6.

5. Privacy of Mixture Model

We provide a brief overview of how privacy is measured in previous binary RRT models and then propose our new privacy measure.

5.1. Previous Work

Model efficiency is not the only performance basis by which researchers should design their studies; respondent privacy is just as important. Without response privacy, respondents may refuse to respond or provide an untruthful response.

5.1.1. Privacy Measure

Ref. [9] provided a measure of privacy loss as described below. Let:

$$\delta =\mathrm{Max}\{{\eta }_1,{\eta }_2\}$$

(12)

where

$$\begin{array}{cc}\hfill {\eta }_1=P\left(S\right|Y)=& \mathrm{probability} \mathrm{of} \mathrm{being} \mathrm{in} \mathrm{the} \mathrm{sensitive} \mathrm{group} \mathrm{given} \hfill \\ & \mathrm{that} \mathrm{the} \mathrm{response} \mathrm{is} “\mathrm{Yes}”;\hfill \\ \hfill {\eta }_2=P\left(S\right|N)=& \mathrm{probability} \mathrm{of} \mathrm{being} \mathrm{in} \mathrm{the} \mathrm{sensitive} \mathrm{group} \mathrm{given} \hfill \\ & \mathrm{that} \mathrm{the} \mathrm{response} \mathrm{is} “\mathrm{No}”.\hfill \end{array}$$

Privacy protection, $PP$ introduced by [10] is defined as

$$PP=\frac{1-\delta }{1-{\pi }_x}.$$

(13)

5.1.2. Unified Measure $\mathbb{M}$

Ref. [3] proposed a unified measure of privacy and efficiency using the following metric:

$$\mathbb{M}=\frac{P{P}^a}{{\mathrm{MSE}}^b},$$

(14)

where a and b are weights based on the importance the researcher places on privacy and efficiency, respectively. Ref. [3] assumed $a=b=1$, arguing equal importance to efficiency and privacy. We follow the same approach here.

5.1.3. Privacy of Proposed Model

The following estimators for privacy loss ${\eta }_1,{\eta }_2$ are derived for the proposed mixture model accounting for measurement error from Section 3.

$$\begin{array}{cc}\hfill {\eta }_1& =P\left(S\right|Y)=\frac{P(Y\cap S)}{P\left(Y\right)}\hfill \\ & =\frac{{\pi }_x\left[m+Ap(1-2m)+(1-2m)(1-p){\pi }_y+(1-2m)(1-A-{\pi }_y)q\right]}{P_y}\hfill \\ \hfill {\eta }_2& =P\left(S\right|N)=\frac{P(N\cap S)}{P\left(N\right)}\hfill \\ & =\frac{{\pi }_x\left[1-q-{\pi }_y(1-p-q)-A(p-q)-m(1-2A(p-q)-2q+2{\pi }_y(p+q-1))\right]}{1-P_y}\hfill \end{array}$$

5.1.4. Unified Measure $\mathbb{M}$ of Proposed Model

Table 4. A comparison of theoretical $PP$ and $\mathbb{M}$ between the three binary RRT models. Calculated with $n=500$, $p=0.7$, ${\pi }_x=0.4$, and ${\pi }_y=\frac{1}{12}$.

q	${\mathit{\pi }}_{\mathit{x}}$	A	m	$\mathbf{MSE}\left[{\widehat{\mathit{\pi }}}_{\mathit{x}}\right]$	$\widehat{\mathbf{MSE}\left[{\widehat{\mathit{\pi }}}_{\mathit{x}}\right]}$	$\mathit{PP}$	$\widehat{\mathit{PP}}$	$\mathbb{M}$	$\widehat{\mathbb{M}}$
Greenberg model
0.00	0.4	1.00	0.01	0.0010	0.0009	0.1117	0.1119	107.6471	123.2866
0.00	0.4	0.95	0.01	0.0011	0.0010	0.1169	0.1168	103.0169	120.1149
0.00	0.4	0.90	0.01	0.0012	0.0011	0.1226	0.1227	98.5821	113.0005
0.00	0.4	1.00	0.05	0.0012	0.0011	0.2234	0.2234	180.9270	199.7414
0.00	0.4	0.95	0.05	0.0014	0.0012	0.2324	0.2320	171.7727	194.7648
0.00	0.4	0.90	0.05	0.0015	0.0013	0.2422	0.2436	162.8959	189.5389
0.00	0.4	1.00	0.10	0.0016	0.0014	0.3488	0.3494	222.2470	245.2008
0.00	0.4	0.95	0.10	0.0017	0.0016	0.3606	0.3604	209.1361	224.7503
0.00	0.4	0.90	0.10	0.0019	0.0017	0.3731	0.3729	196.3539	220.4197
Lovig model
0.15	0.4	1.00	0.01	0.0018	0.0016	0.4398	0.4395	249.8652	271.3308
0.15	0.4	0.95	0.01	0.0019	0.0018	0.4525	0.4516	233.1124	253.8097
0.15	0.4	0.90	0.01	0.0021	0.0020	0.4659	0.4645	216.8621	235.4783
0.15	0.4	1.00	0.05	0.0021	0.0019	0.4978	0.4975	239.3552	257.3392
0.15	0.4	0.95	0.05	0.0023	0.0021	0.5106	0.5111	222.4784	242.4145
0.15	0.4	0.90	0.05	0.0025	0.0023	0.5241	0.5225	206.1068	223.8847
0.15	0.4	1.00	0.10	0.0026	0.0025	0.5665	0.5670	216.0764	228.5641
0.15	0.4	0.95	0.10	0.0029	0.0028	0.5791	0.5792	199.9674	208.7430
0.15	0.4	0.90	0.10	0.0032	0.0030	0.5922	0.5923	184.3541	195.9438
Warner model
0.3	0.4	1.00	0.01	0.0034	0.0032	0.6597	0.6596	195.7034	204.8066
0.3	0.4	0.95	0.01	0.0037	0.0036	0.6711	0.6727	179.7029	186.7708
0.3	0.4	0.90	0.01	0.0042	0.0040	0.6830	0.6817	164.3027	171.1752
0.3	0.4	1.00	0.05	0.0040	0.0038	0.6897	0.6898	173.4890	179.6031
0.3	0.4	0.95	0.05	0.0044	0.0042	0.7005	0.7016	159.0592	167.1141
0.3	0.4	0.90	0.05	0.0049	0.0046	0.7117	0.7139	145.1756	153.5604
0.3	0.4	1.00	0.10	0.0050	0.0048	0.7265	0.7270	145.2988	150.4105
0.3	0.4	0.95	0.10	0.0055	0.0054	0.7366	0.7351	132.9618	135.4669
0.3	0.4	0.90	0.10	0.0062	0.0060	0.7469	0.7456	121.0998	125.1110

performs a simulation study that compares the three models by efficiency, privacy protection, and [3]’s unified measure. Privacy is calculated using the traditional method as defined in (13). The unified measure $\mathbb{M}$ is defined in (14). Several key values are noted in bold.

This is the classical view of model privacy, but now, we pursue a new method of discussing privacy offered by binary RRT models. As expected, the $PP$ and $\mathbb{M}$ match well with their empirical estimates even in the presence of measurement error. We observed the same conclusions as in [3] for efficiency, privacy protection, and $\mathbb{M}$ in the presence of m: the Greenberg model is best in terms of efficiency; the Warner model is best in terms of privacy; the Lovig model is best in terms of $\mathbb{M}$.

5.2. Proposed Measure of Privacy for Binary RRT Models

5.2.1. Description

The traditional method of privacy introduced by [9] can be difficult to interpret. We propose using the odds ratio as a measure of the predictability of participants’ true responses from recorded model responses. The odds ratio (OR) is a statistical measure commonly used in binary logistic regression to quantify the association between two binary variables.

The odds ratio quantifies the change in the odds of the sensitive trait as the reported response changes from a 0 (“No”) to a 1 (“Yes”). Values greater than 1 indicate higher odds of the true response being “Yes”, suggesting better predictability. The odds ratio serves as a valuable metric to assess the predictability of the reported response data from the RRT model.

In our study, we hypothesize that a higher odds ratio corresponds to a lower level of privacy protection offered by the RRT model. Conversely, a lower odds ratio suggests a lower level of predictability and efficiency, but provides greater respondent privacy. We will investigate how different factors, such as the introduction of measurement error and trust in the chosen RRT model, affect the odds ratio.

5.2.2. Privacy Measure Simulation Results

In

Table 5. Estimates averaged over 10,000 simulations for the Lovig mixture model for measurement error in Section 3 with $n=500$, ${\pi }_x=0.4$, ${\pi }_y=\frac{1}{12}$, and $p=0.7$.

q	p + q	A	m	OR
Greenberg model
0	0.7	1.00	0.00	124.93
0	0.7	1.00	0.05	124.62
0	0.7	1.00	0.10	124.62
0	0.7	0.95	0.00	105.53
0	0.7	0.95	0.05	105.74
0	0.7	0.95	0.10	105.74
0	0.7	0.90	0.00	89.93
0	0.7	0.90	0.05	90.05
0	0.7	0.90	0.10	90.05
Lovig model
0.15	0.85	1.00	0.00	13.26
0.15	0.85	1.00	0.05	13.27
0.15	0.85	1.00	0.10	13.27
0.15	0.85	0.95	0.00	11.61
0.15	0.85	0.95	0.05	11.63
0.15	0.85	0.95	0.10	11.63
0.15	0.85	0.90	0.00	10.24
0.15	0.85	0.90	0.05	10.25
0.15	0.85	0.90	0.10	10.25
Warner model
0.3	1.00	1.00	0.00	5.60
0.3	1.00	1.00	0.05	5.60
0.3	1.00	1.00	0.10	5.60
0.3	1.00	0.95	0.00	5.09
0.3	1.00	0.95	0.05	5.11
0.3	1.00	0.95	0.10	5.11
0.3	1.00	0.90	0.00	4.65
0.3	1.00	0.90	0.05	4.66
0.3	1.00	0.90	0.10	4.66

, we simulate the odds ratio in all three models accounting for measurement error.

Table 5 provides several insights into the effect of privacy offered by the three models accounting for measurement error. We observed the same conclusions as in [3] for privacy with this new approach: the Warner model offers the best privacy; the Greenberg model offers the least privacy; the Lovig model offers moderate to high privacy.

The odds ratio offers greater interpretability for a choice of $p,q$ compared to $PP$ from (13). For example, a mixture model was conducted where $p,q$ were chosen as $p=0.7$ and $q=0.15$ and $A=0.95$ and $m=0.05$. The estimated odds ratio for this model would be interpreted by making the claim that the odds of the true answer being “Yes” given that the reported answer was “Yes” is 11.63-times greater than the odds of the true answer being “No”. This contrasts the traditional definition of privacy protection $PP$, where its definition is a conditional probability that may only be interpreted using relative levels between models. While the traditional definition of privacy protection is useful for defining the unified measure $\mathbb{M}$, the logistic regression coefficients hold interpretation value. This new definition for privacy requires no adjustments to the unified measure’s definition.

6. Discussion

The researcher’s choice of the ideal model must take into account measurement error and the unified measure. It has been shown that the effect of measurement error on both model efficiency and privacy is significant. To account for this, the unified measure $\mathbb{M}$ accounts for measurement error since it uses the updated estimators in this paper. Since $\mathrm{MSE}\left[\widehat{m}\right]$ is in the equation for $\mathrm{MSE}\left[\widehat{{\pi }_x}\right]$, $\mathbb{M}$ discounts the performance for choices of $p,q$ where $\mathrm{MSE}\left[\widehat{m}\right]$ increases. Therefore, we recommend utilizing $\mathbb{M}$ for designing a binary RRT study accounting for measurement error.

After fixing the sample size and preliminary question parameters, for all reasonable possibilities of untruthfulness, measurement error, and the sensitive trait, the Lovig model for measurement error (with parameters $p,q$ sufficiently far from 0.5) optimizes the unified measure $\mathbb{M}$. This is preserved when the ratio between privacy protection and efficiency is parameterized using $a,b$. There are two regions of $p,q$ that locally optimize $\mathbb{M}$: when $p\in (0.7,0.8)$ and $q\in (0.05,0.15)$, as demonstrated in Table 4, and when $p\in (0.1,0.3)$ and $q\in (0.7,0.8)$. The former was chosen in the tables because its unified measure is always greater than the latter. Unified measure values for these cases become closer when $a>b$ (indicating prioritization of privacy protection above efficiency) or under higher rates of untruthfulness and measurement error. All options reduce the overall $\mathbb{M}$ values. Researchers should, therefore, choose their parameters in this region so that $\mathbb{M}$ is maximized regardless of untruthfulness, measurement error, or sensitive trait levels.

The Lovig mixture model for measurement error outperforms both the Greenberg and the Warner models in terms of the MSE and unified measure in most cases in the presence of untruthfulness and measurement error. The researcher must choose model parameters that optimize the unified measure while ensuring a high rate of participant cooperation. To this point, [3] noted that the choice from three questions in the mixture model helps improve the respondent cooperation as compared to when they have a choice of two questions, as in the Warner and Greenberg models.

7. Concluding Remarks

The proposed mixture model is the recommended RRT model for collecting sensitive data with the best performance and flexibility for privacy and efficiency compared to existing models. This model accounts for measurement error and untruthful responses using secondary questions to estimate their prevalence. The choice of model parameters significantly impacts both privacy and efficiency. A new logistic regression method is proposed to compare the models comprehensively concerning privacy. The adaptability for conditions of untruthfulness and measurement error allows researchers to choose the most suitable model for their specific needs.

While this paper has outlined a promising design for addressing measurement error within binary RRT models and introduced an innovative privacy measure, several considerations warrant acknowledgment. The real-world implementation of the outlined methods might encounter challenges, particularly with the proposed three-question design. The complexity of this design necessitates a more extensive explanation to participants. This may lead to increased abandonment when combined with sensitive questions, impacting data collection and accuracy. Future research and practical application should address these concerns to ensure the effectiveness and feasibility of these proposed procedures in diverse settings.

This study leverages the comprehensive [3] mixture model to account for the effects of measurement error. The measurement error’s impact on efficiency and privacy has not been discussed in the published literature on binary RRT models. This development will help to improve the efficacy of future studies that make use of binary RRT models.