Bayesian Statistical Inference for Factor Analysis Models with Clustered Data

Abstract

Clustered data are a complex and frequently used type of data. Traditional factor analysis methods are effective for non-clustered data, but they do not adequately capture correlations between multiple observed individuals or variables in clustered data. This paper proposes a Bayesian approach utilizing MCMC and Gibbs sampling algorithms to accurately estimate parameters of interest within the clustered factor analysis model. The mean traversal graph of parameters ensures that the Markov chain converges, and the Bayesian case-deletion model is used to analyze the model’s impact and identify outliers in clustered data using Cook’s posterior mean distance. The applicability and validity of the principal-component-method-based factor analysis model for clustered data are demonstrated by comparing the parameter estimation of this method with the principal component method, the clustered data with and without internal relationships are compared by example analysis, and the anomalous groups are identified by the Cook’s posterior mean distance.

1. Introduction

Clustered data, commonly found in fields like neuroscience and epidemiology, are characterized by a high degree of similarity among observations within the same group, which surpasses the similarity observed between different groups. They comprise interconnected samples, each encapsulating a collective of individuals with correlated characteristics. When these samples are amalgamated, they form a complex clustered dataset with multiple clusters, potentially varying in number of constituent samples. This dataset accentuates intra-group correlations, a focus distinct from longitudinal data, which prioritizes the temporal sequence of observations for individual subjects. The hierarchical structure of clustered data introduces analytical challenges, particularly in statistical analysis, where there is oversight of the internal group. Galbraith and Daniel [1] have demonstrated that neglecting the internal relationships within clustered data can result in substantial biases in parameter estimation, underscoring the imperative to consider these internal associations. The field of clustered data analysis has seen significant advancements in recent years with the emergence of diverse analytical models such as linear models [2,3,4], random effects models [5], semiparametric models [6], nonparametric models [7], logistic regression models [8], and multilevel models [9,10], etc. Each of these methodologies has contributed to the parameter estimation for clustered data, converging on a pivotal insight; disregarding the potential correlations among observations within groups during the modeling process can lead to an underestimation of standard errors, thereby compromising the robustness of inferential conclusions.

The factor analysis model is an effective statistical tool for describing the dependence between multidimensional variables. Spearman [11] first proposed factor analysis when studying intelligence tests. It not only reduces the dimension, but also reflects the internal correlation among variables and solves the problem of intra-cluster correlation effectively. Experiments in cytology performed by Julian [12] demonstrated that ignoring the nested structure leads to serious deviations in the estimates of unknown parameters when the intra-cluster correlation is significant. This proves that the nested structure of clustered data cannot be ignored. Huang et al. [9] proposed a variety of alternative schemes to deal with the nested structure of clustered data. However, the processing methods applicable to the clustered factor model were not mentioned. The mean of cluster value method is a simple and effective method for processing clustered data. It has been successfully applied by many scholars to deal with nested structures. Okech [13] used the mean of clusters to process the nested structure of the data when processing the characteristic data relating to low-income parents’ parenting pressure and economic pressure through the factor model. Galbraith et al. [1] also stated in a paper that using the mean of clusters to process clustered data is an effective way to obtain unknown parameters. Recently, with the rise of Bayesian networks, estimating factor analysis models using Bayesian methods has become a hot topic. The Bayesian factor analysis model, first proposed by Press [14], is capable of obtaining implicit solutions to the unknown parameters of the factor model when considering sample covariances obeying a Wishart distribution. Wirth et al. [15] demonstrated that the MCMC method can obtain better fitting results when estimating the parameters of the factor analysis model for non-clustered data. Zhang et al. [16] proposed a novel Bayesian factor analysis model which achieves the downscaling and feature extraction of high-dimensional multi-omics data by introducing the knowledge of biological mapping and improves the robustness of the model in the processing of noise and sparsity. Hansen et al. [17] proposed a new variational inference algorithm that first uses multiplicative gamma process shrinkage prior to approximation of the posterior distribution of a Bayesian factor model in order to address the poor scalability of factor analytic models for high-dimensional data.

Since Cook [18] introduced the case-deletion model for identifying influential points, the methods of statistical diagnostic of models have become one of the research hotspots, with various diagnostic methods being developed. Among these, Bayesian statistical diagnostics represents a particularly active branch, with statisticians both domestically and internationally widely utilizing this approach. De Finetti [19] explored the problem of how to deal with data outliers through Bayesian subjective probability theory and analyzed the treatment in the case of independent, exchangeable, and partially exchangeable error distributions. Jackson et al. [20] proposed two approximations capable of computing diagnostic statistics for the Bayesian case-deletion model. Zhu et al. [21] examined three Bayesian case-deletion influence measures in the context of complex models such as longitudinal data. Nevertheless, there is a paucity of research on Bayesian case-deletion diagnostics for factor models. Moreover, the statistical diagnosis of factor models for clustered data has not been addressed in the relevant literature. Factor analysis models excel at identifying correlations among observable variables or individuals within clustered datasets where large volumes of data across various fields share unobservable underlying public variables. However, applying these models for parameter estimation and impact analysis within clustered data remains largely unexplored in the existing literature. For simplicity, we outline the following specific contributions of this manuscript:

• We used the results averaged across clusters to deal with the nested structure of the clustered data, where the associations between clusters are inscribed by a common diagonal form of the error matrix;
• We performed a comprehensive analysis of factor analysis models for clustered data, employing both Bayesian and principal component analysis methods, and compared the efficacy of these two methodologies;
• We utilized Cook’s posterior mean distance for statistical diagnostics of factor analysis models within clustered data and identified groups with anomalies through diagnostic statistics.

The remaining sections are structured as follows: Section 2 introduces the process of constructing the factor analysis model within clustered data; Section 3 introduces the basic steps of how to model the factor analysis model of clustered data using the Bayesian method; Section 4 focuses on how to estimate factor analysis models for clustered data using principal component analysis; Section 5 introduces how to diagnose the anomalous groups by using Cook’s posterior mean distance; Section 6 validates the feasibility of the Bayesian method using numerical simulation and compares the estimation effect with that of the principal component method; Section 7 is a example analysis that uses examples to illustrate that the estimation method in this paper can be effectively applied in practice; Section 8 encapsulates the findings and concludes the paper.

2. Model Framework of Factor Analysis for Clustered Data

Suppose the observed variable of the ith individual in the gth cluster is $x_{gi}=(x_{gi1},x_{gi2},$ $\dots ,x_{gip}{)}^T$. For $g=1,2,\dots ,G, i=1,2,\dots ,n_g$, the factor analysis model is expressed as follows:

$$\begin{array}{c}\hfill x_{gi}={\mu }_g+{\mathsf{\Lambda }}_g{F}_{gi}+{\epsilon }_{gi},\end{array}$$

(1)

where ${\mu }_g$ is the p × 1 dimensional mean vector. ${\mathsf{\Lambda }}_g$ is the factor loading matrix of the gth cluster. $F_{gi}={\left(F_{gi1},F_{gi2},\dots ,F_{gim}\right)}^T$ denotes the m-dimensional common factor of the ith individual in the gth group. ${\epsilon }_{gi}$ is the special factor for the ith individual in the gth cluster, where ${\epsilon }_{gi}\sim N_p(0,{\mathsf{\Psi }}_g)$. At the same time, we suppose that ${\epsilon }_{gi}$ is uncorrelated with $F_{gi}$ for all g and i values. Aggregate all the individuals in the gth cluster in the form of a matrix as follows:

$$X_g=1_{n_g\times 1}{\mu }_g^T+F_g^T{\mathsf{\Lambda }}_g^T+{\epsilon }_g,$$

(2)

where

$$\begin{array}{c}{X}_g=\left(\begin{array}{c}{x}_{g1}^T\\ ⋮\\ x_{g{n}_g}^T\end{array}\right)={\left(x_{g1},\cdots ,x_{g{n}_g}\right)}^T,1_{n_g\times 1}{\mu }_g^T={\left(\begin{array}{c}1\\ ⋮\\ 1\end{array}\right)}_{n_g\times 1}\left({\mu }_{g1},\cdots ,{\mu }_{gp}\right)\hfill \\ F_g^T=\left(\begin{array}{c}{F}_{g1}^T\\ ⋮\\ F_{g{n}_g}^T\end{array}\right)={\left(F_{g1},\cdots ,F_{g{n}_g}\right)}^T,{\epsilon }_g=\left(\begin{array}{c}{\epsilon }_{g1}^T\\ ⋮\\ {\epsilon }_{g{n}_g}^T\end{array}\right)={\left({\epsilon }_{g1},\cdots ,{\epsilon }_{g{n}_g}\right)}^T\hfill \end{array}$$

The factor analysis model of all clusters is expressed in diagonal matrix form, which is shown in Equation (3):

$$\begin{array}{c}\hfill X=\mu +\mathsf{\Lambda }F+\epsilon ,\end{array}$$

(3)

where

$$\begin{array}{c}X=diag\left(X_1^T,X_2^T,\dots ,X_G^T\right),\mu =diag\left(\left({\mu }_{1i}{1}_{1\times n_g}\right),\left({\mu }_{2i}{1}_{1\times n_g}\right),\dots ,\left({\mu }_{Gi}{1}_{1\times n_g}\right)\right),\hfill \\ F=diag\left(F_1,F_2,\cdots ,F_G\right),\epsilon =diag\left({\epsilon }_1^T,{\epsilon }_2^T,\dots ,{\epsilon }_G^T\right).\hfill \end{array}$$

3. Bayesian Estimations of Factor Analysis Model for Clustered Data

3.1. Analysis of Prior Distribution

The factor analysis model of a single cluster is shown in Equation (1). Let $\theta$ represent a vector comprising all unknown parameters. For Bayesian analysis, consider that $\theta$ satisfies the following joint priors:

$$\pi \left(\theta \right)=\pi \left({\mu }_g,{\mathsf{\Lambda }}_g,F_{gi},{\mathsf{\Psi }}_g\right)=\pi \left({\mu }_g\right)\pi \left({\mathsf{\Lambda }}_g\right)\pi \left(F_{gi}\right)\pi \left({\mathsf{\Psi }}_g\right).$$

(4)

Ansari et al. [22] mentioned the advantages of using a conjugate prior when dealing with multilevel factor analysis models. This approach simplifies the computation of the posterior distribution, improves computational efficiency, and provides flexibility and stability to the parameters. We follow this research and assume that the variance of the common factors follows an inverse gamma distribution. The prior distributions for the remaining unknown parameters are delineated below:

$$\begin{array}{c}{\mu }_g\sim N_p\left({\alpha }_0,{\mathsf{\Sigma }}_0\right),{\mathsf{\Lambda }}_g={\displaystyle \prod _{j=1}^p}{\mathsf{\Lambda }}_{gj}\stackrel{D}{=}{\displaystyle \prod _{j=1}^p}{N}_m\left({\alpha }_{1j},{\mathsf{\Sigma }}_{1j}\right),F_{gi}\sim N_m\left({\alpha }_1,{\mathsf{\Sigma }}_1\right),\hfill \\ p\left({\mathsf{\Psi }}_g\right)={\displaystyle \prod _{j=1}^p}p\left({\mathsf{\Psi }}_{gj}\right)\stackrel{D}{=}{\displaystyle \prod _{j=1}^p}IG\left(c,d\right),{\mathsf{\Psi }}_g=diag\left({\mathsf{\Psi }}_{g1},\cdots ,{\mathsf{\Psi }}_{gp}\right),\hfill \end{array}$$

where $IG\left(·\right)$ represents the inverse gamma distribution. Aware of the potential variability among different subgroups, we adjust the means and variances of the common factors to generate subgroup datasets with distinct means and variances, thereby enabling a more precise simulation and analysis of the specific characteristics of these subgroups. In addition, we set the random error term to follow a uniform normal distribution, a setting designed to guarantee the feasibility and validity of the method of group means. ${\alpha }_{1j},{\mathsf{\Sigma }}_{1j},c$, and d are fixed hyperparameters; the remaining variables ${\alpha }_0,{\mathsf{\Sigma }}_0,{\alpha }_1$, and ${\mathsf{\Sigma }}_1$ are variable parameters.

3.2. Gibbs Sampling Method

The posterior distribution of parameters is the starting point of Bayesian statistical inference. Since the posterior distribution of $\theta$ involves multiple integrals, we use the Gibbs sampler [23] to extract samples successively from the conditional distribution to form a Markov chain. It is assumed that each parameter is independent, and the probability density function of $x_{gi}$ is $L\left(X|\theta \right)={\displaystyle \prod _{g=1}^G}L\left(X_g|\theta \right)={\displaystyle \prod _{g=1}^G}{\displaystyle \prod _{i=1}^{n_g}}L\left(x_{gi}|\theta \right)$. The specific format is as follows:

$$L\left(x_{gi}|{\mu }_g,{\mathsf{\Lambda }}_g,F_{gi},{\mathsf{\Psi }}_g\right)=\prod _{g=1}^G\prod _{i=1}^{n_g}\frac{1}{{\left(\sqrt{2\pi }\right)}^p{\left|{\mathsf{\Psi }}_g\right|}^{\frac{1}{2}}}exp\left\{-\frac{1}{2}{\left(x_{gi}-{\widehat{\mu }}_g\right)}^T{\mathsf{\Psi }}_g^{-1}\left(x_{gi}-{\widehat{\mu }}_g\right)\right\},$$

(5)

where ${\widehat{\mu }}_g={\mu }_g+{\mathsf{\Lambda }}_g{F}_{gi}$. Since $\theta$ denotes all the unknown parameters, the estimates of each component are calculated based on the conditional distribution satisfied by $\theta$ to compute $x_{gi}^{new}$. As the number of sampling times increases, the distribution of the sampled random numbers becomes closer to the true distribution. $X_g=(x_{g1}^T,x_{g2}^T,\dots ,x_{g{n}_g}^T{)}^T$ is the set of observed variables, and $\pi \left({\mu }_g,{\mathsf{\Lambda }}_g,F_{gi},{\mathsf{\Psi }}_g\right)$ represents the prior distribution of the unknown parameters, according to Equations (4) and (5), the joint posterior distribution $h\left(x_{gi},\theta \right)$ is as follows:

$$\begin{array}{c}\hfill h\left(x_{gi},\theta \right)=L\left(x_{gi}|{\mu }_g,{\mathsf{\Lambda }}_g,{\mathsf{\Psi }}_g,F_{gi}\right)\pi \left({\mu }_g,{\mathsf{\Lambda }}_g,{\mathsf{\Psi }}_g,F_{gi}\right).\end{array}$$

(6)

The posterior distributions of the unknown parameters are inferred from Equation (6) to be $p\left(\theta |X_g\right)$. The specific form is shown in (7):

$$\begin{array}{c}\hfill p\left(\theta |x_{gi}\right)=\frac{L\left(x_{gi}|\theta \right)\pi \left(\theta \right)}{\int L\left(x_{gi}|\theta \right)\pi \left(\theta \right)d\theta }\propto L\left(x_{gi}|{\mu }_g,{\mathsf{\Lambda }}_g,F_{gi},{\mathsf{\Psi }}_g\right)\pi \left({\mu }_g,{\mathsf{\Lambda }}_g,F_{gi},{\mathsf{\Psi }}_g\right).\end{array}$$

(7)

Firstly, the initial values of $\left\{x_{gi}^{\left(0\right)},{\mu }_g^{\left(0\right)},{\mathsf{\Lambda }}_g^{\left(0\right)},F_{gi}^{\left(0\right)},{\mathsf{\Psi }}_g^{\left(0\right)}\right\}$ are given, $\{x_{gi}^{\left(n\right)},{\mu }_g^{\left(n\right)},{\mathsf{\Lambda }}_g^{\left(n\right)},F_{gi}^{\left(n\right)},$${\mathsf{\Psi }}_g^{\left(n\right)}\}$ can be obtained after n steps of Gibbs sampling, and its update steps are as follows:

(1)

Select $x_{gi}^{\left(1\right)}$ from $p\left(x_{gi}^{\left(0\right)}|{\mu }_g^{\left(0\right)},{\mathsf{\Lambda }}_g^{\left(0\right)},F_{gi}^{\left(0\right)},{\mathsf{\Psi }}_g^{\left(0\right)}\right)$;

(2)

Select ${\mu }_g^{\left(1\right)}$ from $p\left({\mu }_g^{\left(0\right)}|x_{gi}^{\left(1\right)},{\mathsf{\Lambda }}_g^{\left(0\right)},F_{gi}^{\left(0\right)},{\mathsf{\Psi }}_g^{\left(0\right)}\right)$;

(3)

Select $F_{gi}^{\left(1\right)}$ from $p\left(F_{gi}^{\left(0\right)}|x_{gi}^{\left(1\right)},{\mu }_g^{\left(1\right)},{\mathsf{\Lambda }}_g^{\left(0\right)},{\mathsf{\Psi }}_g^{\left(0\right)}\right)$;

(4)

Select ${\mathsf{\Lambda }}_g^{\left(1\right)}$ from $p\left({\mathsf{\Lambda }}_g^{\left(0\right)}|x_{gi}^{\left(1\right)},{\mu }_g^{\left(1\right)},F_{gi}^{\left(1\right)},{\mathsf{\Psi }}_g^{\left(0\right)}\right)$;

(5)

Select ${\mathsf{\Psi }}_g^{\left(1\right)}$ from $p\left({\mathsf{\Psi }}_g^{\left(0\right)}|x_{gi}^{\left(1\right)},{\mu }_g^{\left(1\right)},{\mathsf{\Lambda }}_g^{\left(1\right)},F_{gi}^{\left(1\right)}\right)$.

The preceding observations lead to the conclusion that, as the number of cycles approaches infinity, the distribution of $\left\{x_{gi}^{\left(n\right)},{\mu }_g^{\left(n\right)},{\mathsf{\Lambda }}_g^{\left(n\right)},F_{gi}^{\left(n\right)},{\mathsf{\Psi }}_g^{\left(n\right)}\right\}$ gradually approaches the target distribution of $p\left({\mu }_g,{\mathsf{\Lambda }}_g,F_{gi},{\mathsf{\Psi }}_g|X_{gi}\right)$ [24]. This provides sufficient evidence to suggest that the samples taken above might be considered as representing the posterior distribution. The posterior distribution of the unknown parameter $\theta$ is as follows:

$$\begin{array}{c}p\left(x_{gi}|{\mu }_g,{\mathsf{\Lambda }}_g,F_{gi},{\mathsf{\Psi }}_g\right)\sim N_p\left({\mu }_g+{\mathsf{\Lambda }}_g{F}_{gi},{\mathsf{\Psi }}_g\right)\hfill \\ p\left({\mu }_g|x_{gi},{\mathsf{\Lambda }}_g,F_{gi},{\mathsf{\Psi }}_g\right)\sim N_p\left({\left(n_g{\mathsf{\Psi }}_g^{-1}+\sum _0^{-1}\right)}^{-1}\left(n_g{\mathsf{\Psi }}_g^{-1}\overline{M}+\sum _0^{-1}{\alpha }_0\right),{\left(n_g{\mathsf{\Psi }}_g^{-1}+\sum _0^{-1}\right)}^{-1}\right)\hfill \\ p\left(F_{gi}|x_{gi},{\mu }_g,{\mathsf{\Lambda }}_g,{\mathsf{\Psi }}_g\right)\sim N_m\left({\left({\mathsf{\Lambda }}_g^T{\mathsf{\Psi }}_g^{-1}{\mathsf{\Lambda }}_g+\sum _1^{-1}\right)}^{-1}\left({\mathsf{\Lambda }}_g^T{\mathsf{\Psi }}_g^{-1}\left(x_{gi}-{\mu }_g\right)+\sum _1^{-1}{\alpha }_1\right),{\left({\mathsf{\Lambda }}_g^T{\mathsf{\Psi }}_g^{-1}{\mathsf{\Lambda }}_g+\sum _1^{-1}\right)}^{-1}\right)\hfill \\ p\left({\mathsf{\Lambda }}_{gj}^T|x_{gij},{\mu }_{gj},F_{gi},{\mathsf{\Psi }}_{gj}\right)\sim N_m\left({\left(F_{gi}{\mathsf{\Psi }}_{gj}^{-1}{F}_{gi}^T+{\widehat{\mathsf{\Sigma }}}_{1j}^{-1}\right)}^{-1}\left(F_{gi}{\mathsf{\Psi }}_{gj}^{-1}\left(x_{gij}-{\mu }_{gj}^T\right)+{\widehat{\mathsf{\Sigma }}}_{1j}^{-1}{\alpha }_{1j}\right),{\left(F_{gi}{\mathsf{\Psi }}_{gj}^{-1}{F}_{gi}^T+{\widehat{\mathsf{\Sigma }}}_{1j}^{-1}\right)}^{-1}\right)\hfill \\ P\left({\mathsf{\Psi }}_{gj}|x_{gi},{\mu }_g,{\mathsf{\Lambda }}_g,F_{gi}\right)\sim IG\left(\frac{n}{2}+c,\frac{1}{2}{\displaystyle \sum _{i=1}^n}{\left(x_{gij}-{\mu }_{gj}^T-{\mathsf{\Lambda }}_{gj}^T{F}_{gi}\right)}^2+d\right)\hfill \end{array}$$

where $\overline{M}=\frac{1}{n_g}{\displaystyle \sum _{i=1}^{n_g}}\left(x_{gi}-{\mathsf{\Lambda }}_g{F}_{gi}\right)$. The estimated results of the unknown parameters of the factor analysis model for clustered data can be obtained by employing Gibbs sampling at the cluster level. However, it is necessary to ascertain whether the effect of each group within the clustered data is consistent and in line with the assumptions of the established model. Furthermore, it is crucial to determine whether there are clusters that deviate significantly from the original model. Statistical diagnosis of the above models represents an indispensable component of research involving clustered data.

4. Principal Component Method for Factor Analysis Models with Clustered Data

In the initial phase of our investigation, we focus on the factor analysis model for a single cluster with the matrix representation presented in Equation (2). The eigenvalues of the covariance matrix are labeled by ${\lambda }_1,{\lambda }_2,\dots ,{\lambda }_p$, arranged in descending order of magnitude, where ${\lambda }_1$ is the largest. The corresponding orthogonal eigenvector is denoted by ${\xi }_1$. If ${\mathsf{\Lambda }}_g{\mathsf{\Lambda }}_g^T$ is sufficiently large and the variance of the specific factors is considered negligible, ${\mathsf{\Sigma }}_g$ can be approximately replaced by ${\mathsf{\Lambda }}_g{\mathsf{\Lambda }}_g^T$, just like ${\mathsf{\Sigma }}_g\approx {\mathsf{\Lambda }}_g{\mathsf{\Lambda }}_g^T$. Using spectral decomposition, we decompose the covariance matrix with the specific form shown below:

$$\begin{array}{c}\hfill {\mathsf{\Sigma }}_g\approx {\mathsf{\Lambda }}_g{\mathsf{\Lambda }}_g^T=\sum _{i=1}^p{\lambda }_i{\widehat{e}}_{gi}{\widehat{e}}_{gi}^T,\end{array}$$

(8)

where ${\lambda }_i$ is the eigenvalue of the covariance matrix, denoted as ${\mathsf{\Lambda }}_g=diag\left({\widehat{\lambda }}_{g1},{\widehat{\lambda }}_{g2},\dots ,{\widehat{\lambda }}_{gm}\right)$$\in {\mathbb{R}}^{m\times m}$, and ${\widehat{e}}_{gi}$ is the corresponding orthogonal eigenvector, denoted as ${\widehat{E}}_{g=}({\widehat{e}}_{g1},{\widehat{e}}_{g2},\dots ,$${\widehat{e}}_{gm})\in {\mathbb{R}}^{p\times m}$. Since $F_{gi}$ obeys a normal distribution with a non-zero mean in this paper, the estimator for the factor loading matrix, derived from the principal components method, is delineated as follows:

$$\begin{array}{c}\hfill {\widehat{\mathsf{\Lambda }}}_g={\widehat{E}}_g{\mathsf{\Lambda }}_g^{\frac{1}{2}}{b}^{-\frac{1}{2}}{I}_m.\end{array}$$

(9)

The common factor $F_g$ can be estimated using the least-squares algorithm as shown in Equation (10):

$$\begin{array}{c}\hfill {\widehat{F}}_g={\left({\widehat{\mathsf{\Lambda }}}_g\right)}^{-\frac{1}{2}}{\widehat{E}}_g^T\left(X_g^T-{\mu }_g{I}_{1\times n_g}\right).\end{array}$$

(10)

Since all clusters share a common error matrix, the parameter estimates for the factor model in clustered data can be derived by averaging the estimation results from each cluster.

5. Statistical Diagnosis of Factor Model for Clustered Data

5.1. Case-Deletion Model

The case-deletion model is a diagnostic tool used to assess whether observed data points are influential points. It involves comparing the differences in corresponding statistics or parameters after deleting one point to determine whether it is an influential point. The model after deleting the mth group can be expressed as Equation (11):

$$\begin{array}{c}\hfill X_g=1_{n_g\times 1}{\mu }_g^T+F_g^T{\mathsf{\Lambda }}_g^T+{\epsilon }_g.\end{array}$$

(11)

Let $x_{\left(m\right)},{\mu }_{\left(m\right)},F_{\left(m\right)},{\mathsf{\Lambda }}_{\left(m\right)},{\epsilon }_{\left(m\right)}$ represent the dataset after deleting the mth group. The MCMC method is used to estimate parameters after data deletion, and Cook’s posterior mean distance is used to identify the influence between clusters.

5.2. Cook’s Posterior Mean Distance

To study the posterior mean effect of the gth group on unknown parameter $\theta$, $\theta =\left({\mu }_g,{\mathsf{\Lambda }}_g,F_g,{\mathsf{\Psi }}_g\right)$. The Bayesian case-deletion influence measure of deleting the gth cluster is defined by Cook’s posterior mean distance, as shown in Equation (12):

$$\begin{array}{c}\hfill C{M}_g={\left(\widehat{\theta }-\widehat{\theta }\left(g\right)\right)}^T{\mathsf{\Psi }}_{\theta }\left(\widehat{\theta }-\widehat{\theta }\left(g\right)\right),\end{array}$$

(12)

where $\widehat{\theta }=\int \theta p\left(\theta |x_g\right)d\theta$ and $\widehat{\theta }\left(g\right)=\int \theta p\left(\theta |x_g\left(g\right)\right)d\theta$, respectively, represent the posterior mean before and after the deletion of the gth cluster, ${\mathsf{\Psi }}_{\theta }$ represents the inverse matrix of the covariance of $\theta$, and ${\mathsf{\Psi }}_{\theta }$ is a positive definite matrix. If the value of $C{M}_g$ is larger, the difference between $\widehat{\theta }$ and $\widehat{\theta }\left(g\right)$ is greater, and the gth cluster is the strong-influence cluster or the anomalous cluster.

6. Simulation Study

6.1. Random Parameter Generation for the Markov Chain Monte Carlo Algorithm

A clustered dataset is simulated using Monte Carlo methods. The construction of a single cluster factor analysis model is initially considered, with the caveat that the number of samples in each cluster is not necessarily identical. To this end, the number of samples was allowed to adhere to a uniform distribution of $\mathcal{U}[40,50]$, with samples generated sequentially and iteratively from the corresponding conditional distribution. The factor loading matrix is of order 6 × 2, and the initial parameter settings are as follows:

$${\mathsf{\Lambda }}_g={\left(\begin{array}{cccccc}{\mathsf{\Lambda }}_{11}& {\mathsf{\Lambda }}_{12}& {\mathsf{\Lambda }}_{13}& {\mathsf{\Lambda }}_{14}& {\mathsf{\Lambda }}_{15}& {\mathsf{\Lambda }}_{16}\\ {\mathsf{\Lambda }}_{21}& {\mathsf{\Lambda }}_{22}& {\mathsf{\Lambda }}_{23}& {\mathsf{\Lambda }}_{24}& {\mathsf{\Lambda }}_{25}& {\mathsf{\Lambda }}_{26}\end{array}\right)}^T.$$

The initial coefficients of the factor loading matrix are set as ${\mathsf{\Lambda }}_{11}={\mathsf{\Lambda }}_{12}={\mathsf{\Lambda }}_{13}={\mathsf{\Lambda }}_{14}={\mathsf{\Lambda }}_{15}={\mathsf{\Lambda }}_{16}=0.9, {\mathsf{\Lambda }}_{21}={\mathsf{\Lambda }}_{22}={\mathsf{\Lambda }}_{23}={\mathsf{\Lambda }}_{24}={\mathsf{\Lambda }}_{25}={\mathsf{\Lambda }}_{26}=0.1$, ${\mathsf{\Lambda }}_{gj}^T\sim N_2\left({\alpha }_{1j},{\mathsf{\Sigma }}_{1j}\right),j=1,\dots ,6$. In order to ensure that the common factors satisfy different normal distributions in different clusters, it is necessary to assume that the common factor satisfies normal distributions with different means and variances. Furthermore, it is assumed that the mean values follow the uniform distribution of $\mathcal{U}[0,1]$, and the variance follows the uniform distribution of $\mathcal{U}[0,0.5]$, just like $F_{gi}\sim N_2\left({\alpha }_1,{\mathsf{\Sigma }}_1\right),{\alpha }_1\sim$$\mathcal{U}[0,1]$, ${\mathsf{\Sigma }}_1\sim$$\mathcal{U}[0,0.5]$. The random error term is also assumed to follow a normal distribution, just like ${\epsilon }_{gi}\sim N_6\left(0,{\mathsf{\Psi }}_g\right)$, where ${\mathsf{\Psi }}_g$ obeys the inverse gamma distribution, that is, ${\mathsf{\Psi }}_g\sim I{G}_6\left(c,d\right)$. In the prior distribution, the hyperparameters are selected as ${\alpha }_{1j}={\left(0.9,0.1\right)}^T,{\mathsf{\Sigma }}_{1j}=diag\left(0.1,0.1\right),c=3,$ and d = 4. This paper determines the convergence and stability of the Gibbs sampling by traversing the ergodic mean plot, as shown in Figure 1.

$${\widehat{\mu }}_g=\frac{1}{n-l}\sum _{l+1}^n{\mu }_g^{\left(m\right)},{\widehat{\mathsf{\Lambda }}}_{gj}^T=\frac{1}{n-l}\sum _{l+1}^n{\mathsf{\Lambda }}_{gj}^{\left(m\right)},{\widehat{F}}_{gi}=\frac{1}{n-l}\sum _{l+1}^n{F}_{gi}^{\left(m\right)},{\widehat{\mathsf{\Psi }}}_g=\frac{1}{n-l}\sum _{l+1}^n{\mathsf{\Psi }}_g^{\left(m\right)}$$

6.2. Simulation Results of Gibbs Sampling

Gibbs sampling is leveraged to estimate the parameters ${\widehat{\mathsf{\Lambda }}}_g$, ${\widehat{f}}_{gi}$, and ${\widehat{\mathsf{\Psi }}}_g$ through a process executed over 50 iterations. The initial estimates for parameters $f_{gi}$ and ${\epsilon }_{gi}$ are derived from the mean of 1000 samples, each drawn from the corresponding prior distribution. For the generation of the simulated clustered dataset, the number of clusters is variably set to 10, 30, and 50. A comprehensive presentation of the simulation outcomes can be found in

Table 1. Estimated results for the Gibbs sampling algorithm for clustered data.

Clusters	10			30			50
Parameters	Bias	RMSE	SD	Bias	RMSE	SD	Bias	RMSE	SD
$F_1$	−0.0054	0.05185	0.04919	−0.0045	0.04508	0.03047	0.0067	0.05240	0.04949
$F_2$	−0.0045	0.07089	0.04995	−0.0028	0.04621	0.04405	−0.0022	0.04212	0.03937
${\mathsf{\Lambda }}_{11}$	0.0077	0.03157	0.03156	0.0063	0.03155	0.03153	−0.0054	0.02291	0.02291
${\mathsf{\Lambda }}_{21}$	0.0035	0.03170	0.03168	0.0020	0.03147	0.03145	−0.0018	0.02295	0.02294
${\mathsf{\Lambda }}_{31}$	0.0058	0.03161	0.03159	0.0033	0.03139	0.03138	0.0029	0.02323	0.02321
${\mathsf{\Lambda }}_{41}$	−0.0019	0.03162	0.03159	0.0014	0.03171	0.03169	0.0013	0.02288	0.02288
${\mathsf{\Lambda }}_{51}$	−0.0098	0.03178	0.03176	0.0071	0.03155	0.03152	0.0068	0.02291	0.02290
${\mathsf{\Lambda }}_{61}$	0.0043	0.03162	0.03159	0.0081	0.03165	0.03163	−0.0037	0.02305	0.02302
${\mathsf{\Lambda }}_{12}$	0.0041	0.03162	0.03158	0.0038	0.03156	0.03154	0.0031	0.02412	0.02411
${\mathsf{\Lambda }}_{22}$	0.0039	0.03149	0.03146	0.0075	0.03153	0.03150	−0.0070	0.02400	0.02400
${\mathsf{\Lambda }}_{32}$	−0.0092	0.03172	0.03170	0.0061	0.03161	0.03160	−0.0054	0.02440	0.02438
${\mathsf{\Lambda }}_{42}$	−0.0023	0.03183	0.03180	0.0022	0.03167	0.03165	0.0020	0.02411	0.02410
${\mathsf{\Lambda }}_{52}$	0.0063	0.03144	0.03142	0.0070	0.03166	0.03163	0.0069	0.02415	0.02414
${\mathsf{\Lambda }}_{62}$	0.0066	0.03173	0.03170	0.0032	0.03163	0.03161	−0.0022	0.02420	0.02419
${\mathsf{\Psi }}_{11}$	0.0643	0.13252	0.09980	0.0509	0.12967	0.09969	−0.0487	0.12818	0.09865
${\mathsf{\Psi }}_{22}$	−0.0662	0.13391	0.10050	−0.0663	0.13188	0.10016	0.0106	0.13039	0.09804
${\mathsf{\Psi }}_{33}$	−0.0647	0.13144	0.09782	−0.0681	0.13391	0.10050	−0.0637	0.13075	0.10042
${\mathsf{\Psi }}_{44}$	0.0863	0.13012	0.09901	0.0828	0.12700	0.09974	0.0696	0.12622	0.09833
${\mathsf{\Psi }}_{55}$	0.0412	0.12839	0.09846	0.0387	0.12775	0.09789	0.0652	0.12815	0.09924
${\mathsf{\Psi }}_{66}$	0.0650	0.13257	0.09719	−0.0612	0.12995	0.09647	0.0593	0.12607	0.09580

. In this paper, we use Bias to denote the deviation between the true value and the estimated mean, RMSE to indicate the root mean square error, and SD to represent the standard deviation.

$$Bias=\frac{1}{n}\sum _{k=1}^n\left({\widehat{\theta }}^{\left(k\right)}-\theta \right),RMSE=\sqrt{\frac{1}{n}{\sum _{k=1}^n\left({\widehat{\theta }}^{\left(k\right)}-\theta \right)}^2},SD=\sqrt{\frac{1}{n-1}{\sum _{k=1}^n\left({\widehat{\theta }}^{\left(k\right)}-E\left({\widehat{\theta }}^{\left(k\right)}\right)\right)}^2}$$

Table 1 shows that, after many iterations, the bias and mean square error of all parameter estimates are controlled within a tightly constrained range, and the mean square error and variance are relatively close to each other, which, in turn, indicates that the estimation results are close to the real values, revealing the effectiveness of the factor model for dealing with clustered data when using this method. The principal component method, as a classical estimation method for factor analysis models, compares the parameter estimation based on two different methods under the same settings as the simulated dataset in the paper, and the estimation results of the principal component method are as shown in the table below.

6.3. Random Parameters Generation for the Principal Component Algorithm

Since the principal component method ignores the effect of variance, the parameters of the principal component method are set to be approximately the same as the Bayesian method for comparison purposes. The coefficients of the factor loading matrix are set as follows:

$${\mathsf{\Lambda }}_g={\left(\begin{array}{cccccc}{\mathsf{\Lambda }}_{11}& {\mathsf{\Lambda }}_{12}& {\mathsf{\Lambda }}_{13}& {\mathsf{\Lambda }}_{14}& {\mathsf{\Lambda }}_{15}& {\mathsf{\Lambda }}_{16}\\ {\mathsf{\Lambda }}_{21}& {\mathsf{\Lambda }}_{22}& {\mathsf{\Lambda }}_{23}& {\mathsf{\Lambda }}_{24}& {\mathsf{\Lambda }}_{25}& {\mathsf{\Lambda }}_{26}\end{array}\right)}^T.$$

The initial coefficients of the factor loading matrix are set as ${\mathsf{\Lambda }}_{11}={\mathsf{\Lambda }}_{12}={\mathsf{\Lambda }}_{13}={\mathsf{\Lambda }}_{14}={\mathsf{\Lambda }}_{15}={\mathsf{\Lambda }}_{16}=0.9, {\mathsf{\Lambda }}_{21}={\mathsf{\Lambda }}_{22}={\mathsf{\Lambda }}_{23}={\mathsf{\Lambda }}_{24}={\mathsf{\Lambda }}_{25}={\mathsf{\Lambda }}_{26}=0.1$. We assume that the common factor satisfies normal distributions with different means and variances. The mean of $F_g$ is assumed to be uniformly distributed between 0 and 1, and the variance is assumed to be uniformly distributed between 0 and 0.5. $F_{gi}\sim N_2\left({\alpha }_1,{\mathsf{\Sigma }}_1\right),{\alpha }_1\sim$$\mathcal{U}[0,1]$, ${\mathsf{\Sigma }}_1\sim$$\mathcal{U}[0,0.5]$, and the random error ${\epsilon }_{gi}$ is distributed as $N_p\left(0,{\mathsf{\Psi }}_g\right)$. The initial values of ${\mathsf{\Psi }}_g$ are set to be the same as the initial value of ${\mathsf{\Psi }}_g$ under the Gibbs sampling algorithm. The intra-group correlation is also set as $x_{gi}=0.8x_{gi-1}+{\delta }_i,{\delta }_i\sim N_6\left(0,{\mathsf{\Psi }}_g\right)$.

6.4. Simulation Results of Principal Component Algorithms

The parameters of interest, ${\widehat{\mathsf{\Lambda }}}_g$ and ${\widehat{f}}_g$, can be estimated using principal component analysis. To gauge the accuracy of these estimates, the deviation, standard deviation, and mean square error for the pairs of ${\widehat{\mathsf{\Lambda }}}_g$ and ${\mathsf{\Lambda }}_g$, as well as ${\widehat{f}}_g$ and $f_g$, are computed. This process is repeated 1000 times to ensure a robust assessment of how closely the estimated values of the parameters approximate their actual values. The number of clusters is also set to 10, 30, and 50, and the estimated results are depicted in

Table 2. Estimated results for the principal component algorithm method for clustered data.

Clusters	10			30			50
Parameters	Bias	RMSE	SD	Bias	RMSE	SD	Bias	RMSE	SD
$F_1$	−0.54662	0.5250	0.1785	−0.52588	0.5234	0.1797	−0.52138	0.4692	0.1792
$F_2$	−0.52657	0.5393	0.2127	−0.52966	0.5402	0.2156	−0.52526	0.4868	0.2139
${\mathsf{\Lambda }}_{11}$	0.62061	0.8119	0.5276	0.60196	0.7891	0.5168	0.61905	0.7911	0.5129
${\mathsf{\Lambda }}_{21}$	0.61813	0.7897	0.5115	0.61369	0.7987	0.5179	0.60584	0.7908	0.5120
${\mathsf{\Lambda }}_{31}$	0.61785	0.7916	0.5147	0.61234	0.7989	0.5197	0.60830	0.7978	0.5201
${\mathsf{\Lambda }}_{41}$	0.61547	0.8018	0.5208	0.60478	0.7793	0.5119	0.61607	0.8057	0.5234
${\mathsf{\Lambda }}_{51}$	0.61554	0.7893	0.5145	0.61900	0.8053	0.5218	0.60848	0.7975	0.5195
${\mathsf{\Lambda }}_{61}$	0.61820	0.7898	0.5108	0.61384	0.7997	0.5191	0.60794	0.7952	0.5167
${\mathsf{\Lambda }}_{12}$	−0.08237	0.4492	0.4398	−0.07056	0.4552	0.4493	−0.08871	0.4579	0.4488
${\mathsf{\Lambda }}_{22}$	−0.05652	0.4555	0.4503	−0.07448	0.4587	0.4523	−0.07114	0.4588	0.4521
${\mathsf{\Lambda }}_{32}$	−0.08293	0.4546	0.4459	−0.08121	0.4578	0.4502	−0.07911	0.4555	0.4496
${\mathsf{\Lambda }}_{42}$	−0.09258	0.4531	0.4424	−0.08322	0.4550	0.4481	−0.07380	0.4593	0.4522
${\mathsf{\Lambda }}_{52}$	−0.08433	0.4562	0.4449	−0.07415	0.4575	0.4516	−0.08288	0.4601	0.4523
${\mathsf{\Lambda }}_{62}$	−0.07983	0.4568	0.4499	−0.07234	0.4553	0.4450	−0.07302	0.4583	0.4524

.

As can be seen from Table 1 and Table 2, the estimation results of the MCMC method are better than the estimation results of the principal component method for each item, accordingly indicating that the MCMC method is more effective relative to the principal component method. The corresponding influence diagnosis is reliable only if the estimation results are better.

6.5. Statistical Diagnosis of Clustered Data

The dataset under consideration is subjected to statistical diagnosis using the case-deletion method. Cook’s distance is utilized to evaluate the influence of individual clusters by comparing the changes in CM(g)—a measure of leverage and influence—following the removal of different clusters. Within the simulation study, the common factor variable, denoted as $F_g$, for the 8th, 15th, 21st, and 42nd clusters, is incremented by 2 to appraise the validity of the applied method. The variable of $F_g$ is designated as the diagnostic parameter of interest. The outcomes of this analysis are presented in Figure 2.

Figure 2 presents a depiction of Cook’s distance for clusters consisting of 10, 30, and 50 clusters, which are generated through Monte Carlo simulations and subsequently deleted sequentially. Analysis of Figure 2 reveals that, within the clustered data produced by Monte Carlo simulation, the 8th cluster exerts the most significant influence. In the dataset comprising 30 groups, the 8th, 15th, and 21st clusters are identified as potent influence points, with the 21st cluster demonstrating the most pronounced impact. Similarly, in the dataset with 50 clusters, the 8th, 15th, 21st, and 42nd clusters are recognized as having substantial influence, with the 21st cluster again being the most influential. These findings are congruent with the intentional perturbations introduced to the 8th, 15th, 21st, and 42nd clusters, as previously discussed. Consequently, the efficacy of the diagnostic methodology employed can be assessed and deemed valid.

7. Example Analysis

In this section, we detail the experimental analysis of cigarette sales. Our study is based on data derived from a survey of cigarette sales which was conducted across the United States from 1963 to 1992. This dataset is accessible through the Wiley Online Library at https://www.wiley.com/legacy/vkhbwileychi/baltagi/ (accessed on 14 February 2024). The survey, conducted by Baltagi [25], aimed to investigate cigarette consumption patterns across 46 different states within the United States. A comprehensive description of the variables included in the dataset is presented in

Table 3. Description of data variables.

Data Variables	Definition
State	The state surveyed for cigarette data
Year	Time of data investigation
Price	Retail price of cigarettes
Pop	The local population
Pop16	The number of local people older than 16
Cpi	Local consumer price index
Ndi	National Disposable Income
Sales	Per capita sales of cigarettes
Pimin	Minimum price per pack of cigarettes

. The dataset, which is inherently clustered by state, has been modeled as such for analysis. Before conducting factor analysis, the data underwent standardization. Subsequently, the suitability of the data for factor analysis was assessed using the KMO test and Bartlett’s spherical test. The outcomes are detailed in

Table 4. KMO test and Bartlett’s spherical test.

Index		Value
KMO test		0.78
Bartlett’s spherical test	K-squared	72,607
	p-value	<$2.2\times {10}^{-16}$

.

As detailed in Table 4, the KMO test for sampling adequacy is calculated at 0.78, a value that exceeds the threshold commonly accepted for proceeding with factor analysis. Moreover, Bartlett’s test for sphericity yields a p-value of less than 0.05, which is statistically significant and further corroborates the appropriateness of the data for this analytical approach. Figure 3 indicates that the extraction of three common factors is justified, thus leading to the development of a factor analysis model inclusive of these three factors.

Figure 4 illustrates the classification of the eight indicators into three separate categories based on their factor loading coefficients. Specifically, the indicators labeled as Price, Cpi, Pimin, Ndi, and Year are categorized together as representing economic factors. The indicators Pop and Pop16 are united to form the demographic factor. Finally, the Sales indicator is identified as the sole representative of the demand factor.

Examination of Figure 5 reveals that the mean of the trace for each variable stabilizes upon completion of 3000 iterations, suggesting that the Gibbs sampling algorithm has achieved convergence. To ensure reliability, the analysis was conducted with a total of 5000 iterations, with the initial 3000 iterations serving as a burn-in period. The subsequent 2000 iterations were retained for analysis, yielding a set of valid samples. Statistical analysis of these valid samples was then performed. Anderson and Herman [26] have delineated that the core condition for the identifiability of the factor loading matrix is the requirement for ${\mathsf{\Lambda }}_g^T{\mathsf{\Psi }}_g^{-1}{\mathsf{\Lambda }}_g$ to be diagonal. Then, ${\mathsf{\Lambda }}_g$ can be identified. The resulting estimates for the factor loading matrix coefficients are presented in

Table 5. Estimated effects of the Gibbs sampling.

Clusters		46			46			46
Common Factor		Economic			Population			Demand
Parameters	Mean	Median	SD	Mean	Median	SD	Mean	Median	SD
Year	0.9897	0.9893	0.0206	0.0511	0.0510	0.0214	−0.0418	−0.0412	0.0213
Price	0.9608	0.9613	0.0144	−0.2603	−0.2604	0.0152	0.0907	0.0906	0.0164
Pop	0.9911	0.9908	0.0360	0.0083	0.0084	0.0390	−0.0980	−0.0979	0.0392
pop16	0.9908	0.9910	0.0103	0.1074	0.1073	0.0104	−0.0807	−0.0800	0.0110
Cpi	0.9959	0.9959	0.0138	−0.0473	−0.0472	0.0140	−0.0459	−0.0455	0.0147
Ndi	0.9917	0.9920	0.0120	−0.1120	0.1197	0.0129	0.0347	0.0348	0.0123
Sales	0.6296	0.6295	0.0152	0.7726	0.7725	0.0153	0.0767	0.0765	0.0153
Pimin	0.9650	0.9647	0.0271	−0.2448	−0.2461	0.0291	0.0878	0.0889	0.0305

.

According to the results shown in Table 5, the difference between the mean and median of the loading coefficient is small, indicating that the fitting effect of the MCMC method is better. Baltagi [25] studied the variation in cigarette price elasticized across states in this dataset using a heterogeneous coefficient model but did not consider the issue of commonality between states. The clustered factor analysis model can not only obtain the internal relationship among these states, but also describe the common factors among different states. The results are also richer than the analytical effects of the heterogeneous coefficient model. In order to highlight the advantages of clustered data, the quality of the model can be judged by two methods: the Akaike Information criterion (AIC) and Bayesian information criterion (BIC). Akaike [27] proposed a method to calculate the AIC value in a factor analysis model, and the AIC and BIC values calculated through this calculation method are shown in

Table 6. Comparison of AIC and BIC values with Gibbs sampling and principal component analysis.

	MCMC Method		Principal Component Method
	AIC	BIC	AIC	BIC
The within-cluster correlation is considered	398.4	359.2	739.9	688.1
The within-cluster correlation is not considered	4272.9	4192.1	14,129.3	14,077.5

.

As can be seen from Table 6, the AIC and BIC values of the model considering the internal relationship of clusters are much lower than those of the model which does not consider the internal relationship, which indicates that considering the internal relationship of clustered data is better than ignoring the internal relationship of clusters, and, at the same time, it illustrates that the estimation effect of the MCMC method is better than the principal component method.

In addition, the Bayesian case-deletion model was applied to these data in this study as a way to explore those clusters with anomalies or high-influence clusters. When the common factor $F_{gi}$ and factor loading matrix ${\mathsf{\Lambda }}_g$ are the parameters of interest and the rest of the parameters are redundant, the influence diagnostic results are as shown in Figure 6 and Figure 7, respectively.

Figure 6 presents visual evidence suggesting that, based on Cook’s distance, the 17th cluster can be classified as an outlier or a cluster exerting a significant influence on the model regardless of whether parameter $F_g$ or ${\mathsf{\Lambda }}_g$ is the focus of interest. A comparison of the arithmetic means of indicators within the original dataset reveals that the 17th state’s cigarette price is substantially lower than the national average. At the same time, this state has a significantly smaller local population compared to other states. However, the local consumer price index and per capita sales are observed to be higher. These observations lead to the inference that the 17th state may wield a more substantial influence. To investigate the potential masking effect among influential clusters, we performed a sequential exclusion analysis, initially omitting the 17th cluster, before conducting the Bayesian statistical diagnosis. The results of these diagnostic assessments are depicted in Figure 7.

Figure 7 provides a clear depiction revealing that the Cook’s distance values for the parameters of the non-17th clusters exhibit minimal variation. This observation suggests that there is no significant masking effect among these clusters. Concurrently, the figure also indicates that the 17th cluster exerts a more substantial influence on the estimation of the parameters.

8. Discussion

Clustered data, one of the complex data types, are characterized by internal correlation within the same groups and a lack of correlation between different groups. Ignoring this nested structure when inferring can lead to biased conclusions. Utilizing the factor analysis model is an effective approach to address the nested structure inherent in clustered data. This paper presents a factor model specifically tailored for clustered data. Employing the MCMC method in conjunction with Gibbs sampling, we initially estimate the unknown parameters of an individual cluster. The inter-cluster association is depicted through a shared diagonal error matrix. Thereby, the factor model with clustered data is constructed by integrating multiple clusters. Numerical simulations substantiate the model’s and methodology’s validity. Empirical evidence indicates that the clustered data model significantly outperforms models that disregard internal relationships. Furthermore, this study employs Bayes deletion influence to compare diagnostic statistics before and after cluster deletion and utilizes Cook’s posterior mean distance to identify potential outliers among clusters. Both the simulation study and the example analysis yield promising results.

The research concepts presented are extendable to structural equation models with clustered data. As structural equation models are an advanced form of the factor analysis model, the modeling approaches discussed are not only applicable to them but also offer a foundation for future statistical diagnostic research on clustered data across varying models.