Bayesian Treatments for Panel Data Stochastic Frontier Models with Time Varying Heterogeneity

Abstract

This paper considers a linear panel data model with time varying heterogeneity. Bayesian inference techniques organized around Markov chain Monte Carlo (MCMC) are applied to implement new estimators that combine smoothness priors on unobserved heterogeneity and priors on the factor structure of unobserved effects. The latter have been addressed in a non-Bayesian framework by Bai (2009) and Kneip et al. (2012), among others. Monte Carlo experiments are used to examine the finite-sample performance of our estimators. An empirical study of efficiency trends in the largest banks operating in the U.S. from 1990 to 2009 illustrates our new estimators. The study concludes that scale economies in intermediation services have been largely exploited by these large U.S. banks.

1. Introduction

In this paper, we consider two panel data models with unobserved heterogeneous time-varying effects; one with individual effects treated as random functions of time, and the other with common factors whose number is unknown and whose effects are firm-specific. This paper has two distinctive features and can be considered as a generalization of traditional panel data models. First, the individual effects that are assumed to be heterogeneous across units, as well as to be time varying, are treated non-parametrically, following the spirit of the models of Bai (2009, 2013), Li et al. (2011), Kneip et al. (2012), Ahn et al. (2013), and Bai and Carrion-i-Silverstre (2013). Second, we develop methods that allow us to interpret the effects as measures of technical efficiency in the spirit of the structural productivity approaches of Olley and Pakes (1996) and non-structural approaches from the stochastic frontier literature (Kumbhakar and Lovell 2000; Fried et al. 2008). Levinsohn and Petrin (2003), Kim et al. (2016), and Ackerberg et al. (2015) have provided rationales for various treatments for the endogeneity of inputs and the appropriate instruments or control functions to deal with the potential endogeneity of inputs and of technical change based on variants of the Olley-Pakes basic model set up. Although we do not explicitly address entry/exit in this paper, we do address dynamics, as well as the potential endogeneity of inputs and the correlation of technical efficiency with input choice (Amsler et al. 2016). The general factor structure we utilize can pick up potential nonlinear selection effects that may be introduced when using a balanced panel of firms. Our dynamic heterogeneity estimators could be interpreted as general controls for any mis-specified factors, such as selectivity due to entry/exit, that are correlated with the regressors and could ultimately bias slope coefficients. Olley and Pakes (1996) utilize series expansions and kernel smoothers to model such selectivity. Our second estimator instead utilizes a general factor structure, which is a series expansion with a different set of basis functions than those used in the polynomial expansions employed by Olley-Pakes. Alternatively, we can interpret the effects based on a panel stochastic frontier production specification that formally models productive efficiency as a stochastic shortfall in production, given the input use. Van den Broeck et al. (1994) formulate a Bayesian approach under a random effects composed error model, while Koop et al. (1997) and Osiewalski and Steel (1998) provided extensions to the fixed effect model utilizing Gibbs sampling and Bayesian numerical methods, but these studies assumed that the individual effects were time invariant. Comparisons between the Bayes and classical stochastic frontier estimators have been made by Kim and Schmidt (2000). The estimators we consider are specified in the same spirit as Tsionas (2006), who assumed that the effects evolve log-linearly. We do not force the time-varying effects to follow a specific parametric functional form and utilize Bayesian integration methods and a Markov chain-based sampler to provide the slope parameter and heterogeneous individual effects inferences based on estimators of the posterior means of the model parameters.

The paper is organized as follows. Section 2 describes the first model setup and parameter priors. Section 3 introduces the second model and the corresponding Bayesian inferences, followed by Section 4, presenting the Monte Carlo simulations results. The estimation of the translog distance function is briefly discussed and the empirical application results of the Bayesian estimation of the multi-output/multi-input technology employed by the U.S. banking industry in providing intermediation services are presented in Section 5. Section 6 provides the concluding remarks.

2. Model 1: A Panel Data Model with Nonparametric Time Effects

Our first model is based on a balanced design with T observations for n individual units. Observations in the panel can be represented in the form $(y_{it},x_{it}),i=1,\dots ,n;t=1,\dots ,T$, where the index i denotes the ith individual unit, and the index t denotes the tth time period.

A panel data model with heterogeneous time-varying effects is:

$$y_{it}=x_{it}^{}\beta +{\gamma }_{it}+v_{it}$$

(1)

where $y_{it}$ is the response variable, $x_{it}$ is a $1\times p$ vector of the explanatory variables, $\beta$ is a $p\times 1$ vector of the parameters, and ${\gamma }_{it}$ is a nonconstant and unknown individual effect. We make a standard assumption that the measurement error $v_{it}\sim NID(0,{\sigma }^2)$. The time-varying heterogeneity is assumed to be independent across units. This assumption is quite reasonable in many applications, particularly in production/cost stochastic frontier models where the effects are measuring technical efficiency levels. A firm’s efficiency level primarily relies on its own factors such as its executives’ managerial skills, the firm size, and the operational structure, etc., and should thus be heterogeneous across firms. These factors usually change over time, as does the firm’s efficiency level.

For the ith unit, the model is:

$$Y_i=X_i\beta +{\gamma }_i+v_i,i=1,\dots ,n$$

(2)

where $Y_i,X_i$, and ${\gamma }_i$ contain the stacked vectors of dimension T for cross-section i.

When interpreting the effects as firm efficiencies, as is done in stochastic frontier analysis (Pitt and Lee 1981; Schmidt and Sickles 1984), the estimation of time-varying technical efficiency levels is as important as that of the slope parameters.

A difference between our model and many other Bayesian approaches in the literature is that no functional form for the prior distribution of the unobserved heterogeneous individual effects is imposed. Instead of resorting to the classical nonparametric regression techniques (Kneip et al. 2012), a Markov chain Monte Carlo (MCMC) algorithm is implemented to estimate the model. We can consider this to be a generalization of Koop and Poirier (2004) in the case of panel data, including both individual-specific and time-varying effects. Moreover, our model does not rely on the restrictive conjugate prior formulation for the time varying individual-effects.

A Bayesian analysis of the panel data model set up above requires specification of the prior distributions over the parameters (γ, β, σ) and computation of the posterior using a Bayesian learning process:

$$p\left(\beta ,\gamma ,\sigma |Y,X,\omega \right)\propto p\left(\beta ,\sigma ,\gamma \right)\cdot l(Y,X;\beta ,\gamma ,\sigma ).$$

(3)

The prior for the individual effect ${\gamma }_i$ is not assumed to follow a normal distribution; instead, it is only assumed that the first-order or second-order difference of γ_i follows a normal prior.

$$p\left(\gamma \right)\propto {\displaystyle {\prod }_{i=1}^n\exp \left(-\frac{{\gamma }_i^{\prime }Q{\gamma }_i}{2{\omega }^2}\right)}=\exp \left(-\frac{1}{2{\omega }^2}{\gamma }^{\prime }\left(I_n\otimes Q\right)\gamma \right)$$

(4)

where $Q=D^{\prime }D$, and $D$ is the $\left(T-1\right)\times T$ matrix whose elements are $D_{tt}=1$, for t = 1,…,T − 1; $D_{t-1,t}=-1$ for all t = 2,…,T and zero otherwise. The information implied by this prior is that${\gamma }_{i,t}-{\gamma }_{i,t-1}~N\left(0,{\omega }^2\right)$, or $D{\gamma }_i\stackrel{IID}{~}N\left(0,{\omega }^2{I}_{T-1}\right)$. ω is the smoothness parameter that indexes the degree of smoothness. ω can be considered as a hyper-parameter, or it can be assumed to have its own prior, which is explained in the next section. Provided the continuity and first-order differentiability of${\gamma }_{i,t}$, this assumption says that the first derivative of the time-varying function${\gamma }_{i,t}$ in (4) is a smooth function of time. One can assume second-order differentiability instead, which is implied by the condition that ${\gamma }_{it}-2{\gamma }_{i,t-1}+{\gamma }_{i,t-2}~N\left(0,{\omega }^2\right),$$\mathrm{or}{D}^{\left(2\right)}{\gamma }_i\stackrel{IID}{~}N\left(0,{\omega }^2{I}_{T-2}\right)\mathrm{and}$$Q=D^{(2)\prime }{D}^{(2)}$.

A non-informative distribution is assumed for the joint prior of the slope parameter β and the unknown variance term σ².

$$p\left(\beta ,\sigma ,\gamma \right)\propto {\sigma }^{-1}$$

(5)

This is equivalent to assuming that the prior distribution is uniform on $(\beta ,\log \sigma )$.

With the assumptions of the priors above, the joint prior of the model parameters is:

$$p\left(\beta ,\sigma ,\gamma \right)\propto {\sigma }^{-1}{\displaystyle {\prod }_{i=1}^n\exp \left(-\frac{{\gamma }_i^{\prime }Q{\gamma }_i}{2{\omega }^2}\right)}={\sigma }^{-1}\exp \left(-\frac{1}{2{\omega }^2}{\gamma }^{\prime }\left(I_n\otimes Q\right)\gamma \right)$$

(6)

The corresponding sample likelihood function is:

$$l\left(Y, X, \beta ,\gamma ,\sigma \right) \propto {\sigma }^{-nT}\exp \{-\frac{1}{2{\sigma }^2}{\left(Y-X\beta - \gamma \right)}^{\prime }\left(Y-X\beta - \gamma \right)\}$$

(7)

The likelihood is formed by the product of the nT independent disturbance terms, which follow the normal distribution for the idiosyncratic error, assumed to be NID (0, σ²). Applying Bayes’ theorem, the probability density function is updated utilizing the information from the data and to form the joint posterior distribution given by:

$$\begin{array}{cc}\hfill p\left(\beta ,\gamma ,\sigma |Y,X,\omega \right)\propto & {\sigma }^{-\left(nT+1\right)}\exp \{-\frac{1}{2{\sigma }^2}{\left(Y-X\beta -\gamma \right)}^{\prime }\left(Y-X\beta -\gamma \right)\}\hfill \\ \hfill & \times \exp \{-\frac{1}{2{\omega }^2}{\gamma }^{\prime }\left(I_n\otimes Q\right)\gamma \}\hfill \end{array}$$

(8)

The model in (1) and (2) is identified provided we have a proper prior for the ${{\gamma }_i}^{\prime }s$. To accomplish this, we use (4) with a proper prior for $\omega$: $p\left(\omega \right)\propto {\omega }^{-(\overline{n}+1)}\exp \left(-\frac{\overline{q}}{2{\omega }^2}\right),\overline{n}\ge 0,\overline{q}>0$ (see (17) below), where $\overline{q}$ is the sum of squares with $\overline{n}$ observations. The “non-informative” case is to let $\overline{n}$, $\overline{q}\to 0$. We use $\overline{n}=1$ and $\overline{q}={10}^{-6}$ following standard practice (Geweke 1993). The posterior is well-defined and integrable. Such issues have been dealt with by Koop and Poirier (2004), whose spline method is equivalent to the difference prior we adopt.

To proceed with further inference, we need to solve this analytically. However, the joint posterior distribution does not have a standard form and taking draws directly from it is problematic. Therefore, we utilize Gibbs sampling to perform Bayesian inference. The Gibbs sampler is commonly used in such situations because of the desirable result that iterative sampling from the conditional distributions will lead to a sequence of random variables converging to the joint distribution. A general discussion on the use of Gibbs sampling is provided by Gelfand and Smith (1990), who compare the Gibbs sampler with alternative sampling-based algorithms. A more detailed discussion is given in Gelman et al. (2003). Gibbs sampling is well-adapted to sampling the posterior distributions for our model since it is possible to derive the collection of distributions.

The Gibbs sampling algorithm we employ generates a sequence of random samples from the conditional posterior distributions of each block of parameters, in turn conditional on the current values of the other blocks of parameters, and it thus generates a sequence of samples that constitute a Markov Chain, where the stationary distribution of that Markov chain is the desired joint distribution of all the parameters.

In order to derive the conditional posterior distributions of β, γ, and σ, we first rewrite the joint posterior in (8) as:

$$\begin{gathered} p(Y|\beta ,\gamma ,\sigma )\propto {\sigma }^{-nT}\exp \left\{-\frac{1}{2{\sigma }^2}{(Y-X\beta -\gamma )}^{\prime }(Y-X\beta -\gamma )\right\} \\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\propto {\sigma }^{-nT}\exp \left\{-\frac{1}{2{\sigma }^2}{[(Y-X\widehat{\beta }-\gamma )}^{\prime }(Y-X\widehat{\beta }-\gamma )+{(\beta -\widehat{\beta })}^{\prime }\left(X^{\prime }X\right)\left(\beta -\widehat{\beta }\right)]\right\} \end{gathered}$$

(9)

where $\widehat{\beta }={\left(X^{\prime }X\right)}^{-1}{X}^{\prime }\left(Y-\gamma \right)$.

The joint posterior can be rewritten as:

$$\begin{array}{l}p\left(\beta ,\gamma ,\sigma |Y,X,\omega \right)\propto {\sigma }^{-\left(nT+1\right)}\exp \{-\frac{1}{2{\omega }^2}{\gamma }^{\prime }\left(I_n\otimes Q\right)\gamma \}\\ \times \exp \{-\frac{1}{2{\sigma }^2}[{(Y-X\widehat{\beta }-\gamma )}^{\prime }(Y-X\widehat{\beta }-\gamma )+{(\beta -\widehat{\beta })}^{\prime }(X^{\prime }X)(\beta -\widehat{\beta })]\}.\end{array}$$

(10)

From (10), the conditional distribution of β can be shown to follow the multivariate normal distribution with the mean $\widehat{\beta }$ and covariance matrix${\sigma }^2{\left(X^{\prime }X\right)}^{-1}$.

$$p\left(\beta |Y,X,\gamma ,\sigma ,\omega \right)\propto \exp \{-\frac{1}{2{\sigma }^2}{(\beta -\widehat{\beta })}^{\prime }(X^{\prime }X)(\beta -\widehat{\beta })\}$$

(11)

The conditional distribution of $\beta$, therefore, is given by:

$$\beta |\sigma ,\gamma ,\omega ,Y,X\propto f_k\left(\beta |\widehat{\beta },{\sigma }^2{\left(X^{\prime }X\right)}^{-1}\right)$$

(12)

In order to derive the conditional distribution of the individual effect ${\gamma }_i$, we rewrite the joint posterior distribution as:

$$\begin{array}{l}p\left(\beta ,\gamma ,\sigma |Y,X,\omega \right)\propto \\ {\sigma }^{-\left(nT+1\right)}\exp \{-\frac{1}{2{\sigma }^2}{\left(\gamma -Y+X\beta \right)}^{\prime }\left(\gamma -Y+X\beta \right)-\frac{1}{2{\omega }^2}{\gamma }^{\prime }\left(I_n\otimes Q\right)\gamma \}\propto \\ {\sigma }^{-\left(nT+1\right)}\exp \{-\frac{1}{2{\sigma }^2}{\displaystyle {\sum }_{i=1}^n{\left({\gamma }_i-Y_i+X_i\beta \right)}^{\prime }\left({\gamma }_i-Y_i+X_i\beta \right)}-\frac{1}{2{\omega }^2}{\displaystyle {\sum }_{i=1}^n{{\gamma }_i}^{\prime }Q{\gamma }_i}\}\end{array}$$

(13)

Under the assumption that the effects, the ${{\gamma }_i}^{\prime }s$ are independent across units, the conditional posterior distribution of ${\gamma }_i|\beta ,\sigma ,\omega ,\left\{{\gamma }_j,j\ne i\right\},Y,X$ is the same as that of ${\gamma }_i|\beta ,\sigma ,\omega ,Y,X$, and is distributed as a multivariate normal.

$${\gamma }_i|\beta ,\sigma ,\omega ,\left\{{\gamma }_j,j\ne i\right\},Y,X~{\gamma }_i|\beta ,\sigma ,\omega ,Y,X\propto {\varphi }_T\left({\gamma }_i|{\widehat{\gamma }}_i,{\sigma }^2{\omega }^{2}V\right)$$

(14)

where the mean ${\widehat{\gamma }}_i$ and covariance matrix V are ${\widehat{\gamma }}_i={\omega }^{2}V\left(y_i-X_i\beta \right)$ and $V={\left({\sigma }^{2}Q+{\omega }^2{I}_T\right)}^{-1}$ for $i=1,\dots ,n$. The detailed derivation is presented in Appendix A.

The conditional posterior distribution for ${\sigma }^2$ is given below in (15). It is clear that the sum of the squared residuals ${\left(Y-X\beta -\gamma \right)}^{\prime }\left(Y-X\beta -\gamma \right)/{\sigma }^2$ has a conditional chi-squared distribution with nT degrees of freedom, as shown in (16):

$$p\left({\sigma }^2|\beta ,\gamma ,Y,X,\omega \right)\propto {({\sigma }^{-2})}^{nT/2-1}\exp \{-\frac{1}{2{\sigma }^2}{\left(Y-X\beta -\gamma \right)}^{\prime }\left(Y-X\beta -\gamma \right)\}$$

(15)

$$\frac{{\left(Y-X\beta -\gamma \right)}^{\prime }\left(Y-X\beta -\gamma \right)}{{\sigma }^2}|\beta ,\gamma ,\omega ,Y,X\sim {\chi }_{nT}^2.$$

(16)

If the smoothing parameter ω is also assumed to follow its own prior instead of being treated as a constant, then its conditional posterior distribution can also be derived. Suppose that $\frac{\overline{q}}{{\omega }^2}~{\chi }_{\overline{n}}^2$, where $\overline{n},\overline{q}\ge 0$ are hyper-parameters that control the prior degree of smoothness that is imposed on the ${{\gamma }_{it}}^{\prime }s$. Then, the conditional posterior distribution of ${\omega }^2$ is derived as:

$$\frac{\overline{q}+{\displaystyle {\sum }_{i=1}^n{\gamma }_i^{\prime }Q{\gamma }_i}}{{\omega }^2}|\beta ,\sigma ,\gamma ,Y,X~\frac{\overline{q}+{\displaystyle {\sum }_{i=1}^n{\gamma }_i^{\prime }Q{\gamma }_i}}{{\omega }^2}|\gamma ,Y,X~{\chi }_{\overline{n}+n}^2$$

(17)

Generally, small values of the prior “sum of squares” $\overline{q}/\overline{n}$ correspond to smaller values of $\omega$ and thus a higher degree of smoothness. Alternatively, we can choose the smoothing parameter $\omega$ using cross validation, which in a Bayesian context is similar to cross validation for tuning parameters in classical nonparametric regression. We choose the smoothing parameter ω so that the marginal likelihood (obtained as in Perrakis et al. 2014) is maximized.

A Gibbs sampler is then used to draw observations from the conditional posteriors based on (11) through (17). Draws from these conditional posteriors will eventually converge to the joint posterior in (8). Since the conditional posterior distribution of β follows the multivariate normal distribution displayed in (12), it will be straightforward to sample from it. For the individual effects ${\gamma }_i$, sampling is also straightforward since its conditional posterior follows a multivariate normal distribution with a mean vector ${\widehat{\gamma }}_i$ and covariance matrix ${\sigma }^2{\omega }^{2}V$, as expressed in (14).

Finally, to draw samples from the conditional posterior distribution function for the unobserved variance of the measurement error σ term, we have two simple steps. First, we draw samples directly from ${\left(Y-X\beta -\gamma \right)}^{\prime }\left(Y-X\beta -\gamma \right)/{\sigma }^2$, which is shown in (16) to follow a chi-squared distribution with the degree of freedom nT. Next, we assign the values of ${\left(Y-X\beta -\gamma \right)}^{\prime }\left(Y-X\beta -\gamma \right)/(Chi-rnd)$ to ${\sigma }^2$, where $(Chi-rnd)$ is the random generated variable that follows a ${\chi }_{nT}^2$ in the first step.

3. Model 2: A Panel Data Model with Factors

We next consider a somewhat different specification for the panel data model, wherein the effects are treated as a linear combination of unknown basis functions or factors:

$$y_{it}={\alpha }_i+x_{it}\beta +{\varphi }_t{\gamma }_i+v_{it}=x_{it}\beta +{\displaystyle {\sum }_{g=1}^G{\varphi }_{tg}{\gamma }_{ig}}+v_{it}.$$

(18)

Here, ${\varphi }_t$ is a $1\times G$ vector of common factors, ${\gamma }_i$ is a $G\times 1$ vector of individual-specific factor loadings, and ${\alpha }_i$ represents the firm-specific and time invariant effects. For these effects, we retain the Schmidt and Sickles (1984) interpretation of the fixed effects as measures of unit specific time invariant productivity (inefficiency), but we embed it in a Bayesian framework using the Bayesian Fixed Effects Specification (BFES) of Koop et al. (1997). Following their model specification, the BFES is characterized by marginal prior independence between the individual effects. Therefore, the effects are assumed not to be linked across firms, as would be the case for the spatial stochastic frontier considered by Glass et al. (2016).

As for measuring the inefficiency, the essence of the Schmidt and Sickles (1984) device in the Bayesian context is that, during the sth of the total S (MCMC) iterations or paths, inefficiency is constructed as the difference of the individual effect from the maximum effect across firms: $u_i^{(s)}={\alpha }_i^{(s)}-\underset{j=1,\dots ,n}{\max }{\alpha }_j^{(s)}$. Thus, one counts the most efficient firm in the sample as 100% efficient. However, there is uncertainty as to which firm we should use for benchmarking and this is resolved by averaging: ${\widehat{u}}_i=S^{-1}{\displaystyle \sum _{s=1}^S{u}_i^{(s)}}$ to account for both the parameter uncertainty, as well as the uncertainty regarding the best performing firm. The efficiency level of the most efficient firm in the sample approaches 1 when S → ∞. This method has much in common with the Cornwell et al. (1990) (CSS) estimator of time and firm specific productivity effects. The difference is that at each path of the Gibbs sampler, we have new draws for the ${{\alpha }_i}^{\prime }s$, a new value for $\underset{j=1,\dots ,n}{\max }{\alpha }_j^{(s)}$, and thus a new value for $u_i^{(s)}$. While CSS have one set of estimates and therefore a single firm to use as the benchmark, in the Bayesian approach, we have draws from the posterior of the ${{\alpha }_i}^{\prime }s$. There is also uncertainty as to which firm is the benchmark since we are simulating from the finite sample distribution of the ${{\alpha }_i}^{\prime }s$ and thus, we re-compute $\underset{j=1,\dots ,n}{\max }{\alpha }_j^{(s)}$ and the value of $u_i^{(s)}$ each time.

The method can be extended to the case in which the time effects are nonlinear, e.g., where ${\alpha }_{it}={\displaystyle {\sum }_{l=1}^L{\omega }_{il}{t}^l}$. With this specification, we can allow for a firm-specific polynomial trend, where ${\omega }_{il}$ represents the firm-specific coefficients. Of course other covariates can also be included in the time effects if so desired.

The model can be written for the ith unit as:

$$\underset{\left(T\times 1\right)}{Y_i}={\alpha }_i{\iota }_T+\underset{\left(T\times k\right)}{X_i}\underset{\left(k\times 1\right)}{{\beta }_{}}+\underset{\left(T\times G\right)}{{\mathrm{\Phi }}_{}}\underset{\left(G\times 1\right)}{{\gamma }_i}+\underset{\left(T\times 1\right)}{v_i},i=1,2,\dots ,n$$

(19)

or for the $t$th time period as:

$$\underset{\left(n\times 1\right)}{Y_t}=\underset{\left(n\times 1\right)}{\alpha }+\underset{\left(n\times k\right)}{X_t}\underset{\left(k\times 1\right)}{{\beta }_{}}+\underset{\left(n\times G\right)}{{\mathrm{\Gamma }}_{}}\underset{\left(G\times 1\right)}{{\varphi }_t}+\underset{\left(n\times 1\right)}{v_t},t=1,2,\dots T$$

(20)

where $\mathrm{\Phi }=\left[\begin{array}{c}{\varphi }_1^{\prime }\\ ⋮\\ {\varphi }_T^{\prime }\end{array}\right]$, and $\mathrm{\Gamma }=\left[\begin{array}{c}{\gamma }_1^{\prime }\\ ⋮\\ {\gamma }_n^{\prime }\end{array}\right]$. If we set ${\varphi }_{1t}$ then ${\gamma }_{it}$ acts as an individual-specific intercept. Effectively, the first column of $\mathrm{\Phi }$ contains ones. The model for all observations can be written as $Y=X\beta +\left(I_n\otimes \mathrm{\Phi }\right)\gamma +v=X\beta +\left(I_T\otimes \mathrm{\Gamma }\right)\varphi +v$, where $\gamma =vec\left(\mathrm{\Gamma }\right)$ and $\varphi =vec\left(\mathrm{\Phi }\right)$.

This model setting follows that in Kneip et al. (2012), and it satisfies the following structural assumption, which is Assumption 1 from Kneip et al:

Assumption 1:

For some fixed $L\in \{0,1,2,\dots \}<T$, there exists an L-dimensional space $L_T$, where, $\left\{{\varphi }_i(1),{\varphi }_i(2),\dots ,{\varphi }_i(T)\right\}\in L_T$ such that the time-varying individual effect ${\varphi }_i(t)={\varphi }_t{\gamma }_i$ holds with probability 1.

We define the priors similarly to Model 1. Regarding the slope parameter $\beta$ and variance of the noise term $\sigma$, we continue to assume a non-informative prior: $p\left(\beta ,\sigma \right)\propto {\sigma }^{-1}$. For the common factors, it is reasonable to assume that:

$$p\left({\varphi }_1,{\varphi }_2,\dots ,{\varphi }_T\right)\propto \exp \left(-\frac{{\displaystyle {\sum }_{t=1}^T{({\varphi }_t-{\varphi }_{t-1})}^{\prime }({\varphi }_t-{\varphi }_{t-1})}}{2{\omega }^2}\right)=\exp \left(-\frac{1}{2{\omega }^2}tr{\mathrm{\Phi }}^{\prime }Q\mathrm{\Phi }\right).$$

(21)

This prior is consistent with the presence of common factors that evolve smoothly over time. The degree of smoothness is controlled by the parameter $\omega$ and by setting ${\varphi }_0=0.$ Smoothness in this context then comes from the specification of the random walk prior above as essentially a spline.

For the loadings, we assume ${\gamma }_i\stackrel{IID}{~}{N}_G\left(\overline{\gamma },\mathrm{\Sigma }\right)$. An alternative that we do not pursue but which may attenuate the proliferation of factors would be to stochastically constrain the loadings to approach zero in the following sense: if $\underset{\left(n\times G\right)}{\mathrm{\Gamma }}=\left[{\gamma }_{\left(1\right)},\dots ,{\gamma }_{\left(G\right)}\right]$, then ${\gamma }_{(1)}\sim N_n(\overline{\gamma },{\psi }^2{I}_n)$, ${\gamma }_{(g)}\sim N_n({\alpha }^g\overline{\gamma },{\lambda }^g{\psi }^2{I}_n)$, for $g=1,\dots ,G$, where $\alpha ,\lambda$ are parameters between zero and one. The posterior kernel distribution is:

$$p\left(\beta ,\sigma ,\varphi ,\gamma |Y,X\right)\propto {\sigma }^{-\left(nT+1\right)}\exp \left[-\frac{{\displaystyle {\sum }_{i=1}^n{\displaystyle {\sum }_{t=1}^T{\left(y_{it}-x_{it}^{\prime }\beta -{\varphi }_t^{\prime }{\gamma }_i\right)}^2}}}{2{\sigma }^2}-\frac{{\displaystyle {\sum }_{t=2}^T{({\varphi }_t-{\varphi }_{t-1})}^{\prime }({\varphi }_t-{\varphi }_{t-1})}}{2{\omega }^2}\right]{\displaystyle {\prod }_{i=1}^{n}p\left({\gamma }_i|\zeta \right)}$$

(22)

where $\zeta$ denotes any hyper-parameters that are present in the prior of ${{\gamma }_{(i)}}^{\prime }s$. When ${\gamma }_{(i)}\stackrel{IID}{~}{N}_G\left(\overline{\gamma },\mathrm{\Sigma }\right)$, we have:

$$\begin{array}{c}p\left(\beta ,\sigma ,\varphi ,\gamma ,\overline{\gamma },\mathrm{\Sigma }|Y,X\right)\propto \\ {\sigma }^{-\left(nT+1\right)}\exp \left[-\frac{{\displaystyle {\sum }_{i=1}^n{\displaystyle {\sum }_{t=1}^T{\left(y_{it}-x_{it}^{\prime }\beta -{\varphi }_t^{\prime }{\gamma }_i\right)}^2}}}{2{\sigma }^2}-\frac{{\displaystyle {\sum }_{t=1}^T{({\varphi }_t-{\varphi }_{t-1})}^{\prime }({\varphi }_t-{\varphi }_{t-1})}}{2{\omega }^2}\right]\\ \times {|\mathrm{\Sigma }|}^{-n/2}\exp \left[-\frac{1}{2}{\displaystyle {\sum }_{i=1}^n{\left({\gamma }_i-\overline{\gamma }\right)}^{\prime }{\mathrm{\Sigma }}^{-1}\left({\gamma }_i-\overline{\gamma }\right)}\right]p\left(\overline{\gamma },\mathrm{\Sigma }\right)\end{array}$$

(23)

where $p\left(\overline{\gamma },\mathrm{\Sigma }\right)$ denotes the prior on the hyper-parameters. A reasonable choice is the $p\left(\overline{\gamma }|\mathrm{\Sigma }\right)\propto$ constant and $p\left(\mathrm{\Sigma }\right)\propto {|\mathrm{\Sigma }|}^{-\left(\overline{\nu }+1\right)/2}\exp \left(-\frac{1}{2}\overline{A}{\mathrm{\Sigma }}^{-1}\right)$, which leads to:

$$\begin{array}{c}p\left(\beta ,\sigma ,\varphi ,\gamma ,\overline{\gamma },\mathrm{\Sigma }|Y,X\right)\propto \\ {\sigma }^{-\left(nT+1\right)}{|\mathrm{\Sigma }|}^{-\left(n+\overline{\nu }+1\right)/2}\exp \left[-\frac{{\displaystyle {\sum }_{i=1}^n{\displaystyle {\sum }_{t=1}^T{\left(y_{it}-x_{it}^{\prime }\beta -{\varphi }_t^{\prime }{\gamma }_i\right)}^2}}}{2{\sigma }^2}-\frac{{\displaystyle {\sum }_{t=1}^T{({\varphi }_t-{\varphi }_{t-1})}^{\prime }({\varphi }_t-{\varphi }_{t-1})}}{2{\omega }^2}-\frac{1}{2}tr\left(A{\mathrm{\Sigma }}^{-1}\right)\right]\end{array}$$

(24)

where $A=\overline{A}+{\displaystyle {\sum }_{i=1}^n\left({\gamma }_i-\overline{\gamma }\right)}{\left({\gamma }_i-\overline{\gamma }\right)}^{\prime }$.

In order to proceed with Bayesian inference, we again use the Gibbs Sampling algorithm. For our model 2 specification, the implementation of Gibbs sampling is rather straightforward since we can analytically derive the conditional posteriors for the parameters in which we are interested. In what follows, we use the notation $Y:=Y-\alpha \otimes {\iota }_T$. The conditional posteriors are:

$$\beta |\sigma ,\varphi ,\gamma ,\overline{\gamma },\mathrm{\Sigma },Y,X~N_k\left(\overline{\beta },{\sigma }^2{\left(X^{\prime }X\right)}^{-1}\right),\mathrm{where}\overline{\beta }={\left(X^{\prime }X\right)}^{-1}{X}^{\prime }\left(Y-\left(I_n\otimes \mathrm{\Phi }\right)\gamma \right)$$

(25)

$$\frac{{\left(Y-X\beta -\left(I_n\otimes \mathrm{\Phi }\right)\gamma \right)}^{\prime }\left(Y-X\beta -\left(I_n\otimes \mathrm{\Phi }\right)\gamma \right)}{{\sigma }^2}|\beta ,\gamma ,\varphi ,\overline{\gamma },\mathrm{\Sigma }~{\chi }_{nT}^2$$

(26)

$$\overline{\gamma }|\beta ,\sigma ,\varphi ,\gamma ,\overline{\gamma },\mathrm{\Sigma },Y,X~\overline{\gamma }|\gamma ,\mathrm{\Sigma },Y,X~N_G\left(n^{-1}{\displaystyle {\sum }_{i=1}^n{\gamma }_i,n^{-1}\mathrm{\Sigma }}\right)$$

(27)

$${\gamma }_i|\beta ,\sigma ,\overline{\gamma },\mathrm{\Sigma },Y,X~N_G\left({\widehat{\gamma }}_i,{\sigma }^2{\left({\mathrm{\Phi }}^{\prime }\mathrm{\Phi }+{\sigma }^2{\mathrm{\Sigma }}^{-1}\right)}^{-1}\right)$$

(28)

where ${\widehat{\gamma }}_i={\left({\mathrm{\Phi }}^{\prime }\mathrm{\Phi }+{\sigma }^2{\mathrm{\Sigma }}^{-1}\right)}^{-1}\left({\mathrm{\Phi }}^{\prime }{e}_i+{\sigma }^2{\mathrm{\Sigma }}^{-1}\overline{\gamma }\right)$, $e_i=y_i-X_i\beta$, for each $i=1,\dots ,n,$

$${\varphi }_t|\beta ,\sigma ,\gamma ,\overline{\gamma },\mathrm{\Sigma },Y,X,\{{\varphi }_{\tau },\tau \ne t\}~N_G\left({\widehat{\varphi }}_t,{\sigma }^2{\omega }^2{\left({\omega }^2{\mathrm{\Gamma }}^{\prime }\mathrm{\Gamma }+2{\sigma }^2{I}_G\right)}^{-1}\right)$$

(29)

where ${\widehat{\varphi }}_t={\left({\omega }^2{\mathrm{\Gamma }}^{\prime }\mathrm{\Gamma }+2{\sigma }^2{I}_G\right)}^{-1}\left({\omega }^2{\mathrm{\Gamma }}^{\prime }{e}_t+{\sigma }^2\left({\varphi }_{t-1}+{\varphi }_{t+1}\right)\right),$ $e_t=y_t-X_t\beta$ for each $t=1,\dots ,T$.

Using a Gibbs sampler, we draw observations from the conditional posteriors from (25) to (29). Draws from the conditional posteriors will eventually converge to the joint posterior (24). The conditional posterior distribution of β follows the multivariate normal (25) and it is straightforward to sample from that distribution. To draw samples from the conditional posterior distribution function for the unobserved variance of the measurement error σ term, we first draw samples directly from the distribution of ${\left(Y-X\beta -\gamma \right)}^{\prime }\left(Y-X\beta -\gamma \right)/{\sigma }^2$, which is shown in (26) to follow a chi-squared distribution with the degree of freedom nT, and then assign the values of ${\left(Y-X\beta -\gamma \right)}^{\prime }\left(Y-X\beta -\gamma \right)/(Chi-rnd)$ to ${\sigma }^2$, where $(Chi-rnd)$ is the generated random variable that follows ${\chi }_{nT}^2$ in the first step.

For the mean parameter $\overline{\gamma }$, sampling is also straightforward since its conditional posterior follows a multivariate normal distribution. The variance matrix $\mathrm{\Sigma }$ follows an inverted Wishart distribution. For the unknown common factors ${\gamma }_i$ and the corresponding factor loadings ${\varphi }_t$ we can draw directly from multivariate normal distribution following (28) and (29). Finally, the individual firm effects ${\alpha }_i$ can be drawn using the procedure in Koop et al. (1997). This involves standard computations as the Bayesian fixed effects are drawn for normal posterior conditional distributions. The difficult distributional issues involved in deriving the analytical finite sample distribution of the parameters and estimates of relative efficiency are resolved through the MCMC procedure used to generate $u_i^{(s)}={\alpha }_i^{(s)}-\underset{j=1,\dots ,n}{\max }{\alpha }_j^{(s)}$, a fact that has been mentioned by Koop et al. (1997). ${\alpha }_i^{(s)}$ can be calculated from the posterior of ${\beta }_i^{\left(s\right)}$ and ${\gamma }_i^{\left(s\right)}$ and thus $u_i^{(s)}$ can also be calculated.

In our discussion of Model 2, we have treated the number of finite factors (G) as known. However, we can also utilize Bayesian techniques to develop inferences on G. Classical inferential approaches have been proposed by Bai and Ng (2007), Onatski (2009), and Kneip et al. (2012). We consider models with at most L finite factors G = 1, 2,…,L. Suppose $p(\theta ,{\mathrm{\Gamma }}_G)$ and $L(\theta ,{\mathrm{\Gamma }}_G;Y,G)$ denote the prior and likelihood, respectively, of a model with G factors, where $\theta$ is the vector of parameters common to all models (such as β and σ) and ${\mathrm{\Gamma }}_G$ denotes a vector of parameters related to the factors and their loadings, φ and γ. The marginal likelihood is $M_G(Y)={\displaystyle \int }L\left(\theta ,{\mathrm{\Gamma }}_G;Y,G\right)p\left(\theta ,{\mathrm{\Gamma }}_G\right)d{\mathrm{\Gamma }}_{G}d\theta$. For models with different numbers of factors, say G and G’, we can consider the Bayes factor in favor of the first model and against the second:

$$BF=\frac{{\displaystyle \int }L\left(\theta ,{\mathrm{\Gamma }}_G;Y,G\right)p\left(\theta ,{\mathrm{\Gamma }}_G\right)d{\mathrm{\Gamma }}_{G}d\theta }{{\displaystyle \int }L\left(\theta ,{\mathrm{\Gamma }}_{G^{\prime }};Y,G^{\prime }\right)p\left(\theta ,{\mathrm{\Gamma }}_{G^{\prime }}\right)d{\mathrm{\Gamma }}_{G^{\prime }}d\theta }=\frac{M_G(Y)}{M_{G^{\prime }}(Y)}$$

(30)

Computation of the marginal likelihood requires the computation of the integral in the numerator $\mathcal{P}(\theta |Y,G)$ with respect to $\varphi$ and $\gamma$. As this is not available analytically, we adopt the following approach.

We first specify:

$$\mathcal{P}\left(\theta |Y,G\right)={\displaystyle \int }L\left(\theta ,{\mathrm{\Gamma }}_G;Y,G\right)p\left(\theta ,{\mathrm{\Gamma }}_G\right)d{\mathrm{\Gamma }}_G={\displaystyle \int }\frac{L\left(\theta ,{\mathrm{\Gamma }}_G;Y,G\right)p\left(\theta ,{\mathrm{\Gamma }}_G\right)}{q\left({\mathrm{\Gamma }}_G\right)}q\left({\mathrm{\Gamma }}_G\right)d{\mathrm{\Gamma }}_G,$$

(31)

where $q({\mathrm{\Gamma }}_G)$ is a convenient importance sampling density. We factor the importance density as $q\left({\mathrm{\Gamma }}_G\right)={\displaystyle {\prod }_{t=1}^T{q}_t^{\varphi }\left({\varphi }_t\right)}{\displaystyle {\prod }_{i=1}^n{q}_i^{\gamma }\left({\gamma }_i\right)}$, where $q_t^{\varphi }$ and $q_i^{\gamma }$ are univariate densities. The densities are chosen to be univariate Student’s t-distributions with five degrees of freedom, with parameters matched to the posterior mean and standard deviation of MCMC draws for $\varphi$ and γ, respectively. The integral is then calculated using standard importance sampling, which is quite robust. The standard deviations are multiplied by constants $h_{\varphi }$ and $h_{\gamma }$, which are selected so that the importance weights are as close to uniform as possible. We use 100 random pairs in the interval 0.1 to 10 and select the values of h for which the Kolmogorov-Smirnov test is the lowest. We truncate the weights to their 99.5% confidence interval, but in very few instances was this found necessary as extreme values are rarely observed. There is evidence that changing the degrees of freedom of the Student’s t provides some improvement, but we did not pursue this further as the final results for the Bayes factors were not found to differ significantly.

Given marginal likelihoods $M_g(Y)$, $g=1,\dots ,G$, the posterior model probabilities can be estimated as1:

$$p_g(Y)=\frac{M_g\left(Y\right)}{{\displaystyle {\sum }_{g=1}^G{M}_g\left(Y\right)}},g=1,\dots ,G$$

(32)

The posterior model probabilities summarize the evidence in favor of a model with a given number of factors.

4. Monte Carlo Simulations

In order to illustrate the model and examine the finite sample performance of the new Bayesian estimators with nonparametric individual effects (BE1) and with the factor model specification for the individual effects (BE2), we carry out a series of Monte Carlo experiments. The performance of the Bayesian estimator is compared with the parametric time-variant estimator of Battese and Coelli (1992) (BC), the estimators proposed by (Cornwell et al. 1990)—within (CSSW) and random effects GLS (CSSG)—and the Kneip et al. (2012) estimator that utilizes a combination of nonparametric regression techniques (smoothing splines) and factor analysis (Bada and Liebl 2014) to model the time-varying unit specific effects. The BC estimates are based on the model (1) where the time-varying effects are given by ${\gamma }_{it}=-e^{-\eta (t-T)}{u}_i.$ The temporal pattern of firm-specific effects ${\gamma }_{it}$ depends on the sign of $\eta$. The time-invariant case corresponds to $\eta =0$. The disturbances $u_i$ are i.i.d. and are assumed to follow a non-negative truncated normal distribution. Estimation of the BC model is carried out by parametric MLE. The CSSW and CSSG estimates are also based on model (1) and specify the time varying effects as ${\gamma }_{it}={\theta }_{i1}+{\theta }_{i2}t+{\theta }_{i3}{t}^2$. Derivations of the within, GLS, and efficient Hausman-Taylor type IV estimators can be found in Cornwell et al. (1990). The KSS estimator requires a bit more discussion. They assume that ${\gamma }_{it}$ is a linear combination of some basis functions ${\gamma }_{it}={\displaystyle \sum _{r=1}^L{\zeta }_{ir}{g}_r(t).}$ In the first step of their three step procedure, they obtain estimates of the slope parameters and nonparametric approximations to ${\gamma }_{it}$ by a least squares regression of Y on X and an approximation of the effects using smoothing splines. In the second step, they obtain the empirical covariance matrix of residuals and in the third step they determine the basis functions and corresponding coefficients. Details can be found in Kneip et al. (2012). Point estimates and standard errors for BE1 and BE2 are posterior moments whose derivation we detailed in Section 2 and Section 3. We averaged the point estimates and the standard deviations of the parameter estimates from all of the simulated paths.

We consider a panel data model with two regressors written as $y_{it}={\beta }_1{x}_{it}^{(1)}+{\beta }_2{x}_{it}^{(2)}+{\gamma }_{it}+v_{it}.$ We generate samples of size n = 50, 100, 200, with T = 20, 50. In each experiment, the regressors $x_{it}^{(j)}(j=1,2)$ are randomly drawn from a standard multivariate normal distribution N(0,I_p) The i.i.d. disturbance term $v_{it}$ is drawn from a standardized $N(0,1)$. Time-varying individual effects are generated by four different DGPs, which specify the effects as following a unit specific quadratic function of a time trend (DGP1), random walk (DGP2) oscillating function given by a linear combination of sine and cosine functions (DGP3), and finally a simple additive mixture of the previous three data generating processes (DGP4). The parameterizations are:

$$\mathrm{DGP}1: {\gamma }_{it}={\theta }_{i0}+{\theta }_{i1}(t/T)+{\theta }_{i2}{(t/T)}^2$$

$$\mathrm{DGP}2: {\gamma }_{it}={\varphi }_i{r}_t$$

$$\mathrm{DGP}3: {\gamma }_{it}={\nu }_{i1}t/t\cos (4\pi t/T)+{\nu }_{i2}t/T\sin (4\pi t/T)$$

$$\mathrm{DGP}4: {\gamma }_{it}={\theta }_{i0}+{\theta }_{i1}(t/T)+{\theta }_{i2}{(t/T)}^2+{\nu }_{i1}t/t\cos (4\pi t/T)+{\nu }_{i2}t/T\sin (4\pi t/T).$$

Here ${\theta }_{ij}(j=0,1,2)$ and ${\varphi }_i$ are i.i.d $N(0,1)$, $r_{t+1}=r_t+{\delta }_t,{\delta }_t\sim i.i.d.N(0,1)$, and ${\nu }_{ij}(j=1,2)\sim i.i.d.N(0,1).$

Gibbs sampling was implemented using 55,000 iterations with a burn-in period of 5000 samples. We only consider every other 10th draw to mitigate the impact of autocorrelation from successive samples from the Markov chain. With regard to the selection of the number of factors, Gibbs samplers for all DGPs rely on an MCMC simulation from models with a G value ranging from one to eight. The true number of factors is 3, 2, 1, and 6 for the four respective DGPs.

The simulation results for all the DGPs are displayed in

Table 1. Monte Carlo simulations for DGP1.

Mean Squared Error for the Individual Effects
n	T	BC		CSSW		CSSG		KSS			BE1			BE2
50	20	0.7284		0.0012		0.0012		0.0039			0.0053			0.0671
	50	0.9371		0.0005		0.0005		0.1255			0.0021			0.0323
100	20	0.8222		0.0008		0.0008		0.0033			0.0031			0.0183
	50	0.8245		0.0003		0.0003		0.0220			0.0018			0.0115
200	20	0.8451		0.0008		0.0008		0.0023			0.0027			0.0101
	50	0.8823		0.0003		0.0003		0.0021			0.0011			0.0083
Estimate and Standard Error for the Slope Coefficients
	T = 20						T = 50
	BC	CSSW	CSSG	KSS	BE1	BE2	BC		CSSW	CSSG		KSS	BE1		BE2
n = 50
EST1	0.5250	0.4961	0.4965	0.4954	0.4981	0.5017	0.5105		0.4991	0.4992		0.4999	0.5013		0.4998
SE1	0.0130	0.0033	0.0032	0.0029	0.0057	0.0042	0.0073		0.0021	0.0020		0.0019	0.0035		0.0027
EST2	0.4856	0.4949	0.4948	0.4919	0.4985	0.5020	0.4969		0.5048	0.5047		0.5053	0.5001		0.5002
SE2	0.0139	0.0035	0.0033	0.0031	0.0055	0.0041	0.0073		0.0020	0.0020		0.0018	0.0032		0.0024
n = 100
EST1	0.4973	0.5018	0.5013	0.5045	0.5023	0.5002	0.4843		0.4999	0.4998		0.4991	0.4999		0.5001
STD1	0.0099	0.0023	0.0022	0.0021	0.0032	0.0027	0.0066		0.0014	0.0014		0.0014	0.0023		0.0018
EST2	0.5047	0.5009	0.5012	0.5022	0.5016	0.5001	0.4995		0.5001	0.5000		0.4990	0.5003		0.5001
STD2	0.0098	0.0022	0.0022	0.0020	0.0032	0.0028	0.0066		0.0014	0.0014		0.0013	0.0022		0.0017
n = 200
EST1	0.4936	0.5013	0.5015	0.5009	0.5000	0.4981	0.5000		0.5007	0.5007		0.5000	0.5012		0.5001
STD1	0.0071	0.0016	0.0016	0.0016	0.0027	0.0022	0.0042		0.0010	0.0010		0.0010	0.0019		0.0015
EST2	0.4983	0.5016	0.5020	0.5019	0.5002	0.4993	0.5027		0.4972	0.4972		0.4969	0.5003		0.5004
STD2	0.0071	0.0016	0.0016	0.0016	0.0027	0.0022	0.0042		0.0010	0.0010		0.0010	0.0018		0.0014

,

Table 2. Monte Carlo simulations for DGP2.

Mean Squared Error for the Individual Effects
n	T	BC		CSSW		CSSG		KSS			BE1			BE2
50	20	0.9202		0.1266		0.1266		0.0182			0.0071			0.0048
	50	0.9052		0.2996		0.2996		0.0238			0.0053			0.0025
100	20	0.8588		0.4553		0.4553		0.0531			0.0040			0.0037
	50	0.9884		0.1065		0.1065		0.0046			0.0028			0.0013
200	20	0.9183		0.6376		0.6375		0.0706			0.0022			0.0027
	50	0.9526		0.0616		0.0616		0.0028			0.0009			0.0008
Estimate and Standard Error for the Slope Coefficients
	T = 20						T = 50
	BC	CSSW	CSSG	KSS	BE1	BE2	BC		CSSW	CSSG		KSS	BE1		BE2
n = 50
EST1	0.4786	0.4857	0.4904	0.5059	0.5010	0.4993	0.4820		0.4811	0.4938		0.4972	0.5052		0.4983
SE1	0.0460	0.0308	0.0298	0.0136	0.0262	0.0037	0.0243		0.0230	0.0227		0.0059	0.0177		0.0029
EST2	0.4664	0.4414	0.4854	0.4599	0.5031	0.4992	0.4840		0.4660	0.4848		0.4988	0.5001		0.4999
SE2	0.0491	0.0326	0.0314	0.0146	0.0261	0.0035	0.0241		0.0226	0.0225		0.0059	0.0174		0.0028
n = 100
EST1	0.4854	0.4818	0.4898	0.5065	0.4997	0.5002	0.5137		0.5360	0.5089		0.4950	0.5101		0.4987
STD1	0.0200	0.0195	0.0188	0.0075	0.0163	0.0028	0.0415		0.0257	0.0254		0.0055	0.0128		0.0018
EST2	0.5005	0.4996	0.5115	0.5101	0.4993	0.5001	0.4482		0.5283	0.5143		0.5127	0.5002		0.4992
STD2	0.0198	0.0189	0.0186	0.0073	0.0164	0.0029	0.0415		0.0256	0.0254		0.0055	0.0130		0.0018
n = 200
EST1	0.5051	0.4995	0.5015	0.4864	0.4927	0.5013	0.4274		0.5097	0.4968		0.5018	0.5032		0.5011
STD1	0.0169	0.0175	0.0171	0.0067	0.0120	0.0021	0.0527		0.0202	0.0200		0.0032	0.0078		0.0013
EST2	0.4895	0.4898	0.5147	0.4951	0.4901	0.5020	0.3996		0.4930	0.5015		0.5042	0.5021		0.5031
STD2	0.0170	0.0175	0.0171	0.0067	0.0121	0.0020	0.0531		0.0204	0.0202		0.0033	0.0077		0.0014

,

Table 3. Monte Carlo simulations for DGP3.

Mean Squared Error for the Individual Effects
n	T	BC		CSSW		CSSG		KSS			BE1			BE2
50	20	3.3477		0.8816		0.8816		0.0130			0.0244			0.0356
	50	3.3639		0.8469		0.8468		0.0082			0.0134			0.0152
100	20	3.5102		0.8309		0.8303		0.0123			0.0116			0.0282
	50	3.7625		0.8357		0.8356		0.0072			0.0028			0.0053
200	20	3.8433		0.8335		0.8333		0.0121			0.0083			0.0116
	50	3.8513		0.8393		0.8392		0.0063			0.0014			0.0019
Estimate and Standard Error for the Slope Coefficients
	T = 20						T = 50
	BC	CSSW	CSSG	KSS	BE1	BE2	BC		CSSW	CSSG		KSS	BE1		BE2
n = 50
EST1	0.5277	0.5250	0.4994	0.4989	0.5012	0.5002	0.4868		0.4871	0.4976		0.5005	0.4991		0.5001
SE1	0.0188	0.0203	0.0197	0.0029	0.0081	0.0038	0.0122		0.0122	0.0120		0.0018	0.0041		0.0025
EST2	0.4905	0.4998	0.5062	0.4930	0.4981	0.4997	0.5259		0.5255	0.5207		0.5052	0.4994		0.5003
SE2	0.0198	0.0215	0.0207	0.0031	0.0078	0.0035	0.0121		0.0120	0.0119		0.0018	0.0042		0.0023
n = 100
EST1	0.4816	0.4768	0.4998	0.5030	0.4961	0.4992	0.4877		0.4863	0.4972		0.4986	0.4995		0.5002
STD1	0.0132	0.0139	0.0134	0.0021	0.0058	0.0025	0.0076		0.0077	0.0076		0.0013	0.0022		0.0017
EST2	0.4907	0.4816	0.5088	0.5028	0.4971	0.4985	0.5024		0.5118	0.5089		0.4993	0.4990		0.5004
STD2	0.0131	0.0135	0.0133	0.0021	0.0057	0.0024	0.0076		0.0077	0.0076		0.0013	0.0023		0.0018
n = 200
EST1	0.5120	0.5103	0.5110	0.5016	0.5012	0.5011	0.4976		0.5012	0.4962		0.4999	0.4981		0.5052
STD1	0.0088	0.0091	0.0089	0.0016	0.0042	0.0013	0.0055		0.0054	0.0054		0.0010	0.0015		0.0011
EST2	0.4885	0.4892	0.5019	0.5029	0.5015	0.5014	0.4874		0.4883	0.4957		0.4973	0.4992		0.4994
STD2	0.0088	0.0091	0.0089	0.0016	0.0041	0.0012	0.0055		0.0055	0.0054		0.0010	0.0016		0.0012

and

Table 4. Monte Carlo simulations for DGP4.

Mean Squared Error for the Individual Effects
n	T	BC		CSSW		CSSG		KSS			BE1			BE2
50	20	0.8042		0.2161		0.2161		0.0030			0.0130			0.0445
	50	0.9478		0.2056		0.2056		0.0890			0.0045			0.0141
100	20	0.8770		0.1382		0.1382		0.0026			0.0112			0.0291
	50	0.8626		0.1337		0.1337		0.0193			0.0028			0.0055
200	20	0.8764		0.1301		0.1301		0.0020			0.0098			0.0108
	50	0.9111		0.1445		0.1445		0.0015			0.0015			0.0021
Estimate and Standard Error for the Slope Coefficients
	T = 20						T = 50
	BC	CSSW	CSSG	KSS	BE1	BE2	BC		CSSW	CSSG		KSS	BE1		BE2
n =50
EST1	0.5521	0.5250	0.5329	0.4995	0.4922	0.4951	0.5031		0.4871	0.4901		0.4999	0.5051		0.5010
SE1	0.0233	0.0203	0.0197	0.0030	0.0039	0.0031	0.0148		0.0122	0.0120		0.0019	0.0032		0.0025
EST2	0.4788	0.4998	0.5014	0.4907	0.5011	0.4977	0.5201		0.5255	0.5246		0.5053	0.5001		0.5003
SE2	0.0248	0.0215	0.0207	0.0031	0.0036	0.0032	0.0147		0.0120	0.0119		0.0018	0.0031		0.0028
n = 100
EST1	0.4732	0.4768	0.4713	0.5017	0.5001	0.5052	0.4720		0.4863	0.4867		0.4985	0.5003		0.5001
STD1	0.0169	0.0139	0.0134	0.0022	0.0031	0.0027	0.0103		0.0077	0.0076		0.0014	0.0021		0.0017
EST2	0.4880	0.4816	0.4836	0.5018	0.5000	0.5041	0.5077		0.5118	0.5117		0.4998	0.5002		0.5003
STD2	0.0167	0.0135	0.0133	0.0021	0.0032	0.0025	0.0103		0.0077	0.0076		0.0014	0.0020		0.0015
n = 200
EST1	0.5029	0.5103	0.5112	0.5011	0.5032	0.5001	0.4891		0.5012	0.5012		0.5003	0.4991		0.5002
STD1	0.0116	0.0091	0.0089	0.0016	0.0028	0.0020	0.0069		0.0054	0.0054		0.0010	0.0018		0.0014
EST2	0.4892	0.4892	0.4934	0.5021	0.5013	0.5020	0.4940		0.4883	0.4886		0.4970	0.4987		0.5013
STD2	0.0116	0.0091	0.0089	0.0016	0.0025	0.0022	0.0070		0.0055	0.0054		0.0010	0.0017		0.0013

. Estimates and standard errors of the slope coefficients β₁ and β₂ are presented in the upper panel of each table, while estimates of the individual effects γ_it and their normalized MSE are displayed in the lower panel of each table. The normalized MSE of the individual effects γ_it is calculated as:

$$R({\widehat{\gamma }}_{it},{\gamma }_{it})=\frac{{\displaystyle {\sum }_{i=1}^n{\displaystyle {\sum }_{t=1}^T{({\widehat{\gamma }}_{it}-{\gamma }_{it})}^2}}}{{\displaystyle {\sum }_{i=1}^n{\displaystyle {\sum }_{t=1}^T{\gamma }_{it}{}^2}}}$$

(33)

Since we have not analyzed the role of correlated effects in these experiments, estimates of the slope parameters should be consistent for CSSW, CSSG, and KSS. Moreover, the BC model utilizes parametric MLE based on i.i.d. normally distributed random disturbances and thus should also yield consistent slope parameter estimates. Results from the four different specifications of the effects clearly demonstrate that point estimates of the slope coefficients for BC, CSSW, CSSG, and KSS are comparable across the various dgps, although variances will of course be smaller for estimators that do a better job of modeling the effects. The BC estimator does a poor job of estimating the effects since the specification we utilize assumes that the effects have the same temporal pattern for the different units. Generalizations of the BC estimator are available that allow the effects to be functions of selected regressors that may change over units, but we do not utilize these extensions in our experiments. Since DGP1 is consistent with the assumptions for the time-varying effects in the CSS model (we use the version of the CSS estimator utilized in the Cornwell et al. (1990) application wherein the unit specific effects were given by a second-order polynomial in the time trend), it is no surprise that the CSSW and CSSG estimators have the best performance compared with the other estimators for this dgp. However, it is also clear from the results of Table 1 that the Bayesian estimators are comparable to those of the CSSW, CSSG, and KSS estimators in terms of the estimates of individual effects. Moreover, for the sample sizes of n = 50, T = 50, and n = 100, T = 50, the Bayesian estimators provide more accurate estimates of individual effects than the KSS estimator. This implies that the performance of the Bayesian estimators is quite effective in estimating the time-varying effects of the smoothed-curve forms, like the second-order polynomials. It is not surprising that the mean squared errors of the Bayesian estimators are consistently much lower than those of the BC estimator for all sample sizes.

DGP2 considers the case where the individual effects are generated by a random walk and the results for these experiments are shown in Table 2. CSSW and CSSG are over-parameterized as they assume that the individual effects are quadratic functions of the time trend and have relatively poorer performances for this dgp. BE1 and BE2 are mostly data driven and impose no functional forms on the temporal pattern of the individual effects. For this relatively simple random walk specification, they outperform the other estimators that rely on functional form assumptions and also have a better estimation performance in terms of the MSE of individual effects than KSS. DGP3 characterizes significant time variations in the individual effects. As we can see from Table 3, the BE1 and BE 2 estimators have a comparable performance to the KSS estimator and outperform it again for experiments with relatively large panels such as (n = 100 and 200). The other estimators, whose effects rely on parametric assumptions of simple functional forms, are largely dominated by the Bayesian estimators.

DGP4 is a mixture of the scenarios for the time varying effects used in DGP1–DGP4. Table 4 indicates that that BE1 and BE2 outperform the BC, CSSW, and CSSG estimators in terms of the MSE of the individual effects and are comparable to KSS.

As we have pointed out, for all the DGPs, the slope parameter estimates are comparable across the six different estimators. However, this is not the case for the individual effects. This is a drawback for the estimation of technical or efficiency change since such measures are usually based on an unobserved latent variable that is estimated using some function of the model residuals. For example, the individual effects correspond to the technical efficiencies in stochastic frontier analysis. Our new Bayesian estimators for the stochastic frontier would appear to be excellent candidates among competing estimators for modeling a production or cost frontier and it is to topic that we now turn to in our empirical model of banking efficiency.

5. Empirical Application: Efficiency Analysis of the U.S. Banking Industry

5.1. Empirical Models

In this section, the two Bayesian estimators we have introduced are used to estimate temporal changes in the efficiency levels of 40 of the top 50 banks in the U.S. ranked by their book value of assets. We consider only 40 of these banks due to missing observations and other data anomalies. The empirical model is borrowed from Inanoglu et al. (2015), who use a suite of econometric specifications, including time-invariant panel data models, time-variant models, and quantile regression methods, to examine issues of “too big to fail” in the banking industry. In our illustration of the new Bayesian estimators, we will only compare the results across the different time-varying stochastic frontier panel estimators we discussed in the last section, along with modifications in the Bayesian estimators, to deal with potential issues of endogeneity. The estimators we utilize are based on different assumptions on the functional form of the time varying effects and provide various treatments for the unobserved heterogeneity, but they are all based on (1), which characterizes a single output with panel data assuming unobserved individual effects.

We will estimate a second order approximation in logs (the translog specification) to a multi-output/multi-input distance function (Caves et al. 1982). The output-distance function $D_O(Y,X)$ is non-decreasing, homogeneous, and convex in multiple outputs $Y$ and non-increasing and quasi-convex in multiple inputs $X$. The translog output distance function takes the following form:

$$\begin{array}{l}-y_{1it}^{}={\eta }_{it}+{\displaystyle \sum _{j=2}^m{\gamma }_j{y}_{jit}^{*}}+\frac{1}{2}{\displaystyle \sum _{j=2}^m{\displaystyle \sum _{l=2}^m{\gamma }_{jl}{y}_{jit}^{*}}}{y}_{lit}^{*}+{\displaystyle \sum _{k=1}^n{\delta }_k}{x}_{kit}^{}+\frac{1}{2}{\displaystyle \sum _{k=1}^n{\displaystyle \sum _{p=1}^n{\delta }_{kp}{x}_{kit}^{}}}{x}_{pit}^{}\\ +{\displaystyle \sum _{j=2}^m{\displaystyle \sum _{k=1}^n{\theta }_{jk}{y}_{jit}^{*}}}{x}_{kit}^{}+v_{it},i=1,\dots ,N;t=1,\dots ,T.\end{array}$$

(34)

Here $y_{{}_{jit,j=2,\dots ,m}}^{*}=\ln (Y_{jit}/Y_{1it})$, $x_{kit}=\ln (X_{kit})$, and the normalization ${\displaystyle \sum _{j=1}^m{\gamma }_j=}1$ results from the homogeneity property of the output distance function in outputs. If we denote $Z=[x_{NT\times n}^{},y_{NT\times (m-1)}^{*},x{x}_{NT\times (n\times (n+1)/2)}^{},y^{*}{y}_{NT\times ((m-1)\times m/2)}^{*},x{y}_{NT\times (m-1)\times n)}^{*}]$ then the model can be written as a simple re-parameterized version of (1):

$$y_{1,it}=z_{it}^{\prime }{\beta }^{*}+{\gamma }^{*}{}_{it}+v^{*}{}_{it}$$

(35)

To allow for the endogeneity of the regressors in z we use the model:

$$z_{it}=\Pi z_{i,t-1}+{\epsilon }_{it}$$

(36)

where ${[v_{it},{\epsilon }_{it}]}^{\prime }~N(0,\mathrm{\Sigma })$. That is, we complete the model with a panel VAR reduced form for the potentially endogenous variables. The likelihood and the posterior distributions are straightforward to derive using the methods we discussed in Section 2 and Section 3. Moreover, since the output distance function is bounded from above by 1, the logarithmic transformation used in specifying the estimating equation in (34) provides a natural justification for the bounded support of the unobserved technical efficiency term specified in the stochastic frontier literature for the unit specific time varying effects ${\gamma }^{*}{}_{it}$.

The elasticities of the distance function with respect to the input and output variables are:

$$s_p={\delta }_p+{\displaystyle \sum _{k=1}^n{\delta }_{kp}}{x}_k^{}+{\displaystyle \sum _{j=2}^m{\theta }_{pj}{y}_j^{*}},p=1,2,\dots ,n$$

(37)

$$r_j={\gamma }_j+{\displaystyle \sum _{l=2}^m{\gamma }_{jl}{y}_j^{*}}+{\displaystyle \sum _{k=1}^n{\theta }_{kj}{x}_k^{}},j=2,\dots ,m.$$

(38)

The individual effects are transformed into relative efficiency levels using the standard order statistics device in Schmidt and Sickles (1984):

$$T{E}_{it}=\exp \{{\gamma }^{*}{}_{it}-{\max }_{i=1,\dots n}{\gamma }^{*}{}_{it}\}.$$

(39)

For the BC estimator, technical efficiency levels can differ, but parsimony is achieved by assuming that all firms have the same temporal pattern.

Clearly, the levels of efficiency can vary substantially for the methods that use the order statistics (the firm with the largest effect) to benchmark the most efficient firm and thus the relative efficiencies of the remaining firms. Typically, this impact is mitigated by data trimming, but with only 40 firms in our study, we decided to avoid doing so when presenting the results below. The BC estimator has no such potential drawback.

5.2. Data

The dataset analyzed is a balanced panel of 40 out of the top 50 U.S. commercial banks based on the yearly data of their Book Value of Assets from 1990 through 2009. The panel size is thus 40 by 20. Missing observations and data anomalies reduced the sample from 50 to 40 firms. The data is merged on a pro-forma basis wherein the non-surviving bank’s data is represented as part of the surviving bank going back in time. The three output and six input variables used to estimate the translog output distance function are: Real Estate Loans (“REL”), Commercial and Industrial Loans (“CIL”), Consumer Loans (“CL”), Premises & Fixed Assets (“PFA”) , Number of Employees (“NOE”), Purchased Funds (“PF”), Savings Accounts (“SA”), Certificates of Deposit (“CD”), and Demand Deposits (“DD”). Additionally, three types of risk proxies are considered: Credit Risk (“CR”), which is approximated by the Gross Charge-off Ratio; Liquidity Risk (“LR”), which is proxied by the Liquidity Ratio; and Market Risk (“MR”), which is proxied by the standard deviation of Trading Returns.

5.3. Empirical Results

The input and output elasticities evaluated at the geometric mean of the sample are displayed in

Table 5. Estimation results.

Model	BC	CSSW	CSSG	KSS	BE1	BE2	BE1*	BE2*
PFA	−0.1267	−0.1067	−0.1243	−0.0448	−0.1221	−0.0505	−0.0972	−0.0555
NOE	−0.1518	−0.2750	−0.2731	−0.2195	−0.1520	−0.0666	−0.1145	−0.0424
PF	−0.1088	−0.0571	−0.0628	−0.0679	−0.0570	−0.1387	−0.0930	−0.1003
SA	−0.3058	−0.1026	−0.1413	−0.1289	−0.1700	−0.3042	−0.1030	−0.2542
CD	−0.2938	−0.2422	−0.2492	−0.1526	−0.2363	−0.2867	−0.1541	−0.2012
DD	−0.0295	−0.0055	−0.0297	−0.0321	−0.0259	−0.0636	−0.0715	−0.0335
REL	0.6302	0.6267	0.6254	0.5468	0.6182	0.6099	0.4103	0.4242
CIL	0.2674	0.2116	0.2053	0.3200	0.2300	0.2630	0.2415	0.2208
CL	0.1024	0.1617	0.1693	0.1332	0.1518	0.1271	0.1013	0.1212
RTS	1.0165	0.7891	0.8804	0.6459	0.7634	0.9102	0.6333	0.6871
Avg.TE	0.7576	0.6094	0.6608	0.5552	0.4584	0.6937	0.7944	0.7889

.2 For the BE2 estimates, the BF for two factors versus one factor is 35.12, while the BF for three versus one factor is 2.23 and the BF for four versus one factor is 1.10. For the KSS estimates, we use the procedure outlined in Kneip et al. (2012, pp. 607–8) with α = 0.05. The KSS procedure estimates two factors. Thus, we have two factors in our empirical illustration for BE2, and KSS. BE1* and BE2* are the Bayesian estimators corrected for endogeneity of the terms that are interacted with the endogenous multiple outputs. From Table 5, we can see that magnitudes and signs of the elasticity estimates across different models are comparable, except for the Demand Deposit, where CSSW gives a significantly lower estimate than all of the other estimators. All of the estimators suggest decreasing returns to scale except BC. However, the returns-to-scale estimate suggested by BC is 1.0165, which is not significantly different from 1. Alternatively, we can say that there is no evidence of increasing returns to scales based on the estimation results. The largest US banks appear to have fully exploited their scale economies in generating intermediation services.

Variations in the temporal pattern of the individual effects are displayed in Figure 1. The BC estimator provides higher efficiency estimates, but also efficiencies that decline through the sample period, while all the other estimators find efficiencies of similar magnitudes that increase slightly and then decline, anticipating the meltdowns of financial institutions beginning around 2007 that led to the Great Recession.

As we can see from the last row in Table 5 and in Figure 1, the scale of the average technical efficiency levels ranges from around 0.63 to 0.73. Turning our attention to the estimated temporal pattern of the technical efficiencies using the Bayesian estimators, we notice that the BE1 and BE2 models display similar trends, but the efficiency levels suggested by BE2 are consistently higher than those by BE1. The same pattern exists for the estimators BE1* and BE2*. The efficiencies are higher when endogeneity is considered in the model. The Bayesian estimators all display an initial slowly increasing pattern in the 1990s, and a decreasing one in the 2000s. The increasing trend in efficiency levels at the beginning of 1990s is probably because of the increased competitive pressure in the financial industry due to the deregulations introduced in the 1980s. The decreasing trend in efficiency levels started before the Great Recession, perhaps because the financial institutions were taking on riskier activities and became less focused on their traditional roles as financial intermediaries when the global pool of fixed-income securities substantially increased.

In order to evaluate the significance of endogeneity in the model, we have calculated the Bayes Factors (BF) in favor of the model with endogeneity. These results are in

Table 6. Estimated efficiencies (evaluated at means) and Bayes factor in favor of endogeneity.

	Without Endogeneity		With Endogeneity		BF†
	BE1	BE2	BE1*	BE2*
1990	0.6216	0.7125	0.7189	0.7192	3.672
1992	0.5915	0.7317	0.6179	0.7003	3.855
1994	0.6718	0.7106	0.7283	0.7146	3.781
1996	0.7103	0.7325	0.7781	0.7613	4.038
1998	0.7317	0.7716	0.7925	0.7845	4.129
2000	0.7612	0.7815	0.8105	0.8006	4.217
2003	0.7120	0.7451	0.8023	0.7942	5.333
2005	0.7101	0.7222	0.7945	0.7943	5.885
2007	0.6787	0.7104	0.7824	0.7745	6.452
2009	0.6513	0.6817	0.7748	0.7663	6.812

^†BF calculation, see Perrakis et al. (2014).

, along with the corresponding Bayes Factors (BF). The level of the Bayes Factor is higher in recent years than it is in early years and is clearly in favor of endogeneity as all Bayes factors exceed 3.5.

6. Conclusions

This paper has proposed a Bayesian approach to treat time-varying heterogeneity in a panel data stochastic frontier model setting. We introduce two new models: one with nonparametric time effects and one with effects that are driven by a number of unknown common factors. In both of the models, we do not impose parametric assumptions on the individual effects other than smoothness and we utilize the Gibbs sampler to implement our Bayesian inferences. The Monte Carlo experiments indicate that the new Bayesian estimators tend to outperform the non-Bayesian alternatives we consider, including the BC, CSS, and the KSS models, under various data generating processes. The new Bayesian estimators are used to analyze the temporal pattern of the technical efficiencies of the largest 40 U.S. banks from 1990 to 2009. The results indicate that the largest banks experienced a decrease in the efficiency with which they provided intermediation services around the time of the Great Recession.