Robust Algorithms for Change-Point Regressions Using the t-Distribution

Abstract

Regression models with change-points have been widely applied in various fields. Most methodologies for change-point regressions assume Gaussian errors. For many real data having longer-than-normal tails or atypical observations, the use of normal errors may unduly affect the fit of change-point regression models. This paper proposes two robust algorithms called EMT and FCT for change-point regressions by incorporating the t-distribution with the expectation and maximization algorithm and the fuzzy classification procedure, respectively. For better resistance to high leverage outliers, we introduce a modified version of the proposed method, which fits the t change-point regression model to the data after moderately pruning high leverage points. The selection of the degrees of freedom is discussed. The robustness properties of the proposed methods are also analyzed and validated. Simulation studies show the effectiveness and resistance of the proposed methods against outliers and heavy-tailed distributions. Extensive experiments demonstrate the preference of the t-based approach over normal-based methods for better robustness and computational efficiency. EMT and FCT generally work well, and FCT always performs better for less biased estimates, especially in cases of data contamination. Real examples show the need and the practicability of the proposed method.

1. Introduction

In regression analyses, the relationship between the response and explanatory variables often changes abruptly at some positions, termed change-points (CPs) or thresholds. Change-point (CP) locations signify the place of changes in the data structure, which is usually pivotal for decision making or for realizing some scientific issues. Hence, CP regression models have been widely used for uncovering the meaning of CPs and regression coefficients in many fields, such as engineering, economics, medicine, climatology, genomics and linguistics. There are some motivating examples of CP detection in reality. In medicine, CPs often denote threshold values that are altered by the effect of some risk factor on the response. For example, the risk of a baby having Down Syndrome (DS) at birth is low and unrelated to the mother’s age until an age threshold, but then afterward, the DS risk grows steeply with the mother’s age [1]. Another use of CP models in medical studies is for monitoring patients’ health, such as detecting heart rate trend [2] or inspecting the activity of epileptics [3]. Besides, in econometrics, CPs often signal the time of market structure changes as a result of government interference or policy changes [4]. More applications include discovering the time of changes in the temperature trend for studying the global warming effect in climatology [5] and finding changes in the DNA copy number in genomics [6].

Outliers frequently occur in regression data due to measurement errors or improperly sampling a fraction of data from a different population. Outliers can lead to distortions in the model fitting. Outliers even have greater impacts on CP regressions since they may not only distort the estimations of model parameters but also twist the segmented structure of data. Fearnhead and Rigaill [6] provided a real example to illustrate the CP detection errors caused by outliers. They remarked that many CP methods are sensitive to outliers because of the modeling assumption of Gaussian noises, with much work on CP regressions being carried out for Gaussian models, such as in [7,8,9,10,11]. Despite that the distortion very likely resulted from outliers and the considerable concern for robust procedures in the literature of CP detection, there are few studies on robust estimations for CP regressions.

In regression analysis, the two common methods for solving outlier problems are outlier diagnostic and robust procedures. The robust approach is more recommended [12,13] since the diagnostic approach has issues such as it may fail due to the masking effect, the probability distribution of the resulting statistics is unknown and the variations of estimators may be underestimated. Hence, in this study, we aim at finding a new robust procedure for CP regressions. Here, we regard an estimation method to be robust in the sense of insensitivity to outliers and heavy-tailed distributions.

In this paper, we propose a robust CP regression model using t-distributions. The longer tail of a t-distribution makes it a more robust approach to CP regression data fitting by giving less weight to observations that are atypical in the computation of CPs and regression parameters. With the t CP regression approach, the normal distribution in each regime is embedded in a broader class of symmetric distributions, with an additional parameter called the degrees of freedom, r. As commented by Lange et al. [14] and Peel and McLachlan [15], the parameter r is helpful for tuning the robustness of the t-based method so that it is crucial for achieving resistant estimations. The proposed method treats CPs as latent class variables representing data points lying in respective regimes so that the new method is free of the difficulty of non-differentiability of the log-likelihood function with respect to CPs regarded as parameters. The computations of t log-likelihood are simplified by writing the t-distribution as a scale mixture of normal distributions. Through two latent variables introduced into the observed data, the expectation and maximization (EM) algorithm is employed to estimate the CP model.

Considering the indefiniteness of the demarcation between two adjacent pieces, we extend the crisp CP model to a fuzzy class model. We create a fuzzy classification t (FCT) method which is more robust than the t-based EM-type algorithm. To make the t CP model more tolerant of high leverage outliers, we further propose a modified version to first moderately prune high leverage points and then employ the new method to the pruned dataset. The rules for deciding the proportion of pruning and the degrees of freedom of t-distributions are also discussed. Through numerous simulations and real applications, we demonstrate the effectiveness and superiority of the proposed methods by comparing them with existing methods.

The rest of this paper is organized as follows. In Section 2, we present some related work in the literature. In Section 3, we develop the new methods and illustrate the determination of the degrees of freedom of the t-distribution, the resistance to high leverage outliers and the robust properties of the proposed methods. In Section 4, experiments with numeric and real examples are provided to show the effectiveness and superiority of the proposed methods. Finally, the discussion and conclusions are stated in Section 5.

2. Related Work

Since change-point (CP) problems are often encountered in practice, numerous methodological approaches have been implemented for estimating CP models, such as likelihood-based estimation, Bayesian analyses, grid-searching approaches and clustering methods. A main difficulty with the likelihood estimations for CP regressions is the non-differentiability of the likelihood function with respect to the CPs viewed as parameters. Researchers have tried to overcome this problem through various approximations or smooth transitions between two adjoining regimes. For example, Muggeo [16] created a segmented (SEG) regression method by means of a simple linearization technique. This method allows simultaneous inference on CPs and regression coefficients, but it is not resistant to outliers and only workable for continuous models. Recently, Lu and Chang [10] conquered the non-differentiable problem by transforming the CP regression model into a mixture regression model and introduced an EM-BP algorithm to derive estimates. There have been more recent methods for solving CP problems. Chakar et al. [17] consider the infeasibility of the dynamic programming algorithm for dependent series. They proposed a robust estimator of the autocorrelation parameter to de-correlate the autoregressive time series. Then, the classical inference for independent series can be utilized to estimate the CPs in the autoregressive series.

There are some recent Bayesian CP analyses. For instance, Ko et al. [18] proposed the Dirichlet process hidden Markov model (DPHMM) to detect CPs without requiring the specification of the number of CPs a priori. Bardwell and Fearnhead [19] presented a Bayesian approach to detect abnormal regions in multiple time series, allowing the presence of changes in a small subset of the time series.

Nonparametric CP methods were also suggested for the data, which cannot be well-approximated by any parametric distribution, such as in the works of Zou et al. [20] and Haynes et al. [21]. Moreover, Frick et al. [7] considered the question of search accuracy in CP analysis. They introduced a simultaneous multiscale CP estimator (SMUCE) to provide the estimates of an unknown step function with the probability of over- and underestimating the number of CPs being controlled. Pein et al. [8] further proposed a heterogeneous simultaneous multiscale change-point estimator (H-SMUCE) for a heterogeneous piecewise constant model. More recently, various researchers have focused on developing computationally efficient methods [22,23]. For more reviews on CP work, we refer to articles [24,25] and references therein.

In spite of recent rapid developments on various aspects of CP detection, there is less work on the robustness to the presence of outliers [6,24]. To alleviate the disturbance of outliers, data may be pre-processed to remove outliers. After editing suspected outliers, the subsequent analysis is still based on the normality assumption. A serious problem with this approach is that the resulted influences may fail to reflect uncertainty in the “cleaned” data; particularly, standard errors may be too small.

The least absolute deviations (LAD) estimation has been used to estimate CP regression models robustly by a few researchers [26,27]. A robust approach connected with the LAD procedure is assuming that regression errors have a Laplace distribution. Yang [28] and Jafari et al. [29] employ Laplace distributions to estimate CP regression models robustly. The restriction of a single CP detection limits their applicability because multiple CPs often exist in reality. Although the LAD method is more robust than the least squares (LS) estimation, it is computationally intensive to optimize the objective function [25].

M-estimation is another popular robust approach in linear regressions, which replaces the least squares criterion with a resistant criterion. Considering the robustness of M-estimation, Fearnhead and Rigaill [6] proposed to associate the penalized cost approaches with a less sensitive loss function for estimating CPs robustly. The authors claimed that a bounded loss, such as the bi-weight loss, is favorable for robust estimations. However, their method relies on a constant variance assumption which is unsuitable in many applications [8]. Recently, a robust estimator based on the Wilcoxon statistic was proposed for estimating the CP of the mean [30], but this approach is restricted to a single CP detection.

The t-distribution has also been employed for robust fitting of a statistical model to the data structure. For example, Lange et al. [14] proposed to use t-distributions for the robust fit of a traditional linear regression model to data. Peel and McLachlan [15] proposed a mixture model of t-distributions for robust clustering. Yao et al. [31] presented a t mixture regression model for estimating model parameters robustly. Lin et al. [32,33] presented the tests on heteroscedasticity in Student t and skew-t-normal regression models, respectively. Although t-distributions have been used in mixture regressions for making resistant estimations, they were barely considered for CP regression problems. Though locating CPs in a piecewise regression model is similar to classifying data into classes of similar objects, CP regression problems are different from mixture regressions in some aspects. For example, data points in CP regressions are ordered according to the order of some partitioning variable, and hence there are restrictions on the memberships of data points belonging to each individual segment. Subsequently, the robust mixture regressions of t-distributions cannot be applied to CP regressions directly. The rare implementation of t-distributions in CP regressions may be because the t likelihood function is non-differentiable with respect to CPs and there is no explicit form for the maximizer of the t-log-likelihood.

A few studies have utilized t-distributions to solve CP regression problems. Osorio and Galea [34] located a single CP for t regression models based on Schwarz Information Criterion (SIC). Their method is restricted to a single CP detection, and the degrees of freedom of t-distributions are not well-estimated. Lin et al. [35] adopted a Bayesian approach to identify the position of changes in variance for a t regression model with the regression coefficients assumed to be constant. The assumption of constant regression coefficients is restrictive and unrealistic. Furthermore, the burden of their computations is heavy because six steps are required in one pass of the Gibbs sampler for locating a single CP only, not mentioned for detecting multiple CPs. That is why they suggested detecting multiple CPs one by one in a hierarchical binary way instead of simultaneous detection. However, the hierarchical binary method for locating multiple CPs may not result in a correct optimum [11]. Moreover, these researchers did not illustrate the resistance of their method against high leverage outliers. It is known that t-distributions are sensitive to high leverage outliers, and these kinds of outliers usually have severe effects on the fit of a regression model to the data.

3. Robust Change-Point Regression Using t-Distributions

3.1. An EMT Algorithm for Change-Point Regression Models

Assume a dataset, $\left\{(x_1^{\prime },y_1),\dots ,(x_n^{\prime },y_n)\right\}$, is randomly drawn from a population of CP regressions, with $x_j=(x_{j1,}\dots ,x_{jp}{)}^{\prime }\in {\Re }^p$ ($(\cdot {)}^{\prime }$ represents the transpose of a vector or a matrix) and $y_j\in \Re$, $j=1,\dots ,n$ expressing the jth independent observation and response, respectively. Suppose that $x_d^{*}=(x_{1d},x_{2d},\dots ,x_{nd}{)}^{\prime }$, for some $d\in \left\{1,\dots ,p\right\}$, is an independent variable that splits the domain of independent variables into sub-domains, such that regression models vary from sub-domain to sub-domain. Thus, we may assume that the dataset $\left\{(x_1^{\prime },y_1),\dots ,(x_n^{\prime },y_n)\right\}$ has been positioned in the increasing order of $x_d^{*}$. Accordingly, CPs can be stated by the rank of the ordered data. Suppose a CP regression model contains c − 1 CPs at ${\tau }_1,{\tau }_2,\dots ,{\tau }_{c-1}$, with $1\le {\tau }_1<{\tau }_2<\dots <{\tau }_{c-1}\le n-1$ (${\tau }_{c-1}<n$ since the last observation cannot be a CP). Set ${\tau }_0=0$ and ${\tau }_c=n$. Then, the CP regression model can be represented as follows. Given $(x_j^{\prime },y_j)$ lying in the ith segment, i.e., ${\tau }_{i-1}<j\le {\tau }_i$,

$$y_{j|i}={\beta }_{i0}+{\beta }_{i1}{x}_{j1}+\dots +{\beta }_{ip}{x}_{jp}+e_{j|i}, i=1,\dots ,c,$$

(1)

where $e_{j|i}$ are independent with mean 0 and variance ${\sigma }_i^2$, $i=1,\dots ,c$. Since CPs can appear randomly among $\left\{1,\dots ,n-1\right\}$, the c − 1 CPs can be viewed as any possible combination of c – 1 numbers drawn from $\left\{1,\dots ,n-1\right\}$. Let $\mathsf{\tau }={\left({\tau }_1,{\tau }_2,\dots ,{\tau }_{c-1}\right)}^{\prime }$ with $1\le {\tau }_1<{\tau }_2<\dots <{\tau }_{c-1}\le n-1$ be a collection of c − 1 CPs. Define a random variable $w_{\mathsf{\tau }}$ as:

$$w_{\mathsf{\tau }}=\left\{\begin{array}{cc}1,\hfill & \mathrm{if} 1\le {\tau }_1<{\tau }_2<\dots <{\tau }_{c-1}\le n-1 \mathrm{are} \mathrm{CPs},\\ 0,\hfill & \mathrm{otherwise},\end{array}\right.$$

(2)

with:

$$P(w_{\mathsf{\tau }}=1)={\epsilon }_{\mathsf{\tau }}$$

and

$$\underset{1\le {\tau }_1<{\tau }_2<\dots <{\tau }_{c-1}\le n-1}{\sum \dots \sum }{\epsilon }_{\mathsf{\tau }}=1.$$

Then, define $z_{ij}$ as:

$$z_{ij}=\left\{\begin{array}{cc}\underset{\begin{array}{c}1\le {\tau }_1<{\tau }_2<\dots <{\tau }_{c-1}\le n-1\\ {\tau }_{i-1}<j\le {\tau }_i\end{array}}{{\displaystyle \sum \dots \sum }}{w}_{\mathsf{\tau }},& 1\le i\le c, 1\le j\le n, \\ 0,& \mathrm{otherwise}.\end{array}\right.$$

(3)

Note that $z_{ij}$, $i=1,\dots ,c$, $j=1,\dots ,n$ are actually the class variables, where

$$z_{ij}=\left\{\begin{array}{cc}1,& \mathrm{if} (x_j,y_j) \mathrm{is} \mathrm{from} \mathrm{segment} i,\\ 0,& \mathrm{otherwise}.\end{array}\right.$$

Thus, the sample $\left(x,y\right)={\left((x_1^{\prime },y_1),\dots ,(x_n^{\prime },y_n)\right)}^{\prime }$ is incomplete with missing data

$$z={\left(z_{11},z_{12},\dots ,z_{cn}\right)}^{\prime }.$$

The EM algorithm is used for estimations. The complete likelihood function is:

$$L^c\left(\mathsf{\theta }|\left(x,y\right),z\right)={\displaystyle \prod _{i=1}^c{\displaystyle \prod _{j=1}^n{\left[{\pi }_i{f}_i(y_j|x_j)\right]}^{z_{ij}}}},$$

where ${\pi }_i$ are mixing proportions and $\mathsf{\theta }$ is the vector of parameters included in regressions, and $f_i(y_j|x_j)$ is the probability density function of response $y_j$ given by regressors $x_j$ in the ith segment. The complete log-likelihood function is given by:

$$l^c\left(\mathsf{\theta }|\left(x,y\right),z\right)={\displaystyle \sum _{i=1}^c{\displaystyle \sum _{j=1}^n{z}_{ij}}\log \left({\pi }_i\ln f_i(y_j|x_j)\right)}$$

(4)

For the E step of EM, we need the conditional expectation of $z_{ij}$ given by $\left(x,y\right)$ under ${\mathsf{\theta }}^0$.

$$E\left(z_{ij}|(x,y),{\mathsf{\theta }}^0\right)=\underset{\begin{array}{c}1\le {\tau }_1<{\tau }_2<\dots <{\tau }_{c-1}\le n-1\\ {\tau }_{i-1}<j\le {\tau }_i\end{array}}{{\displaystyle \sum \dots \sum }}E\left(w_{\mathsf{\tau }}|(x,y),{\mathsf{\theta }}^0\right)$$

$$=\underset{\begin{array}{c}1\le {\tau }_1<{\tau }_2<\dots <{\tau }_{c-1}\le n-1\\ {\tau }_{i-1}<j\le {\tau }_i\end{array}}{{\displaystyle \sum \dots \sum }}P\left(w_{\mathsf{\tau }}=1|(x,y),{\mathsf{\theta }}^0\right).$$

Based on Bayes’ theorem, $P\left(w_{\mathsf{\tau }}=1|(x,y),{\mathsf{\theta }}^0\right)$ is estimated by:

$${\epsilon }_{\mathsf{\tau }}^{*}=\frac{{\displaystyle \prod _{j=1}^{{\tau }_1}{f}_1^0\left(y_j|x_j\right){\displaystyle \prod _{j={\tau }_1+1}^{{\tau }_2}{f}_2^0\left(y_j|x_j\right)\dots {\displaystyle \prod _{j={\tau }_{c-1}+1}^n{f}_c^0\left(y_j|x_j\right){\epsilon }_{\mathsf{\tau }}^0}}}}{\underset{1\le k_1<k_2<\dots <k_{c-1}\le n-1}{{\displaystyle \sum \dots \sum }}{\displaystyle \prod _{j=1}^{k_1}{f}_1^0\left(y_j|x_j\right)\dots {\displaystyle \prod _{j=k_{c-1}+1}^n{f}_c^0\left(y_j|x_j\right){\epsilon }_k^0}}},$$

(5)

where the superscript 0 indicates that parameters are given as ${\mathsf{\theta }}^0$. Then, $E\left(z_{ij}|(x,y),{\mathsf{\theta }}^0\right)$ is estimated by:

$$p_{ij}^{*}=\left\{\begin{array}{cc}\underset{\begin{array}{c}1\le {\tau }_1<{\tau }_2<\dots <{\tau }_{c-1}\le n-1\\ {\tau }_{i-1}<j\le {\tau }_i\end{array}}{{\displaystyle \sum \dots \sum }}{\epsilon }_{\mathsf{\tau }}^{*},& i=1,\dots ,c, j=1,\dots ,n,\\ 0,& \mathrm{otherwise}.\end{array}\right.$$

(6)

Replace $z_{ij}$ in (4) with $p_{ij}^{*}$ in Equation (6) and denote the resulted expression as $Q\left(\mathsf{\theta }|(x,y),{\mathsf{\theta }}^0\right)$. The M step is to maximize

$$Q\left(\mathsf{\theta }|(x,y),{\mathsf{\theta }}^0\right)={\displaystyle \sum _{i=1}^c{\displaystyle \sum _{j=1}^n{p}_{ij}^{*}}}\ln \left({\pi }_i{f}_i(y_j|x_j)\right).$$

(7)

Repeating the steps E and M until the stop criterion is met, we then obtain the estimator of CPs as:

$$\widehat{\mathsf{\tau }}=\underset{1\le {\tau }_1<{\tau }_2<\dots <{\tau }_{c-1}\le n-1}{\arg \max }\left({\epsilon }_{\mathsf{\tau }}\right), \mathsf{\tau }={\left({\tau }_1,{\tau }_2,\dots ,{\tau }_{c-1}\right)}^{\prime }.$$

To robustly estimate the CP regression model given by Equation (1), assume that the regression errors $e_{j|i}$, $j=1,\dots ,n$, follow a t-distribution, with $r_i$ degrees of freedom (DF), scale parameter ${\sigma }_i$ and the density:

$$f(e|\sigma ,r)=\frac{1}{\sigma }h\left(\frac{e}{\sigma }|r\right), h(t|r)=\frac{\mathsf{\Gamma }\left(\frac{r+1}{2}\right)}{{(\pi r)}^{1/2}\mathsf{\Gamma }\left(\frac{r}{2}\right){\left\{1+\frac{t^2}{r}\right\}}^{\frac{1}{2}(r+1)}},$$

(8)

where $h(t|r)$ is the density of a standard t-distribution with r DF, t(r). We first suppose $r_i$s are known, and we discuss the decision of $r_i$s later. The complete log-likelihood is:

$$l^c\left(\mathsf{\theta }|\left(x,y\right),z\right)={\displaystyle \sum _{i=1}^c{\displaystyle \sum _{j=1}^n{z}_{ij}}}\ln \left({\pi }_i{f}_i(y_j-{X^{\prime }}_j{\mathsf{\beta }}_i|x_j,r_i,{\sigma }_i)\right),$$

where $f_i(\cdot )$ is the density of Equation (8), $X_j^{\prime }=(1,x_j^{\prime })=(1,x_{j1},\dots ,x_{jp})$, ${\mathsf{\beta }}_i=({\beta }_{i0},{\beta }_{i1},\dots ,{\beta }_{ip}{)}^{\prime }$, $\mathsf{\theta }={\left({\mathsf{\beta }}_1^{\prime },\dots ,{\mathsf{\beta }}_c^{\prime },{\sigma }_1,\dots ,{\sigma }_c\right)}^{\prime }$. To simplify the computations, we express the t-distribution as a scale mixture of normal distributions (c.f. [12,28]). We introduce into the data vector additional missing data $v=(v_1,\dots ,v_n{)}^{\prime }$, satisfying that given $z_{ij}=1$,

$$y_j-X_j^{\prime }{\mathsf{\beta }}_i|x_j,v_j,z_{ij}=1~N\left(0,{\sigma }_i^2/v_j\right),v_j|z_{ij}=1~\mathrm{gamma}\left(\frac{r_i}{2},\frac{r_i}{2}\right),$$

(9)

where $\mathrm{gamma}(\alpha ,\lambda )$ has density $g(v;\alpha ,\lambda )=\frac{1}{\mathsf{\Gamma }(\alpha )}{\lambda }^{\alpha }{v}^{\alpha -1}{e}^{-\lambda v}$, g > 0. Then, $y_j-X_j^{\prime }{\mathsf{\beta }}_i|x_j,z_{ij}=1$ has a $t(r_i)$ distribution with a scale parameter ${\sigma }_i$. The complete log likelihood function becomes:

$$\begin{array}{l}{l}^c\left(\mathsf{\theta }|\left(x,y\right),z,v\right)\\ ={\displaystyle \sum _{j=1}^n{\displaystyle \sum _{i=1}^c{z}_{ij}\ln }}\left({\pi }_i\frac{\sqrt{v_j}}{{\sigma }_i}\varphi \left(\frac{\sqrt{v_j}(y_j-X_j^{\prime }{\mathsf{\beta }}_i)}{{\sigma }_i}\right)g(v_j|r_i)\right)\\ ={\displaystyle \sum _{j=1}^n{\displaystyle \sum _{i=1}^c{z}_{ij}\ln }}\left({\pi }_i\frac{\sqrt{v_j}}{\sqrt{2\pi }{\sigma }_i}\exp \left(-\frac{v_j{(y_j-X_j^{\prime }{\mathsf{\beta }}_i)}^2}{2{\sigma }_i^2}\right)\frac{{\left(\frac{r_i}{2}\right)}^{r_i/2}}{\mathsf{\Gamma }\left(\frac{r_i}{2}\right)}{v}_j^{r_i/2-1}\exp (-v_j{r}_i/2)\right)\\ ={\displaystyle \sum _{j=1}^n{\displaystyle \sum _{i=1}^c{z}_{ij}\ln }}{\pi }_i-\frac{1}{2}{\displaystyle \sum _{j=1}^n{\displaystyle \sum _{i=1}^c{z}_{ij}\ln }}{\sigma }_i^2-\frac{1}{2}{\displaystyle \sum _{j=1}^n{\displaystyle \sum _{i=1}^c{z}_{ij}}}\frac{v_j{(y_j-X_j^{\prime }{\mathsf{\beta }}_i)}^2}{{\sigma }_i^2}\\ +{\displaystyle \sum _{j=1}^n{\displaystyle \sum _{i=1}^c{z}_{ij}\ln }}\left(\frac{\sqrt{v_j}}{\sqrt{2\pi }}\frac{{\left(\frac{r_i}{2}\right)}^{r_i/2}}{\mathsf{\Gamma }\left(\frac{r_i}{2}\right)}{v}_j^{r_i/2-1}\right)-\frac{1}{2}{\displaystyle \sum _{j=1}^n{\displaystyle \sum _{i=1}^c{z}_{ij}}}{v}_j{r}_i,\end{array}$$

where $\varphi (\cdot )$ is the density of $N(0,1)$. To use EM, given the current estimate ${\mathsf{\theta }}^{(k)}$, since the last two terms do not contain unknown parameters $\mathsf{\theta }$, the computation of $E\left(l^c\left(\mathsf{\theta }|\left(x,y\right),z,v\right)|\left(x,y\right),{\mathsf{\theta }}^{(k)}\right)$ in step E simplifies to computing $E\left(z_{ij}|(x,y),{\mathsf{\theta }}^{(k)}\right)$ and $E\left(v_j|(x,y),{\mathsf{\theta }}^{(k)},z_{ij}=1\right)$. Based on Equation (6):

$$E\left(z_{ij}|(x,y),{\mathsf{\theta }}^{(k)}\right)=p_{ij}^{(k+1)}=\left\{\begin{array}{cc}\underset{\begin{array}{c}1\le {\tau }_1<{\tau }_2<\dots <{\tau }_{c-1}\le n-1\\ {\tau }_{i-1}<j\le {\tau }_i\end{array}}{{\displaystyle \sum \dots \sum }}{\epsilon }_{\mathsf{\tau }}^{(k)},& i=1,\dots ,c, j=1,\dots ,n,\\ 0,& \mathrm{otherwise},\end{array}\right.,$$

(10)

where ${\epsilon }_{\mathsf{\tau }}^{(k)}$ is computed by Equation (5) with the density $f_i(y_j-X_j^{\prime }{\mathsf{\beta }}_i^{(k)}|{\sigma }^{(k)},r_i)$ given by Equation (8).

Moreover, the gamma distribution is the conjugate prior distribution for $v_j$. It is easy to show that:

$$v_j|(x,y),z_{ij}=1,{\mathsf{\theta }}^{(k)} ~ \mathrm{gamma}\left(\frac{r_i+1}{2},\frac{1}{2}\left[r_i+{\left(\frac{y_j-X_j^{\prime }{\mathsf{\beta }}_i^{(k)}}{{\sigma }_i^{(k)}}\right)}^2\right]\right),$$

(11)

$$E\left(v_j|(x,y), {\mathsf{\theta }}^{(k)}, z_{ij}=1\right)=q_{ij}^{(k+1)}=\frac{r_i+1}{r_i+{\left[(y_j-X_j^{\prime }{\mathsf{\beta }}_i^{(k)})/{\sigma }_i^{(k)}\right]}^2}.$$

(12)

Then, step M is to maximize:

$$E\left(l^c\left(\mathsf{\theta }|\left(x,y\right),z,v\right)|\left(x,y\right),{\mathsf{\theta }}^{(k)}\right)$$

$$\propto Q(\mathsf{\theta }|(x,y),{\mathsf{\theta }}^{(k)})={\displaystyle \sum _{j=1}^n{\displaystyle \sum _{i=1}^c{p}_{ij}^{(k+1)}\ln }}{\pi }_i-\frac{1}{2}{\displaystyle \sum _{j=1}^n{\displaystyle \sum _{i=1}^c{p}_{ij}^{(k+1)}\ln }}{\left({\sigma }_i^{(k)}\right)}^2-\frac{1}{2}{\displaystyle \sum _{j=1}^n{\displaystyle \sum _{i=1}^c{p}_{ij}^{(k+1)}}}{q}_{ij}^{(k+1)}{\left(\frac{y_j-X_j^{\prime }{\mathsf{\beta }}_i^{(k)}}{{\sigma }_i^{(k)}}\right)}^2$$

Taking partial derivatives of $Q\left(\mathsf{\theta }|(x,y),{\mathsf{\theta }}^{(k)}\right)$ with respect to all parameters with the Lagrange multiplier under constraint ${\displaystyle \sum _{i=1}^c{\pi }_i}=1$, we obtain the updated estimates:

$${\mathsf{\beta }}_i^{(k+1)}={\left(X^{\prime }{U}_i^{(k+1)}X\right)}^{-1}{X}^{\prime }{U}_i^{(k+1)}Y,$$

(13)

$${\left({\sigma }_i^2\right)}^{(k+1)}=\frac{{\displaystyle \sum _{j=1}^n{p}_{ij}^{(k+1)}}{q}_{ij}^{(k+1)}{\left(y_j-X_j^{\prime }{\mathsf{\beta }}_i^{(k+1)}\right)}^2}{{\displaystyle \sum _{j=1}^n{p}_{ij}^{(k+1)}}},$$

(14)

where $X\in {\Re }^{n\times (p+1)}$ is a matrix with the jth row $X_j^{\prime }=(1,x_j^{\prime })=(1,x_{j1},\dots ,x_{jp})$, $Y\in {\Re }^n$ is a vector with the jth component $y_j$ and $U_i^{(k+1)}$ is a diagonal matrix with the jth diagonal element equal to $p_{ij}^{(k+1)}{q}_{ij}^{(k+1)}$. Based on the above, we propose an EMT Algorithm 1 for estimating CP regression models, as follows.

Algorithm 1. EMT

$\mathrm{Set} s=C_{c-1}^{n-1}$$, {\mathsf{\epsilon }}^{(k)}={\left[{\epsilon }_{\mathsf{\tau }}^{(k)}:\mathsf{\tau }={\left({\tau }_1,{\tau }_2,\dots ,{\tau }_{c-1}\right)}^{\prime },1\le {\tau }_1<{\tau }_2<\dots <{\tau }_{c-1}\le n-1\right]}_{s\times 1}$,   $p^{(k+1)}={\left[p_{ij}^{(k+1)}:1\le 1\le c,1\le j\le n\right]}_{cn\times 1}\in {\Re }^{cn}$, $q^{(k+1)}={\left[q_{ij}^{(k+1)}:1\le i\le c,1\le j\le n\right]}_{cn\times 1}\in {\Re }^{cn}$.   $\mathrm{Let} \eta >0$$, k= 0,{\epsilon }_{\mathsf{\tau }}^{\left(0\right)}=1/C_{c-1}^{n-1}$, $\mathrm{for} \mathrm{each} \mathrm{possible} \mathsf{\tau }$$, \mathrm{and} q_{{}_{ij}}^{(0)}=1$$, i=1,\dots ,c$ $, j=1,\dots ,n$.   $\mathrm{Step} 1. \mathrm{Compute} p^{(0)}$$\mathrm{with} \mathrm{Equation} (6) \mathrm{using} {\mathsf{\epsilon }}^{(0)}$$. \mathrm{Then}, \mathrm{compute} {\mathsf{\beta }}_i^{(0)}$$,{\left({\sigma }_i^2\right)}^{(0)}$,   $i=1,\dots ,c$, with Equations (13) and (14), respectively.   $\mathrm{Step} 2. \mathrm{Compute} p^{(k+1)}$$\mathrm{and} q^{(k+1)}$ with Equations (10) and (12), respectively.   $\mathrm{Step} 3. \mathrm{Compute} {\mathsf{\beta }}^{(k+1)}\equiv {\left({\mathsf{\beta }}_1^{(k+1)},\dots ,{\mathsf{\beta }}_c^{(k+1)}\right)}^{\prime }\in {\Re }^{c(p+1)}$$\mathrm{with} p^{(k+1)}$$\mathrm{and} q^{(k+1)}$ by Equation (13).   $\mathrm{Step} 4. \mathrm{Compute} {\mathsf{\sigma }}^{(k+1)}\equiv {\left({\sigma }_1^{(k+1)},\dots ,{\sigma }_c^{(k+1)}\right)}^{\prime }$$\mathrm{with} {\mathsf{\beta }}^{(k+1)}$$, p^{(k+1)}$$\mathrm{and} q^{(k+1)}$ by Equation (14).   $\mathrm{Step} 5. \mathrm{Update} {\mathsf{\epsilon }}^{(k+1)}$$\mathrm{with} {\mathsf{\beta }}^{(k+1)}$$, {\mathsf{\sigma }}^{(k+1)}$$\mathrm{and} {\mathsf{\epsilon }}^{(k)}$ by Equation (5) with the density given by Equation (8).   $\mathrm{Step} 6. \mathrm{If} ‖{\mathsf{\epsilon }}^{(k+1)}-{\mathsf{\epsilon }}^{(k)}‖<\eta$$, \mathrm{then} \mathrm{stop}; \mathrm{else}, k=k+1$ and go to Step 2.   After the EMT algorithm converges, the estimates of c − 1 CPs are given by   $\widehat{\mathsf{\tau }}=\underset{1\le {\tau }_1<{\tau }_2<\dots <{\tau }_{c-1}\le n-1}{\arg \max }\left({\epsilon }_{\mathsf{\tau }}\right)$$, \mathsf{\tau }=({\tau }_1,{\tau }_2,\dots ,{\tau }_{c-1}{)}^{\prime }$.

3.2. A FCT Algorithm for t CP Regression Models

Since the EM algorithm converges slowly when data involve noises (c.f. [36]), we extend the crisp model to a fuzzy class model for improving the convergence rate and the precision of estimations. Similar to Lu and Chang [10], using the fuzzy partitioning concept, we regard each probable CP combination as a partition of data with a fuzzy membership and then transform them into the pseudo memberships of data belonging to each individual segment. As in previous descriptions, consider a regression model with c − 1 CPs, as given in Equation (1). Suppose $\mathsf{\tau }={\left({\tau }_1,{\tau }_2,\dots ,{\tau }_{c-1}\right)}^{\prime }$, $1\le {\tau }_1<{\tau }_2<\dots <{\tau }_{c-1}\le n-1$, is a collection of CPs with a fuzzy membership ${\alpha }_{\mathsf{\tau }}$ under the conditions $0\le {\alpha }_{\mathsf{\tau }}\le 1$ and $\underset{1\le {\tau }_1<{\tau }_2<\dots <{\tau }_{c-1}\le n-1}{\sum \dots \sum }{\alpha }_{\mathsf{\tau }}=1$. Now, let $z_{ij}$ be the fuzzy membership of observation $(x_j,y_j)$ belonging to the ith piece. Then, $z_{ij}$ can be expressed by ${\alpha }_{\mathsf{\tau }}$ as follows (c.f. [10]):

$$z_{ij}=\left\{\begin{array}{cc}\underset{\begin{array}{c}1\le {\tau }_1<{\tau }_2<\dots <{\tau }_{c-1}\le T-1\\ {\tau }_{i-1}<j\le {\tau }_i\end{array}}{{\displaystyle \sum \dots \sum }}{\alpha }_{\mathsf{\tau }},& i=1,\dots c-1, i\le j\le n-c+i, \\ 0,& i=1,\dots c-1, 1\le j<i \mathrm{or} n-c+i<j\le n,\\ 1-{\displaystyle \sum _{i=1}^{c-1}{\mu }_{ij}},& i=c, 1\le j\le n.\end{array}\right.$$

(15)

Now, the CP model is a fuzzy class model with a fuzzy c-partition, $\left\{z_1,\dots ,z_c\right\}$, $z_i={\left(z_{\mathrm{i}1},\dots ,z_{in}\right)}^{\prime }$, $i=1,\dots ,c,$ and ${\displaystyle \sum _{i=1}^c{z}_{ij}}=1$, $j=1,\dots ,n$. The regression parameters can be estimated by maximizing the fuzzy classification maximum likelihood (FCML) function (c.f. [10,37]):

$$J(\mathsf{\theta })={\displaystyle \sum _{i=1}^c{\displaystyle \sum _{j=1}^n{z}_{ij}^m\ln }}{f}_i((x_j,y_j)|\mathsf{\theta })+w{\displaystyle \sum _{i=1}^c{\displaystyle \sum _{j=1}^n{z}_{ij}^m\ln }}{\pi }_i$$

(16)

subject to ${\displaystyle \sum _{i=1}^c{\pi }_i}=1$, ${\pi }_i>0$, where m > 1, $r\ge 0$, $z_{ij}\in [0,1]$, ${\displaystyle \sum _{i=1}^c{z}_{ij}}=1$, $i=1,\dots ,c$, $j=1,\dots ,n$ and $\mathsf{\theta }={\left({\mathsf{\beta }}_1^{\prime },\dots ,{\mathsf{\beta }}_c^{\prime },{\sigma }_1,\dots ,{\sigma }_c,{\pi }_1,\dots ,{\pi }_c\right)}^{\prime }$. As stated, assume that the CP regression errors, $e_{j|i}=y_j-X_j^{\prime }{\mathsf{\beta }}_i$, $j=1,\dots ,n$, have a t(r_i)-distribution. With the scale mixture expression of t(r) in Equation (9), the FCML objective function in Equation (16) is:

$$\begin{array}{l}{T}_{m,w}(z,\mathsf{\pi },\mathsf{\beta },\mathsf{\sigma })\\ ={\displaystyle \sum _{i=1}^c{\displaystyle \sum _{j=1}^n{z}_{ij}^m\ln f_i(}}{y}_j|{\mathsf{\beta }}_i,{\sigma }_i)+w{\displaystyle \sum _{i=1}^c{\displaystyle \sum _{j=1}^n{z}_{ij}^m\ln {\pi }_i}}\\ ={\displaystyle \sum _{i=1}^c{\displaystyle \sum _{j=1}^n{z}_{ij}^m\ln \left(\frac{\sqrt{v_j}}{\sqrt{2\pi }{\sigma }_i}\exp \left(-\frac{v_i{(y_j-X_j^{\prime }{\mathsf{\beta }}_i)}^2}{2{\sigma }_i^2}\right)\frac{{\left(\frac{r_i}{2}\right)}^{r_i/2}}{\mathsf{\Gamma }\left(\frac{r_i}{2}\right)}{v}_j^{r_i/2-1}\exp (-v_j{r}_i/2)\right)}}+w{\displaystyle \sum _{i=1}^c{\displaystyle \sum _{j=1}^n{z}_{ij}^m\ln {\pi }_i}}\\ =-\frac{1}{2}{\displaystyle \sum _{i=1}^c{\displaystyle \sum _{j=1}^n{z}_{ij}^m\ln }}{\sigma }_i^2-\frac{1}{2}{\displaystyle \sum _{i=1}^c{\displaystyle \sum _{j=1}^n{z}_{ij}^m\frac{v_j{(y_j-X_j^{\prime }{\mathsf{\beta }}_i)}^2}{{\sigma }_i^2}}}+{\displaystyle \sum _{i=1}^c{\displaystyle \sum _{j=1}^n{z}_{ij}^m\ln \left(\frac{\sqrt{v_j}}{\sqrt{2{\pi }_i}}\frac{{\left(\frac{r_i}{2}\right)}^{r_i/2}}{\mathsf{\Gamma }\left(\frac{r_i}{2}\right)}{v}_j^{r_i/2-1}\right)}}\\ -\frac{1}{2}{\displaystyle \sum _{i=1}^c{\displaystyle \sum _{j=1}^n{z}_{ij}^m{v}_j{r}_i+}}w{\displaystyle \sum _{i=1}^c{\displaystyle \sum _{j=1}^n{z}_{ij}^m\ln {\pi }_i}}\end{array}$$

Based on Equation (12), replace $v_j$ with $q_{ij}$ in $T_{m,w}(z,\mathsf{\pi },\mathsf{\beta },\mathsf{\sigma })$. Take the partial derivatives of the Lagrange expression with $T_{m,w}(z,\mathsf{\pi },\mathsf{\beta },\mathsf{\sigma })$ under constraint ${\displaystyle \sum _{i=1}^c{\pi }_i}=1$, then, we obtain the updated estimates:

$${\mathsf{\beta }}_i^{(k+1)}={\left(X{U}_i^{(k)}{X}^{\prime }\right)}^{-1}X{U}_i^{(k)}Y,$$

(17)

$${\sigma }_i^{2^{(k+1)}}=\frac{{\displaystyle \sum _{j=1}^n{z}_{ij}^{m^{(k)}}}{q}_{ij}^{(k)}{\left(y_j-X_j^{\prime }{\mathsf{\beta }}_i^{(k+1)}\right)}^2}{{\displaystyle \sum _{j=1}^n{z}_{ij}^{m^{(k)}}}},$$

(18)

$${\pi }_i^{(k+1)}=\frac{{\displaystyle \sum _{j=1}^n{z}_{ij}^m{}^{{}^{(k)}}}}{n},$$

(19)

where $U_i^{(k)}$ is a diagonal matrix with the jth diagonal element equal to $z_{ij}^m{}^{{}^{(k)}}{q}_{ij}^{(k)}$. Having the regression estimates $\widehat{\mathsf{\theta }}$, we update the CP memberships ${\alpha }_{\mathsf{\tau }}\mathrm{s}$ as follows. Given a CP collection $\mathsf{\tau }$, define:

$$d_{\mathsf{\tau }}^2=-{\displaystyle \sum _{i=1}^c{\displaystyle \sum _{j={\tau }_{i-1}+1}^{{\tau }_i}\ln f_i(y_j|x_j,\widehat{\mathsf{\theta }})}}$$

We then optimize the objective function:

$$J(\mathsf{\alpha })=\underset{1\le {\tau }_1<{\tau }_2<\dots <{\tau }_{c-1}\le n-1}{{\displaystyle \sum \dots \sum }}{\alpha }_{\mathsf{\tau }}^m\exp (d_{\mathsf{\tau }}^2)$$

subject to $\underset{1\le {\tau }_1<{\tau }_2<\dots <{\tau }_{c-1}\le n-1}{\sum \dots \sum }{\alpha }_{\mathsf{\tau }}=1$. $0\le {\alpha }_{\mathsf{\tau }}\le 1$. The updated ${\alpha }_{\mathsf{\tau }}$ is:

$${\alpha }_{\mathsf{\tau }}=\frac{\exp {(-d_{\mathsf{\tau }}^2)}^{-1/(m-1)}}{\underset{1\le t_1\le t_2\le \dots \le t_{c-1}\le n-1}{\sum \dots \sum }\exp \left(-d_t^2/(m-1)\right)},$$

(20)

We propose an FCT Algorithm 2 for robustly fitting CP regression models, as follows.

Algorithm 2. FCT

$\mathrm{Set}s=C_{c-1}^{n-1}$$,{\mathsf{\alpha }}^{(k)}={\left[{\alpha }_{\mathsf{\tau }}^{(k)}:\mathsf{\tau }={\left({\tau }_1,{\tau }_2,\dots ,{\tau }_{c-1}\right)}^{\prime },1\le {\tau }_1<{\tau }_2<\dots <{\tau }_{c-1}\le n-1\right]}_{s\times 1}$,   $\mathrm{Let}\eta >0$$,k=0,{\alpha }_{\mathsf{\tau }}^{\left(0\right)}=1/C_{c-1}^{n-1}$$,\mathrm{for}\mathrm{each}\mathrm{possible}\mathsf{\tau }$$,\mathrm{and}{q}_{{}_{ij}}^{(0)}=1$$,i=1,\dots ,c$$,j=1,\dots ,n$.   $\mathrm{Step}1.\mathrm{Compute}{z}_{ij}^{(0)}$$1\le i\le c$$,1\le j\le n$ by Equation (15).   $\mathrm{Step}2.\mathrm{Compute}{\mathsf{\beta }}_i^{(k+1)}$$,{\sigma }_i^{2^{(k+1)}}$$,{\pi }_i^{(k+1)}$$,i=1,\dots ,c$, by Equations (17)–(19), respectively.   $\mathrm{Step}3.\mathrm{Update}{\mathsf{\alpha }}^{(k+1)}$$\mathrm{by}(20)\mathrm{using}{\mathsf{\beta }}_i^{(k+1)}$$,{\sigma }_i^{2^{(k+1)}}$$,i=1,\dots ,c$, and Equation (8).   $\mathrm{Step}4.\mathrm{Compute}{z}_{ij}^{(k+1)}$$\mathrm{and}{q}_{ij}^{(k+1)}$ by Equations (12) and (15), respectively.   $\mathrm{Step}5.\mathrm{If}‖{\mathsf{\alpha }}^{(k+1)}-{\mathsf{\alpha }}^{(k)}‖<\eta$$,\mathrm{then}\mathrm{stop};\mathrm{else},k=k+1$ and go to step 2.   After FCT converges, the estimates of c − 1 CPs are given by   $\widehat{\mathsf{\tau }}=\underset{1\le {\tau }_1<{\tau }_2<\dots <{\tau }_{c-1}\le n-1}{\arg \max }\left({\epsilon }_{\mathsf{\tau }}\right)$$,\mathsf{\tau }=({\tau }_1,{\tau }_2,\dots ,{\tau }_{c-1}{)}^{\prime }$.

3.3. Decide the Degrees of Freedom of t-Distributions

In this section, we discuss the decision of the degrees of freedom (DF), $r={\left(r_1,\dots ,r_c\right)}^{\prime }$. Typically, one can derive the maximum likelihood estimator (MLE) of r by maximizing the log-likelihood over $r_i$. Another solution is to compute the likelihood over a grid set, $\mathrm{G}=\{1, 2,\dots ,r_{max}\},$ and choose DF from G which has the largest likelihood (c.f. [28]). As is known, t(r) approaches normal distributions for a large enough r. In practice, r is not necessary to be very large for achieving normality. Similar to [31], we set r_max = 15. After performing lots of simulations, we found that choosing $r$ based on a grid set often fails to provide CP estimates that are precise enough. However, numerous simulation results show that both EMT and FCT with $r=1$ (EMT1 and FCT1) always produce rather precise CP estimates, no matter whether data are contaminated or not. Accurate CP estimates are critical to solving CP regression problems. To demonstrate that EMT1 and FCT1 are generally suitable to all kinds of data, we employ EMT and FCT with r = 1, 5, 10 and 15 to diversified datasets randomly generated from the models of 1 and 2 CPs, with data including 0, 5, 10 and 15 noises, respectively. The models considered for generating data are shown in Figure 1 with regression errors following a normal distribution of N(0, 0.5²) (except for the cases in Figure 1, more situations and different kinds of models were also examined, but results are similar, and hence, we only report four cases here). We evaluate the performance based on R replications in each individual circumstance with R equal to 1000 and 200 for models of 1 and 2 CPs, respectively. The evaluation criteria includes (1) the mean of CP estimates ($\overline{\mathrm{CP}}$) along with the standard errors (SECP), and (2) the sum of the mean squared errors (MSES) of model coefficient estimates computed as follows:

$$\mathrm{MSES} \equiv {\displaystyle \sum _{i,j}\mathrm{MSE}\left({\beta }_{ij}\right)},\mathrm{MSE}\left({\beta }_{ij}\right)=\frac{1}{R}{\displaystyle \sum _{r=1}^R{\left({\widehat{\beta }}_{ij(r)}-{\beta }_{ij}\right)}^2},$$

(21)

where ${\widehat{\beta }}_{ij(r)}$ is the estimate of ${\beta }_{ij}$ in the rth replication. The normal-based EM-BP algorithm for CP regressions presented in [10] is also employed and renamed EMN for comparisons. For illustrative purposes, the true regression coefficients and one of the simulated series are shown in Figure 1.

Table 1. The performance of EMT and FCT with different degrees of freedom.

(I) 1 CP Cases
(a) 0 outliers	CP = 25
Method	EMN	EMT1		EMT5	EMT10	EMT15	FCT1	FCT5	FCT10	FCT15
CPhat	25.18	25.15		25.26	25.20	25.19	25.21	25.10	25.11	25.13
SECP	0.168	0.186		0.169	0.166	0.166	0.120	0.109	0.109	0.109
MSES	0.76	1.21		0.80	0.78	0.78	0.98	0.58	0.56	0.55
(b) 15 noises	CP = 33
Method	EMN	EMT1		EMT5	EMT10	EMT15	FCT1	FCT5	FCT10	FCT15
CPhat	18.80	33.74		21.75	24.41	21.75	33.51	31.86	29.65	28.70
SECP	0.137	0.154		0.237	0.272	0.237	0.111	0.130	0.123	0.109
MSES	14.17	0.98		10.68	8.24	10.68	0.77	0.82	1.65	2.38
(c) 1 extreme outlier (2, 8)	CP = 26
Method	EMN	EMT1		EMT5	EMT10	EMT15	FCT1	FCT5	FCT10	FCT15
CPhat	18.55	25.79		25.56	24.70	23.34	26.06	26.06	25.94	25.85
SECP	0.112	0.167		0.147	0.149	0.152	0.064	0.063	0.063	0.063
MSES	1.73	1.02		0.70	0.69	0.82	0.75	0.49	0.47	0.47
(II) 2 CPs Case, CP1 = 28, CP2 = 56
Method	EMN	EMT1	EMT5	EMT10	EMT15	FCT1	FCT5	FCT10	FCT15	Method
CP1	15.79	27.79	21.43	17.64	16.45	28.39	26.14	21.73	19.27	CP1
SECP1	0.252	0.249	0.350	0.324	0.306	0.137	0.216	0.297	0.314	SECP1
CP2	68.39	57.86	62.07	64.85	65.22	56.46	54.42	57.44	60.43	CP2
SECP2	0.064	0.279	0.274	0.352	0.412	0.187	0.167	0.255	0.314	SECP2
MSES	7.97	7.81	7.09	7.48	7.79	3.33	2.62	6.15	10.31	MSES

summarizes the simulation results.

Table 1 demonstrates that EMT1 and FCT1 generally perform better than those with different degrees of freedom (DF) by virtue of more precise CP estimates and less MSE in the estimations of regression coefficients in all cases, especially the cases of data with contaminations. For clean data such as in Figure 1a, all EMTs and FCTs with different DF provide sufficient and comparable estimates of CPs and regression coefficients. However, for data containing noisy points, such as the cases of Figure 1b,d, only EMT1 and FCT1 work well in all, but the other EMTs and FCTs mostly fail to provide sufficient CP estimates. As illustrated by Peel and McLachlan [15], the use of t components provides less extreme estimates of the posterior probabilities of segment membership of the CP model. The longer tail of t-distributions makes the fitted CP models more robust by giving reduced weight to atypical observations in the calculation of CP estimates. Furthermore, the t model-based approach embeds the normal distribution for each segment into a wider class of elliptically symmetric distributions, with an additional parameter called DF which is key for tuning robustness. This illustrates that the long tail of t(1)-distributions makes EMT1 and FCT1 more robust than those with higher DF. Even with normal data involving no contaminations, EMT1 and FCT1 still perform as well as EMN because of the similarity of symmetry between normal and t-distributions. Moreover, the best robustness of EMT1 and FCT1 among all EMTs and FCTs can be seen from the robustness properties of the proposed methods illustrated in Section 3.5. Thus, one may first fit data using EMN, EMT and FCT with several different DF, such as r = 1, 5, 10 and 15, respectively. If the estimations of EMN, EMTs and FCTs are close, one may use EMN, considering that the data are clean and normally distributed. Otherwise, EMT1 and FCT1 should be adopted.

3.4. Resistance against High Leverage Outliers

Similar to the M-estimates for linear regressions, the proposed EMT and FCT are still susceptive to high leverage outliers. Here, high leverage outliers refer to outliers in the space of independent variables, since points far away from the main body of points in the X space have high leverage. Modifications for solving this problem are needed. In regression analyses, the leverage score of an observation $(x_j,y_j)$ is often defined as:

$$h_{jj}=n^{-1}+{(n-1)}^{-1}{\mathrm{MD}}_j,$$

where

$${\mathrm{MD}}_j=(x_j-\overline{x}{)}^{\prime }{S}^{-1}(x_j-\overline{x}), x_j=(x_{j1},\dots ,x_{jp}{)}^{\prime },$$

$\overline{x}$ and $S$ are the sample mean and the sample covariance of $x_j$s, respectively. However, $\overline{x}$ and $S$ are nonresistant to outliers, and leverage points may not be recognized due to the effect of other high leverage points [34]. To lessen the mask effect, we consider replacing $\overline{x}$ and $S$ with the minimum variance determinant (MCD) estimators [38] among all the subsets of $\{x_1,x_2,\dots ,x_n\}$ having h elements, $1\le h\le n$. In choosing h, we adopt the suggestion proposed by Rousseeuw and Leroy [39] and Lopuhaa and Rousseeuw [40], that h = (n + p + 1)/2 should be utilized to have the largest breakdown value for the MCD estimators. Through the fast MCD (FastMCD) algorithm of Rousseeuw and Van Driessen [41] with the small sample corrections proposed by Pison [42], the dataset is trimmed by removing the top $\alpha *100\%$ of the data sorted in the descending order of the leverages of data. Then, EMT/FCT is applied to the trimmed data. Since the proposed EMT/FCT is only sensitive to outliers with rather high leverages, a small value of $\alpha$ such as 0.05 is usually enough in practice. To advance the ability of the proposed algorithms in withstanding outliers, for data involving doubtful high leverage outliers, we suggest employing EMT/FCT after trimming those observations with a modified MD ranking in the top $\alpha *100\%$ of MDs arranged in declining order using a small $\alpha$.

3.5. The Robust Properties of EMT and FCT

In this section, we employ the influence function and gross error sensitivity to illustrate the ability of the proposed algorithms in withstanding outliers. These two criteria have been frequently used to measure robustness in statistics [13]. Suppose $y_1,\dots ,y_n$ are observed data for estimating the parameter $\theta$. An M-estimator is created by minimizing the form:

$${\displaystyle \sum _{j=1}^n\rho \left(y_j;\theta \right)},$$

(22)

where $\rho$ is a function that measures the difference between the assumed distribution and the true one. An M-estimator is derived by solving the equation:

$${\displaystyle \sum _{j=1}^n\phi \left(y_j;\theta \right)}=0,$$

(23)

where $\phi \left(y_j;\theta \right)=\left(\partial /\partial \theta \right)\rho \left(y_j;\theta \right)$. For example, the sample mean and median are the M-estimators for the loss function $\rho \left(y_j;\theta \right)={\left(y_j-\theta \right)}^2$ and $\rho \left(y_j;\theta \right)=\left|y_j-\theta \right|$, respectively. Consider a random sample, $Y_1,\dots ,Y_n$, from a population of density function $f_y(y;\theta )$ and let $\rho \left(y_j;\theta \right)=-\log f_y(y_j;\theta )$. Then, the MLE is also an M-estimator.

The influence function (IF) is useful for evaluating the relative influence of individual observations toward the value of an estimate. The IF of an M-estimator is proportional to its $\phi$ function with the form:

$$\mathrm{IF}(y;\theta )=-\frac{\phi \left(y;\theta \right)}{{\displaystyle \int \left[\frac{\partial \phi \left(y;\theta \right)}{\partial \theta }\right]d{F}_y(y)}},$$

(24)

where $F_y(y)$ denotes the distribution function of Y. An outlier might bring about issues when the influence function of an estimator is unbounded. Many useful robustness measures can be investigated through IF. For example, the gross error sensitivity (g*) denoted by

$$g^{*}=\underset{y}{\sup }\left|\mathrm{IC}(y;F,\theta )\right|$$

(25)

is one important robustness measure. $g^{*}$ can be thought of as the approximate magnitude of the worst influence that the addition of an infinitesimal point mass may have on the value of the associated estimator.

For the proposed EMT, the considered estimator is the MLE associated with the t density having r degrees of freedom:

$$f_y(y;\theta )=\frac{\mathsf{\Gamma }\left(\frac{r+1}{2}\right)}{{(\pi r)}^{1/2}\mathsf{\Gamma }\left(\frac{r}{2}\right){\left\{1+\frac{{\left(y-\theta \right)}^2}{r}\right\}}^{\frac{1}{2}(r+1)}}.$$

Maximizing $\log f_y(y;\theta )$ is equivalent to minimize

$$\ln \left\{1+\frac{{\left(y-\theta \right)}^2}{r}\right\}.$$

The loss function associated with the MLE obtained by EMTr (r denotes DF) is equivalent to:

$$\rho \left(y;\theta \right)=\ln \left\{1+\frac{{\left(y-\theta \right)}^2}{r}\right\}$$

(26)

and hence the $\phi \left(y;\theta \right)$ is:

$$\phi \left(y;\theta \right)=\frac{-2(y-\theta )}{r+{(y-\theta )}^2}.$$

(27)

We only need to analyze $\phi \left(y;\theta \right)$ because IF is proportional to $\phi \left(y;\theta \right)$ according to Equation (24). Note that:

$$\underset{y\to \infty }{\lim }\phi \left(y;\theta \right)=\underset{y\to -\infty }{\lim }\phi \left(y;\theta \right)=0.$$

(28)

Equation (28) indicates that IF approaches to 0 as y approaches to positive or negative infinity. The maximum and minimum values of $\phi \left(y;\theta \right)$ can also be derived by solving

$$\left(\partial /\partial y\right)\phi \left(y;\theta \right)=0.$$

(29)

Therefore, the $\phi \left(y;\theta \right)$ in Equation (27) is bounded and continuous, as exhibited in Figure 2a. Furthermore, Figure 2a shows that for all y values, the $\left|\phi \left(y,\theta \right)\right|$ of EMT1 is closest to 0 among all EMTr, which means that EMT1 is the most robust among all EMTr. Besides, Figure 2a demonstrates that for any given observation y, the $\left|\phi \left(y,\theta \right)\right|$ of EMTr increases with r, and $\left|\phi \left(y,\theta \right)\right|$ tends to be unbounded as $r\to \infty$ and $\left|y\right|\to \infty$. This investigation reflects that the IF associated with EMN is unbounded because t(r) approaches to a normal distribution, as $r\to \infty$. In fact, the loss function corresponding to normal regressions is equivalent to $\rho \left(y;\theta \right)={(y-\theta )}^2$ with $\phi \left(y;\theta \right)=-2(y-\theta )$. Thus, the IF with EMN is unbounded and the gross error sensitivity $g^{*}=\infty$. This is the reason why the normal-based MLE is not robust to outliers.

The plot of IF for the median is also displayed in Figure 2b. The influence of an excessively large or small y on the median is constant. However, the influence of an extremely large or small y on the estimator derived by EMT1 is tiny according to Equation (28) and Figure 2a. In fact, Equation (28) implies that an extremely large or small y can be viewed as an observation which may come from a different but unknown distribution and have no influence (i.e., $\mathrm{IC}(y;F,\theta )=0$) on our estimator. The maximum influence of observations, y, on our estimators can be derived by solving Equation (29). Hence, the proposed estimator has a bounded and continuous IF, and also a finite gross error sensitivity. Thus, the t CP model-based approach is robust from the point of view of robust statistics. EMT1 and FCT1 are more robust than those with DF greater than 1.

4. Experiments and Examples

4.1. Simulation Studies

We used numerical examples to demonstrate the effectiveness and the superiority of the proposed method. In all simulations, the stop criterion and initial values are set as $\eta =0.000005$ and ${\epsilon }_{\mathsf{\tau }}^{\left(0\right)}=1/C_{c-1}^{n-1}$, for each possible combination of c − 1 CPs, $\mathsf{\tau }=({\tau }_1,\dots ,{\tau }_{c-1}{)}^{\prime }$, $1\le {\tau }_1<{\tau }_2<\dots <{\tau }_{c-1}\le n-1$. A sample of size n is randomly generated by selecting n values of the partitioning variable uniformly distributed over the interval [0, 10]. Then, divide the interval [0, 10] evenly into c nonoverlapping subintervals, $\left(d_{i-1},d_i\right]$, $i=1,\dots ,c$, with $d_0=0$ and $d_c=10$. The values of the response variable y are computed according to respective regression settings. Seven methods used for comparisons include:

• Using EM with errors having normal distributions [10] (EMN)
• Rank-based segmented regression method [43] (RSE)
• EMro-t in [44], re-denoted as EM-Tk
• The proposed EMT with DF equal to 1 (EMT1)
• The trimmed version of EMT1 with trimmed proportion, $\alpha =5\%$ (EMT1-tm)
• The proposed FCT with DF equal to 1 (FCT1)
• The trimmed version of FCT1 with trimmed proportion, $\alpha =5\%$ (FCT1-tm)

The comparisons are based on three models under four respective scenarios using sample sizes of 50 and 75 with 1000 and 200 replicates for 1 and 3 CPs models in each simulation run, respectively. For illustrative purposes, Figure 3 displays the true coefficients of the models used, and one of the simulated series including generated data, fitted lines and noises. Four respective scenarios considered for three models are similar and include the cases of data having (1) 0 atypical observations, (2) several background noises, (3) high leverage outliers and (4) heavy-tailed distributions. The evaluation measures for comparisons are (1) the MSE of CP estimates (MSE-CP) described as

$$\mathrm{MSE}(\widehat{\tau })=\frac{1}{R}{\displaystyle \sum _{k=1}^R{\left({\widehat{\tau }}_k-\tau \right)}^2}$$

(30)

where ${\widehat{\tau }}_k$ is the CP estimate at kth replication, and (2) the sum of MSE of the regression coefficient estimates (MSES-reg) as given by Equation (21).

Table 2. Comparisons of 7 methods with models a–c under 4 different situations, respectively.

Model a: 1 CP Discontinuous Model
(a.1) 0 outliers	EMN	RSE	EM-Tk	EMT1	EMT1-tm	FCT1	FCT1-tm
MSE-CP	0.91	37.75	0.50	2.84	2.68	0.43	0.40
MSES-reg	0.57	x1	0.52	0.67	0.71	0.76	0.79
(a.2) 8 (14%) noises	EMN	RSE	EM-Tk	EMT1	EMT1-tm	FCT1	FCT1-tm
MSE-CP	66.98	111.16	135.25	0.87	1.19	0.98	0.92
MSES-reg	2.26	x	0.82	0.65	0.82	0.74	0.98
(a.3) leverage outliers (10, 10)	EMN	RSE	EM-Tk	EMT1	EMT1-tm	FCT1	FCT1-tm
MSE-CP	569.16	x	54.68	573.70	2.80	574.47	0.39
MSES-reg	11,556.91	x	x	10,900.66	0.71	x	0.75
(a.4) errors~t (2)	EMN	RSE	EM-Tk	EMT1	EMT1-tm	FCT1	FCT1-tm
MSE-CP	6.79	44.96	1.17	2.69	2.95	0.41	0.41
MSES-reg	12,603.61	x	2.98	5066.66	3395.77	1.54	0.75
Model b: 1 CP Continuous Model
(b.1) 0 outliers	EMN	RSE	EM-Tk	EMT1	EMT1-tm	FCT1	FCT1-tm
MSE-CP	0.024	0.004	0.014	0.009	0.022	0.004	0.008
MSES-reg	0.600	0.620	0.574	0.831	1.020	0.784	0.875
(b.2)10 (17%) noises	EMN	RSE	EM-Tk	EMT1	EMT1-tm	FCT1	FCT1-tm
MSE-CP	201.574	0.535	208.182	0.573	1.245	0.277	0.143
MSES-reg	14.174	2.939	9.714	0.978	1.745	0.771	1.215
(b.3) leverage outliers (10, 15)	EMN	RSE	EM-Tk	EMT1	EMT1-tm	FCT1	FCT1-tm
MSE-CP	335.479	x	2.428	477.948	0.024	347.620	0.004
MSES-reg	7451.722	x	1.518	9694.875	0.986	7884.283	0.853
(b.4) errors~t (2)	EMN	RSE	EM-Tk	EMT1	EMT1-tm	FCT1	FCT1-tm
MSE-CP	0.038	0.002	0.041	0.017	0.019	0.013	0.011
MSES-reg	1346.945	1.742	2.834	1.640	1.825	1.072	1.218
Model c: 2 CPs Model
(c.1) 0 outlierss	EMN	RSE	EM-Tk	EMT1	EMT1-tm	FCT1	FCT1-tm
MSE-CP1	0.223	0.170	0.490	0.490	0.228	0.271	0.457
MSE-CP2	5.745	0.041	6.142	6.142	5.361	3.218	5.194
MSES-reg	4.798	x	6.019	6.019	8.255	7.580	8.587
(c.2) 10 (12%) noises	EMN	RSE	EM-Tk	EMT1	EMT1-tm	FCT1	FCT1-tm
MSE-CP1	149.402	210.211	208.989	0.355	0.289	0.347	0.464
MSE-CP2	153.532	871.555	69.751	3.830	5.782	6.503	5.972
MSES-reg	7.969	x	7.640	7.809	8.444	5.359	5.901
(c.3) leverage outliers (10, 10)	EMN	RSE	EM-Tk	EMT1	EMT1-tm	FCT1	FCT1-tm
MSE-CP1	1.594	x	268.806	11.273	3.858	3.978	4.012
MSE-CP2	306.429	x	768.323	318.802	13.971	318.088	13.400
MSES-reg	7902.387	x	x	8939.276	9.410	8412.589	8.816
(c.4) errors~t (2)	EMN	RSE	EM-Tk	EMT1	EMT1-tm	FCT1	FCT1-tm
MSE-CP1	3.655	x	2.269	2.969	2.194	0.302	0.472
MSE-CP2	1.137	x	1.037	0.242	0.890	1.603	1.033
MSES-reg	67.314	x	x	4.727	4.914	3.704	4.486

x denotes that MSE is extremely large.

reports the results.

Table 2 shows that EMT1 and FCT1 perform better or at least comparable to the other methods in all cases of all three models, except the case of data contaminated by leverage outliers. However, for the cases with high leverage outliers, the trimmed versions, EMT1-tm and FCT1-tm, work well. Particularly, FCT1 usually performs better than EMT1 because the influence of outliers can be greatly reduced by moderately increasing the power of the fuzzy index m used in FCT1. More findings are stated as follows:

• For datasets including no atypical observations, all EM-type and FCT methods (including EMN, EM-Tk, EM-T, EMT1, EMT-tm, FCT1 and FCT1-tm) work well for all three models, with FCT1 performing better than the others for less biased CP estimates and less MSES-reg with regression coefficient estimates.
• RSE works well for the 1 CP continuous model (model b). However, RSE is not workable in cases of models with discontinuity (model a) or multiple CPs (model c). However, these situations frequently occur in practice.
• For data with background noises such as cases a.2, b.2 and c.2, only the proposed t-based approach works generally well. Particularly, FCT1 provides more precise estimates.
• For datasets including extremely high leverage outliers such as cases a.3, b.3 and c.3, all methods cannot work well, except EMT1-tm and FCT1-tm. In such situations, both trimmed versions provide effective estimates of CPs and regression coefficients for each model, and FCT1-tm performs better for producing less biased estimates on the whole.
• From extensive simulations, the iteration time respectively taken by EMT1 and FCT1 in all cases is always less than other EM-based methods. This evidence indicates that the proposed method is most computationally economic. Especially, it is even more timesaving than the normality-based EMN.

Next, we show that the proposed algorithms are more tolerant of extreme outliers in comparison with the existing methods based on model b in Figure 3. Four cases of extreme contaminations occurring at different locations are considered, with a simulated sample shown in Figure 4.

Cases 1 and 4: 5 (9.1%) identical outliers occur at (0, 7) and (10, 12), respectively. The leverage of outliers in these two cases is high.

Cases 2 and 3: 8 (13.8%) identical outliers occur at (3, 0) and (8, 4), respectively. The leverage of outliers in these two cases is moderate.

Note that in all four cases, the distance of the observed y-value from the predicted y- value is larger than 6 deviations, and the percentage of extreme outliers is more than 9.1% in each case. The estimation results are shown in

Table 3. Comparing the robustness of 7 methods to extreme outliers using model b under four situations.

Case	5 (9.1%) Outliers, CP = 30		6 (10.7%) Outliers, CP = 30		8 (13.8%) Outliers, CP = 33		8 (13.8%) Outliers, CP = 25		5 (9.1%) Outliers, CP = 25		6 (10.7%) Outliers, CP = 25
	MSECP 1	MSEreg 1	MSE-CP	MSEreg	MSECP	MSEreg	MSE-CP	MSEreg	MSECP	MSEreg	MSECP	MSEreg
EMN	25.19	13.15	32.16	16.40	36.84	5.74	14.47	8.42	25.9	85.9	32.0	113
RSE	1.35	x	2.30	x	1.42	0.46	0.07	0.41	1.27	x	2.19	x
EM-Tk	33.44	13.53	46.31	18.82	44.85	7.34	27.11	22.77	34.6	96.0	46.1	146
EMT1	0.01	1.02	3.32	4.40	4.14	1.74	4.10	34.40	0.06	0.96	3.78	30.1
FCT1	0.04	0.78	1.93	8.64	0.03	1.07	0.00	0.59	0.02	0.72	2.40	96.5
EMT1-tm	0.11	1.59	16.38	13.88	1.79	1.78	1.83	22.05	0.16	2.37	17.3	140
FCT1-tm	0.04	1.02	5.22	13.32	0.05	1.43	0.01	3.25	0.01	2.02	5.59	143

¹: MSECP and MSESreg are the MSE of the estimates of CP and regression parameters given by Equations (21) and (30), respectively.

.

Similar to previous comparisons, Table 3 shows that FCT1 performs best in all four cases of extreme contaminations. Particularly, FCT1 and FCT1-tm are much more resistant against radical outliers than other methods for providing estimates as precise as those obtained from clean data. Specifically, in cases 1 and 4, where the drastic outliers (0, 7) and (10, 12) have rather high leverages and their responses are distant from the predicted values for 12 standard deviations, FCT1 methods are still hardly affected by outliers and produce adequate estimates even if extreme outliers account for almost 10% of the whole dataset. Moreover, as explained earlier, FCT1 is more robust than EMT1 in that the fuzzy partitioning and the fuzziness index, m, are helpful for reducing the influence of outliers by giving tiny weights with a large m to the exceptional outliers. Table 3 demonstrates again that the normal-based method, EMN, cannot work well in the presence of extreme outliers. Additionally, while RSE is quite robust to outliers with moderate leverages, it cannot provide satisfied regression estimates for data involving drastic outliers with fairly high leverages.

In summary, the proposed FCT1 method is very robust to outliers, even though the dataset includes extreme outliers with the observed responses far away from the predicted values, by more than 6 standard deviations. Furthermore, FCT1 can still produce sufficient estimates for datasets in the presence of radical outliers with quite high leverages. More importantly, FCT1 is tolerant of the extreme outliers, which account for about 10% of the total data in all four cases of the experiment. In practice, outliers far away from the predicted regression line, by more than 6 standard deviations, are unusual and very likely less than 10% of the whole dataset. Thus, the proposed FCT1 is attainable in real applications, generally.

4.2. Real Data Applications

Example 1. We applied the proposed methods to a dataset involving 107 terrestrial animals gathered by Garland [45] for analyzing the relationship between the maximal running speed (MARS) with the body size. It is understandable that animals of different sizes can reach different maximal running speeds (MARS). McMahon [46] commented that animals are built for elastic similarity and the MARS of animals is proportional to the fourth root of body mass, (mass)^1/4, among elastically similar animals. The plot in Figure 5 shows that a slope change occurs in the relationship between (MARS)^1/4 and (mass)^1/4. Additionally, several exceptionally low points appear near the place where the regression slope converts. It is appreciable that these suspicious points very likely pull down the regression line and hence have a substantial influence on the regression slope.

We respectively apply EMN, EMT and FCT to the MARS data by setting DF = 1. The fitting results in Figure 5 show that EMT produces a much shorter CP at 25 and the line on the right hand side becomes flatter due to the down pull exerted by those extremely low points (see Figure 5, the blue fitted line by EMT and ${\widehat{\beta }}_{21}^{\mathrm{EMT}}=-0.0456$, flatter than those by EMN and FCT1, ${\widehat{\beta }}_{21}^{\mathrm{EMN}}=-0.0892$ and ${\widehat{\beta }}_{21}^{\mathrm{FCT}}=-0.0820$). The CPs produced by EMR (${\widehat{\tau }}^{\mathrm{EMN}}=46$) and FCT (${\widehat{\tau }}^{\mathrm{FCT}}=41$) are very close, but the slope of the left line (${\widehat{\beta }}_{11}^{\mathrm{EMN}}=0.4973$) obtained by EMN is much lower than that by FCT1 (${\widehat{\beta }}_{11}^{\mathrm{FCT}}=0.9118$). This evidence indicates that FCT1 is much more resistant against those extraordinarily low points than EMN. Based on previous simulation results showing that FCT1 usually provides less biased estimates than other models, we believe that the estimated model provided by FCT1 is reasonable and best suited to MARS data. The MARS data can be obtained from [45].

Example 2. We analyzed a bedload transport dataset (downloaded from https://doi.org/10.2737/RDS-2007-0004 (accessed on 26 May 2017)) to study the relationship between the discharge rate (m³/s) and the rate of bedload transport (kg/s). Bedload transport measures the transportation of particles in flowing fluids along a bed. The bedload transportation is often described as occurring in two phases, starting from a relatively slow stage (Phase I) to a faster stage (Phase II). The transition point is the main interest of researchers in studying bedload transportation. Ryan and Porth [47] had fitted the dataset with a piecewise regression model, and they found it difficult because few data were collected at higher flows. The scatter plots of data in Figure 6 show that the transportation rate tends to increase with the water discharge faster after the discharge rate goes beyond 1.5 m³/s. Furthermore, the two points on the upper right having extraordinarily high transportation rates are leverage points.

We implemented EMN, EMT1, EMT1-tm, FCT1 and FCT1-tm with DF = 1 and trimming percentage $\alpha =0.05$ to the data. Figure 6 shows that EMN and EMT1 are heavily affected by two high leverage points and both give an identical CP at 74 (discharge rate, DR = 1.81354). This is irrational for only two points included in Phase II. By contrast, the other three methods are much more robust, resulting in the same CP at 62 (DR = 1.5786) and much lower slope estimates for the second line, especially FCT1 and FCT1-tm. Particularly, the slope of the second line, ${\widehat{\beta }}_{21}^{\mathrm{FCT}1-\mathrm{tm}}=0.0282$, obtained by FCT1-tm is much smaller than those obtained by other methods. These facts illustrate that FCT1 and FCT1-tm are more resistant against unusual observations; especially, FCT1-tm is capable of withstanding high leverage outliers. Our resulting CP estimate, 62, looks reasonable and it is close to the finding of Zhang and Li [48]. As for deriving more accurate estimates for Phase II, just as suggested by Ryan and Porth [47], more data at faster flows are required.

Example 3. We applied the proposed algorithms to the Dow Jones Index closing price ranged from 22 October 2008 to 22 October 2009 (downloaded from https://finance.yahoo.com/quote/%5EDJI/history?ltr=1 (accessed on 20 July 2021)). Due to the financial crisis of 2008 with a bankruptcy of Lehman Brothers, U.S. stock market price series suffered a huge fluctuation from late 2008 to 2009. Thus, the U.S. stock market price series during this period is worthy of investigations to illustrate variance changes in real situations. Adopting an approach common in finance, we consider the stock returns, $R_t=\left(P_{t+1}-{\mathrm{P}}_t\right)/P_t$, where $P_t$ is the closing price on day t. Since the values of $\left\{R_t\right\}$ are too small, we applied our methods to the series $\left\{R_t\right\}$, $R_t^e=100\exp \left(R_t\right)$. The descriptive statistics of $\left\{R_t^e\right\}$ in

Table 4. Descriptive statistics of $\left\{R_t^e\right\}$.

Sample Size	Mean	SD	Minmum	Median	Maximum	Skewness	Kurtosis
252	101.11	2.1451	92.59	100.06	111.49	0.573	7.074

and the box plots of the whole data and the 4 sub-segments derived by EMN in Figure 7 reveal that the series $\left\{R_t^e\right\}$ follows a fat-tailed distribution and includes several extreme outliers. Therefore, t-regression models are better-suited to the series $\left\{R_t^e\right\}$.

Lin et al. [35] studied the same data and concluded three CPs at 35, 133 and 182. We set c = 4 and applied EMN, EMT1 and FCT1 to $\left\{R_t^e\right\}$, separately. EMN produces 3 CPs at 32, 122 and 182, which are close to those of Lin et al. [35]. However, the box plots in Figure 7b show that the last three segments obtained by EMN contain extreme outliers. Hence, the CPs estimated by EMN are suspicious due to its sensitivity to outliers and heavy-tails. Moreover, the estimated CPs of Lin et al. are obtained by the binary hierarchic segmentation, which does not generally provide the true optimum (cf. [11,49]). On the other hand, EMT1 and FCT1 produced 3 CPs at 21, 32 and 108 and 21, 32 and 151, in which the second segment contains only 11 observations and the difference of variations between the first two segmentations is not statistically significant. Furthermore, the scree test for choosing the number of CPs (see the discussion in Section 5) suggests that the optimal number of segments is 3. The box plots in Figure 8 show that several outliers appear in the last two segments. According to previous experiments, FCT1 is powerful in resisting atypical observations and it is more robust than EMT1. Thus, the CP estimates, 33 and 151, provided by FCT1 should be the optimal times of variance changes in the return series.

5. Conclusions and Discussion

5.1. Discussion

Computation cost is usually a concern in CP detection methods. The capacity of a robust method in resisting outliers for different sizes of datasets is important. For most statistical methods, it is harder to achieve robustness with a small dataset because estimations are easily affected by anomalous observations for small amounts of data. In order to show the high robustness of the proposed method even for small samples, we used datasets consisting of tens or hundreds of data points in the experiments. When fitting a regression model with c − 1 CPs to a dataset of size n by EMT or FCT, we consider $C_{c-1}^{n-1}$ potential collections of CPs, simultaneously. Therefore, the computational efficiency can be impeded as the size of the dataset or the number of CPs increases. In fact, except for the sample size and the number of CPs, other factors may influence the computational cost, such as the model variations, the model structures, such as continuity or discontinuity at CPs, and the discrepancies of models between two contiguous segments. According to our simulations executed on a computer of an Intel Core i7-7700HQ CPU with 2.80 GHz and 8 Gb of Ram, the proposed methods are suitable for datasets of sizes no more than 2000, 1000 and 500 for locating one, two and three CPs, respectively. For large datasets, a possible approach is to cut the sample size to the favorable size and then derive preliminary estimates by using the reduced data. One possible way for reducing the data size is to remove the data of even rank in the ordered dataset arranged in the increasing order of the partitioning variable, $x_d^{\ast }$, then repeatedly reduce the data until the favorable size is reached. After deriving the rudimentary estimates, we can further estimate CPs and regression parameters by employing EMT to the nearby data of the preceding CPs again.

The determination of the number of CPs is important in CP detection. However, it is difficult when lacking likelihood-based tests for the number of CPs because the regularity conditions fail and there is little knowledge about the asymptotic distribution of the test statistics even for Gaussian cases. Most CP studies usually adopted some information criteria or informal tests for deciding the number of segments. Here, we employed the idea of Hawkins’s scree test [49] to investigate the progressive changes in the residual sum of squared errors ($RSS(c-1)-RSS(c)$, c is the number of segments fitted to the data), with the changes in the MSE described as:

$$\mathrm{MSE}(c)=\frac{\mathrm{RSS}(c)}{[n-c(p+1)]}=\frac{{\displaystyle \sum {(y_i-{\widehat{y}}_i)}^2}}{[n-c(p+1)]},$$

and the changes in the maximized log-likelihood multiplied by −2 (−2lnL) (i.e., $F(c-1)-F(c)$, $F(c)$ is −2 multiplying the optimized log-likelihood obtained from a model of c pieces fitted to the data). Additionally, Ciuperca’s [50] generalized Bayesian Information Criterion (BIC) was also utilized.

The suggested indexes were employed to the four models, shown in Figure 9, with the data contaminated by background noises, which includs outliers with a quite high leverage. For data contaminated by high leverage outliers, the trimmed version with $\alpha =0.05$ was used and the results were similar to those of cases including no outliers. Hence, we do not present the results of cases with high leverage outliers here. We report the results of both EMT1 and FCT in

Table 5. Deciding the number of segments for the models in Figure 9.

(i) Using EMT1
try	0 CP c = 1, 15 noises				1 CP c = 2, 15 noises				2 CPs c = 3, 10 noises				3 CPs c = 4, 10 noises
c	RSS 1	–2lnL 2	MSE 3	BIC	RSS	–2lnL	MSE	BIC	RSS	–2lnL	MSE	BIC	RSS	–2lnL	MSE	BIC
1	x 4	x	x	115	x	x	x	92	x	x	x	−5	x	x	x	90
2	−9	−59	−1	133	98	997	2	79	18	37	0.13	3	190	430	3.88	33
3	* 5	*	−891	467	11	−172	0	108	8	−220	0.37	11	35	−50	0.72	30
4	0	1	−43	517	−3	−217	0	159	3	17	−0.02	62	4	−39	0.04	74
(ii) Using FCMT1
try	0 CP c = 1, 15 noises				1 CP c = 2, 15 noises				2 CPs c = 3, 10 noises				3 CPs c = 4, 10 noises
c	RSS 1	–2lnL 2	MSE 3	BIC	RSS	–2lnL	MSE	BIC	RSS	–2lnL	MSE	BIC	RSS	–2lnL	MSE	BIC
1	x	x	x	115	x	x	x	88	x	x	x	−10	x	x	x	90
2	−8	−59	−0.52	133	88	311	1.69	77	7	290	0.08	0	185	1885	3.78	36
3	−7	*	−0.54	167	0	−263	−0.17	111	14	39	0.26	15	36	−45	0.73	36
4	8	810	−0.24	216	8	−630	0.01	158	1	51	−0.03	67	7	29	0.12	75

¹. RSS: the change of RSS, i.e., RSS(c − 1) − RSS(c). ². –2lnL: the change of −2 multiplying maximized log-likelihood, $F(c-1)-F(c)$. ³. MSE: the change of MSE, i.e., MSE(c − 1) − MSE(c). ⁴. Optimal c denoted by fluorescent yellow color. ⁵. Extremely small number (unreasonable results due to using a wrong c).

. For models including zero or one single CP, all four suggested indexes (RSS, MSE, −2lnL and BIC) provide a correct c for each respective case. For the models of multiple CPs, the BIC and −2lnL index fails to obtain the correct c. The RSS and MSE indexes work well in all circumstances. It can be easily seen from Table 5 that both the indexes RSS and MSE decreased progressively for c less than the true value, and then the decrease becomes less again once c goes beyond the true value. These two indexes provide a clear indication of the optimal c. Therefore, the two suggested indexes, RSS and MSE, are useful for determining the number of segments. However, in some applications, the cut-point may not be so clearly indicated by these indices. In such cirsumstances, one may fit the data with several possible c’s based on RSS and MSE and then choose the best fitted model on the basis of some criteria in regression analysis. Although there is no solid inferential base for the suggested method, it may suffice as a rough and practical tool.

On the other hand, the penalized likelihood approach has been utilized in recent studies, such as [7,8], to infer both the number of CPs and their locations. For large datasets, one may consider the penalized approaches with a penalty added into the objective likelihood to estimate the number and the locations of CPs simultaneously. However, the selection of a proper penalty is not easy because the penalty can have a substantial impact on the estimations for the number of CPs [28]. One may refer to Haynes et al. [51] for information on workable penalties.

In using the trimmed EMT, a small value of $\alpha$ was selected to remove leverage points where the leverage scores rank in the top $\alpha \times 100\%$. Apparently, the choice of $\alpha$ is connected with the results of EMT1-tm and FCT1-tm. A formal statistical test is required for objectively judging whether a datum is a high leverage point based on a cut-point, which is the critical point resulted from a statistical test under a given significance level. Accordingly, high leverage points can be deleted adaptively but not fixed at a subjective value. Moreover, high leverage points may present small residuals and hence provide important information for model fitting. Therefore, more research on incorporating the information of high leverage points with small residuals is also needed to judge high leverage points better. One possible method is to borrow the idea of monitoring introduced by Cerioli et al. [52]. First, the trimmed version with a small $\alpha$ is employed to derive robust estimates of CPs and regression parameters. Then, the technique of monitoring in [52] may be considered for judging the trimmed points to make the most of the dataset. The test statistic for outliers in CP regressions would be better based on the difference in the estimates of CPs and model parameters between different samples. Further research is needed, and it will be covered in our future work.

5.2. Conclusions

This paper proposed a new robust method for CP regressions using t-distributions. The t-distribution provides a longer tailed substitute to the normal distribution and has an additional parameter called DF, which is critical for tuning robustness. These favorable properties make the t CP regression model more tolerant of atypical observations. We considered CPs as random variables and transfered them into latent class variables so that we could implement the EM algorithm to estimate the CPs and regression parameters simultaneously. Moreover, we rewrote the t-distribution as a scale mixture of normal distributions to simplify the computation of t-log-likelihood. We further extended the latent class model to a fuzzy class model. Incorporating fuzzy clustering methods, we created the FCT algorithm to make more robust estimations for CP regressions by FCT than by EMT. The proposed EMT and FCT are computationally easy and efficient compared to the other existing EM-type methods. The proposed method has no problem with the non-differentiability and requires no restrictions on the models, such as the continuity between two adjoining segments needed, only a single CP allowed, or an unchanged regression variance demanded. Some of these restraints are often required in existing methods. To advance the ability of the proposed methods in resisting high leverage outliers, we proposed the modified versions, called EMT-tm and FCT-tm, by first trimming high leverage points moderately and then employing the proposed method to the reduced data. As for the decision of the degrees of freedom, r, the EMT1 and FCT1 (DF r = 1) worked well in all experiments, no matter whether the data were contaminated or not. The analysis of robust properties with the proposed t-based methods also showed that EMT1 and FCT1 were most tolerant of atypical observations. In managing CP regression problems, we suggest respectively fitting data with EMN, EMTs and FCTs using values such as r = 1, 5, 10 and 15. If the estimations of the three methods are close, one may consider that the data are normally distributed, having no abnormal observations, so that EMN can be used for obtaining less varied estimators. Otherwise, EMT1 or FCT1 should be empolyed. In practice, FCT1 can be given priority for its better ability of withstanding atypical observations. For the concern about the trimmed proportions of data, $\alpha$, normally, outliers with extremely high leverage are seldom more than 5%, thus $\alpha$ can be set as 0.05 at the start and increased in case a higher value is needed (cf. [28,47]). Extensive experiments with numerical and real examples showed the effectiveness and the practicability of EMT1 and FCT1. In particular, only the trimmed versions were workable for data containing extremely high leverage outliers, such as the real data of bedload transportations. This evidence shows the effectiveness and the need of the trimmed versions for real applications. Overall, the proposed EMT and FCT approaches are robust to atypical observations and heavy-tailed distributions. Especially, FCT1 generally works well and performs better than EMT1 for providing less biased estimates of CPs and model parameters.