New Statistical Residuals for Regression Models in the Exponential Family: Characterization, Simulation, Computation, and Applications

Abstract

Residuals are essential in regression analysis for evaluating model adequacy, validating assumptions, and detecting outliers or influential data. While traditional residuals perform well in linear regression, they face limitations in exponential family models, such as those based on the binomial and Poisson distributions, due to heteroscedasticity and dependence among observations. This article introduces a novel standardized combined residual for linear and nonlinear regression models within the exponential family. By integrating information from both the mean and dispersion sub-models, the new residual provides a unified diagnostic tool that enhances computational efficiency and eliminates the need for projection matrices. Simulation studies and real-world applications demonstrate its advantages in efficiency and interpretability over traditional residuals.

1. Introduction

Evaluating model adequacy in regression analysis is essential as it directly impacts the validity of the inferences made. Residuals, traditionally stated as the difference between observed and predicted values, serve as key diagnostic tools, offering insights into model assumptions, outliers, and overall specification [1,2,3,4,5,6]. Residual analysis can uncover model flaws that might provide misleading conclusions.

Linear regression models, which assume normally distributed errors, are commonly used due to their simplicity. However, many real phenomena do not meet this assumption. Generalized linear models (GLMs) extend linear models by accommodating a wider class of probability laws from the exponential family, including binomial, gamma, and Poisson distributions [7,8,9]. Nevertheless, this extension complicates the residual analysis. The assumptions of constant variance and independence of errors are more nuanced in GLM, requiring more sophisticated residuals tailored to the specific distribution [10,11,12,13,14]. For example, in binary or count models, the variance depends on the mean, complicating interpretation of the residuals [15,16,17]. Recent studies have applied GLM to diverse contexts, such as economic data [18], quantile regression [19], and environmental data through Birnbaum-Saunders regression models [20,21,22].

Residuals are crucial in various analytical techniques, including variable selection, screening, and machine learning [23]. They help to identify predictor contributions by highlighting patterns, guiding feature selection in high-dimensional data, and improving model performance in methods like gradient boosting [24,25,26]. Moreover, diagnostic methods, such as those discussed in [27,28,29], expand on how residuals can be used for goodness-of-fit tests and model adequacy checks, further demonstrating their versatility. The statistic named predicted residual error sum of squares, based on residuals, is also used to assess predictive performance through cross-validation [30].

Despite their importance, traditional residuals, like studentized or deviance, have limitations when applied to GLM. Advanced methods are needed to address these limitations, especially in handling heteroscedasticity and dependencies [31,32,33]. Recent studies have explored alternative approaches, such as skew-normal inflated models [34] and semi-parametric additive modeling with Birnbaum-Saunders distributions [35]. These approaches contribute to the development of more accurate diagnostic tools in complex settings.

Standardizing residuals in GLM often involve projection matrices, derived from the likelihood maximization process [1,5]. Nonetheless, these matrices can be computationally demanding, particularly for large datasets, decreasing their practical use [16,17,36]. Moreover, projection matrices may not fully capture data variability when changes in the dispersion exist, complicating diagnostics [17,34,37]. Recent research has addressed the above-mentioned limitations and proposed methods to improve residual analysis in GLM. Alternative residuals tailored to specific distributions within the exponential family were investigated in [37,38]. However, these proposals often involve complex calculations or are restricted to specific data types, reducing their broader applicability [39].

Modern approaches, presented in studied for semi-parametric additive models with complex data structures [35] as well as continuous Bernoulli [40] and quasi-Cauchy [41] quantile regression models, underscore the need for innovative diagnostic methods and demonstrate the challenges in dealing with data supported within specific intervals, emphasizing the restrictions of traditional approaches. While these studies represent advancements, important gaps in residual analysis for GLMs remain. Proposed methods, such as skew-normal inflated [34] and semi-parametric additive [35] models, focus on either mean or dispersion models without considering both simultaneously. Moreover, several methods rely on computationally intensive procedures, such as projection matrices, which reduces their applicability in large datasets or models with varying dispersion.

In response to the earlier-mentioned limitations, the standardized combined residual integrates information from both mean and dispersion models. Unlike methods that focus on a single model, this integration offers a more comprehensive approach to residual analysis, improving both computational efficiency and interpretability. Importantly, the standardized combined residuals avoid the need for projection matrices, so reducing computational cost and complexity, particularly for large datasets. Thus, one can propose a methodology that enhances the detection of model inadequacies, particularly in scenarios involving heteroscedasticity or interdependence between observations. Therefore, the present study proposes a novel standardized combined residual that provides both theoretical advancements and practical improvements over existing residuals, including a more efficient computational behavior and enhanced diagnostic capabilities.

The main objectives of this study are: (i) to develop a unified framework for residual analysis in GLMs that integrates both mean and dispersion sub-models, (ii) to assess the computational efficiency of the proposed residual in comparison with traditional methods, and (iii) to evaluate the performance of the proposed residual through extensive simulations and applications to real-world datasets. To achieve these objectives, we employ the Fisher scoring iterative method for parameter estimation in GLMs, which allows the simultaneous modeling of both mean and dispersion effects. The new residual is specifically designed to accommodate this without the need for projection matrices, as mentioned, offering both computational efficiency and clearer interpretation.

The structure of the article is as follows. Section 2 covers the framework for modeling and parameter estimation in the exponential family, emphasizing the relationship between mean and variance. In Section 3, nonlinear models within the exponential family are discussed, exploring both general structure and specific cases. In Section 4, we introduce standardized combined residuals, which integrate information from both mean and dispersion sub-models. The proposed residual effectiveness is demonstrated through simulations and two empirical applications, highlighting its utility in practical contexts. Specifically, Section 5 provides methods for evaluating model fit using simulated envelopes, offering a robust approach to residual analysis. In Section 6, empirical applications of the proposed residual to real datasets are presented, demonstrating its practical utility, while Section 7 provides concluding remarks and future research directions.

Additionally, this article includes appendices that provide detailed mathematical derivations (Appendix A); a description of the Fisher scoring iterative method (Appendix B); a complete example in R code [42] to show the implementation of the proposed methods (Appendix C and Appendix D) containing the results of extensive Monte Carlo simulations, further elucidating the performance of the proposed methods.

2. Modeling and Estimation in Exponential Family Distributions

This section presents the framework for modeling and parameter estimation within the exponential family of distributions, emphasizing the critical relationship between the mean and variance. Understanding this relationship is essential for constructing regression models that accurately capture the variability in real-world data, especially when the variability is not constant but depends on the magnitude of the mean.

2.1. Simultaneous Modeling of the Mean and Dispersion in the Exponential Family

The mean and variance are fundamental characteristics that describe the behavior of a random variable within a distribution, assuming that these moments exist. In many distributions belonging to the exponential family, a specific functional relationship exists between the mean and variance. Accurately modeling this relationship is crucial when the variance changes with the mean, which is often the case in practical applications. Such modeling ensures that the variability observed in the data is properly accounted for, particularly in contexts where the dispersion (or spread) of the data is not constant but varies proportionally with the mean. Let $Y_1,\dots ,Y_n$ be independent random variables, where each $Y_i$, for $i\in \{1,\dots ,n\}$, has a probability density function that can be expressed in the canonical form of the exponential family [5,43,44] as

$$f(y_i;{\theta }_i,{\varphi }_i)=exp\left({\varphi }_i\left(y_i{\theta }_i-b\left({\theta }_i\right)\right)+c(y_i;{\varphi }_i)\right),i\in \{1,\dots ,n\},$$

(1)

with ${\theta }_i$ being the canonical parameter, which uniquely determines the distribution of $Y_i$ within the exponential family; ${\varphi }_i^{-1}>0$ being the dispersion parameter, which scales the variance independently of the mean, capturing additional variability in the data; $b\left({\theta }_i\right)$ being the cumulant function, crucial for determining the mean and variance of the distribution, as well as for deriving higher-order cumulants and moments; and $c(y_i;{\varphi }_i)$ being the normalizing function, ensuring that the probability density function integrates to one.

The mean and variance of the distribution with density described in (1) are given by

$$\mathrm{E}\left(Y_i\right)={\mu }_i=b^{\prime }\left({\theta }_i\right),\mathrm{Var}\left(Y_i\right)={\varphi }_i^{-1}{b}^{″}\left({\theta }_i\right),i\in \{1,\dots ,n\},$$

(2)

where $b^{″}\left({\theta }_i\right)$ is the variance function, denoted as $\mathrm{V}\left({\mu }_i\right)$, which depends only on the mean ${\mu }_i$. The function $\mathrm{V}\left({\mu }_i\right)$ describes the theoretical variance of $Y_i$ under the assumption of constant dispersion. However, the true variance observed in the data, $\mathrm{Var}\left(Y_i\right)$ namely, is influenced by the dispersion parameter ${\varphi }_i$. This parameter ${\varphi }_i$ allows the model to adjust for overdispersion (where the observed variance is greater than the theoretical variance) or underdispersion (where the observed variance is less than the theoretical variance), thereby providing a more accurate reflection of the data variability compared to the standard exponential family model. The flexibility of the exponential family of distributions in modeling is crucial in practical applications, as it enables the model to more accurately reflect the true variability present in the data, particularly in situations where the variability cannot be fully explained by the mean alone. Such flexibility is evident across various distributions within the exponential family, each with its own distinct relationship between the mean, variance, and dispersion parameters.

Table 1. Examples of distributions belonging to the exponential family with their indicated parameter.

Distribution	$\mathit{\theta }$	$\mathit{b}\left(\mathit{\theta }\right)$	$\mathit{\varphi }$	$\mathit{V}\left(\mathit{\mu }\right)$	$\mathit{c}(\mathit{y};\mathit{\varphi })$
Normal	$\mu$	${\theta }^2/2$	$1/{\sigma }^2$	1	$-(1/2)log(2\pi /\varphi )-\varphi y^2/2$
Gamma	$-1/\mu$	$-log(-\theta )$	$1/{\mathrm{CV}}^2$	${\mu }^2$	$(\varphi -1)log\left(y\right)+\varphi log\left(\varphi \right)-log\left(\mathsf{\Gamma }\right(\varphi \left)\right)$
Inverse Gaussian	$-1/2{\mu }^2$	$-\sqrt{-2\theta }$	$\varphi$	${\mu }^3$	$log(\varphi /\left(2\pi y^3\right))/2-\varphi /\left(2y\right)$
Poisson	$log\left(\mu \right)$	$exp\left(\theta \right)$	1	$\mu$	$-log(y!)$
Binomial	$log(\mu /(1-\mu \left)\right)$	$log(1+exp(\theta \left)\right)$	n	$\mu (1-\mu )$	$log\left(\left({\displaystyle \genfrac{}{}{0pt}{}{\varphi }{\varphi y/n}}\right)\right)$

CV is the variation coefficient of the gamma distribution and $\mathsf{\Gamma }$ is the standard gamma function.

summarizes some common distributions within the exponential family. In this table, $\mathrm{CV}$ denotes the coefficient of variation, defined as $\mathrm{CV}=\sqrt{\mathrm{Var}\left(Y_i\right)}/\mathrm{E}\left(Y_i\right)$, and $\mathsf{\Gamma }$ represents the Euler gamma function, which generalizes the factorial function to real and complex numbers [45,46]. Understanding these parameters is crucial for accurately modeling the dispersion and shape characteristics of different distributions, which play an important role in the analysis of real-world data.

The theoretical framework outlined above provides a foundation for understanding how the mean and variance interact within the exponential family of distributions. However, applying this framework in practice requires defining a regression structure that incorporates both the mean and dispersion parameters effectively.

2.2. Regression Structure and Parameter Estimation in Exponential Family Models

Let ${\mathit{x}}_i={(x_{i1},\dots ,x_{ik})}^{\top }$ represent values of the set of k covariates associated with the mean response ${\mu }_i$ for observation i, and ${\mathit{z}}_i={(z_{i1},\dots ,z_{iq})}^{\top }$ be the values of the set of q covariates related to the dispersion parameter ${\varphi }_i$ for the same observation, where $i\in \{1,\dots ,n\}$. These values of covariates capture the effect of different predictors on the mean and dispersion, respectively. Importantly, ${\mathit{x}}_i$ and ${\mathit{z}}_i$ can, but do not have to, be the same, depending on the model specification. This modeling allows for different sets of covariates to influence the mean and the dispersion independently, which is an advantage in modeling with the exponential family.

We assume that the mean ${\mu }_i$ and the dispersion ${\varphi }_i$, as defined in (2), satisfy the relationships stated as

$$\begin{array}{cc}\hfill g\left({\mu }_i\right)& ={\eta }_{1i}={\mathit{x}}_i^{\top }\mathit{\beta }=\sum _{j=1}^k{x}_{ij}{\beta }_j,\hfill \end{array}$$

(3)

$$\begin{array}{cc}\hfill h\left({\varphi }_i\right)& ={\eta }_{2i}={\mathit{z}}_i^{\top }\mathit{\gamma }=\sum _{j=1}^q{z}_{ij}{\gamma }_j,i\in \{1,\dots ,n\},\hfill \end{array}$$

(4)

where $\mathit{\beta }\in {\mathbb{R}}^k$ and $\mathit{\gamma }\in {\mathbb{R}}^q$ are unknown parameter vectors.

The functions g presented in (3) and h in (4) are known as link functions, which are assumed to be monotonic, twice differentiable, and chosen based on the nature of the response variable as well as in the desired relationship between the predictors and the mean or dispersion. The linear predictors established as ${\eta }_{1i}$ in (3) and as ${\eta }_{2i}$ in (4), formed by linear combinations of the covariates with their respective coefficients, encapsulate the effects of the covariates on the mean and dispersion, respectively.

Let $\mathit{\theta }={({\mathit{\beta }}^{\top },{\mathit{\gamma }}^{\top })}^{\top }$ represent the full parameter vector of the model to be estimated, combining both the coefficients related to the mean and dispersion. The log-likelihood function for the model, which forms the basis for parameter estimation, is expressed as

$$\ell \left(\mathit{\theta }\right)=\sum _{i=1}^n{\ell }_i({\mu }_i;{\varphi }_i),i\in \{1,\dots ,n\},$$

(5)

where ${\ell }_i$ is the log-likelihood contribution of observation i formulated as

$${\ell }_i({\mu }_i;{\varphi }_i)={\varphi }_i\left(y_i{\theta }_i-b\left({\theta }_i\right)\right)+c(y_i;{\varphi }_i),i\in \{1,\dots ,n\},$$

with ${\theta }_i$, $b\left({\theta }_i\right)$, and $c(y_i;{\varphi }_i)$ being defined as in (1). Parameter estimation in GLM is typically performed using the method of maximum likelihood (ML), which involves solving the score equations derived by taking the partial derivatives of the log-likelihood function presented in (5) with respect to the parameters. The ML estimates $\widehat{\mathit{\beta }}$ and $\widehat{\mathit{\gamma }}$ are generated by solving the system of score equations given by

$${\mathit{U}}_{\mathit{\beta }}=0,{\mathit{U}}_{\mathit{\gamma }}=0.$$

If the dispersion parameter ${\varphi }_i$ is known and constant, the probability density function obtained in (1) defines the class of exponential family models with the canonical parameter ${\theta }_i$. However, when ${\varphi }_i$ is unknown, the expression established in (1) defines a valid exponential family model only if the function $c(y_i;{\varphi }_i)$ takes the specific form stated as

$$c(y_i;{\varphi }_i)=d\left({\varphi }_i\right)+{\varphi }_{i}a\left(y_i\right)+u\left(y_i\right),i\in \{1,\dots ,n\},$$

where $d\left({\varphi }_i\right)$, $a\left(y_i\right)$, and $u\left(y_i\right)$ are arbitrary functions that ensure the model adheres to the exponential family structure. The specific form of $c(y_i;{\varphi }_i)$ is crucial because it maintains the tractability of the likelihood function and the validity of the model within the exponential family. For further details, see [38].

In many cases, a closed-form solution is not feasible, necessitating the use of numerical optimization methods. A commonly employed approach to obtain the mentioned solution is the Fisher scoring iterative method, which is presented generically as

$$\begin{array}{cc}\hfill {\mathit{\beta }}^{(m+1)}& ={\mathit{\beta }}^{\left(m\right)}+{\left({\mathit{K}}_{\mathit{\beta }\mathit{\beta }}^{\left(m\right)}\right)}^{-1}{\mathit{U}}_{\mathit{\beta }}^{\left(m\right)},\hfill \end{array}$$

(6)

$$\begin{array}{cc}\hfill {\mathit{\gamma }}^{(m+1)}& ={\mathit{\gamma }}^{\left(m\right)}+{\left({\mathit{K}}_{\mathit{\gamma }\mathit{\gamma }}^{\left(m\right)}\right)}^{-1}{\mathit{U}}_{\mathit{\gamma }}^{\left(m\right)},\hfill \end{array}$$

(7)

where m denotes the iteration number; ${\mathit{\beta }}^{\left(m\right)}$ and ${\mathit{\gamma }}^{\left(m\right)}$ represent the parameter estimates at the mth iteration; ${\mathit{U}}_{\mathit{\beta }}^{\left(m\right)}$ and ${\mathit{U}}_{\mathit{\gamma }}^{\left(m\right)}$ are the score function components corresponding to $\mathit{\beta }$ and $\mathit{\gamma }$, respectively; whereas ${\mathit{K}}_{\mathit{\beta }\mathit{\beta }}^{\left(m\right)}$ and ${\mathit{K}}_{\mathit{\gamma }\mathit{\gamma }}^{\left(m\right)}$ are the Fisher information matrices at the mth iteration, which represent the curvature of the log-likelihood function stated in (5) with respect to the parameters. This curvature refers to the negative expected value of the second derivative (Hessian matrix) of the log-likelihood function with respect to the parameters. It captures how sensitive the likelihood function is to changes in the parameters. The Fisher information matrices measure the curvature by quantifying the rate at which the slope of the log-likelihood function changes with respect to the parameters. A higher curvature indicates that the log-likelihood function has a sharper peak around the parameter estimates, leading to more precise estimates.

By substituting the necessary quantities (see Appendix B), the iterative formulas established in (6) and (7) can be rewritten as a weighted least squares process presented as

$$\begin{array}{cc}\hfill {\mathit{\beta }}^{(m+1)}& ={\left({\mathit{X}}^{\top }{\mathsf{\Phi }}^{\left(m\right)}{\mathit{W}}^{\left(m\right)}\mathit{X}\right)}^{-1}{\mathit{X}}^{\top }{\mathsf{\Phi }}^{\left(m\right)}{\mathit{W}}^{\left(m\right)}{\mathit{u}}_1^{\left(m\right)},\hfill \\ \hfill {\mathit{\gamma }}^{(m+1)}& ={\left({\mathit{Z}}^{\top }{\mathit{P}}^{\left(m\right)}\mathit{Z}\right)}^{-1}{\mathit{Z}}^{\top }{\mathit{P}}^{\left(m\right)}{\mathit{u}}_2^{\left(m\right)},\hfill \end{array}$$

(8)

where

• $\mathit{X}$ and $\mathit{Z}$ are the design matrices corresponding to the values of covariates associated with the mean ${\mu }_i$ and dispersion ${\varphi }_i$, respectively;
• ${\mathsf{\Phi }}^{\left(m\right)}$ and ${\mathit{P}}^{\left(m\right)}$ are diagonal matrices, where ${\mathsf{\Phi }}^{\left(m\right)}=\mathrm{diag}\left\{{\varphi }_i^{\left(m\right)}\right\}$ and ${\mathit{P}}^{\left(m\right)}$ depends on the variance function related to the dispersion model, capturing the structure of the data variability at mth iteration;
• ${\mathit{W}}^{\left(m\right)}$ is a weight matrix that typically depends on the variance function $\mathrm{V}\left({\mu }^{\left(m\right)}\right)$ at mth iteration, adjusting for the variability of the response variable;
• ${\mathit{u}}_1^{\left(m\right)}={\mathit{\eta }}_1^{\left(m\right)}+{\mathit{W}}^{1/2\left(m\right)}{\mathit{V}}^{-1/2\left(m\right)}(\mathit{y}-{\mathit{\mu }}^{\left(m\right)})$, where ${\mathit{\eta }}_1^{\left(m\right)}$ is the linear predictor for the mean model, ${\mathit{V}}_{\gamma }=\mathrm{diag}\{-d^{″}\left({\varphi }_1\right),\dots ,-d^{″}\left({\varphi }_n\right)\}$, $\mathit{y}$ is the vector of observed responses, and ${\mathit{\mu }}^{\left(m\right)}$ is the vector of fitted values at mth iteration;
• ${\mathit{u}}_2^{\left(m\right)}={\mathit{\eta }}_2^{\left(m\right)}+{\mathit{V}}_{\gamma }^{-1\left(m\right)}{\mathit{H}}_{\gamma }^{\left(m\right)}(\mathit{t}-{\mathit{\mu }}^{*\left(m\right)})$, where ${\mathit{\eta }}_2^{\left(m\right)}$ is the linear predictor for the dispersion model, ${\mathit{H}}_{\gamma }=\mathrm{diag}\{h^{\prime }\left({\varphi }_1\right),\dots ,h^{\prime }\left({\varphi }_n\right)\}$, with $h^{\prime }\left({\varphi }_i\right)=\partial {\mathit{\eta }}_{2i}/\partial {\varphi }_i$$\mathit{t}$ being the response variable associated with the dispersion, and ${\mathit{\mu }}^{*\left(m\right)}$ is the fitted value for the dispersion at mth iteration.

Upon convergence of the iterative process stated in (8), we obtain the estimates given by

$$\begin{array}{cc}\hfill \widehat{\mathit{\beta }}& ={\left({\mathit{X}}^{\top }\widehat{\mathsf{\Phi }}\widehat{\mathit{W}}\mathit{X}\right)}^{-1}{\mathit{X}}^{\top }\widehat{\mathsf{\Phi }}\widehat{\mathit{W}}{\mathit{u}}_1,\hfill \end{array}$$

(9)

$$\begin{array}{cc}\hfill \widehat{\mathit{\gamma }}& ={\left({\mathit{Z}}^{\top }\widehat{\mathit{P}}\mathit{Z}\right)}^{-1}{\mathit{Z}}^{\top }\widehat{\mathit{P}}{\mathit{u}}_2,\hfill \end{array}$$

(10)

where ${\mathit{u}}_1={\widehat{\mathit{\eta }}}_1+{\widehat{\mathit{W}}}^{1/2}{\widehat{\mathit{V}}}^{-1/2}(\mathit{y}-\widehat{\mathit{\mu }})$ and ${\mathit{u}}_2={\widehat{\mathit{\eta }}}_2+{\widehat{\mathit{V}}}_{\gamma }^{-1}{\widehat{\mathit{H}}}_{\gamma }(\mathit{t}-{\widehat{\mathit{\mu }}}^{*})$. The expressions defined in (9) and (10) can be interpreted as ordinary least squares (OLS) estimators after applying appropriate weights and transformations. Specifically, to reach $\widehat{\mathit{\beta }}$, we regress the auxiliary variable ${\widehat{\mathsf{\Phi }}}^{1/2}{\widehat{\mathit{W}}}^{1/2}{\mathit{u}}_1$ on the covariate matrix ${\widehat{\mathsf{\Phi }}}^{1/2}{\widehat{\mathit{W}}}^{1/2}\mathit{X}$. Similarly, to get $\widehat{\mathit{\gamma }}$, we perform a regression of the auxiliary variable ${\widehat{\mathit{P}}}^{1/2}{\mathit{u}}_2$ on the covariate matrix ${\widehat{\mathit{P}}}^{1/2}\mathit{Z}$.

2.3. Residuals

The corresponding residuals from the above-mentioned weighted regressions are given by

$$r^{\mathit{\beta }}={\widehat{\mathsf{\Phi }}}^{1/2}{\widehat{\mathit{W}}}^{1/2}({\mathit{u}}_1-{\widehat{\mathit{\eta }}}_1),r^{\mathit{\gamma }}={\widehat{\mathit{P}}}^{1/2}({\mathit{u}}_2-{\widehat{\mathit{\eta }}}_2),r_i^{\mathit{\beta }}={\displaystyle \frac{y_i-{\widehat{\mu }}_i}{\sqrt{{\widehat{\varphi }}_i^{-1}{\widehat{V}}_i}}},r_i^{\mathit{\gamma }}={\displaystyle \frac{t_i-{\widehat{\mu }}_i^{*}}{\sqrt{-d^{″}\left({\widehat{\varphi }}_i\right)}}},i\in \{1,\dots ,n\},$$

where ${\widehat{V}}_i=\partial {\widehat{\mu }}_i/\partial {\widehat{\theta }}_i$. These residuals can be standardized using the projection matrix derived from the OLS solution, which normalizes them for better comparability across observations. Standardization is particularly important when dealing with heteroscedasticity, as it ensures that the residuals have consistent variability, making them more effective for model diagnostics. For each component, the projection matrices are formulated as

$$\begin{array}{cc}\hfill {\widehat{\mathit{S}}}^{\beta }& ={\widehat{\mathsf{\Phi }}}^{1/2}{\widehat{\mathit{W}}}^{1/2}\mathit{X}{\left({\mathit{X}}^{\top }\widehat{\mathsf{\Phi }}\widehat{\mathit{W}}\mathit{X}\right)}^{-1}{\mathit{X}}^{\top }{\widehat{\mathit{W}}}^{1/2}{\widehat{\mathsf{\Phi }}}^{1/2},\hfill \end{array}$$

(11)

$$\begin{array}{cc}\hfill {\widehat{\mathit{S}}}^{\gamma }& ={\widehat{\mathit{P}}}^{1/2}\mathit{Z}{\left({\mathit{Z}}^{\top }\widehat{\mathit{P}}\mathit{Z}\right)}^{-1}{\mathit{Z}}^{\top }{\widehat{\mathit{P}}}^{1/2}.\hfill \end{array}$$

(12)

Then, the standardized ordinary residuals are defined as

$$r_{{\mathrm{s}}_i}^{\mathit{\beta }}={\displaystyle \frac{y_i-{\widehat{\mu }}_i}{\sqrt{{\widehat{\varphi }}_i^{-1}{\widehat{V}}_i(1-{\widehat{s}}_{ii}^{\beta })}}},r_{{\mathrm{s}}_i}^{\mathit{\gamma }}={\displaystyle \frac{t_i-{\widehat{\mu }}_i^{*}}{\sqrt{-d^{″}\left({\widehat{\varphi }}_i\right)(1-{\widehat{s}}_{ii}^{\gamma })}}},i\in \{1,\dots ,n\},$$

where ${\widehat{s}}_{ii}^{\beta }$ and ${\widehat{s}}_{ii}^{\gamma }$ are the diagonal elements of the projection matrices ${\widehat{\mathit{S}}}^{\beta }$ and ${\widehat{\mathit{S}}}^{\gamma }$, respectively. The calculation of $r_{{\mathrm{s}}_i}^{\beta }$ depends on obtaining the projection matrix ${\widehat{\mathit{S}}}^{\beta }$. After assessing the residuals, another important statistic to consider in evaluating the model fit within the exponential family is the deviance, which is a key metric used for assessing how well the model fits the data. The deviance was first translated as deviation in [47].

The deviance function for the mean is established as the difference between the log-likelihood function of the saturated model (with n parameters) and the model of interest (with k parameters, where $k<n$), evaluated at the ML estimate $\widehat{\mathit{\beta }}$, and is given by

$$D^{*}(\mathit{y},\widehat{\mathit{\mu }})=2\sum _{i=1}^n{\varphi }_i\left(y_i({\tilde{\theta }}_i-{\widehat{\theta }}_i)-b\left({\tilde{\theta }}_i\right)+b\left({\widehat{\theta }}_i\right)\right),$$

where ${\tilde{\theta }}_i=\theta \left(y_i\right)$ and ${\widehat{\theta }}_i=\theta \left({\widehat{\mu }}_i\right)$ represent the ML estimates of $\theta$ for the saturated model (with n parameters) and the model under study (with $k<n$ parameters), respectively. Similarly, the deviance function for the dispersion can be defined as

$$D^{*}(\mathit{y},\widehat{\mathit{\varphi }},\widehat{\mathit{\mu }})=2\sum _{i=1}^n\left(t_i({\tilde{\varphi }}_i-{\widehat{\varphi }}_i)+(d\left({\tilde{\varphi }}_i\right)-d\left({\widehat{\varphi }}_i\right))\right),$$

where ${\tilde{\varphi }}_i$ is the solution to $d^{\prime }\left({\tilde{\varphi }}_i\right)=-t_i$. For further details on the deviance, see [1,5].

The deviance residuals are commonly used in GLM to assess the contribution of each observation to the overall model fit. The standardized deviance residuals for the mean and dispersion are given, respectively, by 

$$\begin{array}{cc}\hfill d_{\mu }({\widehat{\mu }}_i,{\widehat{\varphi }}_i)& ={\displaystyle \frac{d_{\mu }^{*}({\widehat{\mu }}_i,{\widehat{\varphi }}_i)}{\sqrt{1-{\widehat{s}}_{ii}^{\beta }}}},d_{\varphi }({\widehat{\mu }}_i,{\widehat{\varphi }}_i)={\displaystyle \frac{d_{\varphi }^{*}({\widehat{\mu }}_i,{\widehat{\varphi }}_i)}{\sqrt{1-{\widehat{s}}_{ii}^{\gamma }}}},i\in \{1,\dots ,n\},\hfill \end{array}$$

(13)

where ${\widehat{s}}_{ii}^{\beta }$ and ${\widehat{s}}_{ii}^{\gamma }$ are the ith diagonal elements of the projection matrices defined in (11) and (12), respectively. The terms $d_{\mu }^{*}({\widehat{\mu }}_i,{\widehat{\varphi }}_i)$ and $d_{\varphi }^{*}({\widehat{\mu }}_i,{\widehat{\varphi }}_i)$ represent the signed square roots of the deviance components for the mean and dispersion, respectively, and are established, for $i\in \{1,\dots ,n\}$, as 

$$\begin{array}{cc}\hfill d_{\mu }^{*}({\widehat{\mu }}_i,{\widehat{\varphi }}_i)& =\mathrm{sign}(y_i-{\widehat{\mu }}_i){(2y_i({\tilde{\theta }}_i-{\widehat{\theta }}_i)-2(b\left({\tilde{\theta }}_i\right)-b\left({\widehat{\theta }}_i\right)))}^{1/2},\hfill \\ \hfill d_{\varphi }^{*}({\widehat{\mu }}_i,{\widehat{\varphi }}_i)& =\mathrm{sign}({\widehat{t}}_i+d^{\prime }\left({\widehat{\varphi }}_i\right)){(2t_i({\tilde{\varphi }}_i-{\widehat{\varphi }}_i)+2(d\left({\tilde{\varphi }}_i\right)-d\left({\widehat{\varphi }}_i\right)))}^{1/2}.\hfill \end{array}$$

These terms capture the contribution of each observation to the deviance, with the sign indicating whether the observed value is above or below the fitted value.

3. Nonlinear Models in the Exponential Family

As the modeling of exponential family distributions extends beyond linear relationships, it becomes necessary to explore nonlinear formulations that offer greater flexibility in capturing complex data patterns. In this section, we transition to nonlinear models within the exponential family framework, focusing on the general structure and specific cases such as the nonlinear negative binomial model.

3.1. General Structure of Nonlinear Models

Let $Y_1,\dots ,Y_n$ be independent random variables, each following the distribution with density as defined in (1), where ${\mu }_i$ and ${\varphi }_i>0$ satisfy the nonlinear structures given by

$$\begin{array}{cc}\hfill g\left({\mu }_i\right)& ={\eta }_{1i}=f_1\left({\mathit{x}}_i^{\top }\mathit{\beta }\right),\hfill \end{array}$$

(14)

$$\begin{array}{cc}\hfill h\left({\varphi }_i\right)& ={\eta }_{2i}=f_2\left({\mathit{z}}_i^{\top }\mathit{\gamma }\right),i\in \{1,\dots ,n\},\hfill \end{array}$$

(15)

where $\mathit{\beta }\in {\mathbb{R}}^k$ and $\mathit{\gamma }\in {\mathbb{R}}^q$ are unknown parameter vectors with $k+q<n$. Here, ${\eta }_{1i}$ and ${\eta }_{2i}$ are the nonlinear predictors, ${\mathit{x}}_i^{\top }=(x_{i1},\dots ,x_{ik})$ and ${\mathit{z}}_i^{\top }=(z_{i1},\dots ,z_{iq})$ are observations of k and q known and fixed covariates, respectively, for each $i\in \{1,\dots ,n\}$.

The link functions g and h are strictly monotonic and twice differentiable. The functions $f_1$ and $f_2$ are continuous and differentiable, ensuring that the Jacobian matrices, ${\mathit{J}}_1=\partial {\mathit{\eta }}_1/\partial \mathit{\beta }$ and ${\mathit{J}}_2=\partial {\mathit{\eta }}_2/\partial \mathit{\gamma }$ namely, have full ranks k and q, respectively.

The log-likelihood function for the structures presented in (14) and (15) is stated as in (2), with the jth component of the score function with respect to $\mathit{\beta }$ defined as

$$\begin{array}{cc}\hfill {\displaystyle \frac{\partial \ell \left(\mathit{\theta }\right)}{\partial {\beta }_j}}& =\sum _{i=1}^n{\displaystyle \frac{\partial {\ell }_i({\mu }_i;{\varphi }_i)}{\partial {\mu }_i}}{\displaystyle \frac{\partial {\mu }_i}{\partial {\eta }_{1i}}}{\displaystyle \frac{\partial {\eta }_{1i}}{\partial {\beta }_j}}=\sum _{i=1}^n{\varphi }_i(Y_i-{\mu }_i)\sqrt{{\displaystyle \frac{{\omega }_i}{V_i}}}{\displaystyle \frac{\partial {\eta }_{1i}}{\partial {\beta }_j}},j\in \{1,\dots ,k\},\hfill \end{array}$$

where $V_i=\partial {\mu }_i/\partial {\theta }_i$ and ${\omega }_i={\left(\partial {\mu }_i/\partial {\eta }_{1i}\right)}^2/V_i$. The score function can be expressed in matrix form as ${\mathit{U}}_{\mathit{\beta }}={\mathit{J}}_1^{\top }\mathsf{\Phi }{\mathit{W}}^{1/2}{\mathit{V}}^{-1/2}(\mathit{Y}-\mathit{\mu }),$ where ${\mathit{J}}_1=\partial {\mathit{\eta }}_1/\partial \mathit{\beta }$ is an $n\times k$ matrix, $\mathsf{\Phi }=\mathrm{diag}\{{\varphi }_1,\dots ,{\varphi }_n\}$, $\mathit{W}=\mathrm{diag}\{{\omega }_1,\dots ,{\omega }_n\}$, $\mathit{V}=\mathrm{diag}\{V_1,\dots ,V_n\}$, $\mathit{Y}={(Y_1,\dots ,Y_n)}^{\top }$, and $\mathit{\mu }={({\mu }_1,\dots ,{\mu }_n)}^{\top }$. Assuming that $c(Y_i;{\varphi }_i)=d\left({\varphi }_i\right)+{\varphi }_{i}a\left(Y_i\right)+u\left(Y_i\right)$, the score function with respect to $\mathit{\gamma }$ is given by

$$\begin{array}{cc}\hfill {\displaystyle \frac{\partial \ell \left(\mathit{\theta }\right)}{\partial {\gamma }_j}}& =\sum _{i=1}^n{\displaystyle \frac{\partial {\ell }_i({\mu }_i;{\varphi }_i)}{\partial {\varphi }_i}}{\displaystyle \frac{\partial {\varphi }_i}{\partial {\eta }_{2i}}}{\displaystyle \frac{\partial {\eta }_{2i}}{\partial {\gamma }_j}}=\sum _{i=1}^n(t_i+d^{\prime }\left({\varphi }_i\right)){\displaystyle \frac{\partial {\varphi }_i}{\partial {\eta }_{2i}}}{\displaystyle \frac{\partial {\eta }_{2i}}{\partial {\gamma }_j}},j\in \{1,\dots ,q\},\hfill \end{array}$$

where $t_i=Y_i{\theta }_i-b\left({\theta }_i\right)+a\left(Y_i\right)$. Similarly, the score function for $\mathit{\gamma }$ can be expressed as ${\mathit{U}}_{\mathit{\gamma }}={\mathit{J}}_2^{\top }{\mathit{H}}_{\gamma }^{-1}(\mathit{t}-{\mathit{\mu }}^{*}),$ where ${\mathit{J}}_2=\partial {\mathit{\eta }}_2/\partial \mathit{\gamma }$ is an $n\times q$ matrix, ${\mathit{H}}_{\gamma }=\mathrm{diag}\{h^{\prime }\left({\varphi }_1\right),\dots ,h^{\prime }\left({\varphi }_n\right)\}$, $h^{\prime }\left({\varphi }_i\right)=\partial {\mathit{\eta }}_{2i}/\partial {\varphi }_i$, $\mathit{t}={(t_1,\dots ,t_n)}^{\top }$, and ${\mathit{\mu }}^{*}={({\mu }_1^{*},\dots ,{\mu }_n^{*})}^{\top }$.

The Fisher information matrix is derived from the second-order derivatives of the log-likelihood function with respect to the parameter vector. For $j,l\in \{1,\dots ,k\}$, we have

$${\displaystyle \frac{{\partial }^2\ell \left(\mathit{\theta }\right)}{\partial {\beta }_j\partial {\beta }_{\ell }}}=\sum _{i=1}^n{\varphi }_i{\displaystyle \frac{\partial }{\partial {\beta }_{\ell }}}\left(\sqrt{{\displaystyle \frac{{\omega }_i}{V_i}}}\right)(Y_i-{\mu }_i){\displaystyle \frac{\partial {\eta }_{1i}}{\partial {\beta }_j}}-\sum _{i=1}^n{\varphi }_i\sqrt{{\displaystyle \frac{{\omega }_i}{V_i}}}{\left({\displaystyle \frac{\partial {\mu }_i}{\partial {\eta }_{1i}}}\right)}^2{\displaystyle \frac{\partial {\eta }_{1i}}{\partial {\beta }_j}}{\displaystyle \frac{\partial {\eta }_{1i}}{\partial {\beta }_l}},$$

and for $j,l\in \{1,\dots ,q\}$, we reach

$${\displaystyle \frac{{\partial }^2\ell \left(\mathit{\theta }\right)}{\partial {\gamma }_j\partial {\gamma }_l}}=\sum _{i=1}^n\left(t_i+d^{\prime }\left({\varphi }_i\right)\right){\displaystyle \frac{\partial {\eta }_{2i}}{\partial {\gamma }_j}}{\displaystyle \frac{\partial }{\partial {\gamma }_l}}\left({\displaystyle \frac{\partial {\varphi }_i}{\partial {\eta }_{2i}}}\right)+\sum _{i=1}^n{d}^{″}\left({\varphi }_i\right){\left({\displaystyle \frac{\partial {\varphi }_i}{\partial {\eta }_{2i}}}\right)}^2{\displaystyle \frac{\partial {\eta }_{2i}}{\partial {\gamma }_j}}{\displaystyle \frac{\partial {\eta }_{2i}}{\partial {\gamma }_l}}.$$

For $j\in \{1,\dots ,k\}$ and $l\in \{1,\dots ,q\}$, we obtain

$${\displaystyle \frac{{\partial }^2\ell \left(\mathit{\theta }\right)}{\partial {\beta }_j\partial {\gamma }_l}}=\sum _{i=1}^n\left(Y_i-{\mu }_i\right)\sqrt{{\displaystyle \frac{{\omega }_i}{V_i}}}{\displaystyle \frac{\partial {\varphi }_i}{\partial {\eta }_{2i}}}{\displaystyle \frac{\partial {\eta }_{1i}}{\partial {\beta }_j}}{\displaystyle \frac{\partial {\eta }_{2i}}{\partial {\gamma }_l}}.$$

Since $\mathrm{E}(Y_i-{\mu }_i)=0$ and $\mathrm{E}(t_i-{\mu }_{T_i})=0$ (under regularity conditions), the expected values of the above-presented derivatives are established as

$$\mathrm{E}\left({\displaystyle \frac{{\partial }^2\ell \left(\mathit{\theta }\right)}{\partial {\beta }_j\partial {\beta }_l}}\right)=-\sum _{i=1}^n{\varphi }_i{\omega }_i{\displaystyle \frac{\partial {\eta }_{1i}}{\partial {\beta }_j}}{\displaystyle \frac{\partial {\eta }_{1i}}{\partial {\beta }_l}},\mathrm{E}\left({\displaystyle \frac{{\partial }^2\ell \left(\mathit{\theta }\right)}{\partial {\gamma }_j\partial {\gamma }_l}}\right)=-\sum _{i=1}^n{\displaystyle \frac{d^{″}\left({\varphi }_i\right)}{{\left(h^{\prime }\left({\varphi }_i\right)\right)}^2}}{\displaystyle \frac{\partial {\eta }_{2i}}{\partial {\gamma }_j}}{\displaystyle \frac{\partial {\eta }_{2i}}{\partial {\gamma }_l}},\mathrm{E}\left({\displaystyle \frac{{\partial }^2\ell \left(\mathit{\theta }\right)}{\partial {\beta }_j\partial {\gamma }_l}}\right)=0.$$

Thus, the Fisher information matrix for $\mathit{\theta }$ is a block diagonal matrix given as ${\mathit{K}}_{\mathit{\theta }\mathit{\theta }}=\mathrm{diag}\{{\mathit{K}}_{\mathit{\beta }\mathit{\beta }},{\mathit{K}}_{\mathit{\gamma }\mathit{\gamma }}\}$, where ${\mathit{K}}_{\mathit{\beta }\mathit{\beta }}$ and ${\mathit{K}}_{\mathit{\gamma }\mathit{\gamma }}$ are the Fisher information matrices for $\mathit{\beta }$ and $\mathit{\gamma }$, respectively. These expected values can be expressed in matrix form as

$${\mathit{K}}_{\mathit{\beta }\mathit{\beta }}=\mathrm{E}\left(-{\displaystyle \frac{{\partial }^2\ell (\mathit{\beta };\mathit{\gamma })}{\partial \mathit{\beta }\partial {\mathit{\beta }}^{\top }}}\right)={\mathit{J}}_1^{\top }\mathsf{\Phi }\mathit{W}{\mathit{J}}_1,{\mathit{K}}_{\mathit{\gamma }\mathit{\gamma }}=\mathrm{E}\left(-{\displaystyle \frac{{\partial }^2\ell (\mathit{\beta };\mathit{\gamma })}{\partial \mathit{\gamma }\partial {\mathit{\gamma }}^{\top }}}\right)={\mathit{J}}_2^{\top }\mathit{P}{\mathit{J}}_2,$$

where $\mathit{P}={\mathit{V}}_{\gamma }{\mathit{H}}_{\gamma }^{-2}$ and ${\mathit{V}}_{\gamma }=\mathrm{diag}\{-d^{″}\left({\varphi }_1\right),\dots ,-d^{″}\left({\varphi }_n\right)\}$. The Poisson distribution is commonly employed for modeling count data. However, it may present issues, particularly in cases of overdispersion, where the observed variance exceeds the mean. In such situations, the Poisson model becomes inadequate because it inherently assumes that the mean and variance are equal.

A similar issue arises with the binomial model, where the dispersion parameter is fixed as the value n; see Table 1. However, in a regression context, n is typically known and does not vary, which can limit the model flexibility in accommodating overdispersion.

3.2. Nonlinear Negative Binomial Model

In recent years, the negative binomial distribution has been increasingly utilized as an alternative in situations where overdispersion is present. Overdispersion occurs when the observed variance exceeds the mean, a scenario in which the Poisson regression model becomes inadequate because it assumes that the mean and variance are equal. Ignoring overdispersion can lead to relevant consequences, such as incorrect or underestimated standard errors in parameter estimates.

Let Y be a random variable following a negative binomial distribution with parameters $\mu >0$ and $\varphi >0$, with its probability function given by

$$f(y;\mu ,\varphi )={\displaystyle \frac{\mathsf{\Gamma }(y+\varphi )}{\mathsf{\Gamma }(y+1)\mathsf{\Gamma }\left(\varphi \right)}}{\left({\displaystyle \frac{\varphi }{\varphi +\mu }}\right)}^{\varphi }{\left({\displaystyle \frac{\mu }{\varphi +\mu }}\right)}^y,y\in \{0,1,\dots \}$$

where, as mentioned, $\mathsf{\Gamma }$ is the gamma function. We use the notation $Y\sim \mathrm{NB}(\mu ,\varphi )$ for the negative binomial distribution, and its expected value and variance are given by $\mathrm{E}\left(Y\right)=\mu$ and $\mathrm{Var}\left(Y\right)=\mu +{\mu }^2/\varphi$, respectively. When the parameter $\varphi$ is known, the negative binomial distribution belongs to the exponential family, with the canonical parameter being $\theta =log(\mu /(\mu +\varphi ))$, the cumulant function $b\left(\theta \right)=-\varphi log(\mu /(\mu +\varphi \left)\right)$, the normalizing function $c(y;\varphi )=\mathsf{\Gamma }(y+\varphi )/\left(\mathsf{\Gamma }\right(y+1\left)\mathsf{\Gamma }\right(\varphi \left)\right)$, and the variance function $\mathrm{V}\left(\mu \right)=\mu +{\mu }^2/\varphi$.

Consider $Y_1,\dots ,Y_n$ to be independent random variables such that $Y_i\sim \mathrm{NB}({\mu }_i,{\varphi }_i)$, for $i\in \{1,\dots ,n\}$, where the dispersion parameter ${\varphi }_i$ is unknown. We assume a regression structure in which ${\mu }_i$ and ${\varphi }_i$ satisfy the nonlinear relationships defined in (14) and (15), respectively. Let the vector of model parameters be $\mathit{\theta }={({\mathit{\beta }}^{\top },{\mathit{\gamma }}^{\top })}^{\top }$ and the corresponding log-likelihood function be expressed as

$$\ell \left(\mathit{\theta }\right)=\sum _{i=1}^n\left(y_{i}log\left({\mu }_i\right)+{\varphi }_{i}log\left({\varphi }_i\right)-(y_i+{\varphi }_i)log({\mu }_i+{\varphi }_i)+log\left(\mathsf{\Gamma }({\varphi }_i+y_i)\right)-log\left(\mathsf{\Gamma }\left({\varphi }_i\right)\right)-log\left(\mathsf{\Gamma }(y_i+1)\right)\right),$$

where ${\mu }_i$ is the mean and ${\varphi }_i$ the dispersion parameter, for $i\in \{1,\dots ,n\}$. The score functions for $\mathit{\beta }$ and $\mathit{\gamma }$ can be represented in matrix form as ${\mathit{U}}_{\mathit{\beta }}={\mathit{J}}_1^{\top }\mathit{W}(\mathit{y}-\mathit{\mu })$ and ${\mathit{U}}_{\mathit{\gamma }}={\mathit{J}}_2^{\top }{\mathit{W}}_{\gamma }^{-1}\mathit{c}$, where ${\mathit{J}}_1=\partial {\mathit{\eta }}_1/\partial \mathit{\beta }$ is an $n\times k$ matrix, ${\mathit{J}}_2=\partial {\mathit{\eta }}_2/\partial \mathit{\gamma }$ is an $n\times q$ matrix, $\mathit{W}=\mathrm{diag}\{{\omega }_1,\dots ,{\omega }_n\}$ with ${\omega }_i={(\partial {\mu }_i/\partial {\eta }_{1i})}^2\mathrm{V}{\left({\mu }_i\right)}^{-1}$, ${\mathit{W}}_{\gamma }=\mathrm{diag}\{h^{\prime }\left({\varphi }_1\right),\dots ,h^{\prime }\left({\varphi }_n\right)\}$ with $h^{\prime }\left({\varphi }_i\right)=\partial {\eta }_{2i}/\partial {\varphi }_i$, $\mathit{y}={(y_1,\dots ,y_n)}^{\top }$, and $\mathit{\mu }={({\mu }_1,\dots ,{\mu }_n)}^{\top }$.

According to [48], the Fisher information matrix for $\mathit{\theta }={({\mathit{\beta }}^{\top },{\mathit{\gamma }}^{\top })}^{\top }$ is block-diagonal and given by ${\mathit{K}}_{\mathit{\theta }\mathit{\theta }}=\mathrm{diag}\{{\mathit{K}}_{\mathit{\beta }\mathit{\beta }},{\mathit{K}}_{\mathit{\gamma }\mathit{\gamma }}\}$, where ${\mathit{K}}_{\mathit{\beta }\mathit{\beta }}$ and ${\mathit{K}}_{\mathit{\gamma }\mathit{\gamma }}$ are the Fisher information matrices for $\mathit{\beta }$ and $\mathit{\gamma }$, respectively. Thus, we can represent ${\mathit{K}}_{\mathit{\beta }\mathit{\beta }}$ and ${\mathit{K}}_{\mathit{\gamma }\mathit{\gamma }}$ in matrix form as

$${\mathit{K}}_{\mathit{\beta }\mathit{\beta }}=\mathrm{E}\left(-{\displaystyle \frac{{\partial }^2\ell (\mathit{\beta };\mathit{\gamma })}{\partial \mathit{\beta }\partial {\mathit{\beta }}^{\top }}}\right)={\mathit{J}}_1^{\top }\mathit{W}{\mathit{J}}_1,{\mathit{K}}_{\mathit{\gamma }\mathit{\gamma }}=\mathrm{E}\left(-{\displaystyle \frac{{\partial }^2\ell (\mathit{\beta };\mathit{\gamma })}{\partial \mathit{\gamma }\partial {\mathit{\gamma }}^{\top }}}\right)={\mathit{J}}_2^{\top }\mathit{P}{\mathit{J}}_2,$$

where $\mathit{P}=\mathit{T}{\mathit{W}}_{\gamma }^{-2}$ and $\mathit{T}=\mathrm{diag}\{\mathrm{E}\left(c_1^{\prime }\right),\dots ,\mathrm{E}\left(c_n^{\prime }\right)\}$, with $\mathit{c}={(c_1,\dots ,c_n)}^{\top }$ being the residual vector for the dispersion model. The deviance function from the mean, assuming ${\varphi }_i$ is known, for $i\in \{1,\dots ,n\}$, is given by

$$D^{*}(\mathit{y},\widehat{\mathit{\mu }})=2\sum _{i=1}^n\left({\varphi }_{i}log\left({\displaystyle \frac{{\widehat{\mu }}_i+{\varphi }_i}{y_i+{\varphi }_i}}\right)+y_{i}log\left({\displaystyle \frac{y_i({\widehat{\mu }}_i+{\varphi }_i)}{{\widehat{\mu }}_i(y_i+{\varphi }_i)}}\right)\right).$$

In practice, ${\varphi }_i$ is replaced by its ML estimate ${\widehat{\varphi }}_i$. When $y_i=0$ for some i, the ith component of the deviance is stated as

$$2{\varphi }_{i}log\left({\displaystyle \frac{y_i+{\varphi }_i}{{\widehat{\mu }}_i+{\varphi }_i}}\right).$$

The construction of ordinary residuals depends on the Fisher scoring method used to obtain the ML estimate; see Appendix B.

By replacing the matrices $\mathit{X}$ and $\mathit{Z}$ with ${\mathit{J}}_1$ and ${\mathit{J}}_2$, respectively, we can express the Fisher scoring method as an iterative weighted least squares process presented as

$$\begin{array}{cc}\hfill {\mathit{\beta }}^{(m+1)}& ={\left({\mathit{J}}_1^{\top }{\mathsf{\Phi }}^{\left(m\right)}{\mathit{W}}^{\left(m\right)}{\mathit{J}}_1\right)}^{-1}{\mathit{J}}_1^{\top }{\mathsf{\Phi }}^{\left(m\right)}{\mathit{W}}^{\left(m\right)}{\mathit{u}}_1^{\left(m\right)},\hfill \end{array}$$

(16)

$$\begin{array}{cc}\hfill {\mathit{\gamma }}^{(m+1)}& ={\left({\mathit{J}}_2^{\top }{\mathit{P}}^{\left(m\right)}{\mathit{J}}_2\right)}^{-1}{\mathit{J}}_2^{\top }{\mathit{P}}^{\left(m\right)}{\mathit{u}}_2^{\left(m\right)},\hfill \end{array}$$

(17)

where ${\mathit{u}}_1^{\left(m\right)}={\mathit{J}}_1{\mathit{\beta }}^{\left(m\right)}+{\mathit{W}}^{1/2\left(m\right)}{\mathit{V}}^{-1/2\left(m\right)}(\mathit{y}-{\mathit{\mu }}^{\left(m\right)})$ and ${\mathit{u}}_2^{\left(m\right)}={\mathit{J}}_2{\mathit{\gamma }}^{\left(m\right)}+{\mathit{V}}_{\gamma }^{-1\left(m\right)}{\mathit{H}}_{\gamma }^{\left(m\right)}$$(\mathit{t}-{\mathit{\mu }}^{*\left(m\right)})$. Upon convergence, we have

$$\begin{array}{cc}\hfill \widehat{\mathit{\beta }}& ={\left({\mathit{J}}_1^{\top }\widehat{\mathsf{\Phi }}\widehat{\mathit{W}}{\mathit{J}}_1\right)}^{-1}{\mathit{J}}_1^{\top }\widehat{\mathsf{\Phi }}\widehat{\mathit{W}}{\mathit{u}}_1,\hfill \end{array}$$

(18)

$$\begin{array}{cc}\hfill \widehat{\mathit{\gamma }}& ={\left({\mathit{J}}_2^{\top }\widehat{\mathit{P}}{\mathit{J}}_2\right)}^{-1}{\mathit{J}}_2^{\top }\widehat{\mathit{P}}{\mathit{u}}_2,\hfill \end{array}$$

(19)

with

$${\mathit{u}}_1={\widehat{\mathit{\eta }}}_1+{\widehat{\mathit{W}}}^{1/2}{\widehat{\mathit{V}}}^{-1/2}(\mathit{y}-\widehat{\mathit{\mu }}),{\mathit{u}}_2={\widehat{\mathit{\eta }}}_2+{\widehat{\mathit{V}}}_{\gamma }^{-1}{\widehat{\mathit{H}}}_{\gamma }(\mathit{t}-{\widehat{\mathit{\mu }}}^{*}).$$

The expressions stated in (18) and (19) can be interpreted as the OLS estimates of $\mathit{\beta }$ and $\mathit{\gamma }$, respectively. In other words, to obtain $\widehat{\mathit{\beta }}$, consider the regression of the auxiliary variable ${\widehat{\mathsf{\Phi }}}^{1/2}{\widehat{\mathit{W}}}^{1/2}{\mathit{u}}_1$ on the covariate ${\widehat{\mathsf{\Phi }}}^{1/2}{\widehat{\mathit{W}}}^{1/2}{\mathit{J}}_1$, and to reach $\widehat{\mathit{\gamma }}$, consider the regression of the auxiliary variable ${\widehat{\mathit{P}}}^{1/2}{\mathit{u}}_2$ on the covariate ${\widehat{\mathit{P}}}^{1/2}{\mathit{J}}_2$. Consequently, the ordinary residuals (see Appendix B) from these regressions are given by

$$\begin{array}{cc}\hfill r^{\mathit{\beta }}& ={\widehat{\mathsf{\Phi }}}^{1/2}{\widehat{\mathit{W}}}^{1/2}({\mathit{u}}_1-{\widehat{\mathit{\eta }}}_1),r^{\mathit{\gamma }}={\widehat{\mathit{P}}}^{1/2}({\mathit{u}}_2-{\widehat{\mathit{\eta }}}_2).\hfill \end{array}$$

Thus, considering the residuals based on the sub-models for the mean and dispersion, we have

$$r_i^{\mathit{\beta }}={\displaystyle \frac{y_i-{\widehat{\mu }}_i}{\sqrt{{\widehat{\varphi }}_i^{-1}{\widehat{V}}_i}}},r_i^{\mathit{\gamma }}={\displaystyle \frac{t_i-{\widehat{\mu }}_i^{*}}{\sqrt{-d^{″}\left({\widehat{\varphi }}_i\right)}}},i\in \{1,\dots ,n\}.$$

These residuals can be standardized using the projection matrices from the OLS solutions stated in (18) and (19). Specifically, we obtain

$$\begin{array}{cc}\hfill {\widehat{\mathit{S}}}^{\mathit{\beta }}& ={\widehat{\mathsf{\Phi }}}^{1/2}{\widehat{\mathit{W}}}^{1/2}{\mathit{J}}_1{\left({\mathit{J}}_1^{\top }\widehat{\mathsf{\Phi }}\widehat{\mathit{W}}{\mathit{J}}_1\right)}^{-1}{\mathit{J}}_1^{\top }{\widehat{\mathit{W}}}^{1/2}{\widehat{\mathsf{\Phi }}}^{1/2},\hfill \end{array}$$

(20)

$$\begin{array}{cc}\hfill {\widehat{\mathit{S}}}^{\mathit{\gamma }}& ={\widehat{\mathit{P}}}^{1/2}{\mathit{J}}_2{\left({\mathit{J}}_2^{\top }\widehat{\mathit{P}}{\mathit{J}}_2\right)}^{-1}{\mathit{J}}_2^{\top }{\widehat{\mathit{P}}}^{1/2}.\hfill \end{array}$$

(21)

Therefore, the corresponding standardized residuals are formulated as

$$r_{{\mathrm{s}}_i}^{\mathit{\beta }}={\displaystyle \frac{y_i-{\widehat{\mu }}_i}{\sqrt{{\widehat{\varphi }}_i^{-1}{\widehat{V}}_i(1-{\widehat{s}}_{ii}^{\mathit{\beta }})}}},r_{{\mathrm{s}}_i}^{\mathit{\gamma }}={\displaystyle \frac{t_i-{\widehat{\mu }}_i^{*}}{\sqrt{-d^{″}\left({\widehat{\varphi }}_i\right)(1-{\widehat{s}}_{ii}^{\mathit{\gamma }})}}},i\in \{1,\dots ,n\},$$

where ${\widehat{s}}_{ii}^{\mathit{\beta }}$ and ${\widehat{s}}_{ii}^{\mathit{\gamma }}$ are the diagonal elements of the matrices presented in (20) and (21), respectively.

We have analyzed various types of residuals within the context of GLM, focusing on their roles in model diagnostics and goodness-of-fit evaluation. However, in scenarios where both the mean and dispersion are modeled simultaneously, it is beneficial to consider a combined residual that encapsulates the information from both aspects.

4. Standardized Combined Residuals

In this section, we introduce the concept of standardized combined residuals, designed to integrate information from both the mean and dispersion sub-models into a single diagnostic tool. This unified approach facilitates a more comprehensive assessment of model adequacy. We begin by presenting a new standardized combined residual and then extend the concept to accommodate broader model structures.

4.1. New Standardized Combined Residual

The combined residual is calculated by summing the residuals from the mean and dispersion components. Using the estimates from the formulas given in  (16) and (17), the combined residual for each observation i is defined as

$$r_i^{\mathit{\beta }\mathit{\gamma }}=(y_i-{\widehat{\mu }}_i)+(t_i-{\widehat{\mu }}_i^{*}),i\in \{1,\dots ,n\},$$

(22)

where $y_i$ is the observed response, ${\widehat{\mu }}_i$ is the estimated mean from the mean sub-model, $t_i$ is a function related to the dispersion component, and ${\widehat{\mu }}_i^{*}$ is the estimated value related to $t_i$. By combining these residuals, we capture discrepancies in both models.

To standardize the combined residual stated in (22), we calculate its variance. Using the variances and covariances from the equations presented in (A2), (A5), and (A6) of Appendix A, the variance of $r_i^{\mathit{\beta }\mathit{\gamma }}$, denoted by ${\zeta }_i$, is given as

$${\zeta }_i={\varphi }_i^{-1}{b}^{″}\left({\theta }_i\right)-d^{″}\left({\varphi }_i\right),i\in \{1,\dots ,n\},$$

(23)

where, as mentioned, $b^{″}\left({\theta }_i\right)$ is the second derivative of the cumulant generating function $b\left({\theta }_i\right)$ and $d^{″}\left({\varphi }_i\right)$ is the second derivative of the dispersion function $d\left({\varphi }_i\right)$. The term ${\varphi }_i^{-1}{b}^{″}\left({\theta }_i\right)$ established in (23) represents the variance from the mean sub-model, while $d^{″}\left({\varphi }_i\right)$ accounts for variability in the dispersion sub-model.

With ${\zeta }_i$ computed, we define the standardized combined residual as

$$r_{{\mathrm{S}}_i}^{\mathit{\beta }\mathit{\gamma }}={\displaystyle \frac{(y_i-{\widehat{\mu }}_i)+(t_i-{\widehat{\mu }}_i^{*})}{\sqrt{{\widehat{\zeta }}_i}}},i\in \{1,\dots ,n\},$$

where ${\widehat{\zeta }}_i$ is the variance ${\zeta }_i$ evaluated at the estimated parameters ${\widehat{\mu }}_i$ and ${\widehat{\varphi }}_i$. This standardization allows for comparison across observations and aids in identifying outliers or model inadequacies by scaling the residuals.

Furthermore, we can estimate the covariance matrices of the parameter estimators $\widehat{\mathit{\beta }}$ and $\widehat{\mathit{\gamma }}$ to assess the variability in the model parameters. The approximate covariance matrices are presented as

$$\mathrm{Cov}\left(\widehat{\mathit{\beta }}\right)\approx {\left({\mathit{J}}_1^{\top }\mathsf{\Phi }\mathit{W}{\mathit{J}}_1\right)}^{-1},\mathrm{Cov}\left(\widehat{\mathit{\gamma }}\right)\approx {\left({\mathit{J}}_2^{\top }\mathit{P}{\mathit{J}}_2\right)}^{-1},$$

where ${\mathit{J}}_1$ and ${\mathit{J}}_2$ are the Jacobian matrices of the mean and dispersion models, respectively, and $\mathsf{\Phi }$ and $\mathit{P}$ are diagonal weight matrices. These covariance matrices help to quantify the uncertainty in the estimated parameters due to the data variability. Using these covariance estimates, the normal equations for parameter estimation, originally expressed in (18) and (19), can be approximated as

$$\left({\mathit{J}}_1^{\top }\mathsf{\Phi }\mathit{W}{\mathit{J}}_1\right)\widehat{\mathit{\beta }}\approx {\mathit{J}}_1^{\top }\mathsf{\Phi }\mathit{W}{u}_1,\left({\mathit{J}}_2^{\top }\mathit{P}{\mathit{J}}_2\right)\widehat{\mathit{\gamma }}\approx {\mathit{J}}_2^{\top }\mathit{P}{u}_2,$$

where $u_1$ and $u_2$ are the working responses for the mean and dispersion models. The left-hand sides represent the information matrices, adjusted by projection matrices, projecting residuals onto the parameter space. This simplifies the parameter estimation process by solving weighted least squares problems.

Assuming that the estimated matrices $\widehat{\mathit{W}}$, $\widehat{\mathsf{\Phi }}$, and $\widehat{\mathit{P}}$ are good approximations of their true counterparts $\mathit{W}$, $\mathsf{\Phi }$, and $\mathit{P}$, respectively, we can simplify the covariance for the working responses as

$$\widehat{\mathrm{Cov}}\left(u_1\right)\approx {\widehat{\mathit{W}}}^{-1}{\widehat{\mathsf{\Phi }}}^{-1},\widehat{\mathrm{Cov}}\left(u_2\right)\approx {\widehat{\mathit{P}}}^{-1},$$

where $\widehat{\mathit{W}}$, $\widehat{\mathsf{\Phi }}$, and $\widehat{\mathit{P}}$ are the diagonal matrices of weights and adjustments.

With these approximations, the covariance matrices for $r^{\mathit{\beta }}$ and $r^{\mathit{\gamma }}$ can be derived. In regression, the covariance of residuals is expressed using projection matrices given by

$$\widehat{\mathrm{Cov}}\left(r^{\mathit{\beta }}\right)\approx (\mathit{I}-{\widehat{\mathit{S}}}^{\mathit{\beta }}),\widehat{\mathrm{Cov}}\left(r^{\mathit{\gamma }}\right)\approx (\mathit{I}-{\widehat{\mathit{S}}}^{\mathit{\gamma }}),$$

where $\mathit{I}$ is the identity matrix, and ${\widehat{\mathit{S}}}^{\mathit{\beta }}$ and ${\widehat{\mathit{S}}}^{\mathit{\gamma }}$ are the projection matrices for the mean and dispersion models, respectively. These matrices reflect the variability explained by the models, while $\mathit{I}-\widehat{\mathit{S}}$ captures the unexplained portion.

To account for the covariances of $u_1$ and $u_2$, we use the standardized combined residual by summing the individual standardized residuals. For observation i, it is defined as

$$r_{{\mathrm{S}}_i}^{\mathit{\beta }\mathit{\gamma }}={\displaystyle \frac{r_i^{\mathit{\beta }}+r_i^{\mathit{\gamma }}}{\sqrt{(1-{\widehat{s}}_{ii}^{\mathit{\beta }})+(1-{\widehat{s}}_{ii}^{\mathit{\gamma }})+2{\widehat{m}}_{ii}}}},i\in \{1,\dots ,n\},$$

(24)

where $r_i^{\mathit{\beta }}$ and $r_i^{\mathit{\gamma }}$ are the standardized residuals from the mean and dispersion models, respectively; ${\widehat{s}}_{ii}^{\mathit{\beta }}$ and ${\widehat{s}}_{ii}^{\mathit{\gamma }}$ are the ith diagonal elements of ${\widehat{\mathit{S}}}^{\mathit{\beta }}$ and ${\widehat{\mathit{S}}}^{\mathit{\gamma }}$, representing the leverage of observation i in each model; and ${\widehat{m}}_{ii}$ is the ith diagonal element of the covariance matrix $M=\mathrm{Cov}(r^{\mathit{\beta }},r^{\mathit{\gamma }})$, capturing the covariance between the residuals of the two sub-models.

The denominator stated in (24) represents the combined variance of the residuals, incorporating both individual variances and their covariance. This standardization ensures that $r_{{\mathrm{S}}_i}^{\mathit{\beta }\mathit{\gamma }}$ has a unit variance, allowing for consistent comparison across observations and aiding in the detection of outliers or influential data points. However, evaluating the standardized combined residual given in (24) requires calculating the projection matrices ${\widehat{\mathit{S}}}^{\mathit{\beta }}$ and ${\widehat{\mathit{S}}}^{\mathit{\gamma }}$, as well as the covariance matrix. These calculations can be computationally expensive, especially for large datasets where matrix operations become complex. The difficulty arises from the need for matrix inversions and multiplications involving the design matrices, which scale poorly with the number of observations. Due to this, the practical implementation of the standardized combined residual presented in (24), while it theoretically sounds good, may be limited in applications with large datasets.

4.2. Background for the Generalized Standardized Combined Residual

Next, we develop a generalized form of the combined residual by relaxing certain assumptions on the functional form of $c(y_i;{\varphi }_i)$. These conditions were previously used to derive variance expressions, but by removing them, we focus on a broader estimation of the dispersion parameter ${\varphi }_i$, which allows greater flexibility in model specification.

Let $Y_1,\dots ,Y_n$ be independent random variables, where each $Y_i$, for $i\in \{1,\dots ,n\}$, follows the density stated in (1). Then, the probability density function of $Y_i$ is given by

$$f(y_i;{\theta }_i,{\varphi }_i)=exp\left({\displaystyle \frac{{\theta }_i{y}_i-b\left({\theta }_i\right)}{{\varphi }_i}}+c(y_i;{\varphi }_i)\right),i\in \{1,\dots ,n\},$$

(25)

where ${\theta }_i$ is the canonical parameter, ${\varphi }_i>0$ is the dispersion parameter, b is a known function related to the cumulant generating function, and $c(y_i;{\varphi }_i)$ is a normalizing function.

Building on the nonlinear relationships defined in (14) and (15), we now provide a more detailed explanation of the link functions and covariate effects.

The mean ${\mu }_i$ and dispersion parameter ${\varphi }_i$ are linked to the covariates through nonlinear functional relationships established such as in (14) and (15), that is, $g\left({\mu }_i\right)={\eta }_{1i}=f_1\left({\mathit{x}}_i^{\top }\mathit{\beta }\right)$ and $h\left({\varphi }_i\right)={\eta }_{2i}=f_2\left({\mathit{z}}_i^{\top }\mathit{\gamma }\right)$, for $i\in \{1,\dots ,n\}$, where, as mentioned, g and h are known link functions, ${\mathit{x}}_i$ and ${\mathit{z}}_i$ are vectors of covariates values for the mean and dispersion sub-models, respectively, and $\mathit{\beta }$ and $\mathit{\gamma }$ are the parameter vectors to be estimated. The functions $f_1$ and $f_2$ represent the nonlinear effects of the covariates on the mean and dispersion components, allowing the model to capture more complex relationships between covariates and the response.

The log-likelihood function for the model based on all observations is given by

$$\ell \left(\mathit{\theta }\right)=\sum _{i=1}^n{\ell }_i({\mu }_i;{\varphi }_i)=\sum _{i=1}^n\left\{{\displaystyle \frac{{\theta }_i{y}_i-b\left({\theta }_i\right)}{{\varphi }_i}}+c(y_i;{\varphi }_i)\right\},$$

(26)

where $\mathit{\theta }$ represents all the parameters in the model.

To derive the score function for the parameter vector $\mathit{\beta }$, we calculate the gradient of the log-likelihood function with respect to $\mathit{\beta }$. Applying the chain rule, we obtain

$${\mathit{U}}_{\mathit{\beta }}={\displaystyle \frac{\partial \ell \left(\mathit{\theta }\right)}{\partial \mathit{\beta }}}=\sum _{i=1}^n{\displaystyle \frac{\partial {\ell }_i}{\partial {\theta }_i}}{\displaystyle \frac{\partial {\theta }_i}{\partial {\mu }_i}}{\displaystyle \frac{\partial {\mu }_i}{\partial {\eta }_{1i}}}{\displaystyle \frac{\partial {\eta }_{1i}}{\partial \mathit{\beta }}}.$$

(27)

Recognizing that $\partial {\ell }_i/\partial {\theta }_i=\left(y_i-b^{\prime }\left({\theta }_i\right)\right)/{\varphi }_i$ and noting that $b^{\prime }\left({\theta }_i\right)={\mu }_i$, the formulation presented in (27) simplifies to

$${\mathit{U}}_{\mathit{\beta }}=\sum _{i=1}^n{\displaystyle \frac{(y_i-{\mu }_i)}{{\varphi }_i}}{\displaystyle \frac{\partial {\theta }_i}{\partial {\mu }_i}}{\displaystyle \frac{\partial {\mu }_i}{\partial {\eta }_{1i}}}{\displaystyle \frac{\partial {\eta }_{1i}}{\partial \mathit{\beta }}}.$$

(28)

To express the equation stated in (28) in matrix form, we define:

• ${\mathit{J}}_1=\partial {\eta }_1/\partial \mathit{\beta }$ as the $n\times k$ Jacobian matrix for the mean model;
• $\mathsf{\Phi }=\mathrm{diag}\{{\varphi }_1,\dots ,{\varphi }_n\}$ as the diagonal matrix of dispersion parameters;
• $\mathit{W}=\mathrm{diag}\{{\omega }_1,\dots ,{\omega }_n\}$, where ${\omega }_i={(\partial {\mu }_i/\partial {\eta }_{1i})}^2/V_i$, for $i\in \{1,\dots ,n\}$;
• $\mathit{V}=\mathrm{diag}\{V_1,\dots ,V_n\}$, where $V_i=\mathrm{Var}\left(Y_i\right)={\varphi }_i{b}^{″}\left({\theta }_i\right)$, for $i\in \{1,\dots ,n\}$;
• $\mathit{y}={(y_1,\dots ,y_n)}^{\top }$ and $\mathit{\mu }={({\mu }_1,\dots ,{\mu }_n)}^{\top }$.

Then, we have that the score function is formulated as ${\mathit{U}}_{\mathit{\beta }}={\mathit{J}}_1^{\top }{\mathsf{\Phi }}^{-1}\mathit{W}(\mathit{y}-\mathit{\mu })$. This formulation shows that the score function is a weighted sum of the residuals $(\mathit{y}-\mathit{\mu })$, where the weights take into account the variance structure and the link function. The information matrix for $\mathit{\beta }$, which quantifies the information of the data about the parameters, is given by the expected value of the negative second derivative of the log-likelihood function stated as

$${\mathit{K}}_{\mathit{\beta }\mathit{\beta }}=\mathrm{E}[-{\partial }^2\ell \left(\mathit{\theta }\right)/\partial \mathit{\beta }\partial {\mathit{\beta }}^{\top }]={\mathit{J}}_1^{\top }{\mathsf{\Phi }}^{-1}\mathit{W}{\mathit{J}}_1.$$

This matrix plays a crucial role in parameter estimation and inference, as it is used to compute standard errors and conduct hypothesis tests.

Next, we derive the score function for $\mathit{\gamma }$, the parameter vector associated with the dispersion model. Starting with the partial derivative of the log-likelihood function with respect to the jth element of $\mathit{\gamma }$, denoted by ${\gamma }_j$, and applying the chain rule, we obtain

$$\begin{array}{cc}\hfill {\displaystyle \frac{\partial \ell \left(\mathit{\theta }\right)}{\partial {\gamma }_j}}& =\sum _{i=1}^n{\displaystyle \frac{\partial {\ell }_i}{\partial {\varphi }_i}}{\displaystyle \frac{\partial {\varphi }_i}{\partial {\eta }_{2i}}}{\displaystyle \frac{\partial {\eta }_{2i}}{\partial {\gamma }_j}},j\in \{1,\dots ,q\},\hfill \end{array}$$

where the derivative $\partial {\ell }_i/\partial {\varphi }_i$ represents how the log-likelihood function changes with respect to the dispersion parameter ${\varphi }_i$ and is given by  

$${\displaystyle \frac{\partial {\ell }_i}{\partial {\varphi }_i}}=-{\displaystyle \frac{{\theta }_i{y}_i-b\left({\theta }_i\right)}{{\varphi }_i^2}}+{\displaystyle \frac{\partial c(y_i;{\varphi }_i)}{\partial {\varphi }_i}},i\in \{1,\dots ,n\}.$$

To simplify the notation in the model, we define the term $t_i$, which captures the change in the log-likelihood function with respect to ${\varphi }_i$. Specifically, we express $t_i$ as

$$t_i={\displaystyle \frac{\partial {\ell }_i}{\partial {\varphi }_i}}=-{\displaystyle \frac{{\theta }_i{y}_i-b\left({\theta }_i\right)}{{\varphi }_i^2}}+c^{\prime }(y_i;{\varphi }_i),i\in \{1,\dots ,n\},$$

(29)

where $c^{\prime }(y_i;{\varphi }_i)$ is the derivative of $c(y_i;{\varphi }_i)$ with respect to ${\varphi }_i$. This quantity $t_i$ can be interpreted as a residual that reflects the variability associated with the dispersion parameter.

Next, we use $t_i$ to derive the score function for the dispersion parameters $\mathit{\gamma }$. Recognizing that $\partial {\varphi }_i/\partial {\eta }_{2i}=h^{\prime }{\left({\varphi }_i\right)}^{-1}$, where $h^{\prime }\left({\varphi }_i\right)$ is the derivative of the dispersion link function with respect to ${\varphi }_i$, we can express the score function for $\mathit{\gamma }$ as

$${\displaystyle \frac{\partial \ell \left(\mathit{\theta }\right)}{\partial {\gamma }_j}}=\sum _{i=1}^n{t}_i{h}^{\prime }{\left({\varphi }_i\right)}^{-1}{\displaystyle \frac{\partial {\eta }_{2i}}{\partial {\gamma }_j}},j\in \{1,\dots ,q\},$$

(30)

We can further express the formulation stated in (30) in matrix form, leading to the score function for $\mathit{\gamma }$ given by

$${\mathit{U}}_{\mathit{\gamma }}={\mathit{J}}_2^{\top }{\mathit{H}}_{\gamma }^{-1}\mathit{t},$$

(31)

where ${\mathit{J}}_2=\partial {\eta }_2/\partial \mathit{\gamma }$ is the Jacobian of the dispersion model, ${\mathit{H}}_{\gamma }=\mathrm{diag}\{h^{\prime }\left({\varphi }_1\right),\dots ,h^{\prime }\left({\varphi }_n\right)\}$ is the diagonal matrix of link function derivatives, and $\mathit{t}={(t_1,\dots ,t_n)}^{\top }$ are the dispersion residuals. The expression stated in (31) shows that the score function for $\mathit{\gamma }$ is a weighted sum of the components of $\mathit{t}$, incorporating both the link function and covariate effects, and reflecting the role of the dispersion parameter ${\varphi }_i$ in the model. To derive the second derivative of the log-likelihood function presented in (26) with respect to ${\gamma }_j$ and ${\gamma }_l$, we differentiate the score function stated in (30) with respect to ${\gamma }_l$ to obtain

$$\begin{array}{cc}\hfill {\displaystyle \frac{{\partial }^2\ell \left(\mathit{\theta }\right)}{\partial {\gamma }_j\partial {\gamma }_l}}& =\sum _{i=1}^n\left\{{\displaystyle \frac{\partial t_i}{\partial {\varphi }_i}}{\left({\displaystyle \frac{\partial {\varphi }_i}{\partial {\eta }_{2i}}}\right)}^2+t_i{\displaystyle \frac{{\partial }^2{\varphi }_i}{\partial {\eta }_{2i}^2}}\right\}{\displaystyle \frac{\partial {\eta }_{2i}}{\partial {\gamma }_j}}{\displaystyle \frac{\partial {\eta }_{2i}}{\partial {\gamma }_l}},j,l\in \{1,\dots ,q\},\hfill \end{array}$$

(32)

where $t_i^{\prime }=\partial t_i/\partial {\varphi }_i$ describes how the dispersion residual changes with respect to ${\varphi }_i$, and ${\partial }^2{\varphi }_i/\partial {\eta }_{2i}^2$ represents the second derivative of ${\varphi }_i$ with respect to ${\eta }_{2i}$, capturing the curvature of the link function. The terms $\partial {\eta }_{2i}/\partial {\gamma }_j$ and $\partial {\eta }_{2i}/\partial {\gamma }_l$ are the derivatives of the linear predictor with respect to the parameters. The curvature of the link function indicates how changes in ${\varphi }_i$ affect the linear predictor ${\eta }_{2i}$ through the link function.

The second derivative stated in (32) is crucial for constructing the Fisher information matrix for $\mathit{\gamma }$. Under regularity conditions, the expected value of the score function is zero, that is, $\mathrm{E}\left({\mathit{U}}_{\mathit{\gamma }}\right)=\mathbf{0}$. This fundamental property ensures that ML estimators are unbiased, meaning that the average score across repeated samples equals zero. Substituting the expression for the score function of $\mathit{\gamma }$ from the formula given in (31), we get

$$\mathrm{E}\left(\sum _{i=1}^n{t}_i{\displaystyle \frac{\partial {\varphi }_i}{\partial {\eta }_{2i}}}{\displaystyle \frac{\partial {\eta }_{2i}}{\partial {\gamma }_j}}\right)=0,j\in \{1,\dots ,q\}.$$

Since $\partial {\varphi }_i/\partial {\eta }_{2i}$ and $\partial {\eta }_{2i}/\partial {\gamma }_j$ are deterministic functions (given the data and parameter values), it follows that $\mathrm{E}\left(t_i\right)=0$, for $i\in \{1,\dots ,n\}$. This result shows that the expected value of the dispersion residuals $t_i$ is zero, similar to the property that the mean residuals $(y_i-{\mu }_i)$ have zero expectation. Next, we consider the expected value of the second derivative of the log-likelihood function expressed in (26) with respect to ${\gamma }_j$ and ${\gamma }_l$, as shown in the formula given in (32). Taking expectation, we obtain

$$\mathrm{E}\left({\displaystyle \frac{{\partial }^2\ell \left(\mathit{\theta }\right)}{\partial {\gamma }_j\partial {\gamma }_l}}\right)=\sum _{i=1}^n\mathrm{E}\left(t_i^{\prime }\right){\left({\displaystyle \frac{\partial {\varphi }_i}{\partial {\eta }_{2i}}}\right)}^2{\displaystyle \frac{\partial {\eta }_{2i}}{\partial {\gamma }_j}}{\displaystyle \frac{\partial {\eta }_{2i}}{\partial {\gamma }_l}},j,l\in \{1,\dots ,q\},$$

where $t_i^{\prime }=\partial t_i/\partial {\varphi }_i$ represents the sensitivity of the dispersion residuals to changes in the dispersion parameter ${\varphi }_i$. The term $\mathrm{E}\left(t_i^{\prime }\right)$ captures the expected curvature of the log-likelihood function established in (26) with respect to ${\varphi }_i$.

Defining $\mathit{T}=\mathrm{diag}\{-\mathrm{E}\left(t_1^{\prime }\right),\dots ,-\mathrm{E}\left(t_n^{\prime }\right)\}$, we can express the Fisher information matrix for $\mathit{\gamma }$ in matrix form as ${\mathit{K}}_{\mathit{\gamma }\mathit{\gamma }}={\mathit{J}}_2^{\top }\mathit{T}{\mathit{H}}_{\gamma }^{-2}{\mathit{J}}_2,$ where ${\mathit{J}}_2=\partial {\eta }_2/\partial \mathit{\gamma }$ is the Jacobian of the dispersion model, containing the derivatives of the linear predictors ${\eta }_{2i}$ with respect to $\mathit{\gamma }$, and ${\mathit{H}}_{\gamma }=\mathrm{diag}\{h^{\prime }\left({\varphi }_1\right),\dots ,h^{\prime }\left({\varphi }_n\right)\}$ is a diagonal matrix of the derivatives of the link function h with respect to ${\varphi }_i$. The matrix $\mathit{T}$ encapsulates the expected information provided by each observation about the dispersion parameters.

This Fisher information matrix quantifies the amount of information the data provide about the dispersion parameters $\mathit{\gamma }$, and it is crucial for parameter estimation and inference, such as computing standard errors and confidence intervals. Under the assumption that the mean and dispersion parameters are orthogonal (that is, the cross-partial derivatives of the log-likelihood with respect to $\mathit{\beta }$ and $\mathit{\gamma }$ are zero), the Fisher information matrix for the full parameter vector $\mathit{\theta }={({\mathit{\beta }}^{\top },{\mathit{\gamma }}^{\top })}^{\top }$ is block-diagonal stated as

$${\mathit{K}}_{\mathit{\theta }\mathit{\theta }}=\mathrm{diag}\{{\mathit{K}}_{\mathit{\beta }\mathit{\beta }},{\mathit{K}}_{\mathit{\gamma }\mathit{\gamma }}\}.$$

(33)

The block-diagonal structure presented in (33) simplifies the estimation process, as it allows us to estimate $\mathit{\beta }$ and $\mathit{\gamma }$ separately without loss of efficiency.

4.3. Generalized Standardized Combined Residual

Our goal is to develop a residual that incorporates the Fisher scoring iterative process, an optimization algorithm used to obtain ML estimates. This process is represented as an iterative weighted least squares procedure, leveraging the quadratic approximation of the log-likelihood function.

For the mean parameters $\mathit{\beta }$, this iterative procedure is expressed as in (16) by

$${\mathit{\beta }}^{(m+1)}={\left({\mathit{J}}_1^{\top }{\mathit{W}}^{\left(m\right)}{\mathit{J}}_1\right)}^{-1}{\mathit{J}}_1^{\top }{\mathit{W}}^{\left(m\right)}{\mathit{u}}^{\left(m\right)},$$

where ${\mathit{W}}^{\left(m\right)}$ is the weight matrix at mth iteration, reflecting the variance of the observations and the link function, and ${\mathit{u}}^{\left(m\right)}$ is the working response vector, which incorporates the current estimates and residuals.

Similarly, for the dispersion parameters $\mathit{\gamma }$, the Fisher scoring update is given as

$${\mathit{\gamma }}^{(m+1)}={\left({\mathit{J}}_2^{\top }{\mathit{P}}^{\left(m\right)}{\mathit{J}}_2\right)}^{-1}{\mathit{J}}_2^{\top }{\mathit{P}}^{\left(m\right)}{\mathit{u}}^{*\left(m\right)},$$

(34)

where ${\mathit{P}}^{\left(m\right)}={\mathit{T}}^{\left(m\right)}{\left({\mathit{H}}_{\gamma }^{\left(m\right)}\right)}^{-2}$ is the weight matrix for the dispersion model at mth iteration. This matrix combines the expected information from ${\mathit{T}}^{\left(m\right)}$ with adjustments from the link function derivatives. The working response vector for the dispersion model, ${\mathit{u}}^{*\left(m\right)}={\mathit{J}}_2{\mathit{\gamma }}^{\left(m\right)}+{\left({\mathit{T}}^{\left(m\right)}\right)}^{-1}{\mathit{H}}_{\gamma }^{\left(m\right)}\mathit{t}$ namely, accounts for the current parameter estimates and the dispersion residuals $\mathit{t}$.

In the expression stated in (34), ${\mathit{H}}_{\gamma }^{\left(m\right)}$ and ${\mathit{T}}^{\left(m\right)}$ are evaluated using the current parameter estimates ${\mathit{\gamma }}^{\left(m\right)}$. This iterative weighted least squares procedure refines the parameter estimates by solving a sequence of linear approximations to the score equations. At each iteration, the weights and working responses are updated based on the current estimates, progressively improving the parameter estimates until convergence.

By developing a residual based on this iterative process, we can create diagnostic measures that reflect the influence of individual observations on the estimation of both mean and dispersion parameters. These residuals help to identify outliers or influential data points that may disproportionately affect the model fit.

Upon convergence of the iterative weighted least squares procedure, we obtain the estimate of the dispersion parameters $\widehat{\mathit{\gamma }}$. Specifically, the converged estimate is given by

$$\widehat{\mathit{\gamma }}={\left({\mathit{J}}_2^{\top }\widehat{\mathit{P}}{\mathit{J}}_2\right)}^{-1}{\mathit{J}}_2^{\top }\widehat{\mathit{P}}{\mathit{u}}^{*},$$

(35)

where $\widehat{\mathit{P}}$ is the weight matrix evaluated at the converged estimates, ${\mathit{J}}_2$ is the Jacobian matrix of the dispersion model, and ${\mathit{u}}^{*}$ is the working response vector for the dispersion sub-model.

The expression presented in (35) shows that $\widehat{\mathit{\gamma }}$ can be interpreted as the weighted least squares estimator of the transformed response ${\mathit{u}}^{*}$ on the covariates represented by ${\mathit{J}}_2$, with weights given by $\widehat{\mathit{P}}$.

By pre-multiplying both the response and the covariate matrix by ${\widehat{\mathit{P}}}^{1/2}$, we can view the estimation process as an OLS problem formulated as

$$\widehat{\mathit{\gamma }}={\left({\left({\widehat{\mathit{P}}}^{1/2}{\mathit{J}}_2\right)}^{\top }\left({\widehat{\mathit{P}}}^{1/2}{\mathit{J}}_2\right)\right)}^{-1}{\left({\widehat{\mathit{P}}}^{1/2}{\mathit{J}}_2\right)}^{\top }\left({\widehat{\mathit{P}}}^{1/2}{\mathit{u}}^{*}\right).$$

This formulation provides an intuitive understanding of the estimation process, emphasizing the role of the weights in adjusting for heteroscedasticity in the dispersion model.

The residuals for the dispersion parameters $\mathit{\gamma }$ can be defined based on the least squares interpretation. The ordinary residual vector for $\mathit{\gamma }$ is expressed as

$${\mathit{r}}^{\mathit{\gamma }}={\widehat{\mathit{P}}}^{1/2}({\mathit{u}}^{*}-{\mathit{J}}_2\widehat{\mathit{\gamma }}).$$

(36)

This expression interprets the residuals as the difference between the observed working responses ${\mathit{u}}^{*}$ and the fitted values ${\mathit{J}}_2\widehat{\mathit{\gamma }}$, scaled by the square root of the weight matrix $\widehat{\mathit{P}}$.

By substituting ${\mathit{u}}^{*}$ given in (34) into the definition of the residual vector ${\mathit{r}}^{\mathit{\gamma }}$ from the expression presented in (36), we obtain

$${\mathit{r}}^{\mathit{\gamma }}={\widehat{\mathit{P}}}^{1/2}\left({\mathit{J}}_2\widehat{\mathit{\gamma }}+{\widehat{\mathit{T}}}^{-1}{\widehat{\mathit{H}}}_{\gamma }\mathit{t}-{\mathit{J}}_2\widehat{\mathit{\gamma }}\right)={\widehat{\mathit{P}}}^{1/2}{\widehat{\mathit{T}}}^{-1}{\widehat{\mathit{H}}}_{\gamma }\mathit{t}.$$

Recall that the weight matrix is defined as $\widehat{\mathit{P}}=\widehat{\mathit{T}}{\widehat{\mathit{H}}}_{\gamma }^{-2}$. Then, its square root is ${\widehat{\mathit{P}}}^{1/2}={\widehat{\mathit{T}}}^{1/2}{\widehat{\mathit{H}}}_{\gamma }^{-1}$. Substituting this into the expression for the residuals, we get

$${\mathit{r}}^{\mathit{\gamma }}=\left({\widehat{\mathit{T}}}^{1/2}{\widehat{\mathit{H}}}_{\gamma }^{-1}\right){\widehat{\mathit{T}}}^{-1}{\widehat{\mathit{H}}}_{\gamma }\mathit{t}={\widehat{\mathit{T}}}^{-1/2}\mathit{t}.$$

This simplification shows that the residuals ${\mathit{r}}^{\mathit{\gamma }}$ are scaled versions of the dispersion residuals $\mathit{t}$, adjusted by the inverse square root of the expected information from the Fisher information matrix. Therefore, for each observation, the individual residual is expressed as

$$r_i^{\mathit{\gamma }}=t_i/{\left(-\mathrm{E}\left(t_i^{\prime }\right)\right)}^{1/2},i\in \{1,\dots ,n\},$$

where $-\mathrm{E}\left(t_i^{\prime }\right)$ represents the negative expected value of the derivative of $t_i$ with respect to ${\varphi }_i$, effectively capturing the curvature of the log-likelihood function with respect to the dispersion parameter.

Building on the residuals from both the mean and dispersion models, we can define a combined residual that incorporates information from both components. Specifically, the combined residual for observation i is defined as

$$r_i^{\mathit{\beta }\mathit{\gamma }}=(y_i-{\widehat{\mu }}_i)+t_i,i\in \{1,\dots ,n\},$$

(37)

reflecting the total deviation of the observed response from its expected value, accounting for both the mean and dispersion effects.

To standardize the combined residual presented in (37) and facilitate comparison across observations, we compute its variance given by

$$\begin{array}{cc}\hfill {\zeta }_i& =\mathrm{Var}\left(r_i^{\mathit{\beta }\mathit{\gamma }}\right)=\mathrm{Var}\left((y_i-{\widehat{\mu }}_i)+t_i\right)=\mathrm{Var}\left(y_i\right)+\mathrm{Var}\left(t_i\right)+2\mathrm{Cov}(y_i,t_i),\hfill \end{array}$$

(38)

where $\mathrm{Var}\left(y_i\right)={\varphi }_i{b}^{″}\left({\theta }_i\right)$ represents the variance of the response variable from the mean model, and $\mathrm{Var}\left(t_i\right)$ captures the variance of the dispersion residuals. The covariance term $\mathrm{Cov}(y_i,t_i)$ accounts for any dependency between the response variable and the dispersion residuals. Calculating $\mathrm{Var}\left(t_i\right)$ and $\mathrm{Cov}(y_i,t_i)$ in the expression defined in (38) requires careful consideration of the statistical properties of $t_i$. As defined previously in (29), $t_i$ represents the change in the log-likelihood function with respect to ${\varphi }_i$. The term $c^{\prime }(y_i;{\varphi }_i)=\partial c(y_i;{\varphi }_i)/\partial {\varphi }_i$ depends on the specific form of the function $c(y_i;{\varphi }_i)$, which is model-dependent. Without specific assumptions about $c(y_i;{\varphi }_i)$, the direct computation of $\mathrm{Var}\left(t_i\right)$ and $\mathrm{Cov}(y_i,t_i)$ can be challenging. However, these terms are crucial for accurately quantifying the combined residual variance and ensuring robust model diagnostics.

To address this challenge, we can use asymptotic approximations or empirical estimates. Under regularity conditions and assuming a large sample size, $\mathrm{Var}\left(t_i\right)$ can be approximated using the observed information as $\mathrm{Var}\left(t_i\right)\approx -\mathrm{E}\left(t_i^{\prime }\right)=-\mathrm{E}(\partial t_i/\partial {\varphi }_i).$ Similarly, $\mathrm{Cov}(y_i,t_i)$ may be approximated by considering the covariance between the score functions for $\mathit{\beta }$ and $\mathit{\gamma }$. If the mean and dispersion parameters are orthogonal (that is, their cross-derivatives are zero), the covariance term may be negligible. Substituting these approximations, the variance of the combined residual simplifies to ${\zeta }_i=\mathrm{Var}\left(y_i\right)+\mathrm{Var}\left(t_i\right)$, for $i\in \{1,\dots ,n\}$. Thus, the standardized combined residual is expressed as

$$r_{{\mathrm{S}}_i}^{\mathit{\beta }\mathit{\gamma }}={\displaystyle \frac{(y_i-{\widehat{\mu }}_i)+t_i}{\sqrt{{\varphi }_i{b}^{″}\left({\theta }_i\right)-\mathrm{E}\left(t_i^{\prime }\right)}}},i\in \{1,\dots ,n\}.$$

In practice, the unknown quantities are replaced with their estimates evaluated at the fitted values as ${\widehat{\zeta }}_i={\widehat{\varphi }}_i{b}^{″}\left({\widehat{\theta }}_i\right)-\widehat{\mathrm{E}}\left(t_i^{\prime }\right)$, for $i\in \{1,\dots ,n\}$, yielding the estimated standardized combined residual formulated as

$$r_{{\mathrm{S}}_i}^{\mathit{\beta }\mathit{\gamma }}={\displaystyle \frac{(y_i-{\widehat{\mu }}_i)+{\widehat{t}}_i}{\sqrt{{\widehat{\zeta }}_i}}},i\in \{1,\dots ,n\}.$$

(39)

This standardized residual presented in (39) combines the deviations from both the mean and dispersion models into a single metric, making it easier to identify observations that may be outliers or exert undue influence on the model fit. By appropriately scaling the combined residual, we ensure consistent interpretation across observations, facilitating effective diagnostic assessment.

In the case where the response variable follows a negative binomial distribution, calculating $\mathrm{Var}\left(t_i\right)$ and $\mathrm{Cov}(y_i,t_i)$ for the standardized combined residual may not be straightforward due to the complexity of the function $c^{\prime }(y_i;{\varphi }_i)$. This function arises from the derivative of the normalization term $c(y_i;{\varphi }_i)$ in the exponential family representation of the negative binomial distribution. To estimate $\widehat{\mathrm{Var}}\left(c^{\prime }(y_i;{\widehat{\varphi }}_i)\right)$ and $\widehat{\mathrm{Cov}}\left(y_i,c^{\prime }(y_i;{\widehat{\varphi }}_i)\right)$, we can employ numerical methods. Specifically, these estimates are computed by evaluating the expectations over the distribution of $Y_i$ using the fitted values of the model parameters. The estimated variance of $c^{\prime }(y_i;{\widehat{\varphi }}_i)$ is given by

$$\begin{array}{cc}\hfill \widehat{\mathrm{Var}}\left(c^{\prime }(y_i;{\widehat{\varphi }}_i)\right)& =\widehat{\mathrm{E}}\left(c^{\prime }{(y_i;{\widehat{\varphi }}_i)}^2\right)-{\left(\widehat{\mathrm{E}}\left(c^{\prime }(y_i;{\widehat{\varphi }}_i)\right)\right)}^2,i\in \{1,\dots ,n\},\hfill \end{array}$$

and the estimated covariance between $y_i$ and $c^{\prime }(y_i;{\widehat{\varphi }}_i)$ is represented as

$$\begin{array}{cc}\hfill \widehat{\mathrm{Cov}}\left(y_i,c^{\prime }(y_i;{\widehat{\varphi }}_i)\right)& =\widehat{\mathrm{E}}\left(y_i{c}^{\prime }(y_i;{\widehat{\varphi }}_i)\right)-\widehat{\mathrm{E}}\left(y_i\right)\widehat{\mathrm{E}}\left(c^{\prime }(y_i;{\widehat{\varphi }}_i)\right),i\in \{1,\dots ,n\}.\hfill \end{array}$$

To compute these above-mentioned expectations numerically, we use the probability function of the negative binomial distribution evaluated at the estimated parameters. Hence, the expectations are approximated as

$$\widehat{\mathrm{E}}\left(c^{\prime }(y_i;{\widehat{\varphi }}_i)\right)=\sum _{j=0}^{\infty }{c}^{\prime }(j;{\widehat{\varphi }}_i)\widehat{P}(Y_i=j),\widehat{\mathrm{E}}\left(c^{\prime }{(y_i;{\widehat{\varphi }}_i)}^2\right)=\sum _{j=0}^{\infty }{c}^{\prime }{(j;{\widehat{\varphi }}_i)}^2\widehat{P}(Y_i=j),\widehat{\mathrm{E}}\left(y_i{c}^{\prime }(y_i;{\widehat{\varphi }}_i)\right)=\sum _{j=0}^{\infty }j{c}^{\prime }(j;{\widehat{\varphi }}_i)\widehat{P}(Y_i=j),$$

where $\widehat{P}(Y_i=j)$ is the estimated probability that $Y_i$ takes the value j, given by the negative binomial probability function evaluated at the estimated parameters ${\widehat{\mu }}_i$ and ${\widehat{\varphi }}_i$, given by

$$\widehat{P}(Y_i=j)=\left({\displaystyle \genfrac{}{}{0pt}{}{j+r_i-1}{j}}\right){\left({\displaystyle \frac{{\widehat{\mu }}_i}{{\widehat{\mu }}_i+r_i}}\right)}^j{\left({\displaystyle \frac{r_i}{{\widehat{\mu }}_i+r_i}}\right)}^{r_i},$$

with $r_i=1/{\widehat{\varphi }}_i$. Since the sums over j from 0 to infinity are infinite, in practice, we truncate the sums at a sufficiently large value ${\mathit{J}}_{\max }$, where the probabilities become negligible. This truncation allows for numerical computation while maintaining accuracy.

For example, we can choose ${\mathit{J}}_{\max }$ such that ${\sum }_{j=0}^{{\mathit{J}}_{\max }}\widehat{P}(Y_i=j)\ge 1-\epsilon$, where $\epsilon$ is a small number (for instance, ${10}^{-6}$). This ensures that we capture almost all the probability mass in our calculations. By computing these sums numerically for each observation i, we obtain the estimates $\widehat{\mathrm{Var}}\left(c^{\prime }(y_i;{\widehat{\varphi }}_i)\right)$ and $\widehat{\mathrm{Cov}}\left(y_i,c^{\prime }(y_i;{\widehat{\varphi }}_i)\right)$ needed for the variance ${\widehat{\zeta }}_i$ of the combined residual. Substituting these estimates into the expression for the standardized combined residual, we have

$$r_{{\mathrm{S}}_i}^{\mathit{\beta }\mathit{\gamma }}={\displaystyle \frac{(y_i-{\widehat{\mu }}_i)+{\widehat{t}}_i}{\sqrt{{\widehat{\varphi }}_i{b}^{″}\left({\widehat{\theta }}_i\right)+\widehat{\mathrm{Var}}\left(c^{\prime }(y_i;{\widehat{\varphi }}_i)\right)+2\widehat{\mathrm{Cov}}\left(y_i,c^{\prime }(y_i;{\widehat{\varphi }}_i)\right)}}},i\in \{1,\dots ,n\}.$$

(40)

Our approach allows us to compute the standardized combined residuals defined in (40) numerically, accommodating the complexities inherent in the negative binomial distribution. Thus, we maintain the diagnostic capabilities of the residuals while ensuring that the calculations remain feasible in practice.

5. Evaluation of Model Fit Using Simulated Envelopes

This section explores methods for assessing model fit through simulated envelopes, focusing on the analysis of residuals. The proposed techniques provide a robust approach to identifying model misspecifications and aberrant observations, even when traditional residual plots may be inadequate.

5.1. Simulated Envelopes for Residual Analysis

The residuals proposed in this study do not limit to normality. Consequently, the conventional thresholds of $[-2,2]$ commonly used in residual plots for evaluating model fit against observation indices, covariate values, or predicted values may not be appropriate. To address this limitation, we use a method based on the empirical quantiles derived from simulated envelopes for residual analysis. This method serves as a helpful diagnostic tool, accurately capturing the underlying probabilistic structure of the residuals, even in situations where key assumptions regarding the data or model are not satisfied.

Simulated envelope-based probability plots are highly effective for evaluating the goodness of fit of statistical models and detecting potential outliers. Residuals that fall outside the bounds of the simulated envelope are flagged as outliers [49]. The procedure for constructing these plots is rigorously outlined in Algorithm 1. A visual representation of this algorithm is provided in the accompanying flowchart of Figure 1. To facilitate understanding of how this procedure can be implemented in practice, we provide a complete example in R code; see Appendix C. This example demonstrates the simulation and residual analysis steps discussed in this section, helping readers bridge the gap between theory and application.

Algorithm 1 Construction of simulated envelopes for residual analysis

1: Input: Fitted model, number of simulations m, sample size n 2: Output: Simulated envelopes for residuals, diagnostic plots 3: Step 1: Model fitting and initial residual calculation 4: Fit the initial model to the observed data to obtain the estimated residuals. The residuals can be ordinary residuals, deviance residuals, or any other residual type relevant to the model being analyzed. 5: Compute the residuals from the fitted model. 6: Step 2: Simulation procedure 7: for $j=1$ to m do 8:     Generate a simulated sample of n independent observations based on the fitted model parameters. 9:     Refit the model to the simulated sample. 10:   Calculate and store the absolute values of the residuals from the refitted model. 11: end for 12: Step 3: Construction of empirical quantiles 13: for $i=1$ to n do 14:    Obtain the empirical quantiles: 2.5th percentile, mean, and 97.5th percentile, for each ordered set of residuals. These quantiles represent the interval where 95% of the residuals are expected to fall, assuming the model is correctly specified. 15: end for 16: Step 4: Diagnostic plotting 17: Construct a quantile versus quantile (QQ) plot by plotting the ordered residuals from the original data against the theoretical quantiles of the standard normal distribution. 18: Overlay the simulated envelope, constructed from the empirical quantiles, onto the plot. 19: Step 5: Model fit assessment 20: Assess the model fit by evaluating whether the residuals lie inside the simulated envelope. 21: if the residuals fall within the simulated envelope then 22:     Conclude that the model provides an adequate fit. 23: else 24:     Identify potential model inadequacies. 25:     Optional step: Model check 26:  If important deviations are observed, consider checking the model. Possible check includes re-specifying the model structure, adjusting for heteroscedasticity, or considering alternative distributions. 27:     Refit the checked model and repeat Steps 1–5. 28: end if 29: Step 6: Application of model-specific diagnostics (if necessary) 30: if applicable (for example, in specific distributions such as binomial or Poisson models) then 31:     Use additional diagnostic checks relevant to the specific type of model (for example, checks for overdispersion, zero-inflation). 32: end if

The adequacy of the model fit is confirmed if, in the QQ plots with simulated envelopes, the residuals lie inside the bounds of the simulated confidence bands. This indicates that the distribution of the residuals from the model closely approximates the expected empirical distribution under the assumed model, thereby validating the model assumptions.

To evaluate the performance of the proposed residuals under different conditions for the mean and dispersion sub-models, we analyze the following scenarios:

(i)

[Model 1] $Y_i\sim \mathrm{Gamma}({\mu }_i;{\varphi }_1)$ with ${\mu }_i^{-1}={\beta }_1+x_i^{{\beta }_2}$ and $log\left({\varphi }_i\right)={\gamma }_1+z_i^{{\gamma }_2}$, where ${\beta }_1=1$, ${\beta }_2=0.8$, ${\gamma }_1=1$, ${\gamma }_2=1.5$, $x_i$ are values from $X\sim \mathrm{U}(0.1,1.1)$, $z_i$ are values from $Z\sim \mathrm{U}(0.4,1.4)$, and $i\in \{1,\dots ,80\}$.

(ii)

[Model 2] $Y_i\sim \mathrm{BN}({\mu }_i;{\varphi }_1)$ with $log\left({\mu }_i\right)={\beta }_1+x_i^{{\beta }_2}$ and $log\left({\varphi }_i\right)={\gamma }_1+z_i^{{\gamma }_2}$, where ${\beta }_1=1.6$, ${\beta }_2=0.8$, ${\gamma }_1=-0.3$, ${\gamma }_2=0.5$, $x_i$ are values from $X\sim \mathrm{U}(0.1,1.1)$, $z_i$ are values from $Z\sim \mathrm{U}(0.4,1.4)$, and $i\in \{1,\dots ,80\}$.

The choice of these parameters was made to reflect realistic scenarios where both the mean and dispersion sub-models vary according to different covariates, allowing for a comprehensive evaluation of the residuals performance under varying conditions.

Figure 2 shows plots with envelopes for Models 1 and 2, generated under the correct model and $n=80$. In these plots, the solid blue lines are the 2.5th and 97.5th percentiles of 100 simulated points for each observation, whereas the dashed blue line is the median. The empirical distributions of the residuals match the estimated distributions, indicating a good fit. Thus, we conclude that the standardized combined residual proves satisfactory performance across all analyzed scenarios, as all observed quantiles lie inside the simulated envelopes, suggesting a suitable fit.

5.2. Residual Analysis under Incorrect Model Specification

One of the common errors in modeling a dataset is assuming constant dispersion, while the real data-generating process may involve varying dispersion. Such misspecification can importantly impact residual analysis, as the empirical distribution of the residuals may deviate substantially from expectations based on correct model assumptions.

To detect this type of misspecification, we evaluate the empirical distribution of model residuals using probability plots. Typically, these plots are employed to assess model fit under correct specifications. However, when model assumptions are violated, the residual distributions can exhibit important deviations, such as skewness or kurtosis, that differ from those of a standard normal distribution. Even if the residuals appear approximately normally distributed, caution is necessary. Relying on conventional thresholds of the standard normal distribution—for example, $[-2,2]$—to identify outliers may lead to misleading conclusions, as the true residual distribution may not adhere to this standard. To address this, we propose using thresholds derived from simulated envelopes. Specifically, we employ the 2.5th and 97.5th percentiles of the empirical distribution of simulated residuals as cut-off bands in residual plots. This employment is computationally efficient because it leverages the residuals already simulated.

To illustrate our diagnostic method, we present examples where dispersion modeling is ignored, meaning that a model with constant dispersion is assumed. In these simulations, the true dispersion sub-model is given by $log\left({\varphi }_i\right)={\gamma }_1+z_i^{{\gamma }_2}$, where the covariate value $z_i=1$, for $i\in \{1,\dots ,25\}$, and $z_i=1.5$, for $i\in \{26,\dots ,50\}$. The degree of heterogeneity is defined as $\lambda =\max \{{\varphi }_1,\dots ,{\varphi }_n\}/\min \{{\varphi }_1,\dots ,{\varphi }_n\}$. Note that $\lambda$ depends on the true values of the parameters ${\gamma }_1$ and ${\gamma }_2$. For models where the response follows a continuous distribution (for example, normal, gamma, or inverse Gaussian —IG—), we consider ${\gamma }_1=1$ and ${\gamma }_2=2$, implying $\lambda \approx 20$; if ${\gamma }_2=2.46$, then $\lambda \approx 40$; and if ${\gamma }_2=3.07$, then $\lambda \approx 100$. For a response following a negative binomial distribution, we consider ${\gamma }_1=1$ and ${\gamma }_2=3.07$, implying $\lambda \approx 100$; if ${\gamma }_2=3.54$, then $\lambda \approx 200$; and if ${\gamma }_2=4$, then $\lambda \approx 400$. However, during parameter estimation, varying dispersion is disregarded, and only the mean of the response variable is modeled. Thus, we analyze the following scenarios:

(i)

[Model 1] $Y_i\sim \mathrm{Gamma}({\mu }_i;{\varphi }_i)$ with ${\mu }_i^{-1}={\beta }_1+x_i^{{\beta }_2}$, where ${\beta }_1=1$, ${\beta }_2=0.8$, and $x_i$ are values from $X\sim \mathrm{U}(0.1,1.1)$, for $i\in \{1,\dots ,50\}$.

(ii)

[Model 2] $Y_i\sim \mathrm{BN}({\mu }_i;{\varphi }_i)$ with $log\left({\mu }_i\right)={\beta }_1+x_i^{{\beta }_2}$, where ${\beta }_1=1.6$, ${\beta }_2=0.8$, and $x_i$ are values from $X\sim \mathrm{U}(0.1,1.1)$, for $i\in \{1,\dots ,50\}$.

Figure 3 and Figure 4 illustrate the behavior of residuals when the response variable follows gamma and negative binomial distributions, respectively. In both cases, the residuals disagree with the model specification, as the residuals fall outside the simulated envelope. This disagreement becomes more pronounced with increasing heterogeneity, denoted by $\lambda$. However, for the negative binomial response model, the detection of incorrect specification is more subtle. As $\lambda$ increases, the probability plots show only slight differences, likely due to the robustness of the parameter estimation process across varying dispersion parameters.

The residual plots for detecting atypical observations are presented in Figure 5 and Figure 6, which show the distribution of residuals versus observation indices for the gamma and negative binomial models, respectively. A consistent pattern is evident across all plots, initially indicating an incorrect model specification. Typically, a good model fit is characterized by randomly distributed residuals. Notably, the combined residuals more prominently highlight aberrant observations, as these points deviate further from the established cut-off bands. The proposed method using simulated envelopes offers an improvement over traditional methods of residual analysis, which often rely on fixed thresholds, such as $[-2,2]$, that assume normality of residuals. This assumption is frequently violated in complex models, especially in GLM with varying dispersion. By constructing envelopes based on empirical quantiles from simulations, our diagnostic method provides a more accurate reflection of the underlying distribution of residuals, even when traditional residual plots may fail to capture deviations from model assumptions.

The above-mentioned construction makes our method more robust and reliable for detecting model misspecifications and outliers, as demonstrated in the performance evaluation across different simulated scenarios. It is also noteworthy that the new limits for detecting outliers differ from the conventional $[-2,2]$ range in several situations. For instance, in a model with a negative binomial distribution, the upper limit is less than 2; see Figure 6.

In the previous sections, we developed and evaluated the effectiveness of various residuals for diagnosing the goodness of fit in models with both mean and dispersion components. We now transition from theoretical considerations to practical illustrations of these concepts by means of empirical applications.

6. Empirical Applications

This section presents two empirical applications where the proposed standardized combined residual is used to analyze real datasets, highlighting its utility in detecting model inadequacies and identifying potential outliers.

6.1. Application 1: Shear Force in Snack Products

The dataset analyzed in this application, as presented in [1], originates from an experiment conducted at the Nutrition Department of the Faculty of Public Health, University of São Paulo, Brazil. This experiment was conducted to evaluate a new type of snack product, specifically focusing on the impact of replacing hydrogenated vegetable fat—traditionally used to fix the aroma in snacks—with canola oil. Five different formulations of the snack were tested over a period of 20 weeks, varying in the proportions of fat and canola oil, as detailed below:

• Characteristic A—22% fat, 0% canola oil.
• Characteristic B—0% fat, 22% canola oil.
• Characteristic C—17% fat, 5% canola oil.
• Characteristic D—11% fat, 11% canola oil.
• Characteristic E—5% fat, 17% canola oil.

The experimental design involved collecting 15 samples for each formulation in even-numbered weeks, resulting in a total of 150 observations per formulation across the study period. The primary response variable in this study was the force required to shear the snack, which serves as a measure of the product texture. The covariates include the group (A, B, C, D, or E) representing the snack formulation, and the week of collection.

Step 1: 

Model fitting and initial residual calculation  

We denote $y_{ijk}$ as the shear force for the kth replicate of the ith group in the jth week, where $k\in \{1,\dots ,15\}$ represents the replicates within each group-week combination; $j\in \{2,4,6,\dots ,20\}$ denotes the even-numbered weeks during which measurements were taken, and  i corresponds to the different snack formulations, specifically groups A, B, C, D, and E. To analyze the data, we apply a joint modeling approach for both the mean and dispersion of the gamma-distributed response variable. The systematic components of the model are expressed as ${\mu }_{ij}={\beta }_0+{\beta }_i+{\beta }_6{\mathrm{week}}_j+{\beta }_7{\mathrm{week}}_j^2$ and $log\left({\varphi }_{ij}\right)={\gamma }_0+{\gamma }_i+{\gamma }_6{\mathrm{week}}_j+{\gamma }_7{\mathrm{week}}_j^2$, where the baseline parameters ${\beta }_0$ and ${\gamma }_0$ represent the effects on the mean and dispersion, respectively, for Group A (22% fat, 0% canola oil); the coefficients ${\beta }_i$ and ${\gamma }_i$ ($i\in \{B,C,D,E\}$) account for the deviations in the mean and dispersion associated with groups B, C, D, and E relative to Group A; the terms ${\beta }_6$ and ${\gamma }_6$ capture the linear effect of the week on the mean and dispersion; while ${\beta }_7$ and ${\gamma }_7$ capture potential nonlinear trends over time through the quadratic term for the week. This model structure allows us to assess the impact of different formulations and the passage of time on the texture (shear force) of the snack products, while also accounting for variability within and between different formulations.

Step 2: 

Simulation procedure  

To assess the adequacy of the model, we generate simulated samples and refit the model for each simulated sample. For each simulation, the absolute values of the residuals are computed and stored. This step follows the approach detailed in Appendix D.2, where the behavior of various residuals is empirically evaluated through Monte Carlo simulations.

Step 3: 

Construction of empirical quantiles  

After generating and refitting the model m times, we calculate the empirical quantiles (2.5th percentile, mean, and 97.5th percentile) for each ordered set of residuals, as described in Appendix D, particularly in the context of gamma-distributed response variables as shown in

Table A2. Empirical means, standard errors, skewness, and kurtosis of standardized ordinary residuals $r_p^{\beta }$, deviance $d_{\mu }({\widehat{\mu }}_i;{\widehat{\varphi }}_i)$, and combined residuals $r_p^{\beta \gamma }$ for the model: $Y_i\sim \mathrm{Gamma}({\mu }_i;{\varphi }_i)$ with ${\mu }_i^{-1}={\beta }_1+x_i^{{\beta }_2}$ and $\log \left({\varphi }_i\right)={\gamma }_i+z_i^{{\gamma }_2}$, ${\beta }_1=1$, ${\beta }_2=0.8$, ${\gamma }_1=1$, ${\gamma }_2=1.5$, $x_i$ are values from $X\sim \mathrm{U}(0.1;1.1)$, $z_i$ are values from $Z\sim \mathrm{U}(0.4;1.4)$, and $i\in \{1,\dots ,30\}$.

	Mean			Standard Error			Skewness			Kurtosis
$\mathit{i}$	${\mathit{r}}_{\mathit{p}}^{\mathit{\beta }}$	${\mathit{d}}_{\mathit{\mu }}({\widehat{\mathit{\mu }}}_{\mathit{i}};{\widehat{\mathit{\varphi }}}_{\mathit{i}})$	${\mathit{r}}_{\mathit{p}}^{\mathit{\beta }\mathit{\gamma }}$	${\mathit{r}}_{\mathit{p}}^{\mathit{\beta }}$	${\mathit{d}}_{\mathit{\mu }}({\widehat{\mathit{\mu }}}_{\mathit{i}};{\widehat{\mathit{\varphi }}}_{\mathit{i}})$	${\mathit{r}}_{\mathit{p}}^{\mathit{\beta }\mathit{\gamma }}$	${\mathit{r}}_{\mathit{p}}^{\mathit{\beta }}$	${\mathit{d}}_{\mathit{\mu }}({\widehat{\mathit{\mu }}}_{\mathit{i}};{\widehat{\mathit{\varphi }}}_{\mathit{i}})$	${\mathit{r}}_{\mathit{p}}^{\mathit{\beta }\mathit{\gamma }}$	${\mathit{r}}_{\mathit{p}}^{\mathit{\beta }}$	${\mathit{d}}_{\mathit{\mu }}({\widehat{\mathit{\mu }}}_{\mathit{i}};{\widehat{\mathit{\varphi }}}_{\mathit{i}})$	${\mathit{r}}_{\mathit{p}}^{\mathit{\beta }\mathit{\gamma }}$
1	−0.0681	−0.1901	−0.0633	1.0069	1.0473	1.0426	0.6518	0.0229	−1.6062	0.3642	−0.2360	2.9941
2	0.1376	0.0352	0.1295	1.2066	1.1792	0.9105	0.3711	−0.0892	−0.6799	−0.1334	−0.3099	0.2738
3	−0.0911	−0.2193	−0.0826	0.9845	1.0308	1.0341	0.6961	0.0606	−1.4265	0.3802	−0.2877	2.4229
4	−0.0654	−0.1620	−0.0551	1.0403	1.0691	1.0100	0.4759	0.0433	−1.2266	−0.1053	−0.4049	1.3695
5	0.0408	−0.1035	0.0218	1.0884	1.0788	0.8775	0.7183	0.0388	−1.8128	0.3847	−0.3716	3.5617
6	−0.0752	−0.1766	−0.0710	1.0448	1.0744	1.0416	0.5371	0.0464	−1.2865	0.1252	−0.3143	1.6389
7	0.0568	−0.0897	0.0466	1.1075	1.0969	0.9690	0.6480	−0.0024	−0.9617	0.1305	−0.4083	0.8442
8	0.0802	−0.0409	0.0756	1.1169	1.0936	0.9398	0.6013	−0.0229	−0.8558	0.3787	−0.2179	0.7340
9	0.1150	−0.0191	0.1128	1.0787	1.0251	0.6807	0.6784	0.0249	−2.0631	0.3947	−0.3186	5.3635
10	0.0056	−0.1215	0.0204	1.0252	1.0285	0.8790	0.6721	0.0293	−1.9264	0.3551	−0.2699	4.7902
11	−0.0916	−0.2250	−0.0857	0.9613	1.0251	1.0574	0.6735	−0.0072	−1.2530	0.3850	−0.2520	1.6858
12	−0.0650	−0.1516	−0.0471	1.0229	1.0411	0.9241	0.4629	0.0932	−0.5624	−0.2520	−0.5480	−0.1751
13	0.0638	−0.0612	0.0579	1.0820	1.0748	0.9571	0.5854	−0.0347	−0.9877	0.1551	−0.2563	1.1088
14	−0.0942	−0.2283	−0.0873	0.9743	1.0290	1.0520	0.7399	0.0479	−1.3609	0.5323	−0.2109	2.1490
15	0.0059	−0.1462	−0.0058	1.0640	1.0737	0.9096	0.7334	0.0395	−1.7826	0.3235	−0.4162	3.4976
16	−0.0241	−0.1645	−0.0194	1.0013	1.0359	0.9927	0.6848	−0.0055	−1.1316	0.3598	−0.3201	1.2526
17	0.0183	−0.1264	0.0210	1.0286	1.0448	0.9516	0.6639	−0.0099	−1.0117	0.1783	−0.3976	1.0217
18	0.0409	−0.0710	0.0315	1.0911	1.0818	0.9075	0.5661	0.0108	−1.9305	0.1144	−0.2480	5.1510
19	0.0227	−0.1132	0.0254	1.0315	1.0427	0.9057	0.6252	−0.0231	−1.8073	0.2184	−0.3521	3.6561
20	0.0523	−0.0366	0.0622	1.1244	1.1179	0.8100	0.3460	−0.0199	−1.4416	−0.3247	−0.4876	2.1293
21	0.2360	0.1138	0.2038	1.2859	1.2057	0.8969	0.4666	−0.0862	−0.7408	0.0220	−0.2888	0.4351
22	−0.0534	−0.2005	−0.0506	1.0163	1.0415	0.9697	0.7783	0.0948	−1.3372	0.4787	−0.3966	1.7910
23	−0.1021	−0.1970	−0.0892	1.0148	1.0521	1.0181	0.4988	0.0796	−1.0082	−0.0953	−0.4596	0.7226
24	−0.0968	−0.1919	−0.0987	1.0644	1.1025	1.0552	0.4335	0.0694	−0.8332	−0.3531	−0.6255	0.2011
25	−0.1021	−0.2108	−0.1049	1.0456	1.0892	1.0939	0.5640	0.0527	−1.3735	0.0698	−0.3166	2.0412
26	−0.0757	−0.1747	−0.0750	1.0639	1.1027	1.0617	0.4483	0.0358	−1.1779	−0.2370	−0.4699	1.1893
27	−0.0344	−0.1896	−0.0438	1.0327	1.0671	0.9761	0.7311	0.0517	−1.4981	0.2856	−0.4978	2.2192
28	0.1027	−0.0232	0.0702	1.1020	1.0902	0.8793	0.5139	−0.1023	−2.1642	0.0722	−0.2594	5.7557
29	−0.0879	−0.2298	−0.0737	0.9649	1.0198	1.0101	0.7244	0.0576	−1.2519	0.3863	−0.3727	1.4555
30	0.0895	−0.0173	0.0767	1.2022	1.1978	0.9942	0.3846	−0.0833	−0.7137	−0.1861	−0.3712	0.2669

.

Step 4: 

Diagnostic plotting  

Then, we plot the ordered residuals from the original data against the theoretical quantiles of the standard normal distribution. The simulated envelopes are overlaid on the plot, as shown in Figure 7. This aligns with the simulation-based diagnostics discussed in Appendix D, which confirm the effectiveness of using simulated envelopes for assessing model adequacy.

Step 5: 

Model fit assessment  

The parameter estimates provided in

Table 2. ML estimates for the parameters from the joint modeling of mean and dispersion with a gamma response fitted to the snack data.

	Mean			Dispersion
Effect	Estimate	Standard Error	p-Value	Estimate	Standard Error	p-Value
Constant	36.990	3.208	0.00	−1.560	0.214	0.00
Group B	−10.783	1.684	0.00	−0.477	0.161	0.00
Group C	−3.478	1.763	0.00	−0.050	0.160	0.07
Group D	−14.829	1.614	0.00	−0.815	0.161	0.00
Group E	−15.198	1.622	0.00	−0.817	0.161	0.00
week	5.198	0.525	0.00	−0.155	0.039	0.00
week2	−0.189	0.212	0.00	0.005	0.001	0.00

reveal that Group A exhibits the highest shear force, indicating a firmer texture compared to the other groups, particularly groups D and E, which show the lowest shear forces. Additionally, Group A shows the highest variability in terms of the coefficient of variation. The standardized combined residual introduced in this study integrates information from both the mean and dispersion models, providing a more comprehensive diagnostic tool. Figure 7 presents the probability plot with simulated envelopes for these residuals, finding no evidence of model misfit, as the residuals fall within the expected bounds. This finding is consistent with the results from the Monte Carlo simulations presented in Appendix D.2, which validate the reliability of the standardized combined residual for diagnosing model adequacy in similar scenarios.

6.2. Application 2: Root Production in Apple Shoots

In this application, we examine a dataset provided in [50], which explores the standardized combined residual when the model response variable is discrete. The data come from an experiment on apple crops, focusing on the number of roots produced by 270 micropropagated shoots. These shoots were grown under two different photoperiod conditions (8 or 16 h of light exposure) and five varying concentrations of benzylaminopurine (BAP) cytokinin. The variables used in the model are defined as follows:

• $Y_i$—Number of roots for the ith shoot;
• $X_{i1}$—Photoperiod (0 = 8 h, 1 = 16 h);
• Group 1—BAP cytokinin concentration of 2.2;
• Group 2—BAP cytokinin concentration of 4.4;
• Group 3—BAP cytokinin concentration of 8.8;
• Group 4—BAP cytokinin concentration of 17.6;
• Group 5—BAP cytokinin concentration of 18.6.

Step 1: 

Model fitting and initial residual calculation  

We perform joint modeling of the mean and dispersion for the response variable, assuming a negative binomial distribution. The systematic component of the model is specified as $log\left({\mu }_i\right)={\beta }_0+{\beta }_1{x}_{i1}+{\beta }_j$ and $log\left({\varphi }_{ij}\right)={\gamma }_0+{\gamma }_1{x}_{i1}+{\gamma }_j$, where the index i represents the individual micropropagated shoot, for $i\in \{1,\dots ,270\}$. Each shoot is subject to one of two photoperiod conditions (8 or 16 h of light exposure) and one of five BAP cytokinin concentration groups, whereas the index j corresponds to the BAP cytokinin concentration group, for $j\in \{1,\dots ,5\}$, representing the five levels of cytokinin concentration tested in the experiment. The parameter ${\beta }_0$ represents the baseline effect on the mean number of roots (response variable) for Group 1 (BAP concentration of 2.2) under the 8-hour photoperiod condition. Similarly, ${\gamma }_0$ represents the baseline effect on the dispersion for the same group and condition. The coefficients ${\beta }_1$ and ${\gamma }_1$ capture the effect of increasing the photoperiod from 8 to 16 h on the mean and dispersion, respectively. The terms ${\beta }_j$ and ${\gamma }_j$ account for deviations in the mean and dispersion associated with each BAP concentration group relative to Group 1. The choice of setting ${\beta }_2=0$ and ${\gamma }_2=0$ implies that Group 1 serves as the reference group, allowing us to interpret the other groups’ effects as deviations from this baseline. This model structure enables us to quantify both the direct effect of photoperiod on root production and how this effect varies across different cytokinin concentrations, providing a comprehensive understanding of the factors influencing root production in the experiment.

Step 2: 

Simulation procedure  

To evaluate the model fit, we generate simulated samples and refit the model for each simulated dataset. For each simulation, the absolute values of the residuals are computed and stored. This procedure is aligned with the approach detailed in Appendix D, where similar simulations were conducted to assess the behavior of residuals in negative binomial models; see

Table A4. Empirical means, standard errors, skewness, and kurtosis of standardized ordinary residuals $r_p^{\beta }$, deviance residuals $d_{\mu }({\widehat{\mu }}_i;{\widehat{\varphi }}_i)$, and combined residuals $r_p^{\beta \gamma }$ for the model: $Y_i\sim \mathrm{NB}({\mu }_i;{\varphi }_1)$ with $\log \left({\mu }_i\right)={\beta }_1+x_i^{{\beta }_2}$ and $\log \left({\varphi }_i\right)={\gamma }_i+z_i^{{\gamma }_2}$, ${\beta }_1=1.6$, ${\beta }_2=0.8$, ${\gamma }_1=-0.3$, ${\gamma }_2=0.5$, $x_i$ are values from $X\sim \mathrm{U}(0.1;1.1)$, $z_i$ are values from $Z\sim \mathrm{U}(0.4;1.4)$, and $i\in \{1,\dots ,30\}$.

	Mean			Standard Error			Skewness			Kurtosis
$\mathit{i}$	${\mathit{r}}_{\mathit{p}}^{\mathit{\beta }}$	${\mathit{d}}_{\mathit{\mu }}({\widehat{\mathit{\mu }}}_{\mathit{i}};{\widehat{\mathit{\varphi }}}_{\mathit{i}})$	${\mathit{r}}_{\mathit{p}}^{\mathit{\beta }\mathit{\gamma }}$	${\mathit{r}}_{\mathit{p}}^{\mathit{\beta }}$	${\mathit{d}}_{\mathit{\mu }}({\widehat{\mathit{\mu }}}_{\mathit{i}};{\widehat{\mathit{\varphi }}}_{\mathit{i}})$	${\mathit{r}}_{\mathit{p}}^{\mathit{\beta }\mathit{\gamma }}$	${\mathit{r}}_{\mathit{p}}^{\mathit{\beta }}$	${\mathit{d}}_{\mathit{\mu }}({\widehat{\mathit{\mu }}}_{\mathit{i}};{\widehat{\mathit{\varphi }}}_{\mathit{i}})$	${\mathit{r}}_{\mathit{p}}^{\mathit{\beta }\mathit{\gamma }}$	${\mathit{r}}_{\mathit{p}}^{\mathit{\beta }}$	${\mathit{d}}_{\mathit{\mu }}({\widehat{\mathit{\mu }}}_{\mathit{i}};{\widehat{\mathit{\varphi }}}_{\mathit{i}})$	${\mathit{r}}_{\mathit{p}}^{\mathit{\beta }\mathit{\gamma }}$
1	0.0800	−0.1576	0.0784	1.0543	1.0440	1.0303	1.1830	−0.0145	1.0567	1.6297	−0.2099	1.2593
2	−0.1285	−0.3632	−0.1148	0.9790	1.0834	0.9020	0.8964	−0.1011	0.7846	0.6302	−0.3266	0.3963
3	0.0521	−0.1940	0.0507	1.0299	1.0495	1.0129	1.1202	−0.0737	0.9810	1.4170	−0.2828	1.0352
4	0.0664	−0.1639	0.0637	1.0782	1.0744	1.0498	1.0030	0.0108	0.9080	0.8986	−0.4082	0.6631
5	−0.0020	−0.2557	−0.0025	1.0275	1.0544	0.9994	1.1614	0.0250	1.0509	1.3966	−0.3572	1.0897
6	0.0681	−0.1624	0.0659	1.0745	1.0688	1.0474	1.0042	0.0051	0.9082	0.8893	−0.3812	0.6530
7	−0.0718	−0.3220	−0.0684	0.9586	1.0343	0.9302	1.0829	−0.0360	0.9278	1.2114	−0.3675	0.7840
8	−0.1105	−0.3527	−0.1037	0.9465	1.0401	0.9102	1.0490	−0.0747	0.8936	1.0519	−0.3181	0.6783
9	−0.1257	−0.3551	−0.1152	0.8751	0.9842	0.8423	0.9571	−0.1475	0.8891	0.9414	−0.3212	0.8004
10	0.0243	−0.2173	0.0238	1.0112	1.0400	0.9847	1.0850	−0.0388	0.9744	1.1698	−0.3251	0.8798
11	−0.0044	−0.2560	−0.0046	1.0181	1.0488	1.0032	1.2489	0.0140	1.0970	1.7721	−0.2085	1.3381
12	−0.0398	−0.2538	−0.0381	0.9745	1.0296	0.9511	0.8840	−0.0380	0.7806	0.5482	−0.4332	0.3003
13	−0.0670	−0.3078	−0.0641	0.9572	1.0285	0.9328	1.1419	−0.0571	0.9733	1.4541	−0.2718	0.9966
14	0.0434	−0.2018	0.0431	1.0196	1.0434	1.0044	1.1190	−0.0618	0.9772	1.3927	−0.2649	1.0077
15	0.0335	−0.2275	0.0310	1.0548	1.0662	1.0233	1.1033	0.0519	1.0104	1.1712	−0.5602	0.9200
16	−0.0280	−0.2797	−0.0269	0.9978	1.0400	0.9800	1.2202	0.0170	1.0566	1.8418	−0.3072	1.3151
17	−0.0786	−0.3355	−0.0763	0.9710	1.0383	0.9467	1.1916	0.0270	1.0297	1.5911	−0.3855	1.1109
18	0.0472	−0.1847	0.0453	1.0313	1.0588	1.0002	0.9745	−0.1008	0.8750	0.8118	−0.2650	0.5907
19	0.0579	−0.1931	0.0562	1.0403	1.0593	1.0149	1.0319	−0.0463	0.9250	0.8743	−0.3993	0.6241
20	−0.0049	−0.2136	−0.0073	1.0074	1.0394	0.9587	0.8862	0.0134	0.8140	0.5088	−0.5098	0.3754
21	−0.1951	−0.4552	−0.1689	0.9736	1.1166	0.8616	0.9830	−0.1202	0.8286	1.0077	−0.3096	0.6065
22	0.0570	−0.1967	0.0560	1.0409	1.0551	1.0235	1.0913	−0.0007	0.9649	1.1634	−0.4341	0.8314
23	0.0643	−0.1635	0.0620	1.0729	1.0719	1.0470	0.9652	−0.0001	0.8688	0.8063	−0.4161	0.5635
24	0.0578	−0.1561	0.0571	1.0357	1.0446	1.0123	0.9089	−0.0156	0.8136	0.6062	−0.4052	0.3901
25	0.0559	−0.1771	0.0540	1.0592	1.0703	1.0342	1.0613	−0.0404	0.9528	1.2137	−0.2377	0.9341
26	0.0624	−0.1613	0.0604	1.0590	1.0676	1.0318	0.9458	−0.0259	0.8515	0.7540	−0.3824	0.5307
27	0.0724	−0.1926	0.0694	1.0702	1.0731	1.0468	1.0881	0.0216	0.9785	1.0999	−0.5273	0.8026
28	−0.0610	−0.2829	−0.0571	0.9281	1.0002	0.8965	0.9891	−0.1037	0.8868	0.8720	−0.2565	0.6463
29	0.0468	−0.2085	0.0457	1.0385	1.0554	1.0217	1.1541	−0.0216	1.0146	1.3559	−0.2962	0.9869
30	−0.0928	−0.3137	−0.0850	0.9592	1.0384	0.9066	1.0092	−0.0420	0.8788	1.0390	−0.2298	0.7176

.

Step 3: 

Construction of empirical quantiles  

After generating and refitting the model m times, we calculate the empirical quantiles (2.5th percentile, mean, and 97.5th percentile) for each ordered set of residuals, consistent with the methodology applied in Appendix D.

Step 4: 

Diagnostic plotting  

Then, we plot the ordered residuals from the original data against the theoretical quantiles of the standard normal distribution. The simulated envelopes are overlaid on the plot, as shown in Figure 8. This technique of using simulated envelopes is validated in Appendix D, where the reliability of the standardized combined residuals in the context of negative binomial models is confirmed; see Table A4.

Step 5: 

Model fit assessment  

The parameter estimates reported in

Table 3. ML estimates for the parameters in the joint modeling of the mean and dispersion with a negative binomial response fitted to the apple dataset.

	Mean			Dispersion
Effect	Estimate	Standard Error	p-Value	Estimate	Standard Error	p-Value
Constant	1.78	0.091	0.00	2.34	0.527	0.00
Photoperiod	−0.92	0.151	0.00	−3.79	0.463	0.00
Group 2	0.26	0.119	0.03	0.46	0.485	0.34
Group 3	0.23	0.110	0.03	0.92	0.491	0.06
Group 4	0.17	0.112	0.12	0.69	0.484	0.15
Group 5	−0.87	1.013	0.16	10.08	105.6	0.92

indicate that Group 1 exhibits the highest root production among the shoots, while groups 4 and 5 have the lowest root counts. Additionally, Group 1 shows the greatest variability in terms of the coefficient of variation. The probability plot of the standardized combined residual, shown in Figure 8, strongly suggests that the proposed model does not adequately fit the data.

Given the inadequacy of the initial model, we propose a simplified systematic component that only accounts for the photoperiod effect formulated as follows:

$$\begin{array}{cc}\hfill log\left({\mu }_i\right)& ={\beta }_0+{\beta }_1{x}_{i1},log\left({\varphi }_{ij}\right)={\gamma }_0+{\gamma }_1{x}_{i1}.\hfill \end{array}$$

(41)

Table 4. ML estimates for the parameters in the simplified joint modeling of the mean and dispersion with a negative binomial response fitted to the apple dataset.

	Mean			Dispersion
Effect	Estimate	Standard Error	p-Value	Estimate	Standard Error	p-Value
Constant	1.96	0.419	0.00	2.77	0.038	0.00
Photoperiod	−0.91	0.458	0.00	−3.68	0.152	0.00

presents the ML estimates for the simplified model specified in (41).

The probability plot for the standardized combined residual from the checked model is presented in Figure 9. Unlike Figure 8, where most points are outside the simulated envelopes, the checked model probability plot shows that only a few points, particularly in the tails, are outside the envelopes. This suggests that the checked model provides a much better fit to the data, consistent with findings from the simulations of the negative binomial model presented in Appendix D; see Table A4.

7. Conclusions and Future Work

In this work, we introduced a new residual—the standardized combined residual—designed for classes of linear and nonlinear models within the exponential family, aimed at evaluating the goodness of fit in various scenarios. The need for this new residual arises from the limitations observed in traditional residuals, particularly when dealing with models that incorporate both mean and dispersion sub-models. Traditional residuals often fall short in adequately capturing the complexities of such models, especially in the presence of heteroscedasticity or nonconstant variance.

The standardized combined residual offers several advantages over existing methods. An important advantage is its construction, which integrates the Fisher iterative scoring process for both the mean and dispersion models. This allows for a unified diagnostic analysis, as it considers a single group of residuals that encapsulate information from both sub-models, rather than requiring separate analyses for each component. Our approach simplifies the evaluation process, particularly in generalized linear models and other models where variance is not constant. Moreover, the proposed residual does not require the computation of projection matrices, which are often computationally intensive and challenging to interpret. This results in reduced computational costs and enhances the practicality of the diagnostic process. The standardized combined residual also behaves similarly to the deviance residual, which is widely used in practical applications, making it an accessible tool for practitioners. A notable feature of the standardized combined residual is its independence from the method of maximization used to estimate model parameters. Whether using traditional optimization techniques or more advanced methods like simulated annealing [51], the residual relies solely on the parameter estimates, making it versatile and robust across different scenarios.

The empirical applications presented in this article, including the analysis of shear force in snack products and root production in apple shoots, showed the utility of the standardized combined residual in practical settings. In both cases, the new residual provided clear insights into model fit and highlighted inadequacies that were not as evident with traditional residuals. Specifically, the application to the apple shoot data underscored the importance of considering both the mean and dispersion in the model, as the initial model was found to be inadequate when evaluated with the new residual.

However, the standardized combined residual has certain limitations. Its effectiveness depends on the correct specification of both the mean and dispersion models. In scenarios where either sub-model is misspecified, the residual may not fully capture the inadequacies. Additionally, while it is computationally less intensive than methods requiring projection matrices, it still relies on accurate parameter estimation, which can be challenging in complex models or with limited data.

While the detailed derivation of certain asymptotic properties of the estimators is not the primary focus of this work, the developed framework provides clear insights into the robustness of the methods employed. The estimators, obtained through maximum likelihood methods and the Fisher scoring procedure, are expected to exhibit desirable properties, such as consistency and asymptotic normality, under regular conditions presented in Appendix B. The regular conditions applied, particularly in the context of exponential family distributions, support the notion that the proposed methodology provides accurate estimates for large samples, as further elaborated in the appendix. This reinforces the practical utility of the estimators, even in the absence of formal proofs of all properties in the present work.

Although the standardized combined residual offers several advantages, particularly in simplifying the diagnostic process and reducing computational costs, some trade-offs are present. By combining the information from both the mean and dispersion sub-models into a single residual, some level of detail may be sacrificed compared to using separate residuals for each component. However, this trade-off was a deliberate decision to balance simplicity with practicality, aiming to create an efficient and unified diagnostic tool. For most practical applications, the benefits of a straightforward interpretation and reduced computational burden outweigh the potential loss of specificity from separate residual analysis.

Future work should explore the extension of this residual to other classes of models, particularly those outside the exponential family. There is also potential for further refinement of the residual to enhance its robustness against model misspecifications. Additionally, the development of software tools to automate the computation and interpretation of the standardized combined residual would greatly benefit practitioners, making advanced diagnostic methods more accessible in applied settings.

In summary, while the standardized combined residual offers a unified and computationally efficient method for evaluating models that incorporate both mean and dispersion components, it also balances the trade-off between simplicity and the potential loss of granularity. Nonetheless, its development represents a step forward in statistical diagnostics, with the potential for broader applications and continued refinement in future research.