Partial Least Squares Regression for Binary Data

Abstract

Classical Partial Least Squares Regression (PLSR) models were developed primarily for continuous data, allowing dimensionality reduction while preserving relationships between predictors and responses. However, their application to binary data is limited. This study introduces Binary Partial Least Squares Regression (BPLSR), a novel extension of the PLSR methodology designed specifically for scenarios involving binary predictors and responses. BPLSR adapts the classical PLSR framework to handle the unique properties of binary datasets. A key feature of this approach is the introduction of a triplot representation that integrates logistic biplots. This visualization tool provides an intuitive interpretation of relationships between individuals and variables from both predictor and response matrices, enhancing the interpretability of binary data analysis. To illustrate the applicability and effectiveness of BPLSR, the method was applied to a real-world dataset of strains of Colletotrichum graminicola, a pathogenic fungus. The results demonstrated the ability of the method to represent binary relationships between predictors and responses, underscoring its potential as a robust analytical tool. This work extends the capabilities of traditional PLSR methods and provides a practical and versatile solution for binary data analysis with broad applications in diverse research areas.

1. Introduction

Due to the great technological advances nowadays, it is increasingly common to find datasets that contain a large number of variables that have been measured in a given sample. The set of variables can range from quantitative to binary, nominal or ordinal, and, very often, a combination of several types.

The rapid growth of data science has led to an increasing demand for statistical techniques that can handle large and complex datasets.

In this context, traditional univariate statistical techniques, repeated for each variable, do not capture the structure of the data, especially the relationships among variables, so it is necessary to use multivariate techniques that offer a more powerful approach. These methods focus on the study of relationships among variables, similarities and differences among individuals and the variables responsible for them, treating all available information simultaneously.

Probably the most used multivariate statistical technique is Principal Component Analysis (PCA). Pearson [1] proposed PCA over 120 years ago, which was later formalized by Hotelling [2,3]. PCA is a dimensionality reduction technique that can be used to summarize the information contained in a large dataset. PCA works by finding a set of orthogonal axes, called principal components, that capture the most variation in the data. It was not until a few years later, in 1971, that K.R. Gabriel [4] proposed a simultaneous graphical representation of the rows and columns of a data matrix, related to PCA, which he called the biplot.

This type of methods allow for visualizing the structure of the data, avoiding making use only of a list of p-values associated with the performed analysis. Biplots can also allow the visualization of more complex multivariate models such as Canonical Variate Analysis or Multivariate Analysis of Variance (MANOVA) [5,6].

All of these techniques were developed for continuous variables, but when the measured variables are of a categorical type, either binary, nominal or ordinal, these representations are not adequate.

There are several dimension reduction methods for categorical data, for example, Item Response Theory (see, for example, [7]) or Factor Analysis for categorical data (see, for example, [8]), but none of these techniques have an associated biplot representation.

We will focus here on biplot representations based on logistic responses that are closely related to the mentioned techniques. It is worth highlighting the work of Vicente-Villardon et al. [9], Demey et al. [10] who developed a biplot for a single matrix of binary data. These proposals are based on the generalized bivariate models proposed by Gabriel [11].

In many practical situations, we have two or more data matrices, with symmetric or asymmetric roles. In this work, we focus on the particular case in which we have two data matrices with asymmetric roles, a set of predictor variables and a set of response variables. Generally, the goal when we have this situation is to find models that allow predicting the responses from the predictors. In both datasets, it is possible to have variables of all the types mentioned above.

For two multivariate data matrices with one playing the role of responses, one of the most used multivariate statistical techniques is Partial Least Squares Regression (PLSR) [12,13]. PLSR is a regression technique that can be used to predict a set of response variables from a set of predictor variables. PLSR works by finding a set of linear combinations of the predictor variables that are most correlated with the response variables. The classical PLSR models were developed mainly for continuous data and to obtain dimension reductions in both responses and predictors, in a similar way to PCA with the difference being that the focus is on prediction rather than variance maximization. A biplot representation for PLSR that can help in the interpretation of the results was proposed by Oyedele and Lubbe [14].

More recently, Vicente-Gonzalez and Vicente-Villardon [15] proposed a PLSR method where all the responses are binary and predictors are continuous. This extends PLS generalized linear regression [16] for a single response variable (PLS1) to several responses (PLS2) in a non-trivial way because it implies a dimension reduction not present in the former work. This proposal goes with a biplot for both matrices, a continuous one for the predictors and a logistic one for the binary responses.

In the context of genomic data, [17,18] proposed a procedure for a single response and numerical predictors but using Iteratively Re-weighted Least Squares (IRWLS), similar to the procedures of [16]. The work of Vicente-Gonzalez and Vicente-Villardon [15] is also an extension of those for several responses.

When the two matrices are binary, none of the procedures mentioned before are suitable. In this paper, we propose a PLSR (PLS2) procedure when both matrices, responses and predictors, have binary variables. We use the logistic biplots, mentioned before, to make a triplot representation that combines individuals and variables of the two initial matrices.

Our proposal can be framed on the PLS regression method as described in [19] in contrast with other approaches as Structural Equation Modeling (SEM) with mainly two methods: covariance-based SEM (CB-SEM) and Partial Least Squares SEM (PLS-SEM). Following [20], CB-SEM is used to check (confirm or reject) theories and their underlying hypothesis by determining how closely a proposed theoretical model can reproduce the covariance matrix for an observed sample dataset. On the other hand, PLS is a causal predictive approach to SEM that focuses on explaining the variance of the dependent variables.

CB-SEM may work with any kind of data, including binary, but the purpose of the method is explaining covariances and not the variables. As pointed by [20] (p. 11), PLS-SEM is similar but not equivalent to PLSR as described in [19,21]. PLSR is a regression-based method that explores the (linear) relationships between multiple independent variables and single or multiple dependent variables. It is different from regression because the method derives composite factors for independent variables. It differs from PLS-SEM because this relies on prespecified networks of relationships between constructs, as well as between the constructs and their measures. For a more complete comparison of PLS-SEM and PLSR, you can check [22].

Our proposal is then related to the PLS regression method, using a generalized linear model for several binary responses and including also a biplot representation for the interpretation of the results.

The database that initially motivated the development of this method is studying the relation among the origins of nine types of corn strains and the presence or absence of genes. The data are composed of two binary matrices, the first one containing indicators of the nine classes and the second a set of binary genetic polymorphisms.

In Section 2, we describe Partial Least Squares and the generalization to two sets of binary variables; Section 3 describes biplots and its construction for the PLS models, and Section 4 presents an example to illustrate the proposed models. The conclusions are in Section 5. Section 6 contains some software notes and the availability of the data.

2. Partial Least Squares

Let ${\mathbf{X}}_{I\times J}$ be a multivariate matrix of predictors with J variables and ${\mathbf{Y}}_{I\times K}$ be a multivariate matrix of responses with K variables, both measured in I individuals.

When we have only a few predictors (fewer than the number of individuals) that are not strongly correlated, Multivariate Linear Regression (MLR) could be employed to predict the responses. However, when the predictors are highly collinear or when there is a large number of them, traditional linear regression models may be inadequate because of the absence of parameter estimates. This situation commonly arises, among others, in genomic studies, where a set of thousands of gene expressions is available as independent variables.

Partial Least Squares Regression (PLSR) [12,13,19] is one of the most recognized and widely applied methods for modeling the relationships between two datasets, particularly when both sets of variables are continuous. The method is particularly useful in Chemometrics [21], although it has been applied in many other disciplines.

When the responses are continuous, Oyedele and Lubbe [14] introduced an associated biplot representation, although earlier, less formal versions exist, such as in [23]. More recently, PLS-Biplot has been applied to team effectiveness [24].

When the response matrix contains binary variables, Partial Least Squares Discriminant Analysis (PLS-DA) [25] is often employed, which essentially fits a PLS regression to a dummy or fictitious variable. Bastien et al. [16] introduced a PLS model for a single binary response, analogous to the PLS-1 model. More recently, Vicente-Gonzalez and Vicente-Villardon [15] proposed a non-trivial extension for multiple binary variables, incorporating dimension reduction for binary data and resulting in a PLS-2 model, which serves as an alternative to PLS-DA.

In regression, to address the discriminant problem, logistic regression (LR) can be used instead of Multivariate Linear Regression, allowing the binary nature of the responses to be captured. While this is often a valid approach, LR faces the same limitations as MLR regarding the number of individuals, variables, and collinearity. Additionally, the separation problem may also arise.

Currently, no PLSR alternative exists for situations where both the predictor and response matrices are binary.

The aim of this section is to extend PLSR to cases where both the predictor and response matrices consist of binary variables, by using logistic functions instead of linear ones. This approach, referred to as Binary Partial Least Squares Regression (BPLSR), enables us to account for the binary nature of the responses and addresses the limitations of traditional PLSR methods when dealing with binary data.

This generalization is based on the adaptation of the NIPALS algorithm for the binary data proposed in previous works by Vicente-Gonzalez and Vicente-Villardon [15]. The adaptation consists of replacing the linear regression step in the NIPALS algorithm by a logistic regression step. This allows the procedure to capture the binary nature of the data. Previously, we made an outline of the classical PLS-2 model.

2.1. Partial Least Squares Regression: NIPALS Algorithm

PLS seeks to identify linear combinations of continuous predictors $\mathbf{X}$ that optimally predict a set of continuous responses $\mathbf{Y}$. Here, we outline the classical NIPALS algorithm [26]. Both sets are usually column-centered and probably standardized. PLS operates by extracting latent variables that capture the most significant variance and covariance between predictors and responses. It uses the iterative extraction of components to build a predictive model that can handle collinearity and high-dimensional data effectively.

The latent model for predictors $\mathbf{X}$, in S dimensions, is

$$\mathbf{X}=\mathbf{T}{\mathbf{P}}^T+\mathbf{E}=\widehat{\mathbf{X}}+\mathbf{E},$$

(1)

where ${\mathbf{P}}_{J\times S}$ are variable loadings, ${\mathbf{T}}_{I\times S}$ are scores for individuals, and ${\mathbf{E}}_{I\times J}$ is a matrix of residuals.

In the same way, $\mathbf{Y}$ is the latent model

$$\mathbf{Y}=\mathbf{T}{\mathbf{Q}}^T+\mathbf{F}=\widehat{\mathbf{Y}}+\mathbf{F},$$

(2)

where ${\mathbf{Q}}_{K\times S}$ are loadings, ${\mathbf{T}}_{I\times S}$ are scores, and ${\mathbf{F}}_{I\times K}$ are residuals.

Observe that both models share the matrix of scores that best predict the responses. The NIPALS algorithm to obtain the parameters can be described as follows:

Note that by removing the step where predictors are updated in the NIPALS algorithm, we would end up with the principal components for the responses. The procedure alternatively updates the scores in matrix $\mathbf{T}$ using the information of responses and predictors.

Here, ${\mathbf{P}}^T\mathbf{P}=\mathbf{I}$ and ${\mathbf{Q}}^T\mathbf{Q}=\mathbf{I}$, and $\mathbf{T}$ contains the scores from $\mathbf{X}$ that best explain the set of responses. We have the relationship

$$\mathbf{T}=\mathbf{X}\mathbf{P}.$$

(3)

A potential issue with this and subsequent algorithms is that they may have multiple starting points, leading to different solutions. The algorithm generates sequences of decreasing residual sum of squares values and eventually converges to at least a local minimum.

If we need the regression coefficients to express the $\mathbf{Y}$ variables as functions of the predictors $\mathbf{X}$, we have  

$$\widehat{\mathbf{Y}}=\mathbf{X}\mathbf{B}.$$

(4)

Considering that $\widehat{\mathbf{Y}}=\mathbf{T}{\mathbf{Q}}^T$, we have

$$\widehat{\mathbf{Y}}=\mathbf{T}{\mathbf{Q}}^T=\mathbf{X}\mathbf{P}{\mathbf{Q}}^T.$$

(5)

Thus, the regression coefficients in terms of the original variables are

$$\mathbf{B}=\mathbf{P}{\mathbf{Q}}^T.$$

(6)

We have to note that the updating steps in the Algorithm 1 minimize the sum of squares of the residuals that can also be understood as a lost function:

$$\sum _{i=1}^I\sum _{j=1}^J{(y_{ij}-{\widehat{y}}_{ij})}^2=\sum _{i=1}^I\sum _{j=1}^J{(y_{ij}-\sum _{s=1}^S{\mathbf{t}}_{is}^t{\mathbf{q}}_{js})}^2=tr\left({(\mathbf{Y}-\widehat{\mathbf{Y}})}^T(\mathbf{Y}-\widehat{\mathbf{Y}})\right)=tr\left({(\mathbf{Y}-\mathbf{T}{\mathbf{Q}}^T)}^T(\mathbf{Y}-\mathbf{T}{\mathbf{Q}}^T)\right)$$

(7)

for the responses and in a similar way for the predictors.

Algorithm 1 NIPALS algorithm.

1: procedure NIPALS($\mathbf{X},\mathbf{Y},S$) 2:     for $s=1\to S$ do 3:         ${\mathbf{t}}_{\left(s\right)}\leftarrow {\mathbf{y}}_{\left(j\right)}$ for some j. Or any other choice.                          ▹ Init: ${\mathbf{t}}_{\left(s\right)}$ 4:         repeat 5:            ${\mathbf{p}}_{\left(s\right)}\leftarrow {\mathbf{X}}^T{\mathbf{t}}_{\left(s\right)}/\Vert {\mathbf{X}}^T{\mathbf{t}}_{\left(s\right)}\Vert$                                                   ▹ Update: ${\mathbf{p}}_{\left(s\right)}$ 6:            ${\mathbf{t}}_{\left(s\right)}\leftarrow {\mathbf{Xp}}_{\left(s\right)}$                                                                  ▹ Update: ${\mathbf{t}}_{\left(s\right)}$ 7:            ${\mathbf{q}}_{\left(s\right)}\leftarrow {\mathbf{Y}}^T{\mathbf{t}}_{\left(s\right)}/\Vert {\mathbf{Y}}^T{\mathbf{t}}_{\left(s\right)}\Vert$                                                   ▹ Update: ${\mathbf{q}}_{\left(s\right)}$ 8:            ${\mathbf{t}}_{\left(s\right)}\leftarrow {\mathbf{Yq}}_{\left(s\right)}$                                                                  ▹ Update: ${\mathbf{t}}_{\left(s\right)}$ 9:         until ${\mathbf{t}}_{\left(s\right)}$ does not change 10:         $\mathbf{X}\leftarrow \mathbf{X}-{\mathbf{t}}_{\left(s\right)}{\mathbf{p}}_{\left(s\right)}^T$                                                                   ▹ Update: $\mathbf{X}$ 11:         $\mathbf{Y}\leftarrow \mathbf{Y}-{\mathbf{t}}_{\left(s\right)}{\mathbf{q}}_{\left(s\right)}^T$                                                                  ▹ Update: $\mathbf{Y}$     return $\mathbf{T}=\left[{\mathbf{t}}_{\left(1\right)},\dots ,{\mathbf{t}}_{\left(S\right)}\right]$, $\mathbf{P}=\left[{\mathbf{p}}_{\left(1\right)},\dots ,{\mathbf{p}}_{\left(S\right)}\right]$ and $\mathbf{Q}=\left[{\mathbf{q}}_{\left(1\right)},\dots ,{\mathbf{q}}_{\left(S\right)}\right]$

2.2. PLS for Binary Responses

When responses are binary, the updating steps of the NIPALS algorithm (steps 7 and 8) are inadequate because the implicit linear relationship it assumes is not adequate. This is similar to how linear regression is unsuitable for binary responses, with logistic regression being the more appropriate method. The difference in step 11 makes also no sense.

Recently, Vicente-Gonzalez and Vicente-Villardon [15] generalized the NIPALS algorithm to handle a set of binary responses by incorporating a logistic relationship with the latent variables.

The expected values for the responses are now probabilities, and we use the logit function as the link function. Let $\widehat{\mathbf{Y}}={\mathbf{\Pi }}^y$, then we have

$$\mathrm{logit}\left({\mathbf{\Pi }}^y\right)=\mathbf{1}{\mathbf{q}}_0^T+\mathbf{T}{\mathbf{Q}}^T.$$

(8)

This equation generalizes Equation (2), except that we need to include a vector ${\mathbf{q}}_0$ with intercepts for each variable, as binary variables cannot be centered in the same way as continuous. Each probability ${\pi }_{ik}^y$ can be expressed as

$${\pi }_{ik}^y={\displaystyle \frac{e^{\left(q_{k0}+{\sum }_{s=1}^S{t}_{is}{q}_{ks}\right)}}{1+e^{\left(q_{k0}+{\sum }_{s=1}^S{t}_{is}{q}_{ks}\right)}}}.$$

(9)

Or in matrix form, if we have

$${\mathbf{M}}^y=\mathbf{1}{\mathbf{q}}_0^T+\mathbf{T}{\mathbf{Q}}^T,$$

(10)

then

$${\mathbf{\Pi }}^y=e^{{\mathbf{M}}^y}÷(1+e^{{\mathbf{M}}^y}),$$

(11)

where the operations apply to each element of the matrices.

The updating steps of the responses have to be changed for binary responses. Rather than minimizing the cost function in Equation (7), we minimize the cost defined as

$$L^y=\sum _{i=1}^I\sum _{k=1}^K\left[-y_{ik}log\left({\pi }_{ik}^y\right)-(1-y_{ik})log\left(1-{\pi }_{ik}^y\right)\right].$$

(12)

We interpret here the function as a cost to minimize rather than a likelihood to maximize. We look for the parameters $\mathbf{T}$, $\mathbf{Q}$, and ${\mathbf{q}}_0$ that minimize the cost function. There are not closed-form solutions for the optimization problem, so an iterative algorithm, in which we obtain a sequence decreasing values of the lost function at each iteration, is used. We use the gradient method in a recursive way calculating one component at each time fixing the previous one. The update for each parameter is as follows:

$$\begin{array}{c}\hfill q_{j0}\Leftarrow q_{j0}-\alpha {\displaystyle \frac{\partial L^y}{\partial q_{j0}}}=q_{j0}-\alpha \sum _{i=1}^I({\pi }_{ij}^y-y_{ij});\end{array}$$

(13)

$$\begin{array}{c}\hfill t_{is}\Leftarrow t_{is}-\alpha {\displaystyle \frac{\partial L^y}{\partial t_{is}}}=t_{is}-\alpha \sum _{j=1}^J{q}_{js}({\pi }_{ij}^y-y_{ij});\end{array}$$

(14)

$$\begin{array}{c}\hfill q_{js}\Leftarrow q_{js}-\alpha {\displaystyle \frac{\partial L^y}{\partial q_{js}}}=q_{js}-\alpha \sum _{i=1}^I{t}_{is}({\pi }_{ij}^y-y_{ik}),\end{array}$$

(15)

for some $\alpha$. Or, in matrix form,

$$\begin{array}{c}\hfill {\mathbf{q}}_0\Leftarrow {\mathbf{q}}_0-\alpha {({\mathbf{\Pi }}^y-\mathbf{Y})}^T{\mathbf{1}}_I;\end{array}$$

(16)

$$\begin{array}{c}\hfill {\mathbf{t}}_{\left(s\right)}\Leftarrow {\mathbf{t}}_{\left(s\right)}-\alpha ({\mathbf{\Pi }}^y-\mathbf{Y}){\mathbf{q}}_{\left(s\right)};\end{array}$$

(17)

$$\begin{array}{c}\hfill {\mathbf{q}}_{\left(s\right)}\Leftarrow {\mathbf{q}}_{\left(s\right)}-\alpha {({\mathbf{\Pi }}^y-\mathbf{Y})}^T{\mathbf{t}}_{\left(s\right)},\end{array}$$

(18)

where ${\mathbf{t}}_{\left(s\right)}=(t_{1s},\dots ,t_{Is})$ and ${\mathbf{q}}_{\left(s\right)}=(q_{1s},\dots ,q_{Js})$. Then, the adaptation of the algorithm is straightforward. We present a recursive procedure that calculates one component at a time while keeping the previously computed components fixed.

In practice, the choice of $\alpha$ can be avoided by using a pre-programmed optimization routine. This algorithm is implemented in the package MultBiplotR [27] developed in the R language [28].

In the same way as for the continuous case, the regression coefficients in terms of the original variables are

$$\mathbf{B}=\mathbf{P}{\mathbf{Q}}^T.$$

(19)

Algorithm 2 is essentially the same as Algorithm 1 except that we have made the necessary adaptations to cope with binary responses. Now, the relation between the manifest and latent variables is logistic rather than linear. Observe that is also a linear relation on the logits, i.e., a generalized linear model.

Algorithm 2 NIPALS-Binary responses.

1: procedure NIPALSBin($\mathbf{X},\mathbf{Y},S,\alpha$, $ϵ$) 2:     Set $\mathbf{T}=\left[\right]$, $\mathbf{P}=\left[\right]$ and $\mathbf{Q}=\left[\right]$ (empty matrices) 3:     ${\mathbf{q}}_0={\displaystyle \frac{1}{I}}{\mathbf{Y}}^T{\mathbf{1}}_I$ or any other good guess.                                        ▹ Init: ${\mathbf{q}}_0$ 4:     repeat 5:         ${\mathbf{\Pi }}^y=e^{\mathbf{1}{\mathbf{q}}_{\mathbf{0}}^{\mathbf{T}}}÷(1+e^{\mathbf{1}{\mathbf{q}}_0^T})$ 6:         ${\mathbf{q}}_0\Leftarrow {\mathbf{q}}_0-\alpha {({\mathbf{\Pi }}^y-\mathbf{Y})}^T{\mathbf{1}}_I$                                                     ▹ Update: ${\mathbf{q}}_0$ 7:     until ${\mathbf{q}}_0$ does not change 8:     for $s=1\to S$ do 9:         Init: ${\mathbf{t}}_{\left(s\right)}\leftarrow random$ or any other good guess. 10:         Init: ${\mathbf{q}}_{\left(s\right)}\leftarrow random$ or any other good guess. 11:         repeat 12:            ${\mathbf{p}}_{\left(s\right)}\leftarrow {\mathbf{X}}^T{\mathbf{t}}_{\left(s\right)}/\Vert {\mathbf{X}}^T{\mathbf{t}}_{\left(s\right)}\Vert$                                                 ▹ Update: ${\mathbf{p}}_{\left(s\right)}$ 13:            ${\mathbf{t}}_{\left(s\right)}\leftarrow {\mathbf{Xp}}_{\left(s\right)}$                                                                ▹ Update: ${\mathbf{t}}_{\left(s\right)}$ 14:            $\mathbf{Q}=[\mathbf{Q};{\mathbf{q}}_{\left(s\right)}]$, $\mathbf{P}=[\mathbf{P};{\mathbf{p}}_{\left(s\right)}]$ and $\mathbf{T}=[\mathbf{T};{\mathbf{t}}_{\left(s\right)}]$▹ Add column ${\mathbf{q}}_{\left(s\right)}$ to $\mathbf{Q}$, ${\mathbf{p}}_{\left(s\right)}$ to $\mathbf{P}$ and ${\mathbf{t}}_{\left(s\right)}$ to $\mathbf{T}$ 15:            ${\mathbf{M}}^y=\mathbf{1}{\mathbf{q}}_0^T+\mathbf{T}{\mathbf{Q}}^T$ 16:            ${\mathbf{\Pi }}^y=e^{{\mathbf{M}}^y}÷(1+e^{{\mathbf{M}}^y})$ 17:            $L_{old}^y\leftarrow (-1){\mathbf{1}}_I^T((\mathbf{Y}\odot {\mathbf{\Pi }}^y)+((1-\mathbf{Y})\odot (1-{\mathbf{\Pi }}^y))){\mathbf{1}}_J$ 18:            repeat 19:                ${\mathbf{q}}_{\left(s\right)}\Leftarrow {\mathbf{q}}_{\left(s\right)}-\alpha {({\mathbf{\Pi }}^y-\mathbf{Y})}^T{\mathbf{t}}_{\left(s\right)}$ 20:                ${\mathbf{q}}_s\leftarrow {\mathbf{q}}_{\left(s\right)}$       ▹ Change the $s-th$ column ${\mathbf{q}}_s$ of $\mathbf{Q}$ for the new guess ${\mathbf{q}}_{\left(s\right)}$ 21:                ${\mathbf{M}}^y=\mathbf{1}{\mathbf{q}}_0^T+\mathbf{T}{\mathbf{Q}}^T$ 22:                ${\mathbf{\Pi }}^y=e^{{\mathbf{M}}^y}÷(1+e^{{\mathbf{M}}^y})$ 23:            until ${\mathbf{q}}_{\left(s\right)}$ does not change 24:            repeat 25:                ${\mathbf{t}}_{\left(s\right)}\Leftarrow {\mathbf{t}}_{\left(s\right)}-\alpha ({\mathbf{\Pi }}^y-\mathbf{Y}){\mathbf{q}}_{\left(s\right)}$ 26:                ${\mathbf{t}}_s\leftarrow {\mathbf{t}}_{\left(s\right)}$       ▹ Change the $s-th$ column ${\mathbf{t}}_s$ of $\mathbf{T}$ for the new guess ${\mathbf{t}}_{\left(s\right)}$ 27:                ${\mathbf{M}}^y=\mathbf{1}{\mathbf{q}}_0^T+\mathbf{T}{\mathbf{Q}}^T$ 28:                ${\mathbf{\Pi }}^y=e^{{\mathbf{M}}^y}÷(1+e^{{\mathbf{M}}^y})$ 29:            until ${\mathbf{t}}_{\left(s\right)}$ does not change 30:            $L_{new}^y\leftarrow (-1){\mathbf{1}}_I^T((\mathbf{Y}\odot {\mathbf{\Pi }}^y)+((1-\mathbf{Y})\odot (1-{\mathbf{\Pi }}^y))){\mathbf{1}}_J$ 31:         until $\mid L_{old}^y-L_{new}^y\mid <ϵ$ 32:         $\mathbf{X}\leftarrow \mathbf{X}-{\mathbf{t}}_{\left(s\right)}{\mathbf{p}}_{\left(s\right)}^T$                                                               ▹ Update: $\mathbf{X}$     return ${\mathbf{q}}_0$, $\mathbf{T}=\left[{\mathbf{t}}_{\left(1\right)},\dots ,{\mathbf{t}}_{\left(S\right)}\right]$, $\mathbf{P}=\left[{\mathbf{p}}_{\left(1\right)},\dots ,{\mathbf{p}}_{\left(S\right)}\right]$ and $\mathbf{Q}=\left[{\mathbf{q}}_{\left(1\right)},\dots ,{\mathbf{q}}_{\left(S\right)}\right]$

For a direct extension of this algorithm, see [29], implemented in the package BiplotML [30].

2.3. Partial Least Squares Regression for Two Binary Matrices

Now both ${\mathbf{X}}_{I\times J}$ and ${\mathbf{Y}}_{I\times K}$ contain binary variables measured in the same I individuals. The goal is still predicting $\mathbf{Y}$ from $\mathbf{X}$ and describing the common structure of predictors and responses.

We denote the expected values of $\mathbf{X}$ as $E\left[\mathbf{X}\right]={\mathbf{\Pi }}^x$ and the expected values of $\mathbf{Y}$ as $E\left[\mathbf{Y}\right]={\mathbf{\Pi }}^y$. Then, we have

$$\mathrm{logit}\left({\mathbf{\Pi }}^{\mathrm{y}}\right)=\mathbf{1}{\mathbf{q}}_0^T+\mathbf{T}{\mathbf{Q}}^T;$$

(20)

and

$$\mathrm{logit}\left({\mathbf{\Pi }}^{\mathrm{x}}\right)=\mathbf{1}{\mathbf{p}}_0^T+\mathbf{T}{\mathbf{P}}^T.$$

(21)

These Equations are generalizations of the decompositions of the matrices $\mathbf{X}$ and $\mathbf{Y}$ for the continuous case in Equations (1) and (2). The only difference is that we need to add the intersection vector for each variable (${\mathbf{p}}_0$ and ${\mathbf{q}}_0$, respectively). This is because, unlike the continuous case, the variables cannot be centered beforehand.

The relation among the observed and latent variables can be written as

$${\pi }_{ik}^y={\displaystyle \frac{e^{\left(q_{k0}+{\sum }_{s=1}^S{t}_{is}{q}_{ks}\right)}}{1+e^{\left(q_{k0}+{\sum }_{s=1}^S{t}_{is}{q}_{ks}\right)}}};{\pi }_{ij}^x={\displaystyle \frac{e^{\left(p_{j0}+{\sum }_{s=1}^S{t}_{is}{p}_{js}\right)}}{1+e^{\left(p_{j0}+{\sum }_{s=1}^S{t}_{is}{p}_{js}\right)}}}.$$

(22)

That is, it is a logistic rather than linear relation.

Now, we have to design an algorithm to calculate the model parameters. The procedure alternatively updates the common scores in $\mathbf{T}$ using the information of responses and predictors. We use the gradient method and the lost functions as follows:

$$L_y=\sum _{i=1}^I\sum _{k=1}^K\left[-y_{ik}log\left({\pi }_{ik}^y\right)-(1-y_{ik})log\left(1-{\pi }_{ik}^y\right)\right],$$

(23)

and

$$L_x=\sum _{i=1}^I\sum _{j=1}^J\left[-x_{ij}log\left({\pi }_{ij}^x\right)-(1-x_{ij})log\left(1-{\pi }_{ij}^x\right)\right].$$

(24)

We update the scores using a gradient descent procedures as in Equations (16)–(18) for the responses. The extension for predictors in $\mathbf{T}$ is straightforward:

$$\begin{array}{c}\hfill {\mathbf{p}}_0\Leftarrow {\mathbf{p}}_0-\alpha {({\mathbf{\Pi }}^x-\mathbf{X})}^T{\mathbf{1}}_I\end{array}$$

(25)

$$\begin{array}{c}\hfill {\mathbf{t}}_{\left(s\right)}\Leftarrow {\mathbf{t}}_{\left(s\right)}-\alpha ({\mathbf{\Pi }}^x-\mathbf{X}){\mathbf{p}}_{\left(s\right)}\end{array}$$

(26)

$$\begin{array}{c}\hfill {\mathbf{p}}_{\left(s\right)}\Leftarrow {\mathbf{p}}_{\left(s\right)}-\alpha {({\mathbf{\Pi }}^x-\mathbf{X})}^T{\mathbf{t}}_{\left(s\right)},\end{array}$$

(27)

NIPALS Algorithm for Two Binary Matrices

Then, we will search for the parameters $\mathbf{T}$, $\mathbf{P}$ y ${\mathbf{p}}_0$ for the predictors and $\mathbf{T}$, $\mathbf{Q}$ y ${\mathbf{q}}_0$ for the responses. We will obtain the scores $\mathbf{T}$ for $\mathbf{X}$ that better predict $\mathbf{Y}$ with a logistic regression model. The scores are also related to predictors with a logistic response model.

The calculations can be organized in an alternating algorithm that calculates the parameters ${\mathbf{t}}_{\left(s\right)}=(t_{1s},\dots ,t_{Is})$, ${\mathbf{p}}_{\left(s\right)}=(p_{1s},\dots ,p_{Js})$ and ${\mathbf{q}}_{\left(s\right)}=(q_{1s},\dots ,q_{Ks})$ for each dimension s, while fixing the parameters already obtained in the previous dimensions. First, we need to calculate the constant (dimension 0) ${\mathbf{p}}_{\left(0\right)}=(p_{10},\dots ,p_{J0})$ and ${\mathbf{q}}_{\left(0\right)}=(p_{10},\dots ,q_{K0})$ separately. This is so because data cannot be centered beforehand.

The algorithm is as follows.

In logistic regression, there is a potential problem known as separation. This occurs when the data are perfectly separable by a linear combination of the predictors. In this case, the maximum likelihood estimator does not exist and tends to infinity [31]. Even if the separation is not perfect (quasi-separation), the estimator can be highly unstable.

To avoid separation, the usual solution is to use a penalized likelihood [32]. This adds a penalty term to the likelihood function that encourages the coefficients to be smaller. In our case, we will use the Ridge penalty [33]:

$$L^y=\sum _{i=1}^I\sum _{k=1}^K\left[-y_{ik}log\left({\pi }_{ik}^y\right)-(1-y_{ik})log\left(1-{\pi }_{ik}^y\right)\right]+\lambda \sum _{k=1}^K\sum _{s=1}^S{q}_{ks}^2.$$

Gradient adaptation for the penalized cost function is automatic:

$${\displaystyle \frac{\partial L^y}{\partial q_{ks}}}=\sum _{i=1}^I{t}_{is}({\pi }_{ik}^y-y_{ik})+2\lambda q_{ks}.$$

To address the separation problem, we can introduce a regularization parameter $\lambda$. This parameter penalizes large coefficients, making the solution less likely to be separated.

The BPLSR algorithm (Algorithm 3) can be sensitive to the random start. To obtain the best possible solution, it is recommended to test several starting points. In practice, it has been observed that the solutions from different starting points produce similar predictions of the responses.

Algorithm 3 PLSR for two binary matrices: BPLSR.

1: procedure BPLSR($\mathbf{X},\mathbf{Y},S,\alpha$, $ϵ$) 2:     Set $\mathbf{T}=\left[\right]$, $\mathbf{P}=\left[\right]$ and $\mathbf{Q}=\left[\right]$ (empty matrices) 3:     ${\mathbf{q}}_0={\displaystyle \frac{1}{I}}{\mathbf{Y}}^T{\mathbf{1}}_I$ and ${\mathbf{p}}_0={\displaystyle \frac{1}{I}}{\mathbf{X}}^T{\mathbf{1}}_I$ or any other good guesses.                        ▹ Init: ${\mathbf{q}}_0$ and ${\mathbf{p}}_0$ 4:     repeat 5:         ${\mathbf{\Pi }}^y=e^{\mathbf{1}{\mathbf{q}}_{\mathbf{0}}^{\mathbf{T}}}÷(1+e^{\mathbf{1}{\mathbf{q}}_0^T})$ 6:         ${\mathbf{q}}_0\Leftarrow {\mathbf{q}}_0-\alpha {({\mathbf{\Pi }}^y-\mathbf{Y})}^T{\mathbf{1}}_I$                                                                            ▹ Update: ${\mathbf{q}}_0$ 7:     until ${\mathbf{q}}_0$ does not change 8:     repeat 9:         ${\mathbf{\Pi }}^x=e^{\mathbf{1}{\mathbf{p}}_{\mathbf{0}}^{\mathbf{T}}}÷(1+e^{\mathbf{1}{\mathbf{p}}_0^T})$ 10:         ${\mathbf{p}}_0\Leftarrow {\mathbf{p}}_0-\alpha {({\mathbf{\Pi }}^x-\mathbf{X})}^T{\mathbf{1}}_I$                                                                            ▹ Update: ${\mathbf{p}}_0$ 11:     until ${\mathbf{p}}_0$ does not change 12:     for $s=1\to S$ do 13:         Init: ${\mathbf{t}}_{\left(s\right)}$, ${\mathbf{q}}_{\left(s\right)}$ and ${\mathbf{p}}_{\left(s\right)}$ with random values or any other good guesses. 14:         $\mathbf{Q}=[\mathbf{Q};{\mathbf{q}}_{\left(s\right)}]$, $\mathbf{P}=[\mathbf{P};{\mathbf{p}}_{\left(s\right)}]$ and $\mathbf{T}=[\mathbf{T};{\mathbf{t}}_{\left(s\right)}]$    ▹ Add column ${\mathbf{q}}_{\left(s\right)}$ to $\mathbf{Q}$, ${\mathbf{p}}_{\left(s\right)}$ to $\mathbf{P}$ and ${\mathbf{t}}_{\left(s\right)}$ to $\mathbf{T}$ 15:         repeat 16:            ${\mathbf{M}}^x=\mathbf{1}{\mathbf{p}}_0^T+\mathbf{T}{\mathbf{P}}^T$ 17:            ${\mathbf{\Pi }}^x=e^{{\mathbf{M}}^x}÷(1+e^{{\mathbf{M}}^x})$ 18:            repeat 19:                ${\mathbf{p}}_{\left(s\right)}\Leftarrow {\mathbf{p}}_{\left(s\right)}-\alpha {({\mathbf{\Pi }}^x-\mathbf{X})}^T{\mathbf{t}}_{\left(s\right)}$ 20:                ${\mathbf{p}}_s\leftarrow {\mathbf{p}}_{\left(s\right)}$       ▹ Change the $s-th$ column ${\mathbf{p}}_s$ of $\mathbf{P}$ for the new guess ${\mathbf{p}}_{\left(s\right)}$ 21:                ${\mathbf{M}}^x=\mathbf{1}{\mathbf{p}}_0^T+\mathbf{T}{\mathbf{P}}^T$ 22:                ${\mathbf{\Pi }}^x=e^{{\mathbf{M}}^x}÷(1+e^{{\mathbf{M}}^x})$ 23:            until ${\mathbf{p}}_{\left(s\right)}$ does not change 24:            repeat 25:                ${\mathbf{t}}_{\left(s\right)}\Leftarrow {\mathbf{t}}_{\left(s\right)}-\alpha ({\mathbf{\Pi }}^x-\mathbf{X}){\mathbf{p}}_{\left(s\right)}$ 26:                ${\mathbf{t}}_s\leftarrow {\mathbf{t}}_{\left(s\right)}$       ▹ Change the $s-th$ column ${\mathbf{t}}_s$ of $\mathbf{T}$ for the new guess ${\mathbf{t}}_{\left(s\right)}$ 27:                ${\mathbf{M}}^x=\mathbf{1}{\mathbf{p}}_0^T+\mathbf{T}{\mathbf{P}}^T$ 28:                ${\mathbf{\Pi }}^x=e^{{\mathbf{M}}^x}÷(1+e^{{\mathbf{M}}^x})$ 29:            until ${\mathbf{t}}_{\left(s\right)}$ does not change 30:            ${\mathbf{M}}^y=\mathbf{1}{\mathbf{q}}_0^T+\mathbf{T}{\mathbf{Q}}^T$ 31:            ${\mathbf{\Pi }}^y=e^{{\mathbf{M}}^y}÷(1+e^{{\mathbf{M}}^y})$ 32:            $L_{old}^y\leftarrow (-1){\mathbf{1}}_I^T((\mathbf{Y}\odot {\mathbf{\Pi }}^y)+((1-\mathbf{Y})\odot (1-{\mathbf{\Pi }}^y))){\mathbf{1}}_J$ 33:            repeat 34:                ${\mathbf{q}}_{\left(s\right)}\Leftarrow {\mathbf{q}}_{\left(s\right)}-\alpha {({\mathbf{\Pi }}^y-\mathbf{Y})}^T{\mathbf{t}}_{\left(s\right)}$ 35:                ${\mathbf{q}}_s\leftarrow {\mathbf{q}}_{\left(s\right)}$▹ Change the $s-th$ column ${\mathbf{q}}_s$ of $\mathbf{Q}$ for the new guess ${\mathbf{q}}_{\left(s\right)}$ 36:                ${\mathbf{M}}^y=\mathbf{1}{\mathbf{q}}_0^T+\mathbf{T}{\mathbf{Q}}^T$ 37:                ${\mathbf{\Pi }}^y=e^{{\mathbf{M}}^y}÷(1+e^{{\mathbf{M}}^y})$ 38:            until ${\mathbf{q}}_{\left(s\right)}$ does not change 39:            repeat 40:                ${\mathbf{t}}_{\left(s\right)}\Leftarrow {\mathbf{t}}_{\left(s\right)}-\alpha ({\mathbf{\Pi }}^y-\mathbf{Y}){\mathbf{q}}_{\left(s\right)}$ 41:                ${\mathbf{t}}_s\leftarrow {\mathbf{t}}_{\left(s\right)}$       ▹ Change the $s-th$ column ${\mathbf{t}}_s$ of $\mathbf{T}$ for the new guess ${\mathbf{t}}_{\left(s\right)}$ 42:                ${\mathbf{M}}^y=\mathbf{1}{\mathbf{q}}_0^T+\mathbf{T}{\mathbf{Q}}^T$ 43:                ${\mathbf{\Pi }}^y=e^{{\mathbf{M}}^y}÷(1+e^{{\mathbf{M}}^y})$ 44:            until ${\mathbf{t}}_{\left(s\right)}$ does not change 45:            $L_{new}^y\leftarrow (-1){\mathbf{1}}_I^T((\mathbf{Y}\odot {\mathbf{\Pi }}^y)+((1-\mathbf{Y})\odot (1-{\mathbf{\Pi }}^y))){\mathbf{1}}_J$ 46:         until $\mid L_{old}^y-L_{new}^y\mid <ϵ$     return ${\mathbf{p}}_0$, ${\mathbf{q}}_0$, $\mathbf{T}=\left[{\mathbf{t}}_{\left(1\right)},\dots ,{\mathbf{t}}_{\left(S\right)}\right]$, $\mathbf{P}=\left[{\mathbf{p}}_{\left(1\right)},\dots ,{\mathbf{p}}_{\left(S\right)}\right]$ and $\mathbf{Q}=\left[{\mathbf{q}}_{\left(1\right)},\dots ,{\mathbf{q}}_{\left(S\right)}\right]$

The BPLSR algorithm can also suffer from the separation problem. A good strategy could be trying different values of $\lambda$ and checking how the values of the lost change as a function of the regularization parameter. We could also check the change in the values of the model parameter with changes in the regularization parameter as is normally performed in Ridge Regression. We include the regularization parameter in the routines for the calculation of the algorithms. A good strategy could be also using generalized cross-validation as in [34]. As a general rule, the regularization parameter depends on the size of the matrices. We are working in simulations to identify good values of the regularization parameter in different scenarios.

If we eliminate the part where ${\mathbf{p}}_0$ and $\mathbf{P}$ are updated in Algorithms 2 and 3, we obtain a reduced dimension approximation of a single data matrix analogous to a multidimensional logistic two-parameter model in Item Response Theory; see [35,36]. It is also similar to Principal Component Analysis for binary data as in [37,38,39,40], or to Factor Analysis for binary data [8].

3. Biplots

A biplot [4] is a type of graph that simultaneously displays two sets of information: the scores of the observations and the loadings of the variables in a reduced-dimensional space. It is commonly used in techniques like Principal Component Analysis (PCA) and Partial Least Squares (PLS) regression, among many others, to visualize high-dimensional data in two or three dimensions.

Scores represent the projections of the original data points (observations) onto a reduced number of components or latent variables. Loadings indicate the contributions of the original variables to these components, showing how each variable relates to the reduced dimensions.

In a biplot, observations (samples) are often represented by points, while the loadings (variables) are shown as vectors (arrows). The angles and lengths of the arrows help interpret how variables are correlated and how they contribute to the variation in the data.

Formally, a biplot represents the approximate decomposition of a data matrix $\mathbf{X}$ into the product of two lower-rank matrices (typically two or three) as follows:

$$\mathbf{X}=E\left[\mathbf{X}\right]+\mathbf{E}=\widehat{\mathbf{X}}+\mathbf{E}=\mathbf{A}{\mathbf{B}}^T+\mathbf{E}$$

(28)

This decomposition approximates the original matrix as accurately as possible based on specific criteria, with $\mathbf{A}$ and $\mathbf{B}$ capturing the main structure and $\mathbf{E}$ representing the residuals or error. In fact, a biplot decomposes the expected approximate structure of the data matrix into the product of two other matrices. The matrix $\widehat{\mathbf{X}}$ captures the majority of the data’s characteristics based on a predefined criterion.

Matrices $\mathbf{A}$ and $\mathbf{B}$ can be represented as points into a two- or three-dimensional space in such a way that the inner product

$$x_{ij}\approx {\widehat{x}}_{ij}={\mathbf{a}}_i^T{\mathbf{b}}_j$$

(29)

approximates the element $x_{ij}$.

Biplots were initially proposed in connection to PCA [4] but have been used also accompanying other techniques as Multivariate Analysis of Variance (MANOVA) [5,41], Correspondence Analysis [42] among many others. In the context of PLS for continuous data, it is clear that Equations (1) and (2) define a biplot. The properties of those representations can be developed by Oyedele and Lubbe [14] and Vicente-Gonzalez and Vicente-Villardon [15].

Most biplots are based on the assumption of linear relationships between observed and latent variables and are primarily designed for continuous data. For binary data, however, these representations are not ideal, much like how linear regression struggles to accurately model relationships with a binary response.

Vicente-Villardon et al. [9] introduces a method where the relationship between the observed variables and the dimensions is modeled using a logistic approach. This method is linked to psychometric techniques, such as Item Response Theory or latent trait analysis. The original paper proposes an algorithm based on alternating generalized regressions using the Newton–Raphson method to maximize the likelihood. However, the algorithm faces the same challenge as logistic regression when separation occurs. Specifically, for some binary variables, individuals displaying the presence of a characteristic are fully separated from those without it (by a hyperplane) in the final representation. This method is referred to as the logistic biplot.

Demey et al. [10] introduced a new algorithm that combines Principal Component Analysis, Cluster Analysis, and Logistic Regression to create this type of biplot. The result is termed an “external logistic biplot” because it uses a two-step approach (principal coordinates for individuals and logistic regressions for variables) rather than simultaneously obtaining the row and column markers. This simplification helps to avoid the separation problem but comes at the cost of reduced goodness of fit.

More recently, Babativa-Márquez and Vicente-Villardón [29] proposed an algorithm that improves parameter fitting by employing the nonlinear conjugate gradient algorithm.

More recently, in the context of PLS methods, Vicente-Gonzalez and Vicente-Villardon [15] proposed a generalization of the NIPALS algorithm for binary responses, where the components are obtained iteratively using the gradient method. The paper also introduced a procedure to address the separation issue encountered in earlier algorithms for logistic biplots.

3.1. Logistic Biplots

Linear biplots decompose the expected values of the approximation into a product of two lower-rank matrices. In contrast, the logistic biplot decomposes the expected probabilities using the logit link function, similar to how generalized linear models operate.

For any binary data matrix ${\mathbf{X}}_{I\times J}$, if we call $E\left[\mathbf{X}\right]=\mathbf{\Pi }$ the decomposition, using the logit link, we have

$$\mathrm{logit}\left(\mathbf{\Pi }\right)={\mathbf{1}}_I{\mathbf{b}}_0^T+{\mathbf{AB}}^T.$$

(30)

The constants ${\mathbf{b}}_0^T$ have to be included because data cannot be centered beforehand as in the continuous case. With this decomposition, the inner product of the biplot markers (coordinates) is the logit of a expected probability, except for a constant,

$$logit\left({\pi }_{ij}\right)=b_{j0}+{\mathbf{a}}_i^T{\mathbf{b}}_j.$$

(31)

The logits are easily converted into probabilities

$${\pi }_{ij}={\displaystyle \frac{e^{\left(b_{j0}+{\mathbf{a}}_i^T{\mathbf{b}}_j\right)}}{1+e^{\left(b_{j0}+{\mathbf{a}}_i^T{\mathbf{b}}_j\right)}}}.$$

(32)

Then, by projecting a row marker ${\mathbf{a}}_i$ onto a column marker ${\mathbf{b}}_j$, we obtain, except for a constant, the expected logit and thus the expected probability for the entry $x_{ij}$ of $\mathbf{X}$. The constant $b_{j0}$ serves to determine the exact point where the logit is zero or any other value.

Due to the generalized nature of the model, the geometric interpretation closely resembles that of linear biplots. Computational procedures are analogous to those employed in previous cases, with the addition of the constant term. For example, in two dimensions, we can determine, on the direction of the vector ${\mathbf{b}}_j=\left(b_{j1},b_{j2}\right)$, what point predicts a given probability p. Let $\left(x,y\right)$ be the point satisfying the equation

$$y={\displaystyle \frac{b_{j2}}{b_{j1}}}x.$$

Prediction also verifies that

$$\mathrm{logit}\left(p\right)=b_{j0}+b_{j1}x+b_{j2}y.$$

Therefore, we obtain

$$x={\displaystyle \frac{\left(\mathrm{logit}\left(p\right)-b_{j0}\right)b_{j1}}{b_{j1}^2+b_{j2}^2}};y={\displaystyle \frac{\left(\mathrm{logit}\left(p\right)-b_{j0}\right)b_{j2}}{b_{j1}^2+b_{j2}^2}}.$$

(33)

The point in the direction of ${\mathbf{b}}_j$ that predicts $0.5$ ($logit\left(0,5\right)=0$) is

$$x={\displaystyle \frac{-b_{j0}{b}_{j1}}{b_{j1}^2+b_{j2}^2}};y={\displaystyle \frac{-b_{j0}{b}_{j2}}{b_{j1}^2+b_{j2}^2}}.$$

Using Equation (33), we can position markers for different probabilities along the direction of ${\mathbf{b}}_j$, creating a graded scale similar to a coordinate axis. The interpretation remains fundamentally similar to that of linear biplots; however, markers representing equidistant probabilities may not be spaced equally in the graph.

As an illustrative simple example with data taken from [43], the data that companies collect can include the expected, such as your name, date of birth, and email address, as well as more unusual details, such as your pets, hobbies, height, weight, and even personal preferences in the bedroom. They may also store your banking information and links to your social media accounts, along with any data you share there.

How companies use these data varies depending on their business, but it often leads to targeted advertising and optimizing website management.

The infographic taken from the site https://clario.co/blog/which-company-uses-most-data/, (accessed on 26 January 2025), is shown in Figure 1. The picture shows information on your face, voice, and environment that some internet companies collect.

The original data are transformed into a binary matrix, where the rows represent internet companies, the columns represent the types of information collected, and each entry is marked as 1 if the company collects that information, and 0 if it does not. The data matrix is shown in

Table 1. Internet companies.

Company	Face Rec.	Env. Rec.	Prod. Rec.	Contacts	Voice Rec.	Image Lib.	Language
Facebook	1	1	1	1	1	1	1
Instagram	1	1	1	1	1	1	1
Tinder	1	1	0	1	0	1	1
Grindr	1	1	0	1	0	1	1
Uber	1	0	0	1	0	1	1
TikTok	1	1	1	1	1	1	1
Strava	0	0	0	1	0	1	1
Spotify	0	0	0	0	1	0	1
Myfitnesspal	0	0	0	1	0	1	0
Clubhouse	1	0	0	1	1	0	1
Credit Karma	0	0	0	0	0	1	0
Twiter	1	1	0	1	1	1	1
Airbnb	1	1	0	0	0	1	1
Lidl Plus	0	0	1	0	0	0	1
American Airlines	0	0	0	0	0	1	1
eBay	0	0	1	0	0	1	1
Sleepcycle	0	0	0	0	1	0	1
Paypal	0	0	0	1	0	0	1
Slimming World	0	0	0	0	0	1	0
Whatsapp	0	0	0	1	0	1	1
Zoom	1	1	0	1	0	0	1
Protect Scotland	0	0	0	1	0	0	1
CoStar	0	0	0	1	0	1	0
Offerup	0	0	0	1	1	0	0
Doordash	0	0	0	1	0	0	1
Facetune	1	1	0	0	0	1	1
Google Docs	0	0	0	1	0	0	1
Google Sheets	0	0	0	1	0	0	1
Gmail	0	0	0	1	0	0	1
VSCO	0	0	0	0	0	1	1

. With these data, we perform a logistic biplot that summarizes the information in a two-dimensional graph. We obtain a joint representation of rows and columns of the data matrix. Distances among companies are interpreted as similarities, i.e., companies lying near on the biplot display have similar profiles in relation to the collected information. Angles between variables (kind of information) are interpreted as correlations. Acute small angles mean strong positive correlations, near straight angles mean strong negative correlations, and almost rights angles mean no correlation. Projecting the row markers onto column markers, we have the expected probability for each entry.

Figure 2 shows the graphical results as a typical logistic biplot with scales for probabilities 0.05, 0.25, 0.5, 0.75, and 0.95. We use percents (5, 25, 50, 75, and 95) rather than probabilities to simplify the plot. We can use any other values for the probabilities.

The plot displays a two-dimensional scattergram, where points represent the rows (companies) of the data matrix, and vectors or directions correspond to the variables. Distances among points for companies are inversely related to their similarity, for example, points representing Facebook, Instagram and TikTok coincide, indicating that the three companies have exactly the same profiles (the three collect all the available information). Companies with a single point have unique patterns, for example, Offerup, Uber or eBay, and groups with close points have several characteristics in common.

As previously mentioned, the angles between the variable directions provide an approximation of the correlations between them. For example, face recognition, environment recognition, and languages have high positive correlations because the increasing probabilities point in the same direction, i.e., have small acute angles between them. Collecting the contacts is not correlated to this group because they form a straight angle with it. Contacts and image library are negative correlations because the probabilities increase in opposite directions.

The angles are connected to tetrachoric correlations due to the close relationship between this representation and the factorization of the tetrachoric correlation matrix as we will explain later.

Projecting companies into the direction of a variable, we have the expected probabilities given by the approximation. Figure 3 shows the projections onto the variable “Face Recognition”. The point predicting 0.5 is marked with a circle because it is usually the cut point to predict the presence or absence of the characteristic. We can see that “TikTok”, “Instagram” and “Facebook” project on the upper part of the direction, meaning that they have the highest expected probabilities of using face recognition on their sites. Together with “Twitter”, “Zoom”, “Grinder”, and “Tinder” have expected probabilities higher than 0.95. The actual expected probabilities for each company can be approximated using the graded scaled placed along the direction. Companies with expected probabilities higher than 0.5 normally predict the presence of the character.

To simplify the plot, we can place marks for just a few probabilities, for example, 0.5 and 0.75 indicating the point where the expected logit is 0 (probability 0.5) and the direction of increasing probabilities. Figure 4 shows a simplified logistic biplot with probabilities 0.5 and 0.75. The directions are the same as in the previous plots but the scales are simplified, keeping just the cut for the presence prediction and the direction for higher probabilities.

We can use the expected probabilities from Equation (32) to generate predicted binary values: $\left({\widehat{x}}_{ij}=1\right)$ if ${\pi }_{ij}\ge 0,5$ and 0 otherwise. This yields an expected binary matrix $\widehat{\mathbf{X}}=\left({\widehat{x}}_{ij}\right)$. In the plot, this indicates that for each variable, the plane is divided into two prediction regions by a line perpendicular to the variable’s arrow and passing through the point predicting 0.5. The region on the side of the arrow predicts the presence of the characteristic, while the opposite region predicts its absence. Figure 5 illustrates this statement. In the figure, points in light blue have observed presences, and points in red have observed absences. We can see that all the points lie on the correct prediction regions.

A guide for the interpretation of the biplot can be found also in the supplementary material by Demey et al. [10].

A common overall goodness-of-fit measure is the percentage of correctly classified entries of the data matrix. To assess the quality of representation for individual rows (individuals) and columns (variables), we can calculate these percentages separately. We can also calculate the percentage of true positives (sensitivity) and true negatives (specificity).

To evaluate the quality of representation for each binary variable, we need a suitable measure that generalizes the measures of predictiveness for continuous data as in [44]. Equation (32) essentially defines a logistic regression model for each variable when the row coordinates are fixed, which allows to evaluate goodness of fit using pseudo-$R^2$ measures commonly associated to that model, for example, MacFaden, Cox–Snell or Nagelkerke.

Table 2. Measures of fit for columns.

	Deviance	D.F	p-Val	Nagelkerke	Cox–Snell	MacFaden	% Correct	Sensitivity	Specificity
Face recog.	36.60	2.00	0.00	0.96	0.70	0.91	100.00	100.00	100.00
Environtment recog.	42.47	2.00	0.00	0.96	0.76	0.92	100.00	100.00	100.00
Product recog.	45.91	2.00	0.00	0.96	0.78	0.90	96.67	80.00	100.00
Your contacts	35.03	2.00	0.00	0.91	0.69	0.82	93.33	100.00	80.00
Voice recog.	16.98	2.00	0.00	0.61	0.43	0.45	90.00	75.00	95.45
Image library	12.69	2.00	0.00	0.47	0.34	0.31	80.00	88.89	66.67
Language	47.98	2.00	0.00	0.97	0.80	0.93	100.00	100.00	100.00
Total	237.65	14.00	0.00	0.88	0.68	0.77	94.29	94.79	93.86

contains fit measures for the Internet Companies Data. It contains the fit measures for each column separately and for the complete table.

In the data table, 94% of all entries are correctly predicted by the previously described procedure as shown in Figure 5. However, it is more insightful to examine the indices for each individual variable. This is because, in some cases, only a few variables are accurately represented. Such a situation can arise when dealing with a large set of variables, where only a small subset is actually relevant to the problem. For instance, in a genetic polymorphisms matrix, only certain variables may be of real interest.

In our example, all variables show a reasonable percentage of correct classifications, except for “Image Library”, which correctly classifies only 80% of all values and 66.67% of the negatives. Additionally, we include the deviance for each variable in the model, using the latent dimensions as predictors, along with a p-value as an indicator of model fit. This measure, potentially adjusted for multiple comparisons, could be used to select the most significant variables as demonstrated in [10]. We also show pseudo-$R^2$ measures. All have high values except for “Image Library” and “Voice Recog.” that are worse predicted or represented on the graph.

By assuming that the observed categorical responses are discretized versions of continuous processes, we can leverage the close relationship between the presented model and the factorization of tetrachoric correlations to perform a factor analysis.

Consider a binary variable $X_j$. This variable is assumed to arise from an underlying continuous variable $X_j^{*}$ that follows a standard normal distribution. There is a threshold ${\tau }_j$, which divides the continuous variable $X^{*}$ into two categories.

The relationship between $X_j$ and $X_j^{*}$ can be expressed as

$$X_j=\left\{\begin{array}{cc}0\hfill & \mathrm{if}{X}_j^{*}\le {\tau }_j\hfill \\ 1\hfill & \mathrm{if}{X}_j^{*}>{\tau }_j\hfill \end{array}\right.$$

The tetrachoric correlations are the correlations among the $X_j^{*},j=1,\dots ,J$. Let $\mathbf{R}$ be a $J\times J$ matrix containing the tetrachoric correlations among the J binary variables, and let ${\tau }_j,j=1,\dots ,J$ be the thresholds.

We can factorize the matrix $\mathbf{R}$ as

$$\mathbf{R}\cong \mathbf{\Lambda }{\mathbf{\Lambda }}^T$$

(34)

where $\mathbf{\Lambda }$ contains the loadings of a linear factor model for the underlying variables.

It can be shown that there is a close relation between the factor model in Equation (34) and the model in Equation (32). You can find the details in [8]. The model in Equation (32) is actually equivalent to the multidimensional logistic two-parameter model in Item Response Theory (IRT).

The factor model loadings $\mathbf{\Lambda }=({\lambda }_{js},j=1,\dots ,J;s=1,\dots ,S)$ and the thresholds ${\tau }_j,j=1,\dots ,J)$ can be calculated from the parameters for the variables in our model as

$${\tau }_j=b_{j0}{\left(1-\sum _{s=1}^S{b}_{js}^2\right)}^{-1/2}$$

(35)

$${\lambda }_{js}=b_{js}{\left(1-\sum _{s=1}^S{b}_{js}^2\right)}^{-1/2}$$

(36)

In this way, we provide a classical factor interpretation to our model, adding the loadings and communalities (sum of squares of all dimension loadings of each variable). Loadings measure the correlation among the dimensions or latent factors, and the communalities the amount of variance of each variable explained by the factors. For our data, the loadings and communalities are in

Table 3. Thresholds, loadings, and communalities.

	Thresholds	Loadings		Communalities
		Dim1	Dim2
Face recog.	−0.02	0.14	0.96	0.94
Environtment recog.	−0.05	−0.14	0.96	0.94
Product recog.	−0.11	−0.83	0.50	0.93
Your contacts	0.04	0.96	0.17	0.95
Voice recog.	−0.10	0.64	0.60	0.77
Image library	0.03	−0.80	0.26	0.71
Language	0.11	−0.06	0.97	0.94

.

The two-dimensional solution explains 88% of the variance. All the communalities are higher than 0.71, meaning that a good amount of the variance is explained by the dimensions. The communalities serve also to select important variables. Variables with low communalities may be explained by other dimensions or have little importance for the description of the rows.

We can see also that face recognition, environment recognition and language are mostly related to the first dimension, while product recognition and your contacts are related to the first dimension. Voice recognition is related to both.

3.2. Triplot Representations for Binary Partial (Least Squares) Regression

Equations (1) and (2) define two biplots for the continuous case that share the scores for individuals in $\mathbf{T}$. Some of the properties of those biplots have been described by Oyedele and Lubbe [14].

In the same way, Equations (1) and (8) define two biplots that share scores in $\mathbf{T}$, but now the biplot for responses is a logistic biplot as described in Section 3.1. The details of this representation can be found in [15]. For this case, Equation (19) defines a biplot for the regression coefficients in terms of the observed variables that can help in determining the more important predictors to explain the responses.

The equations in (20) and (21) define two logistic biplots with the particularity that both share the row scores in $\mathbf{T}$. We can plot those together with the loadings for responses $\mathbf{Q}$ and predictors $\mathbf{P}$ to obtain a joint representation of individuals, responses, and predictors, i.e., a triplot with three sets of markers. Both biplots and then the triplot are interpreted using the rules in Section 3.1.

The measures of quality (correct classifications, sensitivity, specificity, pseudo-$R^2$, factor loadings, or communalities) for matrix $\mathbf{Y}$ can be used to identify which responses are correctly explained and what dimensions are needed for the explanation. Global qualities can be used to determine the number of dimensions needed for an adequate prediction, taking into account that some of the responses may not be well predicted by the model. In this case, the angles among the directions for responses or the projections of the scores on them are less important because they are not optimized to explain the variability in data but to capture the relations with the predictors. The same occurs for the joint interpretation of rows and columns.

4. Illustrative Example

Anderson et al. [45] estimate that approximately 30% of plant diseases are caused by phytopathogenic fungi. These fungi can have a significant impact on both ecology and agriculture, causing crop losses and environmental damage.

We focus on a specific fungus, Colletotrichum graminicola (or Glomerella graminícola). This fungus is a major pathogen of corn, causing a disease called anthracnose. Anthracnose produces spots on the leaves and stems of corn plants, which can lead to crop losses. There are some studies, such as that of O’Connell et al. [46], which have collected the economic impact it has had in the United States.

The data used in this study come from a project carried out by the CIALE. The project collected samples of corn plants infected with Colletotrichum graminicola from nine different countries: Argentina (AR), Brazil (BR), Canada (CA), Croatia (HR), Slovenia (SI), France (FR), Portugal (PT), Switzerland (CH), and the United States (US). The response matrix has the indicators (dummy variables) for the countries.

The predictors are a set of binary polymorphisms obtained from the RNA sequencing of the fungus Colletotrichum graminicola, which yielded a total of 13,183 variables. However, the sequencing method generates duplicate information, a variable for the presence and another for the absence, that is complementary; so we first removed those variables for the absence. This reduced the number of variables to 6419. We then selected the 2379 variables that were most significant based on their correlation with the response variable, i.e., the presence or absence of anthracnose.

The statistical analysis was carried out with the statistical software R [28]. We used the MultBiplotR package [27], which includes functions for performing the proposed method including biplot analysis.

The biplot representation of the responses is shown in Figure 6. Rather than the rows, the convex hulls containing all the points of each origin are represented.

The BPLSR model achieved an overall classification accuracy of 90.94% considering the binary variables separately. The classification accuracy for each country, together with some additional measures of fit, are shown in

Table 4. Columns fit measures.

	Deviance	D.F	p-Val	Nagelkerke	Cox–Snell	MacFaden	% Correct	Sensitivity	Specificity
Argentina	3.60	2.00	0.17	0.10	0.03	0.08	94.17	0.00	100.00
Brazil	95.40	2.00	0.00	0.96	0.60	0.94	100.00	100.00	100.00
Canada	35.30	2.00	0.00	0.48	0.29	0.37	85.44	100.00	82.35
Croatia	7.48	2.00	0.02	0.13	0.07	0.10	87.38	0.00	100.00
France	20.36	2.00	0.00	0.34	0.18	0.26	85.44	0.00	97.78
Portugal	1.37	2.00	0.50	0.06	0.01	0.05	97.09	0.00	100.00
Slovenia	1.81	2.00	0.41	0.08	0.02	0.07	97.09	0.00	100.00
Switzerland	19.89	2.00	0.00	0.39	0.18	0.33	93.20	22.22	100.00
USA	30.31	2.00	0.00	0.42	0.25	0.32	77.67	72.22	78.82
Total	215.52	18.00	0.00	0.43	0.21	0.35	90.83	51.46	95.75

.

In our example, the strains from Brazil correctly classify all of their individuals. The rest have lower percentages of correct classification. Argentina, Croatia, France, Portugal, and Slovenia have 0% sensitivity, that is, none of the presences are correctly classified. Switzerland has a sensitivity of 22.22%. This kind of classification is more useful when we have several (possibly) correlated variables rather than a set of indicators for several classes.

In this particular case, in which we have the indicators of some disjoint classes, we can calculate another classification assigning each individual to the class with higher expected probability. The results of this classification are shown in

Table 5. Classification of the countries.

	Brazil	Canada	France
Argentina	0	0	6
Brazil	20	0	0
Canada	0	18	0
Croatia	0	0	13
France	0	0	13
Portugal	0	0	3
Slovenia	0	0	3
Switzerland	0	0	9
USA	0	18	0

.

With this classification, only 49.51% of the strains are correctly classified into their origin. All the strains from Brazil, Canada, and France are correctly classified, but the strains from the USA are mistakenly grouped with those from Canada, while the remaining strains (Argentina, Croatia, France, Portugal, Slovenia, and Switzerland) are classified under France. This likely suggests that the characteristics of all European varieties (including Argentina) are quite similar, while Canada and the USA also share many similarities, with only Brazil showing distinct genetic characteristics. Experts have informed us that the Argentinian strains originated in Europe, which further supports this observation.

We do not include the predictors and its interpretation in this plot because of the big amount. We will show the interpretation in the next step of the analysis.

In conclusion, we identify three groups with different genetic profiles according to the origin of the strains: Brazil, North America (USA and Canada) and Europe (Argentina, Croatia, France, Portugal, Slovenia, and Switzerland). We repeat the analysis, recoding the whole set into those groups named Brazil, North America, and Europe (including Argentina).

The biplot for the new responses is in Figure 7. It is quite clear that all the groups are correctly classified.

Table 6. Columns fit measures for responses of grouped data.

	Deviance	D.F	p-Val	Nagelkerke	Cox–Snell	MacFaden	% Correct	Sensitivity	Specificity
Brazil	93.55	2.00	0.00	0.95	0.60	0.92	100.00	100.00	100.00
Europe	139.67	2.00	0.00	0.99	0.74	0.98	100.00	100.00	100.00
N. America	129.56	2.00	0.00	0.99	0.72	0.97	100.00	100.00	100.00
Total	362.78	6.00	0.00	0.98	0.69	0.96	100.00	100.00	100.00

contains measures of fit for the responses. All measures are very high, so no further comment is needed.

Table 7. Thresholds, loadings, and communalities.

	Thresholds	Dim1	Dim2	Communalities
Brazil	−0.02	−0.65	−0.72	0.94
Europe	0.00	−0.03	0.99	0.98
N. America	0.00	0.92	−0.37	0.98

contains the loadings and the communalities that are very high for these data, which means that the relations among the latent variables with the responses are very high.

The procedure is also helpful in identifying the variables that contribute to the prediction by examining the predictors most strongly associated with the latent variables used for prediction. We can add the predictors to the previous biplot to obtain a triplot. The resulting plot is shown in Figure 8.

Due to the high number of predictors, the complete plot is too crowded and difficult to see, and we may select just one fraction of the variables. We can use, for example, predictors that have high communalities, or any other measure of fit.

We are interested in the predictors that point in the direction of each response and are highly related to the latent variables. For example, we can calculate the squared cosines between the directions of predictors and responses to identify the most important polymorphisms for each group. Those cosines are easily calculated from the matrices in Equation (20). If the vector ${\mathbf{l}}_P={\left({\mathbf{P}}^2{\mathbf{1}}_K\right)}^{1/2}$ contains the lengths of the row vectors of $\mathbf{P}$ (directions of predictors on the plot) and ${\mathbf{l}}_Q={\left({\mathbf{Q}}^2{\mathbf{1}}_I\right)}^{1/2}$ contains the lengths of the row vectors of $\mathbf{Q}$ (directions of responses on the plot), where the exponents apply to each element of the matrices, then the cosines are in the matrix

$$\mathbf{C}={\mathbf{D}}_{{\mathbf{l}}_P}^{-1}\mathbf{P}{\mathbf{Q}}^T{\mathbf{D}}_{{\mathbf{l}}_Q}^{-1}$$

(37)

where ${\mathbf{D}}_{{\mathbf{l}}_P}^{-1}$ and ${\mathbf{D}}_{{\mathbf{l}}_Q}^{-1}$ are the diagonal matrices with the lengths in the diagonal. Each column of $\mathbf{C}$ contains the cosines of all the predictors with a single response; the near are cosines to 1 or −1, the closer are the direction of the responses and predictors.

For each group we select the variables that have squared cosines higher than 0.9 and communalities higher than 0.85 to select also those related to the latent dimensions. The limits for the selection can be changed to select more or fewer predictors.

Figure 9 shows the variables most related to the direction that predicts Brazil, and

Table 8. Fit measures of the variables most related to Brazil.

Polymorfism	Deviance	D.F	p-Val	Nagelkerke	Cox–Snell	MacFaden	% Correct	Sensitivity	Specificity
725993	79.24	2.00	0.00	0.87	0.54	0.80	99.03	100.00	98.81
851495	78.00	2.00	0.00	0.86	0.53	0.79	99.03	100.00	98.81
852127	78.00	2.00	0.00	0.86	0.53	0.79	99.03	100.00	98.81
1162451	79.95	2.00	0.00	0.86	0.54	0.79	98.06	95.00	98.80
1162499	79.95	2.00	0.00	0.86	0.54	0.79	98.06	95.00	98.80
1092206	87.65	2.00	0.00	0.90	0.57	0.84	99.03	95.24	100.00
453086	83.11	2.00	0.00	0.90	0.55	0.84	99.03	100.00	98.81
453152	83.11	2.00	0.00	0.90	0.55	0.84	99.03	100.00	98.81
198811	80.72	2.00	0.00	0.88	0.54	0.82	99.03	100.00	98.81
519877	83.11	2.00	0.00	0.90	0.55	0.84	99.03	100.00	98.81
416611	90.04	2.00	0.00	0.93	0.58	0.89	100.00	100.00	100.00
416649	90.04	2.00	0.00	0.93	0.58	0.89	100.00	100.00	100.00
416658	90.04	2.00	0.00	0.93	0.58	0.89	100.00	100.00	100.00
443422	85.55	2.00	0.00	0.89	0.56	0.82	99.03	95.24	100.00
443436	85.55	2.00	0.00	0.89	0.56	0.82	99.03	95.24	100.00
443446	85.55	2.00	0.00	0.89	0.56	0.82	99.03	95.24	100.00
141702	90.04	2.00	0.00	0.93	0.58	0.89	100.00	100.00	100.00
82966	83.59	2.00	0.00	0.90	0.56	0.85	99.03	100.00	98.81
79763	80.21	2.00	0.00	0.86	0.54	0.79	98.06	95.00	98.80

the goodness of fit.

Figure 10 shows the variables most related to the direction that predicts Europe.

The number of variables highly associated with European tribes is significantly lower than that observed for Brazilian or American tribes. Nevertheless, the goodness of fit, as shown in

Table 9. Fit measures of the variables most related to Europe.

Polymorfism	Deviance	D.F	p-Val	Nagelkerke	Cox–Snell	MacFaden	% Correct	Sensitivity	Specificity
247731	87.77	2.00	0.00	0.77	0.57	0.62	92.23	91.49	92.86
247828	87.77	2.00	0.00	0.77	0.57	0.62	92.23	91.49	92.86
409898	112.74	2.00	0.00	0.89	0.67	0.79	95.15	96.36	93.75
81758	85.96	2.00	0.00	0.76	0.57	0.61	92.23	95.35	90.00
230685	112.20	2.00	0.00	0.89	0.66	0.79	96.12	93.88	98.15

, remains robust. Interestingly, all genes are directly related to the response variable, except for gene “409898”, which shows an inverse relationship. Notably, this gene has a higher goodness of fit compared to the rest, highlighting its distinct contribution to the model. However, it is worth noting that the classification performance for the European strains does not reach the near-perfect results observed for Brazil. In particular, the percentage of cases correctly classified, as well as the sensitivity and specificity, do not reach 100%. This indicates a slightly lower overall predictive accuracy when analyzing the European dataset compared to the Brazilian one.

Figure 11 shows the variables most related to the direction that predicts North America.

The number of genes associated with the North American strain is significantly higher than those associated with the European and Brazilian strains. Only 10 genes are directly associated with the North American strain, while the rest have an inverse relationship with the response variable. The goodness of fit, as shown in

Table 10. Fit measures of the variables most related to North America.

	Deviance	D.F	p-Val	Nagelkerke	Cox–Snell	MacFaden	% Correct	Sensitivity	Specificity
132701	81.10	2.00	0.00	0.73	0.54	0.57	89.32	98.28	77.78
1132920	82.01	2.00	0.00	0.78	0.55	0.66	94.17	91.78	100.00
1156764	71.05	2.00	0.00	0.72	0.50	0.59	92.23	89.33	100.00
1193214	76.35	2.00	0.00	0.75	0.52	0.62	93.20	90.54	100.00
373779	68.57	2.00	0.00	0.65	0.49	0.48	85.44	98.15	71.43
897892	94.50	2.00	0.00	0.81	0.60	0.67	92.23	100.00	81.82
897945	86.08	2.00	0.00	0.77	0.57	0.63	93.20	96.88	87.18
962865	72.23	2.00	0.00	0.69	0.50	0.54	91.26	93.94	86.49
306992	90.08	2.00	0.00	0.79	0.58	0.65	93.20	98.39	85.37
762077	73.39	2.00	0.00	0.71	0.51	0.57	91.26	91.43	90.91
290929	83.85	2.00	0.00	0.74	0.56	0.59	85.44	98.15	71.43
553897	99.66	2.00	0.00	0.84	0.62	0.72	94.17	98.41	87.50
614143	78.79	2.00	0.00	0.71	0.53	0.55	86.41	100.00	72.00
928455	82.02	2.00	0.00	0.73	0.55	0.57	85.44	100.00	70.59
964199	77.15	2.00	0.00	0.70	0.53	0.54	83.50	95.83	72.73
227545	83.45	2.00	0.00	0.74	0.56	0.59	89.32	100.00	76.60
384381	87.77	2.00	0.00	0.77	0.57	0.62	89.32	100.00	76.60
384414	87.77	2.00	0.00	0.77	0.57	0.62	89.32	100.00	76.60
389429	68.86	2.00	0.00	0.65	0.49	0.49	77.67	91.30	66.67
16579	79.07	2.00	0.00	0.72	0.54	0.56	83.50	72.73	95.83
62965	77.16	2.00	0.00	0.72	0.53	0.57	90.29	93.85	84.21
236870	111.89	2.00	0.00	0.91	0.66	0.84	98.06	98.51	97.22
307532	76.90	2.00	0.00	0.71	0.53	0.54	87.38	96.55	75.56
307580	76.90	2.00	0.00	0.71	0.53	0.54	87.38	96.55	75.56
307650	84.81	2.00	0.00	0.75	0.56	0.60	88.35	98.25	76.09
307689	84.81	2.00	0.00	0.75	0.56	0.60	88.35	98.25	76.09
311118	95.52	2.00	0.00	0.83	0.60	0.70	94.17	96.92	89.47
312553	95.52	2.00	0.00	0.83	0.60	0.70	94.17	96.92	89.47
312571	95.52	2.00	0.00	0.83	0.60	0.70	94.17	96.92	89.47
316107	95.52	2.00	0.00	0.83	0.60	0.70	94.17	96.92	89.47
316117	95.52	2.00	0.00	0.83	0.60	0.70	94.17	96.92	89.47
344964	84.37	2.00	0.00	0.77	0.56	0.63	92.23	88.89	94.03
345029	84.37	2.00	0.00	0.77	0.56	0.63	92.23	88.89	94.03
349829	84.37	2.00	0.00	0.77	0.56	0.63	92.23	88.89	94.03
351745	87.15	2.00	0.00	0.79	0.57	0.66	93.20	91.43	94.12
486315	71.90	2.00	0.00	0.70	0.50	0.54	91.26	92.65	88.57
622094	75.72	2.00	0.00	0.70	0.52	0.54	87.38	96.55	75.56
627527	77.08	2.00	0.00	0.71	0.53	0.55	87.38	96.55	75.56
659194	78.95	2.00	0.00	0.72	0.54	0.56	88.35	96.61	77.27
664929	73.80	2.00	0.00	0.68	0.51	0.52	86.41	96.49	73.91
665038	73.80	2.00	0.00	0.68	0.51	0.52	86.41	96.49	73.91
502591	71.17	2.00	0.00	0.67	0.50	0.51	89.32	96.67	79.07
502699	71.17	2.00	0.00	0.67	0.50	0.51	89.32	96.67	79.07
504331	71.17	2.00	0.00	0.67	0.50	0.51	89.32	96.67	79.07
226711	70.15	2.00	0.00	0.66	0.49	0.49	86.41	100.00	72.00
226746	70.15	2.00	0.00	0.66	0.49	0.49	86.41	100.00	72.00
228690	62.55	2.00	0.00	0.61	0.46	0.44	83.50	100.00	67.92
478047	85.07	2.00	0.00	0.79	0.56	0.66	95.15	94.29	96.97
478073	85.07	2.00	0.00	0.79	0.56	0.66	95.15	94.29	96.97
478117	85.07	2.00	0.00	0.79	0.56	0.66	95.15	94.29	96.97
478148	85.07	2.00	0.00	0.79	0.56	0.66	95.15	94.29	96.97
480059	85.07	2.00	0.00	0.79	0.56	0.66	95.15	94.29	96.97
481176	85.07	2.00	0.00	0.79	0.56	0.66	95.15	94.29	96.97
234113	77.47	2.00	0.00	0.71	0.53	0.54	86.41	98.18	72.92
364080	81.82	2.00	0.00	0.74	0.55	0.59	91.26	96.77	82.93
364102	81.82	2.00	0.00	0.74	0.55	0.59	91.26	96.77	82.93
364195	81.82	2.00	0.00	0.74	0.55	0.59	91.26	96.77	82.93
374329	82.47	2.00	0.00	0.73	0.55	0.58	79.61	92.00	67.92
375412	75.50	2.00	0.00	0.70	0.52	0.53	87.38	96.55	75.56
265758	88.81	2.00	0.00	0.80	0.58	0.68	94.17	94.12	94.20
266113	88.81	2.00	0.00	0.80	0.58	0.68	94.17	94.12	94.20
266125	88.81	2.00	0.00	0.80	0.58	0.68	94.17	94.12	94.20
266177	88.81	2.00	0.00	0.80	0.58	0.68	94.17	94.12	94.20
266186	88.81	2.00	0.00	0.80	0.58	0.68	94.17	94.12	94.20
6107	78.86	2.00	0.00	0.72	0.53	0.56	87.38	98.21	74.47
104827	67.60	2.00	0.00	0.64	0.48	0.48	83.50	100.00	69.09

, is high, indicating good model performance. Nevertheless, the percentage of correctly classified cases does not reach 100%. Interestingly, in certain cases, either sensitivity or specificity achieves perfect accuracy, demonstrating the ability of the method to capture certain aspects of the data with exceptional precision.

5. Conclusions

• Partial Least Squares Regression (PLSR) has been assessed as a good alternative to Multivariate Linear Regression for exploring the relationships between two datasets, particularly in situations where predictor variables are either highly numerous or exhibit significant collinearity. This evaluation highlights the versatility and effectiveness of PLSR in addressing complex modeling challenges.
• We propose an extension of the PLS technique applicable to scenarios involving binary predictors and responses. This novel approach, termed Binary Partial Least Squares Regression (BPLSR), is based on a generalization of the NIPALS algorithm, adapted to accommodate binary data structures, thereby broadening the applicability of PLSR techniques.
• Custom functions implementing the BPLSR method for binary data have been developed in the R programming environment. These functions are included in the MultBiplotR package, which has been prepared for submission to the CRAN repository to ensure wider broader accessibility and usability for researchers and practitioners.
• The usefulness and performance of the proposed BPLSR technique was validated through a real-world case study, focusing on the classification of Colletotrichum graminicola strains. The results showed promising classification accuracy, highlighting the potential of BPLSR as a powerful tool for binary data analysis.

6. Software and Data Notes

The example of the logistic biplot has been calculated using the function BinaryLogBiplotGD of the package MukltBiplotR [27] for R [28] that is available in CRAN. The next versions of the package will include these data and example as an illustration of the function.

The PLS for binary data has been calculated using the functions BinaryPLSR and Biplot.BinaryPLSR of the same package. The functions will be included in the next version of CRAN. In the meantime, the last version of the package and the actual script can be obtained upon request to the corresponding author.

The data used in the example can also be obtained upon request.