Dichotomization of Multilevel Variables to Detect Hidden Associations

Abstract

A test of independence is commonly used to determine differences (or associations) between samples in a nominal level measurement. Fisher’s exact test and Chi-square test are two of the most widely applied tests of independence used in the data analyses in different areas such as information technologies, biostatistics, psychology and health sciences. In some cases, contingency tables with null entries (also called random zeros) arise, particularly if the number of samples is small, and the variables analyzed are multilevel. This situation becomes a problem because if one or more entries in a contingency table are zero or have small values, then the tests of independence produce unreliable results. In this paper, we propose a method to address that issue. The method merges one or more levels of the variables analyzed to create contingency tables with only one degree of freedom, avoiding applying a test of independence on contingency tables with random zeros. The source code (Python) of the method is publicly available for use. The results obtained using our method give a complete panorama of the associations between the variables of a data set. To show the effectiveness of our approach to find dependencies between variables, we use four data sets publicly available on the Internet.

1. Introduction

The test of independence is one of the most common tools of inferential statistics applied to areas such as health, psychology and biostatistics. This type of test is used in many studies to determine differences between samples in a nominal level measurement [1].

For example, in [2] the authors applied the Chi-square test on collected data from three psychiatric hospitals in Ogun state, Nigeria, to show the associations between socio-demographic variables and perceived hindrance of mental services. In [3], the Chi-square test was used to know the dependency between the physical activity and school adaptation in two groups of students. In [4], a test of independence was used to support the evidence of the association between the effect of early treatment responses of subjects with major depressive disorder treated with medicines or placebo. Chen, Haung and Ng [5] applied the Chi-square test to evaluate the association between single-nucleotide polymorphisms (SNPs) with multiple diseases. Adamu et al. [6] used the Chi-square test to determine the dependency of the variables of a data set of various cancer types recorded in northeastern states of Nigeria. In [7], the psychological profile of patients with reactions to placebo during a drug provocation test was evaluated by Fisher’s exact test.

On the other hand, for more than three decades the validity of independence tests with contingency tables with small expected cell counts has been extensively researched and discussed [8,9]. In practice, when the number of levels of the variables analyzed is high, the contingency tables with random zeros usually arise, especially if the number of samples is small. In extreme cases, the result of tests can be unreliable, even using Fisher’s exact test. Some tools such as Minitab [10] do not print any if the expected cell counts is less than two or three, depending on the number of levels in the variables.

This paper proposes the use of a method that allows the building of contingency tables with one degree of freedom. For this purpose, new synthetic (but human-understandable) categories are created, allowing a direct application of Chi-square or Fisher’s exact tests of independence.

Our method executes an exhaustive exploration of the combinations between the different values of variables. This can help to discover hidden associations between variables. The results are shown graphically to assist in the finding’s identification process. An implementation of the method is publicly available at http://u.pc.cd/JjCrtalK (accessed on 1 December 2022).

In [11], Sharpe suggests, if possible, avoiding contingency tables of more than one degree of freedom. If the above is not possible, he mentions that one of the methods to identify associations in the data is to use partitioning. The method proposed in this article does an exhaustive search on all possible partitions.

Recently, Zeng et al. [12] developed iSuc-ChiDT, a method to identify succinylation sites in proteins. That method encodes the Chi-square statistical difference table to identify positional attributes in the protein. The iSuc-ChiDT resembles our method; however, the purpose of our method is completely different.

The rest of this paper is organized in four Sections. Section 2 explains the creation of contingency tables and the tests of independence, Chi-square and Fisher’s exact test. The proposed method, and the data sets used to test the proposal are shown in Section 3. Section 4 shows the results of the application of our methods on three data sets. Finally, conclusions are presented in the last section.

2. Preliminaries

2.1. Contingency Tables

In simple words, a contingency table is a two-way layout of the number of items in the sample set that fall into different categories [13,14]. It is a cross-classification table in which the association between the variables (or if they are not related to each other) can be seen.

A two-way contingency table is created from a sample by using two qualitative variables (henceforth, we will refer them as V${}_1$ and V${}_2$, respectively). The number of columns corresponds to the levels (distinct values) of V${}_1$ (m), and the number of rows corresponds the levels of V${}_2$ (n). Therefore, the number of entries or cells is $m\times n$.

Table 1. Example of a 2 × 2 contingency table.

V1	V2		Total
	C1	C2
B${}_1$	a	c	a + c
B${}_2$	b	d	b + d
Total	a + b	c + d	N

shows an example of a two-way contingency table for two dichotomous variables V${}_1$ and V${}_2$. The levels of V${}_1$ are C${}_1$, C${}_2$, whereas for V${}_2$ they are B${}_1$, B${}_2$. The content of each cell is computed from the sample as follows:

a: The number of elements that simultaneously belong to categories B${}_1$ and C${}_2$.

b: The number of elements that simultaneously belong to categories B${}_2$ and C${}_1$.

c: The number of elements that simultaneously belong to categories B${}_1$ and C${}_2$.

d: The number of elements that simultaneously belong to categories B${}_2$ and C${}_2$.

The letters $a,b,c$ and d are the observed frequencies. The sum of rows and columns of a contingency table is called the marginal totals. The values $a+c$ and $b+d$ form the marginal rows. The values $a+b$ and $c+d$ are the marginal columns. N is the size of the sample. The degrees of freedom are computed with $df=(m-1)\times (n-1)$; therefore, for the example of Table 1, $df=1$.

Once a two-way contingency table is created, a test of independence can be applied to discover an association between the variables V${}_1$ and V${}_2$. In the next Subsections, we will explain two of the most common tests of independence.

2.2. Chi-Square Test

This test of independence is used to determine whether there is a significant association between two variables [15,16,17,18].

The null hypothesis (H${}_0$) of the Chi-square test of independence is that there is no association between the variables. The alternative hypothesis (H${}_1$) assumes there is some association between V${}_1$ and V${}_2$.

The Chi-square statistic is defined as follows:

$${\chi }^2=\sum _i^{}\frac{{(O_i-E_i)}^2}{E_i}$$

where

$O_i$ is the number of samples in category (cell) i,

$E_i$ is the expected value of category i, i.e., $E_{i,j}=\frac{T_i{T}_j}{N}$; $T_i$ is the total of row i, and j is the total of column j.

The value of ${\chi }^2$ and the degrees of freedom are used to search the p-value in the table of distribution of probability of Chi-square. Typically, if the p-value is greater than 0.05 (level of significance), then V${}_1$ and V${}_2$ have no association (H${}_0$ is accepted); otherwise; the variables have an association (H${}_0$ is rejected).

It is common in practice to consider that the test is reliable if $E_i>5$ for at least 80% of the number of cells.

2.3. Fisher’s Exact Test

In some cases, the expected frequencies are lower than five ($E_i<5$) for more than 20% of cells in contingency tables. To asses the association between variables in these scenarios, it is convenient to apply Fisher’s exact test of independence [19,20].

For this test, the null hypothesis (H${}_0$) is that the relative proportions of V${}_1$ are independent of V${}_2$, i.e., the variables have no association. The alternative hypothesis (H${}_1$) is that V${}_1$ and V${}_2$ have an association.

For Table 1, the probability of H${}_0$ to be true is computed with:

$$p=\frac{\left(\begin{array}{c}a+c\\ a\end{array}\right)\left(\begin{array}{c}b+d\\ b\end{array}\right)}{\left(\begin{array}{c}N\\ a+b\end{array}\right)}$$

where $\left(\begin{array}{c}n\\ k\end{array}\right)$ is the number of subsets of k elements chosen from a set with n elements.

Similar to the Chi-square test, if $p<0.05$ (or other level of significance), then H${}_1$ is accepted, and H${}_0$ is rejected. Otherwise, H${}_0$ is accepted and H${}_1$ rejected.

2.4. Problem Formulation

Consider two categorical variables V${}_1$ and V${}_2$, suppose the following:

• V${}_1$ has L${}_1$ levels;
• V${}_2$ has L${}_2$ levels.

A test of independence between these variables requires a contingency table $\mathbb{T}$ with (L${}_1-1$) × (L${}_2-1$) degrees of freedom. The number of entries in $\mathbb{T}$ is L${}_1\times$ L${}_2$. A widely accepted rule is that all of the observed and expected frequencies in contingency tables should be at least five; this criteria is used in the Scipy library. Some other tools, such as Minitab, do not display the p-value when any expected count is less than one because the results can be invalid.

In some cases, the data analyzed produces contingency tables that contain one or more cells with low values, or even with zero entries. In these scenarios, the results of the Chi -square test and Fisher’s exact test are unreliable. To face this problem, in the next section, we propose a simple method that merges levels of variables and apply a brute force approach to explore all the possible combinations. Due to this merging of levels, the contingency tables are less likely to have cells with zero values. This can help in find hidden associations between pairs of variables.

3. Materials and Methods

3.1. Method

To avoid the scenarios, such as the one explained in the problem formulation subsection, we propose to transform the original variables V${}_1$ and V${}_2$, into new synthetic dichotomous variables $\mathcal{A}$ and $\mathcal{B}$, respectively. These new variables contain the fusion of different levels of each variable.

In general, for each synthetic variable ($\mathcal{A}$ or $\mathcal{B}$), the exact number of fusions of levels is the semi-sum of the number of combinations of $L_1$ or $L_2$ taking r levels at a time. This is computed as follows:

$$N_{\mathcal{A}}=\lfloor \frac{1}{2}\sum _{r=1}^{L_1}\frac{L_1!}{r!(L_1-r)!}\rfloor$$

(1)

$$N_{\mathcal{B}}=\lfloor \frac{1}{2}\sum _{r=1}^{L_2}\frac{L_2!}{r!(L_2-r)!}\rfloor$$

(2)

where

$N_{\mathcal{A}}$ is the number of fusions of the $L_1$ levels of variable $V_1$,

$N_{\mathcal{B}}$ is the number of fusions of the $L_2$ levels of variable $V_2$, and

$⌊x⌋$ is the floor of the value x.

Based on these synthetic and dichotomous variables, $\mathcal{A}$ and $\mathcal{B}$, the number of contingency tables that can be created is computed with $N_{\mathcal{B}}\times N_{\mathcal{B}}$. The general structure of these tables is shown in

Table 2. Structure of tables of contingency with synthetic variables.

	$\mathcal{B}$
$\mathcal{A}$	Category I ${}_{\mathcal{B}}$	Category II ${}_{\mathcal{B}}$	Total
Category I ${}_{\mathcal{A}}$	a	c	a + c
Category II ${}_{\mathcal{A}}$	b	d	b + d
Total	a + b	c + d	N

.

The Categories I${}_{\mathcal{A}}$ and II${}_{\mathcal{A}}$ of Table 2 contain one level of V${}_1$ or the merge of more than one level of V${}_1$. These mixtures of levels are mutually exclusive.

For the Categories I${}_B$ and II${}_B$ is the same but considering the variable V${}_2$. The entries $a,b,c$ and d of Table 2 are computed as was explained for Table 1.

Example 1.

Suppose we have the following two multilevel variables.

• V${}_1$ with levels: {single, married, divorced, widower};
• V${}_2$ with levels: {Private, self-employment, never-worked}.

For V${}_1$, we have $L_1=4$, whereas for V${}_2$ the number of levels is $L_2=3$. The number of possible combinations for the fusions of variables $\mathcal{A}$ and $\mathcal{B}$ is computed with Equations (1) and (2):

$$N_{\mathcal{A}}=\lfloor \frac{1}{2}\sum _{r=1}^{L_1}\frac{L_1}{r(L_1-r)}\rfloor =\lfloor \frac{1}{2}\left(\frac{4!}{1!(4-1)!}+\frac{4!}{2!(4-2)!}+\frac{4!}{3!(4-3)!}+\frac{4!}{4!(4-4)!}\right)\rfloor =7$$

$$N_{\mathcal{B}}=\lfloor \frac{1}{2}\sum _{r=1}^{L_2}\frac{L_2}{r(L_2-r)}\rfloor =\lfloor \frac{1}{2}\left(\frac{3!}{1!(3-1)!}+\frac{3!}{2!(3-2)!}+\frac{3!}{3!(3-3)!}\right)\rfloor =3$$

A total of 21 tests of independence are needed to completely explore the possible associations between V${}_1$ and V${}_2$. For this small example, it is possible to show all the fusions of levels; these are listed in

Table 3. List of all possible fusions of variables’ levels.

$\mathcal{A}$		$\mathcal{B}$
Category I${}_{\mathcal{A}}$	Category II${}_{\mathcal{A}}$	Category I${}_{\mathcal{B}}$	Category II${}_{\mathcal{B}}$
1 single	1 married, divorced, widower	1 private	1 self-employment, never-worked
2 married	2 single, divorced, widower	2 self-employment	2 private, never-worked
3 divorced	3 single, married, widower	3 never-worked	3 private, self-employment
4 widower	4 single, married, widower
5 single, married	5 divorced, widower
6 single, divorced	6 married, divorced
7 single, widower	7 married, divorced

.

Algorithm 1 shows the complete method for completely exploring the associations between categorical variables with two or more levels. An implementation of this method in the Python programming language is publicly available at http://u.pc.cd/JjCrtalK (accessed on 1 December 2022).

The computational load of Algorithm 1 is high. Considering only the most costly steps, we have the following. The generation of all combinations of n elements, taking r at a time (Line 26) has a cost of $\mathcal{O}\left(\begin{array}{c}\left(\begin{array}{c}n\\ 1\end{array}\right)+\left(\begin{array}{c}n\\ 2\end{array}\right)+...+\left(\begin{array}{c}n\\ \lfloor \frac{n}{2}\rfloor \end{array}\right)\end{array}\right)\approx \mathcal{O}\left(2^n\right)$. In addition, the creation of the synthetic variables, that is, the level partitioning of each variable, has a linear cost with respect to the size of the data set. The Processing with categorical variables of more than 12 levels with hundreds of pieces of data takes several minutes on everyday personal computers and hours if the number of samples are thousands.

Algorithm 1: Exhaustive test of independence for categorical variables

3.2. Data Sets

Table 4. Data sets used for the experiments.

Data Set	Total of Instances	Missing Values	Features
			Nominal	Numeric	Other
BI Software recommendation	100	No	10	1	No
Scholarship	100	No	3	1	No
Zoo	101	No	1	2	Boolean (15)
Breast Cancer	286	Yes	6	1	Interval (3)

shows the main characteristics of the data sets used for the experiments; a brief description of these is given below:

• Scholarship data set. This is a synthetic data set. It was created by the authors of this article to demonstrate the presented method. It contains four attributes: (a) score, numerical type with values between 5 and 10; (b) type of scholarship, with possible values A (assigned to excellent students), B (assigned to very good students), C (assigned to good students) and No (students without a scholarship); (c) class attendance level for each student, with possible values low, medium and high; and (d) like cars: a dichotomous variable randomly assigned to each student, with possible values Yes and No. This data set can be downloaded from http://u.pc.cd/JjCrtalK (accessed on 1 December 2022).
• Zoo data set was created by Richard Forsyt, and it is publicly available on https://archive.ics.uci.edu/ml/datasets/Zoo (accessed on 1 December 2022). The data set has 16 features. Each sample is assigned to one out of seven classes or types. The name of the animals is also given.
• Breast Cancer data set. The samples are described by 9 attributes and 1 class, some of which are interval, and some of which are nominal. It can be downloaded from https://archive.ics.uci.edu/ml/datasets/Breast+Cancer (accessed on 1 December 2022).

3.3. Data Preparation

The following general procedures were considered for the application of the proposed method:

• Real valued variables are not analyzed; therefore they are identified automatically (in Python), and they are not processed.
• In some data sets, categorical variables are encoded as integers or Boolean values. In these cases, a manual transformation of the values from integer type into categorical type was carried out.
• Samples with missing values in one or more variables were removed from data sets.
• Some variables do not provide important information, so they were removed manually. Examples of these variables are names, identifiers, consecutive numbering, etc.

4. Results

Tests of independence are commonly used to find possible associations between pairs of categorical variables. Our method allows detecting hidden associations between this type of variables, which otherwise would be very difficult to discover, even using the Chi-square test or Fisher’s exact test manually. The implementation of the method includes the generation of a graph to facilitate the understanding of the results. These are shown in the next section.

4.1. BI Software Recommendation Data Set

This data set has 100 samples and 11 features. We get rid of the following variables: product_id (the identifier of the product), which is not significant; rating, which is a real number between one and five; category, which has 18 levels.

Figure 1 shows the number of associations detected by the method. The proposed method discovered that the highest number of associations in the BI Software recommendation data set occurred for the variables industry and pricing.

Table 5. An example of the 253 relationships found between the industry and price variables. The contingency table is shown.

		Pricing
		(Freemium)	(Open Source, Enterprise)
Industry	(Manufacturing, Fashion, Utilities)	15	17
	(Marketing, Consultancy Telecommunications, IT Pharma, Food, Academia, Retail)	17	51

p-value = 0.0002.

shows one of the contingency tables in which an association is detected with Chi-square test. Variable Industry has 11 levels: manufacturing, fashion, utilities, marketing, consultancy telecommunications, IT pharma, food, academia and retail; whereas variable Princing has tree levels: (freemium, opensource and enterprise). Manually finding the combinations that are statistically relevant is a time-consuming task. Here is where the method is useful. The implementation of the method provides the results as a figure, the tables of contingency and the corresponding p-values.

4.2. Scholarship Data Set

For the case of the Scholarship synthetic data set, the score was treated as a categorical variable.

In this data set, the method detects many associations between the variables score, type of scholarship and class attendance level, see Figure 2. It is important to remember that tests of independence only identify associations, not causation. However, for this data set, the results found are reasonable, since students with high grades have some type of scholarship and generally attend more classes than other students. On the other hand, liking cars has nothing to do with grades or scholarships. As the values of this variable were imputed randomly, the number of associations found is very low.

The values shown in the cells of Figure 2 indicate the number of associations detected between pairs of variables.

4.3. Zoo Data Set

In this data set, all the Boolean values were treated as dichotomous (categorical). The variable animal-name was deleted manually. The variables legs (number of legs) and type (category of animal) were treated as categorical variables. Figure 3 shows the associations detected by the method. The legs and type variables are the ones with the greatest number of associations.

In this data set, we manually removed some variables to more clearly show the associations; these are presented in Figure 4. It can be seen that variable feathers has a relationship with variable airborne, which sounds logical. Variable fins has a relationship with variable hair. This could be because animals with fins do not have hair. A veterinarian or animal expert could explain the logic behind the associations detected by the method or whether they make any real sense. It is also possible that the method simply detected associations by the distribution of the data. In either case, the method presents the possible associations between variables.

4.4. Breast Cancer Data Set

The Breast Cancer data set contains variables of type interval; these are age, tumor-size and inv-nodes; the variable deg-malig has values 1, 2 and 3. The rest of the variables are of type categorical. All non categorical features were transformed into categorical ones. Figure 5 shows the results obtained by the application of the method. It is noticeable that for this data set, the computation time was more than 7 min on a MacBook Pro, Apple M1 processor with 8 GB RAM.

In this data set, the variables with the highest number of associations are inv-nodes and tumor-size. Although the interpretations on the associations detected are out of the scope of the expertise of the authors, the information retrieved by the method could be useful for an oncologist.

5. Conclusions

Independence tests are a very powerful tool for determining the association of two categorical variables. In the literature, there has been much discussion about the validity of these tests when one or more cells with low values or zeros are presented in the contingency tables. In this article, this problem is tackled from a computational point of view, proposing a method that minimizes the degree of freedom of the contingency tables by grouping the levels of each analyzed variable. In this way, the hidden association between the variables can be detected.

The implementation of the method in software generates a visual representation of the total associations found between the variables, allowing the expert to more easily identify if the association found has any substantial and real meaning.

To demonstrate the method, the synthetic data set “Scholarship” was generated in which some type of scholarship is assigned to students with high grades, and no type of scholarship to students with low grades. In addition, class attendance is higher in students with a scholarship than in the others. Furthermore, the set Scholarship has a variable whose values were generated pseudo-randomly. The proposed method finds a high number of associations between the variables score, type of scholarship and class attendance level, and two associations between the random variable, just as expected. The Scholarship data set, and the Python implementation of the proposed method are publicly available for download.

The method was also tested with real-world data sets, also publicly available on the Internet, finding a high number of associations between several variables.

The proposed solution allows identifying associations between multilevel categorical variables, performing an exhaustive search among all possible combinations of levels. The results are displayed graphically, facilitating the identification of the pairs of variables with the greatest number of possible associations. However, because many of these partitions might not be important, the researcher still has to manually analyze the generated crosstabs. This is a disadvantage of the method in which we currently work. Furthermore, we are improving the method in the computational part because we noticed that with variables of ten or more levels, the processing can take hours. An evolutionary computing approach to solve this problem is still under development.