3.2.4. Dependencies between Research Methods and Other Features of the Analyzed Studies

In this section, we investigate whether there are relationships between the research method used and other article features such as citations, popularity, number of pages and year of publication. As before, in order to perform statistical analysis, we used binary variables describing the use of the peculiar research method in the articles. The variables describing article features are the following: Cited Reference Count, Times Cited WoS Core, Times Cited All Databases, 180 Day Usage Count, Since 2013 Usage Count, Number of Pages and Publication Year (all variables defined in the Web of Science specification [60]). All the above variables except the last one are continuous-type variables and the last one is categorical. In the case of the last variable, the matter is simple. In order to check its relationship with binary variables describing the research methods used, we used the Chi-squared test of independence as before. To check the relationship between binary variables and the other six continuous variables, we calculated the point biserial correlation coefficient. Note that one of the assumptions of the point biserial correlation is the fact that the continuous variable is normally distributed. To check this assumption we plotted histograms, quantile-to-quantile plots and performed the Shapiro–Wilk test of normality for all six continuous variables. The results are shown in Table 6, Figures 5 and 6.

**Table 6.** *p*-values of the Shapiro–Wilk test of normality applied for variables: Cited Reference Count, Times Cited WoS Core, Times Cited All Databases, 180 Day Usage Count, Since 2013 Usage Count, Number of Pages.


**Figure 5.** Histograms of the variables: Cited Reference Count, Times Cited WoS Core, Times Cited All Databases, 180 Day Usage Count, Since 2013 Usage Count, Number of Pages.

**Figure 6.** Q–Q plots of the variables: Cited Reference Count, Times Cited WoS Core, Times Cited All Databases, 180 Day Usage Count, Since 2013 Usage Count, Number of Pages.

From the histogram plots in Figure 5 we can see that only the distribution of the Number of Pages variable is approximately bell-shaped and therefore looks like a normal distribution. The quantile-to-quantile plots from Figure 6 confirm that only the Number of Pages variable may be normally distributed (because the values are arranged along a straight line). However, if we look at Table 6, we see that the *p*-values of the Shapiro–Wilk test of normality of all the considered variables are small (*p*-value < 0.05) and therefore we must reject the null hypothesis that a sample came from a normally distributed population. Since one of the assumptions of point biserial correlation was not met, we could not use this method to investigate the relationship between binary research method variables and the variables describing article features. However, we used a different solution. We grouped the values of continuous variables into one of three categories: low, medium and high, according to the scheme described in Table 7, and then we used the Chi-squared test of independence as before. The Publication year variable is already a categorical variable. However, due to the fact that it has nine values and the sample size is small, we also grouped its values into three categories. The remaining variables were grouped so that the size of each class was at least 10 and that all classes were more or less equal.


**Table 7.** Qualifying intervals for variables.

We performed the Chi-squared test of independence for all pairs such that the first variable in the pair was a binary variable describing the research method used and the second variable in the pair was a continuous variable describing the features of the article. The results of the analysis are shown in Table 8.

**Table 8.** *p*-values from the Pearson's Chi-squared test of independence (where CRC stands for Cited Reference Count, CW for Times Cited WoS Core, CA for Times Cited All Databases, 180U for 180 Day Usage Count, 2013U for Since 2013 Usage Count, PY for Publication Year and NoP for Number of Pages).


The statistical analysis based on the Pearson's Chi-squared test of independence showed that in most cases there is no statistically significant evidence that there is a statistical relationship between research methods and article features (*p*-value > 0.05). The analysis showed that only in six cases (highlighted in bold in Table 8) there is a significant statistical dependency between certain specific research methods and article features (*p*-value < 0.05). We discuss these dependencies based on the results from Table 9 below and in Figure A2 in Appendix A.

For the two first relationships from Table 9, we have enough value in each cell of the contingency table so we can conclude that there is a statistical relationship between the Analysis of created values method and 180 Day Usage Count variable—the use of this method translates into popularity among readers. There is also a statistical relationship between the Analysis of created values method and the number of pages of the article. In this case, it is easy to see from the chart that the use of this research method is related to the reduction of the number of pages of the article in which this method is used.


**Table 9.** Contingency tables of the Pearson's Chi-squared test of independence for the variables with statistically significant dependency.

> Unfortunately, the other four contingency tables from Table 9 have at least one cell with value smaller than five; therefore, we should apply Yates's correction for continuity. However Yates's correction for continuity is mainly applied for 2 × 2 contingency tables. This is not our case, so we have to use a different statistical test to resolve the remaining four cases. We performed Fisher's exact test, which is also commonly employed when sample sizes are small or the data are very unequally distributed among the cells of the contingency table. The results of the Fisher exact test can be seen in Table 10.

> **Table 10.** *p*-values from the Fisher exact test of independence (where RM6 stands for Research method 6, NoP for number of Pages and CRC for cited reference count).


From the Fisher exact test, it follows that there are two more statistical relationships (*p*-value < 0.05) between the analysis of participants' motivations method and the number of pages of the article, the state-of-the-art review method and the cited reference count. Again from Table 9 and Figure A2, we can draw some conclusions. It appears that use of the analysis of participants' motivations method is related to the increase of the number of pages of the article. Moreover, it seems that use of state-of-the-art review method has a positive impact on Cited Reference Count.

The last relationship we looked for was the relationship between the number of methods used in the articles and the features of the article. As before, we conducted the Chi-squared test of independence. For the purposes of the analysis, the variable describing the number of methods used was divided into four categories: one method, two methods, three methods, and 4–5 methods. Due to the small number of the articles with four or five methods used, these articles were grouped into one category. The obtained results (compare Table 11) showed that there is no statistically significant evidence that there was a statistical relationship between the number of methods used in the articles and the features of the article (*p*-value > 0.05).

**Table 11.** *p*-values from the Chi-squared test of independence (where NoM stands for Number of Methods, CRC for Cited Reference Count, CW for Times Cited WoS Core, CA for Times Cited All Databases, 180U for 180 Day Usage Count, 2013U for Since 2013 Usage Count, PY for Publication Year and NoP for Number of Pages).

