*3.2. Statistical Analysis*

3.2.1. Number of Methods per Article

On average, 1.89 methods were used per article. Figure 3 visualizes the number of research articles using a specified number of methods. It can be noted that a majority of the analyzed articles used at most two methods.

**Figure 3.** The number of research articles using specified numbers of methods.

3.2.2. Changes in the Popularity of Using Particular Methods

Changes in the number of articles using the identified methods appearing in subsequent years were also analyzed. The chart below shows the yearly numbers of articles using the seven most common methods and strategies, in the period 2012–2020. We can observe that although the analysis of organizational structure has been the most widely used method since 2016, it has recently lost its popularity, falling behind the analysis of created values. In turn, the analysis of the e-participation process, which enjoyed a peak in interest in 2015, has now largely lost its relevance. A similar decline in interest can be observed in relation to the analysis of collaboration model, which peaked in 2018. (as can be seen below in Figure 4).

**Figure 4.** The number of research studies using the following methods: (1) analysis of organizational structure/design, (2) analysis of created values, (3) analysis of e-participation process, (4) analysis of participants' behavior, (5) analysis of collaboration model, (6) analysis of participants' motivations, (7) analysis of communication model, (8) analysis of innovation process.

#### 3.2.3. Dependencies between Research Methods

In this section we answer the question of whether there are any dependencies between the various research methods. It is common that when we want to investigate the relationship between variables, we calculate the classical Pearson's correlation coefficient. However, Pearson's correlation coefficient should only be applied to check the dependency between two continuous variables. In our situation this is not the case because the variables describing the usage of research methods are binary variables, answering the question of whether a particular method was used or not. When we are looking for relationships between binary or categorical variables, the commonly used statistical test is Pearson's Chi-squared test of independence. We performed Pearson's Chi-squared test between each pair of variables out of all 15 variables, describing the research methods in Table 2. The results can be seen in Table 3.

**Table 3.** *p*-values from Pearson's Chi-squared test of independence applied to each pair of research method variables (where, for example, RM1 stands for Research Method 1). The assignment of particular methods and strategies to the labels numbered from RM1 to RM15 is described in Table 2.


The statistical analysis based on Pearson's Chi-squared test of independence showed that in most cases there was no statistically significant evidence of a statistical relationship between research methods (*p*-value > 0.05). The analysis showed that only in seven cases (highlighted in bold in Table 3) was there a significant statistical dependency between certain specific research methods (*p*-value < 0.05). We discuss these dependencies based on the results from Table 4 below and in Figure A1 in the Appendix A.

It must be noted that one of the assumptions of Pearson's Chi-squared test of independence is the fact that the value of the contingency table cell should be five or more in at least 80% of the cells, and no cell should have a value less than one. Unfortunately, all the contingency tables from Table 4 have at least one cell with a value smaller than five; therefore, the assumption above was not met. Since this was the case, we applied Yates's correction for continuity (Yates's Chi-squared test) [59]. The results can be seen in Table 5.

After Yates's correction there were only five cases with significant statistical dependency between certain specific research methods (*p*-value < 0.05). However, three of them were statistically highly significant (*p*-value < 0.001).


**Table 4.** Contingency tables of Pearson's Chi-squared test of independence for the variables with statistically significant dependency. The assignment of particular methods and strategies to the labels numbered from RM1 to RM15 is described in Table 2.

**Table 5.** *p*-values from Yates's Chi-squared test of independence.


Finally, we can conclude that there are five statistically significant relationships between research method variables: A relationship between analysis of created values and analysis of collaboration model, between analysis of participants' behavior and analysis of participants' motivations, between analysis of collaboration model and analysis of innovation process, between categorization of the implemented projects and state-of-the-art review, and finally between analysis of platform usability and analysis of the impact of AI algorithms. Note that the Chi-squared test of independence does not not give an answer as to what kind of dependency exists between variables. It only answers the question of whether there is dependency between variables. To find the limits on what can be shown from the analysis we looked at the contingency tables and corresponding figures and checked if we were able to draw any conclusions from them. From Table 5 and Figure A1 we can suppose that the latter four relationships rely on the fact that in the vast majority of cases, both of these methods were not used simultaneously. In the case of the relationship between analysis of created values and analysis of collaboration model, we can hypothesize that the discontinuation of the analysis of created values method was associated with an increase in the applicability of the analysis of collaboration model method. However, in this case the relationship between variables was not obvious.
