*2.4. Data Collection and Analysis*

The study was ethically reviewed according to the human research ethics governance framework, and participants were asked to complete an informed consent form prior to data collection. An oCam screen recording program was used to record the online health information retrieval behavior of each participant on the computer screen, which was transcribed for coding and analysis following the study's completion. The content analysis framework of the participants' online health information retrieval behavior included "keyword selection", "Boolean logic query", "limited scope for query", "information browsing", and "information source".

The reliability of the content analysis was measured using inter-rater reliability [26], in which higher consistency results indicate higher reliability of the analysis. The reliability coefficients for the coding results were calculated according to the formulae of mutual agreement, mean agreement, and reliability coefficients, as follows

$$\text{Mean agreement} = \frac{2M}{N1 + N2}$$

$$\text{Reliability coefficient} = \frac{n \times \text{Mean mutual agreement}}{1 + (n - 1) \times \text{Mean aggregation}}$$

*M*: The number of variables for which the coding result was fully agreed between two persons.

*N*1: The total number of variables coded by the first coder.

*N*2: The total number of variables coded by the second coder.

*n*: The number of coders.

Content analysis of the data could only be performed following the determination of the reliability coefficient. Two coders coded 30 samples and calculated a mean agreement of 0.67 and a reliability coefficient of 0.80.

### **3. Results**

In this study, the Delphi method was used to encode the data content of the responses to the survey results, using descriptive statistics to restore the current status of college students' online information retrieval behavior.
