Construction of Questionnaire-Scale USOTIC “Social Networks in Primary and Secondary School Students: Use and Digital Coexistence”

Abstract

In the present work, the construction process, namely the design, elaboration and validation of a questionnaire-scale is described to evaluate the attitudes and interactions in the way of living in the digital world of social networks in primary and secondary students. The current aim was to analyse the means, uses and risks, as well as the sources of training and advice those students have to interact with the diverse range of devices with an internet connection that provide access to social networks. The participating sample for validation was 1073 students from the Primary and Secondary Education stages of centres with organic dependency: Private, concerted and public. The obtained results express the satisfactory metric quality of the questionnaire, thus presenting a theoretically based and operationally defined instrument, considering four relevant dimensions: Special, environmental, perceptual and motivational.

1. Introduction

The way children and young people coexist in the world of technology and social networks has a great impact and echo in today’s society due to the attitudes that are reflected through them, causing conflicts that alter coexistence inside and outside the classroom, sometimes moving beyond the barriers of educational centres. These conflicts can arise due to inappropriate use of social networks, which makes them vulnerable [1]. This vulnerability can be generated in this group as a result of a lack of knowledge of the repercussions and the importance of incorrect use of social networks, and on too many occasions, due to a lack of digital training and critical thinking in their use [2].

The aim of this work was to develop an evaluation instrument using a questionnaire-scale to explore the digital coexistence of children and adolescents through social networks, as a means of obtaining information and analysing the risks for Primary and Secondary School pupils when interacting on social networks. It focuses on dimensions such as the use they make of them and their relationship with learning, digital coexistence, participation and attitudes. The information obtained may constitute relevant data to detect the key elements of the use of social networks in children and adolescents and guide them towards taking preventive measures, fostering the development of critical thinking and, in certain cases, correcting incorrect use.

In order to carry out the interest of this research, a questionnaire entitled “Study of Social Networks in primary and secondary education students” was employed. In this article, we present its reliability and validity.

2. Fundamentals

In today’s society, technology has become a necessity in our way of life, causing a cultural change and transforming our ways of communication, interaction and knowledge. This transformation triggers a break from the linear form of communication, giving birth to a new form of interaction and communication through ICTs, opening a window to the digital world, transferring young people to scenarios where they are the protagonists of their own actions: The way they adhere to the diversity of media content, which they share, disseminate and comment on with other users [3]. In this sense, the content offered to them through the use of ICTs facilitates interaction and socialisation, as they share hobbies, interests and tastes for the same activities with their peers (music, sports, travel, etc.). This way of interacting will be determined by their attitude, which can lead them to become vulnerable to new scenarios, which, through ICTs, turn them into actors defined by their attitudes, altering digital coexistence. In this line, there is a diversity of studies, such as Project SAVE [4], the ANDAVE school anti-violence project [5], the programme to combat bullying launched by the regional government [6], the Cyber-experts programme [7] and the Cybermentors programme for the safe and responsible use of ICTs [8]. Reports of the state observatory on coexistence were carried out in 2008, as were studies by [9], all with the same objective of preventing disruptive attitudes that make young people vulnerable and improving conflicts in digital coexistence.

In line with this research, together with professional experience in the field of education, the interest in analysing the way in which Primary and Secondary School pupils live together in the digital world was born, and to carry it out, the construction of a questionnaire (Appendix A) was implemented.

Importance of ICTs

Technologies offer a wide range of possibilities for interaction, communication and knowledge through a diversity of tools with easy access to an internet connection. In this sense, children and adolescents are increasingly acquiring devices with a variety of attractive digital applications that allow them to open up to the digital world in a quick and easy way, but without being aware of the risks and consequences to which they are exposed, making them vulnerable.

Vulnerability is determined as the dimension in which adolescents have found a way of life in Social Networks and mobile devices in which “they use them very naturally and playfully because they have been socialised in technological contexts through processes of self-learning and informal knowledge” [10]. Through them, they show their creativity, as it allows them to personalise their pages, and make and publish their own musical productions, photos and videos with the aim of making them visible and obtaining likes or opinions from others. This makes them more vulnerable as they expose their lives and there is a lack of privacy of which they are unaware, which can lead to future consequences and repercussions because adolescents’ knowledge of technologies is mainly instrumental [11].

In this sense, we consider it relevant that in order to understand the process of inclusion of children and adolescents in the digital world of social networks, it is necessary to be clear about the diversity of concepts related to digital life, starting from the definition of a social network, which is detailed below:

“A structure made up of people who are interconnected through the internet by different types of links: affective, work, sexual, friendship, etc. It constitutes a dynamic group in continuous evolution, where there are people who join, others who appear and others who disappear. They allow contact with unknown people in a simple way” [12]. They are formed between children and adolescents who have common interests, which allows them to form friendships, follow up on past relationships and get to know people intimately to a certain extent.

In this line, there are studies [13,14] in which the most attractive device for adolescents is the mobile phone, which is becoming more and more advanced and sophisticated, allowing them to do a multitude of things and giving them a degree of self-esteem, freedom and personal intimacy. These devices, from adolescence onwards, serve them for fun, leisure and communication, through the resources available on them, such as chatting, sending messages, sharing photos, events, etc. Thus, they cover their basic psychological needs: Making themselves visible, reaffirming their identity in front of a group, having fun or simply being connected to their friends.

This way of relating is characterised by the ease with which they overcome embarrassment, overcome shyness or fear of ridicule, express their own identity, create personal profiles and establish relationships, in which they express emotions, which encourages participation; although, in some cases, exclusion can occur, which leads to two important aspects: On the one hand, it facilitates the creation of links, and on the other hand, their destruction.

Along these lines, [15] highlight three positions in their studies that shed light on the motives and uses those adolescents give to ICTs:

• The effects produced by media content affect and condition adolescents’ behaviour since they act as models of that content.
• Adolescents integrate the use of content into their daily lives in an active way so that they can choose what is interesting and convenient for them.
• The use of devices becomes part of a social process in adolescents playing an important role in the construction of identity.

In this sense, the use of ICTs can be understood from the perspective that they are part of everyday life for adolescents, and consequently, they are linked to problems of conflicts and risks that adolescents face in a new digital scenario. Among the risks in which adolescents may find themselves, we highlight the following:

• Not having a critical capacity in the selection and publication of content, being irresponsible in terms of exposing their personal information and privacy to a world in which they do not know who is behind the screen [15].
• Abuse in the use of technologies sometimes causes them to become disordered, leading to online addiction [12].
• The protection of being on the other side of the internet gives rise to disruptive behaviour among adolescents towards others, making them vulnerable to falling victim to cyberbullying [16]. Different forms of cyberbullying can take place through electronic insults, harassment by sending offensive messages, impersonation, disclosure and coaxing of personal information and exclusion and ostracism [17].

3. Materials and Methods

The aim of this study was to construct and validate a questionnaire-scale to explore the digital coexistence of primary and secondary school pupils with social networks, analysing the means, uses and risks, as well as the sources of training and advice they obtain for interacting with them.

3.1. Data Collection Procedure

The questionnaires were completed on a scheduled basis at different educational centres that had previously been asked to participate on a voluntary basis. In this study, the questionnaires were completed in person in paper format at the different schools by appointment, day and time, established by the head teacher at the school. It was carried out in the classrooms of the educational centre, under the supervision of the person responsible for the research and authoring of this questionnaire [1] during the months of March, April and May of the 2018–2019 academic year. To this end, before the students began the questionnaire, they were informed of the aim and purpose of the study and the importance of their participation in it, which was voluntary and anonymous. In addition, they were made aware of the law on data protection and confidentiality for research purposes when completing the questionnaire.

3.2. Participants in the Pilot Study

In order to carry out the statistical studies required for the validation of the questionnaire under study, we relied on a pilot study involving 1073 Primary and Secondary Education students from the province of Madrid, 47% of whom were male and 52% female.

As for the type of centre according to its organic nature, of the total number of participants in the study, 17.1% belonged to private centres, 43.7% to public centres and 39.2% to subsidised centres. Regarding the distribution of the sample by stage, of the total number of respondents, 17.3% correspond to the 4th stage of Primary Education, 17.5% correspond to the 5th level of Primary Education, 18.3% correspond to the 6th stage of Primary Education, 13% correspond to the 1st stage of Secondary Education, 12.8% correspond to the 2nd stage of Secondary Education, 12.7% correspond to the 3rd stage of Secondary Education and 8.4% correspond to the 4th stage of Secondary Education. In relation to the distribution of the sample by age, 21.4% of the total participants were aged between 9 and 10, 34.7% were aged between 11 and 12, 24.9% were aged between 13 and 14, 3.3% were aged between 15 and 16 and 3.7% were aged between 17 and 18.

3.3. Instrument

To address the proposed general objective, the questionnaire is designed and used as a tool for data collection as determined by studies such as [18,19]. The choice of the instrument was in response to the quantitative methodology and nature of the data we are concerned with acquiring, which in our case is related to the motivation and perception of various attributes of the classroom space [20].

4. Justification for the Development of the Questionnaire

Regarding the identification of the dimensions in the development of this questionnaire, we started with a review of authors such as [21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38]. We were able to explore how the different contributions they provide can be grouped into the following specific argumentative dimensions in relation to digital coexistence:

• Culture of uses, which includes device access, internet connection, social networks and connection time. This is in line with authors [22,23,25,30,38], in which their studies analyse the use of smartphones, social networks and the culture of connectivity in children and adolescents.
• Need for and dependence on networks. Authors such as [32,33,34,35] establish how the need for children and adolescents to be connected makes them dependent on them, and these will be linked to the purpose of achieving prestige on the different social networking sites.
• Security and harassment on the networks. As a reference, we take legal aspects such as articles 7 and 84 of the Spanish Data Protection Agency [39], which establish the minimum requirements and responsibilities for minors’ access to the internet and social networks. In addition, we include the studies by [29,36], in which they demonstrate the need to train critical citizens in the face of the challenges of technologies for their safety and protection, and the dimension is elaborated. Given this need to train critical citizens, one of the key elements is attitude, as shown in studies by [21,24,26]; therefore, we find that the factor of attitude towards the usefulness and purpose of interaction through social networks is a factor that will determine the risks to which children and adolescents may be subjected in the digital world, such as cyberbullying, invasion of privacy and intimacy, etc., as well as the risks to which they may be exposed.
• Utility and purpose. It is worth highlighting in this aspect how networks can be used as a means to promote learning, as stated by authors such as [40,41,42], who highlight the potential of learning networks as a means to offer a greater wealth of access to different sources and resources that facilitate learning.
• Another aspect to take into account is the sources of training from the educational centre (teachers, counsellors and external agents); in this line, as a reference, we take the Horizon report [43] in relation to the time students spend connected to the internet where they interact with others and exchange a diversity of information. In this sense, the role of the different training agents is essential to promote critical thinking and the responsible use of social networks.
• A final dimension that can be seen in the different studies consulted is in the area of family support. In this sense, authors such as [28,31,37] focus on how to help pupils in the process of socialisation and development in order to develop in a society that requires competent digital citizenship.

5. Validation Process of the Tool

5.1. Content Validation

Following the literature review, and once an initial structure based on theoretical dimensions had been established, items were developed for each group. This initial battery of items consisted of 63 items, distributed across the five dimensions.

This initial proposal was submitted to the judgement of experts, who assessed it on the basis of three indicators: Quality in the wording of the question, coherence with the object of study and relevance and importance within the instrument in terms of the information it can provide.

Expert judgement is a strategy with many advantages; Ref. [44] summarises its importance as follows:

… the theoretical quality of the response we obtain from the person, the level of depth of the assessment offered, its ease of implementation, the fact that it does not require many technical or human requirements for its execution, the possibility of using different strategies to collect information is very useful for determining knowledge on difficult, complex, novel or little studied contents and topics, and the possibility of obtaining detailed information on the subject under study.

In this sense, and following [44], there are discrepancies regarding the number of experts that should participate in this validation process, following, in order of the dates of publication, Ref. [45], who suggest that the number should range between 15 and 20; Ref. [46], who places them between 15 and 35; Ref. [47], between 7 and 30 and Ref. [48], between 15 and 25.

Even so, there is another relevant issue, namely the consideration of “expert”. For this, we rely on the coefficient of expert competence [49]. In this reference, several self-assessed indicators are proposed by the proposed experts, and in our case, we summarise these indicators around three factors: Teaching experience in the field, level of publications and research experience in the field of knowledge.

In short, 15 experts were selected, who obtained scores above 0.85 in the coefficient of expert competence (CEC).

Table 1. Coefficient average of expert competence (CEC).

	Teaching	Publications	Investigation	CEC
Expert media	0.93	0.89	0.88	0.90

shows the overall data, although the overall scores can be found in Appendix B.

5.2. Expert Validation Data

Each expert gave a score between 0–1 to each item and for each factor (quality, coherence, relevance), obtaining a final score for each item as an average of the previous scores.

The initial version of the questionnaire had 67 items, with 4 of a socio-demographic nature and 63 of content on the dimensions. After judgement, the questionnaire was reduced to 44 items, whereby 4 socio-demographic items and 40 research items were maintained. The eliminated questions did not exceed an average of 0.80 in the overall total score. Appendix C shows the item-by-item scores for each expert, as well as the overall score for each one.

The configuration of the instrument after the expert assessment is presented in

Table 2. Distribution of items in the questionnaire.

Factor	Identification Factor	Items
1	Culture of network use	19, 20, 21, 22, 23, 27 y 31
2	Network dependency	1, 3, 4, 5, 6, 7, 8, 10, 15 y 16
3	Safety and harassment on networks	24, 25, 26, 28, 29 y 30
4	Use of networks for learning	11, 12, 13, 14, 17 y 18
5	Teacher support	32, 33, 34, 35 y 36
6	Family support	2, 9, 37, 38, 39, 40

Source: Distribution data.

.

Finally, the recommendation made by the experts to establish seven levels of response on the Likert scale was taken into account in the final drafting of the instrument. This was taken on board in the final drafting of the instrument: 1 (NEVER), 2 (RARELY), 3 (SOMETIMES), 4(OFTEN), 5 (NORMALLY), 6 (USUALLY) and 7 (ALWAYS).

5.3. Construct Validation

Once the questionnaire had been updated with the experts’ contributions, the statistical study was carried out by means of confirmatory factor analysis. To this end, the pilot test was carried out with the aforementioned participants.

The factor analysis was carried out, in which the number of factors to be extracted was not fixed, nor were the minimum values to be excluded. Varimax rotation was applied, as well as the KMO test and Bartlett’s test of sphericity as shown in

Table 3. KMO and Bartlett tests.

Kaiser-Meyer-Olkin Measure of Sampling Adequacy		0.832
Bartlett’s test of sphericity	Approx. Chi-cuadrado	10,731.409
	gl	780
	Sig.	0.000

.

From these data we can assume that as the KMO value is 0.0832 (close to the ideal value of 1), it indicates a remarkable fit of the data [50].

Likewise, the level of Bartlett’s test (sig = 0.000) allows us to affirm that there is perfect significance, considering the adjustment of the variables by means of the ideal factor analysis.

In terms of principal components, six factors explain 57.765% of the total variance. From the sixth factor onwards, percentages of less than 2% of the variance are explained and should therefore be discarded (

Table 4. Factorial analysis.

Factor	Initial Eigenvalues
	% of Variance	Cumulated %
1	16.621	16.621
2	13.579	30.199
3	7.932	38.131
4	7.360	45.491
5	6.334	51.825
6	5.940	57.765

Source: Research data.

).

The model was then established using the structural equation procedure with the support of AMOS software version 24.

In the first analysis, the non-statistical values did not support the proposed factorial model, so we proceeded to rectify the model according to our own data. Seven items with low correlations were eliminated. The results obtained for this second model are presented in

Table 5. Values of the proposed model.

p Value	RMSEA	CFI	TLI	NFI
0.07	0.049	0.964	0.949	0.921
>0.05	<0.50	>0.90	>0.90	>0.90

Source: Research data.

.

According to [51], the obtained values confirm this assessment.

After review, it was considered that the remaining items could explain the information of those eliminated.

The questionnaire was therefore structured as shown in Figure 1. We then list the appropriate statistics, which, by virtue of sample size and response format, best fit the study (

Table 6. Structure of the questionnaire.

Factor	Identification Factor	Items
1	Culture of network use	19, 20, 21 y 31
2	Network dependency	1, 4, 6, 7, 8, 10, 15 y 16
3	Safety and harassment on networks	25, 26, 28, 29 y 30
4	Use of networks for learning	11, 12, 13, 14, 17 y 18
5	Teacher support	32, 33, 34, 35 y 36
6	Family support	9, 37, 38, 39, 40

Source: Research data.

).

6. Results

A reliability study was carried out on the items that make up the final structure (

Table 7. Cronbach’s global alpha of the instrument.

Alpha of Cronbach	N of Elements
0.729	33

), whereby Cronbach’s alpha was calculated for the entire questionnaire, by category, item by item, as well as the value if these items were eliminated.

Analysed independently, each dimension yields the values shown in

Table 8. Cronbach’s alpha for each dimension of the instrument.

Factor	Identification Factor	Alfa of Cronbach
1	Culture of network use	0.613
2	Network dependency	0.699
3	Safety and harassment on networks	0.701
4	Use of networks for learning	0.709
5	Teacher support	0.805
6	Family support	0.734

Source: Research data.

.

Taking into account the length of the questionnaire (33 items) and according to [52], the alpha values indicate that the instrument can be considered reliable overall. It should be noted that the data corresponding to dimensions 1 and 2 are below a value of 0.7, but even so, they can be accepted as reliable data.

7. Conclusions

In recent decades, interest in the presence of social networks and their involvement in learning processes has been increasing, and it is necessary to identify how this is carried out and what functions are given to these uses in students in compulsory education.

The obtained results express the satisfactory metric quality of the questionnaire, thus presenting a theoretically grounded and operationally defined instrument by considering four relevant dimensions: Spatial, environmental, perceptual and motivational. This approach can be considered relevant and differentiated in relation to other instruments, both in its theoretical approach and in its operationalization in items, thus making it consistent. The analysis of its technical characteristics shows adequate overall reliability in terms of internal consistency, which facilitates its applicability in different contexts.

On the other hand, it should be noted that the instrument presents an optimal content validity, guaranteed by the consistency and rigour of the theoretical work and the evaluations that the selection of experts from education and university research professionals conferred on the configuration of the instrument’s items.

To conclude, the proposed instrument represents a relevant contribution from a theoretical approach due to the approximation in the elaboration and dimensions that it addresses.