**4. Discussion**

Studies show that immunization, one of the greatest public health achievements, is occasionally hampered by strong biological, social, and cultural reactions of the public [68], which is why the media and communication are extremely important for this issue. In such conditions, online media, which today have become the primary source of information, are of special importance, especially because of the fact that many parents receive information about vaccinations mostly through online sources [10]. However, despite the growing literature on vaccination and the role of online media in modern times, and thus in the field of medicine and public health, we are still trying to explore and understand parents' attitudes about vaccinating children, as well as why and how parents have di fferent levels of trust in vaccines. This is particularly important given the research by Larson et al. [44], who found that socioeconomic status, media information, and attitudes and motivations regarding health care, as well as knowledge and awareness of the need for vaccines, are related to indecision to give the vaccine. This is especially present in less developed countries.

Thence, research on the impact of online media on parents' attitudes towards child vaccination is, according to the authors' knowledge, the first integrated scientific study in three countries of the Western Balkan region—Montenegro, Serbia, and Bosnia and Herzegovina. Therefore, the aim of the research was to investigate the level of influence that online media, as a form of marketing communication, have on the formation of attitudes toward the vaccination of children, in order to direct e ffectively and e fficiently the potentials of this form of communication towards the strengthening and improvement of public health. Discussion of achieved results is presented below.

The obtained results confirmed the hypothesis that the identified characteristics of the respondents (gender, country of origin, age, marital status, and number of children) have a significant influence on attitudes towards vaccination of children, which is correlated with the results of other studies [69] in which demographic characteristics stand out as predictors of vaccines. Furthermore, the contingency analysis showed that the majority of respondents have high confidence in vaccines, when their characteristics, such as gender, country of origin, age, marital status, and number of children would be analyzed. The next largest category are parents who have a neutral attitude, i.e., are indi fferent to the vaccination of their children, because this category consists of 27.87% of respondents. In third place are parents who completely trust in vaccines, and they make up 18.9% of all surveyed parents. The penultimate category is that of parents with low confidence in vaccines, and they represent 15.32% of all surveyed parents. In the last place are parents who do not believe in the positive e ffect of vaccinating their children. They make up only 4.02% of the surveyed parents in this study. Based on the obtained results, we may conclude that more than half of the respondents have a high or complete level of confidence in vaccines. However, about one-fifth of respondents do not trust or have a low level of trust in vaccines, which is correlated with the results of other studies [22–24] which confirm that distrust leads to vaccination rejection. These results sugges<sup>t</sup> that certain actions must be taken in order to improve the existing situation. In that sense, the implementation of social marketing may be of grea<sup>t</sup> importance in order to strengthen trust in vaccines. Thus, social marketing in the field of immunization has a general social character, that is, it implies the implementation of marketing strategies in order to achieve specific goals of behavior oriented to the common good. In other words, the adequate implementation of social marketing may influence the change of behavior, that is, it may influence the strengthening of trust in vaccines. Thus, social marketing becomes important factor of improving public health.

Furthermore, the analysis of the variance of the respondents' data on the stated characteristics showed that the expected value for each individual characteristic (except for the country) di ffers in relation to the attitudes of the parents, i.e., the level of trust in vaccines. These results are in correlation with the results of other studies, in which it is pointed out that important factors of influence are gender, age, level of education, and knowledge about immunization [53–56].

Findings of the research have shown a strong link between online media and parents' attitudes, which is correlated with the results of some studies in other countries [8,32]. Namely, the analysis of equality of expected values for the characteristics of the research on parents 'attitudes towards vaccines in the situation when using online media as a source of information has shown that respondents' attitudes di ffer significantly depending on which aspect of online media is used as a source of information. In accordance with the above, the results of the descriptive statistics have shown that 37.7% of cases used online health websites, then blogs and forums (33.6%), when collecting online data about vaccines and indicate that the two sources were the most trusted online media outlets. For 11.7% of respondents, Facebook is the primary online source for information about vaccines, while 4.2% prefer other social networks (Twitter, Instagram). The participation of parents who were more informed about the arguments about the pro-vaccine (49.3%) and arguments against the vaccination of children (50.7%) is almost equal, which makes sense, because all those who ask for additional information should want to know both positive and negative arguments on this topic. In collecting information on vaccination through online media, 53.2% of parents stated that their spouse was also involved, while 46.8% of respondents stated that they collected the information for themselves. When asked how they rate their understanding of the material they received through online media, 55.6% of respondents consider their understanding to be average and 16.8% have a complete understanding, while the remaining respondents are not satisfied with their understanding of the information they obtained online. The "pressure" of pro-vaccine campaigns was felt by 13.74% of parents, while 40.55% felt the "pressure" of anti-vaccine campaigns, and the remaining respondents did not feel the pressure of these campaigns. Lastly, it is interesting that only 11.48% of respondents share vaccine information with others online, while 88.52% do not, although they use online sources to inform themselves. These results may be explained by the fact that parents in the analyzed area still use "o ffline" sources of information to form attitudes about vaccines compared to online media. The most common source of information are doctors, i.e., pediatricians (27.4%), while online media are used in combination with information obtained from doctors or family members and friends. Since this is the way they

ge<sup>t</sup> information, they still do not have the habit of sharing their views on vaccines to other parents in public, i.e., online. Another reason is that parents in this study stated that they felt "the pressure of the online campaign against vaccines" (39% of respondents), so they did not want to be part of that campaign and impose their views through online media on other parents.

Based on the results obtained through logistic regression, we may conclude that the following demographic groups of parents are especially vulnerable to the formation of attitudes about vaccines: women, because as mothers they are more sensitive and easier to "scare" them with certain texts about harmfulness of vaccines for the health of their children; parents of younger age ("millennials") who are influenced by digital technologies and use them in all spheres of life, including raising children, and it is not surprising that they are the most vulnerable age group of parents, who under the influence of online media form a negative attitude towards vaccines; parents living in common law marriage, as well as parents with more children, one of the key reasons being that more children give little free time to parents to devote to researching and finding additional sources of information to form attitudes about vaccines, so they rely to online sources. Precisely these di fferences that exist in di fferent demographic groups when forming a negative attitude towards vaccines show the following: parents regularly follow texts about the negative e ffects of vaccines through online media and, based on this information, form di fferent attitudes towards vaccines (logistic regression, for example, showed that mothers are more likely to form a negative attitude towards fathers, etc.), then that parents look di fferently at texts from online media about the negative e ffects of vaccines and thus view di fferently the importance of this information in forming attitudes towards vaccines ("millennials" will be more often influenced by information from online media compared to parents of older generations, etc.), parents di fferently estimate the truth of texts from online media about the negative e ffect of vaccines on children's health (parents with more children are more likely to think these texts are true compared to parents who have one or two children, etc.), and parents react di fferently to these texts and some of them believe that they are texts that are not based on scientific facts (parents with a doctorate), while there are parents who believe that the texts are quite true (parents with a lower level of education in in relation to doctors of science, etc.), but in the end there are di fferences in the impact of this information obtained from online media on the final formation of attitudes about vaccines for di fferent demographic categories of parents.

On the other hand, the analysis of the η2 coe fficient has shown that the greatest value in the opinion of parents that the texts on the online media about the negative e ffects of the vaccine influence the formation of the parents' attitude (η<sup>2</sup> = 0.216). Namely, the research confirmed that parents were mostly under pressure from the negative campaign about vaccines through the online media (39% of respondents). Then, the research showed that parents trust the information they receive from medical websites the most (51.4% of respondents). Finally, the dominant group in the study consisted of mothers (70.1%), i.e., females, and the results of the logistic regression indicated that mothers were more influenced by the online campaign against vaccines. The combination of these factors has led to the fact that online media really play a significant role in forming the negative attitudes of respondents towards vaccines. Thus, based on the applied methods, we may conclude that online media, as a form of marketing communication, has a significant influence on the formation of parents' position on vaccination of children, that is, trust in vaccines.

On the other hand, the ANOVA test and the eta coe fficient confirmed that the country of origin of the respondents does not a ffect their attitude towards the vaccination of children. However, based on logistical regression, we concluded that, despite the fact that parents who come from these three di fferent countries (Serbia, Montenegro, and Bosnia and Herzegovina) have approximately the same attitudes towards vaccines, the influence of online media on their attitude is di fferent. It turned out that parents from Bosnia and Herzegovina are somewhat more susceptible to forming a negative attitude towards vaccines under the influence of online media than parents from Serbia and Montenegro. One of the potential reasons for this result may be found in the demographic characteristics of the respondents by country. While the demographic structure of the surveyed parents in Serbia and

Montenegro is approximately the same, in Bosnia and Herzegovina, certain deviations have been noticed, which primarily refer to the level of education. Namely, in Bosnia and Herzegovina, there is a smaller number of respondents with the highest education (1.7%) compared to respondents with other lower levels of education. The dominant education is high school (44.2%), which, according to the results of logistic regression, has the greatest chance of forming a negative attitude of parents towards vaccines based on online media.

Finally, the authors tried to investigate whether there was a di fference in attitudes of parents coming from di fferent countries, regarding the influence of online media on formation of attitudes. When it comes to potential di fferences in the analyzed countries, the results of the survey have shown that the attitudes of the parents are the same, regardless of whether the respondents are from Montenegro, Serbia, or Bosnia and Herzegovina (η<sup>2</sup> = 0). This should not be surprising having in mind the fact that these are developing countries, which have relatively similar legislation in the field of vaccination, and that all of them seek to harmonize public health policies in accordance with European standards. Thus, based on the above analysis, it may be noticed that there is no high value of deviation per state in the relation of the analyzed variable.

Based on all the above, we may conclude that there are a number of factors that a ffect parents' attitudes toward vaccinating children and that online media are an important factor that determines parental behavior. In addition, research has shown that social marketing may be an important determinant of strengthening trust in vaccines, as well as improving of public health.

#### **5. Conclusions and Implications**

Today, information technologies have changed the paradigm of communication between medical professionals and public. The wide availability of information through the penetration of the mass media has played a significant role in encouraging parents [70–72] to form attitudes based on facilitated access to the media, especially when it comes to online media.

Concerns about vaccination have become a global phenomenon and led to the search for answers to the question of who has the greatest influence on the parental attitude toward vaccination: medical professionals, internet, or family and social environment [73]. In this regard, the widespread availability of information through online sources plays a significant role in the formation of attitudes [74]. Accordingly, in recent years, the role of Facebook as a source of pro and antivaccine information has also been analyzed [75,76]. Hence, in this area, social marketing has become especially important, in which activities are aimed at changing or maintaining the behavior of people for the benefit of individuals and society as a whole.

A large number of studies have been published over the years on the degree of exposure to vaccines [18–20]. There is also research on the impact of certain forms of online media (e.g., di fferent forms of social media) on parents' attitudes towards child vaccination. However, although several studies have examined parents' attitudes toward vaccination of children, few of them, according to the authors, have integrated research in the way given in this study. Thus, unlike most previous research on vaccines, in which arguments for or against vaccination were mainly emphasized, the authors wanted to determine if there was a correlation between online media and parents' attitudes towards vaccination through analysis in three countries in the Western Balkans region (Montenegro, Serbia, and Bosnia and Herzegovina), bringing the topic in the context of the application of social marketing in order to strengthen trust in vaccines and improve public health.

The authors developed and empirically tested a model that examined the relationship between influencing factors and parents' attitudes toward vaccination of children. The authors used advanced descriptive statistics, as well as the ANOVA method, which allows to determine the individual influences of the analyzed factors related to the attitudes of parents about the vaccination of children, especially through the prism of online media. Logistic regression was applied to obtain a more precise answer to the question of the relationship between the demographic characteristics of the respondents and their negative attitude towards vaccines based on information from the online media. Furthermore, in order to further examine the importance of a certain way of using online media on the formation of attitudes, i.e., parents' trust in vaccines, the eta-coe fficient was used. The analysis of the η2 coe fficient showed that the greatest influence on the formation of attitudes, i.e., parents' trust in vaccines, when we look at the online media, have texts about the negative e ffects of vaccines that may be found on such sources.

The study has shown that the analyzed variables from the model have a significant influence on parents' attitudes toward vaccination of children, and that they strongly reflect on the level of confidence in vaccines. Further, the study has shown that their impact varied depending on the factor being observed.

These conclusions may create more implications for decision-makers.

Based on the achieved results, the authors sugges<sup>t</sup> that decision-makers should pay more attention to contemporary forms of social marketing, as well as online media, in order to focus their potential more on improvement of public health, as well as to avoid the harmful impact that these forms of communication may have an opinion on vaccines. In a broader sense, the authors conclude that these forms of communication a ffect not only attitudes about vaccination but the improvement of public health, which opens space for further research on this topic.

Professional support must be present in all forms of application of online media and social marketing in the field of immunization as important segmen<sup>t</sup> of public health. In this regard, the provision of information by health professionals and the quality of their information are essential for the decision to vaccinate or otherwise [77,78], which is extremely important for communication through social media, which are very common today.

Decision makers have to be aware that positive attitudes are key to a high level of confidence and that it is necessary to integrate a number of factors in order to maximize the application of social marketing in the field of medicine and public health.

In addition to the practical, the authors believe that this paper has a significant theoretical contribution. Namely, these results, except expanding the base of empirical research on the application of online media and social marketing in order to strengthen the trust in vaccines, o ffer added value to the existing literature by analyzing this concept in di fferent countries and according to di fferent factors. Additionally, the analysis is brought into the context of improving public health, which makes the work a special value. Finally, this analysis goes beyond the national framework and presents an analysis in a multi-country context, thus contributing to theorizing on the topic of social marketing and online media in the field of medicine and public health in an international context.

Finally, having in mind actual debates on the far-reaching consequences that the world will su ffer globally due to external shocks caused by the new Covid-19 corona pandemic, we believe that, in the future, more detailed analyses of the impact of online media on vaccine confidence may be conducted exactly in this area. Not only in less developed countries, but also in developing countries, it is evident that the e ffects of the pandemic of the new coronavirus Covid-19 will be manifested through direct influences on people's perception and attitudes about vaccination, especially when it comes to vaccination of children. In order to avoid the negative e ffects that online media may produce, decision-makers need to develop adequate online communication and social marketing strategies, for which the findings o ffered in this study may be very helpful and useful.

**Author Contributions:** Conceptualization, B.M., A.J.S., and B.D.; methodology, B.M. and T.B.V.; software, T.B.V.; validation, B.M., A.J.S., and B.D.; formal analysis, B.M., B.D., and E.B.; investigation, B.M. and A.J.S.; resources, B.M., B.D., and E.B.; data curation, B.M., A.J.S., B.D., and T.B.V.; writing—original draft preparation, B.M., A.J.S., and T.B.V.; writing—review and editing, B.M., B.D., and E.B.; visualization, B.D.; supervision, B.M. and B.D. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **Appendix A Survey: The Impact of Online Media on Parents' Attitudes toward Vaccination of Children**

Respected,

This survey aims to determine the impact of online media on the attitudes of parents about the vaccination of children. The research is conducted for scientific purposes, with the aim of improving the situation in the subject area. The questionnaire contains 20 questions. The research is anonymous, and the results will be observed at the aggregate level.

> Thank you for your participation!

	- • Male
	- • Female
	- • 18–24
	- • 25–29
	- • 30–34
	- • 35–40
	- • 41–45
	- • More than 45
	- • Montenegro
	- • Serbia
	- • Bosnia and Herzegovina
	- • Primary school
	- • Secondary school
	- • College
	- • Faculty
	- • Specialist
	- • Master
	- • PhD
	- • Married
	- • Extramarital union
	- • Divorced
	- • Widow
	- • 1
	- • 2
	- •3


	- • Doctors
	- • Family/relatives
	- • Professional journals
	- • Books
	- • Flyers/brochures
	- • Internet/social networks
	- • Mass media (TV, newspapers, etc.)
	- • Friends/associates
	- • Other (please specify) \_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_
	- • Facebook
	- • Other social media (Twitter, Instagram, etc.)
	- • Wikipedia
	- • Blogs/forums
	- • Medical web sites
	- •Other web resources (please specify) \_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_
	- •I didn't use internet resources
	- •Yes

	- • Little or no understanding at all
	- • Poor understanding
	- • Average understanding
	- • Above average understanding
	- • Complete understanding
	- • Arguments about pro-vaccine
	- • Arguments against vaccine
	- • Yes
	- • No
	- • Conversation
	- • Brochures/informative sheets
	- • Suggested web pages/online sources
	- • Referring to media

	- • Yes

	- • Yes, I have felt the "pressure" from pro-vaccines campaigns
	- • Yes, I have felt the "pressure" from anti-vaccines campaigns
	- • No, I haven't felt the "pressure" from pro-vaccines campaigns
	- • No, I haven't felt the "pressure" from anti-vaccines campaigns
	- • I don't believe at all
	- • Low level of trust
	- • Neutral
	- • High level of trust
	- • I really completely believe
	- • Vaccination is necessary in order to prevent illnesses
	- • I believe my pediatrician
	- • Vaccinations should be individual choice of parents
	- • Immunity of illness is better than immunity of vaccination
	- • Vaccination is ideal: once vaccinated, children may not ge<sup>t</sup> the illness against which they were vaccinated
	- • It is necessary to have as more vaccinations as possible
	- • Without vaccination, a child may become ill and, consequently, cause others to ge<sup>t</sup> it.
	- • Vaccines are also given for illnesses that children probably will not get.
	- • Vaccination is mostly safe for children
	- • Vaccines consist of harmful substances
	- • Plan of vaccination includes too many vaccines at the same time
	- • Children ge<sup>t</sup> more vaccines than is useful for them
	- • Vaccinations are harmful and they should be avoided
	- • I am satisfied with the efforts that the State (relevant institutions) put forth regarding the vaccination

Note: Likert's questions are rated on a scale from 1 to 5, where 1 = I completely disagree, 2 = I disagree, 3 = I cannot judge, 4 = I agree, and 5 = I completely agree.

	- • I regularly read texts in online media about negative effects of vaccines
	- • Texts in online media about negative effects of vaccines significantly influence on my attitudes about vaccines
	- • Texts in online media about negative effects of vaccines are mostly correct
	- • Texts in online media about negative effects of vaccines are mostly based on fears and speculations, not on scientific facts

• Texts in online media about negative effects of vaccines today have more importance than they deserve

Note: Likert's questions are rated on a scale from 1 to 5, where 1 = I completely disagree, 2 = I disagree, 3 = I cannot judge, 4 = I agree, and 5 = I completely agree.

	- • Social networks
	- • Medical web resources/sites
	- • Blogs
	- • Forums
	- • Online newspapers and magazines
	- • Other online resources \_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_
