Developing the Social Ecology of Occupational Zoonoses Instrument: A Comprehensive Tool for Measuring Social and Behavioral Factors in Agricultural Settings

Abstract

This article presents the development and validation of a new instrument measuring social, cultural, and behavioral factors influencing exposure to occupational zoonoses in agricultural settings. The Social Ecological Model (SEM) and the Standards for Educational and Psychological Testing were used as guiding frameworks to ensure the instrument’s validity. The instrument’s content was compiled by combining the results of a scoping literature review and an expert qualitative study. The instrument items were drafted, organized, and underwent a meticulous process of revision and adjustment. It was translated into Spanish and tested in one-to-one cognitive interviews with five volunteer agricultural workers. Exploratory factor analysis was conducted for construct discrimination, and bivariate regression analyses were conducted to explore the association with exposure indicators. Evidence of validity was obtained from four out of five sources of validity evidence according to the AERA/APA’s Standards of Educational and Psychological Testing. The Social Ecology of Occupational Zoonoses (SEOZ) was successfully used to determine social and behavioral factors associated with a higher risk of exposure to occupational zoonoses. Further use of the SEOZ can provide valuable insights into developing effective interventions to improve the health and well-being of agricultural workers.

1. Introduction

Agricultural workers in animal production systems are constantly confronted with numerous zoonotic threats during their work shifts, making them highly susceptible to infections [1]. Therefore, it becomes crucial to understand the intenerate web of social and behavioral factors that influence their preventive behavior. Understanding these factors holds the key to informing the development or improvement of interventions to reduce exposure risks [2]. Among the most high-risk working environments for zoonotic exposure are dairy cattle operations, with individuals working or residing on farms, farm visitors, service providers, and veterinarians being particularly vulnerable [3,4,5]. Pathogenic agents endemic to dairy farms can cause diseases in farmers, farm workers, service providers, and consumers of dairy products. Salmonella spp. [6,7], E. coli (O157:H7) [4,8,9,10,11], Campylobacter jejuni [12,13,14], and Cryptosporidium parvum [15] are the most commonly found pathogens and pose significant threats to the persons exposed to these. It is, therefore, critical to address zoonotic threats and develop effective preventive measures to ensure the overall health and well-being of agricultural workers.

Preventing zoonotic diseases in animal–human interfaces can be challenging, as it is influenced by the complex social and cultural systems that drive preventive behavior [16]. It has been demonstrated that the behavior of the at-risk person can affect their exposure to infectious agents [17,18]. Comprehending the intricacies of the social and behavioral factors at play is essential for crafting interventions that can effectively mitigate the impact of occupational zoonoses. The Social Ecological Model (SEM) serves as a pragmatic framework, intricately structured into hierarchically organized levels (Figure 1), each contributing to a comprehensive understanding of the factors influencing preventive behaviors [19,20]. At the intrapersonal level, individual characteristics and attitudes come to the forefront, influencing how one perceives and responds to zoonotic risks. Interpersonal dynamics within social relationships dictate the exchange of information, support, and behavioral norms, thereby shaping preventive behaviors. Moving up the hierarchy, the organizational level delves into the role of workplaces and the policies that foster or impede preventive practices. At the community level, the collective social context and cultural norms come into focus, impacting how individuals and groups interact with zoonotic risk. The enabling environment, representing the broader sociopolitical and economic factors, exerts its influence by facilitating or constraining access to resources necessary for preventive action. Together, these components intricately weave the fabric of the SEM framework, enabling a comprehensive exploration of the multifaceted influences on zoonotic disease prevention behaviors. This model provides a useful framework for better understanding of the multiple factors that impact preventive behaviors on farms.

To the extent of our knowledge, no tools are available to measure the social ecological factors influencing the prevention of infectious diseases in agricultural workers. The creation of novel instruments constitutes a fundamental facet of epidemiological research. Equally important is the validation of these instruments, which allows the correct interpretation of research findings and improves the overarching integrity of the entire research.

For validation purposes, there are many sources of theoretical and methodological frameworks that can guide the systematic assessment of an instrument’s robustness. The Standards for Educational and Psychological Testing (namely the Standards hereafter) is considered the best practice in psychometrics and provides guidelines for developing and evaluating psychometric instruments [21]. The core of the measurement validity knowledge in the Standards is based on the affirmation that validity of a measurement is not as an absolute characteristic but is instead a continuum of evidence that can be accumulated from various sources [22]. Therefore, according to the Standards, there are five sources of evidence to evaluate measurement performance: evidence based on content, evidence based on response processes, evidence based on internal structure, evidence based on the relationship of responses with other variables, and evidence based on the consequences of applying the instrument. These new definitions and concepts adopted from evidence based science have resulted in the Standards becoming a comprehensive framework to guide researchers through developing and validating psychometric instruments to ensure that research results are valid and meaningful [21,23].

This study aimed to develop and demonstrate evidence of measurement validity of an instrument measuring social, cultural, and behavioral factors influencing exposure to occupational zoonoses in agricultural settings. For this purpose, we describe the process of developing a new instrument for measuring the social ecological factors related to the prevention of occupational zoonotic diseases at the human–animal interface. Furthermore, we demonstrate and discuss its diverse sources of evidence of validity using the Standards for Educational and Psychological Testing as a guidance framework.

2. Materials and Methods

As an essential task of a larger research project, a psychometric instrument titled the Social Ecology of Occupational Zoonoses (SEOZ) was meticulously developed. This instrument served as a channel for evaluating the multilayered impact that personal, interpersonal, and organizational factors have on the exposure to occupational zoonoses in agricultural settings [24]. The instrument development was based on three fundamental principles: (1) appropriateness, (2) quality and clarity, and (3) validity. To assure these, a stepwise development process was conducted, including content assessment studies, critical revisions and translation procedures, and evaluations of four sources of measurement validity evidence.

2.1. Content of the Instrument

The instrument’s content was determined in a deductive manner guided by the Social Ecological Model (SEM) [19,20]. As mentioned in the introduction section, this model comprises factors that describe the ability to affect or change preventive behavior (Figure 1). It was decided beforehand that the instrument will focus on measuring factors only in the first 3 levels, i.e., intrapersonal, interpersonal, and organizational measures. This strategic decision stems from the overall objective of the project in providing grounds that allows the development and improvement of current farm-based prevention programs.

Two approaches were conducted independently to assess the instrument’s content: (i) a scoping literature review and (ii) a qualitative study with experts.

2.1.1. Scoping Literature Review

The aim of the scoping literature reviews was to provide a comprehensive overview of existing research that report using the SEM framework in relation to the prevention of infectious diseases in any setting. For this purpose, the following search string was developed: (“social-ecological” OR “socio-ecological”) AND (model OR framework) AND (prevention OR intervention) AND (“infectious diseases” OR “communicable diseases” OR “transmissible diseases”), which was translated and applied to the following repositories: Science Direct, PubMed, EBSCO (Medline and Academic search premiere), and Web of Science core collection. An initial manual selection of articles was conducted based on the titles and abstracts. Then, the selected articles underwent full-text review and data extraction for further analysis.

2.1.2. Experts Qualitative Study

A crowdsourcing participatory qualitative study with subject matter experts was conducted to assess the specific SEM components that could be relevant to the prevention of occupational zoonosis in animal production settings. The inclusion criteria aimed at selecting individuals with expertise and practical experience in animal production, zoonotic diseases, or related fields, who have contributed to interventions or research in zoonosis prevention. This included university veterinarians, epidemiologists, psychologists, and occupational health specialists involved in preventing occupational zoonoses. Their selection criteria included not only their professional roles but also their willingness and availability for participation, effective communication abilities, a minimum of three years’ experience in the field, and expertise demonstrated by the publication of at least one article in the topic within the last two years.

A responder-driven sampling strategy, akin to a snowball sampling approach, was employed. In this method, the initial participants that comply with the inclusion criteria were selected. Subsequently, each of these participants, acting as “seeds,” referred additional experts who fulfilled the stipulated criteria. This approach not only facilitated the identification of relevant subject matter experts but also maintained the study’s focus on those actively engaged in preventing occupational zoonoses. The use of responder-driven sampling allowed for the organic expansion of the participant pool, ensuring a diverse array of perspectives and insights crucial for a comprehensive qualitative assessment.

Expert participants were invited and asked to respond to a semi structured questionnaire with open-ended questions seeking to elucidate the potential relevant components of the SEM for the prevention of zoonotic diseases in farms, from the perspectives of the experts.

The responses underwent qualitative data analysis using framework-guided thematic analysis [25]. Briefly, the data, comprising expert responses, were reviewed to identify recurring patterns, nuances, and salient points. Subsequently, the SEM framework was applied as a lens through which the emerging themes and subthemes were systematically categorized and organized. These thematic insights were intricately linked to the specific levels of the Social Ecological Model, facilitating a comprehensive understanding of their alignment and relevance within this conceptual framework.

2.2. Item Creation and Review

Based on the findings of the previous content studies, another comprehensive scoping literature search was performed, but in this case, it was aimed to identify relevant previously validated instruments containing constructs or items identified in the content studies. For content sections not identified in the literature review, original items were drafted from scratch in English. The items were reviewed using Fowler Jr (2013) [26] recommendations, i.e., avoid double-barreled questions, low clarity, ambiguousness, use of jargon, and unbalanced or biased questions. Additionally, the items underwent a thorough quality evaluation on correct spelling, wording, structure, and grammar. The questionnaire was initially organized according to the levels of the SEM. At this stage, the items were also presented for review to four volunteer graduate students from the Departments of Animal Sciences and Clinical Sciences of Colorado State University (CSU). Suggestions, questions, comments, or concerns were recorded and later discussed and adjusted (e.g., rewording, dropping, merging, splitting, etc.).

2.3. Translation Process

The target population demographic is mostly Spanish-speaking laborers; thus, two native-speaker translators, both possessing veterinary and epidemiology backgrounds (EC and JP) and bilingual proficiency, independently translated the questionnaire into Spanish. The two versions were contrasted, and differences were discussed and conciliated. Notably, the translators took careful consideration to prevalent local cultural jargon used by the farm workers. The Spanish version was presented to a third bilingual expert with field experience (NR) for fine adjustments.

2.4. Cognitive Interviews

After approval from the CSU–Institutional Review Board (CSU-IRB), in-person cognitive interviews were conducted with five volunteer workers from a nearby dairy farm. Briefly, the participants were informed of the purpose of the interview and were prompted to provide their perception of the questionnaire’s relevance, intelligibility, and overall clarity. Field notebooks documented the interview proceedings, and the gathered data was analyzed and used for refining the questionnaire items. Remarkably, these interviews led to revisions in both the item sequence and the questionnaire’s format, enhancing its general usability.

2.5. Data Collection and Participants

Upon comprehensive review and integration of all received feedback and recommendations, the refined instrument was delivered to several volunteer workers from dairy farms in Northern Colorado. In adherence to the ethical protocol, a short verbal introduction along with an informed consent letter were presented to the potential participants. Modest monetary compensation was offered to cover their time and any additional transportation fees incurred due to participation. The selection of workers included those stationed in various areas, including milking parlors, calf rearing, maternity, and hospital sections. This strategic selection aligned with the recognized high-risk profile for exposure to zoonotic diseases in these specific work sites. The instrument was administered in a one-to-one interview by a trained researcher (JP) following Hartge and Cahill’s [27] recommendations. Alongside, as part of the general project activities, microbiological samples were collected from the workers’ protective clothing and analyzed in the laboratory to confirm the presence of zoonotic agents. A detailed description of this process can be found in our earlier publication, Palomares et al. [28].

2.6. Statistical Analysis

In preparation for data analysis, variables and observations marked by a considerable proportion of missing responses exceeding 20% were excluded. Subsequently, a Minimum Residual Factoring Exploratory Factor Analysis (EFA) characterized by varimax rotation and median imputation was conducted. The determination of the appropriate number of retained factors was guided by a parallel analysis. An item’s allocation to factors, as well as its interpretability, was based on loadings exceeding |0.5|. The derived factor scores were further included as dependent variables in regression analysis, to explore their statistical correlation with laboratory outcomes. All statistical analyses were conducted with R statistical software [29].

3. Results

3.1. Evidence of Validity Based on Contents

3.1.1. Literature Review

Following a full-text evaluation, a total of 19 manuscripts emerged, presenting compelling evidence of the utilization of the Social Ecological Model (SEM) within different dimensions of infectious disease prevention (refer to Figure 2). Remarkably, the majority of the scrutinized articles were centered around the prevention of HIV transmission, accounting for 9 out of the 19 manuscripts. Additionally, three articles focused on sexual health behavior and various other sexually transmitted diseases. Malaria, dengue, hepatitis C, and tuberculosis prevention were addressed in one paper each. Further enriching this array, two manuscripts directly addressed the prevention of zoonotic diseases, while another explored the potential links between social behavior and infectious agents in a general context. At the culmination of this examination, it was established that 16 distinct content items aligned with the overarching framework of the SEM.

3.1.2. Qualitative Study

From this task, eighteen experts were identified and contacted through email. Eight of the experts responded positively (44% response rate). Participant experts were two university veterinary hospital faculty members, one dairy industry consultant, one extension specialist, two epidemiologists and public health experts, and two occupational safety/industrial hygiene experts. Using the framework-guided thematic analysis, the initial coding yielded 58 codes. These codes were arranged into five content items that aligned with the SEM levels.

The 22 content items (16 from the literature review and 5 from experts) were arranged into the SEM components and used as the source of content for the instrument (Figure 3).

3.2. Evidence of Validity Based on Response Processes

From the second scoping comprehensive literature review on validated instruments, we identified and included eight constructs. These included: risk perceptions, self-efficacy, competing demands, norms and culture, work climate and culture, organizational responsibility, and organizational support. Additionally, 143 questions were composed from scratch to compliment the missing content items.

From the review process involving the team of researchers and the volunteer graduate students, the items were adjusted, combined, or removed. One hundred and twenty five (125) items remained after the initial revision and were organized into sections corresponding with the levels of the SEM. In addition, sections on demographics and preventive practices measures were included as per the greater research project objectives. From the cognitive interviews, seven additional items that induced bias, had an inclination for falsified responses, or seemed difficult for the responders were dropped.

A total of 54 workers were contacted and invited to participate. Of those, 42 workers volunteered to respond to the SEOZ instrument (78% response rate). The main reasons for non-participation were time restrictions or lack of transportation availability after the interview. On average, the instrument was completed in 35–40 min. The filled database had approximately 4% of missing values. Sixteen items and one responder were dropped due to their high proportion of missing responses (>20%); this procedure reduced the proportion of missing values to 1.65%.

3.3. Evidence of Validity Based on Internal Structure

Using the results of the parallel analysis, we obtained ten factors that had equal or larger eigenvalues than those randomly simulated, which were retained (Figure 4). Results from the EFA (Exploratory Factor Analysis) are presented in the Supplementary Materials. The retained ten factors accounted for 71% of the total variance (eigenvalues ranging from 9.13 to 5.15). Of the 102 items, 74 (73%) were relevant contributors to the component’s variance (correlation > |0.5|).

Construct discrimination was evident across most of the factors. The first factor (contributing with 8.95% of the total variance) contained the organizational construct of management perception and sense of farm belonging. The second factor (contributing with 8.63% of total variance) contained an intrapersonal construct involving aspects of knowledge and risk perception. The self-efficacy construct was compiled within the third factor (contributing with 8.02% of total variance). The fourth factor (contributing with 7.38% of total variance) contained the construct of perception toward peers and supervisors. The fifth factor (contributing with 7.37% of total variance) was a construct of attitudes towards reporting and seeking health care due to injuries. Similarly, the sixth factor (contributing with 6.89% of total variance) is a construct of attitudes towards reporting and seeking health care when aware of having symptoms of infection. Job satisfaction was compiled in the seventh factor (contributing with 6.67% of total variance). Safety culture and climate were compiled in the eighth factor (contributing with 6.2% of total variance). Furthermore, organizational responsibility and job control were constructs discriminated in the ninth and tenth factors, respectively (contributing with 5.79% and 5.04% of total variance).

3.4. Evidence of Validity Based on Relationships with Other Variables

Evidence of validity based on the relationships of questionnaire responses and laboratory outcomes was assessed a posteriori by regression analysis with factors as dependent variables and laboratory results from workers’ samples as dependent variables [28]. A significant association was found between four factors and exposure to potentially dangerous zoonotic agents. Self-efficacy and negative workplace perceptions were risk factors of exposure (OR: 1.43, 95% CI 1.11–2.22; and 1.22, 95% CI 1.02–1.53, respectively). In contrast, knowledge, risk perception, and good perceptions of supervisors and coworkers were protective factors (OR: 0.91, 95%CI 0.82–0.99; and 0.89, 95%CI 0.79–0.98, respectively). Please refer to our previous publication for further detail regarding these results [28].

4. Discussion

We successfully developed and evaluated an instrument tailored to measure the intricate complexity of social, cultural, and behavioral factors shaping the prevention of zoonotic diseases within the context of occupational settings in animal production. Guided by the Social Ecological Model as a theoretical framework, we adeptly curated the instrument’s content. Complimentarily, our approach adhered methodologically to the concepts and methods introduced by the AERA, APA, and NMCE within the Standards for Educational and Psychological Testing, effectively obtaining diverse sources of validity evidence. This developmental process was driven by our commitment to ensuring the precision and integrity of data collection, with a critical aim to not only inform but also enhance the development or refinement of prevention interventions.

We successfully used two complementary sources of content: a scoping review of the literature and a qualitative elicitation of factors from experts. Both literature reviews and expert assessment have been broadly used to assess content [30,31]. Finding convergence and complementarity between both approaches is evidence that the content items are relevant and representative of the content items and constructs that are intended to be measured [30,32].

Clarity and quality were effectively achieved through extensive review steps and cognitive interviews. These methods have been recommended as the preferred method for assessing response processes [33]. The significance of ensuring clarity and quality within research instruments cannot be overstated. Ambiguous or unclear questions can lead to participant confusion, potentially yielding inaccurate or incomplete responses. Moreover, such issues may compromise the validity and reliability of the collected data, thereby undermining the robustness of the research results. Conversely, a meticulously refined instrument enhances participant comprehension, thus bolstering the integrity of responses and ultimately augmenting the validity of the research findings.

Assessing the instrument’s dimensions with exploratory factor analysis provided evidence of validity based on internal structure [34]. While most components showed evident discrimination of constructs, we found individual items that did not belong to the construct. However, these few items had lower loadings < |0.6| than others, which were more representative of the constructs. Moreover, an exploration of potential sources of ambiguity or potential overlaps is recommended. Adjusting these items to more cohesively align with their respective constructs could further enhance the instrument’s robustness and fidelity in capturing the intended dimensions. This process, however, would benefit from a judicious balance to ensure that any modifications remain coherent with the original context and purpose of the instrument.

Finding significant associations between SEM factors and laboratory results indicating exposure to zoonotic diseases is evidence of validity based on relationships with other variables. Notably, factors such as self-efficacy, knowledge perception, peer and supervision perceptions, and management perceptions have been previously identified as drivers of occupational preventive behavior [35,36,37]. The implications of these findings resonate particularly in the efforts to develop or improve prevention programs to reduce occupational risks in similar settings. These identified influential factors can lead to targeted interventions and strategies. For instance, tailored prevention programs can be formulated focusing on enhancing self-efficacy levels among workers, promoting knowledge and risk perception, fostering positive peer and supervision interactions, and optimizing management engagement. As a potential result, it is expected that, empowering workers with the confidence to practice preventive measures, enhancing their understanding of risks, and cultivating a supportive organizational culture can cumulatively enhance occupational health and safety practices.

Evidence of validity based on the consequences of applying the instrument refers to the direct or indirect effects caused by the application of an instrument [38]. In this study, we did not evaluate whether the instrument’s application negatively affected the workers. However, following the ethical recommendations, we ensured that participation in the study did not cause any harm or negative consequences to the participants. We ensured that survey responses could not be traced back to its respondents to avoid any potential confidentiality breach. In addition, we tried not to interfere with the workers’ routine. All the participants were asked to participate in the study voluntarily and could opt out of participation at any time. Requesting permission from the farm’s administration, performing the interviews during non-shift hours, and maintaining strict confidentiality of participants are indicators of our efforts to avoid negative repercussions.

Although the accumulative amount of validity evidence found for the SEOZ instrument is promising, the findings presented here should not be considered without considering its limitations. We did not conduct a quantitative content study for assessing the content of the instrument, but a scoping review and a qualitative expert study approach were appropriate for the study’s purpose and the research project’s goals. The cognitive interviews were conducted in a small sample of a very homogeneous group of workers. In retrospect, one of the study’s limitations was the restricted access to diverse farms and a relatively small sample of workers, hindering the study’s heterogeneity. To address this issue from today’s standpoint, potential strategies could have been employed, such as forming collaborative partnerships with agricultural organizations, using stratified sampling to ensure varied farm types and regions, extending the data collection period, offering incentives to participant farms, and utilizing a mix of recruitment methods (e.g., online, telephone, in-person). These lessons can guide future research to enhance its representativeness and robustness. In addition, the language used, and the translation was conducted for a specific target population. Another limitation is that the SEOZ was developed for this in a specific research project, so its reliability was not evaluated. More extensive testing is required with a larger and heterogeneous sample to overcome these limitations.

5. Conclusions

In conclusion, developing and validating a robust psychometric instrument required greater effort than perceived or anticipated. We believe the SEOZ instrument’s development significantly contributes to occupational health and safety in agricultural settings. The SEOZ instrument can provide valuable insights into preventing and controlling zoonotic diseases in farm settings by measuring social, cultural, and behavioral factors. This instrument can inform the development of effective interventions to improve agricultural workers’ and their communities’ health and well-being.