**Table A1.** PRISMA Checklist \*.



**Table A1.** *Cont*.


\* According to [46].

## **Appendix B**

**Table A2.** Quality Criteria Checklist \*.



**Table A2.** *Cont*.

\* According to [54].

## **References**


## *Article* **Design and Development of an Instrument on Knowledge of Food Safety, Practices, and Risk Perception Addressed to Children and Adolescents from Low-Income Families**

**Sueny Andrade Batista <sup>1</sup> , Elke Stedefeldt <sup>2</sup> , Eduardo Yoshio Nakano <sup>3</sup> , Mariana de Oliveira Cortes <sup>4</sup> , Raquel Braz Assunção Botelho <sup>1</sup> , Renata Puppin Zandonadi <sup>1</sup> , António Raposo 5,\* , Heesup Han <sup>6</sup> and Verônica Cortez Ginani 1,\***


**Abstract:** In the fight against foodborne diseases, expanding access to information for different groups is needed. In this aspect, it is crucial to evaluate the target audience's particularities. This study constructed and validated an instrument containing three questionnaires to identify the level of knowledge, practices, and risk perception of food safety by low-income students between 11 and 14 years old. The following steps were used: systematic search of the databases; conducting and analyzing focus groups; questionnaires development; and questionnaires analysis. After two judges' rounds, the final version was reached with 11 knowledge items, 11 practice items, and five risk perception items. The content validation index values were higher than 0.80. The adopted methodology considered the students' understanding and perceptions, as well the appropriate language to be used. Besides, it allowed the development of questionnaires that directly and straightforwardly covers the rules set by the World Health Organization for foodborne disease control called Five Keys to Safer Food (keep clean; separate raw and cooked; cook thoroughly; keep food at safe temperatures; and use safe water and raw materials). Its use can result in a diagnosis for elaborating educational proposals and other actions against foodborne illness in the most vulnerable population.

**Keywords:** adolescents; children; food safety; hygiene practices; low-income; risk perception

## **1. Introduction**

Even though the prevalence of outbreaks of foodborne disease (FBD) is underreported, it results in socioeconomic losses worldwide. In 2010, there were 600 million cases, with 420,000 deaths, caused mainly by agents of diarrheal disease. FBD represents a challenge for food security, human health, economic prosperity, agriculture, market access, tourism, and sustainable development.

Another important aspect is an imbalance in the disease's distribution with a higher incidence in vulnerable populations, either economically, due to comorbidities, or by the age group (children and the elderly). In addition to being more susceptible to disease complications, such as dehydration, these populations may be more exposed to diseasecausing agents, mainly biological ones, viruses, parasites, and bacteria). Fungi and their

**Citation:** Batista, S.A.; Stedefeldt, E.; Nakano, E.Y.; Cortes, M.d.O.; Botelho, R.B.A.; Zandonadi, R.P.; Raposo, A.; Han, H.; Ginani, V.C. Design and Development of an Instrument on Knowledge of Food Safety, Practices, and Risk Perception Addressed to Children and Adolescents from Low-Income Families. *Sustainability* **2021**, *13*, 2324. https://doi.org/ 10.3390/su13042324

Academic Editor: Rajeev Bhat

Received: 14 January 2021 Accepted: 17 February 2021 Published: 20 February 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

associated toxins have also been characterized as a significant safety problem worldwide [1]. This fact may result from a lack of knowledge and an environment conducive to such agents' proliferation, such as inadequate conditions for handling and preserving food in the domestic environment [2].

A study carried out by Hadler et al. [3] identified this situation of greater vulnerability. Children and the elderly in poverty in the United States showed a higher incidence of Salmonellosis. The study highlighted the fact that 91.1% (n = 48.111) of the illnesses were acquired domestically. European Union data corroborate with the importance of domestic kitchens in the epidemiological scenario of FBD [4]. In Brazil, of the total number of FBD outbreaks reported between 2007 and 2017, 38.3% (n = 2922) of the cases occurred in homes, considered the places with the highest occurrence of FBD [5].

Responsibility for food safety must be taken together. Government agencies must fulfill their role in educating, informing, regulating, supervising, and ensuring fair and reliable business relationships. Besides complying with regulations and protecting food from contamination, producers from different sectors of the economy must provide consumers with clear and accurate information on handling food. Finally, consumers must recognize their role, following the relevant instructions, and adopting appropriate food hygiene measures [6,7].

Nevertheless, the high incidence of FBD in the home environment is possibly related to failures in adopting safe procedures during food handling. Individuals who handle food have a great responsibility in these cases, and they are considered the main ones responsible for foodborne outbreaks. Numerous studies have been developed to explore the food handler's involvement in controlling FBD [8–14]. These studies aimed at understanding these individuals' knowledge, practices, attitudes, behaviors, risk perception, and optimistic bias to develop diagnostic strategies that subsidize effective actions to face FBD. For this purpose, the studies use instruments developed for the target population according to the research proposal, such as specific questionnaires. They usually approach an adult audience, composed of food handlers from food services or consumers from these establishments.

Understanding such factors is essential since research related to consumers in general and food handlers demonstrated that unsafe handling practices are frequent, despite happening through an acceptable level of knowledge [8,9,15–20]. There is a need to cover other knowledge areas and age groups [8]. Children and adolescents are important targets for food safety training actions. They are at a stage where their cognitive structures reach their highest development level [21], considered crucial for developing knowledge, skills, and habits, including food safety. Therefore, with early intervention in young people's education, it is possible to influence future behaviors and habits before they are formed, benefiting the student himself, his family, and people who will be under their responsibility in the future. A solid foundation for healthy behaviors is then established [22–24].

Other aspects reinforce the importance of considering them, such as the great potential as agents of transformation [23–25], evidence of being ready to fulfill in the preparation of meals [17,22,26–29] and finally, a tendency to feel more confident in their perceptions of personal risk [24].

Despite concrete evidence about this group's important role in the area, the developed studies are more dedicated to adult consumers and their food handling practices. The fact reveals the need for more significant investments in studies and actions with this audience [28,30].

Studies directed to understand the difference in the socioeconomic status of risk factors for FBD are essential. Mainly considering the large proportion of FBD originating from the domestic environment [4,5,31,32] and the greatest vulnerability of populations in poverty situations. In this way, it will be possible to strengthen the individual role, including economically less affluent classes, as an essential link in the food supply chain [2,3].

When considering socioeconomic aspects, the studies point out significant factors that impact knowledge and the adoption of good food safety practices, such as level of

education, place of residence [33], and location of the school (rural or urban), the latter when it comes to children and adolescents [34]. Other factors, such as low income and low awareness, contribute to a lesser willingness to adhere to an environment compatible with food safety in developing countries. In these countries, the difficulty of access to drinking water, safe cooling, fuel cost for cooking or adequate reheating of food, high environmental temperatures, inadequate sanitation, and low education levels discourages adopting appropriate practices [35–37].

Therefore, the connection between food safety education, school subjects, and the home environment becomes essential [38]. However, it is observed that studies with individuals in the school stage [8–14] present heterogeneity of methodologies, often adapted from other audiences, and not all of them present evidence of validation. They do not focus on economically vulnerable populations, and there may be bias not being applicable in adverse realities. Pawlowski [39] emphasizes that an instrument's development must necessarily include the target audience to formulate health programs and policies based on reality. In this case, challenges inherent to low-income individuals need to be considered to ensure that their specificities are addressed [39].

Interventions for younger consumers and investment in consumer education can reduce FBD, and savings are expected of up to ten dollars for every dollar invested [40]. In this sense, this research aims to construct and validate a structured questionnaire to assess knowledge, practices, and risk perception related to food safety in low-income students aged 11 to 14 years. The results serve as a starting point to design educational actions that aim to improve the population's food safety in which food challenges are faced daily.

## **2. Materials and Methods**

This exploratory research was conducted in the Federal District (FD)—Brazil and it was approved by the Ethics Committee of the College of Health Sciences of the University of Brasilia—CEP/FS UnB (CAAE nº 02033218.0.0000.0030). To participate in the study, three schools were selected, by convenience criteria, within a sample of the Project "Healthy eating and the school food production chain: what happens in public schools in the Federal District (FD)". The schools selected classes according to the schedule availability. The students' participation took place by presenting the terms of consent and free and informed consent.

In the form of three questionnaires, the instrument was developed for students aged 11 to 14 years. The environment of public schools was chosen to meet the criterion of low income since, according to the Programme for International Student Assessment— 2015 (PISA) [41], this target audience has the lowest ESCS (Index of Economic, Social, and Cultural Status) in Brazil.

A quality instrument's development requires well-defined steps and rigorous procedures that guarantee reliable indicators [39,42]. The methodology chosen for the study recommends a process with the inclusion of specific tasks and methods carried out in a determined time sequence. These steps indicate the instrument's psychometric strength and the guarantee of the constancy of its content with the constructs.

Thus, the development of the instrument was based on the model described by Pasquali [43]. The theoretical procedures respected the following steps: (i) systematic search in the databases; (ii) conducting and analyzing focus groups; (iii) questionnaires development; and (iv) questionnaires analysis (Figure 1).

zil).

**Figure 1.** Theoretical procedures for constructing the questionnaires for assessing knowledge, practices, and risk perception related to food safety for low-income students (Federal District- Bra-**Figure 1.** Theoretical procedures for constructing the questionnaires for assessing knowledge, practices, and risk perception related to food safety for low-income students (Federal District- Brazil).

## *2.1. Systematic Search in Databases 2.1. Systematic Search in Databases*

In addition to the database registered in Figure 1, the following documents were used to design the preliminary version of the questionnaire: Brazilian food safety resolutions based on texts from Codex Alimentarius [44]—RDC 216 and IN 16 [45,46]; *Base Nacional Comum Curricular* (BNCC)- Common Base National Curriculum [47], Protection Motivation Theory [48], Five Keys to Safer Food [49], and instruments developed by Da Cunha, Stedefeldt and De Rosso [50], De Andrade et al. [8], and Haapala and Probart [17]. In addition to the database registered in Figure 1, the following documents were used to design the preliminary version of the questionnaire: Brazilian food safety resolutions based on texts from Codex Alimentarius [44]—RDC 216 and IN 16 [45,46]; *Base Nacional Comum Curricular* (BNCC)- Common Base National Curriculum [47], Protection Motivation Theory [48], Five Keys to Safer Food [49], and instruments developed by Da Cunha, Stedefeldt and De Rosso [50], De Andrade et al. [8], and Haapala and Probart [17].

## *2.2. Focus Groups (FG) Development and Analysis*

*2.2. Focus Groups (FG) Development and Analysis*  Theoretical procedures, the focus of this study aims to obtain validity guaranteed by semantic and construct analyses. Thus, it was necessary to define the instrument's properties, using the focus group methodology [51]. The method offers the possibility of examining students' understanding and perceptions about food safety and verifying the appropriate language to address the target audience. It considers the view of different subjects and social contexts for less accessible information, which in many cases can only be Theoretical procedures, the focus of this study aims to obtain validity guaranteed by semantic and construct analyses. Thus, it was necessary to define the instrument's properties, using the focus group methodology [51]. The method offers the possibility of examining students' understanding and perceptions about food safety and verifying the appropriate language to address the target audience. It considers the view of different subjects and social contexts for less accessible information, which in many cases can only be raised through qualitative techniques [18,51,52].

raised through qualitative techniques [18,51,52]. Structured with an average of nine participants per group, 40 min in length, and heterogeneous concerning sex, the FGs were held in schools in the urban and rural areas of the public-school system in FD, which serves the low-income population [41]. The focal group conduction was carried out by a moderator with two transcribers' participation, following a pre-established script of 21 questions contemplating the Five Keys to Safer Food (1—keep clean; 2—separate raw and cooked; 3—cook thoroughly; 4—keep food at Structured with an average of nine participants per group, 40 min in length, and heterogeneous concerning sex, the FGs were held in schools in the urban and rural areas of the public-school system in FD, which serves the low-income population [41]. The focal group conduction was carried out by a moderator with two transcribers' participation, following a pre-established script of 21 questions contemplating the Five Keys to Safer Food (1—keep clean; 2—separate raw and cooked; 3—cook thoroughly; 4—keep food at safe temperatures; and 5—use safe water and raw materials) [49].

safe temperatures; and 5—use safe water and raw materials) [49]. The focus groups were transcribed and analyzed with the content analysis technique and by the IraMuTeQ Software (Interface de R pour les Analyses Multidimensionnelles de Textes et de Questionnaires) 0.7 alpha [53,54], aiding in the questionnaires development. The software's used followed the instructions in the manual provided by Camargo The focus groups were transcribed and analyzed with the content analysis technique and by the IraMuTeQ Software (Interface de R pour les Analyses Multidimensionnelles de Textes et de Questionnaires) 0.7 alpha [53,54], aiding in the questionnaires development. The software's used followed the instructions in the manual provided by Camargo and Justo [55], and Descending Hierarchical Classification (DHC) analysis was performed.

## *2.3. Questionnaires Development*

In preparing the items, the importance of assessing the knowledge, practices, and risk perception was considered. The transmission of the knowledge with low application in the practical context, associated with the low perception of risk, possibly results in unsafe practices [56]. From this premise, the dimensions assessed were defined.

Determining knowledge and food safety practices among consumers, in general, are of scientific interest, as exemplified by a study by Meysenburg et al. [18]. The study shows that unsafe handling practices are frequent, although they happen through an acceptable level of knowledge [18]. The repetition of these practices points to a significant gap that can be filled by developing educational actions capable of altering the handler's risk perception [8].

Risk perception plays a role in risk management and control in anticipating crises and supporting people in prevention strategies [57]. Another vital phenomenon included and considered in the risk perception was the optimistic bias [58]. The concept of incorporated optimistic bias is represented by the belief that individuals, who in most conditions, visualize their chances of experiencing health and safety problems as inferior to those of their peers (co-workers, friends, and family) [8,59,60].

Therefore, investigating the relationship between practice and knowledge, permeated by risk perception, will contribute to the elucidation of educational strategies aimed at an age group with little knowledge in this field [28,30].

Three possible responses were used to measure the "knowledge" construct, "yes", "no", and "I do not know". One point was assigned for each correct answer, and zero points were assigned to the wrong answer or "I do not know" [16]. Concerning the "practices" construct, a three-point frequency scale was used—"never", "sometimes", and "always".

As for the "risk perception" construct, the first version's Likert scale was based on a study developed by Swaney-Stueve et al. [61]. The study proposes a new model of pictorial scale—the emoji scale. Thus, a five-point scale was associated with emojis, ranging from extremely low to extremely high.

## *2.4. Questionnaires Analysis*

## 2.4.1. Semantic Analysis

The validity procedures started with the semantic analysis of the first version of the instrument, containing three questionnaires, separately addressing knowledge, practices, and risk perception. This stage was developed with the lowest level of skill of the target population: students aged between 11 and 12 years in a FD school. The "brainstorming" technique was used, and the steps described by Pasquali [43] allowing student participation with suggestions for reformulating issues that are not understood. It was performed with this age group to ensure that the items were understandable to the group with the lowest skill, and therefore understandable for the most skilled group.

## 2.4.2. Construct Analysis (Round 1 and 2)

For construct validation, 22 specialists were available to analyze the first version of the instrument. Of these, 16 participated in the analysis of the second version. For the selection of judges, at least one of the following items was used: (i) minimum master's degree or, (ii) research development in the areas of the studied constructs or, (iii) performance in the area of food safety or, (iv) performance in the area of school meals. This validation aimed to verify the adequacy of the latent attribute(s) behavioral representation, providing information on the representativeness and clarity of each item of the questionnaires [43].

Initially, experts received the necessary information and guidance on the instrument's evaluation process, including possible participation in other rounds. All instructions for filling out the Consent Form were provided. After consent, they could access the instrument to facilitate its full view and the link that directed them to the SurveyMonkey® platform.

The items were evaluated for simplicity, clarity, relevance, credibility, variety, and evaluation of the corresponding construct. The knowledge questionnaire was also assessed for

balance and the Likert scales to measure responses adequately. For the correct completion of the evaluation, the constructs' constitutive definition and the items' adequacy criteria were provided. After the judges' participation, their answers and considerations were transcribed to the Microsoft Office Excel® 2007 program.

The Content Validation Index (CVI) was used to evaluate the judges' contributions [62], which allows observing the judges' level of agreement on each item. The item's continuity was limited to a CVI more significant than or equal to 80% (≥0.80) [43]. Three types of CVI were applied: Item-CVI (I-CVI), Questionnaire-CVI (Q-CVI), and Scale-level CVI (S-CVI). Thus, the first refers to the values obtained from each item that makes up the questionnaires according to the criteria of simplicity, clarity, relevance, credibility, and the possibility of evaluating the corresponding construct. The second refers to the values obtained referring to the balance criteria of the knowledge questionnaire and variety criteria of the three questionnaires. Finally, the third refers to the scales used to measure the responses to each questionnaire.

## 2.4.3. Elaboration of the Final Questionnaires

The final instrument was developed based on the answers obtained in the previous step. Additionally, the four researchers' consensus for the judges' positions was also considered, with three dietitians, and one statistician, all postgraduate.

## **3. Results and Discussion**

The construction process resulted in an instrument with three questionnaires that separately contemplate each dimension studied, with 11 items of knowledge, 11 items of practice, and five risk perception items (Appendix A). The steps carried out allowed the development of an instrument that contemplates the main aspects of food safety covered in the Five Keys to Safer Food [49], within a perspective specific to the reality that will be applied. When considering the reality and particular challenges inherent to the studied public, successful communication on food safety is possible within a diagnostic perspective to carry out effective interventions [63]. For Slovic (1987), those who promote and regulate health must understand how people think and respond to risk. If there is no such understanding, the formulated policies may be ineffective.

The biggest challenge during the research was to make the language simple and straightforward. The same difficulty was reported in developing the "EQ-5D-Y" instrument for quality of life-related to children and adolescents' health. The instrument was an adaptation of the "EQ-5D-3L", intended for adults, to suit the new audience. The process resulted in changes, especially in the wording (including illustrations, titles, and response options), to optimize item comprehension and improve data quality [64,65].

In a study to assess the psychometric properties of a Japanese version of the Dutch questionnaire on eating behavior for children, there was a review process by teachers. The idea was to ensure that the final version covered the students' cognitive development and avoided any incomprehensible writing [66].

In this study, specialized judges in the education area were crucial due to the difficulty of adapting the language for children and adolescents. The contributions allowed to create an appropriate wording according to the audience's stage of cognitive development.

The results of the steps required to prepare the questionnaires are detailed below.

## *3.1. Search in Databases*

The studies in the literature allowed us to obtain essential information for the subsequent steps. However, some points stood out for the characterization of the instrument. When writing the items, the students' ability to think logically was considered, according to Piaget's development in which they find themselves—formal operative [20] and Pasquali's [43] criteria for understandable and objective writing.

The three questionnaires that make up the instrument had an item structure similar to that developed by Da Cunha, Stedefeldt and De Rosso [50], and De Andrade et al. [8]. The decision on using the Five Keys to Safer Food [49], based on the possible lack of information on the topic by students, proved to be easy to understand and familiar to students. Additionally, these are more disseminated and more accessible information to teachers and guardians of children. The complementary search at BNCC [47] was important to verify the syllabus required for students aged 11 to 14 years from the Brazilian public education network. From the information obtained, it was possible to establish the expected level of understanding of the topic.

The risk perception questionnaire was based on concepts from the World Health Organization [67], from ISO 31000:2009 [68], and a study developed by Slovic [57]. The inclusion of the optimistic bias [8,16,58,60] brought the possibility of broadening the discussion on children's topics. Some researchers suggest that children are also susceptible to the phenomenon of optimistic bias [60,69], but the scientific literature is still scarce.

The scale for measuring the risk perception questionnaire responses was based on the WHO document [2]. The risk perception questionnaire assessed the relationship between FBD symptoms and their severity and intensity based on the content used.

## *3.2. Conducting and Analyzing Focus Groups (FG)*

The four FG (n = 36) allowed the language structure's adjustment, including in the questionnaires terms and forms of communication mentioned in the discussions. The study design made it possible to include different perceptions and knowledge since social inequalities exist between urban and rural scenarios. The integration of the two realities enabled the understanding of the universe experienced by students in different contexts.

The content analysis of the discussions that emerged in the focus groups identified the language that the public understands, the knowledge of greater and lesser mastery, the habits, and different views related to food safety. This information supported the construction of the items. The units of record (UR) evoked are shown in Table 1, where it was possible to observe the most discussed Keys to Safer Food by analyzing the statements related to each recording unit [49] and the most relevant evocations (Table 2).


**Table 1.** Themes, units of record (UR), and relative and absolute frequencies of the evocations of the focus groups conducted with low-income students (n = 36) ages 11 and 12 (FD—Brazil).

**Table 2.** Keys to Safer Food and selected individual evocations related to the focus groups' food safety conducted with low-income students (n = 36) with school ages of 11 and 12 (FD—Brazil).


The results obtained in the content analysis corroborate the findings from the DHC analysis of the IRaMuTeQ Software. From the DHC, the *corpus*, which is the text composed of the FG's coded transcripts, was categorized into six classes, named according to the central narrative addressed in the discussions (Figure 2).

The UR hand hygiene/bacteria contamination has the highest number of evocations (39%) in content analysis and DHC analysis (46.9%). Students demonstrated knowledge about the importance of hand hygiene and the frequency and way of performing the procedure, understand that the human body is a source of contamination for food, and associate it with bacteria.

central narrative addressed in the discussions (Figure 2).

**Figure 2.** Dendrogram of the descending hierarchical classification with the generated classes, the percentage value about the total of the analyzed corpus, and interclass relations of the focus groups conducted with 11-year-old low-income students (n = 36) (FD—Brazil). **Figure 2.** Dendrogram of the descending hierarchical classification with the generated classes, the percentage value about the total of the analyzed corpus, and interclass relations of the focus groups conducted with 11-year-old low-income students (n = 36) (FD—Brazil).

> The UR hand hygiene/bacteria contamination has the highest number of evocations (39%) in content analysis and DHC analysis (46.9%). Students demonstrated knowledge about the importance of hand hygiene and the frequency and way of performing the procedure, understand that the human body is a source of contamination for food, and associate it with bacteria. In the BNCC [47] for students aged six to 14 years old in Brazil, these subjects are present in life and evolution. At the age of six, body hygiene habits (such as washing hands before eating) and their relationship to health maintenance should be discussed. At In the BNCC [47] for students aged six to 14 years old in Brazil, these subjects are present in life and evolution. At the age of six, body hygiene habits (such as washing hands before eating) and their relationship to health maintenance should be discussed. At the age of nine, the student must have appropriate attitudes to prevent diseases associated with them, based on knowledge of transmitting some microorganisms (viruses, bacteria, and protozoa). Hand washing is a widely discussed topic, and students have more access to it. A study developed by Osei Tutu et al. [19] to assess students' knowledge and food safety practices, aged between seven and 21 years, showed that most students responded correctly about hand washing.

> The results obtained in the content analysis corroborate the findings from the DHC analysis of the IRaMuTeQ Software. From the DHC, the *corpus*, which is the text composed of the FG's coded transcripts, was categorized into six classes, named according to the

"There are things that smell bad, but it is good to eat. Some look bad, but they are good. Some are bad, but they are good." Sanitation of fruits and vegetables "Some foods have to put on bleach, like lettuce and tomatoes. And there are some foods that we have to boil." "You have to soak the fruit, and then put a little bleach in it, then let it soak, to remove most of the microbes." "I do not eat a thing with bleach in my food." "Bleach? That is crazy, dude?"

the age of nine, the student must have appropriate attitudes to prevent diseases associated with them, based on knowledge of transmitting some microorganisms (viruses, bacteria, and protozoa). Hand washing is a widely discussed topic, and students have more access to it. A study developed by Osei Tutu et al. [19] to assess students' knowledge and food Then, the UR safe/unsafe food stood out (40%). Considering that the "three" and "four" classes resulting from the DHC correspond in content to this registration unit, it is noted that they stood out similarly (36.7%). Different perspectives were observed. When asked how to check when the food is fit for consumption, they responded by checking the expiration date on processed foods' packaging and the food's appearance and smell. Research with domestic food handlers has shown that some determine that food is safe because of its appearance or smell, both unreliable indicators for food safety [18].

There were differences between the participants. Many related the concept to nutritional terms for the concept of safe and unsafe food. A study developed by Gavaravarapu et al. [52] obtained similar results. Many teenagers initially confused the concept of safe food with nutritious food, listing many foods such as vegetables, cereals, milk, and meat as safe foods.

However, a large portion related the unsafe food to one that shows signs of deterioration, that is out of date and contains pesticides. The consequences of unsafe food consumption and the association between food consumption with pesticides and death were listed. Finally, in an urban school, the food was considered safe when planted and grown in the domestic environment, as there is no use of pesticides. It is noted that students have the perception that pesticides cause deleterious effects, listing symptoms such as stomach pain, diarrhea, headache, eye pain, and desire to sleep.

The concern shown in the students' speeches regarding the importance of pesticidefree consumption comes in line with the significant and active role that consumers assume in the face of the movement towards achieving sustainable production and consumption. The United Nations' National Goals reinforce this movement for the Sustainable Development Goals (SDGs) [70] and the Food Guide for the Brazilian Population [71]. The Guide shows that an adequate and healthy diet derives from a socially and environmentally sustainable food system, encouraging the domestic cultivation of organic foods and exchanging between neighbors to expand access to a greater diversity of foods. Such practice leads to the production of tasty foods and protects the environment and health.

Consumers have been looking for organic foods, stimulated by health and sustainability benefits [72,73]. Consequently, children and adolescents are influenced by the environments in which they are part of. According to Bronfenbrenner's Ecological System Theory, different levels influence the construction of these individuals' lifestyles, emphasizing the importance that the environment has on them. Thus, family (parents, siblings), school (teachers, colleagues), shared activities, culture, history, customs, laws, and the economic system can influence their education and choices [74–76].

A relevant point was food consumption with signs of deterioration, common among rural school students. Thus, an instrument to investigate this habit is needed. It is noted that individuals who recognize that these foods are considered unsafe still consume such foods. PISA -2015 [41] points out that Brazilian students from rural schools had the lowest ESCS index. This lower socioeconomic level may imply less knowledge about the harms of spoiled food and the low availability of food inputs, leading to unsafe food consumption.

Furthermore, studies have pointed out that food and nutritional insecurity are more prevalent in rural areas, characterized by lower family income and low education, contributing to establishing this situation [67,77–80]. Similar data were found in a study conducted with students in China [9]. It was observed that a large majority knew that expired foods could not be consumed. Only a small number still thought that expired foods could be consumed after boiling or heating or as long as they looked good. In this case, the consumption of unsafe food, expired food, may be related to their belief that appearance is more important than the expiration date information.

The consumption of unsafe foods due to a deterioration process or expired dates causes short-term and long-term consequences. These can be chronic kidney diseases, neurological disorders, reactive arthritis, and irritable bowel syndrome. Foods containing mycotoxin are also worrisome due to their carcinogenic potential [81].

Note that the most explored keys were keeping clean and using safe water and safe raw materials in both analyses. However, topics that make up these keys, such as sanitation of the environment and utensils and drinking water, were not discussed in the focus groups. The keys: separate raw and cooked food, cook the food well, and keep food at safe temperatures obtained less space within the narratives. However, it was possible to observe cross-contamination knowledge involving fungi and food shelf life at different storage temperatures. The most debated topics are related to the direct and more usual handling of food, such as handwashing and checking food before eating. The absence of some topics and the relatively impoverished speeches of others can be attributed to the absence of more direct questions about the focus groups' subjects. Moreover, there are items related to cooking and cleaning responsibilities, which may not be part of their usual routines.

As for the sanitation of fruits and vegetables, the participants recognize the importance of adequate sanitation, but a large portion is unaware of the process that comprises three stages: washing, disinfection or sanitization, and rinsing. The disinfection or sanitization stage was the discussion point that caused the most divergence of opinions from bleach statements. This product has sodium hypochlorite as its active ingredient. This step is recommended by the Brazilian legislation [45,46], and the products must be regularized in the competent agency of the Ministry of Health, the National Health Surveillance Agency (ANVISA). Consumers have access to this information through the Food Guide for the Brazilian population [82].

Food handlers and consumers in a study developed by De Andrade et al. [8] asked whether washing vegetables and soaking them in water with vinegar is enough for this food to be safe for consumption. They obtained 57.8% and 40%, respectively. Given these data, it is inferred that the knowledge of Brazil's appropriate process may be a deficit in different audiences. Gavaravarapu et al. [52] showed that in the developed FGs, there was also recognition of the importance of cleaning fruits and vegetables and reported that raw foods should be washed carefully before consumption. However, there is no report on how hygiene should be carried out.

Finally, the concepts of gravity [67] and vulnerability [48] became evident in the evocations.

"Unsafe food can cause illness, even death, diabetes, infection, a coffin."

"The child gets more ill when he eats something bad; his intestines are more sensitive; the intestines of the adult are firmer."

## *3.3. Questionnaires Development (Questionnaires Analysis)*

From the information obtained with the previous steps, the first version of the instrument was prepared. The questions were distributed among the three dimensions: 12 items of knowledge, 12 practice items, and 10 risk perception items.

Three sessions were held for the semantic analysis stage, with four students each, heterogeneously concerning sex. A construct was worked on in each session. The terms that students understood poorly were replaced by suggestions made by them or by other terms used during the explanation in this step. It was noted that very long questions hinder the correct understanding of the item. Subsequently, it was found that such questions did not meet the criteria of simplicity and clarity [43].

In the construct analysis stage, 22 judges agreed to participate, 59% have a master's degree (n = 13), 32% have a doctorate degree (n = 7), 4.5% post-doctorate degree (n = 1) and 4.5% specialization (n = 1). The judges' group comprised a pedagogue and a psychopedagogue to contribute specifically to the language's adequacy.

In the first round of the construct analysis, 28 items were approved (82.4%); they presented I-CVI values ≥ of 0.80. It is noted that all items have relevance and credibility. The variety criterion was met in the three questionnaires with a mean Q-CVI = 0.94. As for the balance criterion, Q-CVI = 0.91 was obtained. The average of the I-CVI values for each item in each questionnaire is shown in Table 3. It should be noted that all items, regardless of approval, were adjusted for clarity and simplicity.

Regarding the Likert scale evaluation, the scale used in the risk perception questionnaire was approved with S-CVI = 0.82, and the scale of the practice's questionnaire presented S-CVI = 0.77, not approved.

After rewriting, removing items, and replacing the Likert scales used according to the judges' suggestions, the questionnaires was resent for further evaluation. Regarding the scale used in the "risk perception" construct, as emojis are very useful in evoking emotions and attitudes [83], they can confuse the correct measurement of responses since students can associate with feelings and check the emoji that reflects their mood. Thus, it was replaced by another five-point scale, ranging from "no chance" to "100% chance". Another five-point scale replaced the Likert scale of the "practices" construct, ranging from "never" to "always". A study by Dalmoro and Vieira [84] points out that the three-point scale is less reliable and less able to demonstrate the interviewee's opinion.


**Table 3.** Average I-CVI values of the items of the versions submitted to the construct analysis of each questionnaire that make up the instrument for assessing knowledge, practices, and risk perception related to food safety for low-income students (FD-Brazil).

In the second assessment of the construct analysis, 16 judges agreed to participate. All items obtained I-CVI ≥ 0.80. The variety criterion was met in the three questionnaires with a mean Q-CVI = 0.94. As for the balance criterion, Q-CVI = 0.94 was obtained. The average I-CVI values for each item in each questionnaire are shown in Table 3. The two Likert scales obtained S-CVI = 0.94.

After completing the specialists' second evaluation, the researchers responsible for the study discussed the items resulting in the third version of the questionnaires. They analyzed the need for modifications to obtain the final version. They included items deemed pertinent and restructuring sentences to better understand and modify the Likert scale of the risk perception questionnaire.

As seen in the semantic analysis stage and according to some judges' positions, students could have difficulties understanding the concept of risk. Thus, it was decided to break down the risk perception items in their two dimensions—gravity and probability [67]. The term "chance" was used for the probability dimension because they are synonymous in the semantic context, and the epidemiological definition is not considered [85].

As the two risk dimensions were worked on separately in the risk perception questionnaire, there was a need to develop two scales to measure the responses (Appendix A). The probability dimension was measured using a five-point Likert scale associated with the ruler. The severity dimension was measured using a four-point scale associated with the figures that characterize symptoms at each severity level. Items 1.1 and 2.1 were coded from "one" to "four", with "one" being "no severity" and "four" being "high severity". The optimistic bias will be obtained by subtracting the score from item "two" (2 and 2.1) by the score from item "one" (1 and 1.1). Positive and significant differences indicate the trend of optimistic bias [14]. E.g., if in questions (1 and 1.1), the respondent obtained a score of 5 and 4 respectively, the final score was 1. In questions (2 and 2.1), he obtained scores of 4 and 2, with a final score 2. When checking the optimistic bias, it turns out that 2-1 presents a positive difference, being possible to identify the phenomenon.

The entire process of restructuring the questionnaires is shown in Figure 3.

**Figure 3.** Restructuring steps for the knowledge, practices, and risk perception assessment instrument related to food safety for low-income students (FD—Brazil). **Figure 3.** Restructuring steps for the knowledge, practices, and risk perception assessment instrument related to food safety for low-income students (FD—Brazil).

## **4. Conclusions 4. Conclusions**

The food safety scenario revealed in the discussions, and systematic database search made it possible to understand the addressed dimensions and constructs. Those were knowledge, practices, and risk perceptions, providing an instrument that considered the public's reality and perspective, especially the issue language, to be simple, straightforward, playful, and understandable. The achievement of the objective was possible through a rigorous validation process. This process resulted in a reliable and appropriate instrument consisting of three questionnaires that helped determine a starting point for the formulation of public policies aimed at the universe of low-income children and adolescents. This study's limits are represented by the fact that it does not include verifying the reliability coefficients (Conbrach's alpha and temporal stability) and validity (regression analysis and internal consistency analysis) of the items that make up the instrument. However, this is planned for future studies. The food safety scenario revealed in the discussions, and systematic database search made it possible to understand the addressed dimensions and constructs. Those were knowledge, practices, and risk perceptions, providing an instrument that considered the public's reality and perspective, especially the issue language, to be simple, straightforward, playful, and understandable. The achievement of the objective was possible through a rigorous validation process. This process resulted in a reliable and appropriate instrument consisting of three questionnaires that helped determine a starting point for the formulation of public policies aimed at the universe of low-income children and adolescents. This study's limits are represented by the fact that it does not include verifying the reliability coefficients (Conbrach's alpha and temporal stability) and validity (regression analysis and internal consistency analysis) of the items that make up the instrument. However, this is planned for future studies.

the ruler. The severity dimension was measured using a four-point scale associated with the figures that characterize symptoms at each severity level. Items 1.1 and 2.1 were coded from "one" to "four", with "one" being "no severity" and "four" being "high severity". The optimistic bias will be obtained by subtracting the score from item "two" (2 and 2.1) by the score from item "one" (1 and 1.1). Positive and significant differences indicate the trend of optimistic bias [14]. E.g., if in questions (1 and 1.1), the respondent obtained a score of 5 and 4 respectively, the final score was 1. In questions (2 and 2.1), he obtained scores of 4 and 2, with a final score 2. When checking the optimistic bias, it turns out that

2-1 presents a positive difference, being possible to identify the phenomenon.

The entire process of restructuring the questionnaires is shown in Figure 3.

**Author Contributions:** Conceptualization, methodology, validation, formal analysis, investigation, data curation, writing—original draft preparation, and writing—review and editing—S.A.B.; methodology, validation, formal analysis, writing—review and editing, and supervision E.S.; validation, formal analysis—E.Y.N.; writing—original draft preparation, and writing—review and editing— M.d.O.C.; writing—original draft preparation, and writing—review and editing—R.B.A.B.; writ-**Author Contributions:** Conceptualization, methodology, validation, formal analysis, investigation, data curation, writing—original draft preparation, and writing—review and editing—S.A.B.; methodology, validation, formal analysis, writing—review and editing, and supervision E.S.; validation, formal analysis—E.Y.N.; writing—original draft preparation, and writing—review and editing— M.d.O.C.; writing—original draft preparation, and writing—review and editing—R.B.A.B.; writing review and editing, and funding acquisition—R.P.Z.; writing—review and editing, and funding acquisition—A.R.; writing—review and editing, and funding acquisition—H.H.; Conceptualization, methodology, validation, formal analysis, writing—original draft preparation, writing—review and editing, supervision, project administration, and funding acquisition—V.C.G. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research was funded by FAPDF (nº 166/2020).

**Institutional Review Board Statement:** The study was conducted according to the guidelines of the Declaration of Helsinki, and approved by Ethics Committee of the College of Health Sciences of the University of Brasilia—CEP/FS UnB (CAAE nº 02033218.0.0000.0030). have read and agreed to the published version of the manuscript. **Funding:** This research was funded by FAPDF (nº 166/2020). **Institutional Review Board Statement:** The study was conducted according to the guidelines of the **Funding:** This research was funded by FAPDF (nº 166/2020). **Institutional Review Board Statement:** The study was conducted according to the guidelines of the view and editing, supervision, project administration, and funding acquisition—V.C.G. All authors have read and agreed to the published version of the manuscript. **Funding:** This research was funded by FAPDF (nº 166/2020). have read and agreed to the published version of the manuscript. **Funding:** This research was funded by FAPDF (nº 166/2020).

ing—review and editing, and funding acquisition—R.P.Z.; writing—review and editing, and funding acquisition—A.R.; writing—review and editing, and funding acquisition—H.H.; Conceptualization, methodology, validation, formal analysis, writing—original draft preparation, writing—review and editing, supervision, project administration, and funding acquisition—V.C.G. All authors

ing—review and editing, and funding acquisition—R.P.Z.; writing—review and editing, and funding acquisition—A.R.; writing—review and editing, and funding acquisition—H.H.; Conceptualization, methodology, validation, formal analysis, writing—original draft preparation, writing—re-

ing—review and editing, and funding acquisition—R.P.Z.; writing—review and editing, and funding acquisition—A.R.; writing—review and editing, and funding acquisition—H.H.; Conceptualization, methodology, validation, formal analysis, writing—original draft preparation, writing—review and editing, supervision, project administration, and funding acquisition—V.C.G. All authors

ing—review and editing, and funding acquisition—R.P.Z.; writing—review and editing, and funding acquisition—A.R.; writing—review and editing, and funding acquisition—H.H.; Conceptualization, methodology, validation, formal analysis, writing—original draft preparation, writing—review and editing, supervision, project administration, and funding acquisition—V.C.G. All authors

**Informed Consent Statement:** Informed consent was obtained from the parents of all subjects involved in the study. Declaration of Helsinki, and approved by Ethics Committee of the College of Health Sciences of the University of Brasilia—CEP/FS UnB (CAAE nº 02033218.0.0000.0030). Declaration of Helsinki, and approved by Ethics Committee of the College of Health Sciences of the University of Brasilia—CEP/FS UnB (CAAE nº 02033218.0.0000.0030). **Institutional Review Board Statement:** The study was conducted according to the guidelines of the Declaration of Helsinki, and approved by Ethics Committee of the College of Health Sciences of the **Institutional Review Board Statement:** The study was conducted according to the guidelines the Declaration of Helsinki, and approved by Ethics Committee of the College of Health Sciences of the University of Brasilia—CEP/FS UnB (CAAE nº 02033218.0.0000.0030).

**Data Availability Statement:** The study did not report any data. **Informed Consent Statement:** Informed consent was obtained from the parents of all subjects involved in the study. **Informed Consent Statement:** Informed consent was obtained from the parents of all subjects involved in the study. University of Brasilia—CEP/FS UnB (CAAE nº 02033218.0.0000.0030). **Informed Consent Statement:** Informed consent was obtained from the parents of all subjects in-

*Sustainability* **2021**, *13*, x FOR PEER REVIEW 14 of 21

*Sustainability* **2021**, *13*, x FOR PEER REVIEW 14 of 21

have read and agreed to the published version of the manuscript.

*Sustainability* **2021**, *13*, x FOR PEER REVIEW 14 of 21

*Sustainability* **2021**, *13*, x FOR PEER REVIEW 14 of 21

**Acknowledgments:** This research was partially supported by Coordination for the Improvement of Higher Education Personnel (CAPES)—scholarship. **Data Availability Statement:** The study did not report any data. **Data Availability Statement:** The study did not report any data. **Informed Consent Statement:** Informed consent was obtained from the parents of all subjects involved in the study. **Data Availability Statement:** The study did not report any data. volved in the study. **Data Availability Statement:** The study did not report any data.

**Conflicts of Interest:** The authors declare no conflict of interest. **Acknowledgments:** This research was partially supported by Coordination for the Improvement of Higher Education Personnel (CAPES)—scholarship. **Acknowledgments:** This research was partially supported by Coordination for the Improvement of Higher Education Personnel (CAPES)—scholarship. **Acknowledgments:** This research was partially supported by Coordination for the Improvement of **Acknowledgments:** This research was partially supported by Coordination for the Improvement of

### **Appendix A. Instrument Addressed for Low-Income Children and Adolescents on Food Safety Knowledge, Practices, and Risk Perception Conflicts of Interest:** The authors declare no conflict of interest. **Conflicts of Interest:** The authors declare no conflict of interest. **Appendix A. Instrument Addressed for Low-Income Children and Adolescents on**  Higher Education Personnel (CAPES)—scholarship. **Conflicts of Interest:** The authors declare no conflict of interest. Higher Education Personnel (CAPES)—scholarship. **Conflicts of Interest:** The authors declare no conflict of interest.


Gender: ( ) Female ( ) Male Date: \_\_\_\_ / \_\_\_\_ / 202\_\_\_

Gender: ( ) Female ( ) Male Date: \_\_\_\_ / \_\_\_\_ / 202\_\_\_

### **Make an X ONLY A RECTANGLE that matches your answer. Make an X ONLY A RECTANGLE that matches your answer.**  Gender: ( ) Female ( ) Male Date: \_\_\_\_ / \_\_\_\_ / 202\_\_\_ Gender: ( ) Female ( ) Male Date: \_\_\_\_ / \_\_\_\_ / 202\_\_\_

**ATTENTION:** GETTING SICK = having stomach pain and/or diarrhea and/or vomiting and/or headache and/or fever because you ate contaminated food. **Make an X ONLY A RECTANGLE that matches your answer. ATTENTION:** GETTING SICK = having stomach pain and/or diarrhea and/or vomiting and/or headache **ATTENTION:** GETTING SICK = having stomach pain and/or diarrhea and/or vomiting and/or headache **Make an X ONLY A RECTANGLE that matches your answer. Make an X ONLY A RECTANGLE that matches your answer.** 

Series \_\_\_\_\_\_\_\_\_\_ Age: \_\_\_\_\_\_\_\_ years old Area where you live: ( ) Urban ( ) Rural

Series \_\_\_\_\_\_\_\_\_\_ Age: \_\_\_\_\_\_\_\_ years old Area where you live: ( ) Urban ( ) Rural


2. What is the chance that a COLLEAGUE who studies with you will get sick from having eaten the same food served at your school? studies with you will get sick from having eaten the same food served at your school? 2. What is the chance that a COLLEAGUE who studies with you will get sick from having eaten the same food served at your school? 2. What is the chance that a COLLEAGUE who studies with you will get sick from having eaten the same food served at your school? 2. What is the chance that a COLLEAGUE who studies with you will get sick from having eaten the same food served at your school?

2. What is the chance that a COLLEAGUE who

2.1 If YOUR COLLEAGUE gets sick from eating food served at the school you study, how serious could it be? food served at the school you study, how serious could it be? 2.1 If YOUR COLLEAGUE gets sick from eating food served at the school you study, how serious could it be? 2.1 If YOUR COLLEAGUE gets sick from eating food served at the school you study, how serious could it be? 2.1 If YOUR COLLEAGUE gets sick from eating food served at the school you study, how serious could it be?

2.1 If YOUR COLLEAGUE gets sick from eating

3. What is the chance that a person will die from eating contaminated food?

3. What is the chance that a person will die from eating contaminated food?

3. What is the chance that a person will die from eating contaminated food?

*Sustainability* **2021**, *13*, x FOR PEER REVIEW 15 of 21

**Make an X ONLY A RECTANGLE that matches your answer.**  Never = never did this action Rarely = does this action from time to time Sometimes = do this action sometimes Often = do this action many times Always = this action is part of your daily life

1) Do you wash your hands with soap and water/soap/detergent before eating?

(11) Does unsafe food to eat always smell foul, look strange, and have a different texture?

(11) Does unsafe food to eat always smell foul, look strange, and have a different texture?

Unsafe food: food that can make you sick.

Code \_\_\_\_\_\_\_\_ (filled in by the researcher)

(5) Eating food that was out of the fridge for a long time after it was done can make you sick?

(5) Eating food that was out of the fridge for a

Ex: Food prepared for lunch that stayed until dinner time on the stove.

(1) Do you always need to use soap/soap/detergent to wash your hands correctly?

(6) Eating foods with a bad smell, bad taste, different texture than usual or moldy, can make you sick?

(7) Removing the moldy part of bread before eating reduces or eliminates the chance of you becoming ill?

(8) Eating food made in a kitchen that contains flies and other insects can make you sick?

(9) To eat raw fruits and vegetables, do you need to wash them using bleach?

(2) Is using a paper towel to clean a dirty board of raw meat enough to be able to use this board to cut bread?

Enough: when you don't need to do anything else.

(3) Should raw meats be kept in the refrigerator on shelves below ready-to-eat foods?

Examples of ready-to-eat foods: cake, cooked rice, baked beans.

9) Do you help in preparing food or food at home?

7) Do you eat bread after removing a moldy part?

9) Do you help in preparing food or food at home?

7) Do you eat bread after removing a moldy part?

9) Do you help in preparing food or food at home?

7) Do you eat bread after removing a moldy part?

9) Do you help in preparing food or food at home?

9) Do you help in preparing food or food at home?

8) Do you eat fruits without washing them?

8) Do you eat fruits without washing them?

8) Do you eat fruits without washing them?

8) Do you eat fruits without washing them?

prepare meals or food?

prepare meals or food?

prepare meals or food?

prepare meals or food?

prepare meals or food?

**Examples of fruit: apple, grape, orange.** 

5) Do you eat expired foods that have a good smell, normal appearance, and texture?

3) Do you store food in the refrigerator in closed packages or containers with a lid?

2) When you open a milk carton, do you leave it out of the fridge for more than an hour?

5) Do you eat expired foods that have a good smell, normal appearance, and texture?

5) Do you eat expired foods that have a good smell, normal appearance, and texture?

4) Before eating the food, do you look at the expiration date on the packaging?

4) Before eating the food, do you look at the expiration date on the packaging?

**IF YOU SCORE IN THE PREVIOUS QUESTION, THE OPTIONS RARELY, SOMETIMES, OF-TEN OR ALWAYS OR ALWAYS, ANSWER THE NEXT QUESTIONS:**  10) Do you wash your hands with soap and water/soap/detergent before preparing or helping to

**Examples of fruit: apple, grape, orange.** 

**IF YOU SCORE IN THE PREVIOUS QUESTION, THE OPTIONS RARELY, SOMETIMES, OF-TEN OR ALWAYS OR ALWAYS, ANSWER THE NEXT QUESTIONS:**  10) Do you wash your hands with soap and water/soap/detergent before preparing or helping to

**Examples of fruit: apple, grape, orange.** 

**IF YOU SCORE IN THE PREVIOUS QUESTION, THE OPTIONS RARELY, SOMETIMES, OF-TEN OR ALWAYS OR ALWAYS, ANSWER THE NEXT QUESTIONS:**  10) Do you wash your hands with soap and water/soap/detergent before preparing or helping to

**Examples of fruit: apple, grape, orange.** 

**IF YOU SCORE IN THE PREVIOUS QUESTION, THE OPTIONS RARELY, SOMETIMES, OF-TEN OR ALWAYS OR ALWAYS, ANSWER THE NEXT QUESTIONS:**  10) Do you wash your hands with soap and water/soap/detergent before preparing or helping to

**IF YOU SCORE IN THE PREVIOUS QUESTION, THE OPTIONS RARELY, SOMETIMES, OF-TEN OR ALWAYS OR ALWAYS, ANSWER THE NEXT QUESTIONS:**  10) Do you wash your hands with soap and water/soap/detergent before preparing or helping to

**Examples of fruit: apple, grape, orange.** 

7) Do you eat bread after removing a moldy part?

6) Do you eat raw or soft yolk eggs?

6) Do you eat raw or soft yolk eggs?

6) Do you eat raw or soft yolk eggs?

6) Do you eat raw or soft yolk eggs?

(6) Do you eat raw or soft yolk eggs? Always = this action is part of your daily life **Make an X ONLY A RECTANGLE that matches your answer.** 

*Sustainability* **2021**, *13*, x FOR PEER REVIEW 17 of 21

**Make an X ONLY A RECTANGLE that matches your answer.**  Never = never did this action Rarely = does this action from time to time Sometimes = do this action sometimes Often = do this action many times

10) Do you wash your hands with soap and water/soap/detergent before preparing or helping to

**Examples of fruit: apple, grape, orange.** 

3) Do you store food in the refrigerator in closed packages or containers with a lid?

\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_

4) Before eating the food, do you look at the expiration date on the packaging?

10) Do you wash your hands with soap and water/soap/detergent before preparing or helping to

10) Do you wash your hands with soap and water/soap/detergent before preparing or helping to

**IF YOU SCORE IN THE PREVIOUS QUESTION, THE OPTIONS RARELY, SOMETIMES, OF-TEN OR ALWAYS OR ALWAYS, ANSWER THE NEXT QUESTIONS:**  10) Do you wash your hands with soap and water/soap/detergent before preparing or helping to

**IF YOU SCORE IN THE PREVIOUS QUESTION, THE OPTIONS RARELY, SOMETIMES, OF-TEN OR ALWAYS OR ALWAYS, ANSWER THE NEXT QUESTIONS:**  10) Do you wash your hands with soap and water/soap/detergent before preparing or helping to

**Examples of fruit: apple, grape, orange.** 

**Examples of fruit: apple, grape, orange.** 

## **References References**

2008.

**Examples of fruit: apple, grape, orange.**  6) Do you eat raw or soft yolk eggs? 1. Riesute, R.; Salomskiene, J.; Moreno, D.S.; Gustiene, S. Effect of yeasts on food quality and safety and possibilities of their inhibition. *Trends Food Sci. Technol.* **2021**, *108*, 1–10, doi:10.1016/j.tifs.2020.11.022. 2. World Health Organization *WHO Estimates of the Global Burden of Foodborne Diseases*, 1st ed.; WHO Library Cataloguing-in-1. Riesute, R.; Salomskiene, J.; Moreno, D.S.; Gustiene, S. Effect of yeasts on food quality and safety and possibilities of their inhibition. *Trends Food Sci. Technol.* **2021**, *108*, 1–10. [CrossRef]

7) Do you eat bread after removing a moldy part?

prepare meals or food?

8) Do you eat fruits without washing them?

prepare meals or food?

prepare meals or food?

*2017*; Wiley-Blackwell Publishing Ltd.: Hoboken, NJ, USA, 2018; Volume 16.

China. *Food Control* **2019**, *95*, 181–188, doi:10.1016/j.foodcont.2018.07.042.

Sweden. *Food Control* **2020**, *113*, 107130, doi:10.1016/j.foodcont.2020.107130.

*Food Control* **2018**, *84*, 345–353, doi:10.1016/j.foodcont.2017.08.011.

7. WHO. *A Guide to Healthy Food Markets*; WHO: Geneva, Switzerland, 2006; ISBN 0201398257.

**2019**, *11*, 40, doi:10.3390/nu11010040.

doi:10.1016/J.FOODRES.2019.01.006.

prepare meals or food?

in Beijing, China: A cross-sectional study. *PLoS ONE* **2017**, *12*, e0187208, doi:10.1371/journal.pone.0187208.

the different degrees of risk perception. *Food Res. Int.* **2019**, *116*, 126–134, doi:10.1016/j. foodres.2018.12.058.

prepare meals or food?

9) Do you help in preparing food or food at home?

8) Do you eat fruits without washing them?


9) Do you help in preparing food or food at home?

9) Do you help in preparing food or food at home?

8) Do you eat fruits without washing them?

9. Cheng, Y.; Zhang, Y.; Ma, J.; Zhan, S. Food safety knowledge, attitude and self-reported practice of secondary school students

10. de Freitas, R.S.G.; da Cunha, D.T.; Stedefeldt, E. Food safety knowledge as gateway to cognitive illusions of food handlers and

11. Luo, X.; Xu, X.; Chen, H.; Bai, R.; Zhang, Y.Y.; Hou, X.; Zhang, F.; Zhang, Y.Y.; Sharma, M.; Zeng, H.; et al. Food safety related knowledge, attitudes, and practices (KAP) among the students from nursing, education and medical college in Chongqing,

12. Marklinder, I.; Ahlgren, R.; Blücher, A.; Ehn Börjesson, S.M.; Hellkvist, F.; Moazzami, M.; Schelin, J.; Zetterström, E.; Eskhult, G.; Danielsson-Tham, M.L. Food safety knowledge, sources thereof and self-reported behaviour among university students in

13. Ovca, A.; Jevšnik, M.; Kavčič, M.; Raspor, P. Food safety knowledge and attitudes among future professional food handlers.

8. de Andrade, M.L.; Rodrigues, R.R.; Antongiovanni, N.; da Cunha, D.T. Knowledge and risk perceptions of foodborne disease by consumers and food handlers at restaurants with different food safety profiles. *Food Res. Int.* **2019**, *121*, 845–853,

8) Do you eat fruits without washing them?

6. Dubugras, M.T.B.; Pérez-Gutiérrez, E. *Perspectiva Sobre a Análise de Risco na Segurança dos Alimentos*; PAHO: Rio de Janeiro, Brazil,

5. Draeger, C.; Akutsu, R.; Zandonadi, R.; da Silva, I.; Botelho, R.; Araújo, W.Brazilian Foodborne Disease National Survey:


## *Article* **The Impact of Climate Change on the Food (In)security of the Siberian Indigenous Peoples in the Arctic: Environmental and Health Risks**

**Elena Bogdanova 1,2,\* , Sergei Andronov 2,3 , Andrei Soromotin <sup>4</sup> , Gennady Detter <sup>5</sup> , Oleg Sizov <sup>6</sup> , Kamrul Hossain <sup>7</sup> , Dele Raheem <sup>7</sup> and Andrey Lobanov 2,3,\***


**Abstract:** Climate change represents a global challenge that impacts the environment, traditional lifestyle and health of the Indigenous Peoples in the Arctic zone of Western Siberia and threatens their food security. Reindeer are an important food source for this population since reindeer herding products are used as traditional nutrition and effective preventive means and remedies for adapting to the cold and geomagnetic activity in the High North. Longer off-season periods, high summer and winter temperatures, melting ice, and forest and tundra fires have a significant impact on the trampling and degradation of reindeer pastures. These effects may lead to massive reindeer losses and changes in the traditional diet of the Indigenous Peoples in the Arctic, which result in increases in the prevalence of respiratory diseases, overweight and hypertension. This study applied a multidisciplinary approach based on ecological and medical research methods with the inclusion of socioeconomic analysis. The primary sources included data on the longitudinal dynamics of air temperature as a climate change indicator and reindeer livestock populations (1936–2018), consumption of reindeer products and physiological impacts on the Yamal Indigenous population collected during expeditions to the Arctic zone of Western Siberia in 2012–2018.

**Keywords:** climate change; food security; reindeer herding; reindeer losses; overgrazing; traditional nutrition; health risks; Arctic Indigenous Peoples; Yamal-Nenets Autonomous Okrug

## **1. Introduction**

Food security, health and well-being of the Arctic Indigenous communities depend on reindeer herding. There have been rapid social, economic, and political changes in the Russian Arctic during the last half of the twentieth century. It has impacted Indigenous traditional livelihoods, community social networks, and their lifestyle. Furthermore, the Indigenous Peoples are increasingly exposed to risks associated with climate change [1–3] which is presenting challenges for reindeer herders in the Arctic. In recent decades, Arctic and subarctic regions have experienced significant warming trends [4–7], which can have devastating environmental impacts [8]. The dynamics of the increase in the average annual

**Citation:** Bogdanova, E.; Andronov, S.; Soromotin, A.; Detter, G.; Sizov, O.; Hossain, K.; Raheem, D.; Lobanov, A. The Impact of Climate Change on the Food (In)security of the Siberian Indigenous Peoples in the Arctic: Environmental and Health Risks. *Sustainability* **2021**, *13*, 2561. https://doi.org/10.3390/su13052561

Academic Editors: António Raposo and Heesup Han

Received: 19 January 2021 Accepted: 22 February 2021 Published: 27 February 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

temperature in the Arctic has, on average, an effect that is two times greater than that in the temperate latitudes [9]. By the end of the twenty-first century, the global mean temperature is expected to rise 1–5 ◦C [9]. This increase in temperature has already affected reindeer herding activities [10–13], traditional lifestyle, well-being and health of the Indigenous Peoples in Scandinavia and the Russian Arctic [1,2,14]. Adaptive capacity [15,16] and coping strategies [17] for changes in the terrestrial ecosystem caused by climate change are urgently required [14,18,19].

A growing number of studies are providing insights into how climate change is being experienced in the Arctic, by northerners, beyond the variables included in climate change models, and how Arctic people are being affected [18,20,21]. Human dimensions of climate change have been enriched with the studies of vulnerability which are increasingly considering the multiple variables that drive exposure sensitivities, and research on the interconnections between climate and non-climatic factors (industrial development of the Arctic, changing socio-ecological and economic policies, etc.) that have acted synergistically to affect individuals and communities [18,22–24]. The effect of the political context could be so large as to conceal the impact of other natural factors on reindeer populations such as climate change. However, a gradual increase of reindeer populations in the northern part of Russia in the 1960s can be associated with changes in atmospheric circulation patterns [25]. Therefore, in our research, we are mostly focused on the impact of climate change on reindeer herding practices. However, the importance of other non-climatic factors as socio-economic drivers has also been considered.

Climate change is expected to have both positive and negative impacts on reindeer herding [3,17,26,27]. Regarding the positive effects, climate change has increased the duration of the snowless season and temperatures during the growing season, which have positively impacted the productivity of plants and the growth of the reindeer's forage base [27,28]. In spring, early snowmelt and increased availability of fresh forage are favourable for lactating reindeer and the new-born calves [29–32]. These climatic changes have impacted the nomadic routes of reindeer herds and, ultimately, the traditional lifestyles of the local communities because of the decreased availability of their traditional food [33] for most of the year.

Regarding the negative impacts, warming can result in the potential re-emergence of anthrax associated with historic livestock burial sites [33–36], biological pollution, more severe insect harassment, epizootics and more frequent parasite epidemics [3,37–42]. It increases the risk of forest and tundra fires [43] (dry lichen represents an extreme fire hazard). The expansion of the taiga tick has caused the border of tick-borne encephalitis and borreliosis to move north. Longer warming periods in shallow water bodies and the thawing of cesspools previously located in the permafrost can lead to outbreaks of intestinal infections [36,44].

Warmer winters with "varying temperatures and events like rain-on-snow or thawfreeze may lead to more frequent icing of snow and basal ice" [45], which make it difficult for reindeer to access ground lichens [3,46,47]. Warm autumns can also result in the growth of mycotoxin-producing microfungi (moulds) below the snow in reindeer pastures [3,47,48]. The lengthening of off-season periods and high summer temperatures contribute to the trampling (lichen becomes brittle when it dries) and degradation of reindeer pastures [49]. This damage results in less food being available to reindeer in winter and negatively affects the adaptive abilities of their bodies and their resistance to infections (brucellosis, mycobacteriosis, and anthrax), leading to increased losses during extreme weather and decreased offspring survival rates [50]. The trampled lichen takes a long time to recover and negatively impacts reindeer husbandry for several years. The increase in the occurrence of anomalous phenomena (winter thaws and rains, late spring with ice crust, high and prolonged heat) represents a significant threat, as it can result in massive reindeer losses [51,52] because of the decreased availability of pastures and ultimately placing an effect on the subsistence-based livelihoods of the Indigenous populations [53] jeopardizing the food security of the Indigenous Peoples.

Although climate change is projected to increase fish stocks in the Barents Sea capable of offering the availability of foods [54], the decrease of reindeer livestock is more likely expected. However, reindeer meat is a rich source of protein, minerals and essential fatty acids, and its consumption is culturally acceptable in these Arctic communities [55]. The loss of this food source increases the prevalence of chronic bronchitis [56,57], overweight [58] and arterial hypertension [59,60] because consuming reindeer meat, blood and liver helps to prevent these diseases [61–64] and increases the ability of the Indigenous population to adapt to the harsh Arctic living conditions [65–68]. The maintenance of a traditional diet is closely related to the maintenance of the traditional food system [69,70], which is "tightly interwoven with the culturally, socially and environmentally embedded practices of reindeer herding. Indigenous Peoples and their traditional food systems emerged in harmony with nature and contain knowhow on the sustainable use of natural resources in ways that contribute to their health" [71].

The environmental changes in the Russian North [72], and specifically in the Yamal-Nenets Autonomous Okrug (YNAO), have resulted in the melting of ice and the extension of the sea [73,74]. The ice crust formation in the winter of 2013–2014 and the outbreak of anthrax in the summer of 2016 have provoked a discussion on Nenets' reindeer husbandry "crisis", as evidenced by the overgrowth of herds and overgrazing [75,76]. Biologists and local governments emphasized the need to significantly reduce the Yamal reindeer population "for the sake of environmental safety". However, A. Golovnev presented an alternative approach focused on a system of movement: "skilful herd-navigation and quick manoeuvring is the basis of Nenets' traditional rule "*ya puna hayoda*" (land after us remains). Conversely, the consequences of sluggish and stationary behaviour, including huddling around camp for a long time, is reflected in another Nenets proverb: "*yadata habei*" (land is turned upside down)" [77]. These are examples of traditional coping strategies for dealing with the development of adverse weather and pasture conditions that have accumulated and been exchanged within the herding communities in different Arctic countries [78]. However, the rapidly changing Arctic environment and climatic challenges have made these coping strategies ineffective, requiring the development of new methods [46,79].

Climate change and the extensive exploitation of reindeer pastures have resulted in overgrazing [80]. This has caused the Indigenous Peoples to change their nomadic routes (i.e., some reindeer herders in the Priuralsky district of the YNAO have moved to the Laborovskay tundra, which is still rich with lichen). Nomadic reindeer herders of the Tazovskaya, Tanamskaya, Gydanskaya and Yuribeyskaya tundras have experienced the reduction of winter reindeer pastures: there are significantly damaged pastures next to the slaughterhouse in the Antipayutinskaya tundra, near the trading posts ("faktoria") Yuribey and Tanama, surrounding the settlements of Antipayuta and Gyda and near fishing sites on the coast of Ob Bay (north of the settlement of Antipayuta, on the western shore of Yambuto lake, on the coast of Gydan Bay, in the area of oil and gas deposits of the Messoyakhinskaya group) [27]. Thus, climate change has had a strong influence on the Arctic ecosystems and jeopardized the Indigenous Peoples' food security and their well-being [81,82]. In this study, we focused on analysing the longitudinal trends in the climatic parameters, reindeer livestock population and consumption of traditional reindeer products and the physiological impacts on the Yamal Indigenous population. These analyses can provide relevant information about the sustainability of reindeer herding and the Indigenous communities in the Arctic zone of Western Siberia.

## **2. Materials and Methods**

## *2.1. Setting: The Yamal-Nenets Autonomous Okrug (YNAO): Geography, Population and Ethnic Structure*

The YNAO, the geographic focus of our research, is an important region for the Indigenous Peoples of Russia, and it is located in the circumpolar northwest of West Siberia. It has a population of 544,008 [83] who live in an area of 769,250 square kilometres [84] with a population density of 0.71 people per square kilometre. The location of the YNAO (more than half of its territory is above the Arctic Circle) significantly influences the traditional

livelihoods in this region. It is a unique territory because almost half of the minority Indigenous population of the Russian Arctic (about 45,000 people) reside there, including the Nenets, Khanty, Selkups and Komi-Zyryans. A total of 14,600 Indigenous Peoples are nomadic, living in tundra areas [85]. The culture, health and social well-being of Indigenous Peoples are strongly linked to their traditional lifestyle and traditional livelihoods (reindeer herding, fishing, etc.), which are essential for meeting Indigenous Peoples' vital needs and helping them to survive in the severe Arctic areas.

## *2.2. Study Design*

In this paper, we present the results of a quantitative analysis of the impacts of climate change on the reindeer livestock population and consumption of traditional reindeer products and of a follow-up analysis of the increased risks of physiological health effects faced by the Siberian Indigenous Peoples in reindeer herding communities living and practicing nomadism in the remote territories of the YNAO. The objectives of our study were to determine (1) if the longitudinal dynamics of air temperature has impacted reindeer livestock populations in the YNAO in 1936–2018 and (2) if the dynamics of the reindeer livestock population has affected the consumption of reindeer products and the health of Indigenous Peoples in the Yamal Indigenous population in 2012–2018. Non-climatic factors could be also considered to explain the results.

## *2.3. Measurement Tools, Methodology and Study Population*

The study applied a multidisciplinary approach based on ecological and medical research methods with the inclusion of policy and socioeconomic analysis. The primary sources included data on the longitudinal dynamics of air temperature as a climate change indicator, reindeer livestock population, consumption of reindeer products and physiological impacts (prevalence of arterial hypertension) on the Yamal Indigenous population.

The data on the average daily, average monthly and average annual air temperatures for the period from 1936 to 2018 were obtained from the open-source platforms of the Federal Service for Hydrometeorology and Environmental Monitoring in the Russian Federation (Roshydromet) [86]. We used data collected from three meteorological stations in the following locations in the YNAO: in the city of Nadym in the Nadymsky district (65.53333: 72.51667), in the settlement of Novy Port in the Yamalsky district (67.4100: 72.5600) and the settlement of Antipayuta in the Tazovsky district (69.06667: 76.83333). The data on the number of reindeer in the YNAO for the period from 1930 to 2018 were obtained from the Department of Agroindustrial Complex of YNAO [87].

First, the correlation between air temperature and the reindeer population was studied. Spearman's rank correlation coefficient was used to determine the strength of the link between the sets of data. Due to the seasonality observed in the time series, seasonal differencing was applied to eliminate the seasonal component with the use of an autoregressive integrated moving average (ARIMA) model [88,89]. Short-term forecasting (5 years) was carried out using the Almon distributed lag method. Second, a forecasting model of the dynamics of the reindeer population was developed. Third, the correlation between the reindeer livestock population and consumption of reindeer products (reindeer meat, liver and blood) with a follow-up correlation between the consumption of these products and the prevalence of arterial hypertension (as the main reason for mortality [59]) in the Yamal Indigenous population was studied.

The data on socio-demographic characteristics, consumption of reindeer products and health status were collected during expeditions to the Arctic zone of Western Siberia in 2012–2018. The fieldwork was conducted by researchers of the YNAO Arctic Scientific Research Centre, the National Medical Research Centre for Rehabilitation and Balneology, the Northern Arctic Federal University and the Association of Reindeer Herders in YNAO (two of the researchers were Indigenous).

To study the basic patterns of consumption of traditional reindeer products, we initially conducted a cross-sectional screening with the participation of the Indigenous

inhabitants of the Arctic zone of Western Siberia. The inclusion criteria for the respondents were as follows: be over 18 years of age, be of Indigenous origin, be an Indigenous language speaker, be involved in reindeer herding, live a nomadic or semi-nomadic lifestyle and have resided in the tundra or the settlements of the Arctic zone of Western Siberia for over five years. The sequence of the survey was as follows: during an expedition to the settlement between 2012 and 2018, respondents were invited to participate in the survey while undergoing a medical examination conducted by the YNAO Arctic Scientific Research Centre at health care institutions. After the aims and content of the research were explained and consent for participation was obtained, the name of each person invited to take the survey was recorded in a registry, and they were given a questionnaire. The questionnaire was developed in Russian using the methodological recommendations of the Russian Academy of Medical Science (RAMS) Nutrition Institute (1996, 2016) [90,91]. It collected information about socio-demographic factors and the consumption of reindeer products (reindeer meat, liver and blood). Data collection was performed by medical doctors who had been trained in the study procedures with the assistance of Indigenous nurses and researchers. All participants given the questionnaire were interviewed and underwent medical examinations at the beginning of the study by a general practitioner, pulmonologist and cardiologist. If a patient had been found to have severe somatic pathology, they would have been excluded from the survey, but no such cases were found. Participants filled out a confidential paper questionnaire. The consumption of reindeer products was analysed according to the following questions: "How many grams of reindeer meat did you consume over the previous 30 days?", "How many grams of reindeer liver did you consume over the previous 30 days?", "How many grams of reindeer blood did you consume over the previous 30 days?" The participants received information about the programme, both verbally and in writing, and they provided written informed consent. The consent form stated that participation was voluntary and that their confidentiality was assured. Participants' personal data and their answers were anonymised, numbered and entered into de-identified databases.

An analysis of the traditional foods (reindeer meat, liver and blood) was carried out using frequency and survey methods [63,92]. The amount of food consumed was estimated using a catalogue graphically depicting a range of portions of foods and dishes [93]. The amount of traditional products consumed over the previous 30 days was estimated in the survey. The blood pressure of the participants was measured three times according to the Korotkov method, and the presence of arterial hypertension (AH) was established in accordance with the recommendations for AH [94,95].

Statistical analyses were performed using Microsoft Excel 2016 and Statistica for Windows, v. 8.0 (StatSoft Inc., Oklahoma, OK, USA). Significant differences were defined at a *p*-value < 0.05.

## *2.4. Ethics Approval*

The study was approved by the Ethics Committee of the Arctic Scientific Research Centre of YNAO, Salekhard, Russian Federation, on 16 January 2012 (approval protocol No. 01/1-13). The research has been done in accordance with ethical concerns of working with the Indigenous Peoples in the Russian Federation (Constitution of the Russian Federation, Article 69. 14 March 2020). Communication was initiated with the Associations of the Indigenous Peoples and with representatives from national Indigenous communities of the Nadymsky, Yamalsky and Tazovsky districts in YNAO early in research planning. This resulted in an expression of interest from their representatives in having the research conducted in their communities.

## **3. Results and Discussion**

Our main findings showed that climate change (increasing average air temperature) has impacted the reindeer population mostly in the central area of the Arctic zone of Western Siberia (Nadymsky district), as we found a weak correlation between the dynam-

ics of average air temperature and reindeer livestock population in the northern areas (Yamalsky and Tazovsky districts). The weak correlation is explained with the stronger impact of non-climatic factors (i.e., the industrial development of the Arctic, changing socio-ecological and economic policies as well as tendencies of commodity production in the Indigenous reindeer herding communities). If the trend for increasing average annual temperatures continues at the same rate, the growth of the reindeer population in the YNAO will remain steady until 2025. After 2023, the dynamics of the reindeer population will slow down or become negative. Weather and extreme climatic events (the formation of an ice crust over large areas, freezing rain, heat waves) can cause catastrophic collapses in the reindeer population because many areas have insufficient reindeer pasture resources, which reduces the adaptive capabilities of the reindeer and the survival of offspring. Given the cyclical nature of these processes, the most dangerous period will be the spring of 2023–2024. However, these climatic effects and the reindeer population do not have direct impacts on food security, although the health status of the Indigenous population in the YNAO is strongly linked to the consumption of reindeer products.

The results of the correlation analysis of average air temperature and the reindeer population in three districts of the YNAO are presented in Table 1.


**Table 1.** Spearman rank-order correlations of average air temperature and the reindeer population.

The Nadymsky district located in the central part of the YNAO has different types of landscapes, including southern shrub tundra, forest-tundra and northern taiga. In this district, we found a direct strong statistically significant correlation between the average annual air temperature and the number of domesticated reindeer (rs = 0.5; *p* = 0.02). As the average annual temperature increased, the size of the reindeer herd increased, which is probably associated with the increase in the reindeer forage base because of the increase in the productivity of vascular plants in the warming climate. An increase in fodder naturally increases the adaptive reserves of animals, making it possible for them to endure unfavourable periods with fewer losses, increasing the survival of offspring and increasing the number of deer (Figure 1).

For the Yamal region, which is located above the Arctic Circle on the Yamal Peninsula, no reliable correlation was found between the reindeer population and air temperature (rs = 0.169278; *p* = 0.126048) (Figure 2).

For the Tazovsky region, which is located above the Arctic Circle on the Tazovsky, Gydansky and Yavay-Sale Peninsulas, only a weak direct correlation was found between the average annual air temperature and the number of domesticated reindeer (rs = 0.4; *p* = 0.0001) (Figure 3).

*Sustainability* **2021**, *13*, 2561 7 of 23

**Figure 1.** Correlation between the reindeer population and average annual air temperature in the Nadymsky district, 1960–2018. **Figure 1.** Correlation between the reindeer population and average annual air temperature in the Nadymsky district, 1960–2018. *Sustainability* **2021**, *13*, 2561 8 of 23

**Figure 2.** Correlation between the reindeer population and average annual air temperature in the Yamalsky district, 1936– 2018. **Figure 2.** Correlation between the reindeer population and average annual air temperature in the Yamalsky district, 1936–2018.

**Figure 3.** Correlation between the reindeer population and average annual air temperature in the Tazovsky district, 1960– 2018. **Figure 3.** Correlation between the reindeer population and average annual air temperature in the Tazovsky district, 1960–2018.

Understanding of vulnerability of reindeer husbandry requires one assess at least three separate aspects: the external impacts on the social-ecological systems, the ability of these systems to cope and adapt to these impacts, and the extent to which environmental or societal conditions hinder herders in adapting to change [96]. Climate change is one of the important external impacts on reindeer herding. The differences in the impact of climate change on the reindeer population in the Nadymsky, Yamalsky and Tazovsky districts are partly connected with the thermal balance in the northern and central areas of the YNAO. Klokov K.B. et al. [97] stated that the areas for wintering with the best thermal balance are located in the northern (tundra) parts of the Nadymsky, Purovsky and Shuryshkarsky districts of the YNAO. In July–August, the optimal grazing conditions (without going beyond the thermoneutral zone) are in the northern part of the Yamal Peninsula. On the Gydansky and Tazovsky Peninsulas, the thermal balance sometimes goes beyond the thermoneutral zone. The positive impact of climate change on the reindeer population can be explained by the increase in the snowless period and, consequently, the increased productivity of Understanding of vulnerability of reindeer husbandry requires one assess at least three separate aspects: the external impacts on the social-ecological systems, the ability of these systems to cope and adapt to these impacts, and the extent to which environmental or societal conditions hinder herders in adapting to change [96]. Climate change is one of the important external impacts on reindeer herding. The differences in the impact of climate change on the reindeer population in the Nadymsky, Yamalsky and Tazovsky districts are partly connected with the thermal balance in the northern and central areas of the YNAO. Klokov K.B. et al. [97] stated that the areas for wintering with the best thermal balance are located in the northern (tundra) parts of the Nadymsky, Purovsky and Shuryshkarsky districts of the YNAO. In July–August, the optimal grazing conditions (without going beyond the thermoneutral zone) are in the northern part of the Yamal Peninsula. On the Gydansky and Tazovsky Peninsulas, the thermal balance sometimes goes beyond the thermoneutral zone.

vascular plants in the warming climate. Finnish reindeer herders have also noted this favourable effect of climate change on increased forage: a rainier but warmer and longer growing season may increase the growth of vegetation and availability of high-quality forage, such as mushrooms, for reindeer [3,17]. Arctic areas are sensitive to such changes in forage conditions. In the Arctic zone of Western Siberia, reindeer consume more than 600 species of lichens, grasses, shrubs, trees and mushrooms [98], while in Chukotka and Alaska, there are over 1000 species of vascular plants [99]. Thus, reindeer can survive The positive impact of climate change on the reindeer population can be explained by the increase in the snowless period and, consequently, the increased productivity of vascular plants in the warming climate. Finnish reindeer herders have also noted this favourable effect of climate change on increased forage: a rainier but warmer and longer growing season may increase the growth of vegetation and availability of high-quality forage, such as mushrooms, for reindeer [3,17]. Arctic areas are sensitive to such changes in forage conditions. In the Arctic zone of Western Siberia, reindeer consume more than 600 species of lichens, grasses, shrubs, trees and mushrooms [98], while in Chukotka and Alaska, there are over 1000 species of vascular plants [99]. Thus, reindeer can survive without lichen if there are many wet plants under the snow, such as horsetail and evergreen grasses, which contain many vitamins, proteins and microelements. Climate change in Siberia has had environmental effects, including inducing changes in species composition [100–103],

the abundance of animals and plants [104–106] and annual biological cycles [2,107]. It has a strong influence on the grazing cover of reindeer pastures, especially in the central parts of the YNAO, where thinner snow cover, milder weather and shorter periods of low temperatures are favourable for reindeer because of the higher availability of forage. For example, the tendency for the increased growth of deciduous shrubs observed in recent decades is apparently closely related to the higher and more intense summer temperatures in Western Siberia [108]. S. Rasmus et al. also noted the positive effects of the advanced development of vegetation and the consequent higher availability of fresh forage plants for milk-producing dams and their calves, as the calves would be fit by the time the calf marking period starts in the summer [3,107]. However, in the more northern parts of Siberia and Fennoscandia, "foraging conditions have been deteriorated due to hard snow and icy layers formed on the soil and snow cover resulting in declined availability of ground lichens for reindeer. Hard snow and rime accumulated on the branches of trees have also decreased the availability of arboreal lichens" [3]. Walker et al. [109] demonstrated this phenomenon of a faster increase in the aboveground biomass of circumpolar arctic tundra vegetation in more southern Arctic areas: southern tundra subzones exhibited approximately 20–26% biomass increases, whereas northern tundra subzones had increases of 2–7%. However, these processes are much slower in the YNAO. Kovalevskaya N.M. et al. [49] mentioned that the results of their analysis of satellite data suggest that over the past three decades and more, there has been a relatively small (compared to that in other Arctic regions) increase in the productivity of Yamal vegetation and a slight increase in near-surface temperatures. While the degradation of reindeer pastures is increasing (Figure 4) that results in destroying reindeer health (Figure 5) and changing the reindeer diet. forage plants for milk-producing dams and their calves, as the calves would be fit by the time the calf marking period starts in the summer [3,107]. However, in the more northern parts of Siberia and Fennoscandia, "foraging conditions have been deteriorated due to hard snow and icy layers formed on the soil and snow cover resulting in declined availability of ground lichens for reindeer. Hard snow and rime accumulated on the branches of trees have also decreased the availability of arboreal lichens" [3]. Walker et al. [109] demonstrated this phenomenon of a faster increase in the aboveground biomass of circumpolar arctic tundra vegetation in more southern Arctic areas: southern tundra subzones exhibited approximately 20–26% biomass increases, whereas northern tundra subzones had increases of 2–7%. However, these processes are much slower in the YNAO. Kovalevskaya N.M. et al. [49] mentioned that the results of their analysis of satellite data suggest that over the past three decades and more, there has been a relatively small (compared to that in other Arctic regions) increase in the productivity of Yamal vegetation and a slight increase in near-surface temperatures. While the degradation of reindeer pastures is increasing (Figure 4) that results in destroying reindeer health (Figure 5) and changing the reindeer diet. Warming has an unclear effect on Arctic reindeer herding and can also result in higher reindeer losses because of increased risks of entomoses [110] and helminthic diseases [42]. Besides, reindeer are better adapted to hypothermia than to overheating. In hot weather, the physiological activity of metabolic processes in reindeer decreases, and they are not able to accumulate a sufficient supply of nutrients for successful wintering [97]. Therefore, reindeer losses are a likely outcome of climate warming in the Arctic zone of Western Siberia.

without lichen if there are many wet plants under the snow, such as horsetail and evergreen grasses, which contain many vitamins, proteins and microelements. Climate change in Siberia has had environmental effects, including inducing changes in species composition [100–103], the abundance of animals and plants [104–106] and annual biological cycles [2,107]. It has a strong influence on the grazing cover of reindeer pastures, especially in the central parts of the YNAO, where thinner snow cover, milder weather and shorter periods of low temperatures are favourable for reindeer because of the higher availability of forage. For example, the tendency for the increased growth of deciduous shrubs observed in recent decades is apparently closely related to the higher and more intense summer temperatures in Western Siberia [108]. S. Rasmus et al. also noted the positive effects of the advanced development of vegetation and the consequent higher availability of fresh

*Sustainability* **2021**, *13*, 2561 10 of 23

**Figure 4.** Degradation of reindeer pastures in Yamal. **Figure 4.** Degradation of reindeer pastures in Yamal.

Warming has an unclear effect on Arctic reindeer herding and can also result in higher reindeer losses because of increased risks of entomoses [110] and helminthic diseases [42]. Besides, reindeer are better adapted to hypothermia than to overheating. In hot weather, the physiological activity of metabolic processes in reindeer decreases, and they are not able to accumulate a sufficient supply of nutrients for successful wintering [97]. Therefore, reindeer losses are a likely outcome of climate warming in the Arctic zone of Western Siberia.

**Figure 5.** Teeth of a three-year-old deer worn out on the mineral base in the tundra. **Figure 5.** Teeth of a three-year-old deer worn out on the mineral base in the tundra.

The rather weak impact of climate change on the reindeer population in the northern areas of the YNAO (Yamalsky and Tazovsky districts) can be explained by the more significant contribution of non-climatic factors. This region is known as the place with the most intensive industrial development in the Arctic region of Russia. Industrial development is associated with the degradation of reindeer pastures due to the disruptive effect of developing logistic and industrial infrastructure. Anthropogenic activities [2] in recent years and technogenic emissions of combustion products from fuel and energy enterprises into the atmosphere [111–114], as well as metallurgical production and mining, which produce excessive amounts of some macro-and microelements, all of which affect the physiological parameters of reindeer. In the areas near industrial centres, atmospheric pollution affects lichen first, causing its disappearance from the vegetation cover of pastures [51]. Nevertheless, YNAO is the only Arctic region in Russia where the number of domesticated reindeer has constantly increased during the last 50 years even in the conditions of intensive gas and oil extraction, political shifts and during the crisis of the 1990s [115]. It can be a result of the positive impact of industrial development and extending urban areas which provided stable markets for reindeer products. Oil and gas companies gave a stable tax base which was reinvested also in the rural economy including reindeer enterprises [25]. A high degree of nomadism among the Indigenous Peoples and dominance of individual reindeer husbandries (60% of reindeer belong to individual reindeer herders [71]) in YNAO became the preconditions for stronger socio-ecological and economic resilience of reindeer herding livelihood to shifting political context in 1960–2018. While in other Russian Arctic regions, there were dramatic declines in reindeer numbers associated with the years of institutional reforms wherein ownership rights changed; the reorganization of kolkhozes (enterprises with collective ownership) into sovkhozes (enterprises with state ownership). Later from 1991 to 2003, a decrease in the reindeer population was caused by the withdrawal of state subsidies. Finally, up to 2003, the reindeer population in reindeer enterprises decreased to approximately 35% of their previous populations [25] in all Arctic reindeer herding regions (except YNAO) of Russia. The rather weak impact of climate change on the reindeer population in the northern areas of the YNAO (Yamalsky and Tazovsky districts) can be explained by the more significant contribution of non-climatic factors. This region is known as the place with the most intensive industrial development in the Arctic region of Russia. Industrial development is associated with the degradation of reindeer pastures due to the disruptive effect of developing logistic and industrial infrastructure. Anthropogenic activities [2] in recent years and technogenic emissions of combustion products from fuel and energy enterprises into the atmosphere [111–114], as well as metallurgical production and mining, which produce excessive amounts of some macro-and microelements, all of which affect the physiological parameters of reindeer. In the areas near industrial centres, atmospheric pollution affects lichen first, causing its disappearance from the vegetation cover of pastures [51]. Nevertheless, YNAO is the only Arctic region in Russia where the number of domesticated reindeer has constantly increased during the last 50 years even in the conditions of intensive gas and oil extraction, political shifts and during the crisis of the 1990s [115]. It can be a result of the positive impact of industrial development and extending urban areas which provided stable markets for reindeer products. Oil and gas companies gave a stable tax base which was reinvested also in the rural economy including reindeer enterprises [25]. A high degree of nomadism among the Indigenous Peoples and dominance of individual reindeer husbandries (60% of reindeer belong to individual reindeer herders [71]) in YNAO became the preconditions for stronger socio-ecological and economic resilience of reindeer herding livelihood to shifting political context in 1960–2018. While in other Russian Arctic regions, there were dramatic declines in reindeer numbers associated with the years of institutional reforms wherein ownership rights changed; the reorganization of kolkhozes (enterprises with collective ownership) into sovkhozes (enterprises with state ownership). Later from 1991 to 2003, a decrease in the reindeer population was caused by the withdrawal of state subsidies. Finally, up to 2003, the reindeer population in reindeer enterprises decreased to approximately 35% of their previous populations [25] in all Arctic reindeer herding regions (except YNAO) of Russia.

Nowadays, one of the key factors impacting the reindeer population in the northern areas of the YNAO (Yamalsky and Tazovsky districts) is the increase in velvet antlers production. Mainly nomadic reindeer husbandries of Tazovskaya, Messoyakhinskaya, Antipayutinskaya, Tanamskaya tundras are integrated into the commodity production of meat and velvet antlers. This is due to the relatively good logistics, the presence of reindeer slaughtering houses and the possibility to sell them velvet antlers. In this group, over the past four years, the sale of velvet antlers has gradually become the main source of Nowadays, one of the key factors impacting the reindeer population in the northern areas of the YNAO (Yamalsky and Tazovsky districts) is the increase in velvet antlers production. Mainly nomadic reindeer husbandries of Tazovskaya, Messoyakhinskaya, Antipayutinskaya, Tanamskaya tundras are integrated into the commodity production of meat and velvet antlers. This is due to the relatively good logistics, the presence of reindeer slaughtering houses and the possibility to sell them velvet antlers. In this group, over the past four years, the sale of velvet antlers has gradually become the main source of income for reindeer herders, and the delivery of meat is an auxiliary one. Subsistence reindeer

herding, working mainly to provide the family with food and clothing, is widespread in the northern part of the Gydansky Peninsula, on the coast of the Yuratskaya Bay and other parts of the region, logistically remote from settlements, slaughter facilities and large deposits. This is explained by the high cost of gasoline and the cost of exporting reindeer products and delivering goods to the tundra. The further from the logistics centres, the less profitable commodity production becomes due to the high costs [27]. However, the risk of climate change should also not be ignored in the northern areas of the YNAO, as its influence in the neighbouring territories of the Nenets Autonomous Okrug [82] is rather evident based on changes in the vegetation cover of the tundra and reduced availability of lichen for reindeer under the ice and snow.

In general, climate change can have catastrophic effects on forage for reindeer herding in different Arctic areas. Rees et al. [116] argued that climate change impacts are likely to harm the livelihoods of those who practice reindeer husbandry in Norway and Sweden but have a neutral impact in Finland. The main factors that dictate these impacts are the changes in vegetation distribution caused by the changing climate, especially winter temperature and winds. Nevertheless, Rees et al. [116] suggested that the effect of these changes is expected to be relatively small and well within the range of previous experiences of reindeer herders dealing with climate variability. However, in Western Siberia, climate changes (autumn, winter and summer warming) may become a serious challenge for the preservation of nomadic reindeer herding [117].

A forecast model for the number of reindeer correlated to the average annual air temperatures was developed. We analysed the dynamics of average annual temperatures and the number of reindeer using the ARIMA variable method. It was found that the number of reindeer livestock undergoes seasonal fluctuations every 5 and 10 years and exhibits an overall upward trend. The analysis of the number of domesticated reindeer in the YNAO using autocorrelation and partial autocorrelation with the ARIMA model revealed the presence of a decreasing trend and lag in steps 1 and 7 and the presence of seasonality of 10. Based on the data obtained, a forecast for the number of reindeer in five years was developed. According to this forecast, under the optimistic scenario with the continued increasing rates for average annual temperatures, the number of reindeer livestock will increase; under the pessimistic scenario, it will stabilize at the 2016 values (Figure 6).

The ARIMA forecasting model showed that, with the continued increasing rates of average annual temperatures, the growth of the reindeer population in the YNAO will continue until 2025. After 2023, the dynamics of reindeer population growth will slow down or become negative. This model describes only long-term trends. Weather and climatic extremes (the formation of an ice crust over large areas, freezing rain, heat waves) can have catastrophic effects on the reindeer population because, in many areas, reindeer pasture resources are insufficient, which reduces the adaptive capabilities of the reindeer and the survival of offspring. Given the cyclical nature of these processes, the most dangerous period was predicted to be the spring of 2023–2024. However, Klokov K.B. et al. [118] were not as optimistic regarding climate change increasing reindeer livestock populations. With each increase in the average monthly air temperature of 2 ◦C, the boundaries of the thermoneutral zone can move north by about 100 km, which will lead to worse conditions for keeping reindeer, especially in warm years. As a result, the southern part of the Yamal Peninsula may become a zone of "risky reindeer herding", and the zone of comfortable grazing will be reduced.

*Sustainability* **2021**, *13*, 2561 13 of 23

**Figure 6.** Forecast model for the reindeer population in the Yamal-Nenets Autonomous Okrug (YNAO), 1936–2023. **Figure 6.** Forecast model for the reindeer population in the Yamal-Nenets Autonomous Okrug (YNAO), 1936–2023.

The dynamics of the reindeer livestock population will likely have an impact on the food security of Arctic Indigenous Peoples because of the increased availability of reindeer products. We assessed the correlation between the reindeer population and consumption of the most important [65] reindeer products (reindeer meat, liver and blood). A total of 1280 Indigenous inhabitants of the Arctic zone of Western Siberia participated in the study (Table 2). The age class distributions of the participants are presented in Table 3. The average age of all participants was 45.7 ± 14.3 years; 396 (30.9%) of the participants were men, and 884 (69.1%) were women. From year to year, there were no differences between the proportions of males/females and various age classes sampled, as the sample remained constant throughout the study. The dynamics of the reindeer livestock population will likely have an impact on the food security of Arctic Indigenous Peoples because of the increased availability of reindeer products. We assessed the correlation between the reindeer population and consumption of the most important [65] reindeer products (reindeer meat, liver and blood). A total of 1280 Indigenous inhabitants of the Arctic zone of Western Siberia participated in the study (Table 2). The age class distributions of the participants are presented in Table 3. The average age of all participants was 45.7 ± 14.3 years; 396 (30.9%) of the participants were men, and 884 (69.1%) were women. From year to year, there were no differences between the proportions of males/females and various age classes sampled, as the sample remained constant throughout the study.

2012–2018. **Total Indigenous Popula-Table 2.** Data on the respondents recruited for the survey, Yamal-Nenets Autonomous Okrug, 2012–2018.

**Table 2.** Data on the respondents recruited for the survey, Yamal-Nenets Autonomous Okrug,



**Table 3.** Forecast data for the reindeer population in the YNAO, 1936–2023.

The results of the correlation analysis were inconsistent: an increase in the reindeer livestock population was associated with the decreased consumption of reindeer products (Table 4; Figures 7–9). This does not represent a logical outcome of increases in the number of reindeer in herds. However, this phenomenon is frequently discussed in socioeconomic studies and considered to be a clear outcome of transition processes involving traditional lifestyles [27] and the traditional Indigenous economy—for example, the transfer from a subsistence economy in Western Siberia to commodity production. During the last 10 years, rising trends in the export of traditional reindeer products have decreased local Indigenous Peoples' access to venison and had a negative impact on their health [71]. The export potential of the non-edible parts of reindeer (i.e., velvet antlers, reindeer skins, camuses) should be promoted to support the food security of the Indigenous Peoples, while government policies should focus on improving the access of the Indigenous communities to the edible and medicinal portions of the carcass. *Sustainability* **2021**, *13*, 2561 15 of 23 dant protection against free radicals; these effects may explain the high prophylactic ac-

> **Table 4.** Spearman rank-order correlations for the reindeer population and consumption of reindeer products. tivity of venison [62] and its strong positive effects on adapting to cold stress [66] and geomagnetic activity in the Arctic [67]. Venison has been shown to effectively reduce hypertension [59] and the risk of chronic nonobstructive bronchitis [56]. These characteristics


**Figure 7.** Correlation between the reindeer livestock population and consumption of reindeer **Figure 7.** Correlation between the reindeer livestock population and consumption of reindeer meat.

**Figure 8.** Correlation between the reindeer livestock population and consumption of reindeer

meat.

liver.

embedded [119].

meat.

liver.

blood.

hypertension.

tension.

**Figure 8.** Correlation between the reindeer livestock population and consumption of reindeer **Figure 8.** Correlation between the reindeer livestock population and consumption of reindeer liver.

**Figure 7.** Correlation between the reindeer livestock population and consumption of reindeer

dant protection against free radicals; these effects may explain the high prophylactic activity of venison [62] and its strong positive effects on adapting to cold stress [66] and geomagnetic activity in the Arctic [67]. Venison has been shown to effectively reduce hypertension [59] and the risk of chronic nonobstructive bronchitis [56]. These characteristics make reindeer products an important part of the local population's diet. Maintaining a traditional diet is also an important part of the Indigenous culture that strongly contributes to promoting the Indigenous Peoples' health (Figure 13). Saving these nutritious practices has a positive effect on keeping reindeer herding culturally and environmentally

**Figure 9.** Correlation between the reindeer livestock population and consumption of reindeer **Figure 9.** Correlation between the reindeer livestock population and consumption of reindeer blood.

Traditional nutrition is an important part of the Indigenous traditional culture and serves as a remedy to cold stress and increases their adaptation to the severe Arctic climatic conditions. Therefore, as expected, "the Spearman rank-order correlation revealed a strong negative association between the prevalence of arterial hypertension and the consumption of reindeer products" (Table 5; Figures 10–12).

**Table 5.** Spearman rank-order correlations for the consumption of reindeer products and risk of arterial hypertension.


**Figure 10.** Correlation between the consumption of reindeer meat and the prevalence of arterial

**Figure 11.** Correlation between the consumption of reindeer liver and prevalence of arterial hyper-

blood.

blood.

*Sustainability* **2021**, *13*, 2561 16 of 23

**Figure 10.** Correlation between the consumption of reindeer meat and the prevalence of arterial hypertension. **Figure 10.** Correlation between the consumption of reindeer meat and the prevalence of arterial hypertension. **Figure 10.** Correlation between the consumption of reindeer meat and the prevalence of arterial hypertension.

**Figure 9.** Correlation between the reindeer livestock population and consumption of reindeer

**Figure 9.** Correlation between the reindeer livestock population and consumption of reindeer

**Figure 11.** Correlation between the consumption of reindeer liver and prevalence of arterial hypertension. **Figure 11.** Correlation between the consumption of reindeer liver and prevalence of arterial hypertension. **Figure 11.** Correlation between the consumption of reindeer liver and prevalence of arterial hypertension. *Sustainability* **2021**, *13*, 2561 17 of 23

**Figure 12.** Correlation between the consumption of reindeer blood and the prevalence of arterial hypertension. were intended to analyse the impact of climatic factors on the food security of the Indige- **Figure 12.** Correlation between the consumption of reindeer blood and the prevalence of arterial hypertension.

Dissemination of knowledge on the importance of reindeer products for the Indigenous Peoples health as well as strengthening social policy to support reindeer herding husbandries in YNAO could encourage reindeer herders to keep on following nomadic lifestyle and, correspondingly, a traditional diet with a prevalence of reindeer meat and

The main strength of our study was using unique empirical quantitative research data collected from the reindeer herders during expeditions that took place over seven years (2012–2018) and data from the longitudinal monitoring of reindeer livestock in the period 1936–2018 provided by researchers and local authorities. Most similar studies examined fragmented populations and time frames, with unclear results. However, our study had several limitations. The methodological quantitative approach was focused on using a limited number of variables for representing climate and diet changes since we

**Figure 13.** Traditional Yamal Nenets ritual of drinking fresh blood of a slaughtered reindeer in the tundra.

other-by reindeer products.

The results of our previous study showed that there was a dramatic decrease of almost 50% in the consumption of reindeer products by the Indigenous and non-Indigenous Peoples in the YNAO, and only one-third of the studied population still ate venison once or twice daily [65]. This shift threatens their health because a diet rich in venison significantly increases antiatherogenic blood lipid fractions, contributes to the maintenance of normal body weight, and improves microcirculation, tissue fluid exchange and antioxidant protection against free radicals; these effects may explain the high prophylactic activity of venison [62] and its strong positive effects on adapting to cold stress [66] and geomagnetic activity in the Arctic [67]. Venison has been shown to effectively reduce hypertension [59] and the risk of chronic nonobstructive bronchitis [56]. These characteristics make reindeer products an important part of the local population's diet. Maintaining a traditional diet is also an important part of the Indigenous culture that strongly contributes to promoting the Indigenous Peoples' health (Figure 13). Saving these nutritious practices has a positive effect on keeping reindeer herding culturally and environmentally embedded [119]. **Figure 12.** Correlation between the consumption of reindeer blood and the prevalence of arterial hypertension.

*Sustainability* **2021**, *13*, 2561 17 of 23

**Figure 13.** Traditional Yamal Nenets ritual of drinking fresh blood of a slaughtered reindeer in the tundra. **Figure 13.** Traditional Yamal Nenets ritual of drinking fresh blood of a slaughtered reindeer in the tundra.

Dissemination of knowledge on the importance of reindeer products for the Indigenous Peoples health as well as strengthening social policy to support reindeer herding husbandries in YNAO could encourage reindeer herders to keep on following nomadic lifestyle and, correspondingly, a traditional diet with a prevalence of reindeer meat and other-by reindeer products. Dissemination of knowledge on the importance of reindeer products for the Indigenous Peoples health as well as strengthening social policy to support reindeer herding husbandries in YNAO could encourage reindeer herders to keep on following nomadic lifestyle and, correspondingly, a traditional diet with a prevalence of reindeer meat and other-by reindeer products.

The main strength of our study was using unique empirical quantitative research data collected from the reindeer herders during expeditions that took place over seven years (2012–2018) and data from the longitudinal monitoring of reindeer livestock in the period 1936–2018 provided by researchers and local authorities. Most similar studies examined fragmented populations and time frames, with unclear results. However, our study had several limitations. The methodological quantitative approach was focused on using a limited number of variables for representing climate and diet changes since we were intended to analyse the impact of climatic factors on the food security of the Indige-The main strength of our study was using unique empirical quantitative research data collected from the reindeer herders during expeditions that took place over seven years (2012–2018) and data from the longitudinal monitoring of reindeer livestock in the period 1936–2018 provided by researchers and local authorities. Most similar studies examined fragmented populations and time frames, with unclear results. However, our study had several limitations. The methodological quantitative approach was focused on using a limited number of variables for representing climate and diet changes since we were intended to analyse the impact of climatic factors on the food security of the Indigenous Peoples. While non-climatic drivers were also considered to explain the weak correlation of the factors. The studied population was recruited while undergoing a medical examination at health care institutions—municipal hospitals and feldsher-midwife medical stations in remote settlements. Participation was voluntary and did not include all

representatives of the reindeer herding communities of the studied territories, which may limit the generalizability of the findings. It would also be of value to examine food security in reindeer herding and fishing communities and the impact of traditional reindeer and fishing products and plants on the health and wellbeing of the local communities.

## **4. Conclusions**

Studying climate change and its impacts on reindeer herding while considering the input of anthropogenic and technogenic factors can provide new insights into the temporal and spatial warming variabilities in the Arctic zone of Western Siberia. In our study, the different data sets on the dynamics of annual air temperatures and reindeer livestock populations in three districts of the YNAO enabled us to not only study the ongoing climate changes in general but also examine their specific impacts on the Indigenous Peoples' food security. We conclude that as the average annual temperature increases, the size of the reindeer herds will grow, which is probably associated with the increase in the forage base for the reindeer because of the increased productivity of vascular plants in the warming climate. In the YNAO, an increase in the average annual temperature was correlated with an increase in the reindeer population and, in the long term, did not depend on the socio-economic model of the organization of reindeer husbandry, despite the fact that from 1936 to 2019, there were significant political and economic changes. This trend for the impact of climate change was more evident in the central districts of the YNAO.

Climate change is occurring and will continue to occur, faster in higher latitudes than in other regions. Climate change's consequences for livelihoods dependent on reindeer herding should be analysed in the context of the impacts of associated challenges and opportunities, such as exploration of the Arctic's bioresources and involvement of the Indigenous Peoples in bioproduction. Adaptation to climate change could be used as an opportunity to improve the living conditions and food security of the Indigenous Peoples and to sustain their livelihoods in the context of all related issues. Thus, climate change becomes an opportunity for supporting Arctic life and livelihoods. Any policies or regulatory measures should be developed, implemented, monitored and enforced with the full and fair participation of the Indigenous Peoples.

Climate change is already occurring and is unlikely to be curtailed soon, meaning that the effects must be addressed. The impacts of climate change on the Indigenous Peoples can be reduced by working collaboratively to ensure that indigenous interests are respected and that indigenous needs are met without precluding the involvement of others in the region and without being overwhelmed by climate change's detrimental impacts.

The results presented in this work will hopefully encourage dialogue among local practitioners, researchers and policymakers. Our study focused on reindeer husbandry, but the approach is applicable to other traditional Indigenous nature-based livelihoods (e.g., fishing, hunting, and gathering) facing the need to adapt because of the changing climate.

**Author Contributions:** Data Curation, S.A.; Formal Analysis, S.A.; Funding Acquisition, E.B. and A.S.; Investigation, A.L. and E.B.; Methodology, A.L.; Project Administration, E.B.; Resources, S.A.; Software, S.A.; Supervision, A.L.; Validation, S.A. and O.S.; Writing—Original Draft, E.B. and A.L.; Writing—Review and Editing, S.A.; Overall Editing and Assessment, K.H., D.R. and A.S. All authors have read and agreed to the published version of the manuscript.

**Funding:** This study was funded by the Russian Foundation of Basic Research, project number 20-55-71004 and Belmont Forum project number 1729 SERUS.

**Institutional Review Board Statement:** The study was conducted according to the guidelines of the Declaration of Helsinki, and approved by the Ethics Committee of the Arctic Scientific Research Centre of YNAO (protocol No. 01/1-13, 16 January 2012).

**Informed Consent Statement:** Informed consent was obtained from all subjects involved in the study.

**Data Availability Statement:** The study did not report any data.

**Acknowledgments:** We thank the Indigenous communities of the YNAO, Department of Agroindustrial Complex of YNAO, Department of Health Care of YNAO and health care institutions (municipal hospitals and feldsher-midwife medical stations in remote settlements) of the YNAO for their assistance and sharing data.

**Conflicts of Interest:** The authors declare no conflict of interest.

## **References**


## *Article* **Extraction of Essential Oil from River Tea Tree (***Melaleuca bracteata* **F. Muell.): Antioxidant and Antimicrobial Properties**

**Mursleen Yasin 1,\*, Adnan Younis <sup>1</sup> , Fahad Ramzan <sup>1</sup> , Talha Javed 2,3 , Rubab Shabbir <sup>2</sup> , Hamza Armghan Noushahi <sup>4</sup> , Milan Skalicky <sup>5</sup> , Peter Ondrisik <sup>6</sup> , Marian Brestic <sup>6</sup> , Sabry Hassan <sup>7</sup> and Ayman EL Sabagh 8,\***


**Abstract:** Tea tree oil (TTO) from the genus *Melaleuca* L. has antimicrobial, antibacterial, antifungal, and antioxidant properties and is used by the cosmetic, pharmaceutical, and horticultural industries. In Pakistan, *Melaleuca bracteata* can be exploited for essential oil purposes, as this species is well adapted to Pakistan's agroclimatic conditions. The objective of the present study was to evaluate the yield of *M. bracteata* essential oil together with its antioxidant and antimicrobial properties under local prevailing conditions of the subtropics. Essential oil was extracted through the hydrodistillation method. Using this method, six batches of 8 kg samples (fresh leaves and branches) underwent a distillation process for 4–5 h. The average yield obtained was about 0.2%. The GCMS was used to identify the components of extracted essential oil. Eugenol methyl ether is the major component in extracted essential oil, i.e., 96% of the total. A high content of flavonoids and phenolics and a Fe-reducing power ability of *M. bracteata* were observed. The oil was also found effective against *B. subtilis*, *B. cereus*, White rot, and *A. flavus*. Hence, it is concluded that there is a possibility to use TTO for its biocidal properties, and it must also be inspected and then commercialized in Pakistan by the agriculture and cosmetic industries.

**Keywords:** *Melaleuca bracteata*; essential oil; hydrodistillation; eugenol methyl ether; antioxidant; antimicrobial

## **1. Introduction**

Natural products from plants are under immense consideration in the organic world. One such utilization is of essential oils that are complex volatile compounds extracted from leaves, twigs, flowers, seeds, branches, buds, fruits, and roots. These secondary compounds are then stored in secretory cells, glandular trichomes, canals, cavities, and epidermal cells [1]. Use of essential oils (EOs) is expeditiously increased because of their antibacterial, antiviral, antifungal, insecticidal [1], anti-inflammatory [2], and antimicrobial traits [3]. EOs also have antidepressant and anxiolytic activities [4]. These characters are possessed by certain chemical compounds, mainly phenolics, alkaloids, flavonoids, glycosides, and flavonoids [5].

**Citation:** Yasin, M.; Younis, A.; Ramzan, F.; Javed, T.; Shabbir, R.; Noushahi, H.A.; Skalicky, M.; Ondrisik, P.; Brestic, M.; Hassan, S.; et al. Extraction of Essential Oil from River Tea Tree (*Melaleuca bracteata* F. Muell.): Antioxidant and Antimicrobial Properties. *Sustainability* **2021**, *13*, 4827. https:// doi.org/10.3390/su13094827

Academic Editors: António Raposo and Marc A. Rosen

Received: 18 March 2021 Accepted: 22 April 2021 Published: 25 April 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

The Myrtaceae family would rank among the most important families of plants which are enriched with essential oils. Genus Melaleuca is one of the prominent members of the family Myrtaceae that is also well known for its essential oils. *Melaleuca* is indigenous to Australia yet can adjust into several agroclimatic conditions. The volatile oil product of this genus is familiar to the world with the name of tea tree oil (TTO). The species that are under commercial use for essential oil production are *Melaleuca alternifolia*, *M. cajuputi*, *M. bracteata*, and *M. quinquenervia* [6]. Plant extract including oil was used by Aboriginals and was also utilized by soldiers as a general antimicrobial and insect repellent in World War II [7]. Today, TTO is gaining popularity in different industries and being used in aromatherapy, herbal and allopathic medicines, cosmetics, and as natural biocide, fungicide, and preservative. The oil is slightly yellow colored to transparent with a strong camphor-like odor and menthol-like cooling effect [8,9]. The composition of TTO has almost 100 different chemicals, predominantly monoterpenes (terpinen-4-ol, terpinolene, p-cymene, α-pinene, γ-terpinene, 1,8-cineole), sesquiterpenes, and their respective alcohols (monoterpene, alcohol-terpineol) [10].

In Pakistan, the *Melaleuca* genus has been cultivated in the Punjab region for a long time, but its properties and products are yet not investigated here. *Melaleuca bracteata* is one of the species that is well adapted to the agroclimatic conditions of Pakistan [11]. Tea tree oil contains antioxidant, cytotoxic, antifungal, insecticidal, and medicinal properties [12–14]. Reports have shown methyl eugenol and phenylpropanoids to be the prime components of tea tree oil from *M. bracteata*. Moreover, it can be inferred from literature that TTO can be used effectively by the horticulture sector as a chemical-free fungicide, bactericide, and for enhancement of the shelf life and vase life of fruits, vegetables, and flowers. Hence, it is necessary to test the effectiveness and appropriate use of this essential oil so it can be extracted and employed by the local industry more cost-effectively. Therefore, this study aims to investigate the local adapted species of tea tree oil for more exploration and assessment of this essential oil.

## **2. Materials and Methods**

## *2.1. Plant Material*

Fresh plant material including leaves and twigs were collected from the University of Agriculture Faisalabad and Qadir Baksh Nursery Farms Faisalabad, Pakistan (31.43◦ N, 73.07◦ E). The sample for extraction was prepared by clipping leaves and young twigs into smaller pieces.

## *2.2. Extraction Method*

Essential oil was extracted from Melaleuca via the hydrodistillation technique at the Department of Chemistry, University of Agriculture Faisalabad, Pakistan (31.43◦ N, 73.07◦ E). The unit consisted of a distillation flask/tank, condenser, and thermometer. About 8–10 kg of plant material was added in flask and submerged into 12 L of water. Then, the set up was airtightly closed, and the process was started at 250 ◦C temperature. The procedure continued for 4–5 h. Oil was collected in a separating funnel. The %age yield of the extracted oil (*v*/*w*) was calculated by the following formula [15], and then the oil was stored at a cool dry place [16].

Essential Oil Yield % = Volume of extracted oil/Fresh weight of plant material (g) × 100. (1)

## *2.3. Analysis of Tea Tree oil by GCMS*

The extracted oil samples were subjected to compound analysis via the gas chromatography–mass spectrometer technique. The detector of Agilent 5975C was used, and it was provisioned with 7890A GC and column of HP-5MS with 30 m length, 250 micrometer width, and 0.25 micrometer internal diameter with a temperature of 450◦C. The oven program was kept on, and the equilibrium time for the oven was 0 min. The program was run for 36 min, and the temperature of the heater was 240 ◦C. The inert gas

used as a carrier was Helium with a flow rate of 0.7 mL/min. Mass spectrometry was performed in EMV mode at relative voltage of 59eV and scan range from 50 to 550amu. The mass spectrometer was equipped with a gold standard quadrupole analyzer at 150 ◦C (max 200 ◦C). The temperature of MS source was 240 ◦C (max 250 ◦C). The Agilent MSD Productivity ChemStation6 was the software which identified the compounds receiving signals at the same time from both detectors [15].

## *2.4. Antioxidant Analysis*

## 2.4.1. DPPH Radical Scavenging Activity

Radical scavenging activity is the ability of an antioxidant compound to protect cells from free radicals produced during normal cell metabolism. To check this character of tea tree oil, DPPH (2,2-diphenyl-1-picrylhydrazyl) scavenging activity was carried out. For this purpose, 1.0 mL at 0.3 M of 1,1-diphenyl-2-picrylhydrazyl was added to the 2.5 mL solution of the sample and gallic acid standard. Standards of gallic acid were prepared at 20, 40, 60, 80, and 100 ppm. Then, all the test tubes were incubated at room temperature for 20 min and kept in the dark. After that, absorbance was recorded at 518 nm by spectrophotometer (T60 U Spectrophotometer, PG Instruments Ltd., Leicestershire, UK). DPPH 100 ppm was used as control treatment, and absorbance was recorded as Abs control. A blank experiment of 80% ethanol was run as well, and absorbance was calculated as Abs blank. The DPPH scavenging activity was determined from the following expression [17].

DPPH scavenging activity (%) = 100 − {[(Abs sample-Abs blank) × 100]/Abs control]}

## 2.4.2. Total Flavonoid Contents

Flavonoid compounds are antioxidants present in plants and all fruits and vegetables. To evaluate this in tea tree oil, 1.0 mL of samples and catechin standards at 20, 40, 60, 80, and 100 ppm were added in 4.0 mL of distilled water. The solution was prepared in a volumetric flask of 10 mL capacity. After this, 5% NaNO<sup>2</sup> weighing 0.3 mL was mixed in the above prepared solution. 1 M NaOH (2 mL) was added after 1 min, and later, after 5 m, 0.3 mL of 10% AlCl<sup>3</sup> was also mixed. After 60 s, 2 mL of 1 M NaOH was added to make the total volume up to 10 mL by deionized distilled water. We mixed it thoroughly, and absorbance was observed at 510 nm using reagent blank. Catechin equivalent in milligram per gram dry plant material was used to present the total flavonoids in TTO [18].

## 2.4.3. Total Phenolic Compounds

To estimate the phenolic compounds of tea tree oil, gallic acid standards at 20, 40, 60, 80, and 100 ppm were used. To 1.0 mL volume of both samples and standards, 5.0 mL of Folin-Ciocalteu (Fisher chemicals) and 4 mL of Na2CO<sup>3</sup> (7% *w*/*v*) were added with continuous shaking. Then, all the solutions were kept in dark for 30 min, and later on, absorbance was detected at 765 nm with the help of a spectrophotometer (T60 U Spectrophotometer, PG Instruments Ltd., Leicestershire, UK). A blank sample used was a reagent solution. A gallic acid equivalent (GAE) was used to express the amount of total phenolics based on milligram per gram of plant dry weight [19].

## 2.4.4. Reducing Power Ability (RPA) of the Plant

It is determined by adding 1.0 mL gallic acid standard solutions and sample to 0.2 molar solution of phosphate buffer solution (2.3 mL) at 6.6 pH. Then, 2.5 mL of 1% potassium ferricyanide (K3[Fe(CN)6]) was added to the solution. Incubation of the mixture was done at 37 ◦C for 20 m. Then, 10% trichloroacetic acid (2.5 mL) was mixed and then centrifuged at 1000 rpm for 10 m. The supernatant with a 2.5 mL volume was mixed with distilled water and 0.5 mL of 0.1% FeCl3. After 10 m, absorbance was determined at 700 nm using a spectrophotometer (T60 U Spectrophotometer, PG Instruments Ltd., Leicestershire, UK) [20].

## *2.5. Antimicrobial Analysis*

The inhibitory zone (IZ) assay of tea tree oil was tested on microbes using the disc diffusion method as proposed by Al-Abd et al. [21]. Firstly, an agar plate was prepared by spreading the microbe, then paper disks of antimicrobials were added. This method is mostly used to determine the best antibiotic against a new or drug-resistant pathogen. The zone of inhibitions was measured with the help of a transparent scale in millimeters [22]. The bacterial strains tested by this procedure were *Bacillus subtilis* and *Bacillus cereus*. The fungal strains evaluated against tea tree oil were White rot and *Aspergillus flavus*.

## *2.6. Statistical Analysis*

The lab experiments were conducted through a completely randomized design (CRD). Data were analyzed using *Statistix 8.1* software (https://www.statistix.com/ (accessed on: 9 February 2020)), and mean analysis was conducted through Tukey's honestly significant difference test on a 0.05 level of probability [23].

## **3. Results and Discussion**

## *3.1. Essential Oil Yield*

Yield of essential oil via the hydrodistillation method was about 0.2%, as shown in Table 1. The volume of extracted oil could vary based on the agroclimatic and geographical conditions of the area. A study by Siddique et al. [24] reported the %age yield of *Melaleuca bracteata* to range at 0.14 ± 0.01% in Pakistan. The percentage yield of extracted oil of each hydrodistillation batch is shown in Table 1.

**Table 1.** %age Yield of tea tree oil from *M. bracteata* leaves.


## *3.2. GCMS Analysis*

The GCMS analysis of tea tree oil from *M. bracteata* showed the 7 peaks as shown in Figure 1. Table 2 indicates that methyl eugenol is the supreme compound of tea tree oil extracted from *Melaleuca bracteata* species. It represents 96% of the total, and the other six contents were found as just trace elements. The other elements in order of percentage are p-Mentha-2,8-diene-1-ol, Methyl cinnamate, Germacrene D, Terpinolene, Linalool, and alpha-phellandrene. Previous literature has also proclaimed methyl eugenol as the major component of *M. bracteata* oil, with 86.5% of the total volume [24,25]. The latter also indicated methyl cinnamate as the second highest compound, representing 11% of the total. However, another team of scientists documented the percentage of methyl eugenol to be greater than 95% in the tea tree oil of *bracteata*, which is quite related to the present finding [26].

**Table 2.** GC–MS analysis of tea tree oil from *Melaleuca bracteata*.


**Figure 1.** Chromatogram of TTO by GCMS analysis. **Figure 1.** Chromatogram of TTO by GCMS analysis.

**Table 2.** GC–MS analysis of tea tree oil from *Melaleuca bracteata*. **Peak No. Identified Compounds Retention Time (min) % of Total 1** alpha-phellendrene 8.568 0.49% **2** p-Mentha-2,8-diene-1-ol 9.278 0.92% **3** Terpinolene 11.071 0.64% **4** Linalool 11.595 0.50% **5** Methyl cinnamate 19.359 0.77% **6** Methyl eugenol 19.897 96.02% **7** Germacrene D 20.663 0.67% Methyl eugenol (ME) belongs to phenylpropanoids, and its occurrence is found in 450 species of plants. In nature, ME is known for plant chemical defense as an antifungal, antibacterial, antinematode and has toxic activities against insect pathogens [27]. The oil from *Echinophora sibthorpiana* (Apiaceae) also has methyl eugenol. Kivanc [28] found that Methyl eugenol (ME) belongs to phenylpropanoids, and its occurrence is found in 450 species of plants. In nature, ME is known for plant chemical defense as an antifungal, antibacterial, antinematode and has toxic activities against insect pathogens [27]. The oil from *Echinophora sibthorpiana* (Apiaceae) also has methyl eugenol. Kivanc [28] found that its oil (∼0.1%) and ME at 0.05-0.1 % have negative effects on the growth of fungi and bacteria. Later, Kivanc and Akgul [29] identified that eugenol methyl ether at 1000 ppm also detained mycelium and spore formation of *Aspergillus* and *Penicillium* strains. It has also been found to inhibit fungal infestation in peanuts by Sudhakar et al. [30]. ME has been shown to act as a floral synomone in the coevolution of orchid species in the genus *Bulbophyllum* with fruit flies. It was recognized as the best insect repellent and most effective in knockdown and killing effects among the major constituents of EOs [31]. The importance of methyl eugenol can also be inferred from the findings of Rossi et al. [32] that volatile oil from carrot was successfully evaluated to inhibit the growth of *Campylobacter jejuni*, a major gastroenteritis-causing bacterium across the globe. Methyl eugenol, a component of carrot oil, was also found effective separately at a minimum inhibitory concentration of 250 µg/mL.

## its oil (∼0.1%) and ME at 0.05-0.1 % have negative effects on the growth of fungi and bacteria. Later, Kivanc and Akgul [29] identified that eugenol methyl ether at 1000 ppm also *3.3. Biochemical Assays*

detained mycelium and spore formation of *Aspergillus* and *Penicillium* strains. It has also been found to inhibit fungal infestation in peanuts by Sudhakar et al. [30]. ME has been shown to act as a floral synomone in the coevolution of orchid species in the genus The oil extracted from *Melaleuca* species exhibited competent antioxidant properties. The result is supported by Table 3, which shows the antioxidant values of *M. bracteata* oil along with the mean and standard error values.

*Bulbophyllum* with fruit flies. It was recognized as the best insect repellent and most effec-

tive in knockdown and killing effects among the major constituents of EOs [31]. The im-**Table 3.** Average antioxidant values of TTO from *Melaleuca bracteata*.


TFC: total flavonoid contents, TPC: total phenolic compounds, RPA: reducing power ability, DPPH: 2,2-diphenyl-1-picrylhydrazyl.

## 3.3.1. DPPH Scavenging Activity

In Table 3, the average scavenging activity of tea tree oil is displayed. DPPH values for *M. bracteata* oil ranged at 86.848 ± 2.43 mg/mL from the three replicates.

## 3.3.2. Total Flavonoid Contents

From Table 3, it is inferred that TTO has a relatively higher amount of flavonoids, i.e., 951 ± 4.50 mg/mL. Flavonoids are produced in plants as a response to oxidative pressure and photodamage via secondary metabolic processes [33,34].

## 3.3.3. Total Phenolic Compounds of Tea Tree Oil

The TPC of tea tree oil were exploited with respect to the gallic acid standard. The slope expression obtained from the standard curve was used to calculate the concentration of phenolics as a gallic acid equivalent (GAE) in mg/g of plant dry weight basis. Table 3 represents a higher concentration of phenolics, i.e., 110 mg/g of the dry weight of *Melaleuca bracteata* plant.

## 3.3.4. Reducing Power Ability

Antioxidants reduce Fe3+ to Fe<sup>2</sup> , i.e., ferricyanide complex to the ferrous form. This transformation is observed at 700 nm [35]. The reducing ability is amplified with increased absorbance [36]. RPA values were evaluated as equivalent to the standard. About 308 mg/g of the dry weight of the plant was found in *Melaleuca bracteata* oil. A study by Hou et al. [37] provided evidence regarding the antioxidant potential of *Melaleuca bracteata*. When extraction was done with conventional methods, the total phenolic compounds found were 88.6 ± 1.3 mg, and the total flavonoids were 19.4 ± 0.2 mg. The DPPH scavenging activity was high at 86 ± 0.3%. After conditions were optimized, all the values of antioxidants were heightened and found to be at their peak values of 98.7 ± 1.2 mg, 21.6 ± 0.3 mg, and 94.7 ± 0.8% for TPC, TFC, and DPPH, respectively. Siddique et al. [24] also reported that scavenging activity in *M. bracteata* ranged between 35.3 and 89.2 ± 0.4%. This potential increased with the increase of concentration of oil from 20 to 100 µg/mL.

Tea tree oil is an effective antioxidant agent as its commercial species, i.e., *Melaleuca alternifolia*, has also been observed to be enriched with this potential by several reports. One such is the study of Noumi et al. [38] in which DPPH scavenging activity was recorded at 12.5 µg/mL expressed as IC50 values (the concentration of antioxidant at which 50% of the reaction was inhibited). The power of TTO as an antioxidant was also found to be better than that of common synthetic antioxidant butylated hydroxytoluene (BHT) at 30 mM. Previous researchers proposed TTO as a good alternative to maintain the oxidative stability of the food matrix. Additionally, when compared with other natural oxidants such as vitamin C, vitamin E, and quercetin, *Melaleuca alternifolia* oil expressed strong free radical scavenging properties and inhibited lipid peroxidation [39]. This is because of the presence of phenols in this oil, which reduce the aerobic oxidation of organic matter [40]. Hence, it is concluded that the present detection of antioxidants in tea tree oil from *Melaleuca bracteata* is in accordance with previous reports of several scientific studies, and this species could also be explored as an antioxidative essential oil along with its patent plant species (*Melaleuca alternifolia*).

## *3.4. Antimicrobial Properties*

*Bacillus subtilis* and *Bacillus cereus* were tested against tea tree oil from *Melaleuca bracteata*. The zone of inhibition was measured in millimeters for all treatments. It is the measure of a clear region with no microbial growth around the paper disc on the agar surface. The clear area represents the resistance or the effective inhibition of microbial growth by antimicrobial agent. The larger the region is, the greater the antimicrobial effect of the agent. the effect of tea tree oil on the growth of *Bacillus subtilis* and *Bacillus cereus* is elaborated on in Figure 2. The zone of inhibition obtained by the application of *M. bracteata*

*Bacillus subtilis* and *Bacillus cereus* were tested against tea tree oil from *Melaleuca bracteata*. The zone of inhibition was measured in millimeters for all treatments. It is the measure of a clear region with no microbial growth around the paper disc on the agar surface. The clear area represents the resistance or the effective inhibition of microbial growth by antimicrobial agent. The larger the region is, the greater the antimicrobial effect of the agent. the effect of tea tree oil on the growth of *Bacillus subtilis* and *Bacillus cereus* is elaborated on in Figure 2. The zone of inhibition obtained by the application of *M. bracteata* oil

be better than that of common synthetic antioxidant butylated hydroxytoluene (BHT) at 30 mM. Previous researchers proposed TTO as a good alternative to maintain the oxidative stability of the food matrix. Additionally, when compared with other natural oxidants such as vitamin C, vitamin E, and quercetin, *Melaleuca alternifolia* oil expressed strong free radical scavenging properties and inhibited lipid peroxidation [39]. This is because of the presence of phenols in this oil, which reduce the aerobic oxidation of organic matter [40]. Hence, it is concluded that the present detection of antioxidants in tea tree oil from *Melaleuca bracteata* is in accordance with previous reports of several scientific studies, and this species could also be explored as an antioxidative essential oil along with its patent plant

*Sustainability* **2021**, *13*, x FOR PEER REVIEW 7 of 11

species (*Melaleuca alternifolia*).

*2.4. Antimicrobial Properties*

**Figure 2.** Zone of inhibition calculated from *M. bracteata* oil sample and standard against *Bacillus* species.

**Figure 2.** Zone of inhibition calculated from *M. bracteata* oil sample and standard against *Bacillus* species. *Melaleuca* species possess good antibacterial effects against *Bacillus* strains. Previously, in a finding by Goswami et al. [41], the zone of inhibition for *B. subtilis* by *M. bracteata* oil was measured to be 6–7 mm, which showed a moderate antifungal effect of essential oil on the bacterial strain. The growth of *B. subtilis* subsp. *Spizizenii* was also inhibited by this oil species at 250 µg/mL up to 44 mm zone of inhibition [24]. EO from this species was also found to be effective against *Staphylococcus aureus* strains, *S. epidermidis*, *Klebsiella pneumonia*, and *Streptococcus mutans* when tested by Goswami et al. (2017) [41]. The MIC of *M. alternifolia* oil counted for *Bacillus cereus* was 0.3% (*v/v*) by Griffin et al. [42]. *Melaleuca* species possess good antibacterial effects against *Bacillus* strains. Previously, in a finding by Goswami et al. [41], the zone of inhibition for *B. subtilis* by *M. bracteata* oil was measured to be 6–7 mm, which showed a moderate antifungal effect of essential oil on the bacterial strain. The growth of *B. subtilis* subsp. *Spizizenii* was also inhibited by this oil species at 250 µg/mL up to 44 mm zone of inhibition [24]. EO from this species was also found to be effective against *Staphylococcus aureus* strains, *S. epidermidis*, *Klebsiella pneumonia*, and *Streptococcus mutans* when tested by Goswami et al. (2017) [41]. The MIC of *M. alternifolia* oil counted for *Bacillus cereus* was 0.3% (*v*/*v*) by Griffin et al. [42]. Previous literature reported that TTO has a bactericidal activity against *Staphylococcus* species, *Lactobacillus*, *Actinomyces viscosus*, and *E. coli* [43–46]. The antimicrobial traits of *M. alternifolia* have been employed to its terpinen-4-ol compound [47,48]. Moreover, terpinen-4-ol and methyl eugenol from both the species have been recognized to have efficient antibacterial, antifungal, and antinematode properties [14,27].

## *3.5. Antifungal Properties*

The potential of fungicidal effects of tea tree oil was evaluated by measuring its zone of inhibition for *Aspergillus flavus* and *Phanerochaete chrysosporium* (White rot). From Figure 3, it is inferred that the growth of White rot fungi and *A. flavus* was restricted up to 11.67 mm and 10.5 mm by the application of oil from *M. bracteata* species, respectively.

Shin [49] has documented the antifungal aspects of commercial TTO from *M. alternifolia* against *Aspergillus flavus*. The concentration required for the effective control on fungus ranged between 0.3 and 0.7 (% volume by volume). The oil has been found to be effective against other *Aspergillus* species such as *A. niger* with a minimum inhibitory concentration of 0.016 to 0.4 (%volume/volume). The effective antifungal agent is terpinen-4-ol in this oil. Moreover, the zone of inhibition formed by methyl eugenol from the oil of another *Melaleuca* species was 24.3 ± 0.3 mm for pure oil and 46 ± 0.6 mm at 100 µg/mL of oil against *Fusarium oxysporum* [50]. Homeyer et al. [51] observed that within 15 min of exposure, TTO reduced the log of *Fusarium oxysporum* up to -4 as compared to control treatment.

Previous literature reported that TTO has a bactericidal activity against *Staphylococcus* species, *Lactobacillus*, *Actinomyces viscosus*, and *E. coli* [43–46]. The antimicrobial traits of *M. alternifolia* have been employed to its terpinen-4-ol compound [47,48]. Moreover, terpinen-4-ol and methyl eugenol from both the species have been recognized to have efficient an-

The potential of fungicidal effects of tea tree oil was evaluated by measuring its zone of inhibition for *Aspergillus flavus* and *Phanerochaete chrysosporium* (White rot). From Figure 3, it is inferred that the growth of White rot fungi and *A. flavus* was restricted up to 11.67

Shin [49] has documented the antifungal aspects of commercial TTO from *M. alternifolia* against *Aspergillus flavus*. The concentration required for the effective control on fungus ranged between 0.3 and 0.7 (% volume by volume). The oil has been found to be effective against other *Aspergillus* species such as *A. niger* with a minimum inhibitory concentration of 0.016 to 0.4 (%volume/volume). The effective antifungal agent is terpinen-4 ol in this oil. Moreover, the zone of inhibition formed by methyl eugenol from the oil of another *Melaleuca* species was 24.3 ± 0.3 mm for pure oil and 46 ± 0.6 mm at 100 µg/mL of oil against *Fusarium oxysporum* [50]. Homeyer et al. [51] observed that within 15 min of exposure, TTO reduced the log of *Fusarium oxysporum* up to -4 as compared to control

mm and 10.5 mm by the application of oil from *M. bracteata* species, respectively.

tibacterial, antifungal, and antinematode properties [14,27].

*2.5. Antifungal Properties*

treatment.

**Figure 3.** Zone of inhibition calculated from *M. bracteata* oil against White Rot and *Aspergillus fla-***Figure 3.** Zone of inhibition calculated from *M. bracteata* oil against White Rot and *Aspergillus flavus*.

## *vus*. **4. Conclusions**

**6. Conclusions** The present study aimed at the extraction of essential oil from *Melaleuca bracteata* to explore its antioxidant and antimicrobial properties. It has been found from the results that the yield percentage for essential oil is about 2% through the hydrodistillation method, and it is enriched with eugenol methyl ether. A high antioxidant activity is observed when measured on scales for total flavonoids, total phenolics, reducing power ability, and DPPH. The laboratory application of TTO on microbes reduces their activity. Thus, tea tree oil from *Melaleuca bracteata* species has significant potential as an antioxidant and antimicrobial to be used as a natural microbicide in the food and agriculture industries. The *Bracteata* species is well adapted to Pakistan and specifically the Punjab region's The present study aimed at the extraction of essential oil from *Melaleuca bracteata* to explore its antioxidant and antimicrobial properties. It has been found from the results that the yield percentage for essential oil is about 2% through the hydrodistillation method, and it is enriched with eugenol methyl ether. A high antioxidant activity is observed when measured on scales for total flavonoids, total phenolics, reducing power ability, and DPPH. The laboratory application of TTO on microbes reduces their activity. Thus, tea tree oil from *Melaleuca bracteata* species has significant potential as an antioxidant and antimicrobial to be used as a natural microbicide in the food and agriculture industries. The *Bracteata* species is well adapted to Pakistan and specifically the Punjab region's climate. Thus, it must be exploited by the local industry to produce raw and refined oil products to commercialize its use in horticulture, pharmaceuticals, and cosmetics.

**Author Contributions:** Conceptualization, A.Y.; methodology, M.Y.; software, M.Y.; validation, T.J., R.S., F.R. and A.E.S.; formal analysis, M.Y.; investigation, M.Y.; resources, A.Y.; data curation, T.J., R.S. and A.E.S.; writing—original draft preparation, M.Y.; writing—review and editing, M.S., M.B., P.O., A.Y., T.J., H.A.N. and A.E.S.; supervision, A.Y.; project administration, A.Y.; funding acquisition, S.H., A.Y. and A.E.S. All authors have read and agreed to the published version of the manuscript.

**Funding:** The authors was funded by Taif University Researchers Supporting Project grant number (TURSP-2020/142), Saudi Arabia.

**Acknowledgments:** The authors are highly grateful to Taif University Researchers Supporting Project Number (TURSP-2020/142), Saudi Arabia. The authors thank Muhammad Zaryab Khalid for critically reviewing the manuscript.

**Conflicts of Interest:** The authors declare no conflict of interest.

## **References**


## *Article*

## **A Comprehensive Appraisal of the Wild Food Plants and Food System of Tribal Cultures in the Hindu Kush Mountain Range; a Way Forward for Balancing Human Nutrition and Food Security †**

**Abdullah Abdullah <sup>1</sup> , Shujaul Mulk Khan 1,2,\*, Andrea Pieroni 3,4 , Aminul Haq 5,6 , Zahoor Ul Haq 1,7 , Zeeshan Ahmad <sup>1</sup> , Shazia Sakhi <sup>8</sup> , Abeer Hashem 9,10, Al-Bandari Fahad Al-Arjani <sup>9</sup> , Abdulaziz A. Alqarawi <sup>11</sup> and Elsayed Fathi Abd\_Allah <sup>11</sup>**


**Abstract:** The tribal belt of the Hindu Kush mountains is famous for its unique culture, ethnography, wild food plants, food systems, and traditional knowledge. People in this region gather wild plants and plant parts using them directly or in traditional cuisine, or sell them in local markets. However, there is a huge lack of documentation of the food system, particularly that related to wild food plants (WFP). In the current study, we focus on the uses and contributions of WFPs in the traditional tribal food system of the Hindu Kush valleys along the Pakistan–Afghanistan border. Ethnobotanical data were gathered through questionnaire surveys of 84 informants, including 69 men and 15 women, belonging to 21 different villages of the chosen area. In tribal societies men and women rarely mix and thus very few women took part in the surveys. We documented 63 WFP species belonging to 34 botanical families, of which 27 were used as vegetables, 24 as fruits, six in different kinds of chutneys (starters), and six as fresh food species. Fruits were the most used part (41%), followed by leaves (24%), aerial parts (24%), seeds (7%), stems (3%), and young inflorescences (1%). The reported uses of *Carthamus oxyacantha*, *Pinus roxburghii* seeds, and *Marsilea quadrifolia* leaves are novel for the gastronomy of Pakistan. The results reveal that WFPs provide a significant contribution to local food systems and play a role in addressing human nutritional needs, which are usually not met through farming practices. The tribal peoples of the Hindu Kush use WFPs for their nutritional value, but also as a cultural practice—an inseparable component of the tribal community's lifestyle. This important traditional knowledge about the gathering and consumption of WFPs, however, is eroding at an alarming rate among younger generations due to the introduction of fast-food, modernization, and globalization. Therefore, appropriate strategies are imperative not only to safeguard traditional

**Citation:** Abdullah, A.; Khan, S.M.; Pieroni, A.; Haq, A.; Haq, Z.U.; Ahmad, Z.; Sakhi, S.; Hashem, A.; Al-Arjani, A.-B.F.; Alqarawi, A.A.; et al. A Comprehensive Appraisal of the Wild Food Plants and Food System of Tribal Cultures in the Hindu Kush Mountain Range; a Way Forward for Balancing Human Nutrition and Food Security. *Sustainability* **2021**, *13*, 5258. https:// doi.org/10.3390/su13095258

Academic Editor: António Raposo

Received: 17 March 2021 Accepted: 27 April 2021 Published: 8 May 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

plants and food knowledge and practices, as well as the cultural heritage attached to them, but also to foster food security and thus public healthcare via local wild foods in the region.

**Keywords:** gastronomy; livelihood; public healthcare; traditional knowledge; wild food plants; valleys of the Hindu Kush mountain range

## **1. Introduction**

Wild food plants (WFP) have played a crucial role in food systems since the beginning of human civilization [1]. Wild food plants refer to the non-cultivated plant species that are gathered or harvested by local communities from their surrounding ecosystems for utilization in their food systems [2]. Ecosystems in the tribal belt are diverse, being composed of plains, forests, and pastures, providing habitat to various kinds of wild fruits and vegetables [3]. Not only are these WFPs important from a nutritional point of view, but they have important medicinal value [4–7]; various WFPs have been evaluated for their pharmacological properties in many other parts of the world [5,8,9]. Recently, a number of WFPs have been identified as "functional foods" due to their nutritional value. They provide a healthy diet and can contribute to the prevention of some illnesses [10,11]. According to researchers, most WFPs are a rich source of bioactive compounds, such as essential fatty acids, vitamins, and complex sugars [11–13], and can contribute significantly to the alleviation of malnutrition [3]. Their importance in income generation, poverty reduction, nutritional balance, food security, and agricultural diversification has been emphasized in several studies [14,15]. One of the major contributions of WFPs is their use in food security by providing alternative food sources and food diversity in traditional communities [16]. Wild food plants play a crucial role in human survival during times of famine [17] and food shortages; recent examples include their use during the COVID-19 pandemic crisis, and the geopolitical and tribal unrest around the world.

Very few studies have been conducted to document food plants in Pakistan and the area under consideration in this article has never been studied before. Such comparisons are imperative as the Hindu Kush receives less summer precipitation than the Himalayas which gives rise to different kinds of WFPs.

Pakistan is a lower-middle-income country and the 6th most populous in the world [18]. It experiences all four seasons and has a wide diversity of natural resources, especially plants [19], and it still ranks as the 11th most food insecure country in the world [20]. About 60% of the people in the country are food insecure [18]. Food insecurity increases in the tribal belt of the country due to conflicts, distance from cities, and the prevailing dry climate. The Pakistani–Afghan border in the Hindu Kush mountain range has always been the center of tension due to its geopolitical position. Man-made catastrophes, the dramatic increase in the human population, limited access to food, and local livelihood strategies are the other major reasons for food insecurity and poverty in the tribal belt [21]. Wild food plants are an important resource for poor local communities and can be of tremendous help in alleviating hunger and malnutrition if properly managed and utilized. To date, little research has been conducted on the wild vegetable and fruit species used in the food systems of Pakistan in general and Pakistani–Afghan border regions in the Hindu Kush mountain range in particular [16,19,22–28]. The book by Abbasi et al. [29] on the ethnobotanical and nutraceutical aspects of the wild edible vegetables of the Lesser Himalayas is one of the major contributions in this regard.

## *Tribal Culture*

The people of the region practice and enjoy a centuries-old culture known as Pashtunwali or Pukhtunwali. Pashtunwali is a very common cultural heritage of the tribal communities of Pathans living along the Pakistani–Afghan border in the Hindu Kush mountain range. It is a way of life or legal social code of conduct that governs their lives. According to [30] it is a 2000–3000 years old tradition that remains the central pillar of tribal societies. In this era of modernization, Pashtunwali is perceived as culturally idiosyncratic, significantly influencing perceptions, behaviors, and thoughts in everyday life [31,32]. Code of honor (Nang), hospitality (Melmastia), and revenge (Badal) are the three core pillars of Pashtunwali [33]. The code of honor, or self-respect and dignity, is a key component and characteristic of Pashtun society. Melmastia is a sum of conventions and an immediate reward from local societies regarding local cultural values, while Badal is like a debt which ends with revenge. These three cultural customs are the sum of tribal communal expectations from members of their society, as well as from outsiders [34]. These customs are related to WFPs in that Pathans offer food to each other and safeguard their natural resources. Although the tribal societies living along the Pak–Afghan border in the Hindu Kush mountain range are well known for their unique traditional food system, it has not been explored by researchers to document the valuable traditional knowledge about WFPs. The Bajaur area is the most populous, climatically varied, and ethnographically rich in the tribal belt of the Hindu Kush mountain range and it exemplifies the culture and traditional knowledge of the entire Pak–Afghan border and thus it was selected for the current study. It is a region composed of relatively dry valleys in the Hindu Kush mountain range. As for the food system in the tribal belt, especially in the area under investigation, it is worth mentioning that people are financially poor and most of the time they use WFPs. They eat at home most of the time, and the practice of hoteling and eating out in restaurants or other food points is uncommon. All of this strengthens the ties between people and wild food resources, including plants. Climate is one of the main drivers affecting all facets of life including wild food and food systems, with considerable variation in the different administrative units (tehsil) of the study area, i.e., Barang in the south, Nawagai and Chamrkand in the southwest, and Mamund, Salarzai, and Utmankhel tehsils in the north and northeast. Local communities collect wild vegetables and fruits from side valleys, mountains, and cultivated fields. They know the best and most suitable gathering seasons of wild vegetables using traditional ecological knowledge (TEK). The proper time and method of collecting various WFPs is crucial in terms of nutritional value. Women usually cook them in traditional tribal styles. The demand for wild vegetables and fruits increases in different seasons when cultivated fruits and vegetables are not available. Bearing in mind the importance of WFPs in the food system of tribal communities, the current study was designed to (1) document the diversity of WFPs gathered and consumed by tribal communities; (2) assess traditional knowledge about WFPs, their role and contribution in nutrition and the traditional tribal food system; and (3) compare the Hindu Kush valleys with other areas in regard to wild food plants.

## **2. Methodology**

## *2.1. Study Area*

The study region is a tribal area that remained part of the Federally Administered Tribal Areas (FATA) up to 2018. The tribal areas of Pakistan form a long narrow belt along the historical Durand Line (Pakistani–Afghan border). Federally Administered Tribal Areas consisted of six frontier regions (FR), i.e., Kohat, Bannu, Tank, Dera Ismail Khan, Peshawar, and Lakki Marwat, and seven agencies, namely Bajaur, Mohmand, Khyber, Orakzai, Kurram, North Waziristan, and South Waziristan. According to certain writers [35], in the mid-19th century the British faced difficulties and challenges in tax collection and effective control over the tribal belt and, thus, devised a legal and administrative system known as the Criminal Tribes Act of 1871. The law was "designed for the registration, surveillance, and control of certain tribes" [36] that was later enacted under the frontier crimes regulation (FCR). This remained in effect in the tribal belt until 2018. In that year, the government of Pakistan merged the whole tribal belt of FATA into the mainstream province of Khyber Pakhtunkhwa in the hope of promoting harmony and development following the drafting of the constitution of Pakistan. The area selected for the current study is a mountainous territory, with an area of 1296 km<sup>2</sup> , situated in northwestern Pakistan, located at 34–51◦ N

latitude and 71–31◦ E longitude [37]. It shares a 52-km-long border with Afghanistan west of the Panjkora River, possessing three important passes, i.e., Nawa Pass, Ghakhi Pass, and Letaisar [38]. These three passes play a significant role in trade and the sharing of traditional knowledge between both countries. The research area is mainly inhabited by the Utmankhel and Tarkanri tribes, which speak different dialects of Pashto. The Utmankhel tribe resides in the southeast, bordering the districts of Malakand, while the Tarkanri tribe lives in the northwest on the Pakistani–Afghan border of the Hindu Kush mountains. The Utmankhel tribe is divided into the sub-tribes of Alizai, Shamozai, Aseel, Gharshamozai, and Mandal [39]. The Tarkanri tribe comprises the sub-tribes Mamund, Salarzai, Safi, Khalji, and other sub-ethnic groups who live in the former state of Khar [40]. According to Sher 2019, [40], the Utmankhels are descendants of the Kirlan tribe of Waziristan who migrated to the valleys of Arang and Barang in Bajaur during the 12th century. On the other hand, the Tarkanri people are descendants of Afghani Pashtuns. Before the events of 11 September 2001, people of both areas used to visit the two countries easily for various purposes. In short, Afghan culture has had a great impact on the Tarkanri people. Most of the people of the district live in rural parts of the area (Figure 1). i.e., Nawa Pass, Ghakhi Pass, and Letaisar [38]. These three passes play a significant role in trade and the sharing of traditional knowledge between both countries. The research area is mainly inhabited by the Utmankhel and Tarkanri tribes, which speak different dialects of Pashto. The Utmankhel tribe resides in the southeast, bordering the districts of Malakand, while the Tarkanri tribe lives in the northwest on the Pakistani–Afghan border of the Hindu Kush mountains. The Utmankhel tribe is divided into the sub-tribes of Alizai, Shamozai, Aseel, Gharshamozai, and Mandal [39]. The Tarkanri tribe comprises the sub-tribes Mamund, Salarzai, Safi, Khalji, and other sub-ethnic groups who live in the former state of Khar [40]. According to Sher 2019, [40], the Utmankhels are descendants of the Kirlan tribe of Waziristan who migrated to the valleys of Arang and Barang in Bajaur during the 12th century. On the other hand, the Tarkanri people are descendants of Afghani Pashtuns. Before the events of 11 September 2001, people of both areas used to visit the two countries easily for various purposes. In short, Afghan culture has had a great impact on the Tarkanri people. Most of the people of the district live in rural parts of the area (Figure 1).

*Sustainability* **2021**, *13*, x FOR PEER REVIEW 4 of 22

study is a mountainous territory, with an area of 1296 km<sup>2</sup>

surveillance, and control of certain tribes" [36] that was later enacted under the frontier crimes regulation (FCR). This remained in effect in the tribal belt until 2018. In that year, the government of Pakistan merged the whole tribal belt of FATA into the mainstream province of Khyber Pakhtunkhwa in the hope of promoting harmony and development following the drafting of the constitution of Pakistan. The area selected for the current

kistan, located at 34–51° N latitude and 71–31° E longitude [37]. It shares a 52-km-long border with Afghanistan west of the Panjkora River, possessing three important passes,

, situated in northwestern Pa-

**Figure 1.** Map of the study area, showing the 21 studied villages. **Figure 1.** Map of the study area, showing the 21 studied villages.

The district has many natural resources, such as different kinds of stones, including marble, and other minerals. Local communities are involved in the extraction of marble, limestone, manganese, chromites, and talc, among other mineral resources. Geographically, the study area lies at the end of the Hindu Kush mountain range which creates considerable disparity and uncertainty in the pattern of monsoon rains from year to year. The climate of the district varies from extremely hot to bitter cold while traveling from the The district has many natural resources, such as different kinds of stones, including marble, and other minerals. Local communities are involved in the extraction of marble, limestone, manganese, chromites, and talc, among other mineral resources. Geographically, the study area lies at the end of the Hindu Kush mountain range which creates considerable disparity and uncertainty in the pattern of monsoon rains from year to year. The climate of the district varies from extremely hot to bitter cold while traveling from the southeast to the northwest. The weather remains pleasant for most of the summer, but the temperature drops below freezing during winter. The area receives about 800 mm of annual rainfall. Variation in the climatic conditions of the area leads to a considerable diversity of plants and, thus, WFPs [38]. The people are mostly farmers, and a few others are teachers, drivers, cobblers, and businessmen. A considerable number of local people work in other parts of the country or abroad and depend on remittances for their livelihood. The local people have been collecting WFPs for nutritional and ethnomedicinal purposes for countless generations [41,42].

## *2.2. Data Collection*

In the current study, 21 different villages were selected for interviews and gathering data on WFPs using a random table method where 3 villages from each of the tehsils (administrative units) of the study area were selected for questionnaire data collection (Table 1). Two visits to each village were arranged for this purpose between March and November 2016. Interviews were conducted following the code of ethics of both the International Society of Ethnobiology and the American Anthropological Association [42]. Knowledgeable informants were selected through the snowball sampling method [43–45]. A total of 84 informants were interviewed (four from each village) including 69 men and 15 women. The age of the informants ranged from 17 to 85 years, while education level ranged between illiterate to university degree (Table 2). The interviews were conducted in the Pashto language. Participants were asked about the local names, parts used, collection times, recipes and modes of utilization, uses in other dishes, and the sale or marketing of the WFPs species they gathered in the past and still gather today [45]. The first author comes from the study area and has family ties with a few of the villages studied, which helped considerably in gathering data about the uses of WFPs and selecting the informants. Each interviewee was verbally informed that the data gathered would be published and shared globally, which could help them by garnering international interest in their traditional knowledge and use of food plants, as well as the plants they associate with.


**Table 1.** The geographical position of the 21 studied villages in the seven tehsils of the study area.


**Table 2.** Ages and literacy levels of the informants in the study area.

The interviewees were asked about the settlement they live in, as well as their age, place of birth, and occupation. They were also asked the following questions: (1) What wild vegetable species do you collect? (2) What wild fruit species do you collect and use? (3) What species do you use in sauces, salads, and chutneys (4) What WFPs do you use as raw foods (5) What wild herbal drinks do you consume? (6) What wild edible seeds do you collect and use? (7) Do you collect WFPs for marketing purposes? (8) How frequently do you use the plant in a season? (9) What are the names of WFPs in your language? (10) Which parts do you use? (11) What recipes you prepare? (12) What is the best season for collecting? (13) What traditional dishes are made with these WFPs? (14) Did you use any WFPs in the past which you did not mention previously? Thematic analysis was carried out on the basis of our interviews and observations. All the data based on these questions were arranged in tabulated form. After interviewing four people in each village, a short field excursion with the interviewees was arranged to collect all available WFPs.

Collected plant specimens were identified with the help of the Flora of Pakistan [46] and the Wild Plants of the Swat Valley [47], and then plants names were cross-checked against the Plant List database [48]. Plants uses were also compared with previous studies [16,19,22–29,47].

Secondary data were also gathered by collecting and reading local as well as scientific literature about the local names, cuisines, and tribal food systems for finalizing the article in hand.

## *2.3. Data Analysis*

Data on WFPs, i.e., botanical name, local name, plant family, used parts, growth habit, mode of utilization, marketing, use reports, relative frequency of citation, and previous literature that reported their use, were gathered via a questionnaire and literature survey.

## 2.3.1. Relative Frequency of Citation (RFC)

The data on wild food plants, collected during the questionnaire survey, were also analyzed qualitatively via the relative frequency of citation (RFC) to highlight the local significance of each WFP species in the region following [5,49]:

$$\text{RFC} = \frac{\text{FC}}{\text{N}} (0 < \text{RFC} < 1) \tag{1}$$

where FC denotes the number of informants mentioning a specific WFP species, while N is the total number of informants interviewed during the survey.

## 2.3.2. Classification of WFPs Based on Mode of Consumption

All WFPs were grouped into different food categories on the basis of consumption mode following Thakur et al 2017 [3]. Those plant species cooked as food were categorized as vegetables, for example, *Caralluma tuberculata*, *Malva neglecta*, and *Digera muricata*. Species that were consumed without cooking were classified in the category of fruits dry or fresh, i.e., *Ficus carica*, *Morus alba*, and *Celtis caucasica*. Another category consisted of plants used, together with salt and spices, to make sauces, salads, and chutneys, for example, *Mentha longifolia*, *Mentha royleana*, and *Zanthoxylum armatum*. *Salvia moorcraftiana*, *Saccharum bengalensis*, and *Silene conoidea* were considered as raw food species, or those plants, other than fruits, whose parts were consumed raw without cooking.

## **3. Results**

## *3.1. Tribal Food System and WFPs*

Tribal people have a unique and simple food system primarily based on WFPs, dairy products, and cereal crops they cultivate in their fields. Almost every family in the study area raises domesticated animals (sheep, goat, cow, or buffalo) to meet their dairy requirements [50]. They customarily collect WFPs while grazing their animals. They usually have a piece of land where they cultivate local vegetables and crops. They prepare and consume various kinds of cultural foods from these three abovementioned sources. Some of the well-known simple foods of this tribal region include Jawaro Neeny (roasted corn seeds), Ghanamo Neeny (wheat grains are roasted and then a little brown sugar is added to form small balls), Dhal (ground wheat grains are boiled with brown sugar and butter oil), Gongrhi (boiled wheat grains), Shedano rotay (wheat flour is mixed with cow oil and *Carthamus oxyacantha* seeds to make a bread that is baked in a tandoor), and Kakori (wheat flour is mixed with peanuts, resin, walnuts, coconut, and sugar to make a special type of bread). Kakori is prepared mostly when an infant start walking for the first time. Ghunzakhi (wheat flour mixed with vegetable oil, sugar, and *Cuminum cyminm* seeds) is a special type of sweet prepared and gifted mostly by women when they move from their mother's house to their in-law's house. Elderly individuals cook lentils and pulses in a unique traditional style: when the dish is prepared a cup or more of butter oil is added. This dish is offered at special cultural gatherings and ceremonies, such as Ashar. Ashar is a centuries-old tradition of the Pathans in which members of a local community gather to help each other in their work without consideration of compensation, monetary or otherwise. Unfortunately, today, this important cultural tradition is on the verge of extinction due to urbanization, modernization, and globalization. These traditional foods are considered an important feature and significantly contribute to the tribal food system. Tribal people of the study area living in valleys and the foothills collect WFPs to fulfill their food requirements. They are wholly or partially dependent on WFPs resources.

## *3.2. WFPs and Their Taxonomic Categorization*

Overall, 63 folk taxa belonging to 34 botanical families were recorded. A high proportion of these WFPs belong to the Amaranthaceae and Leguminosae families (five species each), followed by Rhamnaceae, Rosaceae, Lamiaceae, and Polygonaceae (four species each), and then Brassicaceae and Moraceae (three species each). The most dominant WFP categories included vegetables (27 species), fruits (24 species), chutneys and sauces (7 species), and raw food species (six species). Field observations showed that the vegetable category was mainly composed of weeds locally called *Gayyah* (unwanted plant species

occurring in or along crop fields). The traditional communities gathered these species for their own food uses, as well as that of their animals.

## *3.3. Traditional Knowledge Related to Gathering WFPs*

Wild food plants were reported to be gathered from diverse localities, including agricultural fields, foothills, and forests. The diversity and availability of plants vary with seasonal changes. Local people possess traditional knowledge regarding the seasonal availability and time of collection of WFPs [51]. Spring (March–May) and Summer (June– October) are the peak seasons for WFPs collection in the study region. Most WFPs are not available from November to February due to the cold temperatures [24]. Leaves and pot herbs grow well in April and August, while the best time for fruits ranges from June to October. Different vegetable species remain available for 2 to 9 months, whereas certain fruit species, such as *Sideroxylon mascatense*, *Punica granatum*, *Vitis jacquamontiana*, *Morrus*, as well as *Ficus* and *Zizphus* species, are only available for a few months.

## *3.4. Shepherds and Their Role in Gathering WFPs*

Shepherd communities, in search of grasses and fodder for their cattle, migrate from one region to another in a cyclic manner. They graze their sheep, goats, and cows on hills and in the mountains and collect WFPs for their own dietary needs. They are totally or partially dependent on WFPs and homemade dairy products.

One elderly participant (81-year-old man) belonging to the Utmankhel - Alizai tribe mentioned his view about the role of WFPs in the food system of shepherds and grass collectors in these words: "In our time, when we used to visit the mountains for cattle grazing or grass and fuelwood collection, and we felt hungry, we ate mint (*Mentha longifolia*) and heart wing sorrel (*Rumex hastatus*) leaves with green chilies, onions, tomatoes, and prickly ash (*Zanthoxylum armatum*) seeds with wheat or barley bread. We ground these on big rocks with the help of small round stones along streams or springs. We used the creeping wood sorrel (*Oxalis corniculate*) as an alternative to *Rumex hastatus* leaves sometimes. Some people used to eat brown sugar with wheat or barley bread". Today, this practice of eating WFPs has decreased due to the modernization, globalization, and urbanization of communities.

## *3.5. WFPs and Their Uses in Traditional Cuisine*

Our findings show that fruits were the most commonly used parts (41%) of WFPs, followed by leaves (24%), aerial parts (24%), seeds (4%), stems (3%), and young inflorescences (2%), (Figure 2). The use of various parts varied from species to species and area to area within the region. Leaves and aerial parts were mostly used as vegetables, in sauces, or consumed raw, while fruits were mostly eaten raw. In terms of growth habit, the recorded WFPs were categorized as 57% herb, 22% tree, 19% shrub, and 2% climber species.

## 3.5.1. Vegetable Species

During the questionnaire survey, local people were asked (1) what wild vegetable species they collect? In response to our question, interviewees mentioned 27 species used as vegetables in the studied villages. According to the use reports, the most cited and used vegetable species were *Nasturtium officinale* (47), *Oxalis corniculata* (44), *Trifolium repense* (44), *Solanum villosum* (37), *Digera muricata* (31), *Portulaca quadrifida* (29), *Malva neglecta* (28), *Vicia sativa* (27), *Cichorium intybus* (21), and *Caralluma tuberculata* (19). *Nasturtium officinale*, locally called *Termera*, was the most used vegetable species in the region and considered a digestive. The species was collected by local communities beside water bodies such as streams, springs, ponds, and lakes from March to November. It was favored for its unique taste and availability. *Oxalis corniculata* was the most readily available species, which grows in the shade of other plants or humid places from March to December. Its widespread availability, long growing season, and special taste made it one of the most consumed vegetable species. *Trifolium repense* is a semi-domesticated species that grows alongside

water bodies. It was widely consumed as a vegetable throughout the region. For example, Chokanr, or *Trifolium repense* leaves with rice, was a very popular traditional dish across the whole tribal belt. *Solanum villosum*, *Solanum americanum*, *Descurainia sophia*, and *Sisymbrium irio* leaves and young shoots were boiled in concentrated milk and offered to patients as well as older people as a digestive. *Sustainability* **2021**, *13*, x FOR PEER REVIEW 9 of 22

**Figure 2.** Contribution of the WFP's parts used in the tribal food system. **Figure 2.** Contribution of the WFP's parts used in the tribal food system.

## 3.5.1. Vegetable Species 3.5.2. Fruit Species

During the questionnaire survey, local people were asked (1) what wild vegetable species they collect? In response to our question, interviewees mentioned 27 species used as vegetables in the studied villages. According to the use reports, the most cited and used vegetable species were *Nasturtium officinale* (47), *Oxalis corniculata* (44), *Trifolium repense* (44), *Solanum villosum* (37), *Digera muricata* (31), *Portulaca quadrifida* (29), *Malva neglecta* (28), *Vicia sativa* (27), *Cichorium intybus* (21), and *Caralluma tuberculata* (19). *Nasturtium officinale*, locally called *Termera*, was the most used vegetable species in the region and considered a digestive. The species was collected by local communities beside water bodies such as streams, springs, ponds, and lakes from March to November. It was favored for its unique In total, 24 species were collected from the wild and consumed raw by local inhabitants. According to the use reports, the most important wild fruit species were *Ficus carica* with 64 use reports, followed by *Berberis lyceum* (58), *Myrtus communis* (53), *Olea ferruginea* (48), and *Sideroxylon mascatense* (48). *Ficus carica* was a very common species in the study region and local people collected its fruits in the morning to avoid the hot weather, as it bears fruit during the hottest months of June and July. *Berberis lyceum* fruits were collected and mostly consumed raw, although some people also used the fruit to make a refreshing juice. *Sideroxylon mascatense*, which occurs in the low elevation hills, was another important fruit species of the study area. People collected ripe *Sideroxylon* fruits in large bottles or pots to maintain their shape and avoid bruising the fruit.

## taste and availability. *Oxalis corniculata* was the most readily available species, which grows in the shade of other plants or humid places from March to December. Its wide-3.5.3. Sauces and Chutneys

spread availability, long growing season, and special taste made it one of the most consumed vegetable species. *Trifolium repense* is a semi-domesticated species that grows alongside water bodies. It was widely consumed as a vegetable throughout the region. For example, Chokanr, or *Trifolium repense* leaves with rice, was a very popular traditional dish across the whole tribal belt. *Solanum villosum*, *Solanum americanum*, *Descurainia sophia*, and *Sisymbrium irio* leaves and young shoots were boiled in concentrated milk and offered to patients as well as older people as a digestive. In response to our questionnaire, informants reported only six species used in making chutney. The custom of herbal sauce or chutney making was not very common throughout the region but included the use of *Mentha longifolia* (40), *Mentha royleana* (34), *Cuminum cyminum* (21), *Zanthoxylum armatum* (37), *Allium ampeloprasum* (14), and *Thymus linearis* (7). Interestingly, all these species are aromatic and were used in fresh and dried forms as well. The local people shade-dried these species in their respective seasons for utilization throughout the year.

## 3.5.4. Species Used Raw

3.5.3. Sauces and Chutneys

3.5.2. Fruit Species In total, 24 species were collected from the wild and consumed raw by local inhabitants. According to the use reports, the most important wild fruit species were *Ficus carica* with 64 use reports, followed by *Berberis lyceum* (58), *Myrtus communis* (53), *Olea ferruginea* (48), and *Sideroxylon mascatense* (48). *Ficus carica* was a very common species in the study This category was also not common in the region. Fresh raw food species were collected from the wild and just masticated or chewed for their flavor or [52] to quench thirst or suppress appetite. Interviews confirmed only six plants, including the young shoots of *Saccharum benghalensis* (17), the fruits of *Silene conoidea* (22), the young shoots of *Salvia moorcraftiana* (9), and the aerial parts of *Indigofera* species (3), were consumed in the

region and local people collected its fruits in the morning to avoid the hot weather, as it bears fruit during the hottest months of June and July. *Berberis lyceum* fruits were collected

juice. *Sideroxylon mascatense*, which occurs in the low elevation hills, was another important fruit species of the study area. People collected ripe *Sideroxylon* fruits in large bot-

tles or pots to maintain their shape and avoid bruising the fruit.

region. Additionally, the seeds of *Carthamus oxyacantha* and *Pinus roxburghii* were used as food.

## 3.5.5. WFPs Used in Herbal Teas, Drinks, and Decoctions

*Myrtus communis*, *Mentha royleana*, and *Mentha longifolia* were used in herbal teas and decoctions. *Myrtus communis* leaves were boiled with green tea and *Mentha royleana* or *Mentha longifolia* were used to prepare herbal drinks. These drinks were used to treat vomiting, diarrhea, heartburn, and other stomach-related problems. The local people kept it in dry form in their houses for off-season uses. *Berberis lyceum*, *Punica granatum*, and *Morus nigra* were used in fresh drinks. The juices of *Berberis lyceum* and *Punica granatum* were considered important drinks in the area. Indigenous people used these plants for cooling the body and quenching thirst. *Berberis* fruits have a very short period of availability and may be available in the market for only 20–30 days. On the other hand, *Punica granatum* fruits were stored by local communities. They stored it in grain bins and used it for a few months. *Morus nigra* is very common and produces large amounts of fruits, but its juice was not common in the study area. Very few informants reported its use as an herbal drink.

## *3.6. Summary of WFP Cuisine*

Various recipes for cooking WFPs were recorded during the interviews with local informants. The traditional communities used WFPs in different ways, and the knowledge of these plants and their mode of consumption were passed down from their ancestors. There were different methods used for the preparation of wild foods depending on the nature of the plant. Wild vegetables, i.e., *Amaranthus vridis*, *Amaranthus crispus*, *Chenopodium album*, *Rumex dentatus*, *Rumex histatus*, *Malva neglecta*, *Digera muricata*, *Nasturtium officinale*, *Lathyrus aphaca*, *Vicia sativa*, *Medicago polymaorpha*, *Medicago denticulata*, *Portulaca quadrifida*, *Cichorium intybus*, *Polygonum aviculare*, *Polygonum plebium*, *Solanum villosum*, *Solanum americanum*, and *Marsilea* were cut into small pieces and boiled in water. After boiling these vegetables, onions were fried in oil, and then tomatoes and garlic were added [22]. After the addition of garlic, a special kind of sauce was made in the traditional tribal style, which is locally called *Dagh* or *Tarka*. Some people also added green chilies depending on preference and taste. Subsequently, the boiled vegetables were added and heated until the poured oil comes out on top of the dish. Some vegetables, such as *Solanum villosum*, *Solanum americanum*, *Descurainia Sophia*, and *Sisymbrium irio*, were boiled in concentrated milk [25]. Bitter vegetables like *Caralluma tuberculata* were cut into small pieces, boiled in water 2–3 times, and then fried in oil with onion, tomatoes, chilies, and condiments. Some people also cooked *Caralluma* with minced meat (*Keema/Qeema*).

## *3.7. Storage and Uses of WFPs in the Off-Seasons*

The storage and drying of WFPs is an important practice that provides stability to the food system throughout the year [24]. In the study area, a number of plants, such as the aerial parts of *Mentha longifolia*, *Mentha royleana*, and *Punica granatum* fruits, were stored by local communities in dried form for consumption during the off-season. Our findings showed that in ancient times, local people used to store fruit of pomegranates in grain bins (which were made of mud), and then they used them during times of need. *Morus alba*, *Sideroxylon mascatense*, and *Prunus armeniaca* (Kakhta) fruits were dried and used in the winter season. In April, *Allium carollianum* leaves were collected by traditional communities to prepare local spices or masala in their homes. They shade-dried its leaves and then used the dried form with *Curcuma longa* (Korkaman), *Cuminum cyminum*, and *Punica granatum* seeds. The use of *Punica granatum* seeds in spices was a custom of the traditional communities. *Myrtus communis* leaves and fruits were stored in homes and used in lassi and herbal tea for their enticing aroma and appetizing flavor.

## *3.8. Relative Frequency of Citation*

In the present study, RFC values ranged from 0.01 to 0.8. The highest RFC was recorded for *Ficus carica* (0.8), followed by *Myrtus communis* (0.7), *Berberis lyceum* (0.7), *Nasturtium officinale* (0.6), and *Olea ferruginea* (0.6) (Table 3). The high RFC values of certain wild fruit species demonstrate their abundant use and extensive knowledge among the tribal communities. *Myrtus communis* and *Berberis lyceum* have high RFC values but both species are facing conservation threats in the area. This continuous practice of gathering WFPs has caused an alarming decrease in the populations of some species. On the other hand, due to the cultural and religious importance of certain species, such as *Ficus carica* and *Olea ferruginea*, some plants are highly valued and thus distributed widely in the region. Therefore, on the basis of our field observations and interviews, we think that the extensive use of WFPs depends not only on the abundance of a species, but also on local preferences and its high nutraceutical value. However, traditionally people have used such food plants irrespective of their knowledge of the plant's impact on health. Rather than thinking about their health benefits, it has been more customary to use wild plants because of long-established traditions, but such uses are of tremendous benefit to these tribal people.

## *3.9. Selling and Marketing of WFPs*

During the interviews, we asked the participants whether they collected or still collect WFPs for marketing purposes. In response, the interviewees reported 10 plant species that are available in the local markets of Bajaur, i.e., Khar, Pashat, Inayt Kalay, Nawagai, Memola, Chenagai, and Tawheed Abad. These 10 marketable species included four wild fruits, five vegetables, and one species used as a flavoring agent. The fruit species included *Sideroxylon mascatense*, *Berberis lyceum*, *Ziziphus sativa*, and *Diospyrus lotus*. *Mentha longifolia*, *Mentha royleana*, *Caralluma tuberculata*, *Spinacia quadrifida*, and *Malva neglecta* were the wild vegetables species, while *Zanthoxylum armatum* was used as a flavoring agent in various traditional dishes. All these species were available fresh in the market except *Sideroxylon mascatense*, whose fruits were available in the market in dried form and during the off-season as well (Table 3).









119

HB = Habit, H = Herb, S = Shrub, T = Tree, Coll. Time = Collection Time, UR = Use reports, RFC = Relative Frequency of Citation, + = Marketable, - = Nonmarketable.

Citation, + = Marketable, - = Nonmarketable.

Wight and

### *3.10. Storage and Uses of WFPs in the Off-Season 3.10. Storage and Uses of WFPs in the Off-Season*

*Sustainability* **2021**, *13*, x FOR PEER REVIEW 17 of 22

A few of these plant species are available in the market and contribute significantly to the local economy. A few of these plant species are available in the market and contribute significantly to the local economy.

Inhabitants of rural areas collect and sell WFPs in the local markets for income generation. Abbasi et al. [25] reported that *Dryopteris ramosa*, *Bauhinia variegata*, *Chenopodium album*, *Portulaca quardifida*, *Nasturtium officinale*, *Malva parviflora*, and Solanum nigrum are sold in the markets of Rawalpindi and Abbottabad, while we did not report the market value for any of these species. According to the findings of Abbas et al. [22], wild vegetable species available in the local markets of Kurram district include *Caralluma tuberculata*, *Mentha spicata*, *Lepidium draba*, *Rumex dentatus*, *Portulaca oleracea*, *Malva neglecta*, *Trifolium repens*, *Stellaria media*, and *Nasturtium officinale*. In the current study, we documented *Caralluma tuberculate* (Figure 3), *Malva neglecta*, and *Mentha* species, which is similar between the two study areas. *Rumex dentatus*, *Malva neglecta*, *Trifolium repens*, *Stellaria media*, and *Nasturtium officinale* were very common species and used by tribal communities as vegetables, but we did not observe them in the markets. Inhabitants of rural areas collect and sell WFPs in the local markets for income generation. Abbasi et al. [25] reported that *Dryopteris ramosa*, *Bauhinia variegata*, *Chenopodium album*, *Portulaca quardifida*, *Nasturtium officinale*, *Malva parviflora*, and Solanum nigrum are sold in the markets of Rawalpindi and Abbottabad, while we did not report the market value for any of these species. According to the findings of Abbas et al [22], wild vegetable species available in the local markets of Kurram district include *Caralluma tuberculata*, *Mentha spicata*, *Lepidium draba*, *Rumex dentatus*, *Portulaca oleracea*, *Malva neglecta*, *Trifolium repens*, *Stellaria media*, and *Nasturtium officinale*. In the current study, we documented *Caralluma tuberculate* (Figure 3), *Malva neglecta*, and *Mentha* species, which is similar between the two study areas. *Rumex dentatus*, *Malva neglecta*, *Trifolium repens*, *Stellaria media*, and *Nasturtium officinale* were very common species and used by tribal communities as vegetables, but we did not observe them in the markets.

**Figure 3.** (**a**) Fruit of *Morus nigra*, (**b**) Fruit of *Morus alba* , (**c**) a young man collecting fruit of Mazri Palm, (**d**) *Caralluma tuberculata*, (**e**) a young man collecting different wild vegetables, and (**f**) Seeds of *Carthamus oxyacantha*. **Figure 3.** (**a**) Fruit of *Morus nigra*, (**b**) Fruit of *Morus alba*, (**c**) a young man collecting fruit of Mazri Palm, (**d**) *Caralluma tuberculata*, (**e**) a young man collecting different wild vegetables, and (**f**) Seeds of *Carthamus oxyacantha*.

### **4. Discussion 4. Discussion**

## *4.1. The Reported WFPs Compared to Previous Studies 4.1. The Reported WFPs Compared to Previous Studies*

We studied the contribution of WFPs to the food system and balancing the nutrition of tribal communities living along the Pakistan-Afghanistan border in the Hindu Kush mountain range. Vegetables were the dominant use category of WFPs across the study region (with 27 different species). Similar studies have also been carried out by other authors on wild vegetables and their nutritional value. For example, Abbas et al [22] re-We studied the contribution of WFPs to the food system and balancing the nutrition of tribal communities living along the Pakistan-Afghanistan border in the Hindu Kush mountain range. Vegetables were the dominant use category of WFPs across the study region (with 27 different species). Similar studies have also been carried out by other authors on wild vegetables and their nutritional value. For example, Abbas et al. [22] reported 53 wild vegetable species from the Kurram district. Likewise, Abbasi et al. [25] documented 45 wild vegetables from the Lesser Himalayas of Pakistan. In comparison to our study area, the district of Kurram and area of the Lesser Himalayas have a rich diversity of wild vegetable species due to greater rainfall and thus suitability for the growth of WFPs. Aziz et al. [19] reported 21 vegetables used in the remote Yasin and Ishkoman valleys of Gilgit Baltistan, while Ahmad et al., 2019 [23], reported 25 wild

vegetable species from Northwest Pakistan. The areas of the Pakistani–Afghan border in the Hindu Kush mountains are characterized by dry and harsh climatic conditions which are quite similar to the climatic conditions of the Yasin and Ishkoman valleys and Northwest Pakistan. Therefore, those areas are home to a small number of wild vegetable species. Another study [28] reported 59 wild vegetables from the district of Harnai Baluchistan. The traditional communities of Harnai district use various species that were not mentioned in our study area, such as *Boerhavia procumbence*, *Erodium cicutarium*, and *Fumaria indica*. This disparity in the use of wild vegetable species may be due to the unavailability of food resources or ethnic differences. On this basis, we believe that the use of wild vegetables varies from area to area due to their availability, the climatic conditions, and ethnography.

In this study, fruits were the second most dominant category with 24 species among the recorded WFPs. On the other hand, Abbasi et al. [16] recorded 35 wild fruit species used by the tribal communities of the Lesser Himalayas, and Khan et al., 2015 [27] recorded 47 species of wild fruits from the Swat Valley. The Lesser Himalayas and Swat Valley exhibit a greater diversity of plants and receive more precipitation during the monsoon season compared to the study region. The area of the Hindu Kush including Bajaur, receives very little precipitation during the summer monsoon. These climatic factors distinguish this area from Swat Valley and the Lesser Himalayas in terms of fruit flora. Ahmad and Pieroni [24] studied 31 wild fruit species from the Thakhte-Sulaiman Hills, Northwest Pakistan. The Thakhte-Sulaiman hills are characterized by a dry and harsh climate with mostly bushy and thorny fruit plants such as *Berberis calliobotrys*, *Ziziphus* species *Cotoneaster microphyllus*, *Cotoneaster minutus*, and *Cotoneaster pruinosus*. On the other hand Perez-Negron and Casas 2007 [54] documented 11 wild fruit species from the district of Dera Ismail Khan. Dera Ismail Khan is also characterized by very harsh climatic conditions and is considered a hub for *Phoenix dactylifera* (Dhaki variety) and *Nannorrhops ritchieana*, especially in the Abdul Khel and Rahmani Khel areas. The Hindu Kush valleys near the Pakistani–Afghan border, including Bajaur, have similar plant taxa such as *Sideroxylon mascatense*, *Nannorrhops ritchieana*, and *Ziziphus* species. In the recent past, ethnobiologists were unwilling to acknowledge the use of *Quercus* fruits as human food [55], but their consumption is very common in various parts of the world, including Turkey [56,57] and the Iberian Peninsula [55]. Mendez-Baceta et al. [52,55] mentioned that in Gorbeialdea *Quercus* acorns are considered food for livestock or eaten during times of food scarcity. In the current study, we reported that *Quercus* acorns were not used as a staple food; however, roasting them is still a hobby in the tribal belt and people come from plain areas to the mountains for acorn collection. Therefore, we suggest that the distribution, diversity, and consumption of wild fruit species vary from region to region depending on their availability, the climatic conditions, nutritional importance, and cultural knowledge.

We also reported various plant species that were used as flavoring agents in sauces and chutneys, i.e., *Mentha* species. Abbas et al. [22] documented that in the Kurram district *Mentha* species were used in salads. Aziz et al. [19] reported that, in the Ishkoman and Yaseen valleys of Gilgit Baltistan, *Thymus linearis* was used in making herbal tea, while in the current study we reported that it was used as a spice. Thus, we believe that variation in the use of WFPs in chutney or sauce making varies from one area to another, perhaps due to ethnography or traditional knowledge. The practice of utilizing raw foods species was also not very common as only six species were reported to be used in raw form. All these plant species were not described as raw food species in other studies previously conducted on the WFPs of Pakistan. *Silene conoidea* was described by Aziz et al., 2020, Abbas et al., 2020, Abbassi et al., 2013b-[19,22,25] as a cooked vegetable species, while in the current study region it was used as a raw food species rather than as a cooked vegetable.

The use of plants in herbal drinks is an important cultural practice among tribal communities. Aziz et al. [19] mentioned that *Carum carvi* seeds, *Elaeagnus angustifolia* bark, and *Thymus linearis* aerial parts were used as seasonings or herbal drinks. In the current research, we reported that the aerial parts of *Thymus linearis* were used as an herbal spice, but not as an herbal drink. The other two species, *Elaeagnus angustifolia* and *Carum carvi*, were not reported in the study area. There is great variation in the climatic conditions and topography of Gilgit Baltistan and areas of the Hindu Kush mountains. Gilgit Baltistan occurs in the Karakoram mountain range, while areas of Pakistani–Afghan border in Bajaur are part of the Hindu Kush mountain range, which suggests that variation in topographic and climatic factors generates variations in floral diversity and traditional uses of plants among local societies.

The tribal belt of Pakistan has a unique food system. Wild food plants are still used in different ways to improve their taste, nutritional value, and health benefits. The local inhabitants prepared many traditional dishes by mixing WFPs with other wild or cultivated vegetables and other foods, i.e., rice, meat, chicken handi, qeema, pulses, Indian squash, potatoes, traditional bread, milk, yogurt, custards, etc. Various other products were also prepared with WFPs, such as juices, syrups, and jams. *Carthamus oxycanthus* seeds were mixed with brown sugar and heated until caramelized, which is locally called *Kaboray* (caramel candy).

Guarrera and Savo [12] discussed this tradition in a review article titled "Wild food plants used in traditional vegetable mixtures in Italy". While investigating the wild vegetables of the Lesser Himalayas, Abbasi et al. [25] also mentioned that some vegetable species are cooked in a mixture with other vegetables or in concentrated milk. Abbas et al. [22] documented in their study of the wild vegetables of Kurram district that "all the seasonal wild vegetables are combined intermittently and a particular dish of 7 to 10 species is cooked during festivals (Nowruz) in the Shia community". Therefore, we suggest that the consumption of WFPs, especially that of wild vegetables, mixed with other foods is a long-standing tradition of human societies to improve their taste, nutritional value, and health benefits.

## *4.2. Management and Conservation of WFPs*

The territory has a great diversity of wild fruit species, including *Ficus carica*, *Ficus palmata*, *Morus alba*, and *Morus nigra*, which occur in abundance across the region. Many of these plants are not currently harvested by humans, but there is potential to use them to meet important dietary requirements. These plants can be conserved in their natural habitats and some of them domesticated as well. Very few people dry and store these fruits for consumption in other seasons of the year when these fruits are not available. There is no plan for the management and use of these species. In the present scenario of COVID-19 pandemic-driven situations, people rely on plant-based recipes to manage health care-related issues [57] and such research findings may be of vast attention for people in the rural areas.

## **5. Conclusions**

The current research represents the first attempt to study the contribution of WFPs to the food system of the tribal belt of Pakistan at the Pak–Afghan border in the Hindu Kush mountain range. The local inhabitants are economically poor and most of the time utilize local food resources at home. The results showed that WFPs are still collected and consumed in the region. In addition to their nutritional value, 10 species have market value and contribute to the economy of local communities. The uses of *Carthamus oxyacantha*, *Pinus roxburghii* seeds, and *Marsillea* leaves are novel for the gastronomy of Pakistan after comparison with other regions of the country. *Myrtus communis*, *Berberis lyceum*, *Prunus armeniaca*, *Nannorrhops ritchieana*, *Zanthoxylum armatum*, and *Vitis jacquemontii* are the species facing conservation issues due to overexploitation, soil erosion, and climatic changes.

Knowledge related to the consumption of WFPs is decreasing in the younger generations of tribal societies. Therefore, more studies are needed to conserve this important cultural gastronomic heritage of the study region where modernization, urbanization, and industrialization are at their peak. We believe that this study will play a crucial role in the conservation and fostering of the traditional knowledge of WFPs cooking and consumption. **Author Contributions:** A.A. carried out the fieldwork, collected data and prepared the first draft of the manuscript. S.M.K. supervised the project, helped in data analyses, provided the logistic support, read all the revisions and correspond the paper. A.P. contributed to the study design and to draft the MS. A.H. (Amunul Haq) helped during Field Work. Z.U.H. helped in specimens processing. Z.A. contributed in data analyses and preparation of GIS map. S.S. helped in literature review and preparation of paper draft. A.H. (Abeer Hashem), A.-B.F.A.-A., A.A.A. and E.F.A. helped in revising and improving the manuscript and provided partial financial support for the research activity and manuscript processing fee. All authors work parallel to produce the research article in hand shape. All authors have read and agreed to the published version of the manuscript.

**Funding:** The authors would like to extend their sincere appreciation to the Deanship of Scientific Research at King Saud University for funding this research (RG-1435-014); to the University of Gastronomic Sciences, Pollenzo, Italy for having funded the editing of the paper.

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Informed consent was obtained from all subjects involved in the study.

**Data Availability Statement:** Not applicable.

**Acknowledgments:** The authors would like to extend their sincere appreciation to the Deanship of Scientific Research at King Saud University for funding this research group NO (RG-1435-014). We also highly acknowledge support of the University of Gastronomic Sciences Pollenzo, Italy for their editing services through professional editor. Special thanks also go to the local people of the study area who generously shared their views about WFPs utilization.

**Conflicts of Interest:** The authors declare no conflict of interest.

## **References**


## *Article* **Analysis of Nutritional Quality Attributes and Their Inter-Relationship in Maize Inbred Lines for Sustainable Livelihood**

**Sapna Langyan 1,2,\*, Zahoor A. Dar <sup>3</sup> , D. P. Chaudhary <sup>1</sup> , J. C. Shekhar <sup>1</sup> , Susila Herlambang <sup>4</sup> , Hesham El Enshasy 5,6 , R. Z. Sayyed 7,\* and S. Rakshit <sup>1</sup>**


**Abstract:** The present investigation was planned to understand the variability and inter-relationship among various nutritional quality attributes of maize kernels to identify potential donors of the respective traits for future hybridization programs. Sixty-three maize inbred lines were processed for the estimation of protein, starch, fat, sugar, 100-kernel weight, specific gravity, and moisture level of the grain. The results reveal that a wide variability among protein, starch, 100-kernel weight, specific gravity, and fat was seen, with special emphasis on the protein concentration that varied from 8.83 to 15.54%, starch (67.43–75.31%), and 100-kernel weight (9.14–36.11 gm). Factor analysis revealed that the protein concentration, starch, and 100-kernel weight, the three major components, comprise 68.58% of the kernel variability. Protein exhibited a significant negative correlation with starch and 100-kernel weight, indicating that an increase in the protein concentration will down-regulate the starch and 100-kernel weight. The inbred lines are proposed as donors for the development of high cultivars for their respective traits, viz., high protein (DMR WNC NY 403 and DMR WNC NY 404), high starch concentration (DMR WNC NY 2163, DMR WNC NY 2219, DMR WNC NY 2234, DMR WNC NY 2408, DMR WNC NY 2437, and DMR WNC NY 2466), high 100-kernel wt. (DMR WNC NY 2113, DMR WNC NY 2213, DMR WNC NY 2233, DMR WNC NY 2234, DMR WNC NY 2414, DMR WNC NY 2435, DMR WNC NY 2465, and DMR WNC NY 2474), sugar (DMR WNC NY 2417), and specific gravity (DMR WNC NY 2418). Genetic distance analysis revealed that DMR WNC NY 397 and DMR WNC NY 404 are the farthest apart inbred lines, having major differences in their protein, fat, starch, and sugar contents, followed by DMR WNC NY 2436 and DMR WNC NY 2394, DMR WNC NY 2212 and DMR WNC NY 2430, DMR WNC NY 396 and DMR WNC NY 2415, DMR WNC NY 404 and DMR WNC NY 2144, and DMR WNC NY403 and DMR WNC NY 2115. Moreover, this study proposes that these possible combinations of lines (in a breeding program) would result in genetic variability with simultaneous high values for the respective characteristics.

**Keywords:** clustering; correlation; fat; protein; starch; sugar; specific gravity

## **1. Introduction**

Food insecurity and hunger affect more than 900 million people worldwide each year. People from poor and underdeveloped and some developing nations are at greater risk. About 5 million hungry people in the world die each year from nutrient deficiency

**Citation:** Langyan, S.; Dar, Z.A.; Chaudhary, D.P.; Shekhar, J.C.; Herlambang, S.; El Enshasy, H.; Sayyed, R.Z.; Rakshit, S. Analysis of Nutritional Quality Attributes and Their Inter-Relationship in Maize Inbred Lines for Sustainable Livelihood. *Sustainability* **2021**, *13*, 6137. https://doi.org/10.3390/ su13116137

Academic Editors: António Raposo and Heesup Han

Received: 20 April 2021 Accepted: 24 May 2021 Published: 29 May 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

causes [1,2]. Women and children are more susceptible to nutrient deficiency due to reproduction and growth demands, respectively [3]. Maize (*Zea mays* L.) is one of the world's important cereal grain crops after rice and wheat. The United States, the European Union, China, Brazil, Mexico, and India are the world's leading producers of maize [4]. Its popularity as a crop is largely due to its diverse functionality as a food source for both humans and animals. Maize has been the major source of the world's protein and calories [5,6]; hence, it is the dietary staple food crop for more than 300 million people [7].

India, since independence, has achieved a manifold increase in food grain production owing to the Green Revolution of the late 1960s. Indian agriculture research, until lately, insisted on increasing productivity per se, rather than quality. India's Human Development Index rank of 130 in the year 2020 reflects a major deficiency in the quality of life of people. Various initiatives of the government have largely remained unsuccessful in bringing down the menace of malnutrition. The latest National Family and Health Survey (NHFS2018-19) data released by the government are indicative of the limited progress made in improving the nutritional status and quality of health services for infants and children, adolescent girls, and women [8]. This is because an adequate diet is not affordable and accessible to all; worldwide, 462 million adults are underweight, and around 45% of deaths among children under 5 years of age are linked to undernutrition [9]. India is one of the nations with a large number of tribal people having malnutrition. This warrants an urgent need to produce maize with sufficient nutrient contents, as maize is a staple and principal food crop for the majority of the tribal people [10]. Various strategies to improve nutritional quality include increasing access to various nutrients, food fortification with supplements, bio-fortification, the introduction of pharmaceutical supplements, and dietary diversification [11,12].

Maize (*Zea mays* L.) is the most widely cultivated crop in all conditions, ranging from tropical to temperate, and regions of the world, providing nutrients as well as raw materials for biomolecules such as starch, fat, and protein [13]. In India, maize occupies a prominent position, and each part of the maize plant is utilized in one or another way, with nothing going to waste [14,15]. The utilization pattern of maize comprises 59% as feed, 17% for industrial purposes, 10% as food, around 10% for export, and 4% for other purposes, including seed [16]. Among all cereals, maize has the highest growth rate with maximum productivity, and due to it possessing the highest genetic yield potential, maize is known as the "queen of the cereals" [17]. The nutritional composition of normal maize comprises 8–13% protein, 68–73% starch, 2–5% fat, 2–4% sugar, fibers, minerals, etc. [18,19]. However, these nutritional attributes are inter-related, and an increase in one may adversely affect the other such as high-fat maize likely having a lower quantity of starch [20,21]. Keeping in view the inter-relationship of nutritional quality parameters when using the information in hybrid breeding programs, the present research work was undertaken to study the variability, the correlation, and the inter-relationship between the nutrient components of 63 different maize inbred lines. Additionally, the genetic distance between 63 different varieties of maize inbred lines was studied in order to select those that have the potential to be used as donor parents for their respective traits for the development of maize cultivars with enhanced protein, sugar, fat, and starch, with a possible contribution to decreasing malnutrition.

## **2. Materials and Methods**

## *2.1. Plant Materials*

The inbred lines were grown in an augmented block design (ABD), with 4 rows per inbred line at 60 cm spacing at a length of 3 m at the Winter Nursery Centre, Hyderabad, India, having a temperature range of 7 to 12 ◦C, sandy soil, 6.5 pH, and low organic matter, during the rabi season. The plants were selfed, and the selfed seeds were used for the biochemical evaluation. Details of the pedigree of the inbred lines are provided in Table 1.


**Table 1.** Nutritional composition of 63 maize inbred lines.


**Table 1.** *Cont.*

## *2.2. Preliminary Analysis*

The samples were oven dried at 90 ◦C to reduce the level of moisture of the grains to meet the accuracy of the results. The kernels were ground to powder by course and fine grinding using a pestle and mortar and finally kept in desiccators for analysis of various nutritional quality parameters.

## *2.3. Estimation of Protein, Moisture, Sugar, Starch, 100-Kernel Weight (100-Kernel wt.), Specific Gravity, and Fat Concentration*

Protein concentration was determined by the micro-Kjeldahl method of AOAC [22]. In this method, the de-fatted samples were digested until the solution became colorless. Further distillation and titration were conducted by using 8 mL NaOH and 0.02 N HCl. The moisture level of the grains was determined by the oven drying method OAC 934.01 by drying at 135 ◦C for 2 h [22]. Total sugars were estimated according to the method of Nelson–Somogyi [23]. Starch concentration was determined according to the method of Clegg [24] using Anthrone reagent. After extraction of starch with perchloric acid, it was further hydrolyzed (in an acidic medium) into glucose, which formed a green color compound on reaction with Anthrone reagent. The optical density was recorded against blank at 620 nm. To calculate the 100-kernel weight, one hundred maize kernels were counted manually, and then these were weighed by an electronic weighing balance with 0.01 g accuracy. The specific density of the kernel was determined by the method of Sangamithra [25]. Fat concentration was estimated according to the method of AOAC by using the solvent extractor system [22]. In this method, the fat concentration of the ground powder was extracted at 40–60 ◦C using non-polar solvent petroleum ether.

## *2.4. Statistical Analysis*

## 2.4.1. Analysis of Variance (ANOVA)

ANOVA was performed to study the significance of the genotypic differences and conducted using Statistical Package for the Social Sciences (SPSS) software. All the biochemical evaluations were conducted in duplicate.

## 2.4.2. Univariate and Multivariate Statistics

The results were expressed as univariate and multivariate statistics. Scott–Knott correlations at significance levels of 1% and 5% and hierarchical cluster analysis (HCA) based on the squared Euclidean distance using Ward's method were performed using Statistical Analysis Software (SAS 9.2 English). Factor analysis for variability component loading was conducted using Statistical Package for the Social Sciences (SPSS) software. The principal component analysis (PCA) method was used as the extraction. The loading plot was prepared using SPSS. The Pearson correlation coefficient |*r*| among 63 maize inbred lines was calculated by the formula given below:

$$r = \frac{\sum XY - \frac{\left(\sum X\right)\left(\sum Y\right)}{n}}{\sqrt{\left(\sum X^2 - \frac{\left(\sum X\right)^2}{n}\right)\left(\sum Y^2 - \frac{\left(\sum Y\right)^2}{n}\right)}}\tag{1}$$

where *X* and *Y* are the variables and *n* is the total number of samples used in the study.

## **3. Results**

## *3.1. Variability Analysis and Factor Analysis*

The ANOVA indicated that the inbred lines differed significantly for all the nutritional quality traits of maize kernel (Table 2), and a wide variability among these quality traits was also observed.

**Table 2.** Mean, standard error, and F-ratio of nutritional quality attributes of 63 maize inbred lines for variability analysis.


\* Significant at the 0.05 probability level; \*\* significant at the 0.01 probability level.

The protein concentration in the maize kernels varied from 8.83 to 15.54%. The highest protein concentration was observed in the DMR WNC NY 403 inbred line (Table 2). The sugar concentration ranged from 3.01 to 5.77%, and the highest sugar concentration was observed in the DMR WNC NY 2417 inbred line. The starch concentration of the maize kernels varied from 67.43 to 75.31%. The highest starch concentration was observed in the DMR WNC NY 2408 inbred line. Meanwhile, the fat concentration varied from 2.04 to 4.47%. The highest fat concentration was observed in the DMR WNC NY 400 inbred line. The 100-kernel wt. varied from 9.14 to 36.11 gm, the highest being exhibited by DMR WNC NY 2234, and specific gravity varied from 0.96 to 1.90 g/cm<sup>3</sup> . Specific gravity data were rightly skewed to their means and contributed towards a high specific gravity. The moisture level of all 63 inbred lines chosen for this study was within acceptable limits.

Factor analysis indicated that the variability among the present inbred lines is mainly contributed by protein and starch concentrations, followed by 100-kernel wt. (Table 3). These three components were extracted based upon principal component analysis, which showed that these alone contribute 68.58% towards kernel variability. The loading plot (Figure 1) revealed that protein, starch concentration, and 100-kernel wt. strongly influence the variability as their loading is close to 1.0. However, the variability does not affect sugar, oil, and specific gravity as their values are below 0.5, i.e., less than 1.0, and as we know, loadings close to −1 or 1 strongly influence the variable, and higher loadings either positively or negatively indicate that the particular variable has a strong effect on the principal component.

**Figure 1.** Loading plot of nutritional quality attributes of 63 maize inbred lines for variability anal-**Figure 1.** Loading plot of nutritional quality attributes of 63 maize inbred lines for variability analysis.

**Table 3.** Component matrix, communities, and total variance of nutritional quality attributes of 63 maize inbred lines for variability analysis. (**a**) Component matrix. (**b**) Communities. (**c**) Total vari-

Protein −0.750 0.073 0.376 Starch 0.714 0.243 −0.272 100-Kernel wt. 0.688 −0.391 0.168 Fat 0.118 0.770 0.107 Specific Gravity 0.083 0.758 0.042 Sugar 0.398 −0.012 0.865  **Initial Extraction**  Protein 1.000 0.709 Oil 1.000 0.618 Sugar 1.000 0.907 Starch 1.000 0.643 100-Kernel wt. 1.000 0.655 Specific Gravity 1.000 0.583

**Component 1 2 3** 

**Initial Eigenvalues Extraction Sums of Squared Loadings** 

**(%) Cumulative (%)** 

**(%) Cumulative (%) Total Variance** 

1 1.725 28.750 28.750 1.725 28.750 28.750 2 1.384 23.074 51.824 1.384 23.074 51.824 3 1.006 16.762 68.586 1.006 16.762 68.586

Extraction method: principal component analysis. Extraction was performed by the principal com-

ance explained.

**Component** 

ponent analysis method.

ysis.

**Total Variance** 

4 .745 12.420 81.006 5 .627 10.452 91.458 6 .513 8.542 100.000

**Table 3.** Component matrix, communities, and total variance of nutritional quality attributes of 63 maize inbred lines for variability analysis. (**a**) Component matrix. (**b**) Communities. (**c**) Total variance explained.


Extraction method: principal component analysis. Extraction was performed by the principal component analysis method.

## *3.2. Correlation Analysis*

Scott–Knott correlation coefficients among various traits such as protein, fat, starch, sugar, 100-kernel wt., and specific gravity of the grains are provided in Table 4. These results reveal that the protein concentration exhibited a significant negative correlation with starch and 100-kernel weight. Contrary to this, the fat concentration showed a significant low positive relationship with specific gravity. Sugar and starch possess a non-significant positive correlation with all the traits under study, except protein.


**Table 4.** Coefficients of nutritional quality attributes of 63 maize inbred lines.

\*\* Correlation is significant at the 0.01 level; \* correlation is significant at the 0.05 level.

## *3.3. Genetic Distance Measurement and Hierarchical Cluster Analysis*

The genetic distance relationship of 63 maize inbred lines, depicted by the squared Euclidean distance based on Ward's method of hierarchical clustering, was obtained based on nutrient composition (Figure 2) data. Cluster analysis was used to reveal the association between the inbred lines used in the present study. Cluster analysis provides various clustering algorithms such as "sequential hierarchical and neighbor clustering". Due to the great heterogeneity within groups, hierarchical cluster analysis can be used to cluster maize inbred lines according to their differences and similarities to further investigate chemical compositional relationships between them. A total of two major clusters are formed at a distance of 25, having 26 and 37 inbred lines in cluster I and cluster II, respectively. On further reducing the distance to 9, three clusters were formed in which cluster I remained unaffected, while cluster II was split into two sub-clusters, viz., C II-SC 1 and C II-SC 2, having 7 and 30 maize inbred lines, respectively. On further reducing the distance to 6, cluster I was split into two sub-clusters, viz., C I-SC 1 and C I-SC 2, having 6 and 20 members, respectively. DMR WNC NY 397 and DMR WNC NY 404 are the farthest apart, having major differences in their protein, fat, starch, specific gravity, 100-kernel wt., and sugar concentration, followed by DMR WNC NY 2436 and DMR WNC NY 2394, DMR WNC NY 2212 and DMR WNC NY 2430, DMR WNC NY 396 and DMR WNC NY 2415, DMR WNC NY 404 and DMR WNC NY 2144, and DMR WNC NY403 and DMR WNC NY 2115. This variability can be exploited in crop improvement programs, particularly for grain quality traits.

**Figure 2.** Depicting hierarchical clustering of 63 maize inbred lines by squared Euclidean distance based on Ward's method of hierarchical clustering based on their nutritional composition differences.

## **4. Discussion**

Maize kernels contain essential components for plant growth and reproduction [23]. However, the nutritional quality of maize is determined by its genetic background, and hence the inbred varieties with a high concentration of proteins, sugars, and other nutritional attributes are desired to combat malnutrition, particularly in sub-Saharan Africa, Southeast Asia, and Latin America [3].

A significant negative correlation between the protein concentration and starch and 100-kernel wt. indicated that an increase in the protein concentration will down-regulate starch and 100-kernel wt. This might be because protein and starch are both mostly present in the endosperm part of the kernel [26]. Although the endosperm contains lower amounts (8%) of protein than the germ (18.4%), it provides the highest quantity, i.e., 78% as compared to the 18% provided by the kernel germ [27]. Contrary to this, the fat concentration showed a significant low positive relationship with specific gravity, indicating that the specific gravity of maize may be increased with an increase in the fat concentration. Therefore, selecting for high fat and specific gravity altogether will not cause any difficulty during crop improvement programs. Maize starch accounts for about 72% of the dry weight of kernels, and it is located in the endosperm, while sugars such as sucrose, maltose, glucose, fructose, and raffinose constitute about 1–3% of the total dry weight of kernels [28]. The major (75%) portion of these free sugars is located in the germ, with the endosperm constituting only 25% of the total sugars [28]. A negative low correlation between protein and fat revealed that breeding efforts cannot be applied for increasing both energy-rich sources simultaneously, and genetic manipulations to increase one may affect the other. Another result of this study is a non-significant positive correlation of sugar and starch with all the traits under study, except protein, as starch contributes mainly to the grain filling characteristics of maize, hence increasing the 100-kernel wt. and, simultaneously, the density of the kernel. Specific gravity was reported to enhance the viability of the grains through an increase in their quality in soybean [29].

Muhammad Saleem et al. [30] and Chaudhary et al. [21] found a positive correlation between protein and sugars. The protein in maize kernels has been studied extensively because it forms the core of the quality of maize. This might be the reason for the above results because sugars are mainly confined to the germ as discussed earlier as well. Our results agree with some other previous findings of Willmot et al. [31]; Clark et al. [32]; Dudley et al. [33]; Liu et al. [34]; Zhang et al. [35]; and Chaudhary et al. [21]. A negative correlation between protein and moisture concentrations indicates an increased moisture level of the grains would decrease the quality of the grain. Therefore, a balanced moisture level is required for improving the quality as well as the quantity of grains. Chaudhary et al. [21] postulated that in maize, the germ accounts for 8–10% of the total grain weight and may contribute 15–20% of total protein, whereas the endosperm accounts for 80– 85% and contributes approximately 80% of protein. Panthee et al. [36] found an inverse relationship between protein and fat concentrations and hence postulated that it will be difficult to improve both attributes simultaneously. Generally, it was observed by many researchers that the protein concentration is negatively and positively correlated with starch and fat concentrations [31,32,34,35]. Similarly, an inverse relation was found between starch and fat [37]. An increase in fat, which might have originated from a bigger germ size, could adversely affect the endosperm volume, thus indirectly contributing towards a lower starch concentration. High-fat maize is usually maize with a higher germ size and a lower endosperm if compared to normal maize genotypes [21]. Few researchers reported that there would be no serious barrier in selecting for both high-protein and high-fat in maize [38,39]. Okporie and Obi [39] and Okporie Oselebe [40] found that sugar also has a very low non-significant positive correlation with starch, which may be attributed to the source and sink relationship. Generally, a high sugar concentration has comparatively low starch synthase activity that may be due to the high Km value of enzymes. Here, both might be there in the equilibrium phase.

Genetic distance analysis presented two main clusters having 26 and 27 clusters, providing the base for selecting parents at a farthest distance in order to attain nutritional variability.

Various strategies for attaining nutritional security can be adopted which are feasible and sustainable as well. Bio-fortification is one of the most important strategies because it is rural-based, where 70% of the resource-poor live, and is cost-effective and sustainable too. Maize provides about 15% of the world's protein (essential amino acids) and 20% of the world's calories [5]. A maize inbred line rich in protein and sugar concentrations has a positive correlation between these nutrients and is highly important for proper human nutrition, growth, health, and immunity and for combating malnutrition.

The outcomes of the present study will help in designing breeding strategies for developing nutritionally improved maize hybrids for the nutritional security of India and the world, and this can answer many of the nutritional quality issues. The authors strongly believe that along with these strategies, bioavailability studies should be carried out to confirm that the targeted nutrient is being readily absorbed in the body and can fulfill the recommended dietary allowance (RDA) because the nutritional quality of the "resource-poor" is much more important than that of the quantity. This approach may help to take a step forward to improve the livelihood security of a nation.

## **5. Conclusions**

The results of the present study reveal a wide variability among protein, starch concentration, and 100-kernel weight in maize inbred lines, and these three contribute to 68.58% of the kernel variability. The inbred lines were proposed as donors for the development of high cultivars for their respective traits, viz., high protein (DMR WNC NY 403 and DMR WNC NY 404), starch concentration (DMR WNC NY 2163, DMR WNC NY 2219, DMR WNC NY 2234, DMR WNC NY 2408, DMR WNC NY 2437, and DMR WNC NY 2466), 100-kernel wt. (DMR WNC NY 2113, DMR WNC NY 2213, DMR WNC NY 2233, DMR WNC NY 2234, DMR WNC NY 2414, DMR WNC NY 2435, DMR WNC NY 2465, and DMR WNC NY 2474), sugar (DMR WNC NY 2417), and specific gravity (DMR WNC NY 2418). Nutritional components in maize inbred lines are also highly correlated, and an alteration in one may positively or negatively affect the other. Although all three were the principal components of variability, an increase in protein will lower the starch concentration. Further, an increase in the weight of the kernels might increase the starch concentration rather than lower the protein concentration of the grain. The development of high-protein maize will affect the grain yield to some extent. However, selecting for high fat would increase the specific gravity which in turn enhances the viability of the grain, providing a wide base for maize hybridization programs.

DMR WNC NY 403 and DMR WNC NY 404 are proposed as high-protein, low-sugar, and low-starch materials, DMR WNC NY 2163, DMR WNC NY 2219, DMR WNC NY 2234, DMR WNC NY 2408, DMR WNC NY 2437, and DMR WNC NY 2466 as high-starch and low-protein materials, and DMR WNC NY 2418 for its high specific gravity and fat level. DMR WNC NY 2234 is proposed as a promising material for its starch concentration and 100-K wt., and DMR WNC NY 2163 and DMR WNC NY 2219 are proposed for their protein, sugar, starch, and specific gravity, only needing to improve their 100-K wt. and fat concentration. Further, DMR WNC NY 2113, DMR WNC NY 2408, DMR WNC NY 2417, and DMR WNC NY 2437 are proposed as excellent materials for their protein, sugar, starch, 100-K wt., and specific gravity, only needing to improve their fat content. Next, DMR WNC NY 2213, DMR WNC NY 2465, and DMR WNC NY 2466 are found as promising for all other traits than protein and fat concentration, whereas DMR WNC NY 2414 and DMR WNC NY 2435 need improvement in their fat and starch concentrations only.

Among the various inbred lines, DMR WNC NY 397 and DMR WNC NY 404, followed by DMR WNC NY 2436 and DMR WNC NY 2394, DMR WNC NY 2212 and DMR WNC NY 2430, DMR WNC NY 396 and DMR WNC NY 2415, DMR WNC NY 404 and DMR WNC NY 2144, and DMR WNC NY403 and DMR WNC NY 2115, were genetically the farthest apart inbred lines, having major differences in their protein, fat, starch, 100-kernel weight, specific gravity, and sugar concentration, and arising from two different clusters as well. In brief, there is a high variability for three major nutritional traits as identified by factor analysis; therefore, these inbred lines can be used as potential donors of the respective traits and would be beneficial to be proposed in breeding programs, as combinations of these lines and their crosses would result in hybrids or genetic variability with high values in one or many traits.

The adoption of maize inbred lines possessing higher proteins and amino acids would result in a significant decrease in malnutrition. Development and consumption of nutrientrich maize varieties would help in preventing malnutrition and in achieving nutritional security more holistically.

**Author Contributions:** Conceptualization, S.L.; investigation and writing of the original draft of the manuscript, S.L.; methodology, S.L., D.P.C., J.C.S., and Z.A.D.; reviewing and editing of the manuscript, S.R.; editing and formal analysis, S.H., H.E.E., and R.Z.S.; funding acquisition, H.E.E. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research was funded by Allcosmos Industries Sdn. Bhd. Arif Efektif Sdn. Bhd., Malaysia with grant Ns. RJ130000.7609.4C187 and RJ130000.7344.4B200.

**Institutional Review Board Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

**Data Availability Statement:** All the data is available in the manuscript.

**Conflicts of Interest:** The authors declare no conflict of interest.

## **References**


## *Article* **Investigating International Students' Perception of Foodservice Attributes in Malaysian Research Universities**

**Anisa Zahwa Akbara <sup>1</sup> , Bee-Lia Chua 1,\* , Heesup Han 2,\* and António Raposo <sup>3</sup>**


**Abstract:** University foodservice is expected to satisfy students' food needs and is one of the necessities at a university. However, serving a community of international students who are multicultural is not an easy task. Thus, it is necessary to recognize international students' needs and wants in order to increase their satisfaction with the overall on-campus dining experience. This study conducted an importance–performance analysis to examine international students' perceived importance and perceived performance of university foodservice attributes. Using a self-administered questionnaire, a total of 620 international students who were studying in Malaysian research universities comprised the sample of this study. The results showed that food price was the most satisfactory foodservice attribute as perceived by international students. Food quality was deemed unsatisfactory and represented the main weakness. Results from this study can assist in determining how international students perceive the quality of the key attributes of on-campus foodservices and identify fields in which improvements are required.

**Keywords:** university foodservice attributes; importance–performance analysis (IPA); international students; Malaysian research universities

## **1. Introduction**

The admissions of global students remain important to the university sector in Malaysia, since Malaysia sought to be acknowledged as a world-class higher education center by the year 2020 [1]. Moreover, foreign students also generate international business possibilities and ties to world trade, become diplomatic allies, and encourage interest in foreign policy [2]. However, to maintain international students in this competitive setting, significant efforts are needed. Efforts were made to attract foreign students by improving the reputation of Malaysian higher education institutions. Therefore, the Malaysian Research Universities (MRU) program has provided Malaysian universities and research institutions with an awareness of the research culture. Among the initiatives are (1) targeting the enrolment of global students, (2) improving Malaysian universities' world ranking, (3) creating more "Malaysian chairs" at universities overseas, and (4) collaborating and cooperating on research and educational issues with world-renowned universities [3]. In addition to these goals, superior service to meet student needs and expectations and maintain student satisfaction and loyalty to their chosen study destinations have become a core objective of these colleges [2,4].

According to Ezeokoli and Ayodele [5], the quality of foodservice is acknowledged as a main academic excellence metric and a significant strategic variable to boost the market share of colleges and service providers, with enduring impacts on the organization and its students. Klassen, Trybus, and Kumar [6] added that foodservice is a fundamental factor

**Citation:** Akbara, A.Z.; Chua, B.-L.; Han, H.; Raposo, A. Investigating International Students' Perception of Foodservice Attributes in Malaysian Research Universities. *Sustainability* **2021**, *13*, 8190. https://doi.org/ 10.3390/su13158190

Academic Editor: Marian Rizov

Received: 16 June 2021 Accepted: 17 July 2021 Published: 22 July 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

that influences student satisfaction. Furthermore, previous studies have shown various foodservice attributes, such as food quality, price fairness, hygiene, food variety, service quality, food price, and ambience, that could predominantly influence university students' satisfaction [7–14]. The foodservice outlet is the primary form for student foodservice in colleges [15]. Drummond and Brefere [16] noted that food can make students become more focused on academic studies and ultimately can affect their academic performance. Therefore, in order to be a study destination for international students, evaluating the role of on-campus dining service is critically important.

University foodservice is one of the amenities that must be taken into consideration in order to accommodate students [17]. Students' preferences for food and services are changing. Students who are dissatisfied with the service quality of a university cafeteria will choose off-campus dining [18]. Most university foodservice operations are outsourced in this modern age, with independent food operators or food firms given a certain number of years of contract to operate the company on campus [19]. Independent operators also run restaurants, cafeterias, food courts, kiosks, vending machines, dining halls and stalls on university premises. This is also the case for Malaysian university foodservices [20]. Consequently, for university foodservice operators, it is important to understand students' needs and satisfy their overall on-campus dining experience, even though some food operators are independent.

On the other hand, university foodservice operators in Malaysia have the most difficult task of serving a community of students who are multicultural and confined to the university environment [21]. Abdullah, Mansor, and Naba [22] found that students were generally dissatisfied with the food quality and price provided by university food operators. Liang and Zhang [15] discovered that the key factors contributing to student dissatisfaction when patronizing a university cafeteria were food quality, price, and value. Klassen, Trybus, and Kumar [6] identified that the most common complaints regarding university foodservice by international students were poor ambience, lack of resources, price, lack of food choices, and low food quality. In the context of university foodservice in Malaysia, there has been relatively limited empirical studies examining the importance and performance of foodservice attributes among international students in Malaysian research universities. To address this issue, this study analyzed the importance and performance of foodservice attributes perceived by international students in Malaysian research universities by utilizing the importance–performance analysis (IPA) method. This method is popular for evaluating service quality in a multitude of areas. IPA examines the "importance" individuals place on any given product/service attribute. Furthermore, the method has been employed in designing corporate marketing strategies, in guiding government planning choices, and in evaluating the organization and supervision of activities and schemes. The IPA method offers a pragmatic theoretical basis for important evaluation. The results of this study could be used to demonstrate which foodservice attributes need to be strategically planned and improved. The objectives of this study are as follows:


## **2. Literature Review**

## *2.1. University Foodservice Attributes*

Many studies have emphasized the effect on customer satisfaction of foodservices [17,19,23,24]. According to El-Said and Fathy [12], foodservice is an important component of overall planning that affects quality of life at university. Particularly, foodservice was verified as an important component of a healthy arrangement that affects the personal satisfaction of students and faculty at universities and colleges [25]. Kim, Lee, and Yuan [26] in their study of college students' satisfaction with university also confirmed that foodservice attributes influenced college student satisfaction.

Student–seller interaction is the type of communication that takes place in a university cafeteria. The interactions between the student and the foodservice personnel are important,

as students can be highly sensitive [27]. Moreover, Chang and Suki [28] added that the staff performance was found to have a statistically significant impact on student satisfaction of the experience of student dining in the cafeteria. The performance of cafeteria employees is particularly important for students in terms of customer friendliness and responsiveness. Hence, interactional quality is an important factor for improving college student satisfaction [27]. Concerning the food quality of on-campus foodservice, Andaleeb and Caskey [10] found that most students noted that, were the food quality to improve, they would more often consider on-campus foodservices. Food quality is a feature of foods acceptable to customers and includes the appearance, smell, taste, and texture of foods [29]. In shaping and affecting customer satisfaction, quality is of essential importance. Customers are the primary drivers of customer satisfaction for food quality [30].

Students also consider cost when purchasing of food at the cafeteria as an important aspect of university foodservice. It is well known that students have limited budgets that affect their food choices; they are always looking for reasonable prices [31]. This is the first problem students have in college foodservice, because students buy food with limited resources, as Nadzirah, Ab-Karim, Ghazali, and Othman [21] claimed. The paid price should be appropriate for the amount of food served to students so that they feel they are getting good value the products and services, producing student satisfaction. The dining atmosphere can be regarded, according to Ha and Jang [32], as critical for affecting customer satisfaction levels, mainly because customer reactions to the setting form part of the consumer experience. For university foodservice, the spatial organization of the sitting, various internal decorations, and the suitability of the sound environment, which are essential to enhanced gratification, dictate the cafeteria atmosphere [33]. Chang and Suki [28] also added that cafeteria design affects the students' food decisions and eating behaviors.

In addition, McLachlan and Justice [34] explained that food selection was found to be an important part of the satisfaction of international students. It was defined as food supplied in on-campus foodservice that matches the culture, religion, taste, etc., of international students. Raman and Chinniah [27] also noted that the variety and diversity of food and comfort in university cafeterias gives students a sense of "home" while on campus. Nevertheless, empirical investigations on international students' perception of foodservice attributes in Malaysian research universities are relatively few. Therefore, this study aimed to demonstrate which foodservice attributes need to be strategically planned and improved as perceived by international students. Foodservice attributes in terms of interactional quality, food quality, atmospheric quality, food price, and food selection are employed in this study.

## *2.2. International Students' University Life Adaptation and its Relation to University Foodservice*

International students are subjected to numerous changes and can struggle to adjust in a number of ways [35]. According to Andrade [36], international students experience more fear, stress, homesickness, loneliness, and lack of social support than domestic students. Thomas and Althen [37] also added that international students may experience a variety of transition issues, including depression/helplessness, hostility toward the host country, anxiety, over-identification with the home country, isolation, homesickness, and loneliness. Transitioning from parental care to independent living entails many novel decision-making experiences and choices, which can have a long-term influence on eating habits and health condition for students [38]. Lin and Yi [39], in agreement with the literature, reported that many foreign Asian students encounter the following when they study abroad: academic distress, language differences, financial concerns, performance anxiety, depression, loneliness, homesickness, relationship issues, non-assertiveness, individualism and bicultural tension, stereotyping, racism, bigotry, and fear. They may have anxieties about readjustment if they are intending to return to their own country. Specifically, there are several categories summarized by Tseng and Newton [40] as the main challenges encountered by international students: socio-cultural adjustment, personal psychological adjustment,

academic adjustment, and general living adjustment. Socio-cultural change leads to culture shock, injustice, and adjusting to unfamiliar traditions, expectations, festivals, and rituals. Personal psychological adjustment is characterized by feelings of depression, solitude, alienation, homesickness, and dissatisfaction. Weather/climate adaptation, transportation systems, bill payment, health care systems, accommodation conditions, and food are examples of general living adjustments.

It has been argued that institutions should play a constructive approach in embracing and supporting global scholars in their academic and social adjustment. The more universities recognize the demands of foreign students, the more effective their students will be [41]. Furthermore, a satisfied student population is a source of favorable outcomes, such as spreading positive word of mouth and recommending the university to others. It can be impossible to obtain overall satisfaction if a negative attitude is formed; this could result in complaints, decreasing loyalty, and negative promotion by word of mouth [42]. Of particular interest to the present study is the fact that international students have discovered that food is a vital aspect of adjusting to Malaysia. International students rely on ethnic dishes from their home countries to alleviate homesickness anxiety [43]. McLachlan and Justice [34] identified food disparities as a component that aids foreign students in dealing with change overload and transformation shock. Furthermore, in order to reach and help international students adapt academically and socially, organizations have to take on a proactive approach. Garg [44] stated that many university administrators are starting to recognize the concept of offering comprehensive campus facilities that are essential to foodservices. Demand for foodservice has steadily increased, especially in universities, as the number of students continues to rise. As a consequence, these higher requirements for college foodservices have placed increasing pressure on operators to satisfy students' requirements [10,45]. One of the primary objectives of universities is student satisfaction. According to Ruetzler, Taylor, and Hertzman [41], campus foodservice has a huge impact on the overall satisfaction of the student population. The canteens provide goods and services that can enhance the life of all those members of today's progressively varied university population with regard to social, financial, educational and personal fulfilment. Students expect greater levels of attention and more friendly service. For most customers, the food quality, cleanliness of the dining area, atmosphere, and freshness of the ingredients are satisfaction factors [46]. Amelia and Garg [9] also stated that the major factors of foodservice satisfaction are first impression, food quality, serving time, and employees. The satisfaction of students in university cafeterias depends entirely on the quality of the food and beverages served in cafeterias, the variety and choices of food and beverages, and fair price and value [9]. Campus administrators have stepped up their efforts to recruit and select only the most talented and experienced foodservice directors in response to this demand.

## *2.3. Importance-Performance Analysis*

Initially established by Martilla and James [47], IPA provides ideas on what products or services a company should concentrate on in order to attain customer satisfaction. It examines quality attributes according to two measures: their effectiveness (satisfaction) and their value to customers (importance). The evaluation of these two dimensions is then combined with a matrix to determine the key factors of satisfaction, propose goals for improvement, and recognize areas of over-conformity and appropriate drawbacks [48]. To simplify data analysis, the IPA method incorporates measurement of consumer-perceived performance and importance into a two-dimensional matrix [47]. Figure 1 depicts a matrix that categorizes attributes into four groups or quadrants to establish goals in allocating finite resources.

locating finite resources.

**Figure 1.** Importance-performance quadrants (Source: Matzlera, 2003). **Figure 1.** Importance-performance quadrants (Source: Matzlera et al. [48]).

Quadrant I, "keep up the good work", refers to the important strength of a product or service and the prospective competitive benefit. The characteristics in this quadrant are regarded to be successful and require continuous investment. Quadrant II is the "concentrate here" area. The attributes in this quadrant are deemed insufficient, constituting major shortcomings and risks to the product's competitiveness. Attributes in Quadrant III are deemed to be comparatively unimportant to consumers and fall under the "low priority" category. They are not doing exceptionally well or are considered to be relatively unimportant to customers. Quadrant IV is a relatively unimportant section of the matrix [48]. The "possible overkill" region includes characteristics that are of low value to consumers but work well, suggesting a possible excess of scarce resources that are being used inefficiently and should be reallocated elsewhere [48]. Quadrant I, "keep up the good work", refers to the important strength of a product or service and the prospective competitive benefit. The characteristics in this quadrant are regarded to be successful and require continuous investment. Quadrant II is the "concentrate here" area. The attributes in this quadrant are deemed insufficient, constituting major shortcomings and risks to the product's competitiveness. Attributes in Quadrant III are deemed to be comparatively unimportant to consumers and fall under the "low priority" category. They are not doing exceptionally well or are considered to be relatively unimportant to customers. Quadrant IV is a relatively unimportant section of the matrix [48]. The "possible overkill" region includes characteristics that are of low value to consumers but work well, suggesting a possible excess of scarce resources that are being used inefficiently and should be reallocated elsewhere [48].

a matrix that categorizes attributes into four groups or quadrants to establish goals in al-

Previous studies [49–52] noted that the IPA method is a tool to assess the performance of the service factor level, which measures the importance and satisfaction of the service innovation process. The importance of measuring and managing customer satisfaction in foodservice is widely acknowledged [53]. The critical factors determining satisfaction and importance must be identified. The best way to attain the greatest level of satisfaction is to decide how scarce resources are deployed. The IPA method is seen as an efficient way to set priorities. The tool appears to be an easy and effective instrument in practice [54]. The decision on how best to allocate scarce funds to maximize satisfaction is very helpful. To maximize fulfillment, IPA is very helpful to decide how best to allot scarce resources. For example, Tzeng and Chang [55] used IPA to explore restaurant service quality. The results of their study provided the direction to improve the service quality in the foodservice industry. Gemmel [56] argued that the IPA technique can help administrators with improvements in order to achieve high customer satisfaction with service quality. Previous studies [49–52] noted that the IPA method is a tool to assess the performance of the service factor level, which measures the importance and satisfaction of the service innovation process. The importance of measuring and managing customer satisfaction in foodservice is widely acknowledged [53]. The critical factors determining satisfaction and importance must be identified. The best way to attain the greatest level of satisfaction is to decide how scarce resources are deployed. The IPA method is seen as an efficient way to set priorities. The tool appears to be an easy and effective instrument in practice [54]. The decision on how best to allocate scarce funds to maximize satisfaction is very helpful. To maximize fulfillment, IPA is very helpful to decide how best to allot scarce resources. For example, Tzeng and Chang [55] used IPA to explore restaurant service quality. The results of their study provided the direction to improve the service quality in the foodservice industry. Gemmel [56] argued that the IPA technique can help administrators with improvements in order to achieve high customer satisfaction with service quality.

## **3. Method**

## *3.1. Measures and Development of Survey Instrument*

The research instrument of this study was a self-administered questionnaire. The questionnaire consisted of demographic questions and relevant measures. The "interactional quality" variable consisted of seven items, while six items were used to measure the "food quality" variable. The "atmospheric quality" variable was composed of six items. In addi-

tion, there were three questions measuring the "food price" variable. The four variables were adapted from El-Said and Fathy [12] and Liang and Zhang [15]. The "food selection" variable consisted of five items, adjusted from Ruetzler, Taylor, and Hertzman [41]. Furthermore, all the measurement items were measured on a 7-point scale, ranging from level 1, which represented "very dissatisfied", to level 7, which signified "very satisfied". Another 7-point scale ranged from 1, which represented "very unimportant", to 7, which signified "very important".

The mean scores of importance and satisfaction for foodservice attributes were obtained. These values then were used to analyze each element in the importance satisfaction grid as a plot. The vertical axis showed satisfaction of international students with the performance of service and service providers on a given attribute, ranging from highly dissatisfied to highly satisfied (7-point Likert scale). The horizontal axis, ranging from very unimportant to very important (7-point Likert scale), represented the importance of the attribute to international students. Satisfaction and importance scores of students were plotted on the IPA grid, which was constructed as the crossing point between mean values of the attribute ratings of importance and satisfaction. Four quadrants were generated by IPA, each with a different marketing strategy. Based on the quadrant in which it fell, each attribute was evaluated.

IPA plot interpretations followed each attribute's composite of importance and satisfaction scores. For instance, attributes high in importance and high in satisfaction proposed that university foodservice was doing a decent task and could continue to assign resources to these attributes. There may be little strategic advantage for additional investment. Particular attention should be paid to attributes of high importance and low satisfaction. A large amount of resources should be invested in improving the efficiency of the attributes. Lastly, low value and high satisfaction attributes could be protected by the university foodservice, but not explicitly allocated any extra funding.

## *3.2. Validity and Reliability*

By using SPSS 20 for Windows, the validity analysis of this study was carried out by computing the correlations between individual items and the score sum of their respective construct. All the correlation coefficients between the items and their respective construct were statistically significant (*p* < 0.05). In this study, after the validity of the instrument was tested, it was distributed to subjects. The test result identified the coefficient of instrument reliability. The reliability coefficient of the instrument in this study was identified using Cronbach's alpha (α) coefficient. The reliability analysis results revealed that the measurement items of the instrument were reliable to measure the five key constructs: interactional quality, food quality, atmospheric quality, food price fairness, and food selection. The results defined that all five key constructs were reliable. The values were included in the very high reliability category (0.80 < α <1.00).

## *3.3. Data Collection and Samples*

The target population and unit analysis consisted of all full-time international postgraduate and undergraduate students enrolled in five public Malaysian Research Universities (MRUs) in Malaysia. They were Universiti Malaya (UM), Universiti Kebangsaan Malaysia (UKM), Universiti Sains Malaysia (USM), Universiti Putra Malaysia (UPM), and Universiti Teknologi Malaysia (UTM). Two stages of sampling techniques were used in this study: stratified sampling (stage 1) and purposive sampling (stage 2). The reason stratified sampling was chosen is because the population of this study was international students in MRUs. There are five separate research universities in Malaysia, and all have international students. Hence, a sample of international students from each university was taken. Subsequently, purposive sampling was done by searching the members (sample) of a group purposefully. Only subjects fit for the study purpose were selected. The inclusion criteria of the study subjects were (1) full-time undergraduate or postgraduate international student at a research university in Malaysia; (2) eating on campus at least three times

at the same foodservice outlet (e.g., cafeteria, food stall) since enrolled at the university; (3) staying in university housing (e.g., hostel, apartment, dorm) at least one semester since enrolled at the university.

The survey was carried out for a 12-week period from 3 October 2018 to 27 December 2018. The questionnaire was distributed by paper-and-pencil and online. The researcher visited the libraries, canteens, and cafeterias of UM, UKM, USM, UPM, and UTM from Monday to Wednesday from October 2018 to December 2018. The researcher was recommended by the librarians to come to the libraries, since many students were available in the libraries from Monday to Wednesday. The subjects were approached in each university's canteen/cafeteria and library and given a piece of paper that contained the online link to the survey. In addition, the researcher attempted to access subjects by seeking assistance from the members of the Indonesian Student Association in each MRU to distribute the questionnaires to international students on campus. Furthermore, the researcher sought assistance from UPM international students to distribute the link via WhatsApp groups to their community in UPM and the other four universities.

## *3.4. Demographic Information*

A total of 620 international students who were studying in MRUs were selected as the sample of this study. Satisfaction and importance means were used to determine where the coordinates of each item fell according to international student experience. The average level of satisfaction with the different elements of university foodservice attributes and the average importance for the total sample of five constructs in foodservice attributes were measured. These values were then used to analyze each element in the importance satisfaction grid as a plot. To provide an overview of international students regarding their demographics as well as the nature of their dining experiences, descriptive characteristics of the sample were examined. The respondents' demographics were analyzed in terms of their gender, home country, name of university, level of higher education, current academic year, and year enrolled in the university.

In this study, more male respondents (58.4%) participated in the survey research than female (41.6%). The respondents came from various home countries, including Indonesia (29.8%), Iraq (11.6%), Iran (10.3%), China (9.4%), Afghanistan (7.7%), and India (7.2%). In addition, there were also participants who came from Korea (6%), Saudi Arabia (5.3%), the Philippines (3.9%), Vietnam (3.4%), Qatar (1.9%), Somalia (1.1%), Egypt (1%), Sri Lanka (0.9%), Ethiopia (0.3%), and Germany (0.2%). Furthermore, the dominant international student group was from UPM (26%), followed by UM (19.8%), USM (19.4%), UTM (18.4%), and UKM (16.6%). The demographic distribution by higher education level. demonstrated that respondents with a bachelor's degree made up the majority of the total respondents (43%), followed by those who were earning a master's degree (41.6%) and a Ph.D. (15.5%). It was revealed that 30.5% of the respondents were in the second year of their program, whereas those who were in the third year made up 27.6% of the total respondents. The participants who answered "other" were 20.8%, meaning that the respondents had studied more than four years since enrolling in the program. The data showed 16.6% of respondents were in the fourth year of their program, whereas only 4.5% respondents were university students in their first year. In summary, the participants of this study were mainly undergraduate international students who were studying in one of these research universities: UM, UPM, UKM, UTM, and USM.

The on-campus eating behavior of international students was measured for the period since the students enrolled at the university. The measurement items are provided in Table 1. With regard to the frequency of visits to the on-campus foodservice outlet(s), most of the respondents (46.8%) visited the university canteen daily. Daily customers were double those who visited the foodservice outlet(s) monthly (23.2%), and monthly customers were fewer than the respondents who visited the university cafeteria(s) weekly (30%).


**Table 1.** International students' on-campus eating behavior (*n* = 620).

The findings revealed that over half of the participants (69.2%) spent their money for meals in the university cafeteria in the range of 6 MYR to 10 MYR per visit, or approximately US \$1.50 to \$2.50. Respondents who spent more than 10 MYR (US \$2.50) took second place (16.4%) followed by respondents who spent less than/equal to 5 MYR (US \$1.25) per visit. A large group of participants indicated that they dined with friends (53.5%), with fewer dining alone (46.5%). In the section asking the reason why international students eat on campus, the respondents were able to choose more than one answer. Therefore, the most significant reason to eat on campus was found to be the price (63.2%). They chose to eat on campus because the meal price was inexpensive. Another significant reason for the respondents was a lack of other options (25.5%). It is not surprising that most of the students stated that they eat with friends in the university canteen, because 9.1% of the participants selected socializing or meeting with friends in the canteen, while 1.2% of the respondents thought the location of the foodservice outlet was convenient for them. Only 0.3% of the respondents stated that "the food is good" in the university cafeteria. Participants could select the food they normally order, with more than one option for this question. The most common choices were snacks (30.2%) and meat, fish, or chicken dishes (29.6%); 10.6% of the respondents mentioned that they like to eat rice dishes, such as fried rice, at the university canteen. Therefore, in term of respondents' on-campus eating behavior, the majority of the respondents were daily consumers and they mostly came to the university foodservice outlet(s) because the price was inexpensive and there was no other option. An overview of the cafeterias in the five Malaysian Research Universities is provided in Appendix A.

## **4. Results**

## *4.1. Overall IPA Grids*

The average value of these five constructs was computed. These values were then used to draw each element on the important satisfaction grid as coordinates. Table 2 illustrates the means of importance and satisfaction (performance) from the five foodservice attributes

in this study as well as the coordinates of each foodservice attribute in the IPA grid. The variables in the table were based on the order of Quadrant I to Quadrant IV. SPSS 20 for Windows was used as a tool for the analysis.


As presented in Figure 2, the horizontal (*x*-axis) axis shows the importance attributes from low to high perceived by international students (overall importance); whereas the vertical axis (*y*-axis) shows the rate of service satisfaction for international students (overall satisfaction), ranging from low to elevated. The grid is characterized for satisfaction and importance for the five variables. As specified in Table 2, each variable was labeled for easy recognition and to shorten the name while being analyzed in SPSS 20 for Windows. The labels were also applied to each item of the variables, as shown in Table 3.

The Quadrant I component was considered to be highly important and highly satisfied. The main driver of customer satisfaction was the food price (FP), and the management task was to guarantee that college foodservice "keep up the good work". The component was high importance and low satisfaction in Quadrant II. The underperformance of food quality (FQ) located in this quadrant is regarded to reflect the main weaknesses of a product and suggests efforts should be "concentrate here".

Subsequently, the component in Quadrant III was rated of low importance and low satisfaction. Food selection (FS) was the attribute of "low priority". Apart from being comparatively unimportant, its bad performance is obviously not an issue. Efficiency should be enhanced only if Quadrant II (greater priority) does not contain the characteristics and/or if the improvements are not too expensive [35]. Components in Quadrant IV were rated of low importance and high satisfaction. Interactional quality (IQ) and atmospheric quality (AQ) could be viewed as areas of "possible overkill". They were relatively unimportant for the customers, but the service was very good at the university dining facilities. *Sustainability* **2021**, *13*, x FOR PEER REVIEW 11 of 20

**Figure 2.** IPA grid for five constructs of university foodservice attributes. **Figure 2.** IPA grid for five constructs of university foodservice attributes.

The items in Table 3 are based on the order of Quadrant I to Quadrant IV. Five IPA grids were developed in order for the five characteristics of university foodservices to be analyzed and identified in terms of weaknesses and strengths. For each attribute, the mean values of importance and satisfaction were calculated and used as the coordinates for each individual grid. The crosshairs and the positioning of the axis in the grids were developed as excellent means of importance and satisfaction for each element (see Figure 3). Hence,

The international students felt that "cleanliness and neatness of staff appearance in the foodservice outlet" (IQ1) was very important, but indicated low satisfaction with it because this item fell into QII. The respondents indicated that "respectfulness of the foodservice outlet staff" (IQ2), "speed of service in foodservice outlet" (IQ5), and "staff knowledge of the food items sold in foodservice outlet" (IQ6) were very important, and the students were satisfied with the service, as the items fell into QI. If the items in QI are further provided, then the customer can suppose that an organization performs excellent work with a highly significant attribute [57]. The foodservice staff should maintain their excellent job in offering friendly service. "Professionalism in service delivery in foodservice outlet" (IQ4) and "Easiness talking staff (able to answer my question)" (IQ7) indicated no particular intervention was required, because the international students indicated these were of low priority, with the items falling into Quadrant III (QIII). The items appear to be not an issue because their poor performance is comparatively irrelevant. "Friendliness of the foodservice outlet staff" (IQ3) registered in the low importance high satisfaction quadrant (QIV), indicating a potential overuse of efforts and resources expended on this item. In this quadrant, the items were relatively unimportant for international students, but the university foodservice did exceptionally well at providing them. Administrators may choose to reassign resources to Quadrant II. Figure 3 shows the IPA grid of

there were different crossing points for the five IPA grids.

*4.2. Individual IPA Grids*

4.2.1. Interactional Quality

interactional quality items.


**Table 3.** Importance-performance means for 27 university foodservice attributes.

## *4.2. Individual IPA Grids*

Each of the five characteristics represent the various component elements. The mean value of importance and satisfaction of each attribute for the 27 items is shown in Table 3. The items in Table 3 are based on the order of Quadrant I to Quadrant IV. Five IPA grids were developed in order for the five characteristics of university foodservices to be analyzed and identified in terms of weaknesses and strengths. For each attribute, the mean values of importance and satisfaction were calculated and used as the coordinates for each individual grid. The crosshairs and the positioning of the axis in the grids were developed as excellent means of importance and satisfaction for each element (see Figure 3). Hence, there were different crossing points for the five IPA grids.

## (1) IPA grid for "interactional quality"

## (2) IPA grid for "food quality"

## **Note:**


**Note:** 


## **Note:**


**Figure 3.** *Cont.*

## (4) IPA grid for "food price"

(5) IPA grid for "food selection"

**Note:**

FP1: The announcement of food price changes

FP2: The reasonableness of the quality of food items provided for the price paid

FP3: The reasonableness of the quantity of food items provided for the price paid

**Note:**


**Figure 3.** Individual IPA grid for items of university foodservice attributes.

## **Figure 3.** Individual IPA grid for items of university foodservice attributes. 4.2.1. Interactional Quality

**5. Discussion and Implications** In accordance with the results of this study, food price attribute was the key driver of international student satisfaction, and management's job is to ensure that the university foodservice "keeps up the good work". Two of three foodservice components, "The announcement of food price changes (FP1)" and "The reasonableness of the quality of food items provided with the price paid (FP3)" emerged as the most satisfactory factors, indicating that international students held highly positive perceptions of university foodservice in these areas. These two components were highly important in attracting international students to eat on campus. Thus, the food prices are critical elements affecting international students' satisfaction. Foodservice operators are advised to focus on these aspects in their promotional efforts of university foodservice in order to capitalize on their importance. Based on the importance–performance analysis, food quality attributes emerged as very important, but international students were not satisfied. Food quality situated in Quadrant II was considered to be underperforming and, as such, represents the product's major weaknesses. Hence, the managers must concentrate on this attribute. "Nutritional benefits of food items (FQ1)", "Appropriateness of food flavor (FQ2)", and The international students felt that "cleanliness and neatness of staff appearance in the foodservice outlet" (IQ1) was very important, but indicated low satisfaction with it because this item fell into QII. The respondents indicated that "respectfulness of the foodservice outlet staff" (IQ2), "speed of service in foodservice outlet" (IQ5), and "staff knowledge of the food items sold in foodservice outlet" (IQ6) were very important, and the students were satisfied with the service, as the items fell into QI. If the items in QI are further provided, then the customer can suppose that an organization performs excellent work with a highly significant attribute [57]. The foodservice staff should maintain their excellent job in offering friendly service. "Professionalism in service delivery in foodservice outlet" (IQ4) and "Easiness talking staff (able to answer my question)" (IQ7) indicated no particular intervention was required, because the international students indicated these were of low priority, with the items falling into Quadrant III (QIII). The items appear to be not an issue because their poor performance is comparatively irrelevant. "Friendliness of the foodservice outlet staff" (IQ3) registered in the low importance high satisfaction quadrant (QIV), indicating a potential overuse of efforts and resources expended on this item. In this quadrant, the items were relatively unimportant for international students, but the university foodservice did exceptionally well at providing them. Administrators may

choose to reassign resources to Quadrant II. Figure 3 shows the IPA grid of interactional quality items.

## 4.2.2. Food Quality

As seen in Figure 3, three items fell into the "concentrate here" quadrant (QII); "Nutritional benefits of food items" (FQ1), "Appropriateness of food flavor" (FQ2), and "Food hygiene and safety" (FQ5) were crucial but students were comparatively unhappy with these items. It can be suggested that attention is needed to improve the satisfaction of international students with these features. "Freshness of food ingredients" (FQ4), which indicates elevated efficiency and high priorities, was situated in the "keep up the good work" quadrant (QI). "Attractiveness of food items displayed" (FQ3) fell into the "possible overkill" quadrant (QIV), showing high satisfaction but low importance. According to this result, one could consider that too much emphasis is placed on this item, considering it is not important to international students. International students were not satisfied with "Appropriateness of food temperature" (FQ6) in university foodservice. However, it was a low priority, since this was also a relatively low priority for students.

## 4.2.3. Atmospheric Quality

According to Figure 3, international students were less satisfied with "Cleanliness of the foodservice outlets" (AQ5) than with other elements. This dimension was considered by students to be essential, but the output was relatively small. This indicates that efforts for enhancement should be focused on this element. There were three items indicated in "low priority" quadrant (QIII): "Decoration of the foodservice outlet" (AQ3), "Noise level" (AQ4), and "Foodservice outlet entertainment, such as music" (AQ6). Under the "keep up the good work" category were "Seat availability" (AQ1) and "Convenience of service hours" (AQ2). No component was located in the "possible overkill" quadrant in this attribute.

## 4.2.4. Food Price

As shown in Figure 3, two items of food price attributes were in the "keep up the good work" quadrant: "The announcement of food price changes" (FP1) and "The reasonableness of the quality of food items provided for the price paid" (FP2). These characteristics were highly important and satisfactory for global students. While "The reasonableness of the quantity of food items provided for the price paid" (FP3) was rated comparatively poor, satisfaction was also rated insufficient. These characteristics did not involve any extra resources because of the small salience; if resources were limited, they could be used if needed in different fields that were more relevant.

## 4.2.5. Food Selection

As depicted in Figure 3, "The variety of foods served" (FS2) and "The choices of foods permit me to meet my religious needs" (FS3) were located in Quadrant I. International students felt these items were both important and satisfactory; management ensured that these fell into the category of "keep up the good work". "The choices of food meet my cultural preferences" (FS1) was located in Quadrant II. This feature was very important but the respondents felt unsatisfied, which represented a priority for management action. The administrators of university foodservice should concentrate on this point. This food selection item was deemed important, but international students were only slightly satisfied with it. The management therefore required special attention for this point. An additional two items were shown in Quadrant III: "The selection of food makes me feel at home" (FS4) and "The selection of food is similar to that in my home country" (FS5).

## **5. Discussion and Implications**

In accordance with the results of this study, food price attribute was the key driver of international student satisfaction, and management's job is to ensure that the university

foodservice "keeps up the good work". Two of three foodservice components, "The announcement of food price changes (FP1)" and "The reasonableness of the quality of food items provided with the price paid (FP3)" emerged as the most satisfactory factors, indicating that international students held highly positive perceptions of university foodservice in these areas. These two components were highly important in attracting international students to eat on campus. Thus, the food prices are critical elements affecting international students' satisfaction. Foodservice operators are advised to focus on these aspects in their promotional efforts of university foodservice in order to capitalize on their importance. Based on the importance–performance analysis, food quality attributes emerged as very important, but international students were not satisfied. Food quality situated in Quadrant II was considered to be underperforming and, as such, represents the product's major weaknesses. Hence, the managers must concentrate on this attribute. "Nutritional benefits of food items (FQ1)", "Appropriateness of food flavor (FQ2)", and "Food hygiene and safety (FQ5)" were three components of food quality that were very important, but international students were less satisfied with these items. It can be suggested that these items should be taken into consideration for cafeteria administrators and can be addressed by providing more vegetables dishes, the availability of juices, etc. University foodservice managers can collaborate with food science and technology departments in their university to create a variety of healthy foods with reasonable prices in the university cafeteria.

On the other hand, "Freshness of food ingredients (FQ4)" was located in "keep up the good work" quadrant, indicating high performance and high priority. The participants felt that this dimension was crucial, and the university operators seemed to perform very effectively in its delivery. The implication is that continued resources should be directed towards improving this item. The college foodservice providers should guarantee that ingredients used for student meals are checked periodically in the college cafeteria for each kitchen counter. The checked components can consist of the expiry date, freshness of the vegetables/spices and herbs, refrigeration methods used in the cafeteria, etc. Meanwhile, two foodservice attributes were relatively unimportant to the international students, but the university foodservice performed very well; these were interactional quality and atmospheric quality. This suggests a possible over-employment of efforts and resources being spent on these attributes. Management may wish to reassign resources to the attributes associated with food quality. In interactional quality items, the foodservice providers must consider the cleanliness and neatness of staff. This item was indicated as very important but had low satisfaction by international students. The findings suggest that special attention is needed for these highly significant performance foodservice components. University foodservice operators need to improve upon these features. The operators can provide training to university foodservice staff in order to increase their knowledge of hygiene and safety. In addition, operators also can conduct audits of the staff.

Meanwhile, in atmospheric quality components, international students deemed "Cleanliness of the foodservice outlets (IQ1)" particularly important yet were less satisfied with it relative to other items. Foodservice managers in the MRUs should pay particular attention to improving international students' satisfaction with this item because it was positioned in the quadrant of "concentrate here". This aspect was regarded as very important for international students, but the production was comparatively low. This should be the focus of improvement initiatives. Foodservice operators should conduct a regular inspection of the appearance of their staff, including the use of gloves when serving meals, the use of safety shoes, the use of aprons, wearing cooking caps to avoid hair falling in the meals, etc. Nevertheless, "keep up the good work" was achieved for "Seat availability" (AQ1) and "Convenience of service hours" (AQ2). The items in Quadrant I are usually concerned with the operation of foodstuffs and should continue to improve the efficiency level. It would therefore be realistic for university foodservice managers to keep on focusing on these attributes in foodservice development on campus. For the food selection components, international students felt they were both important and satisfactory in terms of food variety served and food choices to meet their religious needs. The management should ensure

these items continue to "keep up the good work". Nonetheless, the international students felt that food choices that met their cultural preferences were very important, but they were unsatisfied with the performance. Thus, the university foodservice administrators should focus on this, including increasing the variety of food (e.g., shawarma, takoyaki, sushi, tacos, etc.) to reflect more diversified ethnicity and age groups and to support students with special dietary needs and providing a variety of foods from various countries. The university foodservice operators can collaborate with the international student association and food departments to create, innovate, and provide food choices that can be accepted by international students and be reasonably priced.

The findings are useful in forecasting international student satisfaction and offering good empirical evidence for the importance–performance analysis, which assumes that attribute performance influences international student satisfaction less when the attribute's self-stated significance is poor. If international student satisfaction is highly influenced by attribute performance, the attribute's self-assessed value is high. The findings of this research provide a better understanding of the international students' satisfaction with oncampus foodservice in MRUs. Results from IPA also assist in determining how international students perceive the quality of the key competencies of on-campus foodservices and identify fields in which improvements are further required. The results also are useful in determining how to enhance international students' satisfaction with foodservice attributes in the university.

Overcoming these limitations can provide guidance for future research. First, Indonesian students made up the majority of those who participated in this research. The result could be a non-response factor, which was implemented owing to a lack of representation in the population of these non-respondents. It was possible that most of the perception was based on differences in taste or culture. This may lead to an upward bias in responses. Consequently, the findings of studies on the population of interest should be implemented to counter these limitations. Second, international students in this study came from different cultures around the world. Culture can affect consumer thinking and perceptions, as Li [46] has indicated. For this reason, future research may seek to determine how the relations between the studies differ in terms of service delivery and students of distinct backgrounds and cultural variables. This research is an exciting expansion. Furthermore, the practical data acquired will certainly help food operators in understanding global students' distinctive requirements. Third, the research results may not be generalizable outside this population, because only international students from Malaysian research universities were included in the study population. It is critical that comparative studies be repeated at other Malaysian universities in order to increase generalization. However, the results of the study do not guarantee the same outcomes when various conditions for allocations of different international student population groups are applied. Approved by any clear findings regarding alternative population segmentation criteria, similar assessments must be made for comparative reasons.

## **6. Conclusions**

The empirical findings of this research demonstrated that interactional quality, food quality, atmospheric quality, food price, and food selection attributes in each IPA quadrant were positioned by the mean values of importance and performance. The results are effective in predicting international student satisfaction and provide strong empirical support for the importance–performance analysis, which assumes that international student satisfaction will be influenced less by attribute performance when the self-stated importance of the attribute is low. Where international student satisfaction is influenced by attribute performance, the self-stated importance of the attribute is high. According to the results of the importance–performance analysis, food price was the key driver of international student satisfaction, since the students felt that food price attribute was important and they were satisfied with the attribute. Unfortunately, food quality was considered to be underperforming and, as such, represents the product's major weakness.

Thus, this attribute should be addressed by foodservice managers. Subsequently, food selection was an attribute of low priority. Interactional quality and atmospheric quality could be viewed as areas of performance that were relatively unimportant to international students; however, the university foodservice performed very well in these areas. The importance–performance analysis technique examined international student satisfaction with on-campus foodservice. The findings of this research provide better understanding of international students' satisfaction with on-campus foodservice in MRUs, which can help to meet university quality standards. Results from the IPA can also help to determine how international students perceive the quality of the key competencies of on-campus foodservices and identify fields in which improvements are further required. *Sustainability* **2021**, *13*, x FOR PEER REVIEW 17 of 20 **Author Contributions:** Conceptualization, A.Z.A. and B.-L.C.; methodology, A.Z.A. and B.-L.C.; *Sustainability* **2021**, *13*, x FOR PEER REVIEW 17 of 20 **Author Contributions:** Conceptualization, A.Z.A. and B.-L.C.; methodology, A.Z.A. and B.-L.C.;

> **Author Contributions:** Conceptualization, A.Z.A. and B.-L.C.; methodology, A.Z.A. and B.-L.C.; writing—original draft preparation, A.Z.A.; writing—review and editing, B.-L.C. and H.H.; visualization, A.Z.A.; supervision, H.H. and A.R.; project administration, A.Z.A. and B.-L.C. All authors have read and agreed to the published version of the manuscript. writing—original draft preparation, A.Z.A.; writing—review and editing, B.-L.C. and H.H.; visualization, A.Z.A.; supervision, H.H. and A.R.; project administration, A.Z.A. and B.-L.C. All authors have read and agreed to the published version of the manuscript. **Funding:** This research received no external funding. writing—original draft preparation, A.Z.A.; writing—review and editing, B.-L.C. and H.H.; visualization, A.Z.A.; supervision, H.H. and A.R.; project administration, A.Z.A. and B.-L.C. All authors have read and agreed to the published version of the manuscript. **Funding:** This research received no external funding.

**Funding:** This research received no external funding.

**Institutional Review Board Statement:** Because of the observational nature of the study, and in the absence of any involvement of therapeutic medication, no formal approval of the institutional review board of the local ethics committee was required. Nonetheless, all subjects were informed about the study and participation was on a voluntary basis. The study was conducted in accordance with the Declaration of Helsinki. **Institutional Review Board Statement:** Because of the observational nature of the study, and in the absence of any involvement of therapeutic medication, no formal approval of the institutional review board of the local ethics committee was required. Nonetheless, all subjects were informed about the study and participation was on a voluntary basis. The study was conducted in accordance with the Declaration of Helsinki. **Institutional Review Board Statement:** Because of the observational nature of the study, and in the absence of any involvement of therapeutic medication, no formal approval of the institutional review board of the local ethics committee was required. Nonetheless, all subjects were informed about the study and participation was on a voluntary basis. The study was conducted in accordance with the Declaration of Helsinki.

**Informed Consent Statement:** Not applicable. **Informed Consent Statement:** Not applicable. **Informed Consent Statement:** Not applicable.

**Conflicts of Interest:** The authors declare no conflict of interest. **Conflicts of Interest:** The authors declare no conflict of interest. **Conflicts of Interest:** The authors declare no conflict of interest.

### **Appendix A Appendix A Appendix A**

**Image 1. Cafeteria of Universiti Malaya (UM) Image 1. Cafeteria of Universiti Malaya (UM) Image 1. Cafeteria of Universiti Malaya (UM)**

Source from http://www.ummc.edu.my (accessed on 26 May 2021) Source from http://www.ummc.edu.my (accessed on 26 May 2021)

Source from http://www.ummc.edu.my (accessed on 26 May 2021)

Source from http://www.ukm.edu.my (accessed on 26 May 2021) Source from http://www.ukm.edu.my (accessed on 26 May 2021) **Image 3. Cafeteria of Universiti Sains Malaysia (USM)** Subaidah Restaurant Subaidah Restaurant

Source from http://www.ukm.edu.my (accessed on 26 May 2021) Source from http://www.ukm.edu.my (accessed on 26 May 2021) **Image 3. Cafeteria of Universiti Sains Malaysia (USM)** Subaidah Restaurant Subaidah Restaurant

**Image 2. Cafeteria of Universiti Kebangsaan Malaysia (UKM)** Cafeteria Pusanika Pendeta Zaba College Cafeteria

**Image 2. Cafeteria of Universiti Kebangsaan Malaysia (UKM)** Cafeteria Pusanika Pendeta Zaba College Cafeteria

Source from http://www.ummc.edu.my (accessed on 26 May

Source from http://www.ummc.edu.my (accessed on 26 May

Source from http://www.ukm.edu.my (accessed on 26 May 2021)

### **Image 2. Cafeteria of Universiti Kebangsaan Malaysia (UKM)** 2021) 2021) **Image 2. Cafeteria of Universiti Kebangsaan Malaysia (UKM)** 2021) 2021) **Image 2. Cafeteria of Universiti Kebangsaan Malaysia (UKM)**

*Sustainability* **2021**, *13*, x FOR PEER REVIEW 17 of 20

*Sustainability* **2021**, *13*, x FOR PEER REVIEW 17 of 20

have read and agreed to the published version of the manuscript.

have read and agreed to the published version of the manuscript.

**Conflicts of Interest:** The authors declare no conflict of interest.

**Conflicts of Interest:** The authors declare no conflict of interest.

**Image 1. Cafeteria of Universiti Malaya (UM)** TokMad Cafeteria TokMad Cafeteria

**Image 1. Cafeteria of Universiti Malaya (UM)** TokMad Cafeteria TokMad Cafeteria

**Funding:** This research received no external funding.

**Funding:** This research received no external funding.

**Informed Consent Statement:** Not applicable.

**Informed Consent Statement:** Not applicable.

with the Declaration of Helsinki.

with the Declaration of Helsinki.

**Appendix A**

**Appendix A**

**Author Contributions:** Conceptualization, A.Z.A. and B.-L.C.; methodology, A.Z.A. and B.-L.C.; writing—original draft preparation, A.Z.A.; writing—review and editing, B.-L.C. and H.H.; visualization, A.Z.A.; supervision, H.H. and A.R.; project administration, A.Z.A. and B.-L.C. All authors

**Author Contributions:** Conceptualization, A.Z.A. and B.-L.C.; methodology, A.Z.A. and B.-L.C.; writing—original draft preparation, A.Z.A.; writing—review and editing, B.-L.C. and H.H.; visualization, A.Z.A.; supervision, H.H. and A.R.; project administration, A.Z.A. and B.-L.C. All authors

**Institutional Review Board Statement:** Because of the observational nature of the study, and in the absence of any involvement of therapeutic medication, no formal approval of the institutional review board of the local ethics committee was required. Nonetheless, all subjects were informed about the study and participation was on a voluntary basis. The study was conducted in accordance

**Institutional Review Board Statement:** Because of the observational nature of the study, and in the absence of any involvement of therapeutic medication, no formal approval of the institutional review board of the local ethics committee was required. Nonetheless, all subjects were informed about the study and participation was on a voluntary basis. The study was conducted in accordance

Pendeta Zaba College Cafeteria

Source from http://www.ummc.edu.my (accessed on 26 May

Source from http://www.ummc.edu.my (accessed on 26 May

Source from http://www.ukm.edu.my (accessed on 26 May 2021) *Sustainability* **2021**, *13*, x FOR PEER REVIEW 18 of 20 *Sustainability* **2021**, *13*, x FOR PEER REVIEW 18 of 20

**Image 3. Cafeteria of Universiti Sains Malaysia (USM)** *Sustainability* **2021**, *13*, x FOR PEER REVIEW 18 of 20 *Sustainability* **2021**, *13*, x FOR PEER REVIEW 18 of 20

Source from https://news.usm.my (accessed on 26 May 2021) Source from https://news.usm.my (accessed on 26 May 2021) Source from https://news.usm.my (accessed on 26 May 2021) Source from https://news.usm.my (accessed on 26 May 2021) Source from https://news.usm.my (accessed on 26 May 2021)

Source from https://news.usm.my (accessed on 26 May 2021) Source from https://news.usm.my (accessed on 26 May 2021) Source from https://news.usm.my (accessed on 26 May 2021)

**Image 4. Cafeteria of Universiti Putra Malaysia (UPM)** Kolej 13 Cafeteria Kolej 16 Cafeteria **Image 4. Cafeteria of Universiti Putra Malaysia (UPM) Image 4. Cafeteria of Universiti Putra Malaysia (UPM) Image 4. Cafeteria of Universiti Putra Malaysia (UPM) Image 4. Cafeteria of Universiti Putra Malaysia (UPM)**

Kolej 13 Cafeteria Kolej 16 Cafeteria Kolej 13 Cafeteria

**Image 5. Cafeteria of Universiti Teknologi Malaysia (UTM)** Richiamo Cafe UTM Cafeteria Source from https://k13.upm.edu.my (accessed on 26 May 2021)

**References**

**References**

**References**

**References**

doi:10.1108/13555850910997599.

doi:10.1108/13555850910997599.

ysis. *Int. J. Appl. Psychol.* **2013**, *3*, 19–24.

ysis. *Int. J. Appl. Psychol.* **2013**, *3*, 19–24.

Nigeria. *Dimensions* **2014**, *4*, 147–156.

Nigeria. *Dimensions* **2014**, *4*, 147–156.

doi:10.1016/j.ijhm.2005.01.001.

doi:10.1016/j.ijhm.2005.01.001.

doi:10.1108/13555850910997599.

doi:10.1108/13555850910997599.

ysis. *Int. J. Appl. Psychol.* **2013**, *3*, 19–24.

ysis. *Int. J. Appl. Psychol.* **2013**, *3*, 19–24.

Nigeria. *Dimensions* **2014**, *4*, 147–156.

Nigeria. *Dimensions* **2014**, *4*, 147–156.

doi:10.1016/j.ijhm.2005.01.001.

doi:10.1016/j.ijhm.2005.01.001.

Kolej 13 Cafeteria Kolej 16 Cafeteria Kolej 16 Cafeteria

**Image 5. Cafeteria of Universiti Teknologi Malaysia (UTM)** Richiamo Cafe UTM Cafeteria Source from https://k16.upm.edu.my (accessed on 26 May 2021)

Source from https://www.utm.my (accessed on 26 May 2021) Source from https://www.utm.my (accessed on 26 May 2021)

Source from https://www.utm.my (accessed on 26 May 2021) Source from https://www.utm.my (accessed on 26 May 2021)

Source from https://www.utm.my (accessed on 26 May 2021) Source from https://www.utm.my (accessed on 26 May 2021)

Source from https://www.utm.my (accessed on 26 May 2021) Source from https://www.utm.my (accessed on 26 May 2021)

1. Singh, J.K.N.; Schapper, J.; Jack, G. The importance of place for international students' choice of university: A case study at a

1. Singh, J.K.N.; Schapper, J.; Jack, G. The importance of place for international students' choice of university: A case study at a

2. Arambewela, R.; Hall, J. An empirical model of international student satisfaction. *Asia Pac. J. Mark. Logist.* **2013**, *21*, 555–569,

2. Arambewela, R.; Hall, J. An empirical model of international student satisfaction. *Asia Pac. J. Mark. Logist.* **2013**, *21*, 555–569,

3. Khairani, A.Z.; Razak, N.B. Assessing factors influencing students' choice of Malaysian Public University: A Rasch model anal-

3. Khairani, A.Z.; Razak, N.B. Assessing factors influencing students' choice of Malaysian Public University: A Rasch model anal-

4. Lee, K.-W.; Yuan, J.; Hwang, J.-S.; Kim, H.-S. Doctoral students' selection intention in Hospitality and Tourism Management (HTM) programs in the United States. *J. Hosp. Leis. Sport Tour. Educ.* **2012**, *11*, 140–150, doi:10.1016/j.jhlste.2012.03.001. 5. Ezeokoli, R.N.; Ayodele, K.O. Dimensions of service quality encountered by students on sustainability of higher education in

4. Lee, K.-W.; Yuan, J.; Hwang, J.-S.; Kim, H.-S. Doctoral students' selection intention in Hospitality and Tourism Management (HTM) programs in the United States. *J. Hosp. Leis. Sport Tour. Educ.* **2012**, *11*, 140–150, doi:10.1016/j.jhlste.2012.03.001. 5. Ezeokoli, R.N.; Ayodele, K.O. Dimensions of service quality encountered by students on sustainability of higher education in

6. Klassen, K.; Trybus, E.; Kumar, A. Planning food services for a campus setting. *Int. J. Hosp. Manag.* **2005**, *24*, 579–609,

6. Klassen, K.; Trybus, E.; Kumar, A. Planning food services for a campus setting. *Int. J. Hosp. Manag.* **2005**, *24*, 579–609,

1. Singh, J.K.N.; Schapper, J.; Jack, G. The importance of place for international students' choice of university: A case study at a

1. Singh, J.K.N.; Schapper, J.; Jack, G. The importance of place for international students' choice of university: A case study at a

2. Arambewela, R.; Hall, J. An empirical model of international student satisfaction. *Asia Pac. J. Mark. Logist.* **2013**, *21*, 555–569,

2. Arambewela, R.; Hall, J. An empirical model of international student satisfaction. *Asia Pac. J. Mark. Logist.* **2013**, *21*, 555–569,

3. Khairani, A.Z.; Razak, N.B. Assessing factors influencing students' choice of Malaysian Public University: A Rasch model anal-

3. Khairani, A.Z.; Razak, N.B. Assessing factors influencing students' choice of Malaysian Public University: A Rasch model anal-

4. Lee, K.-W.; Yuan, J.; Hwang, J.-S.; Kim, H.-S. Doctoral students' selection intention in Hospitality and Tourism Management (HTM) programs in the United States. *J. Hosp. Leis. Sport Tour. Educ.* **2012**, *11*, 140–150, doi:10.1016/j.jhlste.2012.03.001. 5. Ezeokoli, R.N.; Ayodele, K.O. Dimensions of service quality encountered by students on sustainability of higher education in

4. Lee, K.-W.; Yuan, J.; Hwang, J.-S.; Kim, H.-S. Doctoral students' selection intention in Hospitality and Tourism Management (HTM) programs in the United States. *J. Hosp. Leis. Sport Tour. Educ.* **2012**, *11*, 140–150, doi:10.1016/j.jhlste.2012.03.001. 5. Ezeokoli, R.N.; Ayodele, K.O. Dimensions of service quality encountered by students on sustainability of higher education in

6. Klassen, K.; Trybus, E.; Kumar, A. Planning food services for a campus setting. *Int. J. Hosp. Manag.* **2005**, *24*, 579–609,

6. Klassen, K.; Trybus, E.; Kumar, A. Planning food services for a campus setting. *Int. J. Hosp. Manag.* **2005**, *24*, 579–609,

Malaysian university. *J. Stud. Int. Educ.* **2014**, *18*, 463–474, doi:10.1177/1028315314523990.

Malaysian university. *J. Stud. Int. Educ.* **2014**, *18*, 463–474, doi:10.1177/1028315314523990.

Malaysian university. *J. Stud. Int. Educ.* **2014**, *18*, 463–474, doi:10.1177/1028315314523990.

Malaysian university. *J. Stud. Int. Educ.* **2014**, *18*, 463–474, doi:10.1177/1028315314523990.
