Next Article in Journal
Financial Analysis of Habitat Conservation Banking in California
Next Article in Special Issue
Role of the Universities as Drivers of Social Innovation
Previous Article in Journal
Private Sector Engagement in the Self-Governance of Urban Sustainable Infrastructure: A Study on Alternative Fueling Infrastructure in the United States
Previous Article in Special Issue
Promotion of Social Innovation through Fab Labs. The Case of ProteinLab UTEM in Chile
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Scientific Competence in Developing Countries: Determinants and Relationship to the Environment

by
José Mauricio Chávez Charro
1,
Isabel Neira
1 and
Maricruz Lacalle-Calderon
2,*
1
Quantitative Economics Department, Universidad de Santiago de Compostela, 15705 Santiago de Compostela, Spain
2
Economic Development Department, Universidad Autónoma de Madrid, 28049 Madrid, Spain
*
Author to whom correspondence should be addressed.
Sustainability 2021, 13(22), 12439; https://doi.org/10.3390/su132212439
Submission received: 12 October 2021 / Revised: 31 October 2021 / Accepted: 1 November 2021 / Published: 11 November 2021

Abstract

:
In 2015, the United Nations General Assembly adopted Agenda 2030 to guarantee sustainable, peaceful, prosperous, and just life, establishing 17 Sustainable Development Goals (SDGs). According to this declaration, pursuing the path of sustainable development requires a profound transformation in how we think and act. People must have scientific competences—not only knowledge of science, but also skills, values, and attitudes toward science that enable them to contribute to the goals proposed. This overall approach, known as Education for Sustainable Development (EDS), is crucial to achieving the SDGs. Scientific competences not only depend on what students learn in their countries’ formal education systems but also on other factors in the environment in which the students live. This study aims to identify the factors that determine scientific competence in students in developing countries, paying special attention to the social and cultural capital and the environmental conditions in the environment in which they live. To achieve this goal, we used data provided by PISA-D in the participating countries—Cambodia, Ecuador, Guatemala, Honduras, Paraguay, and Senegal—and multilevel linear modelling. The results enable us to conclude that achieving scientific competence also depends on the social and cultural capital of the student’s family and on the cultural and social capital of the schools. The higher the score in these forms of capital, the greater the achievement in sciences.

1. Introduction

In 2015, the United Nations General Assembly adopted Agenda 2030 to guarantee sustainable, peaceful, prosperous, and just life, establishing 17 Sustainable Development Goals (SDGs) [1]. According to this declaration, pursuing the path of sustainable development requires a profound transformation in how we think and act. People must have scientific competences—not only knowledge of science but also skills, values, and attitudes toward science that enable them to contribute to the goals proposed. This overall approach, known as Education for Sustainable Development (EDS), is crucial to achieving the SDGs.
Scientific competences not only depend on what students learn in their countries’ formal education system but also on other factors in the environment in which the students live. These factors could thus ensure provision of the scientific competences necessary for preparing students to face the grave environmental problems that have especially strong negative effects on the poorest and developing countries.
This study aims to identify the factors that determine the scientific competence of students in developing countries according to the results of PISA-D, paying special attention to social and cultural capital and to the environmental conditions of the area in which the students live. The study also seeks to fill the gap in studies on education about the environment and sustainability in Africa, south and central America, and southeast Asia.
To achieve these goals, we analysed data from PISA-Development in the participating countries of Cambodia, Ecuador, Guatemala, Honduras, Paraguay, and Senegal [2]. To identify the factors that determine the scientific competence of students in these countries, we performed a multilevel analysis, considering three levels of information: from the student (level 1), from the school (level 2), and from the community (level 3).
The remainder of the paper is organised as follows: Section 2 offers a brief theoretical framework on education, scientific competences, and sustainable development, focusing on the determinants of scientific competences; Section 3 develops the empirical analysis, presenting the data, model, and method used; Section 4 presents the results; finally, Section 5 presents the conclusions.

2. Theoretical Framework

2.1. Education, Scientific Competence, and Sustainable Development

Not all kinds of education foster sustainable development. Therefore, we talk about Education for Sustainable Development (EDS), which is oriented to empowering learners with knowledge, skills, values, and attitudes to take informed decisions and make responsible actions for environmental integrity, economic viability, and a just society [3]. According to Tapio and Willamo [4], the factors that affect human action to face environmental problems can be divided into: (i) individual factors, (ii) social factors, and (iii) ecological factors. Individual factors include knowledge, or the rational logical part of human thinking and, more concretely, knowledge of specific information about the environment and environmental measures [5]. This knowledge is also known as scientific competence. The Programme for International Student Assessment (PISA) defines scientific competence as the ability to use scientific knowledge to understand and make decisions about the natural environment and the changes it undergoes in relation to human action [6]. Social factors include science, as the result of research and in turn as social organization [7]. Science is closely linked to education, which is an important means of protecting the environment [8]. Most environmental problems are due to a lack of environmental knowledge, a term used to mean knowledge and awareness of environmental problems and possible solutions to them [9]. Increasing knowledge of environmental problems can increase people’s concern and awareness [10]. Some authors argue that a common premise for promoting sustainability is to increase people’s awareness and education [11].
In line with the foregoing, the framework of action to achieve SDG 4 (ensure inclusive and equitable quality education and promote lifelong learning opportunities for all), establishes that all students should acquire the theoretical and practical knowledge essential for promoting sustainable development by 2030, establishing as goals: (i) “Percentage of students by age group (or education level) showing adequate understanding of issues relating to global citizenship and sustainability; (ii) Percentage of 15-year-old students showing proficiency in knowledge of environmental science and geoscience” [12] (p. 79).
For years, the OECD’s Programme for International Student Assessment (PISA) has been evaluating the degree to which 15-year-old students nearing the end of compulsory education have acquired key knowledge and skills essential for full participation in modern societies. The competences PISA usually evaluates are reading competence, mathematical competence, and scientific competence [6]. To include developing countries, the OECD also established PISA-D, so that these countries could determine their students’ levels of competence (reading, mathematics, and scientific) and thus improve their public policy on educational issues. Thus, the PISA-D science framework considers scientific competence as key at both the local (intra-country) and international level to enable countries (individually and together) to face the tremendous challenges in water and food supply, disease control, energy production, and adaptation to climate change [6]. Facing all these challenges requires a significant contribution from science and technology. However, “this does not mean turning everyone into a scientific expert but enabling them to fulfil an enlightened role in making choices which affect their environment and to understand in broad terms the social implications of debates between experts” [6] (p. 28). Teaching and learning about science related directly to everyday life make knowledge useful for understanding how the natural world functions, while also teaching students to be informed citizens who are prepared to tackle social issues related to science intelligently [13]. Scientific competence, understood as “the ability to use knowledge and information interactively—that is ‘an understanding of how it [a knowledge of science] changes the way one can interact with the world and how it can be used to accomplish broader goals’” [6] (p. 93), is perceived as a key competence all students must have [14,15]. With this competence, young people can respond to the current environmental and climate crisis by making informed critical decisions that influence their environment. This is the purpose of scientific literacy or competence [6].

2.2. Determinants of Scientific Competence

In scientific education, developing a student’s interest in science probably results in higher levels in understanding of science and environmental awareness [16]. Determining how students can achieve good performance in such scientific competence is a challenge for all countries, especially the least developed. Recent studies argue that achieving scientific competence for 15-year-old youths in the PISA test depends on socioeconomic, family, environmental, and attitudinal factors [17,18,19,20,21] (see Figure A1 in Appendix A). Among these factors, we highlight the following:
The environment. The environment refers specifically to characteristics of the environment of both the school and the students’ households that support success and education through disciplinary and academic climate in the school, as well as through cultural norms and values that motivate students to achieve higher goals [22]. The environment in which the children’s families, schools, and community live their lives affects educational performance and thus students’ behavior and development [23,24,25,26,27,28].
Family factors or cultural capital. Coleman [29] and Bourdieu [30] have studied the relationship between academic achievement and family. Bourdieu has stressed the crucial role of family resources (relational, material, and cultural) in shaping children’s unequal education results. The family thus plays a fundamental role in students’ learning and performance [31]; this influence has been called cultural capital. Citing Bourdieu, Cervini [32] (p. 454) stresses that “cultural capital, then, plays a role of intermediary factor between the student’s social origin (family background) and their learning”. In other words, children of higher social class will possess inherited cultural capital that is valued more highly by the school and will thus have greater success than students without such capital [32].
Social capital. Social capital is defined as a set of relational resources that groups and individuals can access based on their interests [30,33]. Social capital has come to be viewed as a flexible conceptual instrument that can be used to explain a wide range of social problems, including education [22]. Studies of the effect of social capital on academic performance have found a positive correlation between the two [34,35,36].
It is very important to know how these capitals influence students’ achievement in countries with low and medium economic conditions and large cultural differences. It is also important to know how these capitals influence student achievement and how students’ achievement is related to their scientific literacy and thus indirectly to their “climate or ecological consciousness”, starting from the important natural wealth of developing countries.

3. Empirical Analysis

3.1. Data

To empirically analyze the factors determining scientific competence in developing countries, we used data reported by PISA-Development in the participating countries of Cambodia, Ecuador, Guatemala, Honduras, Paraguay, and Senegal [2]. The data come from two questionnaires. The first, completed by students, is the questionnaire on antecedents. It includes information on students’ wellbeing, achievements, and attitudes towards school; learning in their households; and relationship with their parents, classmates, and/or professors; as well as parents’ education and occupation. The second questionnaire, completed by the director of the school, includes information on the school: where it is located, how it is structured and organized, and what the learning environment is like. The PISA-D data represent around one million students 15 years old, 34,604 of whom from a total of 1299 schools completed the assessment (see Table 1). The sampling technique used for PISA-D for each participating country is a stratified sampling design in two stages. The sampling units at the first stage consisted of schools with eligible students (or with the possibility of having such students at the time of the evaluation). Schools were systematically sampled from a comprehensive national list with all PISA-D-eligible schools. The strata were defined for each of the countries according to their characteristics. The sampling units at the second stage consisted of students from the sampled schools. These students were chosen from a complete list of 15-year-old students from each of the sampled schools. A target cluster size (TCS) was set for each country; this value was usually 42 students.
We highlight that over half of the items were identical to those evaluated in PISA-2015, which enabled us to derive further information on the PISA results by connecting them to the scale. The other items were adapted to the PISA framework [6]. All this information considers the students’ personal and socioeconomic characteristics, their social and cultural capital, and the characteristics of the environment that can influence the students’ scientific competence or academic achievement in the sciences. All these characteristics condition these students’ behavior in their relationship to the environment [6]. The data provided by PISA-D were used to evaluate these variables, while considering three levels of information: from the student (level 1), from the school (level 2), and from the community (level 3). Table A1 in Appendix A describes all variables used in this study. The data from the dependent variable were scaled using the Rasch model and expressed by assigning ten plausible values [37], presented on a continuous scale in which 500 points is equivalent to the average of the OECD countries, where the standard deviation is standardized at 100 points [38].
The independent variables associated with cultural capital were selected following Tramonte and Willms, who propose that there are two types of cultural capital, one static and the other relational [39]. The first is associated with possession of cultural goods and intellectual activities, and the second with discussions on cultural and political issues. Static cultural capital can only reflect the decisions and lifestyle of one’s parents, whereas relationship cultural capital reflects how capital is used and transmitted [40].
The variables related to social capital were chosen according to the approaches in Coleman [29], for whom social capital can be presented in three forms: expectations and obligations, information channels, and social norms. In PISA-D [41], the variables that can be included in level (I) are communication within the family, attitude towards school, and relationships between students and teachers. Level (II) includes climate of discipline in the classroom, teacher’s expectations, class size, and whether the school is in a high-crime area. For a more detailed description of each of the variables, see Table 2.

3.2. Specification of Model and Estimation Procedure

To consider the hierarchical structure of the data from PISA-D 2017 and to study the conditioners of students’ scientific competence in developing countries ( Y i j k ), we applied multilevel linear models [42,43]. The econometric model for the estimation is given by Equation (1):
Y i j k = β 0 + β 1 X 1 i j k + β 2 X 2 i j k + γ 1 Z 1 j k + γ 2 Z 2 j k + γ 3 Z 3 j k + β 3 D K + μ 0 j + μ 1 j X 2 i j k + ε 1 i j k
where, (i) indicates the student, who belongs to school (j) from country (k). The variable X 1 i j k is a series of variables related to the student’s personal and socioeconomic characteristics; X 2 i j k is a set of variables associated with the family’s cultural and social capital; Z 1 j k is a group of variables that characterize and approximate the school’s cultural and social capital; Z 2 j k is the set of variables of the school’s surroundings and natural environment; Z 3 j k is the set of variables of the school’s characteristics; and D K represents the dichotomous variables that include the student’s country of residence. Furthermore, μ 0 j is the error of the random effects of the schools’ level, μ 1 j is the random slope for each school relative to the family’s cultural or social capital cultural, and ε 1 i j k is the error term in the students’ level. We start from the assumption that μ 0 j , μ 1 j , and ε 1 i j k follow a normal distribution with mean zero and variances σ 0 μ 2 , σ 1 μ 2 , and σ ε 2 .
The multilevel model used is explained by analyzing its fixed and random parts. The fixed component of the model, expressed by ( β 0 + β 1 X 1 i j k + β 2 X 2 i j k + γ 1 Z 1 j k + γ 2 Z 2 j k + γ 3 Z 3 j k + β 3 D K ), defines the relationship between the student’s academic performance and a set of co-variables of the student or the school, whose estimated slopes are assigned by the parameters β 1 , β 2 , γ 1 , γ 2 , γ 3 , and another fixed effect defined by the slope of the country effect β 3 . The random part of the model composed of ( μ 0 j , μ 1 j X 2 i j k , ε 1 i j k ) enables us to estimate the variances σ 0 μ 2 , σ 1 μ 2 , and σ ε 2 , and includes the remainder ε i j k . This method gives each school the possibility of maintaining its own error component [ β ] 0 + μ 0 j and its own random slope ( β 2 + μ 1 j ) for any explanatory variable of the student’s cultural or social capital. The model can thus indicate whether the effect of any variable of social or cultural capital on scientific competence changes among schools once we control for other characteristics of the educational institution itself considered in the fixed part of the model ( γ Z i j k ). Given the conditions explained, we can analyze possible heterogeneity among schools and these conditions, while at the same time measuring the “average” effect of each variable [40].
Given the hierarchical structure of the data, we specified four multilevel models. Model I considers only the variables of cultural and social capital belonging to the student. Model II incorporates the variables of the family’s cultural and social capital. The goal of Model II is to show whether the cultural and social capital of the student’s family has a different effect, independent of the school the student attends. For this effect, we then incorporate the random slopes of the variables that were statistically significant in Model I. Model III incorporates the variables of social and cultural capital concerning the school. Model IV adds the variables related to the natural environment of the area in which the school is located, as well as the characteristics of the school itself. Model V, like Model II, incorporates the variables that approach cultural and social capital as random slopes of the schools.
All the models include the gender, grade compared, and whether the student has repeated one or more grades in school as control variables. Since the PISA-D test included only seven countries, the multilevel characteristics of our data mean that the country effect is considered as a fixed effect, since capturing the variability between countries requires at least 30 countries [42,44,45].

4. Results

Table 3 shows the results of our estimations.
Of all the variables defining the school’s environment, only “size of the community where the school is located”, “number of days the school is closed”, and whether “the school is near a geologically unstable area” are statistically significant. According to the results of Model IV (see column 5 in Table 3), the larger the community where the school is located, the higher the results for scientific competences. This means that students in large cities have a comparative advantage over students in schools located in less populated areas, which are usually rural. This result is similar to that obtained by Miller and Votruba-Drzal [46], who argue that the urbanization level influences academic achievement based on the differences they find between urban and rural population settlements. The second significant variable—with negative effects on scientific competences—is the number of days the school is closed due to weather or illness, a characteristic typical of developing countries. Finally, schools located in geologically unstable areas show higher student science competences than those in stable areas. This result must be related to the special awareness people develop when they live in such places.
As to the schools’ characteristics, it is very interesting that the school’s infrastructure and resources have no significant effect on scientific competences but that the public or private character of the school does. The results show that public school students achieve a lower level of competence than those educated in private schools. According to the OECD [47], private school students score higher in science than public school students. If the socioeconomic profiles of students and schools are considered, however, public school students score higher than private school students on average in all OECD countries. Similarly, Castro Aristizabal et al. [48] find that 87.2% of school performance differences in science are explained by whether students attend public or private schools.
In addition to the school’s physical environment, its cultural and social capital are very important in explaining the student’s scientific competences. These capitals are conditioned by the socioeconomic characteristics of the community in which the school is located. The results of Model III (column 4 in Table 3) show that the school’s cultural capital, proxied by the “percentage of extremely poor students” and “school learning resource level”, is statistically significant in explaining the level of science competences students achieve. A high percentage of very poor students in the school lowers the science competence levels. According to Alivernini and Manganelli [19], the factor that seems to be most closely associated with the difference between average school outcomes is the school’s average socioeconomic level. Cohen-Vogel et al. [24] term the concentration of students with high poverty levels “ghettoization” of urban centers. Schools in these urban centers achieve poor academic results because most students—poor children and young people without access to adequate housing, health care, and nutrition—find it very difficult to concentrate and learn well. Many factors explain the interaction between social stratification and cultural production in schools and communities where racial, ethnic, and socioeconomic groups show persistent differences in academic performance [49,50].
Our results show that the “school learning resource level” variable is also very important in explaining the scientific competences students achieve. This result is similar to those of Murillo and Román [51], who suggest that schools’ learning resources have significant effects on scientific competences in middle- and low-income countries, even when controlling for students’ socioeconomic characteristics. Furthermore, our results contrast with those obtained for developed countries, such as Spain [6], where schools’ technological resources showed no significant values [20]. These results probably reflect the fact that schools in developed countries have all the elements needed for learning, whereas educational institutions in developing countries do not.
If we now examine the effect of the school’s social capital on the scientific competences students achieve, our estimations of Model III show positive results (column 4 in Table 3). Specifically, a good climate of classroom discipline and good teacher’s expectations yield better student performance in science. These results are in line with Acar [22], who indicates that social capital, in the form of the school’s disciplinary and academic climate, supports success and education. The results also show that cultural norms and values motivate students to achieve higher goals. This criterion reinforces Putnam’s argument that the development of children and youth is strongly determined by the school’s social capital [33]. These results contrast, however, with those of Glewwe and Kremer [52], who cannot draw any general conclusions about which teaching, and school variables increase learning in developing countries.
The values of the variable “student–teacher relationships” are noteworthy, as the relationship is significant but has a negative sign.
If we examine the results of the variance among the schools, we see a decrease when comparing Model II (value 1426.85) to the variance of Model I. This finding indicates that including the variables of families’ social and cultural capital as random slopes (Model II) as well as the variables of the schools’ social and cultural capital (Model V) reveals differences in the results between schools due to the influence of social and cultural capital associated with the student [34].
In addition to all previously discussed variables related to the school (e.g., its physical environment or cultural and social capital), other student-level variables are very important in explaining the results of students’ scientific competence, and thus climate awareness. At this level, we considered not only the individual students’ characteristics but also their cultural and social capital, variables again conditioned largely by the socio-economic environment of the student’s family.
As we can see in Table 3, the results of our estimations show that the student’s cultural capital, proxied by the “number of books at home”, is not only statistically significant but is also the major factor explaining students’ scientific competences. Other authors obtain similar results for mathematics in Latin-American countries [20,53], and for reading competence in OECD countries [39].
As to the student’s social capital, we follow Coleman’s definition as “relationships between children and parents” [29] and proxy this variable with “communication within the family”.
The results in Table 3 indicate that this variable is statistically significant in all models and is positively related to competence in science. This result is in line with Caro [54] and Dufur and Parcel et al. [34], whose studies find a positive interaction of communication between parents and children and parent’s education in the results of the PIRLS 2006 and PISA 2000 tests. The second variable used to proxy the student’s social capital was “Attitude towards school”. This variable is also significant, but only in models that do not include the school’s variables.
Finally, concerning the individual factors, or the student´s individual characteristics, our results show that “gender”, “grade”, and “repeater” are statistically significant in explaining the scientific competences of students. First, we find that boys achieve better science skills than their female classmates, which agrees with the results obtained by Rodríguez-Mantilla et al. [20] in their study of predictors of science performance in PISA-2015 for Spain. For the case of Latin America and the Caribbean, the report from the Third Regional Comparative and Explanatory Study on Education Quality (TERCE) states that the results in science do not follow a definite pattern in terms of gender inequity in the achievement of learning. The gender gap in sciences is statistically significant in a small number of countries, and the advantage by gender is divided [55]. As to the other two characteristics, “grade” and “repeater”, it is important to remember that the learning time is different in developing and poor countries. Children from poor families usually start school at an older age, miss many days of school during primary school, and are more likely to repeat a grade. Many of these children work part-time away from home at very young ages. Moreover, it seems that the time spent in class on the three main PISA subjects varies markedly and the school curriculum does not explore them in as much depth [6]. For these reasons, students who have repeated a year achieve lower results. These results are like those obtained in Gómez Vera’s study of the countries of the southern cone of America based on PISA 2009 [56]. The fact that the students are in a higher grade than the average of other 15-year-olds means that they have achieved a higher level of science competence.
As to the results of country estimates, having Cambodia as a reference, we note that Latin American countries achieve higher performance in science competition, while Senegal and Zambia achieve lower levels.

5. Conclusions

The results enable us to conclude that science competence in the countries participating in PISA-D are low, although the results are even lower in African countries. The achievement of scientific competence is greater in men than in women, and the differentiating results are generally related to the family’s social and cultural capital, to which the students are subjected. The higher the scores in these capitals, the greater their achievement in science. The same occurs with the schools’ cultural and social capital and its influence on the achievement of scientific competence. Schools with a higher number of poor students show lower results in achievement in the sciences, as do schools with fewer learning resources. The social capital of the schools also has a positive influence on science competence, measured as the presence of a good climate of discipline in the classroom and higher expectations from teachers. That the school belonged to a large urban area was also found to be a factor determining higher achievement in the sciences than schools in rural areas with settlements that have small populations.
We must stress the importance of cultural and social capital in achieving scientific competence and the way that such scientific literacy offers the competences students need to respond socially as needed to the environmental crisis. These results show the need to propose further studies to complement this one by considering additional factors that can affect academic performance, such as socio-emotional issues. Additionally, since PISA-D questionnaires do not always capture the most relevant contextual factors for these countries, future research needs to be carried out including more questions about real-life situations in those contexts and/or questions about the level of understanding the students have about their environment. In fact, this paper, as a first approximation in this line, has empirically demonstrated that being near a geological area makes students achieve better grades in science, surely worried about the environment in which they live. Finally, studies that tend to foster better development in the scientific competences are also so necessary in these times of environmental crisis.
To conclude, we argue that the current paradigm of education must change, as economic development alone is insufficient if one neglects the sustainability of the planet [57]. In response to the current environmental and climate crisis, there is an urgent need for young people to develop competences that enable them to make more informed, critical decisions that influence their environment. Scientific literacy prepares them for such decisions [6].

Author Contributions

Conceptualization, J.M.C.C. and I.N.; methodology, J.M.C.C. and I.N.; software, J.M.C.C.; validation, J.M.C.C. and I.N.; formal analysis, J.M.C.C., I.N. and M.L.-C.; investigation, J.M.C.C., I.N. and M.L.-C.; resources, J.M.C.C.; data curation, J.M.C.C.; writing—original draft preparation, J.M.C.C.; writing—review and editing, M.L.-C.; visualization, J.M.C.C., I.N. and M.L.-C.; supervision, J.M.C.C., I.N. and M.L.-C.; project administration, I.N. and M.L.-C.; funding acquisition, I.N. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by the National R&D Programme of the Spanish Ministry of Science, Innovation and Universities through the project ‘Spanish Universities Involvement in Social Innovation Activities’ (SUISIA) with grant number: RTI2018-101722-B-I00.

Data Availability Statement

Data supporting reported results can be found at: https://www.oecd.org/pisa/pisa-for-development/database/ (accessed on 10 December 2020).

Conflicts of Interest

The authors declare no conflict of interest.

Appendix A

Table A1. Description of variables.
Table A1. Description of variables.
Variable NameDescriptionQuestionnaireSurvey ItemResponse
Type
Minimum
Value
Maximum
Value
Dependent Variable
Scientific
Competence
“Ability to explain phenomena scientifically, evaluate and design scientific enquiry, and interpret data and evidence scientifically ” (OECD, 2015).
Independent Variables
Characteristics of environment
School environment
(LEVEL 2)
Size of community where school is locatedTo directorsSC001 1 = Village or rural settlement of under 3000 inhabs.5 = Large city of over 1,000,000 inhabs.
Near highway or freewayTo directorsSC010Q01NADichotomous0 = No1 = Yes
Near busy roads or intersectionsTo directorsSC010Q02NADichotomous0 = No1 = Yes
Near a dump or waste landTo directorsSC010Q04NADichotomous0 = No1 = Yes
Near geologically unstable areaTo directorsSC010Q05NADichotomous0 = No1 = Yes
Near industrial districtTo directorsSC010Q06NADichotomous0 = No1 = Yes
No. of days per year school is closed due to weather or illnessTo directorsSC024Q05NANumericalNo. daysNo. days
Time from home to schoolTo studentsST061 1 = under 15 minutes5 = over 90 minutes
School characteristics
(LEVEL 2)
Type of school: public or privateTo directorsSC006 1 = private2 = public
School resourcesTo directorsSCHRESOURSES 1 = very low resource level5 = very high resource level
Characteristics of the student and their socioeconomic environment
Student characteristics
(LEVEL 1)
Grade compared. Student’s current grade compared to standard grade-age of each countryReported by PISA-DST001 -33
Repeater. Has the student ever repeated a grade?To studentsST009Dichotomous0 = No1 = Yes
GenderTo studentsST004 1 = female2 = male
Family’s cultural capital
(LEVEL 1)
Number of books at homeTo studentsST066Q01NA 14
Mother’s education levelReported by PISA-DMISCED_D 0 = No education *6 = Highest level *
Household poverty indexReported by PISA-DPOVERTY 1 = very poor ***4 = not poor ***
Household properties **Reported by PISA-DHOMEPOS15 −8.17923.9708
School’s cultural capital
(LEVEL 2)
Percentage of very poor students in the schoolTo directorsSC022Q02NA 1 = under 1%6 = over 30%
Availability of instructional resources in the school and use teachers make of themTo directorsINSTRRESCAT 1 = has very basic instructional resources5 = has more complex, expensive resources in the school
Family’s social capital (LEVEL 1)Communication within the familyTo studentsST083 −2.7521.095
School’s social capital
(LEVEL 2)
Student’s attitude towards school. Measures the impact of school on the student, whether the student sees the importance of school for their future.To studentsATSCH 0 = negative attitude10 = full approval and value
Climate of discipline in the classroomTo studentsDISCI 0 = negative attitude10 = full approval and value
Student–teacher relationshipsTo studentsSTTCHREL 0 = lesser degree ****10 = greater degree ****
Teacher’s expectations about students’ success and ability to workTo studentsTCEXPSUC 0 = lesser degree10 = greater degree
Class sizeTo directorsSC005 1 = less than 15 students9 = more than 50 students
Community’s social capital (LEVEL 2)Is the neighborhood in which the school is located a high-crime neighborhood?To directorsSC010Q03NADichotomous0 = No1 = Yes
Notes: * The indexes of parents’ education were obtained by recoding the education levels into the following categories: (0) None, (1) ISCED (International Standard Classification of Education) 1 (primary education), (2) ISCED 2 (lower secondary), (3) ISCED level 3B or 3C (upper secondary, professional/pre-professional), (4) ISCED 3A (general upper secondary) and in some cases ISCED 4 (post-secondary non-tertiary), (5) ISCED 5B (vocational tertiary), and (6) ISCED 5A and in some cases ISCED 6 (tertiary and graduate, leading to an advanced research qualification). ** Household properties refers to the availability of 16 household items, including three country-specific items viewed as measures of household wealth within the country context.*** Household poverty index: This is the index reported by PISA-D as “POVERTY”, which includes four categories: “extremely poor”, “severely poor”, “poor”, and “not poor”, based on the results of another index that measures family resources, for example, whether family shares a hygienic bathroom with others who are not members of their household, whether they have a flush toilet, what material they have on the floor of their house, whether any family member has a bank account, and whether the student has gone hungry in the past month. **** Evaluated using a scale produced by students’ degree of agreement with a series of questions about their interpersonal relationships with their teachers.
Figure A1. Determinants of scientific competence.
Figure A1. Determinants of scientific competence.
Sustainability 13 12439 g0a1

References

  1. UNESCO. Education for Sustainable Development Goal-Learning Objectives; Unesco Publishing: Paris, France, 2017. [Google Scholar]
  2. OECD. Chapter 4: Sample Design. In PISA for Development; Technical Report; OECD Publishing: Paris, France, 2018; pp. 1–29. [Google Scholar]
  3. UNESCO. Roadmap Education for Implementing the Global Action Programme on Education for Sustainable Development; United Nations Educational, Scientific and Cultural Organization: Paris, France, 2014. [Google Scholar]
  4. Tapio, P.; Willamo, R. Developing interdisciplinary environmental frameworks. Ambio 2008, 37, 125–133. [Google Scholar] [CrossRef]
  5. Tapio, P.; Hietanen, O. Epistemology and public policy: Using a new typology to analyse the paradigm shift in Finnish transport futures studies. Futures 2002, 34, 597–620. [Google Scholar] [CrossRef]
  6. OCDE. Marco de Evaluación y de Análisis de PISA para el Desarrollo; OECD Publishing: Paris, France, 2017. [Google Scholar]
  7. Collin, F.; Pedersen, D.B. The Frankfurt School, Science and Technology Studies, and the Humanities. Soc. Epistem. 2013, 29, 44–72. [Google Scholar] [CrossRef]
  8. Dimenäs, J.; Alexandersson, M. Crossing Disciplinary Borders: Perspectives on Learning About Sustainable Development. J. Teach. Educ. Sustain. 2012, 14, 5–19. [Google Scholar] [CrossRef] [Green Version]
  9. Zsóka, Á.; Szerényi, Z.M.; Széchy, A.; Kocsis, T. Greening due to environmental education? Environmental knowledge, attitudes, consumer behavior and everyday pro-environmental activities of Hungarian high school and university students. J. Clean. Prod. 2013, 48, 126–138. [Google Scholar] [CrossRef]
  10. Bamberg, S.; Möser, G. Twenty years after Hines, Hungerford, and Tomera: A new meta-analysis of psycho-social determinants of pro-environmental behaviour. J. Environ. Psychol. 2007, 27, 14–25. [Google Scholar] [CrossRef]
  11. Lozano, R.; Lukman, R.; Lozano, F.J.; Huisingh, D.; Lambrechts, W. Declarations for sustainability in higher education: Becoming better leaders, through addressing the university system. J. Clean. Prod. 2013, 48, 10–19. [Google Scholar] [CrossRef]
  12. UNESCO. Educación 2030: Declaración de Incheon y Marco de Acción; UNESCO Publishing: Paris, France, 2015. [Google Scholar]
  13. DeBoer, G.E. Scientific literacy: Another look at its historical and contemporary meanings and its relationship to science education reform. J. Res. Sci. Teach. 2000, 37, 582–601. [Google Scholar] [CrossRef]
  14. Blanco-López, Á.; España-Ramos, E.; González-García, F.J.; Franco-Mariscal, A.J. Key aspects of scientific competence for citizenship: A Delphi study of the expert community in Spain. J. Res. Sci. Teach. 2015, 52, 164–198. [Google Scholar] [CrossRef]
  15. Lupión-Cobos, T.; López-Castilla, R.; Blanco-López, Á. What do science teachers think about developing scientific competences through context-based teaching? A case study. Int. J. Sci. Educ. 2017, 39, 937–963. [Google Scholar] [CrossRef]
  16. Zhu, Y. How Chinese students’ scientific competencies are influenced by their attitudes? Int. J. Sci. Educ. 2019, 41, 2094–2112. [Google Scholar] [CrossRef]
  17. Gallardo-Gil, M.; Fernández-Navas, M.; Sepúlveda-Ruiz, M.-P.; Serván, M.-J.; Yus, R.; Barquín, J. PISA y la com-petencia científica: Un análisis. Electrónica Investig. Evaluación Educ. 2010, 16, 1–17. [Google Scholar]
  18. Jack, B.M.; Lin, H.-S.; Yore, L.D. The synergistic effect of affective factors on student learning outcomes. J. Res. Sci. Teach. 2014, 51, 1084–1101. [Google Scholar] [CrossRef]
  19. Alivernini, F.; Manganelli, S. Country, School and Students Factors Associated with Extreme Levels of Science Literacy Across 25 Countries. Int. J. Sci. Educ. 2015, 37, 1992–2012. [Google Scholar] [CrossRef]
  20. Rodríguez-Mantilla, J.M.; Fernández-Díaz, M.J.; Olmeda, G.J. PISA 2015: Predictores del rendimiento en Ciencias en España. Rev. Educ. 2018, 380, 75–102. [Google Scholar] [CrossRef]
  21. Mostafa, T.; Echazarra, A.; Hélène, G. The Science of Teaching Science. An Exploration of Science Teaching Practices in PISA 2015; OECD Ilibrary: Paris, France, 2018. [Google Scholar]
  22. Acar, E. Effects of social capital on academic success: A narrative synthesis. Educ. Res. Rev. 2011, 6, 456–461. [Google Scholar]
  23. Bowen, N.K.; Bowen, G.L. Effects of Crime and Violence in Neighborhoods and Schools on the School Behavior and Performance of Adolescents. J. Adolesc. Res. 1999, 14, 319–342. [Google Scholar] [CrossRef]
  24. Cohen-Vogel, L.; Goldring, E.; Smrekar, C. The Influence of Local Conditions on Social Service Partnerships, Parent Involvement, and Community Engagement in Neighborhood Schools. Am. J. Educ. 2010, 117, 51–78. [Google Scholar] [CrossRef]
  25. Cohen, D.J.; Prusak, L. Good Company: How Social Capital Makes Organizations Work; Harvard Business School Press: Boston, MA, USA, 2001. [Google Scholar]
  26. Leventhal, T.; Brooks-Gunn, J. The neighborhoods they live in: The effects of neighborhood residence on child and adolescent outcomes. Psychol. Bull. 2000, 126, 309–337. [Google Scholar] [CrossRef] [Green Version]
  27. Nieuwenhuis, J.; Hooimeijer, P. The association between neighbourhoods and educational achievement, a systematic review and meta-analysis. Neth. J. Hous. Environ. Res. 2015, 31, 321–347. [Google Scholar] [CrossRef] [Green Version]
  28. Wodtke, G.T.; Parbst, M. Neighborhoods, Schools, and Academic Achievement: A Formal Mediation Analysis of Contextual Effects on Reading and Mathematics Abilities. Demography 2017, 54, 1653–1676. [Google Scholar] [CrossRef]
  29. Coleman, J.S. Social capital in the creation of human capital. Am. J. Sociol. 1988, 94, S95–S120. [Google Scholar] [CrossRef]
  30. Bourdieu, P. The forms of capital. In Handbook of Theory and Research for the Sociology of Education; Greenwood Press: New York, NY, USA, 1986; pp. 241–258. [Google Scholar]
  31. Parth, S.; Schickl, M.; Keller, L.; Stoetter, J. Quality Child–Parent Relationships and Their Impact on Intergenerational Learning and Multiplier Effects in Climate Change Education. Are We Bridging the Knowledge–Action Gap? Sustainability 2020, 12, 7030. [Google Scholar] [CrossRef]
  32. Cervini, R. Desigualdades en el logro académico y reproducción cultural en Argentina. Un modelo de tres niveles. Rev. Mex. Investig. Educativa. 2002, 7, 445–500. [Google Scholar]
  33. Putnam, R.D. The Collapse and Revival of American Community; Simon & Schuster: New York, NY, USA, 2000. [Google Scholar]
  34. Dufur, M.J.; Parcel, T.L.; Troutman, K.P. Does capital at home matter more than capital at school? Social capital effects on academic achievement. Res. Soc. Strat. Mobil. 2013, 31, 1–21. [Google Scholar] [CrossRef]
  35. Huang, L. Social capital and student achievement in Norwegian secondary schools. Learn. Individ. Differ. 2009, 19, 320–325. [Google Scholar] [CrossRef] [Green Version]
  36. Mishra, S.; Mishra, S. Social networks, social capital, social support and academic success in higher education: A systematic review with a special focus on ‘underrepresented’ students. Educ. Res. Rev. 2019, 29, 100307. [Google Scholar] [CrossRef]
  37. OECD. PISA 2015 Technical Report; OECD Publishing: Paris, France, 2015. [Google Scholar]
  38. INEVAL. Educación en Ecuador. Resultados de Pisa para el Desarrollo. Available online: http://www.evaluacion.gob.ec/wp-content/uploads/downloads/2018/12/CIE_InformeGeneralPISA18_20181123.pdf2018 (accessed on 2 December 2020).
  39. Tramonte, L.; Willms, J.D. Cultural capital and its effects on education outcomes. Econ. Educ. Rev. 2010, 29, 200–213. [Google Scholar] [CrossRef]
  40. Aracil, A.G.; Gómez, I.N.; Verdú, C.A. Influencia del Capital Social y Cultural en el Conocimiento Financiero de los Adolescentes: Entorno Familiar y Escolar. Rev. Educ. 2016, 374, 94–117. [Google Scholar] [CrossRef]
  41. OECD. Chapter 16: Scaling Procedures and Construct Validation of Context Questionnaire Data; OECD Publishing: Paris, France, 2018. [Google Scholar]
  42. Goldstein, H. Multilevel Statistical Models; John Wiley & Sons: Sussex, UK, 2011. [Google Scholar]
  43. Bryk, A.S.; Raudenbush, S.W. Application of hierarchical linear models to assessing change. Psychol. Bull. 1987, 101, 147–158. [Google Scholar] [CrossRef]
  44. Hox, J.J.; Maas, C.J. Multilevel Analysis. In Encyclopedia of Social Measurement; Elsevier: Amsterdam, The Netherlands, 2005; pp. 785–793. [Google Scholar] [CrossRef]
  45. Aslam, A.; Corrado, L. The geography of well-being. J. Econ. Geogr. 2011, 12, 627–649. [Google Scholar] [CrossRef]
  46. Miller, P.; Votruba-Drzal, E. Urbanicity Moderates Associations between Family Income and Adolescent Academic Achievement. Rural. Sociol. 2015, 80, 362–386. [Google Scholar] [CrossRef]
  47. OECD. PISA 2015 Results. In Overview: Policies and Practices for Successful Schools; OECD Publishing: Paris, France, 2015. [Google Scholar] [CrossRef] [Green Version]
  48. Castro Aristizabal, G.; Giménez, G.; Pérez Ximénez-de-Embún, D. Desigualdades educativas en América Latina, PISA 2012: Causas de las diferencias en desempeño escolar entre los colegios públicos y privados. Rev. Educ. 2017, 376, 33–59. [Google Scholar] [CrossRef]
  49. Carter, P.L. “Black” Cultural Capital, Status Positioning, and Schooling Conflicts for Low-Income African American Youth. Soc. Probl. 2003, 50, 136–155. [Google Scholar] [CrossRef]
  50. Le Mener, M.; Meuret, D.; Morlaix, S. L’accroissement de l’effet de l’origine sociale sur la performance scolaire: Par où est-il passé? Rev. Française Sociol. 2017, 58, 207–231. [Google Scholar] [CrossRef]
  51. Murillo, F.J.; Román, M. School infrastructure and resources do matter: Analysis of the incidence of school resources on the performance of Latin American students. Sch. Eff. Sch. Improv. 2011, 22, 29–50. [Google Scholar] [CrossRef]
  52. Glewwe, P.; Kremer, M. Schools, teachers, and education outcomes in developing countries. Handb. Econ. Educ. 2006, 2, 945–1017. [Google Scholar] [CrossRef]
  53. Cervini, R.; Dari, N.; Quiroz, S. Las determinaciones socioeconómicas sobre la distribución de los aprendizajes escolares: Los datos del TERCE. Rev. Iberoam. Sobre Calid. Efic. Cambio Educ. 2016, 14, 61–79. [Google Scholar] [CrossRef] [Green Version]
  54. Caro, D.H. Parent-child communication and academic performance associations at the within- and between-country level. J. Educ. Res. Online 2011, 3, 15–37. [Google Scholar]
  55. UNESCO-OREALC. Inequidad de Género en los Logros de Aprendizaje en Educación Primaria ¿Qué nos Puede Decir TERCE? Unesco Publishing: Paris, France, 2016. [Google Scholar]
  56. Gómez Vera, G. Los efectos de la repitencia en tanto que politica pública en cuatro países del cono sur: Argentina, Brasil, Chile y Uruguay: Un análisis en base a PISA 2009. Rev. Latinoam. Educ. Comp. 2013, 4, 59–70. [Google Scholar]
  57. Goebel, J.; Fischman, G.E.; Silova, I. Why Measure Un-Sustainable Education. Available online: https://resources.norrag.org/resource/546/why-measure-un-sustainable-education-by-janna-goebel-gustavo-e-fischman-and-iveta-silova#print (accessed on 10 December 2020).
Table 1. Countries participating in PISA-D and their achievement in sciences.
Table 1. Countries participating in PISA-D and their achievement in sciences.
COUNTRYCountry SampleAvg. in Sciences
Cambodia5162330
Ecuador5664399
Guatemala5100365
Honduras4773370
Paraguay4510358
Senegal5182309
Zambia4213309
OECD avg. 493
Total34,604
Source: [2].
Table 2. Descriptive statistics of the variables.
Table 2. Descriptive statistics of the variables.
Characteristics of the Environment
AreaVariableObservationsMeanStandard
Deviation
MinMax
School environment
(LEVEL 2)
Type of community in which the school is located33,7342.59481.28835915
Near highway or freeway28,8030.42224770.493926201
Near busy roads or intersections31,1530.7132860.45223401
Near dump or waste land27,2440.09345180.291069801
Near geologically unstable area27,1480.07540150.26404301
Near industrial district27,1880.07985140.27106801
Number of days school is closed due to weather or illness32,2440.49181241.593598020
Time from home to school32,8981.9160741.11433115
School characteristics
(LEVEL 2)
Public or private school34,3960.95400631.21679909
School resources34,2003.0806431.39928515
Characteristics of the Student and Their Socioeconomic Environment
AreaVariableObservationsMeanStandard DeviationMinMax
Student’s characteristics
(LEVEL 1)
Grade compared34,604−0.29756681.039366−33
Repeater33,6920.3058590.460777101
Gender34,6040.48537740.499793401
Family’s cultural capital
(LEVEL 1)
Number of books at home32,1892.382180.953048714
Mother’s education level30,4812.5277712.02026106
Household poverty index33,5013.2530370.85438314
Household properties32,501−1.7904171.4060211−8.17923.9708
School’s cultural capital
(LEVEL 2)
Percentage of very poor students33,4184.4289011.67312316
Level of instructional resources32,6643.001.41415
Family’s social capital
(LEVEL 1)
Communication within the family29,6151.60e-071.004−2.7521.095
School’s social capital
(LEVEL 2)
Attitude towards school32,4137.5363312.065439010
Discipline in the classroom32,5930.16404531.045713−2.93942.2536
Student–teacher relationships32,4756.5418372.002628010
Teacher’s expectations32,6647.1308772.38518010
Class size32,288773.00826.51165111
Community’s social capital
(LEVEL 2)
Located in a high-crime neighborhood27,4030.12151950.32673601
Source: Developed by authors.
Table 3. Multilevel estimations of PISA-D scientific competences.
Table 3. Multilevel estimations of PISA-D scientific competences.
VariablesModel IModel IIModel IIIModel IVModel V
School environment
Community size where the school is located 5.172 ***4.764 ***
(0.768)(0.717)
Near highway or freeway 2.8902.284
(1.745)(1.630)
Near busy roads or intersections 0.4231.646
(1.923)(1.785)
Near a dump or wasteland −2.562−3.923
(3.244)(2.958)
Near geologically unstable area 8.211 **6.631 *
(3.178)(2.934)
Near industrial district −1.744−1.290
(3.313)(3.104)
No. of days school is closed due to the weather or illness −1.714 ***−1.632 ***
(0.464)(0.430)
Time from home to school −0.504−0.537
(0.310)(0.307)
School characteristics
Public or private −6.376 ***−4.633 ***
(1.314)(1.237)
School resources 0.1700.0601
(0.816)(0.750)
Student characteristics
Grade compared16.32 ***15.83 ***15.07 ***15.05 ***14.73 ***
(0.352)(0.348)(0.409)(0.436)(0.433)
Repeater−10.39 ***−10.10 ***−10.30 ***−9.807 ***−9.616 ***
(0.651)(0.645)(0.762)(0.820)(0.814)
Gender/Male10.31 ***9.727 ***10.44 ***10.23 ***9.641 ***
(0.514)(0.613)(0.600)(0.646)(0.779)
Family’s cultural capital
Number of books at home4.115 ***3.906 ***4.188 ***4.306 ***4.071 ***
(0.308)(0.305)(0.360)(0.390)(0.386)
Mother’s education0.755 ***0.387 *0.776 ***0.721 ***0.477 *
(0.145)(0.173)(0.169)(0.182)(0.216)
Household poverty index0.8743.207 ***0.9601.2122.595 **
(0.622)(0.635)(0.722)(0.778)(0.800)
Household properties1.914 ***0.894 *1.444 **1.062 *0.477
(0.386)(0.421)(0.447)(0.488)(0.533)
School’s cultural capital
Percentage of very poor students −4.654 ***−3.007 ***−2.370 ***
(0.527)(0.563)(0.530)
Level of instructional resources 9.166 ***5.074 ***4.461 ***
(0.714)(0.903)(0.835)
Family’s social capital
Communication within the family4.387 ***4.258 ***4.443 ***4.578 ***4.448 ***
(0.312)(0.329)(0.371)(0.400)(0.421)
School’s social capital
Attitude towards school−0.725 ***−0.645 ***−0.0333−0.007600.0511
(0.131)(0.130)(0.179)(0.192)(0.209)
Discipline in the classroom 1.496 ***1.605 ***1.576 ***
(0.319)(0.341)(0.339)
Student–teacher relationships −2.758 ***−2.940 ***−2.818 ***
(0.219)(0.238)(0.236)
Teacher’s expectations 1.110 ***1.122 ***1.002 ***
(0.186)(0.201)(0.200)
Class size 0.09850.04430.00854
(0.0631)(0.0654)(0.0605)
Community’s social capital
Located in a high-crime neighborhood 2.165−0.3030.184
(2.526)(2.809)(2.591)
Countries, reference Cambodia
Ecuador50.06 ***34.61 ***46.88 ***42.66 ***34.38 ***
(2.984)(2.615)(3.179)(3.238)(3.011)
Guatemala30.55 ***20.63 ***31.42 ***26.65 ***21.48 ***
(2.923)(2.515)(3.114)(3.310)(3.044)
Honduras45.05 ***39.46 ***53.40 ***46.88 ***43.79 ***
(2.899)(2.512)(3.192)(3.395)(3.161)
Paraguay30.01 ***21.02 ***32.44 ***33.60 ***27.88 ***
(2.954)(2.602)(3.326)(3.483)(3.267)
Senegal−27.55 ***−29.86 ***−13.80 ***−23.54 ***−25.28 ***
(3.059)(2.578)(3.268)(3.503)(3.182)
Zambia−5.307−6.003 *5.9244.6223.154
(3.011)(2.627)(3.268)(3.261)(3.031)
Constant313.3 ***311.0 ***297.8 ***311.7 ***307.0 ***
(3.825)(3.773)(5.937)(7.527)(7.288)
Variance of random effects
Student ϭ 2 654.4 ***1607.53 ***482.7 ***407.6 ***1030.99 ***
School ϭ u 0 ) 2 1519.3 ***1426.85 ***1508.0 ***1505.3 ***1415.47 ***
Slopes of X ij ϭ u 1 ) 2
No. of books at home 116.324 *** 125.858 ***
Mother’s education 8.983 *** 8.259 ***
Household properties 18.514 *** 18.274 ***
Communication within the family 10.97 *** 11.27 ***
Attitude towards school 5.637 *** 4.546 ***
Number of students25,60425,60425,60425,60425,604
Number of schools12821282128212821282
2LR100,184.84393.9671,813.8526939.99179.3
Standard errors in parentheses: * p < 0.05, ** p < 0.01, *** p < 0.001.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Chávez Charro, J.M.; Neira, I.; Lacalle-Calderon, M. Scientific Competence in Developing Countries: Determinants and Relationship to the Environment. Sustainability 2021, 13, 12439. https://doi.org/10.3390/su132212439

AMA Style

Chávez Charro JM, Neira I, Lacalle-Calderon M. Scientific Competence in Developing Countries: Determinants and Relationship to the Environment. Sustainability. 2021; 13(22):12439. https://doi.org/10.3390/su132212439

Chicago/Turabian Style

Chávez Charro, José Mauricio, Isabel Neira, and Maricruz Lacalle-Calderon. 2021. "Scientific Competence in Developing Countries: Determinants and Relationship to the Environment" Sustainability 13, no. 22: 12439. https://doi.org/10.3390/su132212439

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop