Assessing the Professionalism of Teacher Educators in Relation to Sustainability: Developing the Teacher Education and Sustainability Scale (TESS)

Abstract

This study focuses on the design and implementation of the Teacher Education and Sustainability Scale (TESS), an instrument to assess the professionalism of teacher educators in relation to sustainability in the context of Agenda 2030 and its 17 Sustainable Development Goals (SDGs). Despite the importance of the SDGs, there is very little research on the identity of teacher educators in relation to sustainability. The Teach4Reach 2.0 project, an international collaboration between the University of Pretoria, the University of Vienna and the University of Innsbruck, aims to fill the research gap in the quantitative measurement of this key area by examining the professionalism of teacher educators in relation to sustainability in order to strengthen their identity. The TESS questionnaire was developed in a structured four-stage process that initially included AI-generated items and was followed by expert refinement and a testing phase with participants from Austria and South Africa. The findings discuss the process of developing the TESS questionnaire and include a critical reflection on AI and on the need for targeted professional development for teacher educators in the field of sustainable development.

1. Introduction

Health crises such as the COVID-19 pandemic, economic and technological challenges such as inflation and digitization, the global climate crisis and the violation of civil rights through wars are events that have a profound impact on the quality of education [1]. To address some of these current challenges, the focus is on moving towards a sustainable future. Agenda 2030 and its 17 Sustainable Development Goals (SDGs) provide a comprehensive framework to guide these efforts and to ensure that key aspects of sustainability are considered and addressed in education reforms [2,3]. Education should support personal transformation and pave the way for the societal changes needed to address global challenges and build a more sustainable world [4]. This requires the integration of new competencies into educational practice to promote resilience and adaptability in future teachers. Teacher educators play a central role in achieving this goal. They can be described as key actors who have a significant impact on the professional development and quality of (future) teachers [5]. They are in a position to empower learners to acquire the knowledge, skills, values and behaviors needed to tackle global challenges and play a role in creating a more just, peaceful, and sustainable world [4]. Izidinia [6] and the European Commission [5] emphasize that responses to socio-cultural and environmental developments have a significant impact on the professionalism of teacher educators. Therefore, the professional development of the identity of teacher educators is an important issue in teacher education. Professional identity is described as a complex interaction involving societal demands and challenges, personal identities based on individual qualifications, attitudes and values, and collective identities that represent the experience of being an integrated member of a group [6,7,8].

Recent literature on the professional identity of teacher educators, particularly in relation to sustainability, is sparse [9]. Schrittesser [10] notes that little attention has been paid to educators who are recruited from the academic staff of colleges and universities, such as professors and junior academics. Their attitudes, practices and teaching activities require independent research, which is still lacking. Although some progress has been made [11,12,13], much of the existing literature focuses primarily on the knowledge and professional development of teachers, rather than that of their instructors. Similarly, various meta-analytical tests and professional identity scales focus on the professionalism of teachers (e.g., [14,15,16,17]). On the topic of sustainability, UNESCO and Educational International conducted a global survey of 58,000 teachers to determine how teachers assess their own ability to contribute to achieving the SDGs. The results showed that only a third of teachers felt prepared for the challenges. They were motivated to teach new skills and topics, but complained of a lack of knowledge, resources and structural support [18].

There are few empirical studies that directly address the professional knowledge and needs of teacher educators (e.g., [9,13,19,20,21,22,23,24,25,26]). These studies highlight the need for future research to examine the normative beliefs of teacher educators in order to develop their professionalism in the 21st century and to meet the requirements of a sustainable transformation. The International Forum for Teacher Educator Development (InFo-TED) has developed a dynamic model that underpins the knowledge base for teacher educators at the personal, local, national and global levels [20]. The model underscores the significance of lifelong learning across 13 areas, addressing social changes, diversity, identities and technological changes, all of which are linked to sustainability issues. Therefore, there is an urgent need for further studies and standardized measurement tools to comprehensively capture the professional identity of teacher educators and to adequately acknowledge their identity in teacher education with a focus on a sustainable future [9,10,27].

To address the research gap on the professionalism of teacher educators in relation to sustainability, the Teacher Educator Sustainability Survey (TESS) questionnaire, developed within the international Teach4Reach 2.0 project, was designed to gain insights from the teacher educator context. The professional understanding of teacher educators is based on Rieckmann [28], Ferrer-Estéves and Chalmeta [29], who point out that a central part of the professionalism of teacher educators is that they are able to deal with changing requirements and influence a shift in consciousness and mentality in favor of values, attitudes and behaviors for a sustainable future [29]. In this context, studies emphasize the potential to strengthen the identity of teachers [6]. In this project, the concept of sustainability was based on the UNESCO Agenda 2030 within the framework of the 17 SDGs [3], with the aim of contributing to the development of scientific knowledge and deeper theoretical understanding of the professionalism of teacher educators. Since the early 1990s, the concept of sustainable development has become an internationally recognized paradigm for the integration of economic, social and environmental development goals [30,31]. In 2015, the United Nations set out 17 Sustainable Development Goals (SDGs) in Agenda 2030, which are to be achieved by 2030 [32], and which take these three dimensions of sustainable development into account. However, at the midpoint of the program, a worrying picture is emerging with regard to the achievement of the SDGs [18,33,34]. The increased action to promote sustainability in the context of education through the UN Decade of Education for Sustainable Development (ESD) (2005–2014) and the Global Programme of Action for ESD (2015–2019) have also failed. Therefore, Education for Sustainable Development: Achieving the Global Sustainable Development Goals (ESD, 2030) was launched as a global framework program [35] to strengthen the contribution towards achieving all 17 SDGs, focusing on five priority action areas: Advancing Policy (1), Transforming Learning and Training Environments (2), Building Capacities of Educators and Trainers (3), Empowering and Mobilizing Youth (4) and Accelerating Sustainable Solutions at the Local Level (5) [36]. Despite extensive efforts and various initiatives, studies show their limited impact [34]. The concept itself has also been criticized. A common criticism of the SDGs is that they are embedded in a neoliberal economic model that prioritizes economic growth and development. Critics argue that this focus can contradict the goals of true environmental sustainability, as the SDGs often support the existing capitalist order instead of tackling the root causes of global crises. Instead of promoting transformative approaches, they risk reinforcing current power structures and inequalities [31,37]. Despite these criticisms, however, the SDGs provide an important framework for guiding sustainable transformation. They address critical global issues and provide a starting point for discussions on how to achieve a just and environmentally sustainable future. By providing a common point of reference, the SDGs can help coordinate global efforts towards meaningful change [38,39].

To support and broaden the discourse on sustainable development in the context of teacher educators and link to the priority action areas, this paper focuses on the development and piloting of the Teacher Education and Sustainability Scale (TESS), a tool to assess the professional understanding of teacher educators in the context of sustainability and the extent to which sustainability is integrated into instructional methods and teaching practices. The TESS questionnaire was developed in a structured four-stage process that included initial artificial intelligence (AI)-generated items, expert refinement and pilot testing with participants from Austria and South Africa. The development process of the questionnaire is described in detail, with particular attention to the pilot phase and its implications for ensuring the quality of the instrument in relation to the use of AI in a first step.

Nowadays, AI plays a central role in research and therefore also in the creation of questionnaires. Russell and Norvig [40] and Nilsson [41] (p. 13) define AI as the process by which computers analyze vast amounts of data and apply extremely complex algorithms to simulate human behavior or reasoning and enable a society to “function appropriately and with foresight in its environment”. With the introduction of ChatGPT at the latest, AI tools were on everyone’s lips. From one day to the next, a wide variety of AI tools were available and accessible to a broad section of the population, making them widely used. The advantages of artificial intelligence are that vast amounts of data can be searched in a short space of time, which also makes it interesting for researchers from various disciplines. By comparing large amounts of data, AI systems can generate novel hypotheses or be part of autonomous research, which promises greater efficiency, accelerated work processes, accuracy and the possibility of exploring different areas [42]. Nevertheless, Landgrebe and Smith [43] argue that AI has not yet been able to master the complex and evolving patterns of human dialogue. Nevertheless, AI is playing an increasingly important role in a dynamic and interactive process of knowledge creation [44]. AI systems can help to analyze large amounts of data and recognize patterns that would be difficult for humans to discern. They can bring together different perspectives and information and thus create a broader knowledge base. At the same time, the question arises as to how AI itself generates knowledge and whether it is able to take the social and context-dependent nature of knowledge into account appropriately. AI can both reinforce and distort knowledge, depending on how it is programmed and what data it processes. When using AI, educators and researchers would need to design materials in such a way that it not only reflects one dominant perspective, but also incorporates the diversity of human experiences and perspectives. This would allow it to function not just as a technical tool, but as an active participant in the process of knowledge generation, helping to promote more inclusive and comprehensive insights.

While AI is penetrating everyday lives increasingly, in many ways it is still an abstract notion. The discourse on the risks and challenges of using AI, some of which cannot yet be assessed and some of which are not yet understood, increasingly supports this abstract view of AI. These aspects lead to an even higher responsibility on the part of scientists to break down the scientific and technical core of AI systems in order to create transparency and promote public discourse on the use of AI [42]. In addition to breaking down the scientific and technical core of AI, creating transparency also requires clear communication about the areas of a scientific process in which artificial intelligence was used, along with a valid justification. Through openness and disclosure, the quality standards of scientific work can be maintained. However, the role of the scientist does not end with mere investigation; rather, it is about subsequently checking and validating the results and findings emanating from AI use. It is therefore advisable to develop a strategy that is based on a clear objective, and which follows the tenets of science.

The seemingly biggest hurdle in using AI in the scientific process is learning how to handle AI ethically and use it in a targeted, constructive manner. There are various studies that discuss the role of AI in research, e.g., in the fight against climate change or for a sustainable future [45,46,47,48]. The benefits of using AI are visible in different areas, as it is able to detect anomalies in data, which is crucial in areas such as medical research, where AI algorithms can detect early disease onset patterns in patient data [49]. In education, AI offers the possibility to analyze individual learning challenges or use individualized tutoring chatbots [15]. In addition, it can support educators in correcting tests and designing competence-based tasks [50]. Bjola [45] (p. 86) describes AI as a new chapter in the development of theory and practice, and point to open questions about how the integration of AI could theoretically and normatively restructure the various disciplines.

Despite all the advantages, however, it is important to bear in mind the challenges that the use of AI brings with it, and this brings the role of the researcher back into the foreground. Since science makes a claim to truth, it is essential to check the accuracy of the sources. This also includes the unconditional identification of copyrights. It is necessary to reflect on the use of AI at all times in order to avoid over-reliance, which might reduce the researcher’s ability and need for critical and creative thinking. In the social sciences, the use of AI can lead to a loss of competences, such as the correct handling of sources or the understanding of the core topic of a manuscript. With all the possibilities, but also challenges, it seems necessary to agree on a legally binding framework, also to ensure fairness among researchers. It is about reflecting on the results presented by AI based on a high-quality understanding of scientific practice, recognizing the limitations and intervening, because the human being as a scientist should remain the driving force in the scientific process. AI is playing an increasingly important role in scientific research by improving data analysis, optimizing experimental design and facilitating new discoveries. However, integrating AI into research brings with it ethical challenges that need to be addressed to ensure fair, transparent and responsible use. Researchers need to be trained to use AI responsibly and adapt their work to the ever-evolving opportunities and challenges.

Against the background of sustainable development, however, the use of AI in particular needs to be scrutinized more closely. AI has the potential to make a significant contribution to sustainable development by addressing challenges in areas such as energy efficiency, resource conservation and environmentally friendly innovation. Nevertheless, the environmental and social challenges of AI must also be taken into account to ensure that its development and application are in line with the principles of sustainable development [51,52,53]. The high energy consumption associated with training complex AI models poses a significant sustainability challenge. These models require considerable computational resources, often provided by large data centers that rely on fossil fuels. The development of energy-efficient algorithms and hardware-optimized processors, as well as the use of renewable energy sources are key to reducing the environmental impact. In addition to energy consumption, resource consumption is also of crucial importance. The production of hardware for AI requires rare raw materials, the extraction of which can be ecologically and socially problematic. Therefore, sustainable development in the AI sector should promote recycling and reuse, and support circular economy models to minimize waste and optimize resource use. Ethical considerations are also central to sustainable AI development. Technology should promote social justice and not exacerbate existing inequalities. This requires transparent, fair systems and broad access to AI technologies to ensure that the benefits are equitably distributed and not reserved for a few large companies. Several strategic actions are needed to promote sustainable development in the context of AI. Investment in research and development is essential to develop energy-efficient algorithms and sustainable hardware technologies. The introduction and enforcement of policies that support sustainable practices in the AI industry and the establishment of regulatory standards for energy consumption and resource utilization are also essential. Raising awareness among developers, businesses and the public about the importance of sustainability in AI and promoting ethical use of the technology is key. In addition, collaboration between industry, research and government is crucial to develop and implement best practices in line with sustainability principles. In summary, the integration of sustainable development concepts into the development and application of AI is not only a technical challenge, but also an ethical imperative. Careful management of the environmental and social impact of AI is essential to ensure that this technology makes a positive contribution to a sustainable future [54].

2. Materials and Methods: A Questionnaire on Teacher Educators’ Professionalism in Relation to Sustainability—TESS as an Example

Questionnaires are regarded as an appropriate research tool to gather valid and reliable data on a research topic [55]. As Smeyers and Smith [56] (p. 7) explain, quantitative research is “erklärung”, i.e., an “explanation” of a phenomenon that can not necessarily be seen [1], in contrast to qualitative research, which provides “verstehen”, i.e., an “understanding” of the phenomenon [22] (p. 7). As such, a professional questionnaire should be well designed in terms of validity and reliability, but also keep in mind factors such as the use of language, clarity, level of difficulty, length [55], layout of the questionnaire, participant and sample size in order to provide useful information [57]. Mann [57] also advises placing easier questions at the beginning of the questionnaire and asking questions that require more thought later in the questionnaire. Another critical aspect, according to Mann [57], is a well-structured cover letter, as this is the researcher’s first contact with potential participants.

Online questionnaires can be used to reach a large population that is not limited to a specific location, without the process of distributing questionnaires being as time consuming [48] as it would be with the administration of hard-copy questionnaires. The structure of a high-quality questionnaire starts with an introduction that explains the purpose of the study and that participation is confidential and voluntary, and that it is possible to withdraw from the study at any time [58]. It also states the approximate time to complete the survey, which in our case was estimated at six to eight minutes. According to a study by Galesic and Bosnjak [59], participants are more likely to take part in surveys that take up to 10 minutes to complete than longer surveys. The researchers were careful to use language that is easy to understand [58], especially given that participants are based in different countries and English might not be everyone’s first language. The researchers drafted the statements and questions with care to avoid leading, loaded, two-fold and double negative questions and statements [58,60,61] throughout the survey.

The first section of the TESS questionnaire consists of eight background questions (drop-down and/or multiple-choice option), where one or more options can be selected depending on the background information required [62]. The background information provides information about the participants’ age, gender, country, level of qualification, current position, specialism, context of experience, etc. This information is important for research and reporting purposes.

After the background information, participants are given instructions explaining what is expected of them when completing the subsequent survey questions. The first set of questions in the TESS are statements based on teacher educators’ personal perceptions of their role in terms of sustainability, teacher professionalism and the education of future teachers. These questions consist of 14 quick, strongly declarative statements that capture various aspects of sustainability in education, with response options on a Likert scale [58,60] that are anchored [61] and provide a rich item pool for collecting research data [60]. The response options are clearly delineated to avoid confusion in selecting the most appropriate option [60]. They consist of the following indicators: strongly disagree, disagree, neutral, agree, strongly agree.

Statements such as “I demonstrate a deep understanding of sustainability concepts and principles” address participants’ knowledge of sustainability. Other prompts include “I incorporate sustainability topics into my teaching” and “I encourage student teachers to think critically about environmental and social issues related to sustainability”, which promotes critical thinking skills and reflects curriculum integration.

In addition, educators are asked about their sustainable practices in education through statements like “I promote sustainable practices within the lecture hall (e.g., waste reduction, energy conservation)”. In addition, the questionnaire uses statements like “I involve students in sustainability-related projects or initiatives within the community” to go deeper into community engagement and emphasize the significance of practical applications.

The item “I have general knowledge of the 17 Sustainable Development Goals of Agenda 2030” emphasizes awareness of global sustainability agendas. In addition, participants are asked to reflect on why they are motivated to establish sustainable learning environments and how they collaborate with others to incorporate sustainability themes into a variety of subject areas (items number eight and nine). The statements “I model sustainable behaviours in my daily life” and “I develop materials to support sustainability practices” not only encourage self-reflection, but also challenge educators to rethink their efforts to develop resources.

Finally, the TESS questionnaire includes questions concerning professional development, such as “I actively seek professional development opportunities related to sustainability education” as well as questions about classroom discussions about sustainability and creating an environment that can shape the training of future teachers.

As DeVilles and Thorpe [60] note, researchers can use an odd or even number of Likert-scale responses depending on their preference and the nature of the questionnaire. The advantage of using an odd number of responses is that participants have the option to choose a neutral response if none of the alternative options apply to them. However, care should be taken to ensure that participants do not lose interest in the questions and use the neutral point as an easy way to complete the survey.

The final section on knowledge about sustainability consists of three open-ended questions that give participants the opportunity to add information related to the research topic from their own perspective [58]: Which sustainability themes/topics do you know most about and are you best placed to teach? Which sustainability themes/topics would you like to learn more about in terms of your professional development? Is there anything else you would like to share with us? The qualitative data from the open-ended questions can provide new insights into the participants’ understanding of sustainability as an alternative measure. This dual approach aims to strengthen the general validity of the study findings by bridging quantitative scores with qualitative feedback. Making use of open and closed-ended questions in a survey is not an uncommon practice in research as it provides different perspectives of participants’ experiences related to the research topic [58]. Participants are, however, encouraged to limit their responses to avoid lengthy feedback [62]. Galesic and Bosnjak [59] found that participants are more likely to give shorter answers later in the survey, which could be another strategy to obtain concise answers on open-ended questions. The TESS questionnaire is designed as a formative assessment tool rather than a summative one. Its purpose is not only to quantify attitudes or knowledge, but also to provide qualitative insights that help identify areas for the professional development of teacher educators.

The TESS questionnaire is available on a Qualtrics platform, making it easy to distribute and potentially reach a larger population [62]. The link to the questionnaire is emailed to participants working in higher education in various educational departments. Although the research group cannot predict the outcome of the sample, the aim is to reach a large sample size in order to limit sampling errors associated with smaller samples and to obtain as true a picture as possible [61,63] of the professionalism of educators in higher education in relation to the Sustainable Development Goals and Agenda 2030. The researchers are aware that recruiting a larger sample might be time-consuming [63] and that the response rate might not be what they expected. They opted to send out the invitations at a time when potential participants are more likely to have time to complete the questionnaire, for example before or after the holidays. Second invitations are also sent out to remind potential participants of the research survey. Another advantage of using an online questionnaire for data collection is that participants can complete the survey in their own time when they find it convenient to do so [23].

Johnson and Christensen [61] recommend that a questionnaire be piloted by five to 10 participants. The TESS questionnaire was piloted with 10 participants from South Africa and Austria. During this process, participants could give feedback on factors such as the overall design of the questionnaire and the wording of questions [58,60,63], highlight information, statements or questions that could potentially lead to misunderstandings [64], test the validity of the scores and “provide an initial evaluation of the internal consistency of the items” [63] (p. 154).

2.1. Instrument Development

The Teacher Education and Sustainability Scale (TESS) was developed in four stages over a period of nine months (October 2023–June 2024). A core team of seven researchers in Austria and South Africa conceptualized and developed the final instrument to measure teachers’ professionalism in the context of sustainability. In Phase 1, the items of the Likert scale were generated with the aid of artificial intelligence and the suitability of the generated items was evaluated within the research team. In Phase 2, the Likert scale was refined and adjusted based on the research aim of the project, and in Phase 3, the full questionnaire with biographical details and open questions was developed with a statistical expert. In the final phase (Phase 4), the instrument was piloted in Austria and South Africa (n = 10). The current study reports on this process.

2.1.1. Phase 1: Generating Initial Items

In the first phase, three draft questionnaires were created by utilizing task-specific prompts on ChatGPT. Prompt 1 was formulated as: “Develop a 10-point scale to assess teacher professionalism in terms of sustainability. The scale should include responses on a 5-point Likert scale” (Questionnaire 1). A second questionnaire was then created by rephrasing the first prompt (Questionnaire 2). Subsequently, a second prompt was used, worded as follows: “Develop a 10-point scale to assess teacher professionalism in relation to sustainability. The scale should include responses on a 5-point Likert scale. Also include five biographical variables at the start of the scale” (Questionnaire 3). The three questionnaires were then compared by a member of the research team to assess their suitability. The criteria for comparison were clarity of wording and alignment with the central research question of the project. On the basis of this analysis, Questionnaire 3 was selected for the next phase.

2.1.2. Phase 2: Refining the Likert Scale Items

In the second phase, Questionnaire 3 was further reviewed and revised in more depth by a member of the research team. For example, the following technical adjustments were made:

• The title of the questionnaire was adjusted from “Teacher Sustainability Professionalism Scale” to the “Teacher Professionalism and Sustainability Scale” (TPSS)
• The instructions for the questionnaire were adapted to the specifics of the research project and made more informative.
• The biographical questions were expanded. Variables such as “Current position”, “Country”, “Subject field” and “Context” were included.
• The statements starting with the phrase “As a teacher, I …” were changed to “As a teacher educator, I”. For example, “The teacher actively seeks …” was reformulated to “As a teacher educator, I actively seek …”
• Some of the item descriptions in brackets were revised—for instance, from “(recycling, conserving resources)” to “(recycling, reducing my ecological footprint, reducing food waste)”.
• Specific word choices were revised throughout the questionnaire. For instance, from “guardians” to “caregivers” and from “classroom” to “lecture hall”.

2.1.3. Phase 3: Developing the Full Questionnaire

The revised questionnaire was sent to an experienced statistician for comment. Several comments were made on specific items, instructions, data brackets and the overall presentation of the questionnaire. The questionnaire was revised accordingly by a member of the research team and presented to the full research team. The questionnaire was then discussed during an online meeting and further adjustments were made. These adjustments included the development of the biographical details section, the addition of more Likert-scale questions, the addition of two open-ended questions and the rephrasing of the introductory texts for the three sections of the questionnaire. The conceptual difference between teachers and teacher educators was emphasized as a point of clarification. The questionnaire needed to focus on teacher educators, rather than the teaching profession in general due to the focus of the study.

At this stage, the questionnaire consisted of Section 1 (background), Section 2 (statements on a Likert scale) and Section 3 (open-ended questions). In addition to the online meeting, the questionnaire was sent to the entire research team for a period of one week for further consideration after the online meeting. Further recommendations were made on the wording of some items, e.g., some context-specific adaptations on qualifications between Austria and South Africa, teaching contexts and subject fields. The title of the instrument was confirmed as the “Teacher Education and Sustainability Scale (TESS)”.

2.1.4. Phase 4: Piloting of the Instrument

Once the research team was satisfied with the completed instrument, it was piloted with teacher educators (n = 6) in Austria and in South Africa (n = 4) (

Table 1. Background of the pilot participants.

Pilot Participants	Gender		Scientific Fields
	Male	Female
Austria	2	4	Teacher education and school research Educational Sciences Educational Philosophy
South Africa	1	3	Educational Psychology Science Education STEM

).

The instrument was uploaded and emailed to the pilot participants. Their responses and feedback were sent to members of the research team in Austria and South Africa respectively and collated by them. Feedback from one participant included that a few statements would be better written in the negative form. This style of phrasing statements is used to avoid respondents falling into a “response set” whereby they mark options without answering them thoughtfully [60,61] ([62] p. 281). Debates around the use of negative statements were considered, but as it is not a lengthy questionnaire, the advantages outweigh the disadvantages of using negative statements [60].

There were references made to statements that should be made clearer, such as the “type of teaching” or “level of teaching” the questionnaire refers to in the “background information” section. Another participant wanted clarity on how “broad knowledge” should be interpreted. Wording that causes confusion can lead to invalid responses [58,60] and affect the validity of the research. It was suggested that the repetition of the phrase “As a teacher educator” at the start of each statement should be removed. Unnecessary wording can have an effect on the time taken to complete the survey and contribute to “participant fatigue” [63]. Constructive, positive feedback included that the “questions and instructions are very straightforward”, which is in line with recommendations by researchers such as DeVilles and Thorpe [60], Johnson and Christensen [61] and Tymms [62]. During the pilot phase, participants could determine how long they needed to complete the questionnaire to avoid “participant fatigue” [63] (p. 154) and how much time was required to answer the questions. In relation to this statement, the feedback was that the questionnaire was “quick to complete”. This is an important factor to consider when creating questionnaires, as researchers do not want to burden participants with unnecessary, time-consuming questions, but at the same time get as many relevant and needed data as possible [62].

At the suggestions of the pilot participants, a picture of the Global Goals was added, an additional open-ended question was added and the automatic responses upon completion of the questionnaire were refined to include the contact details of the lead researchers in the two countries. The Likert scale was designed as a scale from 1 (Strongly Disagree)–5 (Strongly Agree) for all statements.

To improve the TESS questionnaire and make it available to participants, the pilot feedback was summarized in a single document. The feedback and suggestions were discussed amongst the members of the research group and the necessary changes were made accordingly.

3. Results

Table 2. TESS questionnaire Likert-scale items.

Final Instrument Items	Initial Instrument Items
As a teacher educator, I demonstrate a deep understanding of sustainability concepts and principles.	The teacher demonstrates a deep understanding of sustainability concepts and principles.
I incorporate sustainability topics into my teaching.	The teacher incorporates sustainability topics into their curriculum.
I encourage student teachers to think critically about environmental and social issues related to sustainability.	The teacher encourages students to think critically about environmental and social issues related to sustainability.
I do not promote sustainable practices within the lecture hall (e.g., waste reduction, energy conservation).	The teacher promotes sustainable practices within the classroom (e.g., waste reduction, energy conservation).
I model sustainable behaviours in my daily life (e.g., recycling, reducing my ecological footprint, reducing food waste).	The teacher models sustainable behaviours in their daily life (e.g., recycling, conserving resources).
I do not actively seek professional development opportunities related to sustainability education.	The teacher actively seeks professional development opportunities related to sustainability education.
I foster a learning environment that encourages discussions about sustainability.	The teacher fosters a classroom environment that encourages discussions about sustainability.
I collaborate with colleagues to integrate sustainability themes across various subject fields.	The teacher collaborates with colleagues to integrate sustainability themes across subjects.
I involve students in sustainability-related projects or initiatives within the community.	The teacher involves students in sustainability-related projects or initiatives within the community.
I have general knowledge of the 17 Sustainable Development Goals of Agenda 2030.	The teacher communicates the importance of sustainability to parents and guardians.
I do not actively integrate aspects of Agenda 2030 into my teaching.
I am highly motivated to create learning environments to support Agenda 2030 for Sustainable Development.
I collaborate with colleagues in my field at my/other institution(s) to increase sustainability themes in my teaching.
I have developed materials to support sustainability practices.

presents the Likert-scale items of the TESS questionnaire that emerged from the pilot study. The table shows how the initial questions evolved in comparison to the final questions.

One difference between the initial and final questions lies in their wording and clarity. The closing questions are more precise and specific, which avoids misunderstandings and allows for more consistent responses. For example, from “The teacher demonstrates a deep understanding of sustainability concepts and principles” to “I demonstrate a deep understanding of sustainability concepts and principles”. By shifting from a third-person perspective to the first person, the question becomes more personalized, which may encourage respondents to think more deeply about their own practices and beliefs. The initial questions reflect an external observational perspective, while the final questions switch to a first-person perspective. This change aims to make the assessment more introspective and personal. For example, from “The teacher incorporates sustainability topics into their curriculum” to “I incorporate sustainability topics into my teaching”. This change to a first-person perspective helps respondents assess their own actions and commitments more accurately.

The final questions introduce new aspects and topics that were not covered in the initial questions. This extension allows for a more comprehensive collection of relevant information. For example, from “The teacher communicates the importance of sustainability to parents and guardians” to “I have general knowledge of the 17 Sustainable Development Goals in Agenda 2030”. The final question adds a focus on personal knowledge of the Sustainable Development Goals, which was not addressed in the initial questions. A further difference is that the final questions tend to provide more detailed and specific content and help to pinpoint specific actions and behaviors rather than making broad, general statements. For example, from “The teacher models sustainable behaviors in their daily life (e.g., recycling, conserving resources)” to “I model sustainable behaviors in my daily life (e.g., recycling, reducing my ecological footprint, reducing food waste)”. The final version offers more specific examples of sustainable behaviors so that respondents can more easily identify and assess their own practices. New questions were also added to cover aspects that were not addressed in the initial questions but are important to the aim of the study, such as “I foster a learning environment that encourages discussions about sustainability” and “I collaborate with colleagues to integrate sustainability themes across various subject fields”. These additions help to complete the picture of teaching activities related to sustainability and provide a fuller understanding of the teacher educator’s professional identity.

In summary, the final questions represent a refined and expanded version of the initial questions. They are more personalized, cover a broader range of topics and contain more detailed content. These improvements increase the clarity and relevance of the questions and lead to more meaningful and reliable responses. The iterative process of revision and refinement ensures that the questionnaire effectively fulfils the research goals and collects relevant data for the evaluation and improvement of teacher education in sustainability.

4. Discussion and Conclusions

In the Teach4Reach 2.0project, the researchers involved decided that interviewing a large sample of people was the best way to obtain the necessary information on the professionalism of teacher educators in relation to sustainability. Both the construct of sustainability and that of professionalism raise many questions: one prevailing perspective on sustainability is that of Agenda 2030 and the Sustainable Development Goals (SDGs). As the SDGs are strongly focused on economic factors, there is a risk that this impetus will frame the outcomes of the scale under development, leaving very few likely outcomes that are not embedded in the economic framework of these goals. While the SDGs of Agenda 2030 are well-intentioned and address important global challenges, they are also rooted in an economic framework that reflects existing power structures. As a result, they may end up reinforcing the current system rather than challenging it or promoting alternatives beyond capitalism or the Anthropocene. However, the research team agreed on using the SDGs to define sustainability and measure sustainable development as a possible response to socio-cultural and environmental challenges, since Agenda 2030 was adopted by 193 countries of the UN General Assembly in 2015, including Austria and South Africa, where the current study is being conducted. As Izidinia [6] and the European Commission [5] point out, such responses have a significant impact on the professionalism of the educator’s identity, which is described as a complex interaction between societal demands and challenges, and personal and collective identities [6]. The study also aligns with calls for more systemic sustainability initiatives in higher education and the ‘third mission’ of universities to connect their teaching and research with societal challenges [2].

To gain an overview of the topic in Austria and South Africa, the researchers were faced with the task of designing a suitable instrument, a formal, standardized questionnaire, between October 2023 and June 2024. In general, it may be best to use existing questionnaires to ensure valid and reliable data on the research topic [55,56,58]. However, even after a lengthy search, no suitable instrument was found to assess teacher educators’ professional understanding of the SDGs and Agenda 2030, and the extent to which sustainability is integrated into teaching methods and practices, which led to the development of a new questionnaire to provide insights into this field of research. Concepts of ESD are often characterized by a strong tendency towards moralization and the associated individualization of the sustainability debate, implying an evaluation in terms of ‘good/less’ ‘good/bad’ [65,66]. Further, this is an approach that risks perpetuating the attitude–behavior gap, where there is agreement on the desirability of sustainable development, but actual behavioral change lags behind [67]. In the Teacher Education and Sustainability Scale (TESS), the research team took these notions into account and ensured that all important concepts, such as knowledge and understanding, transfer, creation and action in both an individual and collective sense with the idea of integrating economic, social and environmental development were covered through a structured four-stage process consisting of initial AI-generated Likert scale items (Phase 1), refinement and adaptation to the research objective within the research team (Phase 2), expert refinement (Phase 3) and pilot testing with participants from Austria and South Africa (Phase 4).

The questionnaire was structured according to Young [58], taking into account the time limit of Galesic and Bosnjak’s [51], and was designed to be completed within 10 min. The first section of the TESS questionnaire consists of eight background questions that provide information on demographic factors such as age, gender, country, level of qualification, current position, specialism, context of experience, etc. The first set of 14 questions in TESS are statements based on the teacher educators’ personal perceptions of their role in relation to sustainability, teacher professionalism and the preparation of future teachers. They provide information on the respondents’ interest in the issue and their attitudes towards sustainability. Section three, with three open-ended questions on knowledge and awareness of the issue of sustainability, gave the researchers room to explore unexpected responses and follow up on unforeseen factors that may have been overlooked [58]. While open-ended questions allow respondents to formulate their own responses, closed-ended questions allow respondents to specify the response categories most appropriate for their purposes. The five-point scale of Section 2 offers flexibility in that it can be used to look at all five groups in the overall sample, but can also easily combine the two agree positions and/or the two disagree positions when looking at sub-groups.

The TESS questionnaire is designed for the type of self-administered online questionnaire, also known as the ‘mail survey’. Respondents receive the online questionnaire in an email, read the accompanying instructions and complete it at a place of their choice [23]. To facilitate distribution and potentially reach a larger population [62], it will be made available on the Qualtrics system. Self-administered questionnaires should be kept as simple, short and self-explanatory as possible to ensure that instructions are short and clear, response categories are unambiguous, and the line of questioning avoids complicated skip patterns [57,62]. One of the greatest challenges for researchers is to design and develop a standardized questionnaire that meets the data needs and is designed in such a way that each respondent understands the intent of each item and is willing and able to respond [55]. The consideration that the questionnaire cannot be adapted once fieldwork has begun emphasizes the importance of carefully designing and pre-testing of the survey instrument. Therefore, a pilot test was conducted with 10 participants working in higher education in different education sectors in Austria and South Africa. The second important variable affecting the design of the questionnaire was also taken into account, namely the type of sample from which data are to be collected. Since well-designed questionnaires should provide the most complete and accurate information possible, the test phase was very helpful and led to optimization of the questionnaire. The testing process determined whether respondents understood the questions and whether they were able to complete the tasks or had the information required by the questions or prompts. The test phase also provided the most direct evidence of the validity of the questionnaire data for most items and led to a reformulation of the questions to make them more specific and relevant to the participants (see Table 2 for the Likert scale items of the TESS questionnaire resulting from the pilot study).

Additional help in designing a questionnaire came from numerous sources, besides the testing process with representatives of the participants, roughly stratified by age, gender and education. It also came from a literature review, discussions of design challenges with colleagues and experts, with the aim of changing the researchers’ preconceptions. All of this was taken into account in the questionnaire design process. To obtain general input on existing information about the topic, generative artificial intelligence (AI), such as ChatGPT, assisted in the first phase to create three draft surveys with multiple choice and short text questions. AI questionnaires overcome several limitations of traditional surveys, from low response rates and biased answers to respondents’ inability to understand complex questions and researchers’ inability accurately to gauge user sentiment. That said, the use of AI raises ethical and legal concerns, such as a lack of trust in relation to bias or privacy issues. For example, the current study recognizes the risk of potential bias if AI algorithms perpetuate or reinforce existing prejudices and create a more unequal and divided society. The introduction of a four-stage approach to instrument development was intended to compensate for this potential effect. In addition, AI does not have enough contextual information and cannot adapt in a way similar to human experts. For this reason, open questions were included in the TESS questionnaire. There is also the question of whether AI will be able to take all important aspects into account and how creative the approach to generating instrument items will be. In this respect, several versions of the questionnaire were created for the current study. AI also raises important questions about its own sustainability: its operation requires enormous amounts of energy, so it may obscure rather than solve the problem of energy consumption. On the other hand, it was possible in the current study to develop initial ideas and a framework of questions relatively quickly, whereupon the instrument was developed independently of the use of AI. The combination of traditional and AI questionnaires can provide valuable insights from the participants and collect more accurate and reliable data, while carefully considering the risks and challenges [42]. The discourse and reflection phase in the research team during the development of the questionnaire was a crucial factor for the quality of the instrument and the guarantee of achieving the objectives of the study. In this study, the researchers tried to mitigate some of the risks associated with the use of AI by working intensively on the development of the instrument over a longer period of time, holding detailed discussions and setting up several feedback loops. In this study, AI provided initial assistance in generating some ideas, but the subsequent developmental process highlights the importance of critical engagement by humans, the application of scientific thinking and the value of expert knowledge [43]. The development of quantitative instruments should not only rely on AI; for qualitative research, a multi-perspective discussion with experts can add value to the complex field of teacher educators’ professionalism and its measurement.

The TESS questionnaire is the first of its kind to explore the topic of teacher educators’ professionalism and their approach to sustainable development in the context of Agenda 2030. By measuring the professionalism of teacher educators in this regard, the study aims to contribute to the creation of knowledge about sustainability and to leverage the potential influence of teacher educators on the professional practice of future teachers.