Developing and Validating an Instrument for Assessing Learning Sciences Competence of Doctoral Students in Education in China

Abstract

Learning sciences competence refers to a necessary professional competence for educators, which is manifested in their deep understanding of learning sciences knowledge, positive attitudes, and scientific thinking and skills in conducting teaching practice and research. It is of paramount importance for doctoral students in education to develop their competence in the field of learning sciences. This will enhance their abilities to teach and conduct research, and guide their educational research and practice toward greater sustainability. In order to address the shortcomings of current assessment instruments, we constructed a theoretical model for assessing learning sciences competence based on the PISA 2025 framework and Piaget’s theory of knowledge. A three-dimensional assessment framework was designed, along with an initial instrument. Furthermore, the “Delphi method based on large language models (LLM)” was employed to conduct two rounds of expert consultations with the objective of testing and refining the instrument. Throughout this process, we developed a set of guidelines for engaging AI experts to improve interactions with LLM, including an invitation letter to AI experts, the main body of the questionnaire, and the general inquiry about AI experts’ perspectives. In analyzing the results of the Delphi method, we used the “threshold method” to identify and refine the questionnaire items that performed sub-optimally. This resulted in the final assessment instrument for evaluating learning sciences competence among doctoral students in education. The assessment instrument encompasses three dimensions: the knowledge of learning sciences, application of learning sciences, and attitude towards learning sciences, with a total of 40 items. These items integrate Likert scales and scenario-based questions. Furthermore, the study examined potential limitations in the item design, question type selection, and method application of the assessment instrument. The design and development of the assessment instrument provide valuable references for the standardized monitoring and sustainability development of the learning sciences competence of doctoral students in education.

1. Introduction

The learning sciences is an interdisciplinary field, encompassing various disciplines such as neuroscience, cognitive science, psychology, and education [1,2]. It studies the mechanisms of human learning and constructs appropriate learning environments, particularly technology-enhanced environments, with the aim of supporting human learning [3,4]. In 2017, the OECD released the report “Teachers’ Pedagogical Knowledge and the Teaching Profession”, which proposed that teachers’ knowledge should include not only content and classroom management knowledge but also knowledge about learners and learning. This indicates that understanding and applying learning sciences are important skills for teachers [5]. In China, learning sciences are regarded as an important theoretical framework for future educational reforms [6]. In 2019, the Ministry of Education of China issued the “Opinions on Strengthening Research on Educational Science in the New Era” [7], which emphasized the need to “fully utilize the latest achievements and research methods in the fields of cognitive science, neuroscience, and comprehensively apply new technologies such as artificial intelligence to carry out educational research” [7]. This further confirms the important position of learning sciences in educational research. Consequently, it is imperative for every education professional to possess an understanding of learning sciences. In this context, the concept of learning sciences competence has emerged as a crucial indicator for evaluating the competence of educators to enhance teaching and research through the application of learning sciences [8].

Doctoral students in education constitute a significant proportion of the workforce in the field of education. They are divided into two categories: Doctor of Philosophy (Ph.D.) students in Education and Doctor of Education (Ed.D.) students. Ph.D. students in Education are individuals pursuing a Doctor of Philosophy degree in education with the potential to become educational researchers, teacher educators, or scholars engaged in the study of educational theory and practice. Ed.D. students are managers, administrative leaders, curriculum specialists, and other professionals in the field of education who are pursuing a Ph.D. degree in education with the objective of addressing more effectively the challenges encountered at the frontline of education [9]. Despite the existence of clear distinctions between the two groups in terms of their training in academic and practical contexts [10,11], both possess a dual identity as educational researchers and teachers or future teachers. Consequently, the construction of a knowledge competence system with learning sciences at its core has become a common pursuit for both the groups.

It is of significant to focus on the development of specialized assessment instruments for learning sciences competence among doctoral students in education in China. From a global perspective, as of 2023, the number of graduate students in China has reached 3.65 million, ranking second globally, making China a major country in graduate education. According to the “Basic Situation of National Education Development in 2023” report released by the Chinese Ministry of Education, China admitted 153,300 doctoral students and had 612,500 doctoral students enrolled in 2023. Among them, the enrollment volume of doctoral students in education is large and continuously expanding. Conducting research on such a scale of a group can form exemplary practices for the international education field to draw upon. Secondly, a significant proportion of doctoral students in education in China exhibit a relatively narrow and outdated understanding of learning. This is characterized by a view of learning as “knowledge acquisition” and “skill training” [12], which starkly contrasts with the direction of educational reform led by learning sciences. Although research literature on the development of learning sciences competence among educators has grown in recent years [13,14], the majority of the existing studies remain at the theoretical level. There is a clear need for learning sciences competence assessment tools, in particular for doctoral students in education, who represent a significant and under-researched group [15]. This contradiction is reflected in the fact that although the number of related studies has increased and a certain number of assessment tools have appeared, these tools have not been specifically designed to meet the characteristics of doctoral students in education. Furthermore, they lack practicality and scientific rigor, which prevents them from comprehensively assessing the learning sciences competence of doctoral students in education [16].

Therefore, there is an urgent need for a scientifically reliable assessment instrument to gain an in-depth understanding of learning sciences competence among doctoral students in education. This will enable them to remain forward-thinking and innovative in their understanding of learning sciences and to prepare them to become the backbone of global education for sustainable development. The assessment instrument serves as a conduit between doctoral students in education and the field of learning sciences, fostering the continued comprehension of learning sciences among doctoral students in education. The assessment instrument encourages doctoral students in education to review their educational research and practice, diagnose any shortcomings in the field of learning sciences, and develop a plan for continuous improvement. Furthermore, the field of learning sciences is a rapidly evolving interdisciplinary field. The assessment tool requires doctoral students in education to approach learning sciences with a long-term perspective and to consider the integration of knowledge and technology from different disciplines in order to address sustainability challenges in education with innovative solutions. This will not only have a profound impact on the sustainable reform of China’s education doctoral training system, but also provide a valuable reference for countries around the globe when training education doctoral students.

Building upon this foundation, the research questions of this study are as follows:

1. What should be included the assessment of the learning sciences competence of doctoral students in education?

2. How can appropriate assessment instrument items be developed?

3. How to effectively validate assessment instrument?

2. Literature Review

In previous studies, researchers have concentrated on a diverse array of research topics within the field of learning sciences. These include the concepts and theories of learning [17], the stages of development of learning sciences [2], and the role of learning sciences in education [18]. However, a review of past research revealed two themes that are closely related to this study. First, the conceptual definition of learning sciences competence, and second, the development of assessment instruments for learning sciences competence and current status surveys.

2.1. Conceptual Definition of Learning Sciences Competence

Although there are differences in the conceptual definition of learning sciences competence in both the Chinese and English literature, it is generally recognized as an integration of the concepts of competence and learning sciences. In Chinese research, Shang Junjie from the School of Education at Peking University first focused on this concept in 2017 [19], followed by researchers who delved into the dimensions of learning sciences competence, including dimensions such as the knowledge, application, and awareness of learning sciences [20]. In English research, although there is no directly corresponding academic concept, studies generally emphasize that teachers should have the competence to apply learning sciences. The OECD published “The Nature of Learning: using research to inspire practice”, which aims to explore the nature of learning and provide professional guidance for teachers, educational researchers, and educational leaders to improve learning environments [21]. Mayer emphasizes the need to popularize learning sciences perspectives among “novices” and to strengthen research and practice in “applying learning science” [22]. All of this points to the need for educators to apply learning sciences knowledge to solve practical problems. In summary, learning sciences competence refers to a necessary professional competence for educators, which is manifested in their deep understanding of learning sciences knowledge, positive attitudes, and scientific thinking and skills in conducting teaching practice and research.

2.2. Assessment Instruments for Learning Sciences Competence

An assessment instrument is defined as the actual device or method of gathering evidence [23]. In the field of education, assessment instruments are frequently employed to evaluate a teacher’s knowledge, skills, performance, or competence [24]. In recent years, researchers have developed a number of assessment instruments to evaluate the learning sciences competence of educators. In 2014, the Learning Sciences Research Center at East China Normal University conducted a special survey on “Teachers’ Understanding of Neuroscience” for primary and secondary school teachers in the eastern coastal areas of China. The survey resulted in the development of an assessment instrument focusing on neuromyths. The assessment instrument comprises 40 true/false questions in the field of neuroscience [14]. In 2017, the Learning Sciences Laboratory at the School of Education, Peking University initiated the “Enhancing Teachers’ Learning Sciences Competence” project and developed a self-report scale focusing on learning sciences awareness, knowledge, and abilities [25]. In 2023, Zhang Ye developed a learning sciences competence questionnaire for pre-service teachers. This instrument includes three dimensions: learning sciences knowledge, abilities, and awareness, with a total of 48 items [16].

In 2019, The Learning Agency initiated a survey project titled “What Do Teachers Know About The Science of Learning?” and developed a set of context-based test items to assess the teachers’ understanding of learning strategies. In the same year, Deans for Impact released a report entitled “Learning by Scientific Design”, which included the design of a scenario-based test comprising 54 items, which were divided into three categories: understanding, practice, and beliefs in learning sciences principles.

In general, the existing assessment instruments for learning sciences competence have been subjected to rigorous design and validation processes. Nevertheless, there is a dearth of instruments suitable for large-scale surveys and doctoral students in education. The majority of the existing tools rely on self-report scales or scenario-based test items. The former is often lacking in a robust theoretical foundation and depth of inquiry, while the latter, although more comprehensive, lacks breadth [26]. Consequently, we propose the development of a comprehensive and systematic assessment instrument that combines self-report scales with scenario-based assessments. This instrument will integrate competence assessment frameworks with specific content from the learning sciences field.

3. Developing and Validating Learning Sciences Competence Assessment Instrument for Doctoral Students in Education

The objective of our study is to develop and validate an assessment instrument for learning sciences competence among doctoral students in education. The instrument will cover four key stages [27,28]: Firstly, the theoretical foundation of the assessment instrument will be established based on typical competence assessment frameworks and instruments. Secondly, the assessment framework will be designed by integrating content from the learning sciences field. Thirdly, an initial item bank will be developed and items will be screened by referring to the existing assessment instruments. Finally, a modified Delphi method [29] will be employed to scientifically validate the reliability and validity of the assessment instrument.

3.1. The Theoretical Foundation of the Assessment Instrument for Learning Sciences Competence

The theoretical foundation of learning sciences competence assessment instruments is currently incomplete. Consequently, it is necessary to seek theoretical references for instrument design from a broader and relatively mature competence assessment perspective. In April 2022, the Chinese Ministry of Education released the “Compulsory Education Curriculum Scheme and Curriculum Standards” [30], which includes the core competence framework for the science. The framework encompasses four domains: scientific concepts, scientific thinking, inquiry practice, and attitude and responsibility. While this framework provides valuable reference points for identifying competence elements, it does not provide a comprehensive analysis of the structural relationships between these elements. This hinders our dynamic and comprehensive understanding of competence formation processes. In June 2023, the OECD released the PISA 2025 Science Framework (Draft), which explicitly outlines the framework for assessing scientific competence. As illustrated in Figure 1, this framework serves as a foundational theoretical framework for our research [31]. As an internationally typical competence assessment paradigm, the PISA testing project has become a globally influential competence assessment project since its inception in 2000. This framework encompasses not only the fundamental elements of knowledge, skills, and identity recognition but also bridges these elements with impact and demonstration, thereby elucidating the formation process of competence.

A comprehensive analysis of typical competence assessment frameworks revealed that competence is primarily comprised of core elements, including knowledge, attitude, thinking, and competence. Piaget’s constructivist epistemology posits that thinking serves as the mediator of knowledge acquisition [32]. The theoretical perspectives upon which this model is based include the PISA 2025 assessment framework, the core competence framework in the field of science, and Piaget’s constructivist epistemology. Figure 2 illustrates the theoretical model for assessing learning sciences competence. This model elucidates the four core elements of knowledge, attitude, thinking, and competence, as well as their structural relationships. In this model, knowledge and attitude towards learning sciences serve as the foundation, jointly influencing the formation of learning sciences thinking. Thinking then serves as a bridge, facilitating the development of learning sciences competence. The model underscores the fact that the acquisition of learning sciences thinking and the development of learning sciences competencies are embodied and validated in the real world, particularly in the specific scenarios of educational practice and educational research activities.

It is important to highlight that in comparison to typical competence assessment frameworks, such as the Framework for PISA 2025 science assessment, the theoretical model for assessing learning sciences competence that has been constructed is a specific framework that is focused on the assessment of learning sciences competence. Although it draws on the structural relationships of the Framework for PISA 2025 Science Assessment, the underlying goal is to accurately reflect the structural characteristics of learning sciences competence. This model provides a theoretical foundation for the design of assessment frameworks and instruments for learning sciences competence.

3.2. Design of Learning Sciences Competence Assessment Framework

Building upon the theoretical model for assessing learning sciences competence, we have constructed the learning sciences competence assessment framework. This framework comprises three dimensions: the knowledge of learning sciences, attitudes towards learning sciences, and application of learning sciences.

Table 1. Learning sciences competence assessment framework.

Assessment Domain	Secondary Dimension	Tertiary Dimension	Reference
Knowledge of Learning Sciences	Content Knowledge	Overview of Learning Sciences; Research Topics and Main Viewpoints; Theoretical Foundations	[3,4]
	Procedural Knowledge	Various Methodologies Applied by Learning Scientists
	Cognitive Knowledge	Understanding the Rationale Behind Conducting Learning Sciences Research Selecting Appropriate Methodologies to Address Problems Critically Examining Research Findings in Learning Sciences
Application of Learning Sciences (Thinking of Learning Sciences and Competence of Learning Sciences)	Learning	Judging the Effectiveness of Learning Strategies	[22]
	Teaching	Implementing Teaching Practices Based on Learning Sciences
	Assessment	Conducting Teaching Assessment Based on Learning Sciences
	Research	Conducting Research in Learning Sciences
Attitudes towards Learning Sciences	Beliefs in Learning Sciences	Understanding the Value	[25,33]
	Attitudes towards Learning Sciences	Identification; Interest
	Willingness to Engage with Learning Sciences	Participation in Training and Development; Adjusting Teaching and Research

presents the specific contents.

The field of learning sciences encompasses the cultural and intellectual achievements in the field of learning sciences. In accordance with the PISA 2025 framework, the knowledge classification of learning sciences can be divided into three categories: content knowledge, procedural knowledge, and cognitive knowledge. Content Knowledge encompasses the fundamental facts, concepts, viewpoints, and theories that form the basis of learning sciences knowledge. Procedural Knowledge emphasizes the standard procedures and practices utilized by learning scientists to obtain reliable and effective knowledge, including research methods and experimental design skills. Cognitive Knowledge focuses on the structure and defining features of knowledge construction in learning sciences, with an emphasis on understanding the functions and basic principles of learning sciences [2].

The application of learning sciences involves teachers utilizing learning sciences to enhance the scientific nature of teaching practices and research, assisting learners in effective learning processes in real-life situations, and involves the application of learning sciences thinking and competence. The term “thinking of learning sciences” refers to the ability to comprehend the nature, laws, and relationships of learning from a scientific perspective [33]. Conversely, the term “learning sciences competence” is indicative of the capacity to utilize learning sciences to address practical teaching and research issues [34]. Mayer outlines the application of the science of learning in the following manner: In “Psychology Masters’ Advice for Teachers”, the application of learning sciences is divided into three categories: learning, teaching, and assessment [22]. In considering the characteristics of doctoral students in education, the element of “research” is incorporated to form the four core elements of applying learning sciences: learning, teaching, assessment, and research. The scientific understanding of the nature of learning and the effectiveness of learning strategies is essential for the application of learning sciences in the field of education. This understanding informs the selection of learning strategies based on context and the design of learning practices. The evaluation of the effectiveness of learning strategies follows the principles of learning sciences. Finally, the scientific conduct of learning research is a crucial aspect of the application of learning sciences in education.

The attitude towards learning sciences is reflective of the connection, interest, and sense of identification between teachers and learning sciences. It is the fundamental basis for the active application of learning sciences in teaching, research, and practice. This can be examined from three dimensions: beliefs, attitudes, and willingness to apply learning sciences [33]. Beliefs pertain to the cognitive understanding of the value of learning sciences, attitudes represent the emotional inclination towards learning sciences, and willingness denotes the determination to apply learning sciences in practice.

3.3. Selecting and Developing Learning Sciences Competence Assessment Items

A systematic and rigorous strategy was employed to develop the initial assessment item pool. Firstly, in order to ensure the validity of the content of the initial item pool, its scope was limited to the learning sciences competence assessment framework. The framework is organized around three key dimensions: the knowledge of learning sciences, application of learning sciences, and attitudes towards learning sciences. These dimensions provide a foundation and clear direction for item design. Secondly, the construction of items in the item pool was based on an extensive literature review. The items were selected and adapted from the previous studies that fit the theme. For instance, we referenced the study by Li et al. and transformed it into our Q3 [25]. Third, the contextualized question items from the learning sciences competence assessment tools developed by The Learning Agency, Deans for Impact, and other international educational research institutions were selected, Chineseized, and adapted to fit the context. For instance, Q16 was derived from the Deans for Impact questionnaire. In the process, we made the necessary deletions and culturally adapted items in order to facilitate the understanding of Chinese doctoral students in education. Fourth, we made extensive reference to classic books in the field of learning sciences. For instance, we drew upon Meyer’s Applied Learning Sciences: Advice for Teachers from a Master Psychologist [22] to extract and convert the practice cases into assessment questions (e.g., Q21 to Q23). These questions were directly related to teaching practice and effectively assessed the competence of doctoral students in Education to apply learning sciences theories to teaching practice. Finally, in order to ensure the linguistic accuracy of the assessment, all the questions were meticulously translated into Chinese, abridged, and adapted. This process not only aims to ensure linguistic accuracy but also strives to be natural and fluent in presentation, reducing the barriers to understanding caused by cultural differences. It also ensures that all the users, including the Chinese doctoral students in education, can accurately understand the meaning of the questions.

During the development of the initial item pool, we consistently prioritized the relevance of the questions to the doctoral students in education. In comparison to in-service or pre-service teachers, there are heightened external expectations for doctoral students in education in educational research. They are expected to possess deep theoretical roots, critical thinking, and innovation. Consequently, we concentrated on the doctoral students in education’ comprehension of research methods in the learning sciences when evaluating their application of the learning sciences. For example, for Q24 to Q27, we designed a specific situation that is analogous to the typical research activities of doctoral students in education. This was performed with the intention of examining their ability to apply learning sciences to conduct educational research in a real-world setting. For other educators, such as in-service teachers, we may adjust the formulation of the questions to reduce the direct use of technical terms such as “regression analysis” and replace them with assessment content that is more relevant to their daily practice. Furthermore, in consideration of the fact that some Ph.D. students in Education may lack direct experience in the classroom, we have incorporated role-playing and the simulation of real-life teaching scenarios into the design of the questions. This was performed with the intention of providing a bridge between the theoretical and practical aspects of the subject matter, allowing the students to gain a deeper understanding of the questions even if they lack direct teaching experience. In Q20, for instance, the participants assumed the role of “language teachers” and were tasked with guiding the students to explore the connection between the purpose of writing and the genre of the text. By presenting the scenario in a realistic manner, the Ph.D. students in Education were able to rapidly immerse themselves in the role of the teacher.

The item pool development and selection strategy previously described was employed to generate the initial assessment. The instrument is divided into two sections, comprising a total of 43 items. The initial section of the instrument comprises six questions, which seek to ascertain basic information such as gender, age, educational background, academic level, major, and experiences related to learning sciences courses. This information is included to facilitate demographic analysis. The second part of the instrument constitutes the core assessment content, comprising 37 items. The items on the instrument reflect learning sciences competence across the three dimensions: knowledge, application, and attitude towards learning sciences.

Table 2. Preliminary instrument for assessing learning sciences competence of doctoral students in education.

Assessment Domain	Secondary Dimension	Specific Items	References
Knowledge of Learning Sciences	Content Knowledge	Q1. I am able to elucidate the definition of learning sciences. Q2. I am able to provide an overview of the historical and developmental aspects of the field of learning sciences. Q3. I am able to provide a comprehensive list of prominent experts and scholars, as well as journals and academic conferences, in the field of learning sciences, both domestically and internationally. Q4. I am able to delineate the mechanisms and processes underlying the occurrence of learning. Q5. I am able to design appropriate learning environments based on different learning theories for real-life situations. Q6. I am conversant with the techniques of learning analysis, including classroom discourse analysis and online learning analysis. Q7. I am able to delineate the concepts and roles of learning theories. Q8. I am able to delineate the fundamental tenets of behaviorism, cognitivism, constructivism, and humanism as they pertain to the field of learning theory.	[4,35]
	Procedural Knowledge	Q9. I am aware of the advantages, disadvantages, and appropriate contexts for research methods such as experimental and survey research. Q10. I am conversant with the general process of Design-Based Research (DBR) methods. Q11. I am conversant with the methodologies employed in neuroscience research, including neurophysiological techniques such as eye-tracking and the use of wearable devices. Q12. I am conversant with the research methods and technologies based on big data and artificial intelligence, such as educational data mining.	[36]
	Cognitive Knowledge	Q13. It is evident that the essence of learning sciences is the elucidation of the nature of learning and the optimization of learning environment design. Q14. I am able to select the most appropriate learning sciences research methods for a given context with the aim of addressing the problems at hand. Q15. I am able to critically examine the current research outcomes in learning sciences.	[3]
Application of Learning Sciences	Learning Strategies	Q16. Which review strategy is more effective for students when reviewing previously learned concepts: self-testing (e.g., doing practice questions) or re-reading (repeating textbook or notes)? Q17. Which learning strategy is more effective: highlighting key points with a highlighter or pen, or recording knowledge on blank paper? Q18. Which is more effective: integrating information through text and images, or providing text and images separately? Q19. Which learning strategy is more effective: “interleaved practice” (alternating practice with different types of questions) or “blocked practice” (concentrating practice on similar types of questions)?	[37,38]
	Teaching Practices	Q20. As a language instructor endeavoring to facilitate students’ comprehension of the influence of writing purposes on text types, which approach would you select? Activity 1: Have students read news reports and opinion articles from newspapers. Then, in groups, have them discuss how to identify the author’s persuasive intent, the differences between the two, and how to reflect different purposes. Additionally, it is important to consider how to adapt the approach if the purpose of the news report shifts from informative to entertainment. Activity 2: Conduct a newspaper treasure hunt activity where students search for article titles with different writing purposes and record them in a table to visually understand the impact of writing purposes on text types. Q21. When presenting a teaching animation on the formation of thunderstorms, which approach would you select? Approach 1: The 2.5-min animation should be played continuously. Approach 2: The animation can be divided into 16 segments, each approximately 10 s in length, with accompanying descriptions. In order to facilitate the learning process, it is advisable to include a “continue” button, which will allow learners to click and play the next segment. Q22. When instructing students in the operations of algebraic equations, which pedagogical approach would you select? Approach 1: Provide students with algebraic problems for practice and mastery through exercises. Approach 2: Provide illustrative examples for students to learn from initially, and then present analogous problems for practice.	[22]
	Teaching Assessment	Q23. As a mathematics educator, following the instruction of binomial probability problems, a questionnaire was administered to ascertain the extent of student comprehension. The results indicate that the majority of the students believe they have a satisfactory grasp of the concepts. What inferences can be drawn from this data?	[22]
	Scientific Research	You are researching the factors influencing students’ learning effectiveness. Please choose the most appropriate option. Q24. Which question is more suitable for the study of the factors influencing students’ learning effectiveness? A. The relationship between study habits and academic performance B. The correlation between teacher age and teaching effectiveness Q25. After collecting data on student classroom participation, homework completion, and grades, how should you choose the analysis method? A. Summarizing data using descriptive statistics B. Using causal analysis models to explore the relationship between variables Q26. Upon the analysis of the data, it became evident that there was a significant correlation between a specific variable and academic performance. However, the causal relationship between the two remains uncertain. What is the appropriate course of action? A. Conduct regression analysis to explore causality. B. It is recommended that control variables be added in order to eliminate potential influences. Q27. Empirical evidence indicates that a specific learning methodology has a positive effect on academic performance. How can the reliability and validity of the results be ensured? A. Draw conclusions based solely on the results of this study. B. Compare and discuss the results of this study with those of other studies.	[39]
Attitudes towards Learning Sciences	Beliefs in Learning Sciences	Q28. I am aware of the significance of learning sciences in the context of both teaching practice and research. Q29. I am aware of the advantages and limitations of the development of learning sciences research.	[16,33]
	Attitudes towards Learning Sciences	Q30. I am eager to pursue further knowledge in the field of learning sciences. Q31. I am able to utilize learning sciences to address both teaching practice and research questions. Q32. I am able to adhere to the ethical principles of learning sciences research. Q33. I am profoundly interested in the field of learning sciences. Q34. I am proactive in disseminating knowledge about learning sciences to those around me.
	Willingness to Engage with Learning Sciences	Q35. It is my hope that I will have the opportunity to participate in learning sciences-related courses or training activities. Q36. I am eagerly anticipating the opportunity to apply learning sciences in future teaching and research endeavors. Q37. I am prepared to modify my previous teaching and research methodologies in accordance with the findings of learning sciences research in a timely manner.

presents the item content in detail.

It is noteworthy that the questions for knowledge of learning sciences (15 questions) and attitudes towards learning sciences (10 questions) were derived from a five-point Likert scale format. In this format, the participating doctoral students in education were asked to rate the accuracy of a series of statements on a scale of 1 to 5, with higher ratings indicating higher levels of proficiency in that dimension. In the case of the application of learning sciences (12 questions), we employed the concept of situationalized assessment, whereby the questions were integrated with real-life situations and the respondents were required to solve the problems in the simulated real-world environments. Once the questionnaire has been collected, the researcher will assign a value to each question item according to the response situation. The correct value will be assigned a value of “1”, while the incorrect value will be assigned a value of “0”. The total of the resulting values will yield the actual level of competence in learning sciences among the doctoral students in education.

3.4. Validating the Learning Sciences competence Assessment instrument

In order to enhance the reliability and precision of the assessment instrument, an enhanced Delphi method will be employed in order to validate and refine the instrument.

3.4.1. The Validation Method: The Delphi Method Based on LLM

The Delphi method is a research approach that aims to solicit opinions from a panel of experts and strive for a high level of consensus. The Delphi method is distinguished by several key features, including the anonymous participation of experts, multiple rounds of feedback exchange, and evidence-based iterative updates. These distinguishing characteristics differentiate the Delphi method from other foresight research methods [40]. In the field of education, the Delphi method is employed primarily for the establishment of indicator systems, the assessment and modification of scales, and the formation of intervention plans [41]. Nevertheless, it should be noted that the Delphi method is not without its limitations. Potential challenges include the influence of subjective biases among human experts, discontinuity due to expert withdrawal, and the lack of real-time interaction resulting from lengthy processes [42,43].

In response to these challenges, researchers have sought to enhance the Delphi method, leading to the development of various variants. These include the “modified Delphi method”, the “technological foresight Delphi method”, and the “modified Delphi method based on BP neural networks” [44,45]. Our objective is to make groundbreaking advancements on the existing foundation by ingeniously integrating the Delphi method with artificial intelligence technology. This integration introduces large language models (LLMs) to replace traditional human experts in the Delphi method’s polling process [46,47]. This approach not only helps to avoid potential biases caused by subjective factors such as the individual emotions of experts but also greatly enhances research efficiency and shortens the research cycle. This approach introduces new vitality and innovation into the research methods employed in the field of education.

3.4.2. Validation Steps

The Delphi method comprises a series of key steps, including the definition of the research topic, the selection of expert panel members, the preparation of expert inquiry forms, the conduct of multiple rounds of expert consultations, and the analysis of survey results [48]. Building upon the classic approach, we innovatively employ the “LLM-based Delphi method” for validation. The process of the Delphi method based on large language models involves the following steps [47]: (1) it is recommended that a group of LLMs be recruited as inquiry experts and trained using the Prompt command, (2) preparing inquiry instructions that are understandable to AI experts, (3) defining technical indicators and parameters that require in-depth analysis to provide clear guidance for subsequent consultations and data analysis, and (4) collecting data for statistical analysis. This process adheres to the fundamental principles of the Delphi method while leveraging the unique features of LLM applications, thereby providing a solid methodological foundation for research.

• Recruitment of AI Experts

In general, the recruiting of “expert” samples for a Delphi method is not random, but rather based on specific criteria [49]. The training data, model structure, functionality, and application scenarios may vary from one LLM to another, which may lead to differences in the feedback and results obtained when working on the same problem [50]. Consequently, when recruiting AI experts, a number of steps were taken with the objective of ensuring broad representation. First, we delineated the specific skills of the AI experts, such as their applicability to the educational arena and their bilingual natural language processing competence. This helped us to narrow down the candidate LLM models. Secondly, an investigation was conducted into the existing LLM models currently available on the market. This involved an assessment of their training data sources, model architectures, performance evaluation reports, and user feedback. The objective was to ascertain the independence of their Chinese–English databases and the diversity of their algorithms. Finally, the LLMs that met the established criteria were selected. Four recruitment criteria were referenced in order to select the LLMs in the market: the diversity of pre-training databases, diversity of algorithms, diversity of developer backgrounds, and accessibility. Ultimately, 16 LLM-based AI experts were identified.

Table 3. AI experts based on LLM.

Number	Name	Developer	URL	Access Date
1	ERNIE Bot	Baidu	https://yiyan.baidu.com/	accessed on 12 March 2024
2	Spark Desk	Iflytek	https://xinghuo.xfyun.cn/	accessed on 12 March 2024
3	Tongyi Qianwen	Alibaba Group	https://qianwen.aliyun.com/	accessed on 13 March 2024
4	doubao AI	ByteDance	https://www.doubao.com/	accessed on 13 March 2024
5	Baichuan	Baichuan Intelligent Technology	https://www.baichuan-ai.com/	accessed on 13 March 2024
6	360 GPT	360	https://chat.360.com/	accessed on 13 March 2024
7	HunyuanAide	Tencent	https://hunyuan.tencent.com/	accessed on 14 March 2024
8	Kimi Chat	Moonshot AI	https://kimi.moonshot.cn/	accessed on 14 March 2024
9	Tiangong Chat	KUNLUN TECH	https://chat.tiangong.cn/	accessed on 14 March 2024
10	ZhiPu AI	ZhiPu AI	https://chatglm.cn/	accessed on 14 March 2024
11	GPT-3.5	OpenAI	https://openai.com/	accessed on 14 March 2024
12	Copilot GPT	Microsoft	https://copilot.microsoft.com/	accessed on 14 March 2024
13	Claude-3-Sonnet	Anthropic	https://www.anthropic.com/	accessed on 15 March 2024
14	Gemini-Pro	Google	https://poe.com/Gemini-Pro	accessed on 15 March 2024
15	PaLM	Google	https://ai.google/discover/palm2/	accessed on 15 March 2024
16	Mistral-Large	Mistral AI	https://mistral.ai/news/mistral-large/	accessed on 15 March 2024

presents the specific information pertaining to each LLM.

2.

Training of AI Experts

Although LLM has a wide range of potential applications, its direct use in specific scenarios may result in biased results, which is particularly evident in complex educational scenarios [51]. This also demonstrates that the direct application of unguided LLM may not be sufficient for a deeper understanding of the learning sciences domain [52]. In order to overcome such limitations, it is necessary to train the model in a careful, goal-oriented manner. As users, we are unable to directly intervene to access the core dataset or training process of the LLM when interacting with it. However, we can guide the LLM to learn and apply domain-specific knowledge with the help of prompts, which are user-constructed guided commands that are used in the application of the LLM in order to efficiently direct the LLM to produce high-quality, domain-relevant responses [53]. The utilization of prompt instructions is employed to train the LLM-based AI experts in the domain of learning sciences. The core of the process is that prompt instructions not only introduce classic literature, books, and other resources in the learning sciences, but also explicitly provide instructions such as “Deeply understand and internalize these findings”, which leads to a deeper understanding of the body of knowledge in the learning sciences by the LLMs [54]. Consequently, the model is now equipped with a comprehensive understanding of the field of learning sciences, enabling it to draw upon the essential elements of the field with greater accuracy and depth, resulting in more accurate and in-depth responses.

3.

Compiling AI Expert Inquiry Instructions

Based on the preliminary Instrument for Assessing Learning Sciences competence of Doctoral Students in Education, we have compiled a set of inquiry instructions for the AI experts. The instructions consist of three parts: first, the “ Invitation Letter to Experts”, which provides the AI experts with detailed information about the background, content, purpose, and instructions for completing the questionnaire; second, the main body of the instrument inquiry, including specific content of each item, importance ratings (assigned values from 1 to 5), and modification suggestions; and third, the general inquiry about the AI experts, including their familiarity with the inquiry content and the self-assessment of judgment criteria.

4.

Implementing Inquiry

We conducted two rounds of interviews with the AI experts. In the first round of inquiry, we asked the 16 AI experts to rate the importance of the content of each item (assigning values from 1 to 5) and to make suggestions for changes to each item. Based on the feedback from the first round of experts and the results of statistical analysis, we revised and improved the instrument for the second round of inquiry. We continued the interview process until the experts’ opinions converged, at which point we stopped further interviews.

In light of the interaction with the LLMs, the implementation of the Delphi method of expert inquiry involved transforming the traditional instructions of the expert inquiry into a prompt that is suitable for the understanding of large-scale language models. This was performed with the application of the “ZhiPu AI” LLM as an illustrative example, and Figure 3 demonstrates the specific operation flow. In the initial phase, we engage the LLMs in the role of the AI experts by providing them with a comprehensive “Invitation Letter to AI Experts”. This document delineates the objective, content, process, and filing requirements of the inquiry in detail. For instance, the LLM candidates were furnished with explicit instructions. You are an expert in the field of learning sciences and possess a comprehensive understanding of the field of learning sciences. You are one of numerous LLM experts whom I have invited to participate in this endeavor. The objective is to simulate a real-world expert in the Delphi method and provide an in-depth analysis of each of the topics that will be presented. The second step is to reach an agreement with the LLM-based AI expert and send them a “Main Body of the questionnaire” This document requests that the AI expert rate the importance of each of the questions we provide and suggest changes. For instance, it would be advisable to provide LLM with an explicit instruction to the effect that it should assign an importance score (on a scale of 1–5, with the higher the score, the higher the level of importance) to each question item, present the results in a table, and present its insights and suggestions for improving the content of each question item in the last column of the table. In the third step, we requested that the AI expert, based on the large language model, complete a “The general inquiry about Al experts” to assess the familiarity of the content of the correspondence and the basis of judgment. For instance, the expert was asked to assess their familiarity with the topic of the current discussion and to select one of the following five levels of familiarity: “very familiar”, “more familiar”, “basically familiar”, “not too familiar”, or “not familiar”.

3.4.3. Evaluation Indicators

The reliability and validity of the Delphi expert survey results are mainly evaluated by the positive coefficient, concentration degree, authority degree, and coordination coefficient [55,56]. The positive coefficient reflects the degree of expert participation using the ratio of the number of experts participating in the survey to the total number of experts. In our study, which uses large language models (LLM) as survey experts, the positive coefficient is as high as 100%. The degree of concentration is represented by the mean (M) and the full score frequency (F). The mean indicates the average score given by the experts for a particular item, while the full score frequency refers to the ratio of the number of experts who gave full scores for a particular item to the total number of experts who participated in the scoring. A higher mean and full score frequency indicates a higher importance of the item [57]. The authority coefficient (Cr) reflects the degree of the experts’ understanding of the topic, which is represented by the arithmetic mean of the coefficient of familiarity (Cs) and the coefficient of judgment basis (Ca) given by the experts, with a value greater than 0.7 indicating a higher level of authority. The coefficient of coordination reflects the degree of consensus among experts, represented by the coefficient of variation (Cv) and Kendall’s coefficient of harmony W (Kendall’s W), with a smaller coefficient of variation and a larger coefficient of harmony indicating a higher degree of consensus among experts, where the corresponding p-value of W should be less than 0.05.

3.4.4. Item Selection Method

We used the “threshold method” [58] in the item selection process. The threshold method is a statistical analysis method for data screening and quality control by setting thresholds to identify observations in the data that do not meet specific requirements. In the item selection process of this study, the boundary value approach was used to identify and optimize assessment items that may not be appropriate or perform well using objective quantitative metrics.

Threshold screening criteria were calculated as follows: The formula for calculating cut-off criteria for both the frequency of full scores and the arithmetic mean is “threshold = mean − standard deviation”. If the frequency of perfect scores or the arithmetic mean of a question item is lower than their respective cut-off values, it means that the question item is not important enough and needs to be deleted or adjusted [59]. The formula for calculating the criterion for the threshold of the coefficient of variation is ”threshold = mean + standard deviation” [60]. If the coefficient of variation of a question item is greater than the boundary value of the coefficient of variation, it means that there is too much difference in the experts’ ratings of the importance of the question item. Such a question lacks reliability in scoring and needs to be removed or adjusted. In addition, to ensure that important question items are not deleted by mistake, the question items are only deleted if they do not meet all three threshold criteria at the same time. For the items that met only 1–2 cutoff criteria, human experts in the learning sciences were consulted to make modifications to maximize the retention of potentially valuable items.

3.4.5. Analysis of Results

• Analysis of the results of the first round of the Delphi method

In the first round, we invited 16 AI experts to participate, with a positive participation rate of 100.0%. In addition, using the Microsoft^® Excel^® 2021MSO (2405 Build 16.0.17628.20006), we calculated the judgment coefficient (Ca) as 0.8375, the familiarity coefficient (Cs) as 0.8875, and the expert authority coefficient as 0.8625. These values are significantly higher than the base value of 0.7, indicating that the AI experts in this field have sufficient authority. As for the coordination coefficient, through the DView software (A Python tool for calculating Delphi method results), we calculated Kendall’s harmony coefficient W as 0.437; X² as 251.515, with degrees of freedom (df) as 36; and the corresponding asymptotic significance (p) as 0.0, which achieved statistical significance.

The “threshold method” was employed to filter the items. Initially, the scoring results of the initial round of experts were entered into the DView software to calculate the mean, full score rate, and coefficient of variation for each item.

Table 4. Expert inquiry results.

Item	First Round			Second Round			Item	First Round			Second Round
	M	K	CV	M	K	CV		M	K	CV	M	K	CV
Q1	4.813	0.813	0.084	4.688	0.102	0.688	Q20	4.563	0.563	0.112	4.500	0.141	0.563
Q2	4.125	0.188	0.121	4.563	0.112	0.563	Q21	4.563	0.563	0.112	4.438	0.116	0.438
Q3	3.625	0.188	0.222	-	-	-	Q22	4.375	0.438	0.142	4.563	0.112	0.563
Q4	4.875	0.875	0.070	4.938	0.051	0.938	Q23	3.438	0.188	0.260	4.625	0.134	0.688
Q5	4.250	0.375	0.161	4.875	0.070	0.875	Q24	4.813	0.813	0.084	4.625	0.108	0.625
Q6	4.063	0.188	0.141	4.250	0.105	0.250	Q25	4.688	0.750	0.128	4.813	0.084	0.813
Q7	4.750	0.750	0.094	4.625	0.108	0.625	Q26	4.813	0.813	0.084	4.750	0.094	0.750
Q8	4.625	0.625	0.108	4.563	0.112	0.563	Q27	4.938	0.938	0.051	4.813	0.084	0.813
Q9	4.250	0.313	0.136	4.563	0.112	0.563	Q28	5.000	1.000	0.000	4.938	0.051	0.938
Q10	4.063	0.250	0.167	-	-	-	Q29	4.500	0.500	0.115	4.563	0.138	0.625
Q11	3.938	0.188	0.173	-	-	-	Q30	4.813	0.813	0.084	4.875	0.070	0.875
Q12	4.375	0.438	0.142	4.688	0.102	0.688	Q31	4.938	0.938	0.051	4.938	0.051	0.938
Q13	4.938	0.938	0.051	5.000	0.000	1.000	Q32	4.688	0.688	0.102	4.688	0.102	0.688
Q14	4.625	0.625	0.108	4.938	0.051	0.938	Q33	4.750	0.750	0.094	4.688	0.102	0.688
Q15	4.688	0.688	0.102	4.750	0.094	0.750	Q34	4.313	0.313	0.111	4.563	0.112	0.563
Q16	4.500	0.500	0.115	4.563	0.112	0.563	Q35	4.750	0.750	0.094	4.625	0.108	0.625
Q17	3.875	0.125	0.160	4.563	0.112	0.563	Q36	5.000	1.000	0.000	5.000	0.000	1.000
Q18	4.625	0.688	0.134	4.563	0.112	0.563	Q37	4.938	0.938	0.051	4.813	0.084	0.813
Q19	4.500	0.563	0.141	4.625	0.108	0.625

displays the specific values. Subsequently, the thresholds for each parameter in the initial round of inquiries were calculated according to the threshold calculation formula, as shown in

Table 5. Calculation results of thresholds.

	First Round Inquiry			Second Round Inquiry
	Mean	Standard Deviation	Threshold	Mean	Standard Deviation	Threshold
Arithmetic Mean	4.5236	0.3887	4.1349	4.693	0.1757	4.5173
Coefficient of Variation	0.1109	0.0518	0.1627	0.0928	0.0333	0.1261
Full Score Frequency	0.5963	0.2707	0.3256	0.6985	0.1723	0.5262

. According to the threshold filtering criteria, items Q3, Q10, and Q11 were deleted. Although item Q23 did not meet the threshold filtering criteria, it was deemed important to modify it based on the opinions of the AI experts, given the importance of teaching evaluation. The AI experts provided valuable suggestions, including the following: “Increasing the discussion on the diversity of assessment methods” and “Considering the bias in self-assessment results”. These suggestions highlighted the shortcomings of item Q23 in terms of diversity in the assessment methods. Therefore, we transformed item Q23 into a multiple-choice question, with the aim of comprehensively assessing the subjects’ understanding and application of diversified assessment strategies. The question was modified to the following: “If you were a secondary school teacher, which of the following statements would be correct when designing diversified assessment strategies for your students”? The options included: A. Diversified assessment should only include questionnaire surveys and objective tests. B. Peer assessment helps cultivate students’ critical thinking and cooperation skills. C. Student self-assessment is always objective and accurate, without further analysis and guidance needed. D. The purpose of diversified assessment is to avoid the bias caused by relying solely on a single assessment method.

For the items that partially met the threshold filtering criteria, including Q2, Q6, Q17, and Q34, modifications were made based on the suggestions from the AI experts. Regarding item Q2, the AI experts suggested adding specific time points or significant events. However, after consulting with human experts in the field of learning sciences to reduce the difficulty of assessment, it was decided to keep this item unchanged. With regard to item Q6, the AI experts recommended incorporating the latest developments in learning analytics to make it more cutting-edge. Consequently, we modified item Q6 to read: “I am familiar with learning analytics technologies, such as classroom discourse analysis, educational data mining, machine learning, etc.”. With regard to item Q17, the AI experts suggested refining the strategy description and simplifying the content of the stem, indicating possible ambiguity in the item’s wording. Consequently, we revised item Q17 to read: “When reading materials, which learning method is more helpful in improving learning efficiency: using a highlighter or marker to highlight possible key points, or separately recording knowledge in a notebook?”. For item Q34, the AI experts recommended considering the feasibility and integrating the item with actual teaching work. This suggests that items should align with practical teaching tasks and be designed from the perspective of the subjects to create realistic scenario-based assessments. Based on this, we modified item Q34 to: “I am able to provide assistance and guidance in the learning sciences to colleagues and students when necessary”.

2.

Analysis of the results of the second round of the Delphi method

In the second round of inquiry, we invited the same group of 16 AI experts to participate, and we received a similarly high positive response rate of 100.0%. The second-round expert scoring data were input into Excel, where they were used to calculate the judgment coefficient (Ca), the familiarity coefficient (Cs), and the expert authority coefficient. These values were found to be significantly higher than the baseline value of 0.7, indicating that the AI experts in the second round also possess sufficient authority. Regarding the coordination coefficient, we employed the DView software to calculate the Kendall harmony coefficient (W), which was determined to be 0.878, with X² as 104.247, degrees of freedom (df) as 33, and the corresponding asymptotic significance (p) as 0.0, thus reaching a statistically significant level.

In the second round, the “threshold method” for item selection was continued to be employed. Following the input of the second-round expert scoring results into the DView software, the mean, full score rate, and coefficient of variation for each item were recalculated. Table 4 presents the specific numerical values. Furthermore, the threshold values for each parameter were calculated in the second round, as shown in Table 5. In accordance with the screening criteria of the “threshold method”, it was determined that all the items met the statistical standards, and thus, there was no need to delete any items. Following two rounds of inquiry, the AI experts gradually converged on their suggestions for item modifications, resulting in a notable enhancement in the overall statistical outcomes. Consequently, we have concluded the inquiry at this juncture.

4. Discussion

As a comprehensive competence for doctoral students in education, the assessment of learning sciences competence has become an urgent direction for current research. In light of the paucity of assessment frameworks, the dearth of assessment instruments, and the questionable validity of assessment in current research on the assessment of learning sciences competence for doctoral students in education, we draw upon important theoretical frameworks such as the PISA 2025 science literacy assessment framework, the core literacy framework of compulsory education science in China, and Piaget’s constructivist epistemological views. Based on these theoretical foundations, we constructed a theoretical model and assessment framework for assessing learning sciences competence. It is important to emphasize that the theoretical model for the assessment of learning sciences competence that we have constructed is highly relevant and adaptable, specifically designed to target the diagnosis of learning sciences competence. The existing research has indicated that when educators are able to rely on a clear and measurable set of competence frameworks in their professional development process, a significant increase in their work effectiveness becomes possible [61]. Accordingly, the model we constructed not only establishes a more precise assessment framework, but also facilitates the development of learning sciences competencies for doctoral students in education. Although the model was initially applied to a Chinese cohort of doctoral students in education, the principles and core elements of the model’s construction are compatible with the core concepts of classic international competency assessment frameworks, such as the PISA assessment framework. This demonstrates the model’s broad international adaptability and potential for global replication.

In addition, based on an extensive literature review of teacher competence in learning sciences developed by international educational research organizations such as The Learning Agency and Deans for Impact, as well as classic books in the field of learning sciences, we developed an assessment instrument for doctoral students in education that combines Likert scales and scenario-based assessments. The objective of this instrument is to conduct a comprehensive investigation of doctoral students’ knowledge, application, and attitudes towards learning sciences. The instrument was examined in detail and a 34-item assessment instrument was finalized. Most of the question content of the instrument can be found in Table 2. The result analysis section of our first round of the Delphi method shows the detailed modification process and the results of the question items that need to be modified, such as Q2, Q6, Q17, Q23, and Q34.

Moreover, we achieved sustained methodological advancement by employing a large language model (LLM) to supplant traditional human experts in the Delphi method. A novel approach, the “LLM-based Delphi method”, was devised and implemented to assess the scientific validity and effectiveness of the assessment instrument. This method provides a framework for the examination of assessment tools, offering methodological guidance for future research.

Although this study has made some progress, there are still shortcomings. Firstly, in terms of item design, although the listed items in the instrument are necessary for evaluating competence in learning science, they may not cover all the aspects of the field due to space constraints. Therefore, there may be some biases in systematically reflecting the learning sciences competence of doctoral students in education, which need to be improved in subsequent research.

Secondly, with regard to the design of the assessment instrument, the intention is to reduce the difficulty of the assessment instrument by including only multiple-choice questions. Although some stem questions adhere to the principle of situational assessment, the overall approach is flawed in terms of assessing certain key competencies [62]. To comprehensively assess the learning sciences competence of the subjects, future research on assessment instruments could explore the introduction of more diversified question types, such as case analysis and essay questions, to more accurately reflect the subjects’ comprehensive qualities. A case analysis question might describe a classroom in which students are struggling to learn a particular concept despite the teacher’s use of a variety of instructional strategies. The subject is asked to analyze this situation using learning sciences theories to identify potential reasons for the students’ lack of comprehension and to make evidence-based recommendations for instructional adjustments. Essay questions may prompt subjects to discuss the implications of recent research findings in the learning sciences for instructional design, implementation, and evaluation.

Third, in the consideration of testing methods, despite the significant advantages of AI experts in data processing and complex calculations, they still cannot completely replace the unique value of human experts in intuitive judgment and practical experience. Therefore, the key to incorporating LLM into the Delphi method is in balancing the different perspectives of AI experts and human experts to ensure the accuracy and reliability of the results. It must be acknowledged that this study has incorporated the opinions of human experts in the deletion and revision of some topics (e.g., Q23). However, it must also be acknowledged that we have not yet strictly followed a set of rigorous procedures to test the consistency or discrepancy between the insights of human experts and AI experts. This inevitably weakens the diversity of the study results. In the future, we will endeavor to construct a more sophisticated fusion mechanism that will comprehensively integrate the key insights from AI experts and human experts. For instance, two parallel studies could be conducted: one based on the LLM-based Delphi method proposed in this study and the other using the traditional Delphi method. In this latter study, an equal number of human experts in the field of learning sciences would be invited to participate in an in-depth assessment of the learning sciences competence assessment instrument for doctoral students in education. By comparing and contrasting the two sets of Delphi results, we anticipate that the scientific validity and applicability of our findings will be further enhanced. For future researchers interested in adopting the LLM-based Delphi method, we strongly recommend the complementary strategy of comparing the assessment results of AI experts with those of human experts as a way to deepen the understanding of the study topic.

Moreover, future research should not limit the assessment of learning sciences competence to doctoral students in education [63]. It should be extended to various groups, including teacher educators, pre-service teachers, practicing teachers, and education researchers. These groups occupy distinct roles within the field of education. The extent of their learning sciences competence affects the quality of education and the sustained advancement of educational reform. To effectively promote the monitoring and sustainability improvement of learning sciences competence among educational practitioners, it is necessary to establish a regular assessment mechanism. This mechanism would permit the periodic assessment of the level of learning sciences competence among educational practitioners, thereby facilitating the precise designing of programs aimed at enhancing learning sciences competence and sustaining the progress of educational development.

5. Conclusions

The significance of learning sciences has never been greater. As the key drivers of educational practice and research, doctoral students in education play a crucial role. Their learning sciences competence directly impacts the sustainability development of the entire education system. Consequently, it is of the utmost importance to objectively and comprehensively assess their learning sciences competence using appropriate assessment instruments [64]. The doctoral student in education learning sciences competence assessment instrument that we have developed serves not only to evaluate the competence of doctoral students in education but also to provide sustainability guidance for the future training of doctoral students in education. In comparison to previous instruments and questionnaires, our developed instrument integrates Likert scales with scenario-based items, demonstrating greater conciseness and comprehensiveness. It has a broader impact, allowing researchers and institutions associated with it to utilize it. It is our hope that these results will serve as a reference for the regular monitoring and sustainability development of learning sciences competence among doctoral students in education.