Development of the Animal Conservation Digital Learning Aids and Assessments through the Industry-University Collaborative Course

Abstract

Due to the rapid changes caused by globalization and internationalization, this study focused on achieving Sustainable Development Goals (SDGs) 4 and 15 via specific digital learning materials—animal conservation apps—particularly designed for enlarging primary school pupils’ knowledge of biodiversity and conservation of natural habitats, and promoting sustainable development and lifelong learning abilities. Through a collaboration with Taipei Zoo, this study recruited 37 undergraduates who took the Learning Design and Practice course to develop the digital learning-assisted materials, namely animal conservation apps and assessment tools that suited the digital learning materials. In the initiative stage of the course, the undergraduates were required to work as a team and to learn in a group by observing and experiencing the model apps provided in the class. The provided apps were developed in compliance with the ADDIE model. In the middle stage of the course, each team was asked to develop their team app and assessment tools following the ADDIE model. In the final stage, each team’s design results were evaluated based on the digital learning material scale and core competency test evaluation: art domain. The results show that the undergraduates were able to integrate the expertise they gained in the course to developing high-quality digital learning materials. According to the educational professionals’ evaluation, the assessment tools designed by the undergraduates scored high marks.

1. Introduction

The 21st century is the era of the knowledge economy. Recently, Taiwanese society has been faced with demographic changes, a low birth rate, an aging population, multiple interactions of ethnic groups, rapid development of information, an emphasis on the sustainable development of the environment and ecology, and other changes. In order to respond to these changes and satisfy social demands, education in Taiwan appears to be required to handle new and challenging conundrums. Given the above social and educational context, the Ministry of Education of Taiwan (MOE) set up the following long-term goals: (1) The coherence and integrity of the school curriculum was expected to be persistently reinforced, (2) quality-oriented curriculum and teaching was required to be practiced, (3) talent development in accordance with personalities and lifelong learning needed to be implemented, and (4) promoting modern citizens’ lifelong learning abilities, social care, and international vision was equally as important as the impetus of nurturing future talents [1]. In order to achieve these goals, the learning materials adopted in the educational system are required to follow the guidelines for the new curriculum (Curriculum Guidelines of 12-Year Basic Education) published by the MOE (including core competency, aspects of learning, and competence indicators) and education quality.

The practicability and economy of learning materials need to conform to the knowledge trend of the modern age and contemporary learners’ requirements. The learning content is required to be based on diverse versions of textbooks and multiple learning resources, which in turn cultivate students’ abilities to pursue the common good by promoting students’ independent learning skills, various interpersonal interactions, and social participation. On the one hand, it is aimed at extending the learning of all domains and subjects and incorporating project-based, cross-curricular integrative, practical, and experiential courses involved in the 19 issue-centered curriculum (two of these issues related to this study are environmental and outdoor education) in the Curriculum Guidelines of 12-Year Basic Education. On the other hand, the main purpose of Taipei Zoo is to convey conservation knowledge, promote environmental awareness, and be one of the institutions for social education, a research center, and an amusement attraction. Therefore, it may be helpful to integrate the knowledge concerning the environmental and outdoor issues (university), as well as the on-site experience (Taipei Zoo) to cultivate undergraduates’ practical ability and experience in environmental and outdoor education. The support provided by Taipei Zoo is appreciated.

The educational goals of the Department of Learning and Materials Design (LMD) is to cultivate undergraduates’ expertise and skills in developing teaching and learning materials that suit all levels of education in schools and in society [2]. The first-year courses are fundamental and center on Curriculum, Instruction and Teaching; Aesthetic Design; and Computer Sciences. The second-year courses comprise different applied courses involving technology and design application and design approaches for digital learning materials, including Learning Design for Social Studies, Learning Design for Environmental Education, Learning Design for Life and Character Education, Learning Design for Mandarin Phonics, Urban Education and Learning, Game Design and Application, and Layout Design. The teaching objectives of the LMD are to deepen and broaden the content and knowledge of the learning materials; to strengthen professional skills in planning, designing, and producing digital learning materials; and to enable undergraduates to evaluate their design outcomes, as shown in Figure 1 and Figure 2.

Given the above context and fulfilment of SDGs 4 and 15 in students’ learning processes, this study aimed to apply the Learning Design and Practice course to instruct undergraduates in absorbing content knowledge (CK) on environmental protection, animal conservation, and design approaches for digital learning materials for primary school pupils. The main topics of the Learning Design and Practice course comprised environmental and animal ethics, ecological education, curriculum design, and development models. Moreover, the Learning Design and Practice course was expected to develop undergraduates’ design skills (DS) in planning and designing digital learning materials, such as aesthetics, human–machine interface, design tools, and layout design. Undergraduates within this study were considered to be the competent users of CK and DS when qualified to integrate the acquired expertise into managing and producing digital learning materials, and developing assessment tools for learning effectiveness of the digital learning materials. Classroom observations were used in the teaching and learning processes in order to confirm whether the students had the capacity to integrate CK and DS into developing digital learning materials and assessment tools. According to the classroom observations, it was noticed that teaching strategies and learning styles could affect the students’ skills in knowledge transfer and application. It seems to be essential to keep observation records specifically when the students encounter obstacles in the planning and design process. This is because the observation records allow the teacher to instruct the students to reflect upon their learning strategies and the way in which they apply knowledge to solve the problems, which in turn enables the learning content to move forward towards upgrading students’ skills in planning, designing, and evaluating digital learning materials.

The Learning Design and Practice course within this study was the joint force of the industry (Taipei Zoo) and the academia concerning the teaching content, environmental protection, and wildlife conservation education. Apart from this, the course placed particular stress on classroom teaching and practice, curriculum planning, material design, and the integration of CK and DS into developing digital game-based learning materials for primary school pupils. Importantly, the learning content within the digital game-based learning materials was required to arouse users’ desire for exploration and inspection, which could make users feel more interested in understanding animals and exploring the environment [3]. This in turn may expand the users’ knowledge base of environmental and ecological education.

The process of delivering the Learning Design and Practice course embraced the introduction to the theories of design strategies for learning materials, instruction of designing digital learning materials and inspirations, and development of promoting students’ thinking skills and problem-solving competencies. The process could be helpful for undergraduates to absorb knowledge and skills and strengthen their ability to recall and apply the learned expertise [4,5]. Moreover, it is widely acknowledged that the specific instructional system design model, the Analysis, Design, Development, Implementation, and Evaluation model (ADDIE), has functioned as effective guidelines for developing digital game-based learning materials [6], as is evidenced by studies of multimedia-based and digital learning materials [7,8,9]. The undergraduates within the present study were asked to follow ADDIE to develop the particular digital learning materials, namely, animal conservation apps, and design an appropriate assessment suitable for evaluating the apps.

The game-based learning content within the apps included fundamental knowledge of animal conservation and environmental education, and this functioned as an impetus for the primary school pupils (users of the apps) to visit Taipei Zoo. In order to complete the game-based tasks within the apps, the pupils needed to participate in and concentrate on the learning activities regarding animal conservation and environmental education held in the zoo. This in turn allowed the pupils to expand their knowledge of biodiversity, rather than simply focusing on the particular species. The discussions among the undergraduates (the app designers), the lecturer, and the experts in each stage within this study aimed to guide the undergraduates to integrate the acquired knowledge regarding environmental education into the design of digital learning materials for primary school pupils.

Research Purpose

The main purpose of this study was to explore undergraduates’ capacity to develop digital learning materials and assessment tools compatible with the developed learning materials. For the research purpose, the research participants were 37 undergraduates who took the Learning Design and Practice course. Undergraduates attending this course were expected to gain professional skills and expertise in designing and evaluating digital learning materials. Importantly, the course also emphasized undergraduates’ digital skills concerning the application of CK and DS and their competencies for knowledge transfer and transformation. The key issues discussed in this study are as follows: (1) How the undergraduates developed digital learning materials suitable for primary school pupils, and (2) how the undergraduates designed the assessment tools to evaluate the digital learning materials they developed.

2. Literature Review

2.1. Taking Digital Technology as a Trigger for Learning

Digital technologies are auxiliary tools and materials that cannot be ignored for the future citizens of the world. The features of digital learning materials embrace the characteristics of multimedia, individualized teaching, and interactive learning. These features allow learners to learn at their own pace based on their needs, abilities, and preferences, and gain opportunities for self-assessment and immediate feedback during the learning process. Digital learning materials have been widely considered to be an effective impetus for inspiring learners to participate in learning processes [10,11]. Moreover, diverse resources provided within digital learning materials could deepen and expand learners’ knowledge base. More importantly, compared with textbooks, digital learning materials appeared to make the learning content more interesting, attractive, and easier to understand by presenting the content in different forms of information synchronically, e.g., animation shown with narration and images demonstrated along with the key words and signals. In many studies of multimedia learning, using either images together with narration or animation along with narration to show the learning content has been proven to be an effective learning approach [12,13].

Conditioning theory could be applied to designing game-based learning materials [14]. This is because learning with games allows learners to earn rewards, including scores, points, and being upgraded to the next level immediately when completing a task. The process of rewarding could enhance learns’ learning motivation and positive learning behavior. For instance, learners are rewarded or shown inspiring pictures when they give a correct answer. Learners tend to notice and remember important information when gaining the related stimulus responses in the learning process. Notably, however, the tasks and learning content within game-based learning materials need to have different levels of challenges, associated with different enhancement designs. Apart from this, it seems to be essential that learning content within the game-based learning materials be divided into small/short parts, together with prompt and personalized feedback. Conditioning theory asserts that giving feedback appropriately throughout the learning process could attract learners’ attention, and that rewards in the learning process could produce positive reinforcement that encourages learners to keep learning.

Digital learning materials could provide effective graphical information through dynamic images or animation, and this in turn generally assists learners in interpreting and understanding the given information more easily and effectively. Instead of acting as passive acceptors during traditional teaching processes, learners usually become highly active in participating in digital learning processes [6]. This is mainly because digital learning materials could monitor, diagnose, and analyze learning processes automatically whenever necessary. It was found that students who got involved in creating learning materials and assessment tools were more attentive in class and paid more attention to the assessment. This had an overall positive effect on learning outcomes [15]. The related studies went further, finding that the introduction and/or extension of the use of digital learning materials in school settings tended to enhance students’ satisfaction and interests in learning processes [16,17].

2.2. Developing Learning Aids through the ADDIE Model

The Analysis, Design, Development, Implementation, and Evaluation model (ADDIE), as shown in Figure 3, consists of five stages, as follows [6,9,18]: (1) the analysis stage—the requirements of the teaching and learning system should be planned in detail in order to identify the problem, the needs, and the learner’s prior knowledge [19,20]; (2) the design stage—the details of each step involving the design of learning materials should be specified, including learning objectives, design concepts, materials, content, layout, and graphic design, etc. [21]; (3) the development stage—all designs of the learning materials are actually combined according to the steps and details listed in the design stage, including the components, manuals, rules, games, knowledge checkpoints, etc., which should be examined by experts and revised appropriately; (4) the implementation stage—learners are asked to examine learning materials and provide their feedback on the materials (the practice stage can also be regarded as a formative evaluation of learning materials); and (5) the evaluation stage—the summative evaluation of learning materials. The consistency of the content and the objectives, the integrity and applicability of the content, and learners’ satisfaction should be taken into consideration [22].

The significance and benefits of applying ADDIE to managing digital learning materials have been widely acknowledged, as is evidenced by research into e-learning, e-books for teaching and learning purposes, and digital game-based learning [6,7,9,23,24]. In their studies, Chen et al. confirmed that the particular educational e-book designed according to ADDIE successfully enhanced primary school pupils’ learning interests and helped them to learn about Universiade [23]. Other studies went further, asserting that primary school pupils’ “playfulness” and “immersive experience” were found to be improved when learning with the digital learning materials designed particularly based on ADDIE [24]. It can be argued that ADDIE can function as a conceptual framework and effective approach for developing digital game-based learning materials. The present study was concerned with instructing undergraduates in designing digital game-based learning materials, and thus the main ideas of ADDIE in particular were used in teaching processes within this study. Hence, the teaching process of this study started with the analysis of the learners’ needs and learning content. Next, the development of diverse learning components (including the structure and content of the materials) was conducted. Then, trials of the developed digital learning materials, formative evaluations, and revisions were implemented. Finally, experts in the field of education and multimedia design were invited to conduct the summative evaluations of the digital learning materials developed by the undergraduates.

2.3. Designing Scales for Evaluating Learning Effectiveness

The Curriculum Guidelines of 12-Year Basic Education were officially promulgated by the Ministry of Education (MOE) of Taiwan in 2014, and implemented in primary schools (years 1–6), junior high schools (years 7–9), and upper secondary school (years 10–12) nationwide in 2019. Therefore, the Curriculum Guidelines of 12-Year Basic Education were also named the 2019 Curriculum Guidelines in Taiwan, and the coherence and integration of curricula from year 1 to year 12 have been intensively highlighted in Taiwan’s educational system since 2019. Taking initiative, engaging in interaction, and seeking the common good are three fundamental concepts of the 2019 Curriculum Guidelines. In accordance with these three fundamental concepts, schools are expected to assist students in applying their learned knowledge; experiencing the meaning of life; fostering interests in being engaged in the sustainable development of society, nature and culture; and facilitating the attainment of reciprocity and the common good [1]. According to the 2019 Curriculum Guidelines, primary school education embraces eight learning domains, as follows: language arts, mathematics, social studies, natural sciences, arts, integrative activities, health and physical education, and technology. In order to fulfil the goals of the 2019 Curriculum Guidelines, core competencies serve as the basis of curriculum development to ensure coherence and continuity between educational stages, bridging domains and integration between subjects [1]. The concept of core competencies stresses that learning materials should focus on real-life scenarios and holistic development through action and self-development, rather than confining students to the knowledge and abilities taught in the school context. On this basis, schools are required to equip students with competencies, including abilities and attitudes, for daily life and to face future challenges. Given the educational context concerning the 2019 Curriculum Guidelines—as well as considering the digital learning materials (animal conservation apps) within this study specifically designed for primary school pupils—the Learning Design and Practice course in this study not only equipped the undergraduates (the app designers) with aesthetic competencies and design skills, but also instructed them in the expertise of animal conservation and environmental education by the joint force with Taipei Zoo.

Based on the 2019 Curriculum Guidelines, the arts domain in primary school education involves three key competencies, as follows: (1) expression—competencies in media use, artistic creation and displays, and expressions of ideas and emotions; (2) appreciation—competencies in perceiving beauty and understanding values in the arts through participation in aesthetic activities; and (3) practice—competencies in fostering interest in the arts to promote quality of life [25]. More importantly, the 2019 Curriculum Guidelines stress that learning content connected with real-life scenarios is helpful for knowledge transfer and transformation. Students’ learning outcomes and learning processes are equally important and need to be assessed within a context strongly related to students’ daily lives. According to the 2019 Curriculum Guidelines, it is essential that courses and learning materials benefit students in strengthening their abilities to integrate and apply their learned knowledge in real-life scenarios. Owing to the implementation of the 2019 Curriculum Guidelines, the learning content, teaching approaches, learning processes, and learning assessments have been in a challenging transition since 2019. In accordance with the 2019 Curriculum Guidelines, multiple evaluation methods, self- and peer assessments, and student profile assessments, for example, employed in teaching and learning processes are generally perceived to be useful for increasing students’ core competencies in the long run.

With the implementation of the 2019 Curriculum Guidelines, the MOE issued the Standard-Based Assessment of Student Achievement for Primary and Junior High School Students, and formulated the assessment standards for each learning domain, including learning topics and “performance descriptions” [25]. Different from the test-based assessments simply based on exams and scores, the performance descriptions in each learning domain present the state and condition of students’ learning, coupled with five levels of expected and target learning performances. The five stages of each performance description consist of “excellent, good, basic, below basic, and insufficient.” Hence, performance descriptions could show precisely not only whether students achieve the performance demanded or expected, but also which learning stages students are situated in during the process of learning. This in turn may allow the teacher to offer students prompt and suitable assistance that caters to their needs. Since the present study was to instruct undergraduates in designing digital learning materials for primary school pupils, core competencies and performance descriptions highlighted in the 2019 Curriculum Guidelines served as the essential guidelines for developing digital learning materials (animal conservation apps) and the assessment tools in the Learning Design and Practice course within this study.

3. Methodology

3.1. Research Design

The Learning Design and Practice course within this study was implemented and delivered for one semester to cultivate undergraduates to absorb CK and DS regarding environmental protection, animal conservation, and the design of digital learning materials. The undergraduates were guided to develop digital learning materials for primary school pupils and design assessment tools suitable for evaluating the developed learning materials. The framework of this course is shown in Figure 4.

The Learning Design and Practice course encompassed three stages. Stage one was the 10-week lessons. Lessons for the first 4 weeks were seminars and design practice that functioned as the main approaches to equipping the undergraduates with the competencies of CK and DS of designing digital learning materials. During the 4 weeks, the core learning issues and content included the ADDIE model, game-based learning, aesthetics, the design of digital learning materials, and apps—in particular, layout design and the related design technologies. Lessons for the following 3 weeks included lectures coupled with field visits to Taipei Zoo, providing the undergraduates with knowledge regarding animal conservation, environmental education, and related theories. The lessons for the following 2 weeks included the zookeepers guiding the undergraduates to understand the target animals’ morphology, habits, and habitats, and offering practical tips for observing animals. The remaining week involved the lectures centering on the theory of designing and editing assessment tools.

Stage two included the 10-week lessons during which the undergraduates were divided into 5 teams and were required to learn and work as a team. During the lessons for the first week, each team had to review the apps that they developed in Stage one. Furthermore, the undergraduates were asked to consider and develop the evaluation framework and items for the assessment instruments that suited the developed apps. The lessons for the second week involved examining the attributes and functions of the apps, including the learning content, layout design, task-based games for learning purposes, and knowledge checkpoints. During the lessons for the following 3 weeks, the undergraduates were taught to integrate the acquired expertise and technology, coupled with the knowledge of the target animals, into the development of the apps and the assessment tools according to the ADDIE model. In the lessons for the following 3 weeks, the lecturer discussed the developed apps and assessment tools with 2 or 3 teams through the focus group approach (3 h each time, 2 times per week). During the discussions, the lecturer raised open-ended and extended questions to engage the undergraduates in deep, reflective, and critical thinking involving their design. Then, the undergraduates modified their design of the apps and the evaluation items of the assessment tools, and recorded the whole process of learning, group discussions, and design. During the lessons for the remaining 2 weeks, each group presented their app and assessment tools to the class, and gained feedback from classmates and the lecturer. Then, the undergraduates conducted a usability testing that focused on the developed apps in year 3 and year 4 in a primary school by interviewing the pupils. The apps were revised and improved according to the feedback and suggestions collected from the pupils.

In Stage three, 8 experts, including 2 professors in the field of curriculum and teaching, 2 zookeepers with approximately 5 years of working experience in Taipei Zoo, and 4 primary school teachers with over 10 years of teaching experience, were invited to evaluate the apps that the undergraduates developed in Stage two. The hermeneutic method and kappa analysis [26] were used to establish the reliability of the expert evaluation. Moreover, the experts within this study were provided with the prototypes of the apps developed by the undergraduates in each learning stage of the Learning Design and Practice course before the expert evaluation was undertaken. This offered the experts the information to understand whether the undergraduates produced their final version of the apps through group creation. The implementation steps were as follows: A checklist was developed based on the aim and objectives of this study. The 8 experts were evenly divided into 2 groups. Experts in the same group gave scores together. The two groups gave scores separately at the beginning of the evaluation process, and then reached a general consensus on the scores through deliberation. Kappa analysis was conducted after the expert evaluation. Finally, the undergraduates revised and completed the final version of their apps based on the experts’ evaluation and suggestions. Each group of undergraduates was asked to record and submit an academic portfolio, group discussion reports, work distribution chart, and peer assessments in order to ascertain and comprehend each undergraduate’s efforts, learning outcomes, and contribution to the group. As shown in

Table 1. The arrangement of the Learning Design and Practice course.

Stage and Duration	Course	Learning Content	Learning Outcomes
Stage one: 4 weeks	1. CK and DS related to learning material design	1. Learning knowledge and skills about the ADDIE model, game-based learning, aesthetics, learning material design, app design, layout design, and related design technologies, etc.	1. Creating an academic portfolio to record the learning process
Stage one: 3 weeks	2. CK related to animal conservation and environmental education	2. Learning knowledge about animal conservation, environmental education issues, and related theories	2. Creating an academic portfolio to record the learning process
Stage one: 2 weeks	3. A field visit to Taipei Zoo	3. The morphology, habits, and habitats of the target animals and how to observe the animals	3. Creating a worksheet to record the learning process
Stage one: 1 week	4. The knowledge and techniques of developing scales	4. Introducing how to design an appropriate and sufficient scale	4. Creating an academic portfolio to record the learning process
Stage two: 1 week	5. Reviewing and using apps developed in the past	5. Experiencing the apps and discussing the advantages and disadvantages of the design through the focus group	5. Creating an academic portfolio to record the learning process
Stage two: 1 week	6. The content and element involved in the learning materials	6. Discussing the design of the app, learning content, layout, gaming, and knowledge checkpoints through the focus group	6. Discussing and recording the requirements of the design of the materials and scales
Stage two: 3 weeks	7. Developing learning materials and designing scales in line with the materials	7. Applying focus group approaches to solve problems during development	7. Developing the apps with the ADDIE model and designing the scales
Stage two: 2 weeks	8. Presenting the developed apps and scales	8. The teacher and peers provide feedback and opinions after using the apps and scales.	8. Revising the apps based on the teacher’s and peers’ feedback and opinions
Stage two: 1 week	9. Conducting a trial test in a primary school	9. Primary school pupils provide their opinions after using the apps.	9. Revising the apps based on the pupils’ opinions
Stage three	10. Evaluating the apps and scales developed by the undergraduates	10. Inviting experts, scholars, zookeepers, and senior primary school teachers to evaluate the apps and scales	10. Revising the apps and scales again and submitting final version of the apps and scales 11. Submitting the focus group discussion, work distribution chart, and peer assessments

, the analysis of the courses and the learning content corresponded to the undergraduates’ learning outcomes.

3.2. Participants

Purposive sampling was adopted in this study. The participants were 37 second-year undergraduates studying at the Department of Learning and Materials Design (LMD) and taking the Learning Design and Practice course when the study was underway. The participants were divided into 5 groups, and each group was asked to develop an app and a scale to assess the developed app. Before this study was undertaken, all research participants completed the fundamental courses at LMD, including Curriculum and Teaching, and Aesthetic Design, Introduction to Education, Learning Psychology, Teaching Materials and Game-based Learning, and Design and Practice: Level One. Learning Design and Practice is a compulsory course for all second-year undergraduates studying at LMD. The teaching aims of the Learning Design and Practice course is to equip undergraduates with the expertise to integrate the skills of graphic design, digital technologies, and knowledge of teaching and learning to develop learning materials for learners.

3.3. Materials

3.3.1. Core Competency Test: Art Domain (CCTa)

In this study, the CCTa of extended design from previous research results [18] was used as an example to instruct undergraduates in developing their scales. Taking the aspects of learning and competence indicators of the core competency involved in the Curriculum Guidelines of 12-Year Basic Education: Art Domain as the main axis, the scale was compiled according to the development of student performance and learning content, and the knowledge of animal rights/conservation and environmental education provided by Taipei Zoo. It contains expression (3 short-answer-type questions), appreciation (3 short-answer-type questions), and practice (2 short-answer-type questions). The main purpose was to identify whether primary school pupils could develop their abilities of perceptual awareness, aesthetic thinking, and creativity performance after they went through the learning materials (apps in this study). The test was open-ended and graded according to the accuracy of the answers and the degree of detail described. Those who were correct and provided plenty of detail were awarded 3 points, those who were correct but provided insufficient detail were awarded 2 points, those who were partially correct and provided insufficient detail were awarded 1 point, and those who did not answer were awarded 0 points.

3.3.2. Digital Learning Material Scale (DLMS)

This study used previous research results [23] and the ADDIE model to design DLMS. The purpose of this study was to identify whether the learning materials designed by undergraduates conformed to CK and DS learned from the Learning Design and Practice course. The DLMS is divided into four items: material content, instruction, material design, and media application. First, issues regarding accuracy of the content, descriptions of the content, use of learning materials, and design of the architecture and examples in accordance with pupil’s requirements are evaluated in “material content.” Issues concerning operation of the interface, recognition of the instructions, design of the orientation, use of the functions, readability of the texts and paragraphs, legibility of the color, and layout design are in “instruction.” Issues about construction of learning objectives, connections to learners’ prior knowledge, arrangement and development of different materials, design, and availability of the formative evaluation in the apps are measured in “material design”. Finally, issues about the use of images, graphs, photos, drawings, icons, music, soundtracks, videos, animations, dubbing to develop the apps, and the degree of usefulness for learners’ learning are considered in “media application.” There are six questions under each of the four items. Each question has five options, with scores of 5, 4, 3, 2, and 1 from high to low, respectively.

3.3.3. Core Competency Test Evaluation: Art Domain (CCTEa)

Using the CCTEa of the extended design of previous research results [23] and in line with the development of student performance and learning content, this study was compiled according to animal conservation information provided by Taipei Zoo. The purpose was to evaluate whether the assessment tools of the extended design from the designers’ own learning materials conformed to art learning content and learning performance, as well as the content of the learning materials. The content of the scale is divided into three items: expression (3 questions), appreciation (3 questions), and practice (2 questions). Each question has a total of five options, with scores of 5, 4, 3, 2, and 1 from high to low, respectively. Expert validity was established for this scale, and rater reliability was established by the precise interpretation method and kappa analysis.

3.3.4. Level of the Assessment Tool

In this study, the quality of the digital learning materials, the animal conservation apps, was examined and assessed according to the 3-level checklist, which included A+, A, and B, and this was based on the ratings of the evaluation of the digital learning materials issued by the MOE [25]. Level A+ indicated that the quality of the digital learning materials was found to be outstanding, and scored 104 points (inclusive) or above. Level A meant that the quality of the digital learning materials was found to be good, and scored 88 points (inclusive) or above. Level B meant that the quality of the digital learning materials was satisfactory, and scored above 72 points (inclusive).

4. Research Findings

4.1. Undergraduates Are Able to Develop Apps According to the ADDIE Model

The content of the apps designed by five groups of undergraduates needed to include the morphological characteristics, habitats, habits, distribution range, and conservation status of the target animals, which were divided into the following groups, in order. Group 1 designed the capybara app The Fun of Capybara Going to School, which is available on Google Play (https://play.google.com/store/apps/details?id=com.lmdfcapybara.one (accessed on 18 June 2021)). Group 2 designed the anteater app The Feeding State of the Anteater, which is available on Google Play (https://play.google.com/store/apps/details?id=com.lmdfanteater (accessed on 18 June 2021)). Group 3 designed the poison frog app Frog! Do you know?, which is available for download on Google Play (https://play.google.com/store/apps/details?id=com.lmdf.dendrobatidae (accessed on 18 June 2021)) Group 4 designed the tropical rainforest animal canteen app—Food Is Passion, which is available for download from Google Play (https://play.google.com/store/apps/details?id=com.lmdfrainforesteat (accessed on 18 June 2021)). Group 5 designed the tropical rainforest fauna distribution app Tenants of the Tropical Rainforest, which is available on Google Play (https://play.google.com/store/apps/details?id=com.lmdfrainforestlive (accessed on 18 June 2021)). Based on the focus group approach, this study was expected to cultivate undergraduates to plan and develop digital learning materials (the animal conservation apps).

4.1.1. Analysis

In order to use the animal conservation apps in Taipei Zoo, the first step was to analyze the requirements of primary school pupils, for example, by analyzing pupils’ background knowledge and their usage of mobile devices and other apps. After focus group discussion, the undergraduates reached the following consensus:

• Difficulties: Four groups of undergraduates mentioned that pupils only knew shapes and habitats of animals, but did not know the detailed characteristics and habits of animals; after searching for information online and in textbooks, two groups of undergraduates stated that pupils did not know or had no idea about the ecology of the animals in the tropical rainforest and the crisis they are facing today (FG1080415, 1080416).
• Prior knowledge: Three groups of undergraduates believed that pupils’ prior knowledge of animals came from the knowledge written in their textbooks, such as animal lifestyle, animals on land and in water, animal characteristics and classification, etc. Two groups of undergraduates thought that children’s knowledge of animals came from the experience of visiting zoos (FG1080415, 1080416).
• Digital learning material design: Five groups of undergraduates all mentioned that animals and habitats could be designed into the apps to deepen pupils’ understanding of animals; three groups of undergraduates thought it was important to learn about the living habits of animals. They could use picture books or stories to help pupils learn the hunting methods of animals and the type of food that animals could eat (FG1080415, 1080416), with the records of Group 2 in the analysis stage as an example in

  Table 2. The analysis stage of developing the app, using Group 2 as an example.

  Group No.	2
  Name of App	Anteater appߞThe Feeding State of the Anteater
  Problems and Difficulties	Most pupils only knew about anteaters, but had learned little about the differences between mature and baby anteaters. Some pupils generally took pangolins and armadillos to be the same animals.
  Prior Knowledge	Most pupils only knew about the tropical rainforest and some rainforest species. Some pupils had learned little about the living environment of tropical rainforests. In addition, most pupils had learned little about the differences between animals in terms of their shapes and lifestyles, as well as the ecology of tropical rainforests.
  Learning Material Design	Primary school textbooks only introduced fundamental information of ecological environment and species. It was expected that pupils learn more about animals’ habits and habitats when learning with the app (with the comparison of similar creatures living in the same locality). Moreover, the game-based app could also increase pupils’ learning interests.

  .

4.1.2. Design

The teacher and each group of undergraduates discussed the design concept and process of the digital learning materials (the animal conservation apps). After the focus group discussion, the following consensus was reached:

• Four groups of undergraduates mentioned that the learning materials designed should match the content of the units of “Understanding Animals,” “Animal Body and Movement,” or “Animal Assembly” in primary school textbooks and should be extended, deepened, and broadened to include the related knowledge. Group 1 and Group 3 chose to introduce one specie in their apps, and Group 4 and Group 5 selected habitat as the main structure considered, with animals living in the same region (FG1080422, 1080423).
• All five groups of undergraduates thought that instead of using only the knowledge provided by Taipei Zoo, other stories and games related to these species should be added to the apps to increase learners’ interest in learning (FG1080422, 1080423).
• Two groups of undergraduates used photos and videos provided by Taipei Zoo, together with the photos they took and the images they drew to design the apps. Three groups of undergraduates decided to draw the morphological characteristics and habitats of the animals by themselves (FG1080422, 1080423).
• The five groups of undergraduates reached consensus on the design process of the apps. The systematic process was as follows: having a main animal character in the learning material → introduction of the operation methods → animal-related learning content → small games and knowledge checkpoints (FG1080422, 1080423), with the records of Group 4 in the design stage as an example in

  Table 3. The design stage of developing the app, using Group 4 as an example.

  Group No.	4
  Type of Game	Matching games
  Objective	Understanding tropical rainforest animals’ lifestyles, shapes, and main source of food
  Idea	Using matching games to deepen learners’ impressions of the connection of animals and foods Introducing animal knowledge and their main source of food in detail in the animal encyclopedia designed within the app
  Teaching Content	Introducing scarlet ibises’ lifestyle and main source of food Introducing laughing kookaburras’ lifestyle and main source of food Introducing crab-eating mongooses’ lifestyle and main source of food Introducing Tomistomas’ lifestyle and main source of food Introducing Linnaeus’s two-toed sloths’ lifestyle and main source of food
  Design	Asking learners to draw animals, foods, and tropical rainforest images on their own Asking learners to compare the information provided by Taipei Zoo and collected via Web forms with the pictures they drew in step 1.
  System Flow	Game introduction → matching game 1 (animals and shapes) → matching game 2 (animals & foods) → animal encyclopedia

  .

4.1.3. Development

Using the digital learning materials developed in the past as an example, the teacher discussed with each group of undergraduates how to design the fundamental structure of learning content, layout, games, and knowledge checkpoints. Through focus group discussion, each group of undergraduates reached the following consensus:

• Five groups of undergraduates designed matching games for pupils to acquire knowledge directly through matching options. Four groups of undergraduates drew a cute representation of the animal to highlight the features and deepen the learners’ impression (Figure 5). Two groups of undergraduates designed guessing games to strengthen the image and details of the animals (FG1080429, 1080430).
• Group 2 applied a storyline with the learning content of the animal’s morphological characteristics, habitats, habits, distribution range, and protection status to design a picture books in the app, which was different from the previous design and helped pupils compare the real-life appearance of different animals (FG1080429, 1080430).
• The undergraduates of Group 1 and Group 3 chose a single species as the main role to develop the animal conservation apps. In addition, learning contents such as morphological characteristics, habitats, habits, distribution, and conservation status were divided into separate chapters for pupils to learn according to their interests. The undergraduates in Groups 1, 3, 4, and 5 imitated previous designs and showed the morphological characteristics, habitats, habits, distribution range, protection status, and other learning contents of the animals separately in a staggered and mixed way with games and learning content, which was helpful for understanding the lifestyles of various animals (FG1080429, 1080430), with the records of Group 5 in the development stage as an example in

  Table 4. The development stage of developing the app, using Group 5 as an example.

  Group No.	5
  Name of Game	Tenants of the Tropical Rainforest
  Planning	Introducing the six types of habitats of tropical rainforests Allowing pupils to learn about animals and habitats through games Introducing animal characteristics and related knowledge in detail
  Way to Proceed	Step 1: Learners need to choose the habitat they are willing to learn about first. Step 2: Using games to help learners recognize the key animals in the particular habitat. Step 3: Learners are required to choose the animals they are willing to learn more about.

  .

From the apps developed by each group of undergraduates, it was recognized that they were able to integrate, transform, and apply the related CK and DS learned in the course, for instance, (1) in the voice matching game from Group 1, “‘tap the brown spider monkey” played the sound of a brown spider monkey, “tap the crocodile” played the sound of a crocodile, “tap the capybara” played the sound of a capybara, and “tap the parrot” played the sound of a parrot; (2) in the narration of life habits through storytelling from Group 2, the life habits and food of southern tarantulas, giant anteaters, and pangolins were described by picture book stories and anthropomorphic methods; and (3) in the habitat-matching game from Group 5, learners had to choose the outlier among the following groups: toco toucan, macaws, great hornbill; forest floor—Linnaeus’s two-toed sloth, pygmy slow loris, Bornean orangutan, laughing kookaburra; and ground layer—giant anteater, southern tarantula, Formosan Reeve’s muntjac, Malayan tapir (FG1080429, 1080430).

4.1.4. Implementation

The implementation stage was divided into two parts, examination and feedback. The five groups of undergraduates presented the design of their apps and scales on stage, and the teachers and their peers provided opinions and feedback to the presenters. Then, the five groups of undergraduates helped a class of primary school pupils use the apps. In addition, the pupils offered their feedback to the undergraduates. The five groups of undergraduates revised the apps and scales according to the opinions of the teachers, peers, and primary school pupils, with the records of Groups 2 and 3 in the implementation stage as an example in

Table 5. The implementation stage of developing the apps, using Groups 2 and 3 as an example.

Group No.	2	3
Assessment and Feedback	It is recommended to dub the picture book to promote student learning. It is recommended to present the answers and tips at the end of the game for pupils to acquire the related knowledge. It is recommended to add a Back Home button on each page for pupils to feel free to choose pages and the particular learning content.	It is recommended to use more vivid dubbing to promote learning interests. It is recommended to show knowledge tips or learning materials when pupils fail to complete the tasks within the game. Insufficient aesthetics. It is recommended to readjust the color plan. It is recommended to readjust the location of the map. The locations of Central America and South America was wrong.
Modification	The picture book was dubbed in the final version. Tips with knowledge of animals and habitats were added to the game. A Back Home button was added to each page in the app.	The app was redubbed. Teaching materials were added to the game. The color plan was checked and reorganized. Central America and South America were placed in the right locations.

.

4.1.5. Evaluation

The DLMS was used to evaluate the apps developed by the five groups of undergraduates. The kappa values of the evaluation results of two groups of experts were greater than 0.70, showing a high degree of consistency. The evaluation results are summarized in

Table 6. Evaluating the apps developed by five groups of undergraduates through DLMS.

Group No.	1	2	3	4	5
Material Content *	28.0	27.0	23.0	23.0	27.0
Instruction *	27.0	23.0	23.5	22.5	24.5
Material Design *	28.0	28.0	22.0	24.0	27.0
Media Application *	27.0	27.0	22.0	23.5	27.0
Total	110.0	105.0	90.5	93.0	105.5
Certification	A+	A+	A	A	A+
Kappa Values	0.82	0.84	0.78	0.80	0.82

* Full points: 30.

. Results: Groups 1, 2, and 5 achieved more than 105 points and obtained an A+ certification, reaching the excellent level; Groups 3 and 4 achieved 90 or more points with an A-level certification and achieving a good grade.

4.2. Analysis of the Development of Scale of Each Group of Undergraduates

The CCTa was provided to the undergraduates as a case to discuss the design of a scale to evaluate digital learning materials (the animal conservation apps). Before discussion, the undergraduates were asked to recall the experience of using the animal conservation apps, for example, what had been learned from using the apps and which types of queries could be asked to understand primary school pupils’ cognition and knowledge related to the core competency and key elements involved in the Curriculum Guidelines of 12-Year Basic Education: Art Domain. After discussion, the undergraduates in each group reached the following consensus:

• The five groups of undergraduates all agreed that the apps could foster pupils’ academic performance in expression, appreciation, and practice due to the digital learning materials, which contained rich information of animal photos, videos, graphics, texts, games, and knowledge checkpoints. This information offered pupils sufficient knowledge for them to observe, appreciate, and understand the content related to animal conservation and the field of art (FG1080513, 1080514).
• All five groups of undergraduates stated that they had no experience in designing this kind of scale. Therefore, they all modified question stems or answer options by following the content and propositions of the sample case (CSTa) (FG1080513, 1080514), with the scale designed by Group 5 as an example in

  Table 7. The scale designed by Group 5.

  Aspects of Learning	Competence Indicators	Questions and Descriptions
  Expression	1-II-2 Being able to explore visual elements, and express feelings and imagination.	After playing the matching game, do you have a better understanding of the environment and animals in tropical rainforests? Please briefly introduce the tropical rainforest and animals.
  Appreciation	2-II-5 Being able to observe life objects and artwork, and appreciate their own and others’ creations.	After reading the introduction on animals, which animal are you most impressed with? Please draw the most impressive animal or animals, and make a comparison of the differences between the animal(s) you draw and the animal(s) in the app. Do you understand their different attributes?
  Practice	3-II-4 Being able to beautify the living environment through collection or artistic creation.	Please share your work with your classmates and friends, and think about where the materials, colors, or textures could be used in your daily lives.

  .
• Four groups of undergraduates believed that the aspect of expression was a kind of scenario question for pupils to ponder before responding. The undergraduates planned to ask pupils to observe the design, materials, and elements used in the apps, as well as think about the experience of using the apps in order to learn about particular requirements, problems, or ideas from the pupils (FG1080513, 1080514).
• The five groups of undergraduates agreed that it was challenging to formulate questions on the aspect of appreciation. Thus, open-ended questions were believed to be useful for helping pupils to express themselves and obtain information after using the apps (FG1080513, 1080514).

5.

Four groups of undergraduates thought that queries involved in the aspect of practice should be situational questions. They agreed it would be helpful to create or simulate a situation for pupils to integrate their prior knowledge and experience, to apply knowledge learned to a real-life context, to share experiences and feelings with peers, and to provide detailed suggestions to improve the development of the apps (FG1080513, 1080514).

Table 8. Evaluating the CCTEa designed by the five groups of undergraduates.

Group No.	1	2	3	4	5
No. of Questions	6	5	3	4	5
Expression	4.50	4.00	3.50	3.50	4.00
Appreciation	4.50	4.50	3.50	4.50	4.50
Practice	4.50	4.00	3.50	4.00	4.00
Kappa Values	0.90	0.85	0.75	0.85	0.85

shows the evaluation results of the scales designed by the five groups of undergraduates offered by two groups of experts. The kappa value of each group was greater than or equal to 0.70, which indicates that the opinions of both groups of experts were highly consistent. Although the evaluation result of Group 3 was 3.50 (normal), the other four groups all reached the level of excellent and good. The results reveal that the scales designed by undergraduates were approved by both groups of experts, who recognized their relevance. The lower average score in Group 3 may have been due to less information and description being provided, which may have led to precision problems and misunderstanding.

4.3. Comments on the Results

Based on the analysis above, this study suggests that the ADDIE model is one of the most useful instructional design models to help undergraduates organize different types of materials and learning content for developing digital learning materials (the animal conservation apps) step by step. It was found that the undergraduates could integrate the learning contents of relevant learning units (in primary school textbooks) to design their apps in the analysis stage. They were able to analyze pupils’ background/prior knowledge in order to adapt and organize learning materials and content to suit pupils’ requirements in the meantime. These results show that in order to design relevant learning material for learners, the learners’ understanding, comprehension, and capabilities are the key to achieving educational goals [19,20].

In the design stage, five groups of undergraduates were able to plan and design their new apps by analyzing the previously developed apps provided in the course and by learning from the experience of using these apps. Moreover, the knowledge and learning contents derived from primary school textbooks were applied to design the animal conservation apps and further deepen and expand animal conservation- and environmental education-related issues in the apps. In the development stage, four groups of undergraduates applied a game-based learning approach to their topic and learning goals to improve pupils’ learning interest and motivation in the context of using the animal conservation apps. There was only one group of undergraduates that integrated a story-oriented design approach and game-based learning approach to design their app, as a result providing individual pupils with better immersion in the learning content. The undergraduates were able to learn from the process of using the CK and DS learned in the course and developed relevant digital learning materials for pupils with the ADDIE model, which allowed individual undergraduate to learn the approach to solve everyday and future problems (e.g., [27]).

5. Conclusions

5.1. Review of Research Findings

5.1.1. Undergraduates Could Develop Digital Learning Materials According to the ADDIE Model

In this study, the focus group method was used to guide the undergraduates to analyze, design, develop, implement, and evaluate their own design throughout the course. It was found that the five groups of undergraduates were able to discuss the learning content, layout, games, and knowledge checkpoints when using and experiencing the apps developed in the past. In addition, the undergraduates could organize the knowledge and skills learned in the course to develop, modify, and refine the animal conservation apps according to the ADDIE model. Finally, the digital learning materials (the animal conservation apps) designed by each group were evaluated by the DLMS and received A+ to A (reaching excellent/good level).

5.1.2. Undergraduates Could Develop Scales by Adapting Previous Cases

Most of the scales designed by the undergraduates were modified from the sample cases provided in the course. Four groups of undergraduates designed more questions in their scales and the descriptions of questions were detailed and complete, whereas the scale designed by the other group was simple and lacked detailed descriptions. It was found that the undergraduates could apply the knowledge and skills learned in this course, and were able to analyze and imitate the previously developed scales provided in the course to develop their own scales. These scales were evaluated by the experts (the results reaching levels from good to average).

5.2. Recommendations for Future Research

5.2.1. Increasing Conversation among Undergraduates, Field (Taipei Zoo), and Zookeepers May Deepen the Understanding of the Needs in the Field

During the development process, the undergraduates were proactive in analyzing and organizing relevant digital learning materials and learning content, as well as searching for theories and knowledge regarding animal conservation, environmental education, and the requirements of pupils on the Internet and in textbooks. In the future, senior and experienced zookeepers could be invited to participate in focus group discussions. Furthermore, their rich practical experience will be helpful to provide professional knowledge and field experience to undergraduates in developing their critical thinking, reasoning, analytical skills, and overall knowledge related to animal conservation and environmental education.

5.2.2. The Digital Learning Materials (the Animal Conservation Apps) Could Be Used in Taipei Zoo in a Subsequent Study

This study reports on a one-semester Learning Design and Practice course. In the limited course time, a total of five groups of undergraduates integrated and transformed CK and DS into developing digital learning materials (animal conservation apps) and assessment tools. It would be helpful to invite primary school pupils to participate in a field study by using the apps in Taipei Zoo in the future in order to collect detailed learners’ opinions and experience to not only refine and improve the development of the apps, but also to conduct evaluations of the learning effectiveness of the apps.