1. Introduction
Education can improve the literacy of people and is the fundamental approach to national growth [
1,
2]. In particular, the increasingly complex contemporary society has led to various complex problems in fields such as economics, commerce, science and technology, culture, and society that can no longer be solved by merely relying on the knowledge taught in schools. To solve real-life problems, students must be able to flexibly employ learning materials and adapt them to the scenario of the problem [
3,
4]; this capability depends on the practical experience accumulated from frequent use, real-life experience, or practise of learning materials. Therefore, the use of learning materials that enable the practical application of knowledge learned in school is imperative for students, and increasing students’ experience in applying learning materials to practical situations can improve their problem-solving capabilities.
Modern students need problem-solving skills to adapt to rapid changes, meet workplace demands for innovation, and gain a competitive edge academically and professionally [
5];. These skills enable efficient handling of complex issues. Education can foster problem-solving abilities through practise-oriented learning with real-world scenarios, interdisciplinary learning to integrate diverse knowledge, and collaborative learning via teamwork [
6]. Emphasising problem-solving skills not only enhances academic and career success but also boosts confidence and independence in daily life.
In an examination-oriented learning system, the pressures and setbacks experienced by students over the years because of exam participation have caused aversion to and evasion of learning. Exploring whether changing teaching strategies or the representation of learning materials within traditional systems can improve learners’ attitudes and foster problem-solving skills might be a worthwhile concept to investigate and verify.
Animation and manga are media platforms with content including images, text, and videos; thus, teaching materials based on animation and manga can serve as an alternative to improve students’ learning attitudes [
7]. When using media as teaching content, animation and manga provide relaxing and entertaining aspects of learning [
7]. Students in Taiwan are often exposed to various types of advertising and publishing materials using animation and manga as the media; thus, they have already developed a certain level of literacy for animation and manga. Students with such literacy have fewer difficulties grasping, understanding, and critically thinking about the knowledge and viewpoints presented through animation and manga, and appropriate learning models can guide learners to engage in more discussion on the learning content [
8,
9]. In addition, teachers are also participants in the teaching process. Recent studies have shown that using animation and manga as teaching materials can be beneficial for learning outcomes. For example, Wisniawati, Rasiban, and Dewanty’s (2022) study results showed that the use of anime in language listening comprehension learning was able to increase learning motivation and listening skills. This is evidenced by the activities students engaged in during learning and the improved assignment scores they obtained. Some anime programmes are even effective in enhancing learners’ acquisition of specific professional knowledge, such as blood and cells [
10].
This study was conducted to implement teaching materials based on animation and manga to improve students’ problem-solving capabilities, and the potential factors affecting the development of their problem-solving capabilities were explored. The identified factors should be considered by teachers when teaching materials based on animation and manga are employed, thereby improving students’ problem-solving capabilities.
2. Purposes
Current research and the literature have made significant progress in exploring learning methods. However, there is still no consensus on the optimal psychometric model to describe these learning strategies. Different learning methods may exhibit vastly different effects in various contexts, making it challenging for a single model to capture their complexity comprehensively and accurately.
Therefore, this study will solve this problem by utilising a questionnaire tool for data collection. The collected data will be analysed using regression methods, and the results will be compared and validated to identify the most suitable psychometric model for effectively describing and evaluating different learning strategies.
The model we consider can explain how animated or comic-based materials affect learners’ outcomes and enhance problem-solving skills. Nevertheless, as articulated in the research question, a single model cannot adequately explain the influence of such materials on the improvement of problem-solving abilities. In this study, it was assumed that models can effectively predict students’ problem-solving capabilities. Thus, these models can serve as control indicators. Therefore, in this study, models with several distinct factor combinations were developed, and their accuracy in predicting the problem-solving capabilities of students was determined through their comparison and analysis. This enables the identification of assessment indicators that are influential to students when designing and implementing subsequent courses to improve students’ problem-solving capabilities. The purposes of this study were as follows:
In this study, a multimedia teaching system that explained knowledge and principles through animation and manga was developed, enabling students to learn with ease and enjoy learning while reducing learning pressures. The teaching system enabled students to interact with knowledge and facilitated the practical application of knowledge, thereby enhancing their problem-solving capabilities.
3. Literature Review
3.1. Relevant Studies on Comprehension and Application of Animation
Taiwan has been deeply influenced by Japanese culture and subculture. Animation and manga, which are representative of Japanese subculture, are popular media for students in Taiwan. The examination-oriented teaching of the Taiwanese education system has negatively affected students’ interest in learning. In addition, the teaching materials emphasise text reading, and the rigid reading layout and content inevitably exert adverse effects on students’ learning attitudes [
8,
9]. To improve students’ learning outcomes, teaching materials based on animation and manga have been increasingly adopted [
11]. Such teaching materials reduce learners’ cognitive load by changing the representation of materials to improve their readability [
12]. The combination of various sensory elements in animation and manga, such as illustrations and audio, provides more acceptable representations that promote learning among students and stimulate them; thus, students with poor text comprehension skills can gain knowledge through other representations.
3.2. Attitude
Attitude is a mental state that affects actual behaviour. Attitude is a type of psychological state involving an individual’s perception of others and the interpretation of events and objects, from which persistent behavioural tendencies are generated. From the perspective of the theory of reasoned action, attitudes are shaped by personal beliefs and further affect people’s actual behaviours [
13] Therefore, many studies related to information system assessment and learning have selected attitude as an evaluation indicator [
14,
15,
16,
17,
18]. For example, Chou and Luo (2014) evaluated the information system of digital learning using the technology acceptance model, which measures attitude. Russo and Minas’ (2020) study results show that positive learning attitudes significantly enhance problem-solving abilities by increasing student engagement, confidence, and motivation. Students with positive attitudes invest more time and effort, approach problems with an open and proactive mindset, and collaborate effectively with peers. This boosts their creativity, critical thinking, and overall problem-solving efficiency. Teachers should focus on cultivating these attitudes to improve students’ problem-solving skills.
3.3. Problem-Solving Capabilities
Problem-solving capabilities enable an individual to obtain differing solutions for distinct practical problems. Skills for solving such problems are extensive and mutual, incorporating learners’ professional, thinking, and logic construction capabilities. In teamwork, problem-solving also includes members’ collaboration and coordination capacities. Lan (2016) reviewed the relevant literature and proposed four major dimensions of problem-solving capabilities: (1) the ability to identify problems; (2) the ability to break down and clarify problems; (3) the planning, selection, and implementation of strategies, the analysis of one’s conditions and resources, and the construction and selection of feasible solutions; (4) and the ability to examine the cause of the problems and the capacity for self-reflection.
3.4. Self-Regulation Learning
Self-regulation learning is an educational concept that emphasises that learning must be determined by learners and performed by disciplined learners. Using the education system of Hong Kong as an example, Xu and Wu (2013) stated that self-regulation learning is a teaching method that helps students to understand how to conduct learning. In this method, a goal must be set for students to explore and apply knowledge. Therefore, during the process of self-regulation learning, students should (1) have clear goals [
19,
20,
21,
22], (2) possess self-selected learning approaches and techniques, (3) and demonstrate self-control [
23], and (4) teachers must reduce their control of students’ learning. Self-regulation learning includes (1) an emphasis on self-realisation and achieving learning objectives [
24,
25]; (2) the imitation and observation of social contexts [
26]; (3) the influences of social and peer interactions [
27]; and (4) an emphasis on the resolution of cognitive conflicts and the impulse of curiosity [
28]. Therefore, self-regulation is a spontaneous learning model that can effectively cultivate learning attitudes in students, thereby achieving successful outcomes in problem-solving capability-oriented courses that require a high level of self-regulation.
3.5. Multimedia Teaching Materials
Multimedia teaching materials are a type of digital teaching material that incorporates videos, images, audios, and various interactive elements, creating a learning experience that differs from conventional classroom teaching [
29]. Therefore, these materials can stimulate students and produce distinct learning experiences [
30]. The development of network infrastructure and technology provides students with easier access to multimedia teaching materials and enables the simpler operation of the materials, helping students preview and review the course. These materials have extensive applications, particularly in auxiliary teaching or subjects that involve abstract representations. For example, Nelson (2018) introduced multimedia teaching materials into mathematics courses and revealed the interaction between students and the dynamic representation of mathematics.
Therefore, the digitalised features of multimedia teaching materials are not only a suitable presentation method for animation and manga but also possess favourable interactive and feedback functions that can be adjusted to enhance students’ learning. Thus, learning methods become more diverse, in line with the trend toward increasingly diverse and advanced teaching approaches and concepts.
4. Research Method
This study seeks to evaluate the representativeness of various psychometric models in assessing students’ problem-solving capabilities. Thus, the research involves the collection and validation of data through questionnaire contents that can represent the ideas behind the constructs and variables of the models. The comparative analysis will employ the R-squared value from regression analysis as the key metric for evaluation. A questionnaire survey was designed and administered in this study.
4.1. Sample and Participants
We selected a total of 136 students, including 53 junior students from a science and technology university in southern Taiwan and 53 junior students and 30 freshmen from a national university in southern Taiwan, based on the characteristics of snowball sampling. Snowball sampling is particularly suitable for reaching and defining hard-to-access populations. This method allowed us to reach students with similar backgrounds and learning environments, ensuring the representativeness and credibility of our study. By leveraging recommendations from initial samples, we could expand our sample scope, gaining comprehensive and in-depth data.
Additionally, we chose students from these two universities to represent different educational environments and backgrounds, facilitating comparisons between schools and grade levels. Snowball sampling enabled us to collect sufficient data to analyse these differences while ensuring the diversity and representativeness of our sample. This approach aids in understanding students’ learning behaviours and problem-solving abilities, providing insights for future educational strategies.
After excluding the invalid questionnaires, this study yielded 104 valid questionnaires.
4.2. Research Instrument
The questionnaire survey developed and used to collect data included participants’ demographics, and participants’ responses were evaluated on a five-point Likert scale. The dimensions of the questionnaire, which included the comprehension and application of animation and manga, problem-solving attitude, learning attitude, attitude towards self-regulation learning, attitude towards the digitisation of teaching materials, and self-inspection of problem-solving capabilities, were based on the literature review’s content.
Table 1 lists the connotations for each dimension. The development process was as follows:
Define objectives: establish the research goals and required information.
Design structure: create the questionnaire’s format and flow.
Draft questions: develop clear and relevant questions.
Revise: improve the questionnaire based on relevant expert peer feedback.
The reliability and validity of the questionnaire were tested using factor analysis and reliability analysis. Subsequently, the proposed models were verified through regression analysis. The reliability and validity tests can be used to verify the quality of the collected data; this enables the accuracy and authenticity of the research results to be determined [
31]. A Cronbach’s alpha value of ≥0.7 was used as the criterion for the reliability analysis. Factor analysis was adopted to evaluate the validity of the questionnaire, wherein the Kaiser–Meyer–Olkin (KMO) test (KMO value > 0.8) and sphericity test (statistical significance) were employed to determine the suitability of the collected data for factor analysis. Thereafter, the questionnaire items were categorised to test whether the items were in accordance with the researchers’ intended design and whether they were adequate for examining the same factor (usually based on a factor loading of >0.5). Regression analysis is commonly used to examine the strength of correlations between variables, and regression equations are typically employed as models to estimate the relationships between various variables.
In this study, two models were developed using different combinations of factors. Facet matching and reliability and validity tests were repeated through the trial-and-error method. Finally, two types of facet models passed the reliability and validity tests. And the correlationship analysis and regression analysis verified the model’s effect on the research purpose.
The reliability and validity of the two models were tested as follows:
4.2.1. Model 1
The factors in Model 1 were the comprehension and application of animation or manga, problem-solving attitude, and self-inspection of problem-solving capabilities. The KMO value of the sphericity test reached 0.906 > 0.7, reaching statistical significance. Thus, the model was suitable for validity evaluation through factor analysis. The cumulative variance of the rotation sums of the squared loadings of the problem-solving attitude, self-inspection of problem-solving capabilities, and comprehension and application of manga or animation factors in Model 1 reached 69.679%, thus implying that these factors explained approximately 70% of the total variance. The factor loadings of the items were between 0.860 and 0.648. Cronbach’s alpha values were used to determine the reliability of the model. The Cronbach’s alpha values of the three factors were as follows: (1) the value for the comprehension and application of manga or animation was 0.951 > 0.7; (2) the value for problem-solving attitude was 0.981 > 0.7; and (3) the value for self-inspection of problem-solving capabilities was 0.975 > 0.7. These values obtained for Model 1 were greater than the recommended threshold; thus, the questionnaire items in Model 1 possessed satisfactory validity and reliability.
4.2.2. Model 2
The factors in Model 2 were attitude towards self-regulation learning, attitude towards digitisation of teaching materials, and self-inspection of problem-solving capabilities. The KMO value of the sphericity test was 0.921 > 0.7, reaching statistical significance. Thus, the model was suitable for validity evaluation through factor analysis. The cumulative variance of the rotation sums of the squared loadings of attitude towards self-regulation learning, attitude towards the digitisation of the teaching materials, and self-inspection of problem-solving capabilities factors in Model 2 reached 68.847%, thus implying that the factors explained approximately 70% of the total variance. The factor loadings of the items were between 0.577 and 0.804. Therefore, the results indicated that the three variables were valid. Cronbach’s alpha values were used to determine the reliability of the model. The Cronbach’s alpha values of the three factors were as follows: (1) the value for attitude towards self-regulation learning was 0.964 > 0.7; (2) the value for attitude towards the digitisation of teaching materials was 0.937 > 0.7; and (3) the value for self-inspection of problem-solving capabilities was 0.974 > 0.7. These values obtained for Model 2 were greater than the recommended threshold; thus, the questionnaire items in Model 2 presented satisfactory validity and reliability.
4.3. System Development
The proposed teaching system involved using a manga format to create interactive animation and manga multimedia teaching materials. The system contents were based on a video playback application programming interface (API), and a video playback format that could be designed with interactive functions was selected and converted into interactive animation learning material. Students could interact with the contents of the textbook through the interactive functions to learn about the practical application of knowledge.
Figure 1 shows the system that converts comics into video playback. This conversion includes four steps:
Introduction: introducing the video by presenting an example;
Sample description: explaining the contents of the sample;
Using animation to present calculation operations: disassembling the calculation steps and explaining the calculation steps through animation;
Answer display: using animation to display the calculation results.
4.4. Teaching System Design
Experimental Procedure
The experimental procedure for this study involved the following steps:
Step 1: Conduct a literature review and list factors that affect problem-solving (1 month).
Step 2: Design the framework of the course (1 month).
Step 3: Design an application involving multimedia teaching materials based on animation and manga (3 months).
Step 4: Design the survey instrument (2 weeks).
Step 5: Implement the teaching materials in class (1 h per week for 3 weeks).
Step 6: Conduct the questionnaire survey for approximately 1 h after the teaching experiment.
Step 7: Collect the data through the survey from three classes.
Step 8: Perform data analysis using the statistics software IBM SPSS 17.0 (IBM Inc., Armonk, NY, USA).
The experimental teaching process in this study involved the following steps:
Step 1: Commence the learning course.
Step 2: Implement the teaching materials based on animation and manga.
Step 3: Use the teaching content presented by animation and manga as the learning subject.
Step 4: Students complete the learning tasks designed on the basis of the teaching materials and receive feedback on the tasks.
Step 5: Conduct the questionnaire survey in the last week.
Step 6: Teachers review the students’ learning outcomes.
5. Research Results
The predictive power of the two models for problem-solving capabilities was analysed using regression analysis, and the results are as follows.
5.1. Model 1
The factors in Model 1 were the comprehension and application of manga and animation, problem-solving attitude, and self-inspection of problem-solving capabilities.
Table 2 presents the correlation analysis of the factors in Model 1.
The Pearson correlation coefficient between the comprehension and application of manga and animation and self-inspection of problem-solving capabilities was 0.503 < 0.7, indicating a moderately significant correlation. The Pearson correlation coefficient between problem-solving attitude and self-inspection of problem-solving capabilities was 0.720 > 0.7, indicating a strong and significant correlation.
The total variance of the dependent variable explained by the independent variables in Model 1 was R2 = 52.5%, and significance was achieved (p < 0.05); thus, the amount of variance in the dependent variables associated with the independent variables was significant.
According to the coefficient table, problem-solving attitude achieved significance, whereas the comprehension and application of manga and animation did not achieve significance. This result was possibly because problem-solving attitude had some influence on the self-inspection of problem-solving capabilities. However, the factor of the comprehension and application of manga and animation was one of the essential elements in the regression equation, and tests and verifications through other approaches were conducted to determine the influence of this factor. Problem-solving attitude and the comprehension and application of manga and animation had a tolerance of >0, and their Variance Inflation Faction (VIF) was <10; thus, the collinearity was negligible. The regression equation was established as follows:
Unstandardised equation: self-inspection of problem-solving capabilities = 1.715 + 0.065. (comprehension and application of manga/animation) + 0.532 (problem-solving attitude).
Standardised equation: self-inspection of problem-solving capabilities = 0.102 (comprehension and application of manga/animation) + 0.658 (problem-solving attitude).
5.2. Model 2
The factors in Model 2 were attitude towards self-regulation learning, attitude towards digitisation of teaching materials, and self-inspection of problem-solving capabilities. Table 4 presents the correlation analysis of the factors in Model 2.
According to
Table 3 and
Table 4 the Pearson correlation coefficient of attitude towards the digitisation of teaching materials and self-inspection of problem-solving capabilities was 0.614 < 0.7, showing a moderately significant correlation. The Pearson correlation coefficient of attitude towards self-regulation learning and self-inspection of problem-solving capabilities was 0.671 < 0.7, showing a moderately significant correlation.
The total variance of the dependent variable explained by the independent variables in Model 2 was R2 = 47.6%, and the variable summary table revealed that significance was achieved (p < 0.05); thus, the amount of variance in the dependent variable associated with the independent variables was significant.
As shown in the coefficient table, attitude towards digitisation of teaching materials and attitude towards self-regulation learning both achieved significance, and both factors had a tolerance value of >0 and a VIF of <10; thus, the collinearity was negligible. The regression equation was established as follows:
Unstandardised equation: self-inspection of problem-solving capabilities = 1.715 + 0.183 (attitude towards digitise of teaching materials) + 0.396 (attitude towards self-regulation learning).
Standardised equation: self-inspection of problem-solving capabilities = 0.246 (attitude towards digitise of teaching materials) + 0.483 (attitude towards self-regulation learning).
6. Discussion
To measure problem-solving capabilities, two statistically significant models with various factor combinations were developed in this study, and their predictive power for problem-solving capabilities was analysed. The results showed that when the comprehension and application of manga and animation and problem-solving attitude were used as independent variables, the prediction accuracy for problem-solving capabilities was the highest among the two proposed models (R2 = 52.5%); however, the comprehension and application of manga and animation in the model did not achieve significance. This was probably because problem-solving attitude had some influence on self-inspection of problem-solving capabilities. However, the comprehension and application of manga and animation was one of the essential components of the regression equation, and tests and verifications through other approaches were conducted to determine the influence of this factor. Considering the factor combination in which both independent variables achieved significance, the model with attitude towards digitisation of teaching materials and attitude towards self-regulation learning for problem-solving capabilities as the independent variables attained the highest predictive power (R2 = 47.6%).
According to the aforementioned results, attitude towards digitisation of teaching materials and attitude towards self-regulation learning were more adequate factors for effectively interpreting and predicting problem-solving capabilities. However, the combination of comprehension and application of manga and animation and problem-solving attitude achieved the highest predictive power. The focus and contributions of previous studies have been the development of learning instruments or teaching methods, and these studies have verified the effectiveness of a single indicator or model [
14,
15,
16,
17,
18,
32,
33,
34,
35]. In this study, multiple factors were combined to develop models for determining the effects of and relationships between the factors; subsequently, the differences in prediction accuracy were compared and verified. This differentiated the present study from previous studies.
The current development of teaching materials remains inclined towards the use of academic and theoretical principles with educational bases. However, the teaching materials developed in this study were based on learners’ possible reactions and students’ needs at the actual teaching site. Therefore, the content and interactive function that satisfy users’ needs were considered when designing the teaching materials for this study, with the expectation that the developed teaching materials would be verified through practical approaches and experimental teaching.
7. Conclusions
This study developed three statistically significant models to measure problem-solving capabilities, analysing their predictive power. The model using comprehension and application of manga and animation, along with problem-solving attitude, showed the highest prediction accuracy (R2 = 52.5%), although comprehension and application of manga and animation did not achieve significance. The combination of attitudes towards digitization of teaching materials and self-regulated learning also provided a high predictive power (R2 = 47.6%).
The results suggest that while attitudes towards digitization and self-regulated learning are effective for predicting problem-solving capabilities, the combination of manga and animation comprehension with problem-solving attitude offers the highest predictive accuracy. Unlike previous studies that focused on single indicators, this study combined multiple factors, comparing their predictive accuracy to highlight the most effective models.
The two predictive models were developed for use as control indicators in teaching, aiding in the real-time management of students’ learning and serving as parameters for future educational systems aimed at enhancing problem-solving capabilities.
The models’ predictive power can guide teachers in adjusting teaching content. Interactive multimedia teaching materials based on animation and manga were designed to change students’ learning attitudes and enhance practical problem-solving skills.
Suggestions
Several factors in this study did not achieve significance. Thus, the incorporation of different factors in the model should be further examined, or more data should be collected in future studies to verify the significance of these factors.