*3.1. Interface Design*

The system interface is divided into Interfaces A, B and C. A is an intelligent virtual agent, B is a course module and C is a course menu, as shown in Figure 1. The specific functions of the three interfaces are as follows:


### *3.2. Course Model*

The course refers to digital art. In addition to the textual description of the course, picture and video examples are used to help the users understand the course contents. It takes 15 min for the subject to watch the course. Interactive design-related technologies are added in the learning process to improve the user's interest of learning.


**Figure 1.** Affective Tutoring System Interface. **Figure 1.** Affective Tutoring System Interface.

#### *3.2. Course Model 3.3. Agent Model*

The course refers to digital art. In addition to the textual description of the course, picture and video examples are used to help the users understand the course contents. It takes 15 min for the subject to watch the course. Interactive design-related technologies are added in the learning process to improve the user's interest of learning. *3.3. Agent Model*  The agent model can affect the learner's learning emotion, perceive the learner's learning situation and give corresponding emotional guidance. In this study, the semantic analysis of affective computing was used to judge the semantics of dialogue between the user and the agent and provide appropriate emotional feedback to the agent to adjust the user's learning emotion and enhance his/her learning willingness. Therefore, the system set up an agent model as a bridge between the learner and the system. During the recognition, the system will respond to corresponding sentences and change the graphs of the agent model to enhance the learner's willingness to operate. For example, when any positive emotion is recognized in the learner's sentences by the system, the agent module will display a graph of positive emotion, and when any negative emotion is identified from the learner's sentences by the system, the emotion agent model will display a graph of The agent model can affect the learner's learning emotion, perceive the learner's learning situation and give corresponding emotional guidance. In this study, the semantic analysis of affective computing was used to judge the semantics of dialogue between the user and the agent and provide appropriate emotional feedback to the agent to adjust the user's learning emotion and enhance his/her learning willingness. Therefore, the system set up an agent model as a bridge between the learner and the system. During the recognition, the system will respond to corresponding sentences and change the graphs of the agent model to enhance the learner's willingness to operate. For example, when any positive emotion is recognized in the learner's sentences by the system, the agent module will display a graph of positive emotion, and when any negative emotion is identified from the learner's sentences by the system, the emotion agent model will display a graph of negative emotion. If no emotional keywords are contained in the sentence or the system is unable to recognize the input sentence, a dynamic confusion graph without emotional feedback will be displayed. After the user's emotions are recognized by the system, the agent's emotional feedback will be given to the user in the set emotional performance state, and eight kinds of agent emotion feedback will be set up, including happiness, sadness, fear, frustration, anger, surprise, disgust, and doubt, as shown in Figure 2 below.

negative emotion. If no emotional keywords are contained in the sentence or the system is unable to recognize the input sentence, a dynamic confusion graph without emotional feedback will be displayed. After the user's emotions are recognized by the system, the agent's emotional feedback will be given to the user in the set emotional performance state, and eight kinds of agent emotion feedback will be set up, including happiness, sadness, fear, frustration, anger, surprise, disgust, and doubt, as shown in Figure 2 below.

**Figure 2.** The icons of the emotional agent's positive and negative emotions. **Figure 2.** The icons of the emotional agent's positive and negative emotions.

### *3.4. Analysis of Eye Movement Statistics 3.4. Analysis of Eye Movement Statistics*

The proportion and duration of time spent on the ROI blocks of the subject. In this study, an eye tracker gifted by Professor He Hongfa of National Taiwan Normal University was used. It is a research outcome from developing the eye tracker hardware and software in a five-year project for the Top University Project sponsored by the Ministry of Education of the Republic of China. The statistics analysis software developed by this The proportion and duration of time spent on the ROI blocks of the subject. In this study, an eye tracker gifted by Professor He Hongfa of National Taiwan Normal University was used. It is a research outcome from developing the eye tracker hardware and software in a five-year project for the Top University Project sponsored by the Ministry of Education of the Republic of China. The statistics analysis software developed by this study was analyzed according to the calculation formula in the eye tracker's manual, as shown below.


### [Formula Definition: Total fixation counts of fix.txt] *3.5. System Usability Scale*

4. Fixation Counts

[Meaning: Total fixation counts of the subject during the experiment] 5. Percent Time Fixated Related to Total Fixation Duration (%) [Formula Definition: fix.txt; duration totaling/fixation duration in ROI (duration totaling of fix.txt)] [Meaning: Fixation duration in the zone to total fixation duration] *3.5. System Usability Scale*  In this study, the System Usability Scale (SUS) was used. It was developed by Equipment Co., Ltd. in 1986 and mainly used to evaluate the usability of the system. The System Usability Scale is a low-cost, reliable and rapid method to effectively assess the user's subjective feelings toward the system [40]. The Likert scale was used to measure the scale In this study, the System Usability Scale (SUS) was used. It was developed by Equipment Co., Ltd. in 1986 and mainly used to evaluate the usability of the system. The System Usability Scale is a low-cost, reliable and rapid method to effectively assess the user's subjective feelings toward the system [40]. The Likert scale was used to measure the scale from point 1 to 5 from strongly disagree to strongly agree. The formula was, however, divided into even numbers and odd numbers. For even-numbered questions, the numerical value 5 was subtracted from the original scores of the question to be the score available. For odd-numbered questions, the original scores were subtracted from the numerical value 1 to be the score available. Finally, the scores of the even and odd-numbered questions were added and multiplied by 2.5 to obtain the final satisfied scores [41]. The scale was analyzed by the formula, and the statistics selected by the subjects according to the statistics scale

were scored with a total of 100 points for evaluation and analysis. The higher the score, the higher the degree of satisfaction on the system evaluated. statistics scale were scored with a total of 100 points for evaluation and analysis. The higher the score, the higher the degree of satisfaction on the system evaluated.

from point 1 to 5 from strongly disagree to strongly agree. The formula was, however, divided into even numbers and odd numbers. For even-numbered questions, the numerical value 5 was subtracted from the original scores of the question to be the score available. For odd-numbered questions, the original scores were subtracted from the numerical value 1 to be the score available. Finally, the scores of the even and odd-numbered questions were added and multiplied by 2.5 to obtain the final satisfied scores [41]. The scale was analyzed by the formula, and the statistics selected by the subjects according to the

### **4. Data Analysis and Results 4. Data Analysis and Results**  *4.1. System Usability Analysis*

### *4.1. System Usability Analysis*

In order to analyze the user's usability of the system, the statistical analysis was conducted with the System Usability Scale. A total of 15 participants took part in the experiment and all of them filled in the scale, which means that all the 15 scales are effective. In order to analyze the user's usability of the system, the statistical analysis was conducted with the System Usability Scale. A total of 15 participants took part in the experiment and all of them filled in the scale, which means that all the 15 scales are effective.

### 4.1.1. System Usability Scale—Reliability Analysis 4.1.1. System Usability Scale—Reliability Analysis

*Sustainability* **2022**, *14*, x FOR PEER REVIEW 7 of 18

Some of questions in the System Usability Scale are roundabout questions. Therefore, it is required to invert these roundabout questions first, and then to perform the statistical analysis according to the scale statistics filled in by the users to determine the user's usability of the system. From the overall usability scale, the average value is 4.01 with a standard deviation of 0.512. This means that the internal consistency reliability of the scale is high. Figure 3 shows the histogram of the overall usability scale. Some of questions in the System Usability Scale are roundabout questions. Therefore, it is required to invert these roundabout questions first, and then to perform the statistical analysis according to the scale statistics filled in by the users to determine the user's usability of the system. From the overall usability scale, the average value is 4.01 with a standard deviation of 0.512. This means that the internal consistency reliability of the scale is high. Figure 3 shows the histogram of the overall usability scale.

**Figure 3.** Overall Usability Scale. **Figure 3.** Overall Usability Scale.

4.1.2. System Usability Scale—Descriptive Statistics 4.1.2. System Usability Scale—Descriptive Statistics

This refers to the statistical result of each question of the System Usability Scale. Table 1 shows the analysis statistics of the System Usability Scale (percentage of five-point scale) and Table 2 shows the results of the analysis of each question of the Usability Scale. This refers to the statistical result of each question of the System Usability Scale. Table 1 shows the analysis statistics of the System Usability Scale (percentage of five-point scale) and Table 2 shows the results of the analysis of each question of the Usability Scale.

**Table 1.** Analysis Statistics of System Usability Scale (Percentage of five-point scale).



**Table 2.** Results of Analysis of Each Question of the Usability Scale.

The sum of the highest and second highest percentages for each question on a 5-point Likert scale represents agreement. The third highest percentages for each question on a 5-point Likert scale represent neutrality and the second lowest and lowest percentages for each question on 5-point Likert scale represent disagreement. Analysis results are as follows.

Q1: I think I would often use the system. This is a straightforward question with an average number of 4.33 and standard deviation of 0.617. In all, 40% of users are very willing to use the affective tutoring system often for learning, 53.3% of users would like to use the affective tutoring system for learning often and 6.7% of users are generally willing to use the system. As the system is a combination of emotion recognition and the course, emotion feedback is given to users in the aspect of emotion recognition, which has increased the enjoyment of the users. Therefore, it is inferred that most users are satisfied with this system.

Q2: I think the system is too complicated. This is a roundabout question with an average number of 3.47 and standard deviation of 0.915. In all, 13.3% of users strongly disagree that the system is too complicated, 33.4% of users disagree that the system is too complicated, 40% of users are neutral about the complexity of the system and 13.3% of users think that the system is too complicated.

Q3: I think the system is easy to use. This is a straightforward question with an average number of 4.07 and standard deviation of 0.799. In all, 33% of users think the system is very easy to use, 40% of users think the system is easy to use and 26.7% of users are neutral about the complexity of the system.

Q4: I think I need a technician's help to use the system. This is a roundabout question with an average number of 3.40 and a standard deviation of 0.910. In all, 13.30% of users think that it is absolutely necessary to have a technician to help use the system, 26.7% of users think that it is necessary to have a technician to help use the system, 46.70% of users are neutral about whether it is necessary to have a technician to help use the system and 13.3% of users think that it is unnecessary to have a technician to help use the system. Therefore, we are getting to a conclusion that the users do not have sufficient knowledge of affective computing and the functions of the agents need to be introduced. Human resources are required to be of assistance to guide the users.

Q5: I think all functions of the system are integrated well. This is a straightforward question with an average number of 3.80 and standard deviation of 0.775. In all, 20% of users strongly agree that all functions of the system are integrated well, 40% of users agree that all functions of the system are integrated well and 40% of users are neutral about whether all functions of the system are integrated well.

Q6: I think there is too much contradiction in the system. This is a roundabout question with an average number of 4.00 and standard deviation of 0.756. In all, 26.60% of users strongly disagree that there is too much contradiction in the system, 46.70% of users disagree that there is too much contradiction in the system and 26.70% of users are neutral about whether there is too much contradiction in the system.

Q7: I think most people could learn how to use the system fast. This is a straightforward question with an average number of 4.40 and standard deviation of 0.632. In all, 46.60% of users strongly agree that most people could learn how to use the system fast; 46.70% of users agree that most people could learn how to use the system fast and 6.70% of users are neutral about whether most people could learn how to use the system fast.

Q8: I think the system is very difficult to use. This is a roundabout question with an average number of 4.40 and standard deviation of 0.632. In all, 46.60% of users strongly disagree that the system is very difficult to use, 46.70% of users disagree that the system is very difficult to use and 6.70% of users are neutral about whether the system is very difficult to use.

Q9: I think I am very confident of using the system. This is a straightforward question with an average number of 4.33 and standard deviation of 0.816. In all, 53.30% of users strongly agree that they are very confident about using the system, 26.70% of users agree that they are very confident about using the system and 20% of users are neutral about whether they are very confident about using the system.

Q10: I think I have to learn something to use the system. This is a roundabout question with an average number of 3.87 and standard deviation of 1.125. In all, 40% of users strongly agree that they have to learn something to use the system, 20% of users agree that they have to learn something to use the system, 26.70% of users are neutral about whether they have to learn something to use the system and 40% of users disagree that they have to learn something to use the system. The researcher arrived at a conclusion that he/she does not know about affective computing and needs the help of technicians; he/she therefore thinks he/she needs to learn relevant knowledge before using the system.
