A Granularity-Based Intelligent Tutoring System for Zooarchaeology

Featured Application

This study creates an intelligent tutoring system in archaeology for helping students in specialized tasks that require analysis of huge amounts of data. The method proposed here implies the application of classification algorithms that must be trained with a complete data set in order to give accurate results. We have tested our method by developing an intelligent tutoring system in the field of zooarchaeology.

Abstract

This paper presents a tutoring system which uses three different granularities for helping students to classify animals from bone fragments in zooarchaeology. The 3406 bone remains, which have 64 attributes, were obtained from the excavation of the Middle Palaeolithic site of El Salt (Alicante, Spain). The coarse granularity performs a five-class prediction, the medium a twelve-class prediction, and the fine a fifteen-class prediction. In the coarse granularity, the results show that the first 10 most relevant attributes for classification are width, bone, thickness, length, bone fragment, anatomical group, long bone circumference, X, Y, and Z. Based on those results, a user-friendly interface of the tutor has been built in order to train archaeology students to classify new remains using the coarse granularity. A pilot has been performed in the 2019 excavation season in Abric del Pastor (Alicante, Spain), where the automatic tutoring system was used by students to classify 51 new remains. The pilot experience demonstrated the usefulness of the tutoring system both for students when facing their first classification activities and also for seniors since the tutoring system gives them valuable clues for helping in difficult classification problems.

1. Introduction

The use of technology opens new frontiers in learning and improves data mining from different sources in order to improve students’ learning processes [1]. One of the biggest challenges of including communication technologies in learning is the way in which interaction between teachers and students is simulated by automatic methods [2,3]. In this context, tutoring and the possibility of turning an automatic system into an effective instrument for counseling students are stirring a great amount of interest in many areas [4,5]. The interest in this concept is demonstrated by the many initiatives related to intelligent tutoring systems (ITS). For example, Graesser et al. [6] incorporated natural language processing (NLP) to the tutor in order to enhance the student’s learning process. In other studies, authors used gamification techniques to enhance the student’s learning process [7,8] or integrate affect sensors [9]. Other more recent ITS guided students to choose their learning path according to their profile [10].

The use of ITS in archaeology is a very interesting topic because of the great amount of data that is available for analysis. In this discipline, data analysis has been performed by many statistical methods [11], including artificial intelligence (AI) in both supervised and unsupervised forms [12,13,14]. In [15], authors focused on computer vision and gave a visual analysis in archaeology. Random forest [16] has also been used as a supervised learning method to classify archaeological remains. There have also been initiatives [17] which use artificial neural networks in the archaeological domain claiming that they are suitable for the sparse data commonly found in this discipline. Augmented reality and serious games have also been used as tools for improving learning in related subjects such as history or archaeology [18,19]. Virtual reconstructions of ancient cities have also been developed in many different formats [20,21].

Unfortunately, there is a lack of references of ITS in more technical tasks that could help students in more specialized techniques, such as zooarchaeology, which is the topic included in this article. In archaeological sites, there are many different specialized tasks that must be performed by well-trained students [22]. In the case of zooarchaeology, the identification and characterization of animals or species from bone fragments is a crucial task that requires the analysis of many parameters obtained from the characteristics of the remains [16]. First, students require good training in the taxonomical and anatomical characteristics. Later, they must identify the animals and all the information related with their use by humans (trapping, butchering process, discard, technological use, etc.). This data analysis is a great effort and an ITS would be of great help. Moreover, implementing and defining an effective protocol for implementing an intelligent tutor in bone fragment characterization will be also useful in other similar disciplines that also need the analysis of huge amounts of data in archaeology such as the identification of constructive styles [23] or reconstruction of ceramics from small fragments [24].

In this article, we show the steps, for creating an ITS, that have been developed from the application of AI classification algorithms to a data set that includes bone fragments correctly identified in the archaeological site of El Salt. A comparison between different granularities and algorithms is also shown. By choosing the most effective combination, we have developed a user-friendly interface of the ITS with the objective of testing it in the 2019 season in the archaeological site. Finally, we show the results and conclusions of the pilot experience in the archaeological site during August 2019.

This article is structured as follows: Section 2 explains the materials and methods used in the study, Section 3 presents the results, Section 4 discusses the results, and finally Section 5 details the conclusions and future work.

2. Materials and Methods

This section is divided into materials, where the dataset of the study is explained, and methods, where the followed methodology to build the tutoring system and to evaluate it is explained.

2.1. Materials

The used dataset had 3406 instances of archaeological remains and 64 attributes (the last one is the predicted class) extracted from faunal assemblage of Stratigraphic Unit Xa of El Salt (Figure 1 and Figure 2) [25,26,27]. The excavation of “El Salt” can be seen in Figure 1 and examples of the archaeological remains of “El Salt” can be seen in Figure 2 and Figure 3.

The Salt is one of the fundamental sites of the Middle Palaeolithic in the western Mediterranean region due to the entity of its archaeological sequence, for its extraordinary state of conservation, which affects even organic matter and for the integrated and multidisciplinary nature of the research process which takes place in it, with the confluence of different Hispanic, European, and North American Universities and Research Centers. This enclave has been located at the head of the Serpis River, in the locality of Alcoy (Alicante, Spain). It is presented as an outdoor site of about 300 m² located at the foot of a large wall that rises up to 38 m tall. The space inhabited at the foot of the wall was protected by a large roof, which at times of maximum development came to serve as protection for almost the entire surface. Highlights are the strategic condition of its location, in the middle of various biotopes of plain, mountains, river valley, and lake-palm environment, etc. and immersed in a territory of the mountains of Alicante, very rich in diversified resources.

Research at the El Salt site attempts to deepen the knowledge of its paleolithic record, from an integrative and multidisciplinary perspective. This record does not stop growing by virtue of the application of increasingly sophisticated excavation techniques and high resolution analytical procedures. To the traditional material record of this type of deposits, consisting of lithic remains, fauna, anthracological material, etc. Currently, a microscopic and even molecular registry is being added that is decisively contributing to enrich the information and improve its quality. These facts give us the opportunity of obtaining a complete and accurate data-set in zooarchaeology, among others, which is a key step for training classification algorithms as the ones needed for developing the ITS presented here.

In Figure 2, we can observe taphonomic damage and bone remains recovered examples. In Figure 3, we can observe bioestratinomic damage examples.

In

Table 1. Exploratory data analysis of the dataset.

Attribute	Description
Width	Count: 2036, Mean: 14.822829, Std: 8.973862, Min: 0.99, Q1: 8.457500, Q2: 13.43, Q3: 19.32, Max: 71.01
Bone: Ca, T, Cx, F, H, U, Fa1, Mtp, Mt2, Mc4, Mt4, Cr, Vl, Vcd, Mc2, Ta, Hem, I/1, Fa, Mt3, Vc, I1/, Mx, As, Pa, R, Es, Fa2, Fa3, Mt5, nan, Mc5, Ct, Vs, Vt, Mc3, V, Das, Dai, Da, Mt, M3/, M1/, Hy, Mc, P2/, P4/-M1/, M/1, Lt2, Art, Cc, PT, M2/, In, M/P, M/1-M/2, P4/, M3, M1/-M2/, P/2, I, M/X, Pl, P/M, R/, Co, L, Lt1, Lt3, P/4, P/3-P/4, Se, dp2/, P3/, M/x, P4/-P3/, M/3, PTPer8, PTPle, PTPL, P/3, M/, M/2, M, Fa1-V, Fa2-II, Fa2-IV, I/C, Fi, Asta, I/3, d/3, Mc2-3, I/2	Count: 3403, Unique: 93, Top: T, Freq: 374
Thickness	Count: 714, Mean: 4.887451, Std: 2.873843, Min: 0.460000, Q1: 3.072500, Q2: 4.595000, Q3: 6.4, Max: 16.52
Length	Count: 2492, Mean: 36.25859, Std: 28.28524, Min: 2.42000, Q1: 15.94500, Q2: 27.20000, Q3: 47.80000, Max: 284.96000
Bone fragment	Count: 3242, Mean: 189.635102, Std: 204.909877, Min: 1, Q1: 50, Q2: 111, Q3: 500, Max: 555
Anatomical group: Mp, Ma, E, C, A, nan, In, ES, Cr, PT, PTPL, PTPl	Count: 3399, Unique: 11, Top: Mp, Freq: 1275
Long bone circumference	Count: 1378, Mean: 2.526851, Std: 1.107021, Min: 1, Q1: 2, Q2: 2, Q3: 4, Max: 5
X	Count: 1.187000 × 103, Mean: 1.864528 × 103, Std: 6.292228 × 104, Min: 0, Q1: 8.519905, Q2: 2.900000 × 101, Q3: 6.775000 × 101, Max: 2.167891 × 106
Y	Count: 1186, Mean: 1272.859123, Std: 30223.002072, Min: 0, Q1: 11.809257, Q2: 24.500000, Q3: 63, Max: 803982
Z	Count: 1232, Mean: 403.047662, Std: 9767.169421, Min: −143.8, Q1: −0.156130, Q2: 166.2, Q3: 192.6, Max: 342938
Manganeso	Count: 1416, Mean: 1.592514, Std: 0.835553, Min: 1, Q1: 1, Q2: 1, Q3: 2, Max: 5
Concretion	Count: 1177, Mean: 1.806287, Std: 1.033386, Min: 1, Q1: 1, Q2: 1, Q3: 2, Max: 5
Coarse granularity family: Bovidae, Cervidae, Equidae, Leporidae, unknown	Bovidae: 420 Cervidae: 516, Equidae: 240, Leporidae: 2164, Unknown: 66, Total: 3406
Medium granularity family: Bovidae, Canidae, Cervidae, Corvidae, Equidae, Felidae, Leporidae, Phasianidae, Rhinocerotidae, Suidae, Testudinidae, unknown	Bovidae: 420 Canidae: 8, Cervidae: 516, Corvidae: 5, Equidae: 240, Felidae: 24, Leporidae: 2164, Phasianidae: 7, Rhinocerotidae: 2, Suidae: 7, Testudinidae: 9, Unknown: 4, Total: 3406
Fine granularity family: Anatidae, Bovidae, Bufonidae, Canidae, Cervidae, Corvidae, Equidae, Erinaceidae, Felidae, Leporidae, Phasianidae, Rhinocerotidae, Suidae, Testudinidae, Ursidae	Anatidae: 1 Bovidae: 420, Bufonidae: 1, Canidae: 8, Cervidae: 516, Corvidae: 5, Equidae: 240, Erinaceidae: 1, Felidae: 24, Leporidae: 2164, Phasianidae: 7, Rhinocerotidae: 2, Suidae: 7, Testudinidae: 9, Ursidae: 1, Total: 3406

, a descriptive analysis of the sample is shown. There are two columns in the table. In the first one, we have listed the most relevant parameters (according to the classification algorithm with best performance in the coarse, medium, and fine granularities, which will be explained in the following section) and the predicted class ‘Family’ (taxonomic rank between Order and Genus) both in coarse, medium, and fine granularities. Following the name of each parameter, we have included a description of the possible values that it can take. For instance, “Bone” can take the listed 93 unique values, while the anatomical group can take only 11 values. It is worth noting that, in the case of the prediction classes, the coarse granularity can take five values, the medium granularity 12, and the fine granularity 15. The second column of the table describes briefly the attribute. If the attribute is numeric, count (number of not null values), mean, standard deviation (std), minimum (min), first quartile (Q1), second quartile (Q2), third quartile (Q3), and maximum (max) are given. If the attribute is categorical, count (number of not null values), unique (number of categories), top (top category), and frequency (freq, which is the frequency of the top category) are given. In the categorical granularity families, the number of each category is given because it is relevant for the predicting classifier that will be explained in Section 2.2.

2.2. Methods

We used 33% of the remains for testing and 67% for training. A pipeline with a standard scaler with a SMOTE (synthetic minority over-sampling technique) or ADASYN (adaptive synthetic) method was used, together with 10-fold cross-validation. The parameter that we want to maximize is f1-score_macro. We have chosen it because F1-score is the harmonic mean of precision and recall, and the macro option calculates metrics for each label, and finds their unweighted mean, and this does not take label imbalance into account. The used algorithms with their parameters have been:

• Random forest with parameters:

  -

  classifier__n_estimators (number of trees in the forest): [100, 500, 700]

  -

  classifier__max_features (number of features to consider when looking for the best split): [‘auto’, ‘sqrt’, ‘log2’]

  -

  classifier__class_weight (weights associated with classes; if not given, all classes are supposed to have weight one; the balanced mode uses the values to automatically adjust weights inversely proportional to class frequencies in the input data): [‘balanced’,None]

• Support vector machine (SVM) with parameters:

  -

  classifier__kernel (specifies the kernel type to be used in the algorithm): [‘linear’,’rbf’]

  -

  classifier__gamma (kernel coefficient for ‘rbf’, ‘poly’, and ‘sigmoid’): [1 × 10⁻³, 1 × 10⁻⁴]

  -

  classifier__C (penalty parameter C of the error term): [1, 10, 100]

• Naive bayes.
• Neural networks with parameters:

  -

  classifier__solver (the solver for weight optimization. ‘lbfgs’ is an optimizer in the family of quasi-Newton methods. ‘sgd’ refers to stochastic gradient descent. ‘adam’ refers to a stochastic gradient-based optimizer): [‘lbfgs’, ‘sgd’, ‘adam’]

  -

  classifier__alpha (L2 penalty (regularization term) parameter): [1 × 10⁻⁴, 1 × 10⁻⁵]

• k-nearest neighbors (KNN) with parameters:

  -

  classifier__n_neighbors (number of neighbors to use): [3, 5, 7, 9]

More information about the algorithms can be found in the literature [28]. These algorithms have been implemented using the Python library scikit-learn and the ITS has been implemented using Flask.

Regarding the criteria used to distinguish the different taphonomic alterations, zooarchaeological and taphonomic analysis are performed using already established standard methods [29,30]. Faunal remains are taxonomically and anatomically identified whenever possible, while non-identified specimens with insufficient information were classified into three bone categories (long, flat, or articular) and associated with a weight-size category based on bone density, circumference, and thickness of the cortical surface: large-sized >300 kg, medium-sized 100–300 kg, small-sized 5–100 kg, and very small-sized <5 kg [31,32,33]. On the other hand, all remains were analysed taphonomically using macroscopic and microscopic techniques to identify biostratinomic and diagenetic modifications. In the fracture analysis, we classified all of the fragments by fracture type (recent, old fresh, old dry, or indeterminate) following the criteria established by [34] and the morphotypes created by Real [35,36]. All dimensions were measured for each bone fragment. Also, bone surface modifications were observed and quantified to identify different damage caused by anthropogenic activity (percussion, butchering marks, or thermal alteration) or predator action (tooth marks, or digestion), as well as the diverse diagenetic processes that produce alterations on bone surfaces as erosion, sediment concreteness, roots marks, weathering, pigmentation, and trampling [37,38,39,40,41,42,43,44]. Damage caused after the prey’s death and previously their deposit and sedimentation, the biostratinomic process, was classified and analysed individually in a specific database according to the type of damage, origin, agent, location, morphology, distribution, direction, intensity, quantity, and dimensions. Diagenetic processes were recorded based on their presence and the degree of alteration. In the specific case of butchering marks, different activities were established based on the ethnoarchaeological literature [37,45,46,47].

The questionnaire used to test the experience of the tutor in the pilot of El Salt 2019 season followed partially the methodology explained in [48]. The following areas were tested:

• Constructive/active learning: The tutor stimulated us to understand underlying mechanisms/theories.
• Self-directed learning: The tutor stimulated us to search for various resources by ourselves.
• Contextual learning: The tutor stimulated us to apply knowledge to the discussed problem.
• Global score: Overall performance of the tutor.
• Open answer: Give some tips for improvement.

A rating between 1 (strongly disagree) and 5 (strongly agree) was given to questions 1–4.

3. Results

Results are divided into the experiments for the prediction of animals in the coarse, medium and fine granularities, and the ITS development and use. The performance of the different methods in the coarse granularity can be seen in

Table 2. Performance of the different methods in the coarse granularity. Synthetic minority over-sampling technique (SMOTE), support vector machine (SVM), k-nearest neighbors (KNN), adaptive synthetic (ADASYN).

Method (Parameters)	Accuracy	Precision (Weighted)	Recall (Weighted)	F1-Score (Weighted)
Random forest, SMOTE (classifier class weight: balanced, classifier max features: auto, classifier n estimators: 500)	0.86	0.86	0.86	0.86
SVM, SMOTE (classifier C: 100, classifier gamma: 0.001, classifier kernel: rbf)	0.74	0.81	0.74	0.77
Naive Bayes, SMOTE	0.68	0.76	0.68	0.66
Neural Networks, SMOTE (classifier solver: lbfgs, classifier alpha: 1 × 10−5)	0.67	0.75	0.67	0.71
KNN, SMOTE (classifier n neighbors: 3)	0.75	0.79	0.75	0.77
Random Forest, ADASYN (classifier class weight: balanced, classifier max features: auto, classifier n estimators: 100)	0.86	0.85	0.86	0.86
SVM, ADASYN (classifier C: 100, classifier gamma: 0.001, classifier kernel: rbf)	0.72	0.80	0.72	0.75
Naive Bayes, ADASYN	0.68	0.78	0.68	0.65
Neural Networks, ADASYN (classifier solver: adam, classifier alpha: 1 × 10−5)	0.67	0.79	0.67	0.72
KNN, ADASYN (classifier n neighbors: 3)	0.74	0.79	0.74	0.76

, and the confusion matrices of the coarse, medium, and fine granularities using the random forest method can be seen in Figure 4, Figure 5 and Figure 6. In the fine granularity (Figure 6), classes with less than three instances do not appear because a stratified split between training and test has been applied.

In the three granularities, the 10 more relevant attributes for the classification have been (by this order):

• Coarse granularity: width, bone, thickness, length, bone fragment, anatomical group, long bone circumference, X, Y, and Z.
• Medium granularity: bone, width, length, anatomic group, bone fragment, Y, X, Z, thickness and manganese.
• Fine granularity: width, bone, length, bone fragment, anatomic group, Z, Y, X, thickness and concretion.

The best configuration parameters for the random forest were:

• Coarse granularity: ‘classifier__max_features’: ‘auto’, ‘classifier__class_weight’: ‘balanced’, ‘classifier__n_estimators’: 500.
• Medium granularity: ‘classifier__max_features’: ‘sqrt’, ‘classifier__class_weight’: ‘balanced’, ‘classifier__n_estimators’: 700.
• Fine granularity: ‘classifier__max_features’: ‘auto’, ‘classifier__class_weight’: ‘balanced’, ‘classifier__n_estimators’: 700

In Figure 7, we show the interface of the ITS implemented for being used in the season. As we can see in the interface, there is a first region where the user must be identified. Right below this first section, the user should start filling the different characteristics and parameters that have been obtained from the bone fragment. The ITS also gives information about the different options of a parameter when applicable. This is very important since the ITS could be used by archaeologists from different sites, and we must be sure that they agree in the format of the data.

In Figure 8, we show the output screen of the ITS right after a user has finished the characterization of a fragment and decides to ask for a prediction. In this case, the prediction is shown at the beginning of the interface. As soon as the prediction is done, the ITS is ready for another prediction. For this reason, the fields below the prediction do not have any value. It is worth noting that all the information filled by the user has been stored in the ITS server and is available for users upon request. Figure 8 is an example of a table used by students in the archaeological site to compare their own predictions and the ITS answers. We have censored information about the fragment identification and the identity of the students.

This first version of the ITS included the coarse granularity and was used in August 2019 by 3 students to classify 51 remains that were characterized following the format of the same database but belonged to a different archaeological site called “Abric del Pastor”. Abric del Pastor belongs also to Middle Palaeolithic and is near “El Salt”. Two of the three students also answered the questionnaire. The results of the tutor, compared to the predictions of the students frequently agreed even when the remains used for training corresponded to a different site (“El Salt”). Students, due to their limited knowledge and the difficulties of analyzing very fragmented bone remains, usually described the remains based only on the size, which usually agree with the families predicted by the tutor. For example, a lot of bone remains predicted as “small size” are described by the tutor as Bovidae, which is a family included in the category of small animals because most bovids belong to the wild goat (Capra pyrenaica). This information can help students to give more accurate answers or verify their initial guess. On the other side, sometimes the tutor fails and not all the sizes and families are correctly classified. For example, some bones that are correctly described as “small size” by students are characterized as Leporidae by the tutor, which is a mistake. Other mistakes are related to the wrong characterization of birds as Leporidae.

Analyzing the results of the questionnaire, students appreciated the use of the tutor as a complementary tool in their activities. In the subjective open answer, they highlighted two concepts: (1) The need of improving the interface and (2) the need of increasing the accuracy of the predictions.

4. Discussion

Random forests using SMOTE is the best machine learning method for this domain. In

Table 3. F1-score of the random forest in the different granularities.

Animal	Coarse	Medium	Fine
Bovidae	0.57	0.59	0.56
Cervidae	0.69	0.69	0.69
Equidae	0.68	0.63	0.66
Leporidae	0.97	0.94	0.96
Unknown	0.50	0	-
Canidae	-	0	0
Corvidae	-	0	0
Felidae	-	0.22	0.5
Phasianidae	-	0	0
Rhinocerotidae	-	0	0
Suidae	-	1	0
Testudinidae	-	1	1
Anatidae	-	-	0
Bufonidae	-	-	0
Erinaceidae	-	-	0
Ursidae	-	-	0
Total weighted	0.86	0.83	0.85

the F1-score of the different animals is displayed. We can see that coarse and fine granularities have the best F1-scores (0.86 and 0.85 respectively). However, the fine granularity is more suitable because it has more granularity and it classifies better small animals.

Let us now analyze the feedback obtained from the archaeological students and teachers. From the pedagogical point of view, and analyzing their answers, we can see that the ITS moderately stimulates underlying mechanisms but, on the other hand, it strongly stimulates the search for resources by the users. There are a few different opinions about ITS’s ability to stimulate the application of knowledge related to the discussed problem, but the overall performance of the ITS is quite high. Finally, the final tips for improvement were to improve the interface and to have more data in order to have a more accurate prediction.

Although predictions given by the ITS are usually accurate and useful, the presence of some mistakes must be analyzed. There are two ways of improving it. First, the excessive number of characterization as Leporidae can be related to the granularity used in this version. An increment in the number of classes would increase the accuracy of the prediction by increasing the amount of information in the system. The second way of improving the accuracy of the tutor is taking into account the spatial distribution of the fragments in the sites. Since the X, Y, and Z variables are important in the characterization, using information from different archaeological sites in the training would help in the correct characterization of the tutor by avoiding an excessive importance of the bone distribution in a specific site.

5. Conclusions

In this article we have presented a method to create intelligent tutoring systems in archaeology to help students in specialized tasks that require analysis of huge amounts of data. The method proposed here implies the application of classification algorithms that must be trained with a complete data set in order to give accurate results. We have tested our method by developing an intelligent tutoring system in the field of zooarchaeology. We have found that random forest is the method with the best performance in this domain, and the fine granularity approach is the more useful approach for archaeologists because the main doubts appear with small animals, which are not common. In the coarse granularity, random forest was the method which had better results between the tested ones, with an accuracy and f1-score of 0.86 for both. As questionnaires to classify the remains are quite long (63 attributes), the first 10 more relevant attributes for the classification have been shown, which are width, bone, thickness, length, bone fragment, anatomical group, long bone circumference, X, Y, and Z. In addition, a classification using a medium and fine granularity has also been performed obtaining an accuracy of 0.84 and 0.86 and a f1-score of 0.83 and 0.85 respectively.

Regarding the pilot with the tutor, it correctly classified the families in most of the cases. However, some mistakes were also found. Mistakes can be partially corrected by changing the granularity since some of the mistakes are related to the reduced amount of classes in the granularity implemented in the pilot experience. The satisfaction questionnaire showed that the tutor was helpful. Students suggested an improvement in the interface and the accuracy of the predictions.

In future work, more data from different archaeological sites can be included in the training phase in order to increase accuracy and avoid effects related to specific characteristics of a single excavation. Also, natural language processing (NLP) and gamification techniques could be added to the tutor in order to enhance the student’s learning process.