**3. Results**

In order to assess the predictive potential of eConsulta messages regarding the three variables of interest, we first aimed to identify the best combination of algorithms. A total of 3559 messages (169,102 words) corresponding to 2268 teleconsultations (1.57 messages per teleconsultation) were analysed in a framework of 20 di fferent combinations of vector representation of text and machine learning algorithms (Table 3). We assessed the performance of the combinations of algorithms though a stratified 10-fold cross-validation analysis. Figure 1 shows the performance of the most stable algorithm (best metrics, in general) according to the predictor variable.



**Figure 1.** Performance metrics of algorithms.

Specific combinations of algorithms per variable generally perform very well. Table 4 shows the evaluation metrics (mean + standard deviation of the 10 iterations) of the combination of algorithm and numerical representation of the text which has a better performance for each target variable. For all of the cases, the vectors obtained directly from the original texts have been more useful than those enriched with external texts. Table 4 shows that algorithms are generally e ffective, showing they are better when approximating the two binary variables (avoiding the need for a face-to-face visit, increased demand) than the variable "type of query". Thus, eConsulta's classifiers have a promising and robust predictive value, especially for binary variables.


**Table 4.** Results of the best algorithm/text representation combination, according to the variable to be approximated. Average (SD) of the 10 iterations.

As a whole, the results illustrate eConsulta's algorithm classifiers potential predictive value and provide a valuable insight into the implementation of AI methodologies for healthcare teleconsultation.
