*3.7. Implementation Details*

We used scikit-learn (v0.18, [48]) to establish the best predictive model for the individual. Scikit-learn is an open-source Python module integrating a wide range of machine learning algorithms. Scikit learn was integrated in Anaconda (v4.2.13, [49]) and Jupyter Notebooks (4.0.6, [50]) was used in combination with Python (v3.5.2, [49]) for creating the data processing and machine learning pipeline. Jupyter Notebooks is an interactive method to write and run various programming languages, such as Python. The participants, their physical activity data, and the results of the performance of the algorithms and models were saved in an Oracle database (v11g2 XE; [51]). The Oracle database management system is a widely-used SQL-based system for persisting data. The source code and corresponding notebooks of the machine learning procedure is available as open-source software on Github (https://github.com/compsy/personalized-coaching-ml).

For the Web application, we used Flask (Version 0.10.1, [52]), a Python-based Web application microframework for developing Web applications. We used a PostgreSQL database to store information regarding the models and the participants. The machine learning models resulting from the pipeline are exported as Python Pickle files, which were imported into the Web application. The infrastructure-as-a-service provider Heroku is used to host a demo version of the Web application. This Web application is available at http://personalized-coaching.compsy.nl. The Web application is available as open-source software on Github (https://github.com/compsy/personalized-coaching-app). *Sensors* **2018**, *18*, 623 7 of 16

#### **4. Results** the pipeline are exported as Python Pickle files, which were imported into the Web application. The

After optimizing our machine learning models by applying grid search in combination with cross-validation, we assessed the models using the test set. The results are presented here. infrastructure-as-a-service provider Heroku is used to host a demo version of the Web application. This Web application is available at http://personalized-coaching.compsy.nl. The Web application is available as open-source software on Github (https://github.com/compsy/personalized-coaching-app). *Sensors* **2018**, *18*, 623 7 of 16 the pipeline are exported as Python Pickle files, which were imported into the Web application. The

infrastructure-as-a-service provider Heroku is used to host a demo version of the Web application.

#### *4.1. Accuracy and F1-Score on Group Level* **4. Results** This Web application is available at http://personalized-coaching.compsy.nl. The Web application is

Table 2 presents the F1-score and accuracy of the eight different algorithms at the group level. The top three group algorithms based on the mean accuracy and F1-score are: After optimizing our machine learning models by applying grid search in combination with cross-validation, we assessed the models using the test set. The results are presented here. available as open-source software on Github (https://github.com/compsy/personalized-coaching-app). **4. Results**


**Table 2.** Algorithms and their scores for the whole dataset. *4.1. Accuracy and F1-Score on Group Level* Table 2 presents the F1-score and accuracy of the eight different algorithms at the group level. After optimizing our machine learning models by applying grid search in combination with cross-validation, we assessed the models using the test set. The results are presented here.

AdaBoost, Neural Networking, and Support Vector Classifier. Random Forest (RF) 0.737451 (0.003210) 0.819065 (0.003840) 7 KNeighborsClassifier (KNN) 0.749171 (0.005683) 0.829826 (0.005544) 5

Finally, the plus sign indicates outliers on either side of both whiskers.

We visualized the accuracy and F1-score per algorithm using boxplots in Figures 1 and 2. The box represents the second and third quartile groups and the red line indicates the median. The upper whisker visualizes the fourth quartile group and the lower whisker visualizes the first quartile group. Finally, the plus sign indicates outliers on either side of both whiskers. Decision Tree (DT) 0.720535 (0.004787) 0.804220 (0.003006) 8 We visualized the accuracy and F1-score per algorithm using boxplots in Figures 1 and 2. The box represents the second and third quartile groups and the red line indicates the median. The upper whisker visualizes the fourth quartile group and the lower whisker visualizes the first quartile group. Logistic Regression (LR) 0.742125 (0.009821) 0.825725 (0.008487) 6 Random Forest (RF) 0.737451 (0.003210) 0.819065 (0.003840) 7 Decision Tree (DT) 0.720535 (0.004787) 0.804220 (0.003006) 8 We visualized the accuracy and F1-score per algorithm using boxplots in Figures 1 and 2. The box represents the second and third quartile groups and the red line indicates the median. The upper

Logistic Regression (LR) 0.742125 (0.009821) 0.825725 (0.008487) 6

Stochastic Gradient Descent (SGD) 0.767623 (0.005490) 0.853575 (0.004574) 4

**Figure 1.** Algorithm accuracy comparison. **Figure 1.** Algorithm accuracy comparison. **Figure 1.** Algorithm accuracy comparison.


**Figure 2.** Algorithm F1-score comparison. **Figure 2. Figure 2.** Algorithm F1-score comparison. Algorithm F1-score comparison.
