*4.3. Data Compilation*

The data collection was performed without interacting with the subjects during the experiment (i.e., an indirect method). The online questionnaire designed for the survey includes, in addition to the consent form, several questions to determine the initial status of participants as well as to compile the students' opinions after the test. They were asked about their expertise level creating software programs with visual languages and with text-based programming languages. Regarding the post-test, some questions related to the perceived ease of use of App Inventor and VEDILS were included. They deal with the tool usability for (i) connecting/disconnecting via Bluetooth and sending commands to the IoT sensor; (ii) consuming temperature data from the sensor, applying a transformation for changing their measurement scale, removing the outliers and computing the average; and (iii) drawing a chart with the temperature evolution. The answers to these questions follow a five-level Likert scale (1-Strongly disagree, 2-Disagree, 3-Neither agree nor disagree, 4-Agree and 5-Strongly agree). The participants were also asked about their intention to use App Inventor or VEDILS to create more IoT projects. Study data as well as the resources used are linked in the Supplementary Materials.

#### *4.4. Analysis and Findings*

This study aimed at checking whether it is easier for students to develop the proposed IoT mobile app using VEDILS rather than with App Inventor. Ten students (eight men and two women) aged 24 (stddev = 2) participated in the study. All of them were first-year students of a vocational course in web development. By the time the workshop was conducted, they had not ye<sup>t</sup> learned any textual programming tool. They had only studied and used the App Inventor platform. Only one of them had previous experience with traditional text-based coding languages.

All students agreed that it is easy (avg = 4.0, stddev = 0.0) to develop the routines for connecting/disconnecting via Bluetooth and sending commands to the IoT sensor with App Inventor and VEDILS (both tools share the same blocks for that purpose). Regarding the temperature data ingestion and processing steps, most students neither agree nor disagree (avg = 2.77, stddev = 0.40) that these steps are easy to develop with App Inventor. Nevertheless, most students agreed that it is easy to develop these routines with VEDILS (avg = 3.88, stddev = 0.26). This perceived ease of use is even more substantial when developing the temperature evolution chart: the App Inventor *Canvas* component (avg = 2.66, stddev = 0.44) versus the VEDILS *Chart* component (avg = 4, stddev = 0.23). Finally, regarding the question about which tool they would use to develop mobile apps that consume data from IoT sensors and depict them in a chart, 66.6% of the participants chose VEDILS, 11.1% chose App Inventor, whereas the rest did not indicate a preference. In short, all participants rated VEDILS better than App Inventor, except for the student who already had coding skills.

A total of 177 blocks were required for developing the App Inventor version of the app, whereas only 84 were required in VEDILS. In both cases, procedures were used to avoid as much as possible the number of duplicate blocks. Accordingly, the difference in the size of the projects may relate to the students' perceived ease of use for both tools. That difference is particularly pronounced when it comes to presenting the temperature chart because developers must handle many details of the drawing process. The results are also consistent with the qualitative opinions expressed by the students, who highlighted the saving of programming effort required to consume, process and visualise IoT data thanks to the abstractions provided by VEDILS.

#### **5. Evaluating VEDILS Data Processing Blocks with Academics**

While the above section evaluates the components and language extensions provided by VEDILS for IoT computing, this case study solely focuses on the data processing blocks. The main objective is to check the development agility and the usability of the stream blocks compared to the standard built-in blocks for processing data. The design, implementation and analysis of the experimental study are presented below.

## *5.1. Study Design*

The study was performed through six editions of an introductory course of mobile app development with App Inventor/VEDILS between January and February 2018. These courses are part of the Cádiz university's docent innovation program, in which several IT-related courses are regularly delivered to their associated lecturers and researchers.

The reference framework for establishing the hypotheses of this study is based on the potential benefits of certain computer programming paradigms over others [48]. Some authors explored techniques for introducing parallelism concepts, anonymous procedures and higher-order functions into block languages [12–14]. In this particular case of application development, we analyse the ease and agility of using block-based versions of the map-reduce constructs from the functional programming paradigm versus the iterative constructs (i.e., loops) from the imperative programming paradigm. The research questions posed for this study are the following: RQ1—Is there any difference in users' perception of the complexity of the stream processing blocks? RQ2—Is it easier for users to develop apps that collect and process data samples using functional blocks rather than using imperative blocks? and RQ3—Is it faster for users to develop apps that collect and process data samples using functional blocks rather than using imperative blocks?

To find answers to the research questions, the following scenario was carried out. First, all the academics interested in enrolling in the course were arbitrarily allocated in one of the (six) course editions. Each course lasted five hours and the participants were first taught with a short introduction to the educational applications of mobile devices. Next, the instructors explained the fundamentals of visual programming and the VEDILS tool's features.

Second, to reinforce and consolidate what was learned, participants created a number of educational mobile apps. These apps leverage the smartphone sensory and multimedia elements provided by App Inventor as well as the augmented reality capabilities provided by VEDILS. During the course, all the participants had to develop the same apps, except for one that emulates dice rolling. In addition to simulating the dice, in three of the course editions attendants who represented

the control groups had to include an additional routine to calculate the count of odd numbers in a sequence of dice roll samples, whereas in the other three editions, attendants who represented the experimental groups had to program the count of even numbers. For the control groups, the attendants were accordingly taught about the loop statements for data processing, whereas for the experimental groups, the participants were taught about stream blocks.

Finally, the course attendants were asked to develop a citizen science mobile app by themselves. In this vein, smartphones enable to automate data collection and enrich observations with photographs, sound recordings and global positioning system (GPS) coordinates using embedded sensors [49]. The app requirements were: (i) to simulate the input of a numerical measurement of an external phenomenon and (ii) to compute the average of the collected measurements, excluding values out of a permitted value range.

The development of the citizen science app was required to obtain the course completion certificate. The assignment delivery was due within two weeks of course completion. In addition to submitting the developed apps, an online questionnaire had to be filled out. Answers to the questionnaire were analysed using quantitative techniques.

## *5.2. Data Compilation*

A total of 45 users attended the VEDILS course. Data collection was performed without interacting with the subjects through an online questionnaire. The survey included questions related to the participants' knowledge area, age, gender, years of teaching and research experience, highest academic degree obtained and prior expertise in creating computer programs with a visual and/or text-based programming language. Regarding the post-test, questions related to the perceived ease of use of App Inventor and VEDILS were included. These questions pointed to several aspects, such as the use of variables and data lists, control flow statements and loop blocks (for the control groups) and the use of stream blocks (for the experimental groups). In addition, similar questions were included to check the participants' self-confidence when developing the citizen science app. The answers to all the questions were on a five-level Likert scale.

Besides, all the app project files submitted to the learning managemen<sup>t</sup> store (i.e., Moodle) for the instructor's review were subsequently processed through a data integration process for analytic purposes. Among other data, the following were automatically extracted: time spent to develop the app, the number of blocks used, number of debugs and compilations required to complete the app. Study data as well as the resources used are linked in the Supplementary Materials.

#### *5.3. Analysis and Findings*

The 45 participants (17 women and 28 men) were, on average, 41 years old, had 13 years of teaching experience and 11 years of researching experience. Furthermore, 62.22% of the academics had a Ph.D. Their background is as follows: Arts and Humanities (2.22%), Computer Science (20%), Engineering and Architecture (4.44%), Health Sciences (17.78%), Laws and Social Sciences (28.89%) and Natural Sciences (26.67%). In terms of their previous programming experience, from nothing (1) to expert (5), they had scarce visual (avg = 1.82) and textual (2.15) programming skills. Overall, 25 subjects were part of the control groups, whereas the experimental groups were composed of 20 subjects.

Tables 1 and 2 show the users' perception of the stream processing blocks complexity and the ease of development of the app created to collect and process data samples. Data are grouped in the table according to the participants' gender, academic degree, knowledge area and previous experience with visual and textual programming languages.


**Table 1.** Results of the survey with academics: perceived ease of use (the italic font shows the average and chi-squared values whereas the bold one indicates significant differences).

**Table 2.** Results of the survey with academics: ease of development of the app (the italic font shows the average and chi-squared values whereas the bold one indicates significant differences).


Concerning the participants' gender, women perceived that the stream blocks were easier to use (avg = 4.4) than for men (avg = 3.69) but interestingly enough, men were the ones who found the app development more comfortable with those blocks (man's avg = 4.08 vs. woman's avg = 3.33). Furthermore, non-doctorates found the development much easier with stream blocks (avg = 4.25) than the traditional ones (avg = 2.88). Besides, Social Sciences and Humanities (SSH) academics perceived the stream blocks easier to use (avg = 4.75) compared to the Earth & Health Sciences and Engineering (EHSE) lecturers (avg = 3.64). SSH academics also found it difficult (avg = 2.22) to develop the app with the loop blocks, whereas they did not have that much trouble with the stream ones (avg = 3.6).

It is interesting to note (*p* < 0.05) that users with previous experience in visual programming languages perceived loop blocks (avg = 4.57) easier than stream blocks (avg = 3.56). Nevertheless, there is a significant difference (*p* < 0.05) in the fact that academics without experience with visual languages developed the app easier with the map-reduce blocks (avg = 3.71) than with the standard loop blocks (avg = 2.55). As expected, there is also a significant difference (*p* < 0.05) concerning the ease of development of the proposed app with the map-reduce blocks (avg = 3.54) compared to the standard loop blocks (avg = 2.07) for academics without experience with textual languages.

Regarding the apps the lecturers had to create as final assignment of the course, 39 out 45 were correctly developed: 16 apps use the traditional loop blocks and 23 use the stream blocks. Table 3 shows the direct metrics obtained from the app projects. As can be observed, all the apps which had to be developed with stream blocks were completed. The remaining six apps were expected to be developed using the traditional loop blocks. On average, three hours were needed to develop the app with the standard loop blocks, whereas fewer than two hours were required to create the same app with the new stream blocks. That is also tested with a significant difference (*p* < 0.05). Furthermore, the average number of builds and debugs performed for the stream-based apps is fewer than for the loop-based one.


**Table 3.** Indicators of the developed apps (the italic font shows the average and Mann-Whitney U Test values whereas the bold one indicates significant differences).

To sum up, with regard to the question (RQ1), there is no difference in the users' perception of the complexity of the stream processing blocks (avg=3.89) and the loop blocks (avg = 3.72). Concerning whether it is easier for users to develop apps which collect and process data samples with functional blocks rather than with imperative blocks (RQ2), the participants agreed that the development of the requested app was easier with the map-reduce blocks (avg = 3.84) than with loop ones (avg = 2.96). Finally, the indicators obtained for RQ3 point that it is faster for users (100% completion of the projects, a fewer number of debugs required to develop the app and a significant difference (around 38%) in saving development time) to collect and process data samples with functional blocks rather than with imperative blocks.

#### **6. Discussion and Conclusions**

Developing smart user experiences based on IoT technologies is a very complicated task, especially for non-IT professionals. To address these barriers, some of the popular block-based tools aimed at learners in computer programming (e.g., Scratch or App Inventor) were extended with modules to communicate with external hardware. However, they do not provide adequate support for easily ingesting, processing and visualising data from sensors.

In this research, some components and blocks developed explicitly for a custom version of App Inventor, called VEDILS, were proposed. They are devised to facilitate the ingestion of data from sensors in time intervals, to process received data by using a pipelined sequence of mapping, filtering and reducing operations, and finally, to represent them graphically or in a tabular format.

Two studies, namely a quasi-experimental study conducted with students and an experimental with academics, were conducted to evaluate the contribution. From the first study, students considered that it was easier for them to develop the routines for ingesting, processing and visualising data from the external temperature sensor with the VEDILS IoT features rather than with the equivalent components and blocks in App Inventor. From the second study, aimed at only checking the data processing blocks, participants did not perceive stream blocks easier than the loop blocks. Nevertheless, with statistical significance, it was faster for academics to develop the proposed app with the stream blocks, and easier specifically for novice programmers.

Additionally, threats to validity must be taken into account. To maximise the internal validity and the construct validity, we maintained a detailed protocol for both studies. Peer researchers reviewed them, and actions were considered to minimise bias. In the first study, the students completed the app development projects for both App Inventor and VEDILS but in reverse order to minimise the learning effect on the subjects. In the study conducted with academics, they were randomly distributed into different groups. In addition, every course edition was taught with the same instructors (also the authors of this paper). Furthermore, all course attendants were required to develop the same app with the same requirements to ensure the count of minutes spent, debugs and builds required to create the apps were not affected by other factors. Apart from the data automatically extracted from the developed projects (time spent, number of builds and number of debugs), the rest of the variables used for our experiments to measure user perceptions are subjective so that they can also be considered as validity threats.

The limited size of the student sample can be viewed as an external validity threat. Moreover, although the second study has a user sample more extensive than the first one, it is only aimed at academics. As a result, we cannot assure that the obtained findings can be generalised to professionals of other disciplines or conventional users. Hence, more experimentation and analysis are required to evaluate to what extent the findings presented in this work are of relevance for other cases.

It is necessary to consider the limitations of the current work. First, since the new type of language block for providing app developers with a data stream according to a predefined period of time was only incorporated for the standard *BluetoothClient* component and the *BrainwaveSensor* of VEDILS, it is not currently possible to harness it for other built-in App Inventor sensors. In addition, the extension component for using Bluetooth Low Energy (BLE) technology, which is not part of the App Inventor main distribution, is not ye<sup>t</sup> supported for our contribution. Second, the current implementation of the components for visualising data do not allow developers to customise colours, lines widths, font sizes, etc., which are format aspects usually required when designing charts.

Smart homes and buildings, smart cities, mobility and transportation, healthcare, agriculture and industry are some of the main areas of IoT application [1]. The study conducted with students illustrated the potential application of our approach to smart buildings, e.g., for monitoring room temperature. Nevertheless, the contribution presented in the paper is expected to be useful to create IoT applications for other areas. Thus, for example, non-expert programmers (researchers, patients and healthcare professionals) will be able to develop apps for wellness and healthcare purposes without struggling with the complexities of the common mobile programming languages, namely Java or Swift. These kinds of apps are usually data-intensive and require to process users' biometric data, which is ingested from wearable devices, such as smart bands or chest straps, among others.

This research tried to investigate whether the components and extensions presented in the paper contribute to the popularisation of IoT-based mobile app development. In this vein, EUD platforms and, in particular, enriched block-based authoring tools as VEDILS, can simplify development tasks of novice end-user programmers. Furthermore, according to the obtained results, the use of blocks

based on the map-reduce paradigm from functional programming streamlines the development of data processing functions in IoT consumer apps, although more experimentation is required. As future work, we plan to support the BLE extension for App Inventor to improve the customising features of the *Chart* and *DataTable* components.

**Supplementary Materials:** The software for authoring IoT apps with data processing and visualising capabilities can be used through the VEDILS web site http://vedils.uca.es/. Besides, to guarantee the reproducibility of the studies, all the resources developed are available online at http://www.mdpi.com/1424-8220/19/24/5467/s1. This link provides a ZIP package containing: questionnaires and results of both studies, PDF presentations (in Spanish) for the course with academics and the workshop with students and solutions for the different exercises and tutorials.

**Author Contributions:** Conceptualization, I.R.-R.; methodology, I.R.-R. and J.M.D.; software, T.P. and I.R.-R.; validation, J.M.M.; formal analysis, I.R.-R. and J.M.M.; investigation, I.R.-R., J.M.M., and J.M.R.C.; resources, I.R.-R. and J.M.D.; data curation, I.R.-R. and J.M.M.; writing—original draft preparation, I.R.-R., J.M.M. and J.M.R.C.; writing—review and editing, I.R.-R. and J.M.D.; visualization, I.R. and J.M.M.; supervision, I.R.-R.; project administration, J.M.D.; funding acquisition, J.M.D.

**Funding:** This work was developed in the VISAIGLE project, funded by the Spanish National Research Agency (AEI) with ERDF funds under gran<sup>t</sup> ref. TIN2017-85797-R.

**Conflicts of Interest:** The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.
