Networked Symphony Orchestra in Internet of Things Courses ^†

Abstract

Internet of Things (IoT) education is hindered by a deficiency of dynamic and interactive courses, in addition to a lack of components and difficulty in device configuration. These difficulties diminish students’ enthusiasm for IoT initiatives and reduce their drive and involvement. We designed and constructed a networked symphony orchestra using the Lego Mindstorms EV3 package as a project belonging to the IoT subject. Lego Mindstorms EV3 was selected due to its easy configuration. In this study, the knowledge obtained during the subject was utilized. In IoT courses at the University of Guayaquil, there is strong encouragement to apply the studied material to new initiatives. Through the design, the assessment of multiple technologies, and the final implementation of the project described within this paper, students were motivated for the practical application of concepts related to IoT.

1. Introduction

Objects with microprocessors and sensors (such as sensors for motion and/or sound) make up the Internet of Things (IoT) [1]. However, the offer of theoretical rather than practical courses discourage undergraduate students’ interest in IoT. A survey that examined undergraduate IoT courses at Chinese universities was recently released [2]. The majority of the IoT engineering programs were disorganized and lacked practical platforms, making students less motivated and involved in IoT activities. It is important to improve this situation because the practical application of IoT can raise students’ degree of creativity and learning pleasure [3].

Many music compositions are offered as pieces for multiple voices and instruments. It takes years of study to fully comprehend the musical expression, intonation, nuance, and other aspects that the composer notes in scores. The literature offers applications of IoT such as a band playing various musical instruments in a project “the NeXT blues” [4], “CalArts KarmetiK Machine Orchestra” of the California Institute of Arts [5], or ViolinTalk: Violin Robots as IoT devices [6]. These projects are oriented to play sound on real instruments, thus requiring a high budget. In this study, a solution with an affordable budget was proposed.

As in other situations [7,8,9], we can leverage technology to address the issue, which in our case involves enhancing IoT education at the University of Guayaquil. Therefore, we present an IoT project that implements an orchestra in a distributed way, based on commercial LEGO Mindstorms EV3 hardware [10]. Thanks to this project, it was possible to reinforce learning various musical concepts such as device communications, beat, and tempo. Additionally, the project provided music fans the opportunity to watch how the many sections of an orchestra interacted in real time. Furthermore, the user was able to control the orchestra elements with basic instructions such as volume up/down or mute.

The rest of the article is organized as follows. Section 2 presents a survey of the hardware platforms available for the project. Section 3 presents project design, while Section 4 presents an assessment of the students’ perceptions of their motivation with the IoT course. Section 5 concludes this study and suggests recommendations for future research.

2. Materials

One of the advantages of designing IoT systems is that elements cooperate for a common purpose [11]. This behavior is like what occurs in a musical orchestra. Therefore, it is necessary to determine which elements make up the final system. That is why different possible hardware components were investigated before developing the system.

2.1. Hardware Platform

2.1.1. Arduino

Arduino is the cheapest option; the basic kit costs a quarter of the price of a Lego Mindstorms EV3 kit [12]. Extra components can be purchased at very cheap prices. Arduino is an open-source platform for both hardware and software. In addition, it is programmed using the C language, which makes programming easier [13]. However, it is necessary to buy expansion boards and several compatible components to be able to carry out the project [14]. Therefore, the time required to search, assemble, and configure them is a drawback [15]. In addition, the purchase of these components increases the final cost of the platform, making it a less economical option compared to the Lego kit. Similar problems were experienced with other boards such as Raspberry Pi [16].

2.1.2. Lego Mindstorms EV3

Lego Mindstorms EV3 has a complete set of equipment to carry out a multitude of projects in a simple way. This process is based on the union of several plastic blocks (Lego) and the programming of the main block or EV3 brick. This brick contains a 300 MHz ARM9, model Sitara AM1808 (Texas Instruments), which must be programmed [17]. In addition to assembling hardware prototypes by integrating components such as sensors and servomotors, several graphical development environments are used [18]. In addition to the programming environments, there are several application programming interfaces (APIs), such as Java LeJOS EV3 version 0.9.1-beta [19], that support the basic functionalities of the system elements, such as the rotation of a servomotor or the capture of information by sensors [20].

To evaluate the maximum development time of the project (15 days), Mindstorms EV3 was chosen instead Arduino due to the ease of configuring sensors and servomotors, in addition to its ease of programming.

2.2. Software

Among the libraries that allow programming the kit, the following software was assessed.

2.2.1. BrickOS

BrickOS [21] is an alternative operating system that supports Lego Mindstorms RCX blocks. Thanks to the libraries provided by BrickOS, the blocks are programmed in C, C++, and assembler. BrickOS is supported on most Linux distributions and Windows (by CYGWIN), using the gcc or gcc++ compilers.

2.2.2. LeJOS Version 0.9.1-Beta

Unlike BrickOS, LeJOS [19] does not require installing an operating system that replaces the firmware of the Lego RCX, NXT, or EV3 blocks, but rather installs a Java virtual machine, which allows programming the blocks in this language.

2.2.3. Not Quite C

Not Quite C [22] is a language with a syntax similar to C, which has various libraries to support Lego RCX blocks. The main disadvantage is its incompatibility with EV3 blocks, and the need for a development environment of its own, different from that of the EV3 kit. Not Quite C is available for Mac OS and Windows and uses a proprietary version of C as its programming language.

Among the options, LeJOS was selected, since both BrickOS and Not Quite C do not yet have support for the new version of EV3. In addition, since LeJOS is based on the Java virtual machine, it inherits the benefits of this language, such as object-oriented programming, multithreading, recursion, synchronization, exceptions, and multidimensional arrays.

3. System Design

3.1. Members of Orchestra

We distinguish two roles in the orchestra:

• The conductor, who oversees coordinating the rest of the components. The conductor was implemented in the computer.
• The member or musician, who communicates with the conductor. Each musician was implemented by an EV3 brick in charge of playing a score, sent by the conductor.

The distribution of functions is expressed in Figure 1. The orchestra conductor was executed by the computer, which communicates with each EV3 brick to send orders via Bluetooth.

3.2. Modeling Orchestra Behavior

Figure 2 shows the distributed model of the orchestra. This is a clear example of the application of the design methodologies learned in the distributed embedded systems course.

In the initial state, each member of the orchestra remained until the conductor sent the score and a clock time to synchronize. If the connection was established, the state of the connection was skipped. The members already received the score and the clock time sent by the conductor, and were attentive to the reception of the signal to start the execution of the work. If the signal was received, the state of execution was skipped. The members of the orchestra proceeded to play each note and bar of the score sent in the first state. The member was attentive to silence when the conductor ordered it. If this happened, the group went into silence mode. If not, it continued until the end of the piece, in which case, a signal was sent to the conductor that the orchestra completed its performance. When the conductor sent a signal of silence, the members continued the performance of the score without sound. Once the conductor sent a signal to end the silence, the group moved to the performance state. The conductor received the signals indicating the end of the piece from each of the orchestra members.

The design considered the sending of a clock signal by the conductor so that each member synchronized with all the others. On the other hand, the score was sent in the initial state.

3.3. Setting Environment

To configure the development environment, the embedded version of Java “SE Embedded 8” was installed on the EV3 brick which was installed on a microSD memory card. The LeJOS libraries were installed on a computer to implement the orchestra components. Bluetooth was used to implement wireless communication between the EV3 bricks (musicians in the orchestra) and the computer (conductor), through the RNDIS driver [20]. To discover the active devices within a network in the orchestra, the BrickFinder class was used. The BrickFinder class found bricks by discovering neighbors on the network. The discover() method was used to obtain a list of all remote EV3 bricks that were currently visible [23].

4. Results and Discussion

Two IoT groups, each with 35 students, participated in this study. Group A was asked to submit the project proposed in this manuscript as a final project, while Group B was asked to review the state of the art and make a theoretical proposal for a project using IoT.

In a survey, the following characteristics of the IoT course were evaluated: (A) interest in the content, (B) training session interactivity, (C) the importance of the content, (D) acquired ability to implement IoT solutions, and (E) usefulness in the students’ professional future. A Likert scale ranging from 1 to 5 was used: very dissatisfied (1), disappointed (2), acceptable (3), adequate (4), and excellent (5). An attribute was approved when it scored three or higher. Figure 3 and Figure 4 show the results of the survey.

According to the results, Group A gave a higher assessment of each characteristic of the IoT course than Group B. To find out if there was a significant difference between the perceptions of both groups, we used the Wilcoxon rank-sum test. The results are presented in

Table 1. Wilcoxon rank-sum test with results of the student course perception survey.

Evaluated Feature	W Value	p-Value
(A) Interest in the content	704	0.154
(B) Training session interactivity	890.5	0.0001048
(C) Importance of the content	790	0.01335
(D) Acquired ability to implement IoT solutions	851.5	0.00155
(E) Usefulness in the students’ professional future	825.5	0.0006402

.

According to the results of the Wilcoxon test, there was a significant difference (p-value < 0.05) in all the features evaluated. Students in group A gave a higher assessment of the acquired learning and the importance of its application in their future professional practice thanks to the development of the project proposed within the curriculum of the IoT courses at the undergraduate level.

5. Conclusions

As part of the IoT courses at the University of Guayaquil, we built a networked symphony orchestra with the Lego Mindstorms EV3 kit. Lego Mindstorms was used to design and build a networked symphony orchestra because of its simple configuration. The information learned during the course was applied. The knowledge gained in the University of Guayaquil’s IoT courses can be applied to new projects using the project’s design, evaluation of various technologies, and ultimate implementation. The survey results for 35 students in each of the two IoT groups confirmed that students’ interest in the course improved. While Group A was required to solve the project presented in this paper as their final project, Group B was asked to review the state of the art and develop a theoretical proposal for a project with IoT. The following features of the IoT course were assessed in both groups: content interest, training session interactivity, content importance, ability to deploy IoT solutions, and usefulness in one’s future career. According to the results, because of the creation of the project included in the undergraduate IoT course curriculum, students in group A had a higher assessment of their understanding of course content and the significance of applying it to their professional practice.