Next Article in Journal
A Multiprotocol Wireless Sensor Network for High Performance Sport Applications
Previous Article in Journal
Optimized Passive Defense Measures via IR Imaging
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Interconnecting Haptic Interfaces with High Update Rates through the Internet †

by
George Kokkonis
1,*,
Kostas E. Psannis
1,
Sotirios Kontogiannis
2,
Petros Nicopolitidis
3,
Manos Roumeliotis
1 and
Yutaka Ishibashi
4
1
Department of Applied Informatics, University of Macedonia, 54636 Thessaloniki, Greece
2
Department of Mathematics, University of Ioannina, Ioannina, 45110 Epirus, Greece
3
Department of Informatics, Aristotle University of Thessaloniki, 54636 Thessaloniki, Greece
4
Department of Computer Science, Nagoya Institute of Technology, Nagoya 466-8555, Japan
*
Author to whom correspondence should be addressed.
The present paper is an extended version of the paper, presented at IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB), Cagliari, Italy, 7–9 June 2017. In the conference proceedings, the existing transport protocols for supermedia streams were outlined and a performance evaluation of these protocols was made. In this paper existing protocols are presented in detail, and extensive simulation tests are undertaken. Moreover a thorough analysis of the experimental results is made.
Appl. Syst. Innov. 2018, 1(4), 51; https://doi.org/10.3390/asi1040051
Submission received: 6 November 2018 / Revised: 11 December 2018 / Accepted: 12 December 2018 / Published: 18 December 2018

Abstract

:
Real-time transferring of the haptic sense over the Internet is quite a challenging task. This paper outlines the proposed protocols for transferring haptic streams over the Internet. Moreover, it describes the Quality of Service requirements that a network has to fulfill in order to successfully use haptic interfaces with high update rates over the Internet. Extensive simulations and experiments for the performance evaluation of transport protocols for real-time transferring haptic data are carried out. Complements between simulation and real world experiments are discussed. The metrics that are measured for the performance evaluation are delay, jitter, throughput, efficiency, packet loss and one proposed by the authors, packet arrival deviation. The simulation tests reveal which protocols could be used for the transfer of real-time haptic data over the Internet.

1. Introduction

This paper is an extended version of the paper, presented at IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB), 2017 [1]. It presents the existing transport protocols that could be used for transferring haptic streams with high update rates over the Internet. Simulation and emulation tests for transferring haptic streams over the Internet are undertaken. A thorough analysis of these results is presented.
Real-time multi-sensory streams carry audio, video, haptics, and other sensory data [2]. Supermedia data obtain massive variety and volume. This increase of data deteriorates the network status of the Internet.
Researchers [3,4,5,6] try to determine the QoS of the network, in order to maximize the Quality of Experience (QoE) of the user [7] in a real-time internet haptic application. In order to maximize the QoE of the user in a haptic application, the update rate of the haptic stream should rather high, close to 1 KHz. This high update rate should be supported by the QoS requirements of the network, in order to successfully transfer online the haptic feeling real-time. The network performance of the Internet is not stable. It is changing from one area to another and from one hour to another. The recent network conditions of the Internet might permit online real-time haptic applications to flourish. This paper investigates whether the Internet is capable of transferring real-time the haptic sense.
In order to transfer haptic data through the Internet, specific transport protocols should be enforced. Several protocols have been developed for transferring haptic data. The most important are the Application Layer Protocol for Haptic Networking (ALPHAN) Protocol [8], the Smoothed—Synchronous Collaboration Transport Protocol (S-SCTP) [9], the Efficient Transport Protocol (ETP) [10,11], the Interactive Real-time Protocol (IRTP) [12], the Real Time Application Level Protocol For Distributed Interactive Media protocol (RTP/I) [13] and the Real Time Network Protocol (RTNP) [14].
Other protocols that are being widely used to transfer real-time multimedia data, such as UDP, RTP [15], Datagram Congestion Control Protocol (DCCP) [16] and Stream Control Transmission Protocol (SCTP) [17], should be tested for haptic data transferring as well. All the above protocols have been proposed for transferring real-time streams of voice and video, but they have not been tested for transferring the haptic sense. This research tries to investigate whether some of the above real-time protocols could be used for transferring real time the haptic sense. Taking into account the special features of each protocol, the authors will try to propose which protocol suits best for the real-time transfer of the haptic feeling.
The rest of this paper is organized as follows. Section 2 outlines the most recent haptic applications. Section 3 depicts the network conditions that are met in the Internet today. Section 4 presents the simulation scenario that is used for the evaluation of transport protocols. Section 5 analyses the results of the simulation testing. Section 6 discusses the complements, differences and relevancies between simulation and real world experiments. Finally, Section 7 concludes the paper.

2. Internet-Based Haptic Applications

The expansion of the Internet has led to the emergence of haptic applications. Researchers in [18] try to transfer the haptic feeling over the Internet. Network performance, which is often measured through delay and jitter is playing a crucial role in QoE of the user [19]. Other factors that can deteriorate the QoE is the scaling factor in macro-micro teleoperations [20,21] and the inertia of the haptic system [22,23]. Some interesting techniques that can compensate the delay of the network are shown in [24,25,26]. Moreover Fuzzy algorithms [27,28] can be used when the network performance is not sufficient. Computer grids [29] can also offer sufficient computational power to process the demanding Fuzzy algorithms.
Haptic applications could be used to military operations [30], education [31], telesurgery [32], video games [33], and video enhancement [34]. Motion-copying systems (MCS) [35] also use haptics to copy movements of skill experts for mentoring. Emotional communication can be enhanced among users with the use of haptic devices [36]. The virtual reality in now evolving to augmented virtuality [37].
Haptics can help people with visual problems. Navigation and spatial information is offered through haptic sticks to blind people [38,39]. Tactile sensors [40] can be used for remote manipulation and tele-operations. Kinesthetic disabilities can be overcomed with the help of haptic devices [41]. Humanoid robots can imitate the human walking by using haptic sensors [42].
In order to use haptic devises through websites HTML5 offers new java script modules called Web Graphic Library (WebGL). WebGL can manipulate 3D models in a web browser. Researchers in [43] created an HTML5 Haptics (H5H) Plugin. It is compatible with almost all browsers, and it uses “HAPI” to render haptics. It supports almost all haptic devices.

3. The Network Conditions of the Internet

A lot of research [44,45,46] has been conducted for the network conditions of the Internet. Network conditions refer to the amount of traffic that is being transferred through the Internet, the End to End delay, as well as the jitter between source and destination, and the available bandwidth for data transport.
The results from research [3,4,5,6] have concluded that in order to maximize the Quality of Experience (QoE) of the user for haptic streams, the network conditions should satisfy the Quality of Service (QoS) requirements of Table 1.
All the above metrics vary in time and space. They depend on the number of online users, the amount of data that is being transferred at the specific moment of the measurement, and the available equipment of lines and routers. It has been recorded that the amount of data transferred through the web is constantly increasing [47]. Apart from that, the number of online users is increasing as well. The growth of data transfer is compensated by continuing infrastructure upgrades of computer networks.
There are two methods for measuring the network performance. The one is the active and the other is the passive method [44]. As far as the active monitoring is concerned, ICMP packets are sent over the network to monitor the delay, the round trip time, the jitter and the packet loss. Other active monitoring tools for network performance are the ping tool, the traceroute, the capprobe, the dummynet, the netem and the pathchar [45]. The passive method observes the network traffic using some tools that are called sniffers. The most known of them are the Tcpdump, the Wireshark, the Ethereal, the Netflow and the JFlow [44].
In order to monitor the network status, the authors actively measured the average and the standard deviation of the delay, the packet loss rate, and the number of hops of networks between countries and continents. Measurements for the above metrics were made by the authors between Japan and Korea, and between Japan and Greece [48]. A recent measurement has also been made between two cities of Greece, Grevena and Thessaloniki [49]. The distance between those two cities is 175 Km. Two different networks were used for this measurement, the private optical network, GRNET [50], part of the pan-European GEANT network with speeds up to 4 × 10Gbps, on the one hand and one simple 8 Mbps Adsl connection on the other. The results of these measurements are shown in Table 2 and Table 3. For the above experiments, 3000 ICMP packets for each of 0, 6, 11, 15 and 19 o’ clock standard time were sent from one destination to the other.
It is understood that the Internet connection between Japan and Korea satisfy all the restrictions of Table 1 for transferring haptic data through the Internet. For the Internet connection between Japan and Greece, the values of Table 2 are relatively high because they refer to intercontinental pings. The average delay exceeds the limit of Table 1. This is due to the fact that the physical distance between Japan and Greece is much larger than Japan and Korea. That’s why the number of hops is much bigger in this intercontinental connection.
For the Internet connection between the two cities of Greece the results are shown in Table 3. In the case of the simple 8 Mbps Adsl connection the results are slightly worse than the limits in Table 1 for the average delay and jitter. The packet loss is within the limits of Table 1. On the other hand, in the case of the private optical network, GRNET, the results are encouraging. The average delay is only 19.12 ms, the jitter is only 1.70 ms and packet loss is 0.00%. All the above results are much lower than the limits in Table 1, which means that the transport of haptic data through the Internet is feasible under some circumstances.
Another important factor that describes the network conditions of the Internet is the connection speed of the end user. The recent spread of ADSL and VDSL connections provide consumers with connections up to 50 Mbps bandwidth. This bandwidth is by very sufficient for the requirement of 1 Mbps throughput that is being produced from haptic applications, based on Table 1.
We can conclude that the Internet network conditions are now suitable for online real-time haptic applications, especially when these applications take place in near regions.

4. Simulation Scenarioof Existing Protocols

In order to monitor the performance metrics of Table 1 and evaluate the transport protocols, simulations are undertaken. The network simulator that is used is the Network Simulator 2 (NS2) [51]. It is a widely used open source simulator. A lot of common protocols have already been implemented and tested in the NS2. Unfortunately, not many haptic protocols have been applied in NS2 so far.
One transport protocol for haptic applications that havebeen implemented in NS2 is the ETP. Apart from that, a lot of real-time protocols that could be used for haptic applications have been applied in the NS2. Some of them are the RTP, SCTP, DCCP and UDP.
All the above protocols were attached to different nodes in the NS2, as shown in Figure 1. The protocol TCP was mainly applied to the simulation scenario as a traffic generator. All the other protocols try to send a stream of packets with a packet rate of 1000 packets per sec. Most of them have a congestion control algorithm and minimize their sending rate in case of congestion. The haptic packet size that every protocol sends is 64 bytes of data payload [12] plus the overhead of the protocol.
The sample video for the HEVC encoding was the mobile_cif YUV series [52] with 352 × 288 resolution at 24 Hz. The data rate of this video sample after the HEVC encoding with Quantization Parameter QP = 27 and Low-Delay inter-prediction is 642 kbps [53]. This data stream sent over the RTP protocol with a packet size of 1500 bytes.
The audio stream was sent over the RTP protocol with 128 kbps bit rate, packet size 320 bytes and sending rate 50 packets per second.

4.1. Static Network Bandwidth, Delay and Internet Traffic

The square nodes in Figure 1 are routers that are connected with each other through the Internet.
The connection speed between those routers is set to 1, 5, 10, 15, and 20 Mbps for each simulation and is stable for the whole simulation period. The Internet bandwidth of 1 Mbps has been chosen so that a fully congested network can be represented. The Internet bandwidth of 20 Mbps corresponds to a network with no congestion. The Internet bandwidth of 5 Mbps corresponds to a network with low congestion. The connection between the server nodes 6 and 7 is regarded to be the Internet bottleneck of the simulation.
The end to end delay in the connection between nodes 6 and 7 was set to 5, 10, 20, 30, 40, 50, and 60 ms for each simulation and was stable for the whole simulation period. The 5 ms delay is a very small delay that rarely occurs in Internet connections. On the other hand, 50 ms is regarded to be the upper tolerable limit of delay, based on Table 1, that’s why the 60 ms is set as the maximum delay of the simulations. Of course, the end to end delay is changing dynamically in the real world connections. The authors deliberately kept the end to end delay constant throughout each simulation so as to study the behavior of each protocol at the specific delays. This would help the researchers decide which protocols are preferable when the characteristics of the network delay are known. Most of the diagrams depicted in this paper are for an Internet delay of 40 ms which is an acceptable delay, based on Table 1, and commonly encountered in the network. The simulations tests undertaken were 5 (scales for the Internet speed) × 7 (scales for the delay) = for a total of 35 tests.
The connection speed between the nodes 0–5 and server 6, as well as nodes 8–13 and server 7, is 100 Mbps as they are considered to be in the same local area network. The delay in those connections was 1 ms.
The simulation time for each simulation was 20 s. At time 0.5 s the FTP application which was attached at node 0 started to send data. At time 2 s all the other Constant Bit Rate (CBR) applications which were attached to nodes 1–5 will start to send packets with a rate of 1000 packets per second. The packet size varied from node to node depending on the header of the transport protocol.

4.2. Dynamic Network Bandwidth, Delay and Internet Traffic

In order to simulate the dynamic Internet network and test the response of the examined protocols, dynamic traffic generators such as the DelayBox [54] and the TMIX [55] ns2 modules are used. These modules dynamically change the packet loss, the delay and the packet loss of the network. The dynamic traffic is being fed with the help of TMIX through the node 14 and is being channeled to the node 17 of Figure 2. Moreover two Delayboxes are inserted to the main line in order to dynamically change the delay from 1 to 20 ms, the bandwidth from 1 to 20 Mbps and the packet loss from 0 to 1%. The DelayBoxes enforce dynamic network conditions on the TCP flows of the network. The monitored UDP flows are affected indirectly as the TCP flows alter the network conditions.

5. Simulation Results

5.1. Protocol Efficiency

Protocol efficiency is uninfluenced by the network status and is determined by the payload of each application and the header of each protocol Equation (1). As real-time haptic applications enforce very high update rate, the protocol efficiency should be examined. Haptic applications use small payloads The transport protocols should use as small overhead as possible so as not to overshadow the payloads of the haptic applications.
Table 4 illustrates the efficiency of the examined protocols. The UDP protocol shows the highest efficiency As it has the smallest header.

5.2. Packet Loss

Figure 3 and Figure 4 depict the percentage of packet loss inrelation to the delay of the network. This diagram is important as it reveals the correlation between the network delay and the packet loss for each protocol. If the characteristics of the End-to-End delay of the network are known, it can be decided which protocol should be used for the transport of haptic data so as to avoid high values of packet loss.
In Figure 3 the Internet bottleneck is 20 Mbps for all simulations. The delay was set to 5, 10, 20, 30, 40, 50, and 60 ms for each simulation and was stable for the whole simulation period. All the protocols present quite a low packet loss, lower from the limit of 10% of Table 1. The worst performance is presented by protocol the SCTP and the DCCP, but still they have a packet loss lower than 0.76%.
On the other hand, Figure 4 presents much higher values of packet loss. In this scenario, the Internet bottleneck has only 1 Mbps bandwidth for all simulations. It is obvious that the network is congested. Six protocols are trying to send a throughput of at least 3 Mbps over the network with a bandwidth of 1 Mbps. The protocol with the smallest percentage of packet loss is the UDP protocol. The protocol with the higher packet loss is again the SCTP and the DCCP.
Figure 5 and Figure 6 depict the interaction between the packet loss and the bandwidth of the network. Figure 5 depicts a network with static network conditions while Figure 6 depicts a network with dynamic delay and bandwidth. Both charts have similar behavior for bandwidth higher than 5 Mbps. From these charts one can conclude that as the bandwidth increases, the packet loss decreases. For bandwidth over 5 Mbps the packet loss is lower than 1%. When the bandwidth is 1 Mbps, the packet loss is very high, which means that the network is congested. Figure 3, Figure 4 and Figure 5 shows that the DCCP and the SCTP show the worst performance, as far as, the packet loss is concerned when the network is congested. The SCTP and the DCCP protocol use a TCP-like Congestion Control. The sender uses a congestion window and transmits packets until that window is closed. When congestion occurs the congestion window is lowered to half. The DCCP and the SCTP protocols send their packets in bursts. The buffers of the intermediate routers are overflowed and many packets are dropped.

5.3. Throughput

The throughput of a protocol is crucial as higher throughput means higher possibilities for congestion. When the bandwidth of the network is low, protocols with small throughput must be used.
Figure 7 illustrate the throughput of the protocols when the Internet bandwidth is 20 Mbps, so that no congestion should occur in the network. The delay of the Internet is set to 40 ms, as 50 ms is the maximum acceptable delay, based on Table 1. TCP throughput varies between 1660 Kbps and 3300 Kbps, because of the congestion window of the TCP’s congestion control. UDP and RTP protocols pose a steady throughput of 576 and 672 Kbps for haptic data respectively. This means that their sending rate is constant. The higher throughput of RTP is due to the higher header of the protocol. Protocol ETP tries to reach its highest sending rate, but its growth is very slow, due to its congestion control. After 20 s of simulation time it had not yet reached the sending rate of 1000 packets per second. The SCTP protocol presents the highest throughput, after TCP, among the other protocols. Apart from that, it also presents the highest deviation of the throughput. DCCP for the first 6 s presents quite a big deviation of the throughput and it is not stabilized before the 8th second.
Figure 8 illustrates protocols behavior in a dynamic network. The big difference between Figure 7 and Figure 8 is the behavior of the TCP, the ETP, the SCTP, and the DCCP protocol. All the above protocols enforce a tcp-friendly congestion control. When the network bandwidth changes, these protocols changes their transmission rate so as to avoid congestion. The UDP and the RTP protocol show a steady behavior, as they don’t enforce any congestion control.
Figure 9 displays the throughput of the protocols for Internet bandwidth 5 Mbps and delay 40 ms. The bandwidth of 5 Mbps is chosen so that the network is under low congestion. TCP has lowered its throughput that now varies from 1660 Kbps to 2160 Kbps. UDP and RTP protocols display almost the same steady throughput with very small deviations as they do not have a TCP-Friendly rate control. ETP protocol adapts the smallest throughput. It cannot increase its sending rate, because there is some congestion on the network. It can be seen that the congestion control of TCP binds more bandwidth than that of ETP. DCCP shows almost the same performance with the previous simulation. It adapts almost the same steady throughput, with very small deviations, 2 s later than in the previous simulation. SCTP still presents the biggest deviation and does not manage to obtain a steady throughput.

5.4. Jitter

System instability is vulnerable to high values of jitter. Based on Table 1, haptic applications are more sensitive to jitter than other multimedia.
Figure 10 illustrates the jitter of the protocols for an network bandwidth 20 Mbps and delays from 5 to 60 ms. At this high bandwidth no congestion occurs. Almost all of the protocols show very small jitter, lower than 0.6 ms.
Figure 11 shows the jitter for network bandwidth 1 Mbps. It is a fully congested network with a lot of packets lost. The jitter is now much higher than in Figure 10. The protocols with the highest jitter are the DCCP, the TCP and the SCTP. RTP, UDP and ETP protocol have smaller jitter than 6.7 ms, with the ETP exhibiting the best performance with a jitter smaller than 2.6 ms. According to Table 1, the jitter should be lower than 2 ms, a goal that most of the protocols could not achieve, due to the congestion of the network. An interesting observation is that the protocols TCP, SCTP, and DCCP lower their jitter as the delay of the Internet bandwidth increases. All these protocols almost have the same tcp-like congestion control algorithm.
Figure 12 and Figure 13 depict the jitter of the protocols when the Internet bottleneck bandwidth varies from 1 to 20 Mbps. The average Internet delay is near 40 ms. All the protocols show a different behavior when the Internet bandwidth is 1 Mbps and the Network is heavily congested. For higher bandwidth values all protocols present almost the same behavior. As the Internet bandwidth increases, the congestion and as a consequence the jitter effect, are decreasing.

5.5. Packet Arrival Deviation

Packet Arrival Deviation (PAD) is a metric proposed by the authors for the performance evaluation of real-time haptic protocols. It is similar to jitter, but it can offer more precise picture of the real-time network delay conditions than jitter, since it can take into account both the receiver end for the time variation of received packets (packet reception jitter at the receiver end), as well as the ACK packets reception time variation at the sender (packet reception jitter at the sender end). Moreover, it takes also into account the changes in the sending rate of the source focusing on the fluctuations of the Internet bandwidth. Table 1 does not include the PAD, as it is only presently proposed by the authors. The upper limit of PAD should be equal to jitter’s, which is 2 ms.
Figure 14 displays the standard deviation of the packet arrival for different delays. The Internet bandwidth is 20 Mbps. The protocols with the higher standard deviation are the TCP, SCTP and the DCCP protocols. UDP and RTP and ETP protocols present very small standard deviation of packet arrival, lower than 2.5 ms. This difference between TCP, SCTP, DCCP and the UDP, ETP, RTP is the way they send their packets. The first group sends its packets in burst inside a congestion window (CWND), while the second group sends its packets with an almost steady inter packet gap.
Figure 15 and Figure 16 show the Packet Arrival Deviation when the bandwidth of the network changes from 1 to 20 Mbps. SCTP and DCCP and TCP have been excluded from Figure 15 and Figure 16 for 1 Mbps as they showed unacceptable values of PAD. The conclusions of this graph are similar to those of Figure 14. TCP shows the highest PAD as it uses the congestion control windows. UDP and RTP show very little PAD because they do not enforce any congestion control algorithm. For network bandwidth higher than 10 Mbps, there is no network congestion and all protocols show the same behavior.
Summarizing, not all transport protocols should be used for transferring haptic data over the Internet. Protocols that include TCP-like congestion control algorithms, such as the TCP, the SCTP and the DCCP, should be avoided in heavily congested networks. Timely delivery protocols such as the UDP, the ETP and the RTP are more suitable for real-time haptic data transferring. The UDP protocol is being used both by the ETP and the RTP protocol. When the haptic payload is 64 bytes, the efficiency of UDP is 88.88%. This is very crucial, as haptics demand very high sending rate. When the network is congested, the packet loss is rather high (Figure 4). Figure 7 and Figure 9 illustrate network behavior for 20 and 5 Mbps bandwidth with 40 ms delay. In these network conditions the UDP and the RTP protocols have steady throughput. The UDP protocol consumes lower bandwidth, as it is more efficient. The ETP protocol interacts very well when the network conditions are sufficient. UDP and RTP retain lower jitter and PAD from the other protocols in all network conditions. The ETP outperforms the UDP and the RTP when the network is heavily congested (Figure 11 and Figure 12). ETP in order to avoid congestion, it decreases its sending rate by increasing the inter packet gap. Lower packet loss, jitter and PAD are the outcome of this reduction. The RTP and the UDP have better performance in most network conditions. The ETP protocol applies congestion control algorithms and should be used in heavily congested networks.

6. Complements, Differences and Relevancies Between Simulation and Real World Experiments

In Section 3, our previous real world experiment is described, while in Section 4 and Section 5 a simulation experiment has taken place. The simulation tests complement real word experiments.
The real world scenario helped us understand the status of the Internet and define the values of the variables for the simulation experiment. The only variable that is difficult to define is the available bandwidth of the Internet for the whole path. It is a metric that is changing rapidly as it is based on the number of online users and the data that are exchanged.
The real world scenario revealed that a real teleoperation task through the Internet is feasible, while the simulation experiment helped us conclude which protocols suits better for these teleoperation tasks under specific network conditions.
In the real world experiment, it is understood that the experiment results are depending on the physical distance between the source and the destination and the ambiguous network conditions of the Internet. In the simulation tests network conditions are fully controlled, so more accurate results can be produced.
The real world experiment helped the authors define the mean end to end delay, the standard delay deviation (jitter) the packet loss and the number of hops between source and destination. For this experiment the UDP transport protocol was chosen. It is the simplest transport protocol and it is being used for most cases of real-time multimedia applications. Two completely different pairs of source and destination were chosen, in order to examine the dependency between the results on the distance between source and destination.
Despite the ambiguous network condition of the Internet the simulation results of the UDP protocol matches the results of the real world scenario for the connection between Korea and Japan, Table 5. At the specific simulation, the Internet end to end delay was set 32 ms. The Internet bandwidth was set to 20 Mbps. These settings depict a network with no congestion. Both simulation and experiment transport data over the UDP protocol. For the connection between Japan and Greece there were no corresponding simulation tests as the results from the real world experiment were outside the acceptable limits of Table 1.

7. Conclusions

The interconnection of haptic interfaces with high update rate through the Internet is feasible. Haptic data should be transferred by specific timely delivery protocols. Haptic transport protocols should be standardized. Strict QoS requirements should be enforced by Internet Service Providers in order to transfer the haptic feeling over the internet. Simulation and real world experiments revealed which transport protocols should be used for real-time haptic data delivery with high update rates over the Internet. Real-time, lightweight, unreliable, protocols outperform heavyweight reliable protocols. UDP, RTP and ETP protocols are more suitable for interconnecting haptic interfaces than the TCP, the DCCP and the SCTP protocol.

Author Contributions

Conceptualization, G.K., K.E.P., M.R., and Y.I.; Methodology, G.K., S.K., and P.N.; Software, G.K., S.K.; Validation, M.R., K.E.P., and P.N.; Formal Analysis, G.K., and S.K.; Investigation, G.G., and K.E.P.; Resources, K.E.P., P.N., and Y.I.; Data Curation, G.K., and S.K.; Writing-Original Draft Preparation, G.K., and S.K.; Writing-Review & Editing, K.E.P., M.R., and Y.I.; Visualization, G.K., and P.N.; Supervision, K.E.P., Y.I., M.R., and P.N.; Project Administration, G.K., K.E.P., and M.N.

Funding

This research received no external funding.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Kokkonis, G.; Psannis, K.E.; Roumeliotis, M.; Nikopolitidis, P.; Ishibashi, Y. Performance evaluation of transport protocols for realtime supermedia—HEVC streams over the Internet. In Proceedings of the 2017 IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB), Cagliari, Italy, 7–9 June 2017. [Google Scholar]
  2. Cen, Z.; Mutka, M.W.; Zhu, D.; Xi, N. Supermedia Transport for Teleoperations over Overlay Networks. In Networking 2005. Networking Technologies, Services, and Protocols; Performance of Computer and Communication Networks; Mobile and Wireless Communications Systems; Boutaba, R., Almeroth, K., Puigjaner, R., Shen, S., Black, J.P., Eds.; Lecture Notes in Computer Science; Springer: Berlin/Heidelberg, Germany, 2005; Volume 3462, ISBN 978-3-540-25809-4. [Google Scholar]
  3. Eid, M.; Cha, J.; El Saddik, A. Admux: An Adaptive Multiplexer for Haptic–Audio–Visual Data Communication. IEEE Trans. Instrum. Meas. 2011, 60, 21–31. [Google Scholar] [CrossRef]
  4. Iwata, K.; Ishibashi, Y.; Fukushima, N.; Sugawara, S. Qoe assessment in haptic media, sound, and video transmission: Effect of playout buffering control. Comput. Entertain. 2010, 8, 1–12. [Google Scholar] [CrossRef]
  5. Suzuki, N.; Katsura, S. Evaluation of QoS in haptic communication based on bilateral control. In Proceedings of the 2013 IEEE International Conference on Mechatronics (ICM), Vicenza, Italy, 27 February–1 March 2013; pp. 886–891. [Google Scholar]
  6. Isomura, E.; Tasaka, S.; Nunome, T. A multidimensional QoE monitoring system for audiovisual and haptic interactive IP communications. In Proceedings of the 2013 IEEE 10th Consumer Communications and Networking Conference (CCNC 2013), Las Vegas, NV, USA, 11–14 January 2013; pp. 196–202. [Google Scholar]
  7. Hamam, A.; el Saddik, A. Toward a Mathematical Model for Quality of Experience Evaluation of Haptic Applications. IEEE Trans. Instrum. Meas. 2013, 62, 3315–3322. [Google Scholar] [CrossRef]
  8. Al Osman, H.; Eid, M.; Iglesias, R.; El Saddik, A. ALPHAN: Application Layer Protocol for HAptic Networking. In Proceedings of the 2007 IEEE International Workshop on Haptic, Audio and Visual Environments and Games (HAVE 2007), Ottawa, ON, Canada, 12–14 October 2007; pp. 96–101. [Google Scholar]
  9. Dodeller, S.; Georganas, N.D. Transport layer protocols for telehaptics update message. In Proceedings of the 22nd Biennial Symposium on Communications, Queen’s Univeristy, Kingston, ON, Canada, 31 May–3 June 2004. [Google Scholar]
  10. Wirz, R.; Ferre, M.; Marin, R.; Barrio, J.; Claver, J.; Ortego, J. Efficient transport protocol for networked haptics applications. In Haptics: Perception, Devices and Scenarios; Ferre, M., Ed.; Lecture Notes in Computer Science; Springer: Berlin, Germany, 2008; Volume 5024, pp. 3–12. [Google Scholar]
  11. Wirz, R.; Marin, R.; Ferre, M.; Barrio, J.; Claver, J.M.; Ortego, J. Bidirectional Transport Protocol for Teleoperated Robots. IEEE Trans. Ind. Electron. 2009, 56, 3772–3781. [Google Scholar] [CrossRef] [Green Version]
  12. Li, P.; Lu, W.; Sun, Z. Transport layer protocol reconfiguration for network-based robot control system. In Proceedings of the 2005 IEEE Networking, Sensing and Control, Tucson, AZ, USA, 19–22 March 2005; pp. 1049–1053. [Google Scholar]
  13. Mauve, M.; Hilt, V.; Kuhmunch, C.; Effelsberg, W. Rtp/i-toward a common application level protocol for distributed interactive media. IEEE Trans. Multimedia 2001, 3, 152–161. [Google Scholar] [CrossRef]
  14. Uchimura, Y.; Yakoh, T. Bilateral robot system on the real-time network structure. IEEE Trans. Ind. Electron. 2004, 51, 940–946. [Google Scholar] [CrossRef]
  15. Schulzrimie, H.; Casner, S.; Frederick, R.; Jacobson, V. RFC 1889—RTP: A Transport Protocol for Real-Time Applications. January 1996. Available online: https://tools.ietf.org/html/rfc1889 (accessed on 1 December 2017).
  16. Kohler, E.; Handley, M.; Floyd, S. RFC 4340—Datagram Congestion Control Protocol (DCCP). March 2006. Available online: https://tools.ietf.org/html/rfc4340 (accessed on 1 December 2017).
  17. Stewart, R. RFC 4960—Stream Control Transmission Protocol. October 2000. Available online: https://tools.ietf.org/html/rfc4960 (accessed on 1 December 2017).
  18. Marin, R.; Sanz, P.J.; Nebot, P.; Wirz, R. A multimodal interface to control a robot arm via the web: a case study on remote programming. IEEE Trans. Ind. Electron. 2005, 52, 1506–1520. [Google Scholar] [CrossRef]
  19. Huang, P.; Zeng, Q.; Ishibashi, Y. QoE assessment of will transmission using haptics: Influence of network delay. In Proceedings of the 2013 IEEE 2nd Global Conference on Consumer Electronics (GCCE 2013), Tokyo, Japan, 1–4 October 2013; pp. 456–460. [Google Scholar]
  20. Mizoguchi, T.; Nozaki, T.; Ohnishi, K. Stiffness Transmission of Scaling Bilateral Control System by Gyrator Element Integration. IEEE Trans. Ind. Electron. 2014, 61, 1033–1043. [Google Scholar] [CrossRef]
  21. Sakaino, S.; Sato, T.; Ohnishi, K. Multi-DOF Micro-Macro Bilateral Controller Using Oblique Coordinate Control. IEEE Trans. Ind. Informat. 2011, 7, 446–454. [Google Scholar] [CrossRef]
  22. Nozaki, T.; Mizoguchi, T.; Ohnishi, K. Decoupling Strategy for Position and Force Control Based on Modal Space Disturbance Observer. IEEE Trans. Ind. Electron. 2014, 61, 1022–1032. [Google Scholar] [CrossRef]
  23. Hirche, S.; Buss, M. Human-Oriented Control for Haptic Teleoperation. Proc. IEEE 2012, 100, 623–647. [Google Scholar] [CrossRef] [Green Version]
  24. Yalcin, B.; Ohnishi, K. Stable and Transparent Time-Delayed Teleoperation by Direct Acceleration Waves. IEEE Trans. Ind. Electron. 2010, 57, 3228–3238. [Google Scholar] [CrossRef]
  25. Morimitsu, H.; Katsura, S.; Tomizuka, M. Design of force compensator with variable gain for bilateral control system under time delay. In Proceedings of the 2013 IEEE International Symposium on Industrial Electronics (ISIE 2013), Taipei, Taiwan, 28–31 May 2013; pp. 1–6. [Google Scholar]
  26. Natori, K.; Ohnishi, K. A Design Method of Communication Disturbance Observer for Time-Delay Compensation, Taking the Dynamic Property of Network Disturbance Into Account. IEEE Trans. Ind. Electron. 2008, 55, 2152–2168. [Google Scholar] [CrossRef]
  27. Linda, O.; Manic, M. Self-Organizing Fuzzy Haptic Teleoperation of Mobile Robot Using Sparse Sonar Data. IEEE Trans. Ind. Electron. 2011, 58, 3187–3195. [Google Scholar] [CrossRef] [Green Version]
  28. Sim, K.; Byun, K.; Harashima, F. Internet-based teleoperation on an intelligent robot with optimal two-layer fuzzy controller. IEEE Trans. Ind. Electron. 2006, 53, 1362–1372. [Google Scholar] [CrossRef]
  29. Peterlik, I.; Filipovic, J. Distributed Construction of Configuration Spaces for Real-Time Haptic Deformation Modeling. IEEE Trans. Ind. Electron. 2011, 58, 3205–3212. [Google Scholar] [CrossRef]
  30. Elliott, L.; Schmeisser, E.; Redden, E. Development of tactile and haptic systems for U.S. infantry navigation and communication. In Human Interface and the Management of Information: Interacting with Information; Lecture Notes in Computer Science; Springer: Berlin, Germany, 2011; Volume 6771, pp. 399–407. [Google Scholar]
  31. Mohammadi, N.; Murray, I. Developing methodologies for the presentation of graphical educational material in a non-visual form for use by people with vision impairment. In Proceedings of the 2013 IEEE International Conference on Teaching, Assessment and Learning for Engineering (TALE 2013), Bali, Indonesia, 26–29 August 2013; pp. 373–377. [Google Scholar]
  32. Nakajima, Y.; Nozaki, T.; Ohnishi, K. Heartbeat Synchronization with Haptic Feedback for Telesurgical Robot. IEEE Trans. Ind. Electron. 2014, 61, 3753–3764. [Google Scholar] [CrossRef]
  33. Guinan, A.L.; Caswell, N.A.; Drews, F.A.; Provancher, W.R. A video game controller with skin stretch haptic feedback. In Proceedings of the 2013 IEEE International Conference on Consumer Electronics (ICCE 2013), Las Vegas, NV, USA, 11–14 January; pp. 456–457.
  34. Danieau, F.; Fleureau, J.; Guillotel, P.; Mollet, N.; Christie, M.; Lecuyer, A. Toward Haptic Cinematography: Enhancing Movie Experience with Haptic Effects based on Cinematographic Camera Motions. IEEE MultiMedia 2014, 21, 11–21. [Google Scholar]
  35. Yajima, S.; Katsura, S. Multi-DOF Motion Reproduction Using Motion-Copying System With Velocity Constraint. IEEE Trans. Ind. Electron. 2014, 61, 3765–3775. [Google Scholar] [CrossRef]
  36. Hossain, S.; Rahman, A.; El Saddik, A. Measurements of multimodal approach to haptic interaction in second life interpersonal communication system. IEEE Trans. Instrum. Meas. 2011, 60, 3547–3558. [Google Scholar] [CrossRef]
  37. Sutherland, C.; Hashtrudi-Zaad, K.; Sellens, R.; Abolmaesumi, P.; Mousavi, P. An Augmented Reality Haptic Training Simulator for Spinal Needle Procedures. IEEE Trans. Biomed. Eng. 2013, 60, 3009–3018. [Google Scholar] [CrossRef] [PubMed]
  38. Ando, T.; Tsukahara, R.; Seki, M.; Fujie, M.G. A Haptic Interface “Force Blinker 2” for Navigation of the Visually Impaired. IEEE Trans. Ind. Electron. 2012, 59, 4112–4119. [Google Scholar] [CrossRef]
  39. Ahlmark, D.I.; Fredriksson, H.; Hyyppa, K. Obstacle avoidance using haptics and a laser rangefinder. In Proceedings of the IEEE Workshop on Advanced Robotics and Its Social Impacts (ARSO 2013), Tokyo, Japan, 7–9 November 2013; pp. 76–81. [Google Scholar]
  40. Suwanratchatamanee, K.; Matsumoto, M.; Hashimoto, S. Robotic Tactile Sensor System and Applications. IEEE Trans. Ind. Electron. 2010, 57, 1074–1087. [Google Scholar] [CrossRef]
  41. Oonishi, Y.; Oh, S.; Hori, Y. A new control method for power-assisted wheelchair based on the surface myoelectric signal. IEEE Trans. Ind. Electron. 2010, 57, 3191–3196. [Google Scholar] [CrossRef]
  42. Suwanratchatamanee, K.; Matsumoto, M.; Hashimoto, S. Haptic Sensing Foot System for Humanoid Robot and Ground Recognition With One-Leg Balance. IEEE Trans. Ind. Electron. 2011, 58, 3174–3186. [Google Scholar] [CrossRef]
  43. al Osman, H.; Jongeun, C.; el Saddik, A. The HTML5 Haptics Plugin. In Proceedings of the 2012 IEEE International Workshop on Haptic Audio Visual Environments and Games (HAVE 2012), Munich, Germany, 8–9 October 2012; pp. 130–133. [Google Scholar]
  44. Callado, A.; Kamienski, C.; Szabo, G.; Gero, B.; Kelner, J.; Fernandes, S.; Sadok, D. A Survey on Internet Traffic Identification. IEEE Commun. Surv. Tutor. 2009, 11, 37–52. [Google Scholar] [CrossRef]
  45. Finamore, A.; Mellia, M.; Meo, M.; Munafo, M.M.; Rossi, D. Experiences of Internet traffic monitoring with tstat. IEEE Netw. 2011, 25, 8–14. [Google Scholar] [CrossRef] [Green Version]
  46. Hasegawa, M.; Ikeguchi, T. An analysis of the Internet traffic by the method of surrogate data. In Proceedings of the 2002 IEEE International Symposium on Circuits and Systems, Phoenix-Scottsdale, AZ, USA, 26–29 May 2002; pp. III-599–III-602. [Google Scholar]
  47. Labovitz, C.; Iekel-Johnson, S.; McPherson, D.; Oberheide, J.; Jahanian, F. Internet Inter-domain Traffic. In Proceedings of the ACM SIGCOMM 2010 conference, New Delhi, India, 30 August–3 September 2010; pp. 75–86. [Google Scholar]
  48. Ishii, N.; Lee, S.; Ishibashi, Y.; Psannis, K.E.; Kim, J. Experiment on international connection for haptic media communications. In Proceedings of the Tokai-Section Joint Conference of the Eight Institutes of Electrical and Related Engineers, Tokai, Japan, 10–11 September 2009. O-434. [Google Scholar]
  49. Kokkonis, G.; Psannis, K.E.; Roumeliotis, M. Real time Haptic data transferring. In Proceedings of the IEEE 2016 Wireless Days (WD), Toulouse, France, 23–25 March 2016. [Google Scholar]
  50. GRNET: The Greek Research and Technology Network. Available online: https://www.grnet.gr/en (accessed on 20 November 2017).
  51. McCanne, S.; Floyd, S. Network Simulator ns-2. Available online: http://nsnam.isi.edu/nsnam/index.php (accessed on 1 December 2017).
  52. Fitzek, F.; Reisslein, M. Video Traces for Network Performance Evaluation: Yuv 4:2:0 Video Sequences. Available online: http://trace.eas.asu.edu/yuv/index.html (accessed on 20 November 2017).
  53. Kokkonis, G.; Psannis, K.E.; Roumeliotis, M.; Ishibashi, Y. Efficient algorithm for transferring a real-time HEVC stream with haptic data through the internet. J. Real-Time Image Process. 2016, 12, 343–355. [Google Scholar] [CrossRef]
  54. Cao, J.; Cleveland, W.S.; Gao, Y.; Jeffay, K.; Smith, F.D.; Weigle, M.C. Stochastic Models for Generating Synthetic HTTP Source Traffic. In Proceedings of the IEEE INFOCOM 2004, Hong Kong, China, 7–11 March 2004. [Google Scholar]
  55. Weigle, M.C.; Adurthi, P.; Hernandez-Campos, F.; Jeffay, K.; Smith, F.D. Tmix: A Tool for Generating Realistic Application Workloads in ns-2. ACM SIGCOMM Comput. Commun. Rev. 2006, 36, 67–76. [Google Scholar] [CrossRef]
Figure 1. Simulation environment of Network Simulator 2 (NS2).
Figure 1. Simulation environment of Network Simulator 2 (NS2).
Asi 01 00051 g001
Figure 2. Dynamic Network Topology of NS2 with TMIX and DelayBoxes modules.
Figure 2. Dynamic Network Topology of NS2 with TMIX and DelayBoxes modules.
Asi 01 00051 g002
Figure 3. Packet loss for internet bandwidth 20 Mbps.
Figure 3. Packet loss for internet bandwidth 20 Mbps.
Asi 01 00051 g003
Figure 4. Packet loss for internet bandwidth 1 Mbps.
Figure 4. Packet loss for internet bandwidth 1 Mbps.
Asi 01 00051 g004
Figure 5. Packet loss for internet delay 40 ms.
Figure 5. Packet loss for internet delay 40 ms.
Asi 01 00051 g005
Figure 6. Packet loss vs bandwidth.
Figure 6. Packet loss vs bandwidth.
Asi 01 00051 g006
Figure 7. Throughput for internet bandwidth 20 Mbps and delay 40 ms.
Figure 7. Throughput for internet bandwidth 20 Mbps and delay 40 ms.
Asi 01 00051 g007
Figure 8. Throughput vs bandwidth.
Figure 8. Throughput vs bandwidth.
Asi 01 00051 g008
Figure 9. Throughput for internet bandwidth 5 Mbps and delay 40ms.
Figure 9. Throughput for internet bandwidth 5 Mbps and delay 40ms.
Asi 01 00051 g009
Figure 10. Jitter for internet bandwidth 20 Mbps.
Figure 10. Jitter for internet bandwidth 20 Mbps.
Asi 01 00051 g010
Figure 11. Jitter for internet bandwidth 1 Mbps.
Figure 11. Jitter for internet bandwidth 1 Mbps.
Asi 01 00051 g011
Figure 12. Jitter for internet delay 40 ms.
Figure 12. Jitter for internet delay 40 ms.
Asi 01 00051 g012
Figure 13. Jitter vs. bandwidth for dynamic network conditions.
Figure 13. Jitter vs. bandwidth for dynamic network conditions.
Asi 01 00051 g013
Figure 14. Packet arrival deviation for internet bandwidth 20 Mbps.
Figure 14. Packet arrival deviation for internet bandwidth 20 Mbps.
Asi 01 00051 g014
Figure 15. Arrival deviation for internet delay 40 ms.
Figure 15. Arrival deviation for internet delay 40 ms.
Asi 01 00051 g015
Figure 16. Packet Arrival deviation vs bandwidth.
Figure 16. Packet Arrival deviation vs bandwidth.
Asi 01 00051 g016
Table 1. QoS requirements for supermedia streams [3,4,5,6,7].
Table 1. QoS requirements for supermedia streams [3,4,5,6,7].
QoSHapticsVideoAudioGraphics
Jitter (ms)≤2≤30≤30≤30
Delay (ms)≤50≤400≤150≤100–300
Packet Loss (%)≤10≤1≤1≤10
Update Rate (Hz)≥1000≥30≥50≥30
Packet Size (bytes)64–128≤MTU160–320192–5000
Throughput(kbps)512–102425,000–40,00064–12845–1200
Table 2. Internet status for intercontinental communication [48].
Table 2. Internet status for intercontinental communication [48].
Countries ConnectedAvg. Delay (ms)Standard Delay Deviation (ms)Packet Loss (%)No. Hops
Japan–Korea27.010.190.0211
Japan–Greece331.106.301.5326
Table 3. Internet status for communication between cities.
Table 3. Internet status for communication between cities.
Connected CitiesAvg. Delay (ms)Standard Delay Deviation (ms)Packet Loss (%)No. Hops
Grevena—thessalonikh through grnet [50]19.121.7005
Grevena—thessalonikh through adsl line53.195.310.118
Table 4. Protocol Efficiency.
Table 4. Protocol Efficiency.
ETPUDPRTPSCTPDCCP
Header (bytes)12 + 8(UDP)812 + 8(UDP)12 + 4(Chunk INF.)12
Haptic Payload (bytes)6464646464
Efficiency76.19%88.88%76.19%80%84.21%
Table 5. Similarities between simulation and real world experiments.
Table 5. Similarities between simulation and real world experiments.
Delay (ms)Jitter (ms)Packet Loss (%)
Connection between Japan and Korea27.010.190.02
Simulation of Udp Protocol with 20 Mbps Internet Bandwidth32.100.190.02

Share and Cite

MDPI and ACS Style

Kokkonis, G.; Psannis, K.E.; Kontogiannis, S.; Nicopolitidis, P.; Roumeliotis, M.; Ishibashi, Y. Interconnecting Haptic Interfaces with High Update Rates through the Internet. Appl. Syst. Innov. 2018, 1, 51. https://doi.org/10.3390/asi1040051

AMA Style

Kokkonis G, Psannis KE, Kontogiannis S, Nicopolitidis P, Roumeliotis M, Ishibashi Y. Interconnecting Haptic Interfaces with High Update Rates through the Internet. Applied System Innovation. 2018; 1(4):51. https://doi.org/10.3390/asi1040051

Chicago/Turabian Style

Kokkonis, George, Kostas E. Psannis, Sotirios Kontogiannis, Petros Nicopolitidis, Manos Roumeliotis, and Yutaka Ishibashi. 2018. "Interconnecting Haptic Interfaces with High Update Rates through the Internet" Applied System Innovation 1, no. 4: 51. https://doi.org/10.3390/asi1040051

Article Metrics

Back to TopTop