Comparing Usability of Augmented Reality and Virtual Reality for Creating Virtual Bounding Boxes of Real Objects
Abstract
:1. Introduction
2. Mixed Reality User Mapping Interface
- Virtual Object Creation and Deletion: The interface allows users to create and delete virtual 3D geometries to virtually map real objects. This is shown in Figure 2. These geometries represent primitive shapes, which are cuboids, spheres, and cylinders. These virtual objects can be spawned at the user’s command using hand gestures/controller inputs. Users are also given the ability to remove any virtual object from their virtual map at their discretion.
- Manipulating and Labelling Virtual Objects: The system also allows users to label virtual objects using a predetermined list of object labels corresponding to the physical objects used in the user study. Each virtual object’s pose is also modifiable by interacting with its bounding box, allowing users to translate, rotate, and scale these virtual objects.
- Input Methods: All functions provided by the virtual mapping interfaces can be accessed using hand gestures in the AR system or controller inputs in the VR system. In AR, near interactions are supported through finger-tap and hand-grab gestures, while far interactions can be performed using pointed air-pinch gestures which made use of an arm ray that could be directed at virtual objects and their modifiable features. In VR, the triggers of VR controllers are mapped to perform object selection and object manipulation when squeezed, while labeled buttons provide shortcuts for creating and deleting virtual objects when pressed. The different control methods are employed by users to manipulate and create any virtual objects and their semantic labels (shown in Figure 2). These methods form the basis of our comparison.
3. User Study
3.1. User Study Procedure
3.2. Devices
3.2.1. Virtual Reality (VR) Interaction Interface
3.2.2. Augmented Reality (AR) Interaction Interface
3.2.3. Practical Utilization
3.3. Metrics
3.3.1. Objective Metrics
- Three-Dimensional (3D) Intersection Over Union (IoU) (%): The user-generated semantic maps are evaluated using a known ground truth map by examining the volume of the intersection of the 3D bounding boxes of each object over the volume of the union of the bounding boxes. The ground truth map was created by measuring the dimensions of each of the six objects and their position and orientation relative to a fixed start point. Google’s Objectron library is used for computing the 3D IoU given the user-generated 3D bounding box and the corresponding ground truth bound box [16].
- Task Completion Time (s): The total time taken for a user to complete the mapping task.
3.3.2. Subjective Metrics
3.4. Hypotheses
- H1: The VR headset will produce virtual maps with greater accuracy.
- H2: Participants will complete the task quicker with the VR headset.
- H3: Participants will find the controls provided by the VR headset easier to use.
- H4: Participants will find the real-world environment easier to see and interpret when using the AR headset when compared to the VR headset.
- H5: Participants will enjoy using the VR headset for assisted mapping tasks more than the AR headset.
4. Results
4.1. Objective Metrics Results and Analysis
4.2. Subjective Metrics Results and Analysis
4.3. Discussion
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Bozgeyikli, E.; Bozgeyikli, L.L. Evaluating object manipulation interaction techniques in mixed reality: Tangible user interfaces and gesture. In Proceedings of the 2021 IEEE Virtual Reality and 3D User Interfaces (VR), Lisbon, Portugal, 27 March–1 April 2021; pp. 778–787. [Google Scholar]
- Xiao, R.; Schwarz, J.; Throm, N.; Wilson, A.D.; Benko, H. MRTouch: Adding Touch Input to Head-Mounted Mixed Reality. IEEE Trans. Vis. Comput. Graph. 2018, 24, 1653–1660. [Google Scholar] [CrossRef] [PubMed]
- Batmaz, A.U.; Machuca, M.D.B.; Pham, D.M.; Stuerzlinger, W. Do head-mounted display stereo deficiencies affect 3D pointing tasks in AR and VR? In Proceedings of the 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), Osaka, Japan, 23–27 March 2019; pp. 585–592. [Google Scholar]
- Gu, M.; Cosgun, A.; Chan, W.P.; Drummond, T.; Croft, E. Seeing thru walls: Visualizing mobile robots in augmented reality. In Proceedings of the IEEE International Conference on Robot & Human Interactive Communication (RO-MAN), Vancouver, BC, Canada, 8–12 August 2021. [Google Scholar]
- Waymouth, B.; Cosgun, A.; Newbury, R.; Tran, T.; Chan, W.P.; Drummond, T.; Croft, E. Demonstrating cloth folding to robots: Design and evaluation of a 2d and a 3d user interface. In Proceedings of the IEEE International Conference on Robot & Human Interactive Communication (RO-MAN), Vancouver, BC, Canada, 8–12 August 2021. [Google Scholar]
- Szczepanski, R.; Bereit, A.; Tarczewski, T. Efficient Local Path Planning Algorithm Using Artificial Potential Field Supported by Augmented Reality. Energies 2021, 14, 6642. [Google Scholar] [CrossRef]
- Xu, S.; Moore, S.; Cosgun, A. Shared-control robotic manipulation in virtual reality. In Proceedings of the International Congress on human–computer Interaction, Optimization and Robotic Applications (HORA), Ankara, Turkey, 9–11 June 2022. [Google Scholar]
- Gu, M.; Croft, E.; Cosgun, A. AR Point & Click: An interface for setting robot navigation goals. In Proceedings of the International Conference on Social Robotics, Florence, Italy, 13–16 December 2022. [Google Scholar]
- Hoang, K.C.; Chan, W.P.; Lay, S.; Cosgun, A.; Croft, E.A. Arviz: An augmented reality-enabled visualization platform for ros applications. IEEE Robot. Autom. Mag. 2022, 29, 58–67. [Google Scholar] [CrossRef]
- Newbury, R.; Cosgun, A.; Crowley-Davis, T.; Chan, W.P.; Drummond, T.; Croft, E.A. Visualizing robot intent for object handovers with augmented reality. In Proceedings of the IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), Naples, Italy, 29 August–2 September 2022. [Google Scholar]
- Topp, E.A. human–robot Interaction and Mapping with a Service Robot: Human Augmented Mapping. Ph.D. Thesis, KTH, Stockholm, Sweden, 2008. [Google Scholar]
- Cosgun, A.; Christensen, H. Context Aware Robot Navigation using Interactively Built Semantic Maps. arXiv 2018, arXiv:1710.08682. [Google Scholar] [CrossRef]
- Frank, M.; Drikakis, D.; Charissis, V. Machine-learning methods for computational science and engineering. Computation 2020, 8, 15. [Google Scholar] [CrossRef]
- Chaurasia, G.; Nieuwoudt, A.; Ichim, A.E.; Szeliski, R.; Sorkine-Hornung, A. Passthrough+: Real-Time Stereoscopic View Synthesis for Mobile Mixed Reality. Proc. ACM Comput. Graph. Interact. Tech. 2020, 3, 7. [Google Scholar] [CrossRef]
- Vuforia Developer Portal. Available online: https://developer.vuforia.com/ (accessed on 15 July 2023).
- Ahmadyan, A.; Zhang, L.; Ablavatski, A.; Wei, J.; Grundmann, M. Objectron: A large scale dataset of object-centric videos in the wild with pose annotations. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Virtual Online, 19–25 June 2021. [Google Scholar]
- Jost, T.A.; Nelson, B.; Rylander, J. Quantitative analysis of the Oculus Rift S in controlled movement. Disabil. Rehabil. Assist. Technol. 2021, 16, 632–636. [Google Scholar] [CrossRef] [PubMed]
- Shum, L.; Valdés, B.; Loos, H. Determining the Accuracy of Oculus Touch Controllers for Motor Rehabilitation Applications Using Quantifiable Upper Limb Kinematics: Validation Study. J. Med. Internet Res. 2019, 4, e12291. [Google Scholar] [CrossRef]
- Kern, F.; Kullmann, P.; Ganal, E.; Korwisi, K.; Stingl, R.; Niebling, F.; Latoschik, M.E. Off-The-Shelf Stylus: Using XR Devices for Handwriting and Sketching on Physically Aligned Virtual Surfaces. Front. Virtual Real. 2021, 2, 684498. [Google Scholar] [CrossRef]
- Lahoud, F.; Susstrunk, S. Ar in VR: Simulating infrared augmented vision. In Proceedings of the 2018 25th IEEE International Conference on Image Processing (ICIP), Athens, Greece, 7–10 October 2018; pp. 3893–3897. [Google Scholar]
- Lee, C.; Bonebrake, S.; Hollerer, T.; Bowman, D.A. A replication study testing the validity of AR simulation in VR for controlled experiments. In Proceedings of the 2009 8th IEEE International Symposium on Mixed and Augmented Reality, Orlando, FL, USA, 19–22 October 2009; pp. 203–204. [Google Scholar]
Questions | |
---|---|
I1 | The headset’s input system was easy to learn. |
I2 | The headset’s input system was comfortable to use. |
I3 | The headset’s input system helped me achieve the task efficiently. |
I4 | The headset’s input system was responsive to user input. |
I5 * | The headset’s input system was affected by errors (e.g., inaccurate tracking, incorrect interactions, wrong inputs, general bugs). |
V1 | It was easy to see the real-world objects I was mapping using this headset. |
V2 | It was easy to see the virtual objects I was manipulating using this headset. |
V3 | I would be comfortable seeing my surroundings using this headset for long periods of time. |
P1 | I enjoyed using this headset to complete the task. |
P2 * | It was mentally demanding to use this headset to complete the task. |
P3 | It was comfortable to use the headset’s accompanying headset. |
AR | VR | |
---|---|---|
IoU (%) | 0.262 ( = 0.112) | 0.407 ( = 0.157) |
Task Time (s) | 466.6 ( = 21.1) | 333.8 ( = 83.5) |
H | ||
---|---|---|
IoU (%) | VR > AR | 0.001 |
Task Time (s) | VR < AR | <0.001 |
H | ||
---|---|---|
I1—Easy to Learn | VR > AR | 0.006 |
I2—Comfortable to Use | VR > AR | 0.262 |
I3—Perceived Efficiency | VR > AR | 0.005 |
I4—Responsive to Input | VR > AR | 0.001 |
I5—Prone to Errors * | VR < AR | 0.021 |
V1—Seeing Real Objects | AR > VR | 0.006 |
V2—Seeing Virtual Objects | AR > VR | 0.163 |
V3—Visual Comfort | AR > VR | 0.017 |
P1—Perceived Enjoyability | VR> AR | 0.662 |
P2—Mental Load * | VR < AR | 0.180 |
P3—User Comfort | VR > AR | 0.072 |
AR | VR | |
---|---|---|
Bottle | 0.22 ( = 0.19) | 0.40 ( = 0.21) |
Can | 0.20 ( = 0.18) | 0.39 ( = 0.23) |
Apple | 0.25 ( = 0.20) | 0.32 ( = 0.15) |
Futsal Ball | 0.39 ( = 0.16) | 0.54 ( = 0.14) |
Small Box | 0.10 ( = 0.11) | 0.15 ( = 0.11) |
Large Box | 0.42 ( = 0.20) | 0.64 ( = 0.19) |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Kyaw, N.; Gu, M.; Croft, E.; Cosgun, A. Comparing Usability of Augmented Reality and Virtual Reality for Creating Virtual Bounding Boxes of Real Objects. Appl. Sci. 2023, 13, 11693. https://doi.org/10.3390/app132111693
Kyaw N, Gu M, Croft E, Cosgun A. Comparing Usability of Augmented Reality and Virtual Reality for Creating Virtual Bounding Boxes of Real Objects. Applied Sciences. 2023; 13(21):11693. https://doi.org/10.3390/app132111693
Chicago/Turabian StyleKyaw, Nyan, Morris Gu, Elizabeth Croft, and Akansel Cosgun. 2023. "Comparing Usability of Augmented Reality and Virtual Reality for Creating Virtual Bounding Boxes of Real Objects" Applied Sciences 13, no. 21: 11693. https://doi.org/10.3390/app132111693
APA StyleKyaw, N., Gu, M., Croft, E., & Cosgun, A. (2023). Comparing Usability of Augmented Reality and Virtual Reality for Creating Virtual Bounding Boxes of Real Objects. Applied Sciences, 13(21), 11693. https://doi.org/10.3390/app132111693