User Experience Evaluation of a Virtual Reality Tool Used for 3D Modelling in Industrial Design Education: A Study in the Indian Context
Abstract
1. Design Education in the Indian Context
2. Pandemic and Design Education
3. Immersive Media in Design Education
4. VR in 3D Visualisation
- Usability: The ease of use of the VR system and the ability of students to navigate and manipulate the virtual environment.
- Effectiveness: The effectiveness of VR in helping students understand complex 3D design concepts and accurately visualising design ideas.
- Feedback: The availability of real-time feedback to students on their design decisions, allowing them to quickly iterate and improve their designs.
- Immersion: The degree to which students are able to experience a sense of presence in the virtual environment and the impact this has on their learning.
- Collaboration: The ability of students to work together in the virtual environment, including the ability to share and review each other’s work.
- Accessibility: The availability and ease of access to VR systems and the ability of students to use VR outside of the classroom.
- Cost: The cost of VR systems and the associated hardware and software and the impact this has on the ability of institutions to adopt VR design as a teaching tool.
5. Aim of the Paper
6. Methodology
6.1. Participants
6.2. Design Manipulation Check (Conventional 3D Software vs. VR-Based Software)
Collab Option
- Students can import their drawing sketch and obj file of their 3D model, which they have made in 3D software.
- The faculty can use the pen tool in VR to mark overlays on the models and provide corrections by drawing directly on top of the model. They can assess the shape of the model by rotating and moving it virtually, identifying any necessary corrections in terms of sides, curves, and other aspects. This will help the student to understand the corrections instantly.
- The student can snap the 3D model onto the real image to view it in the proper 1:1 scale, and 3D model files from other software’s like Blender, CAD, Rhino, etc., can be imported into this VR tool and users can create a multiple-user collaboration lobby to discuss and work on that file.
- Using the tools, faculty members can easily point out and correct student mistakes, and both can draw and interact with the object and also have liberty to scale it up or down to a 1:1 ratio to understand its real proportions. This is especially helpful when physical interaction is not feasible, which can affect the learning curve of students and create a communication gap.
6.3. Working Definitions
- Ease of use means how comfortably one can use the software.
- Presence is the feeling of realism in a computer-generated environment or world.
- Accessibility is how one can reveal the desired information based on ethnicity (culture, language barrier, gender, and race) and ease.
- Interaction style is the way people how people will interact with the system.
- Visual appeal relates to the attractiveness of the interface.
- Precision is how accurately one can create 3D models with appropriate dimensions.
- Online Collaboration is the way multiple people work together using online platforms for 3D software.
- Error Notification is the way of presenting error messages to reduce human errors in human interaction.
- Natural Human Interaction refers to what they do while interacting among themselves.
- User Control and Freedom include achieving the same task through different processes.
- Implementation of Muscle Computer Interface (muICI): The muscle–computer interface is defined as interacting with computers using the electrical activity of muscles [26].
6.4. Study Design
6.5. Measures
6.6. Statistical Tools Used in this Study
7. Results
7.1. Perceived Usability
7.2. Engagement
7.3. Enjoyment
7.4. Time Consumption
7.5. Acceptance Rating
7.6. NASA-TLX Study
8. Discussion
9. Conclusions
Supplementary Materials
Author Contributions
Funding
Data Availability Statement
Conflicts of Interest
References
- Eames, C.; Eames, R. The Eames Report. April 1958. Des. Issues 1991, 7, 63–75. [Google Scholar] [CrossRef] [Scilit]
- Balaram, S. Design pedagogy in India: A perspective. Des. Issues 2005, 21, 11–22. [Google Scholar] [CrossRef] [Scilit]
- Jena, P.K. Impact of COVID-19 on higher education in India. Int. J. Adv. Educ. Res. 2020, 18, 5. [Google Scholar]
- Ulferts, H. Teaching as a Knowledge Profession: Studying Pedagogical Knowledge across Education Systems; OECD Publishing: Paris, France, 2021. [Google Scholar]
- Dhawan, S. Online learning: A panacea in the time of COVID-19 crisis. J. Educ. Technol. Syst. 2020, 49, 5–22. [Google Scholar] [CrossRef] [Scilit]
- Kamińska, D.; Zwoliński, G.; Maloku, H.; Ibrani, M.; Guna, J.; Pogačnik, M.; Haamer, R.E.; Anbarjafari, G.; Abazi-Bexheti, L.; Bozhiqi, K.; et al. The Trends and Challenges of Virtual Technology Usage in Western Balkan Educational Institutions. Information 2022, 13, 525. [Google Scholar] [CrossRef] [Scilit]
- Singh, J.; Steele, K.; Singh, L. Combining the best of online and face-to-face learning: Hybrid and blended learning approach for COVID-19, post vaccine, & post-pandemic world. J. Educ. Technol. Syst. 2021, 50, 140–171. [Google Scholar]
- Handbook, A.P. All India Council for Technical Education. Available online: https://www.aicte-india.org/sites/default/files/approval/Approval%20Process%20Handbook_2021-22%20(Revised).pdf (accessed on 12 June 2023).
- Dutta, S.J.; Bhattacharjee, R. Integration of virtual laboratories: A step toward enhancing e-learning technology. In Proceedings of the 2019 IEEE 5th International Conference for Convergence in Technology (I2CT), Bombay, India, 29–31 March 2019; pp. 1–5. [Google Scholar]
- Zheng, J.M.; Chan, K.W.; Gibson, I. Desktop virtual reality interface for computer aided conceptual design using geometric techniques. J. Eng. Des. 2001, 12, 309–329. [Google Scholar] [CrossRef] [Scilit]
- Pan, Z.; Cheok, A.D.; Yang, H.; Zhu, J.; Shi, J. Virtual reality and mixed reality for virtual learning environments. Comput. Graph. 2006, 30, 20–28. [Google Scholar] [CrossRef] [Scilit]
- Swan, K. Virtual interaction: Design factors affecting student satisfaction and perceived learning in asynchronous online courses. Distance Educ. 2001, 22, 306–331. [Google Scholar] [CrossRef] [Scilit]
- Hamad, A.; Jia, B. How Virtual Reality Technology Has Changed Our Lives: An Overview of the Current and Potential Applications and Limitations. Int. J. Environ. Res. Public Health 2022, 19, 11278. [Google Scholar] [CrossRef] [Scilit]
- Lee, E.A.; Wong, K.W. A Review of Using Virtual Reality for Learning; Springer: Berlin/Heidelberg, Germany, 2008. [Google Scholar]
- Radianti, J.; Majchrzak, T.A.; Fromm, J.; Wohlgenannt, I. A systematic review of immersive virtual reality applications for higher education: Design elements, lessons learned, and research agenda. Comput. Educ. 2020, 147, 103778. [Google Scholar] [CrossRef] [Scilit]
- Di Natale, A.F.; Repetto, C.; Riva, G.; Villani, D. Immersive virtual reality in K-12 and higher education: A 10-year systematic review of empirical research. Br. J. Educ. Technol. 2020, 51, 2006–2033. [Google Scholar] [CrossRef] [Scilit]
- Banerjee, S.; Chowdhury, A.; Srivastava, A. Creating awareness about health and hygiene during menstrual cycle among Indian adolescent girls using virtual reality. In Advanced Manufacturing Systems and Innovative Product Design: Select Proceedings of IPDIMS 2020; Springer: Singapore, 2021; pp. 327–339. [Google Scholar]
- Mills, K.A.; Brown, A. Immersive virtual reality (VR) for digital media making: Transmediation is key. Learn. Media Technol. 2022, 47, 179–200. [Google Scholar] [CrossRef] [Scilit]
- North, M.M.; North, S.M. The sense of presence exploration in virtual reality therapy. J. Univ. Comput. Sci. 2018, 24, 72. [Google Scholar]
- Yildirim, B.; Topalcengiz, E.S.; Arikan, G.; Timur, S. Using virtual reality in the classroom: Reflections of STEM teachers on the use of teaching and learning tools. J. Educ. Sci. Environ. Health 2020, 6, 231–245. [Google Scholar] [CrossRef] [Scilit]
- Akman, E.; Çakır, R. The effect of educational virtual reality game on primary school students’ achievement and engagement in mathematics. Interact. Learn. Environ. 2020, 31, 1467–1484. [Google Scholar] [CrossRef] [Scilit]
- Chien, S.Y.; Hwang, G.J.; Jong, M.S. Effects of peer assessment within the context of spherical video-based virtual reality on EFL students’ English-Speaking performance and learning perceptions. Comput. Educ. 2020, 146, 103751. [Google Scholar] [CrossRef] [Scilit]
- Sutcliffe, A.; Gault, B.; Shin, J.E. Presence, memory and interaction in virtual environments. Int. J. Hum. Comput. Stud. 2005, 62, 307–327. [Google Scholar] [CrossRef] [Scilit]
- Nielsen, J. How to Conduct a Heuristic Evaluation; Nielsen Norman Group: Dover, DE, USA, 1995; Volume 1, p. 8. [Google Scholar]
- Mohammad, A.; Pedersen, L. Analyzing the Use of Heuristics in a Virtual Reality Learning Context: A Literature Review. Informatics 2022, 9, 51. [Google Scholar] [CrossRef] [Scilit]
- Chowdhury, A.; Ramadas, R.; Karmakar, S. Muscle computer interface: A review. In ICoRD’13: Global Product Development; Springer: Berlin/Heidelberg, Germany, 2013; pp. 411–421. Available online: https://link.springer.com/chapter/10.1007/978-81-322-1050-4_33 (accessed on 12 June 2023).
- Hart, S.G. NASA Task Load Index (TLX) 1986. Available online: https://ntrs.nasa.gov/api/citations/20000021488/downloads/20000021488.pdf (accessed on 12 June 2023).
- Chowdhury, A. Design and development of a stencil for mobile user interface (UI) design. In Research into Design for a Connected World: Proceedings of ICoRD 2019; Springer: Singapore, 2019; Volume 2, pp. 629–639. [Google Scholar]
- Thomas, T.; Tuteja, K.; Chowdhury, A. Advergames Are More Persuasive Among Different Online Advertisements. In Ergonomics for Improved Productivity: Proceedings of HWWE 2017; Springer: Singapore, 2021; pp. 105–117. [Google Scholar]
- Field, A. Discovering Statistics Using IBM SPSS Statistics; Sage: Newcastle upon Tyne, UK, 2013. [Google Scholar]
- McKnight, P.E.; Najab, J. Mann-Whitney U Test. The Corsini Encyclopedia of Psychology; Wiley: Hoboken, NJ, USA, 2010; Volume 1.
- Kyaw, B.M.; Saxena, N.; Posadzki, P.; Vseteckova, J.; Nikolaou, C.K.; George, P.P.; Divakar, U.; Masiello, I.; Kononowicz, A.A.; Zary, N.; et al. Virtual reality for health professions education: Systematic review and meta-analysis by the digital health education collaboration. J. Med. Internet Res. 2019, 21, e12959. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Guan, J.Q.; Wang, L.H.; Chen, Q.; Jin, K.; Hwang, G.J. Effects of a virtual reality-based pottery making approach on junior high school students’ creativity and learning engagement. Interact. Learn. Environ. 2021, 31, 2016–2032. [Google Scholar] [CrossRef] [Scilit]
- Allcoat, D.; von Mühlenen, A. Learning in virtual reality: Effects on performance, emotion and engagement. Res. Learn. Technol. 2018, 26, 2140. [Google Scholar] [CrossRef] [Scilit]
- Mystakidis, S. Distance education gamification in social virtual reality: A case study on student engagement. In Proceedings of the 2020 11th International Conference on Information, Intelligence, Systems and Applications (IISA 2020), Piraeus, Greece, 15–17 July 2020; pp. 1–6. [Google Scholar]
- Lange, A.K.; Koch, J.; Beck, A.; Neugebauer, T.; Watzema, F.; Wrona, K.J.; Dockweiler, C. Learning with virtual reality in nursing education: Qualitative interview study among nursing students using the unified theory of acceptance and use of technology model. JMIR Nurs. 2020, 3, e20249. [Google Scholar] [CrossRef] [Scilit]
- Von Janczewski, N.; Kraus, J.; Engeln, A.; Baumann, M. A subjective one-item measure based on NASA-TLX to assess cognitive workload in driver-vehicle interaction. Transp. Res. Part F Traffic Psychol. Behav. 2022, 86, 210–225. [Google Scholar] [CrossRef] [Scilit]
- Devos, H.; Gustafson, K.; Ahmadnezhad, P.; Liao, K.; Mahnken, J.D.; Brooks, W.M.; Burns, J.M. Psychometric properties of NASA-TLX and index of cognitive activity as measures of cognitive workload in older adults. Brain Sci. 2020, 10, 994. [Google Scholar] [CrossRef] [Scilit]
- Ren, S.; McKenzie, F.D.; Chaturvedi, S.K.; Prabhakaran, R.; Yoon, J.; Katsioloudis, P.J.; Garcia, H. Design and comparison of immersive interactive learning and instructional techniques for 3D virtual laboratories. Presence 2015, 24, 93–112. [Google Scholar] [CrossRef] [Scilit]
- Sampaio, A.Z.; Ferreira, M.M.; Rosário, D.P.; Martins, O.P. 3D and VR models in Civil Engineering education: Construction, rehabilitation and maintenance. Autom. Constr. 2010, 19, 819–828. [Google Scholar] [CrossRef] [Scilit]
- Luo, T.; Zhang, M.; Pan, Z.; Li, Z.; Cai, N.; Miao, J.; Chen, Y.; Xu, M. Dream-experiment: A MR user interface with natural multi-channel interaction for virtual experiments. IEEE Trans. Vis. Comput. Graph. 2020, 26, 3524–3534. [Google Scholar] [CrossRef] [Scilit]
- Choi, S.; Lee, J.; Lee, K.; Yoo, H.J. A 9.02 mW CNN-stereo-based real-time 3D hand-gesture recognition processor for smart mobile devices. In Proceedings of the 2018 IEEE International Solid-State Circuits Conference (ISSCC), San Francisco, CA, USA, 11–15 February 2018; pp. 220–222. [Google Scholar]
- Ali, N.; Ullah, S.; Khan, D. Minimization of students’ cognitive load in a virtual chemistry laboratory via contents optimization and arrow-textual aids. Educ. Inf. Technol. 2022, 27, 7629–7652. [Google Scholar] [CrossRef] [Scilit]
- Chen, L.; Liang, H.N.; Lu, F.; Wang, J.; Chen, W.; Yue, Y. Effect of collaboration mode and position arrangement on immersive analytics tasks in virtual reality: A pilot study. Appl. Sci. 2021, 11, 10473. [Google Scholar] [CrossRef] [Scilit]
- Guerra-Tamez, C.R. The Impact of Immersion through Virtual Reality in the Learning Experiences of Art and Design Students: The Mediating Effect of the Flow Experience. Educ. Sci. 2023, 13, 185. [Google Scholar] [CrossRef] [Scilit]
- Wang, A.; Thompson, M.; Uz-Bilgin, C.; Klopfer, E. Authenticity, interactivity, and collaboration in virtual reality games: Best practices and lessons learned. Front. Virtual Real. 2021, 2, 734083. [Google Scholar] [CrossRef] [Scilit]
- Özgen, D.S.; Afacan, Y.; Sürer, E. Usability of virtual reality for basic design education: A comparative study with paper-based design. Int. J. Technol. Des. Educ. 2021, 31, 357–377. [Google Scholar] [CrossRef] [Scilit]
- Muñoz, R.; Barcelos, T.; Chalegre, V. Defining and validating virtual worlds usability heuristics. In Proceedings of the 2011 30th International Conference of the Chilean Computer Science Society, Curico, Chile, 9–11 November 2011; pp. 171–178. [Google Scholar]
- Huang, H.; Lee, C.F. Factors affecting usability of 3D model learning in a virtual reality environment. Interact. Learn. Environ. 2022, 30, 848–861. [Google Scholar] [CrossRef] [Scilit]
- Hamurcu, A.; Timur, Ş.; Rızvanoğlu, K. An overview of virtual reality within industrial design education. J. Eng. Des. Technol. 2020, 18, 1889–1905. [Google Scholar] [CrossRef] [Scilit]
- Yu, M.; Zhou, R.; Wang, H.; Zhao, W. An evaluation for VR glasses system user experience: The influence factors of interactive operation and motion sickness. Appl. Ergon. 2019, 74, 206–213. [Google Scholar] [CrossRef] [Scilit]
- Aher, M.; Multanwala, H.; Goswami, M.; Chowdhury, A. Visual Human Factor Issues Related to Virtual Reality Devices and Interfaces. Int. J. Des. Allied Sci. 2022, 1, 1–7. [Google Scholar]





| Parameter | Conventional 3D Software Group | VR-Based Software Group |
|---|---|---|
| Usability. Engagement and Enjoyment study | ||
| Mean Age (yrs) | 20.62 | 20.53 |
| Age Range(yrs) | 19–23 | 18–23 |
| Male: n (%) | 15 (44.12) | 15 (44.12) |
| Female: n (%) | 19 (55.88) | 19 (55.88) |
| n | 34 | 34 |
| Cognitive Workload Study | ||
| Mean Age (yrs) | 20.21 | 20.21 |
| Age Range (yrs) | 18–21 | 18–21 |
| Male: n (%) | 5 (35.71) | 5 (35.71) |
| Female: n (%) | 9 (64.29) | 9 (64.29) |
| n | 14 | 13 |
| Sl No. | Parameter | Conventional | VR-Based 3D Modelling Software | Inferences |
|---|---|---|---|---|
| 1 | Ease of use | There are concealed commands within menus and submenus. | It has more ease of use because of hand gestures. | VR-based software is more convenient for 3D model making |
| 2 | Immersion | Negligible immersive quality (presence) | Users can seamlessly interact with the working 3D model (zoom, rotate, scale) view in different angles to get the feel of presence. | VR-based software is more immersive in creating 3D models |
| 3 | Accessibility | Voice commands are included | Voice commands and hand gestures are included | VR-based software has more accessibility for inclusive design |
| 4 | Interaction style | Input devices: using mouse and keyboard, touch-based interaction if touch-based monitor used | Using hand gestures, VR hand controller and wearable device (Microsoft MYO) | VR-based software is more intuitive to use |
| 5 | Visual appeal | Elaborated interface has been used | Visually delight interface | Visually pleasant in terms of UI interfaces in Conventional 3D software as well as in VR-based software. |
| 6 | Precision | More precise 3D models could be created in Conventional 3D software. | Lack of precision in the 3D output models | Conventional 3D modelling software can provide greater precision. However, VR-based modelling software can achieve precision if the created model is exported in a compatible format and edited in conventional 3D software. |
| 7 | Online collaboration | Limited collaborative opportunities | All type of collaboration is possible. | The VR-based system allows for collaboration between users, allowing them to interact with the intended model simultaneously. |
| 8 | Error notification | Present | More lively and feedback notification | VR-based software has better error notifications |
| 9 | Natural interaction | Absent | Hand gestural interaction Voice-based interaction (VOI) | VR-based software has a very visceral way of using hand gestures |
| 10 | User control and freedom | Present | Present | Both are good in terms of user control freedom |
| 11 | Implementation of Muscle–Computer Interface (muICI) | Limited possibilities | Enormous possibilities | In VR-based system Muscle–Computer Interface (mulCI) can be incorporated. |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Banerjee, S.; Chowdhury, A.; Yein, N. User Experience Evaluation of a Virtual Reality Tool Used for 3D Modelling in Industrial Design Education: A Study in the Indian Context. Designs 2023, 7, 105. https://doi.org/10.3390/designs7050105
Banerjee S, Chowdhury A, Yein N. User Experience Evaluation of a Virtual Reality Tool Used for 3D Modelling in Industrial Design Education: A Study in the Indian Context. Designs. 2023; 7(5):105. https://doi.org/10.3390/designs7050105
Chicago/Turabian StyleBanerjee, Shakti, Anirban Chowdhury, and Nilakshi Yein. 2023. "User Experience Evaluation of a Virtual Reality Tool Used for 3D Modelling in Industrial Design Education: A Study in the Indian Context" Designs 7, no. 5: 105. https://doi.org/10.3390/designs7050105
APA StyleBanerjee, S., Chowdhury, A., & Yein, N. (2023). User Experience Evaluation of a Virtual Reality Tool Used for 3D Modelling in Industrial Design Education: A Study in the Indian Context. Designs, 7(5), 105. https://doi.org/10.3390/designs7050105

