Online Virtual Reality Networked Control Laboratory Applied in Control Engineering Education
Summary
This study describes a WebVR-based online virtual reality (VR) laboratory system that provides users with immersive and interactive experimentation capabilities supported by VR devices. The proposed system not only helps to enhance the realism of user participation in online experiments but is also applicable to a wide range of online laboratory frameworks.
Abstract
Online laboratories play an important role in engineering education. This work discusses a WebVR-based virtual laboratory system. The user enters the simulated laboratory environment through a virtual reality (VR) device and interacts with the experimental equipment, similar to hands-on experiments in a physical laboratory. In addition, the proposed system allows users to design their own control algorithms and observe the effects of different control parameters to enhance their understanding of the experiment. To illustrate the features of the proposed virtual laboratory, an example is provided in this paper, which is an experiment on a double inverted pendulum system. The experimental results show that the proposed system allows users to conduct experiments in an immersive and interactive manner and provides users with a complete experimental process from principal design to experimental operation. A solution is also provided to change any virtual laboratory into a WebVR-based virtual laboratory for education and training.
Introduction
With the advancement of the Internet and the popularity of mobile devices, the demand for online education is increasing1. In particular, during periods of widespread epidemics, traditional educational institutions often face challenges in conducting in-person instruction effectively, which highlights the importance of online education as an important pedagogical approach2. Theoretical courses are relatively easy to transfer to online platforms. They can be conducted with the help of tools such as remote video conferencing software and massive open online courses (MOOCs)3. However, practical courses face greater challenges as they require users to perform hands-on experiments in traditional laboratories.
Researchers have made significant contributions to addressing the challenge of making experimental equipment available online. Over the past two decades, extensive studies have been conducted on the concepts and technologies of online laboratories4,5. Online laboratories typically encompass remote laboratories6, virtual laboratories7, and hybrid laboratories8. These online laboratory approaches have found widespread application in various engineering disciplines, including control engineering9, mechanical engineering10, and software engineering11.
While significant progress has been made in terms of the convenience of experimental operations in online laboratories12, users still perceive a lack of realism and similar hands-on practical operations compared to traditional laboratory environments, which affects their overall experience13. This discrepancy in user experience motivates further research and development efforts to enhance realism and engagement in online laboratory environments.
To solve the above problems, virtual reality (VR) technology has been applied in virtual laboratories14 to improve the immersiveness and interactivity of virtual laboratories15. VR-based virtual laboratories provide users with a close-to-realistic experimental experience. Users can complete group assignments in the architectural education process through avatars16, performing the architectural surveying process together immersively, just as they would in a traditional classroom environment. Furthermore, the VR-based virtual laboratories allow users to enter the immersive environment of virtual laboratories and interact with virtual experimental equipment by wearing VR headsets and handles17, improving users’ hands-on abilities18. For different educational purposes, we can design different virtual environments. For example, VR can be combined with gamification theory to enhance engineering education for the general public and to improve the efficiency of disseminating difficult-to-understand knowledge such as sustainable development19.
Similar to online laboratories, particularly virtual laboratories, WebVR-based virtual laboratories have many advantages. Firstly, they break through the time and space limitations of traditional laboratories, and users can conduct experiments anytime and anywhere20. Secondly, online laboratories can provide a safer experimental environment to avoid possible dangers and accidents in experimental operations21. Thirdly, virtual laboratories can also provide more experimental resources and simulation situations to extend users’ experimental scope and experience22. Most importantly, WebVR-based virtual laboratories can stimulate users’ learning interest and initiative and enhance their experimental experience and participation23.
Compared with other VR-based virtual laboratories, WebVR-based virtual laboratory seamlessly combines the merits of VR-based virtual laboratories with web-based online laboratories. Virtual Instrument Systems in Reality (VISIR)24 builds a basic analog electronic remote laboratory by constructing real circuit boards. Users can perform simulated experiments on the web interface to complete real circuit board experiments. Weblab-Deusto8 builds the water tank Field Programmable Gate Array (FPGA) laboratory where users can interact with the three-dimensional (3D) model of the water tank in the web platform without relying on other plug-ins. The system proposed in this paper introduces the capability to seamlessly integrate WebVR as a modular component into the existing virtual laboratory infrastructure. This integration can be achieved without destroying the original architectural framework of the laboratory, thus preserving the basic structure and function of the laboratory. This integration is also applicable to the framework of an online laboratory with separate front end and back end.
The system proposed in this paper is implemented based on Networked Control System Laboratory (NCSLab)25, which inherits the flexibility, interactivity, modularity, and cross-platform features of the NCSLab system. Users can conduct experiments according to different modules and can also customize algorithms and configuration interfaces, providing users with enough space for self-realization. Online experiments are driven in real-time according to the algorithms run by the user. Users can interact with the virtual model to change the inputs of the experimental algorithm when conducting VR experiments and can even change the parameters of the control algorithm through the components so that users can experience the principle of the control algorithm more realistically.
WebVR-based virtual laboratories bring great potential for online education. It can provide an immersive experimental experience, overcome the limitations of traditional laboratories, and promote hands-on practical skills and innovative thinking among users.
Protocol
Representative Results
Discussion
The presented protocol describes a virtual laboratory system that enables users to conduct VR experiments online but also uses a low-cost PC controller28, which is conducive to large-scale application promotion. Users can gain knowledge about the entire experimental process, from principles and algorithms to practical experimental operations. This system allows users to immerse themselves in the experiments, eliminating the reliance on traditional mouse and keyboard input. This system provides an …
Disclosures
The authors have nothing to disclose.
Acknowledgements
This work was supported in part by the National Natural Science Foundation of China under Grant 62103308 and Grant 62073247, in part by the Fundamental Research Funds for the Central Universities under Grant 2042023kf0095, in part by the China Postdoctoral Science Foundation under Grant 2022T150496, and in part by Wuhan University Experiment Technology Project Funding under Grant WHU-2022-SYJS-10.
Materials
| 3DS Max | Autodesk | 3ds Max professional 3D modeling, rendering, and animation software enables you to create expansive worlds and premium designs. https://www.autodesk.com/ca-en/products/3ds-max/overview |
|
| Meta Quest 2 | Meta Platforms | 10036728220341 | meta quest 2 is a standalone virtual reality headset that allows users to experience WebVR content. https://www.meta.com/it/quest/products/quest-2/ |
| Unity | Unity Technologies | Unity is the platform for real-time 3D interactive content creation and operation. All creators, including game developers, artists, architects, automotive designers, film and television, use Unity to bring their ideas to life. The Unity platform offers a complete suite of software solutions for creating, operating, and realizing any real-time interactive 2D and 3D content on cell phones, tablets, PCs, game consoles, augmented reality, and virtual reality devices. https://unity.com/cn |
References
- Martin, F., Sun, T., Westine, C. D. A systematic review of research on online teaching and learning from 2009 to 2018. Comput Educ. 159, 104009 (2020).
- Al-Nsour, R., Alkhasawneh, R., Alqudah, S. Online engineering education: Laboratories during the pandemic – A case study. 2022 Intermountain Eng, Tech Comp. , 1-4 (2022).
- Chirikov, I., Semenova, T., Maloshonok, N., Bettinger, E., Kizilcec, R. F. Online education platforms scale college stem instruction with equivalent learning outcomes at lower cost. Sci Adv. 6 (15), (2020).
- Gamage, K. A. A., et al. Online delivery of teaching and laboratory practices: continuity of university programmes during covid-19 pandemic. Educ Sci. 10 (10), 291 (2020).
- Kefalis, C., Drigas, A. Web based and online applications in stem education. Int J Eng Pedagogy. 9 (4), 76-85 (2019).
- Maiti, A., et al. A framework for analyzing and evaluating architectures and control strategies in distributed remote laboratories. IEEE Trans Learn Technol. 11 (4), 441-455 (2018).
- Liang, Y., Liu, G. Design of large scale virtual equipment for interactive hil control system laboratories. IEEE Trans Learn Technol. 11 (3), 376-388 (2018).
- Rodriguez-Gil, L., Garcia-Zubia, J., Orduna, P., Lopez-De-Ipina, D. Towards new multiplatform hybrid online laboratory models. IEEE Trans Learn Technol. 10 (3), 318-330 (2017).
- Chevalier, A., Copot, C., Ionescu, C., De Keyser, R. A three-year feedback study of a remote laboratory used in control engineering studies. IEEE Trans Educ. 60 (2), 127-133 (2017).
- Wang, N., et al. A novel wiki-based remote laboratory platform for engineering education. IEEE Trans Learn Technol. 10 (3), 331-341 (2017).
- Cruz-Benito, J., et al. Usalpharma: a software architecture to support learning in virtual worlds. IEEE Revista Iberoamericana De Tecnologias Del Aprendizaje. 11 (3), 194-204 (2016).
- Letowski, B., Lavayssière, C., Larroque, B., Schröder, M., Luthon, F. A fully open source remote laboratory for practical learning. Electronics. 9 (11), 1832 (2020).
- Potkonjak, V., et al. Virtual laboratories for education in science, technology, and engineering: a review. Comput Educ. 95, 309-327 (2016).
- Rukangu, A., Tuttle, A., Johnsen, K. Virtual reality for remote controlled robotics in engineering education. IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW). , 751-752 (2021).
- Sermet, Y., Demir, I. Geospatialvr: a web-based virtual reality framework for collaborative environmental simulations. Comput Geosci. 159, 105010 (2022).
- zacar, K., Ortakcı, Y., Küçükkara, M. Y. VRArchEducation: Redesigning building survey process in architectural education using collaborative virtual reality. Comp Graph. 113, 1-9 (2023).
- Li, Y., Lei, Z., He, Y., Liu, Z. A virtual simulation experiment of mechanics: material deformation and failure based on scanning electron microscopy. J Vis Exp. (191), e64521 (2023).
- Alsaleh, S., Tepljakov, A., Kose, A., Belikov, J., Petlenkov, E. Reimagine lab: bridging the gap between hands-on, virtual and remote control engineering laboratories using digital twins and extended reality. IEEE Access. 10, 89924-89943 (2022).
- Han, B., Weeks, D. J., Leite, F. Virtual reality-facilitated engineering education: A case study on sustainable systems knowledge. Comput Appl Eng Educ. 31 (5), 1174-1189 (2023).
- Lei, Z., Zhou, H., Hu, W., Liu, G. Concurrent experimentation in ncslab: a scalable approach for online laboratories. Fut Gen Comp Sys. (148), 139-149 (2023).
- Li, W., Huang, H., Solomon, T., Esmaeili, B., Yu, L. Synthesizing personalized construction safety training scenarios for vr training. IEEE Trans Vis Comput Graph. 28 (5), 1993-2002 (2022).
- Zhang, Y., et al. Virtual simulation system of paste filling for green mining of metallic mine. 2020 International Conference on Virtual Reality and Visualization. , 256-257 (2020).
- Callaghan, M. J., Mccusker, K., Losada, J. L., Harkin, J., Wilson, S. Using game-based learning in virtual worlds to teach electronic and electrical engineering. IEEE Trans Industr Inform. 9 (1), 575-584 (2013).
- Garcia-Zubia, J., et al. Empirical analysis of the use of the VISIR remote lab in teaching analog electronics. IEEE T EDUC. 60 (2), 149-156 (2017).
- Lei, Z., Zhou, H., Hu, W., Liu, G. Unified and flexible online experimental framework for control engineering education. IEEE T Ind Electron. 69 (1), 835-844 (2022).
- Galan, D., Chaos, D., de la Torre, L., Aranda-Escolastico, E., Heradio, R. Customized online laboratory experiments: a general tool and its application to the furuta inverted pendulum [focus on education. IEEE Cont Sys. 39 (5), 75-87 (2019).
- Lei, Z., Zhou, H., Hu, W., Liu, G., Guan, S. Web-based digital twin communication system of power systems for training and education. IEEE T Pow Syst. , (2023).
- Lei, Z., Zhou, H., Hu, W., Liu, G. Controller effect in online laboratories-An overview. IEEE Trans. Learn. Technol. 17, 1-12 (2024).
- Liu, Y., et al. Virtual reality system for industrial training. 2020 International Conference on Virtual Reality and Visualization. , 338-339 (2020).
- Lei, Z., et al. Interactive and visualized online experimentation system for engineering education and research. J Vis Exp. (177), e63342 (2021).
.