literature review for virtual reality

REVIEW article

Virtual reality and collaborative learning: a systematic literature review.

• 1 Centre for Education and Learning, Delft University of Technology, Delft, Netherlands
• 2 Leiden Institute of Advanced Computer Science, Leiden University, Leiden, Netherlands
• 3 Interactive Intelligence, Delft University of Technology, Delft, Netherlands

Background: While research on Virtual Reality’s potential for education continues to advance, research on its support for Collaborative Learning is small in scope. With remote collaboration and distance learning becoming increasingly relevant for education (especially since the COVID-19 pandemic), an understanding of Virtual Reality’s potential for Collaborative Learning is of importance. To establish how this immersive technology can support and enhance collaboration between learners, this systematic literature review analyses scientific research on Virtual Reality for Collaborative Learning with the intention to identify 1) skills and competences trained, 2) domains and disciplines addressed, 3) systems used and 4) empirical knowledge established.

Method: Two scientific databases—Scopus and Web of Science—were used for this review. Following the PRISMA method, a total of 139 articles were analyzed. Reliability of this selection process was assessed using five additional coders. A taxonomy was used to classify these articles. Another coder was used to assess the reliability of the primary coder before this taxonomy was applied to the selected articles

Results: Based on the literature reviewed, skills and competences developed are divided into five categories. Educational fields and domains seem interested in Virtual Reality for Collaborative Learning because of a need for innovation, communities and remote socialization and collaboration between learners. Systems primarily use monitor-based Virtual Reality and mouse-and-keyboard controls. A general optimism is visible regarding the use of Virtual Reality to support and enhance Collaborative Learning

Conclusion: Five distinct affordances of Virtual Reality for Collaborative Learning are identified: it 1) is an efficient tool to engage and motivate learners, 2) supports distance learning and remote collaboration, 3) provides multi- and interdisciplinary spaces for both learning and collaborating, 4) helps develop social skills and 5) suits Collaborative Learning-related paradigms and approaches. Overall, the reviewed literature suggests Virtual Reality to be an effective tool for the support and enhancement of Collaborative Learning, though further research is necessary to establish pedagogies.

1 Introduction

Beginning in the 1980s, academia has studied how to support and enhance Collaborative Learning (CL) in educational settings using technology. Referred to as Computer-Supported Collaborative Learning (CSCL), this pedagogical approach stems from social learning, an educational theory revolving around the idea that “new behavior can be acquired through the observation of other people’s behaviors” ( Shi et al., 2019 ) and focusing on social interaction between learners. CSCL’s strength appears to lie in its flexibility: by using characteristics of technology, both distant and face-to-face collaboration, as well as synchronous and asynchronous collaboration between learners, can be supported ( Stahl et al., 2006 ). As such, CSCL has been attributed numerous affordances, including joint information processing, sharing resources and co-construction of knowledge ( Shawky et al., 2014 ; Jeong and Hmelo-Silver, 2016 ).

An on-going development in the field of CSCL is the use of Virtual Reality, a technology that ‘[transports] a person to a reality (i.e., a virtual environment) which he or she is not physically present but feels like he or she is there’ ( Rebelo et al., 2012 ). These virtual environments (VEs) are shared, simulated spaces that allow distributed users to communicate with each other, as well as to participate in joint activities, making them an effective tool for remote collaboration ( Daphne et al., 2000 ). VEs tend to be highly customizable; their visual representation can be realistic (i.e., similar to reality or containing recognizable elements from reality) or abstract (e.g., three-dimensional representations of abstract concepts) depending on their purpose, making VEs adaptable for many different fields and disciplines ( Jackson et al., 1999 ; Joyner et al., 2021 ). Virtual Reality (VR), then, functions as a human-computer interface, allowing users to access these VEs through a variety of hardware, including flat-surface monitors and displays connected to desktop computers, room-sized devices called CAVE systems that project the VE onto its walls and Head-Mounted Displays (HMDs), helmets or headpieces that visualize the VE individually for each eye. In some cases, users inhabit avatars, virtual embodiments that represent their place inside the VE, though in other cases (such as the aforementioned CAVE systems, where users do not have to wear HMDs), no avatars are required for users to detect each other. Like VEs, the visual representation of avatars can be diverse: avatars can provide realistic depictions of users’ real-life appearances, but can also be visualized as something abstract, such as geometric objects or animals. Using these avatars to mediate interactions with each other, users progressively construct a shared understanding of the VE together ( Girvan, 2018 ). Of particular interest is VR’s ability to “immerse” users, providing them a sense of being inside the VE despite its non-physical, digital nature ( Freina and Ott, 2015 ). This immersion may lead to a state of presence, wherein users begin to behave inside the VE as they would in the physical world ( Jensen and Konradsen, 2018 ). Affordances of VR in education include enhancement of experiential learning ( Le et al., 2015 ; Kwon, 2019 ), spatial learning ( Dalgarno and Lee, 2010 ; de Back et al., 2020 ) and motivation and engagement among different types of learners ( Merchant et al., 2014 ; Chavez and Bayona, 2018 ). While research on VR has generally revolved around discovering its potential to support and enhance education, academics appear to agree that the field of educational use of VR lacks pedagogical practices or strategies, with little focus on how the technology should be implemented to reap its benefits ( Cook et al., 2019 ; Smith, 2019 ; Scavarelli et al., 2021 ).

VR technology has already shown potential for the field of CSCL, improving the effectiveness of team behavior, enhancing communication between group members and increasing learning outcome gains ( Le et al., 2015 ; Godin and Pridmore, 2019 ; Zheng et al., 2019 ). What makes the use of Virtual Reality for Collaborative Learning (VRCL) even more appealing for education is its diversity in hardware and, as a result, the different forms it can take depending on the setting. Whether learners interact with the VEs via display monitors, CAVE systems or HMDs, they all seem to produce positive effects such as positive learning gains and outcomes, as well as engagement and motivation for CL ( Abdullah et al., 2019 ; Zheng et al., 2019 ; de Back et al., 2020 ; Tovar et al., 2020 ).

To advance the field of VRCL, as well as to establish its benefits and affordances, several literature reviews have examined research on VRCL. For example, Muhammad Nur Affendy and Ajune Wanis (2019) , aiming to provide an overview of the capabilities of CL through the adoption of collaborative system in AR and VR, review how VEs are used for different types of collaboration (e.g., remote and co-located collaboration), with different VR hardware (e.g., eye tracking) and multiple intended uses (e.g., increasing social engagement and supporting awareness of collaboration among learners). In comparison, Zheng et al. (2019) evaluate VRCL technology affordances by conducting a meta-analysis as well as a qualitative analysis of VRCL prototypes to explore potential learning benefits; Scavarelli et al. (2021) explore a more theoretical side with the intention to produce educational frameworks for future VRCL-related research, discussing how several learning theories (e.g., constructivism, social cognitive theory and connectivism) are reflected in prior research on the potential of VR as well as Augmented Reality (AR) for social learning spaces.

Together, the literature reviews of Muhammad Nur Affendy and Ajune Wanis (2019) , Zheng et al. (2019) ; Scavarelli et al. (2021) describe a general optimism towards VR in educational settings to support collaboration. The reviews outline VRCL’s strengths as 1) its ability to enhance learning outcomes, 2) its potential to facilitate learning, 3) its effectiveness in supporting remote collaboration between learners, as well as experts and novices, 4) its support for interpersonal awareness between collaborating learners and 5) its diversity, both in terms of its customizability (allowing VEs to better suit objectives) as well as its technology. Affordances of VRCL are identified as 1) social interaction (strengthened by VR’s affordances of immersion and presence), 2) resource sharing (strengthened by VR’s ability to present imaginary elements) and 3) knowledge construction (supported by the two prior affordances of VRCL). Furthermore, challenges and gaps related to (research on) VRCL are outlined. First, accessibility should be considered a primary concern according to Scavarelli et al.,; this does not just relate to the technical accessibility of VR when used in education, but more so to the accessibility of social engagement between learners sharing these virtual learning spaces. Second, they recommend to explore the interplay and connectivity between VEs and the real world, as doing so could reveal new learning theories that innovate VRCL. Third, Zheng et al., suggest that research focus on pedagogical strategies involving VRCL, including how to apply VR to educational settings involving collaboration. Fourth, they propose a focus on finding a balance between using VRCL to recreate (or simulate) existing (“real”) situations and creating new situations that would normally be impossible, considering that prior work has primarily been centered on the former and as such misses out on VR’s potential to do the latter.

Considering that remote collaboration and distance learning, especially since the COVID-19 pandemic, are becoming increasingly important for learners, an understanding of VR’s potential for CL could prove beneficial for the field of education. While research on the topic is apparent, studies focusing on VR’s ability to support and enhance CL are still small in scale ( Zheng et al., 2019 ; Scavarelli et al., 2021 ), accentuating the scarcity of knowledge on the topic. This systematic review specifically centers on scientific research on VRCL, with a particular focus on the empirical knowledge that such literature has established. The aim of this paper is to examine in what ways VR supports and enhances CL according to prior research on these topics; to achieve this, it reports on what VRCL is used for in different fields of education, discusses what research has stated regarding VRCL in terms of affordances and benefits for education, describes the characteristics of VRCL that allow these benefits to come to fruition and provides an insight into the technology behind VRCL, as well as how this compares to the state-of-the-art of VR. In doing so, this study intends to identify possible gaps in the field of VRCL research for possible future studies, in addition to highlighting VRCL’s strengths to support current research. To the best of the authors’ knowledge, this study is the first systematic review on the topic of VRCL. As a means to provide the relevant information, this review addresses the following four research questions.

1. What skills and competences have been trained with use of VRCL (and what should a VRCL environment provide to train these)?

2. What domains and disciplines have been addressed (and why)?

3. What systems have been developed and/or established?

4. What empirical knowledge has been established (and with what methods and/or study designs)?

This section discusses the process of collecting the relevant studies for this literature review. In particular, the inclusion and exclusion criteria, databases and methods used are described.

2.1 Identification

The systematic review used two databases: Scopus and Web of Science. The search query contained the following key elements: 1) collaborative interaction, 2) VR, 3) education, training and learning, 4) simulations of a three-dimensional nature, 5) empirical data and 6) the use of a system (application or prototype). As such, the following search string was used in both databases:

[collaboration OR cooperation OR collaborative OR cooperative OR collaborate OR cooperate] [AND] ["virtual reality” OR “mixed reality” OR “extended reality"] [AND] ["3D” OR 3d OR 3-D OR 3-d OR threedimension* OR three-dimension* OR “three dimension*" OR CGI OR “computer generated” OR “computer-generated” OR model* OR construct*] [AND] [evaluat* OR data OR result* OR observ* OR empiric* OR trial* OR experiment* OR significan* OR participant* OR subject*] [AND] [education OR training OR learning OR university OR school OR vocational] [AND] [system* OR prototyp* OR application* OR program*]

To be considered suitable, papers had to meet five specific inclusion criteria. Firstly, an article had to discuss collaborative or cooperative interaction between human users of a virtual, three-dimensional simulation. Secondly, the article had to include and discuss Virtual-, Augmented-, Mixed Reality (MR) or Extended Reality (XR) as a three-dimensional simulation of a physical space or object(s). While this review focuses on VR for CL, mediums such as AR, MR and XR were included in this search for two reasons. On the one hand, definitions for these mediums appear to overlap to such an extent (with some even considering them too vague and ambiguous ( Tovar et al., 2020 )) that ‘pedagogical advantages of either technologies are [considered] comparable’ ( Sims et al., 2022 ). On the other hand, the mediums in question do not always get defined as separate ones, but rather as different points on one spectrum, commonly referred to as the virtuality continuum, in which ‘“reality” lies at one end, and “virtuality” […] at the other, with Mixed Reality […] placed between’ ( Scavarelli et al., 2021 ). As such, the decision was made to include these mediums, so as to ensure that no pedagogical advantages of VR would be excluded. The third inclusion criterium required an article to include an empirical study (i.e., containing qualitative or quantitative data) for it to be considered suitable. For the fourth and fifth criteria, an article had to contain an educational objective or goal (for human entities) and discuss a system used for educational purposes (for human entities) in order to be eligible.

Additionally, studies would be disqualified from the literature review if they 1) only described a patent, 2) only contained a summary (review) of a conference, 3) only consisted of a literature review, 4) were not accessible to the authors of this study, 5) were not available in English, 6) were a duplicate or a version, edition or release of an older study that already had been included or 7) did not specifically state the number of participants of any experiment involved in the study.

The search query resulted in 1,058 publications for Scopus and 845 studies for Web of Science, resulting in a total of 1,608 studies after duplicates were removed. Using the inclusion and exclusion criteria to filter out ineligible articles (initially based on title and abstract, then on full text), this review resulted in 139 articles analyzed. Results and details of the process (which followed the guidelines of the PRISMA method ( Moher et al., 2009 )) can be seen in Figure 1 . Appendix A shows the complete list of all 139 articles.

FIGURE 1 . Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) diagram of the screening process.

To examine reliability of the selection process, five additional coders screened a random sample of 50 studies individually (10 per coder) using the inclusion and exclusion criteria. After comparing and discussing results, inter-rater reliability (between the first coder and the five coders) was calculated using a Kappa-metric, resulting in a moderate level of agreement of 0.77 ( McHugh, 2012 ) (results can be found in Supplementary Table B1 ).

A taxonomy ( Figure 2 ) was created to help classify all 139 articles. With this review’s research questions in mind, three vital topics were established to function as main categories for the coding process: education, system and evaluation (illustrated in column C1 in Figure 2 ). For RQ1 and RQ2, the first category, education, was established to extract information from the articles, concentrating on six classes. Similarly, information necessary to answer RQ3 was collected by coding attributes related to the second category, system, which included eight classes. Focusing the coding on elements related to the third category, evaluation (with five classes), allowed for extraction of relevant information required to answer RQ4. After the relevant categories, classes (visible in column C2 in Figure 2 ) and attributes (visible in column C3 in Figure 2 ) were decided upon, the classification hierarchy in Figure 2 was constructed, partially based on scientific literature ( Bloom et al., 1956 ; Schreiber and Asnerly-Self, 2011 ; Motejlek and Alpay, 2019 ), to provide assistance during the coding process. For an in-depth description of the motivation behind this classification hierarchy, please see Supplementary Appendix C . While the required information for some of these attributes could easily be inferred directly from each study, other attributes required the first coder to deduce which attributes were applicable.

FIGURE 2 . Classification hierarchy used for coding, including percent agreement ( p a ) and Cohen’s kappa (K) between first and second coder on the right.

To assess reliability of the first coder, a second coder classified articles with the taxonomy ( Supplementary Table D1, D2, D3 ). Inter-rater reliability between the two coders for 30 randomly selected studies was 0.60 (with a percent agreement of 0.85), considered a moderate level of agreement ( McHugh, 2012 ). Additionally, Figure 2 shows the inter-rater reliability for each individual class.

3 Descriptive results

In this section, discussion of descriptive results is divided into three sections according to the structure of the taxonomy. An overview of all results (according to the taxonomy) can be found in Figure 3 .

FIGURE 3 . Results of coding of data found in the literature, according to the taxonomy.

3.1 Education

As a first dimension, elements related to education were analyzed. A majority of the selected articles focused on VRCL in tertiary education (i.e., university), discussing possible uses for students. Educators providing support (e.g., scaffolding) for learners proved most prominent, though not all studies discussed this topic. While a wide selection of educational domains were discussed, computer sciences and social sciences were the most popular fields. Most studies specifically focused on synchronous collaboration. Prevalent among learning paradigms and educational approaches were problem-based learning (PBL) and constructivism. The specific results related to this dimension are found in Table 1 .

TABLE 1 . Distribution of Education-related attributes.

In contrast to the high number of articles focusing on tertiary education (64.0%), primary education was central in 10.8% while 5.0% discussed VRCL in secondary education. A small percentage of studies (6.5%) focused on types of learners outside of formal education (e.g., on-the-job training). In relation to the educators, a little over half of the studies reported on educators supporting the learners by providing varying degrees of scaffolding (55.4%). For 20.9% of cases, educators provided presentations and lectures inside the VE, providing a more passive learning experience. On a broader scope, the studies showed a wide variety of educational domains and fields of expertise to which VR was applied. While approximately a quarter of studies reviewed (25.9%) reported use of VRCL for education, specific domains that were often discussed included computer science, robotics, ICT and informatics (12.2%), social sciences (11.5%) medical fields (9.4%) and engineering (8.6%).

Also shown in Table 1 is the appearance of different types of social learning: 62.6% of studies reviewed discussed synchronous (collaborative) interaction, while in comparison a much lower 18.0% discussed asynchronous (cooperative) interaction. For a 10th of the studies, an expert-novice type of social learning was apparent (9.4%). On the topic of educational approaches and learning paradigms, 29.5% of articles did not seem to discuss any specific approaches. Among those that did, constructivism and PBL were featured substantially (33.1% and 41.0%, respectively), while paradigms such as experientialism, situated learning and distributed cognition were discussed less frequently. Other educational approaches, discussed in 35.3% of articles, included self-regulation and shared regulation (e.g., Al-Hatem et al., 2018 ) as well as cognitive apprenticeship (e.g., Bouta and Retalis, 2013 ). Looking at the learning goals and outcomes, the cognitive domain proved to be popular (50.4%), whereas affective and psychomotor domains were featured much less (7.9% and 5.0%, respectively). Other goals and outcomes included general student engagement (discussed in 31.7%) and support of collaboration amongst learners (60.4%).

The second dimension took a closer look at systems used in the studies, including aspects related to the hardware used (e.g., devices, types of control) as well as users’ interaction with VEs (e.g., degree of virtuality, virtual embodiment). A majority of the studies reviewed did not use VR technologies such as HMD-based VR (HMD VR), but instead focused on monitors and displays when discussing VRCL. Most studies chose general purpose controls (e.g., mouse and keyboard) over more advanced hardware such as positional tracking. A majority of studies provided their participants with full-body embodiment (e.g., avatars) and the ability to manipulate virtual objects while inside the VEs. Approximately a quarter of studies used systems for edutainment purposes (i.e., learning by having fun), while system use for training or therapeutic purposes was less common. Table 2 shows these results in detail.

TABLE 2 . Distribution of System-related attributes.

Results showed a clear preference for 3D (non-HMD) simulations, i.e., a virtual simulation of a (physical) environment projected on a surface or display that is not a Head-Mounted Display (and, as such, is considered less immersive): this degree of virtuality was far more prominent in the reviewed studies (78.4%) compared to the lesser implemented AR/MR (16.5%) and HMD VR (7.2%). The hardware used in these studies reflected this: a large amount (89.2%) implemented flat-surface monitors and displays to present VRCL environments. These studies commonly used desktop computer set-ups that included a keyboard, mouse and monitor, though in the case of AR and MR, surface-based mobile devices were often used. When using the system in a larger setting (i.e., larger group size), studies utilized projector-based (but still flat) surfaces to display the VE (e.g., Bower et al., 2017 ). In some cases, several types of these flat-surface displays were being used in different phases of a study (e.g., Nuñez et al., 2008 ). Cases that used CAVE systems (3.6%) included ImmersaDesks, CAVE-like devices that derive from the original CAVE systems. Studies that involved HMD VR used devices like the Oculus Rift and HTC Vive, while studies revolving around AR and MR implemented devices like the HoloLens. Some studies involved multiple devices to compare effects based on the difference (e.g., monitor-based vs HMD VR, as discussed in Vallance et al., 2015 ) while others discussed implementation of HMD VR and AR-related devices as possible future directions without using these in their experiments. With regard to user interaction, studies that implemented general purpose controls used simple computer keyboard and mouse, though some cases also involved video game controllers such as the Nintendo Wiimote and Nunchuck ( Li et al., 2012 ). Apart from the more default specialized controls such as 3DoF and 6DoF controllers or mobile device-based touch screens, studies also discussed a wide variety of other tools in this category, including multi-touch tabletops, haptic feedback devices, Xbox Kinect and gesture-sensing data gloves. While scarce, gaze control and positional tracking (15.1% and 11.5%, respectively) was primarily found in studies that used (mobile-based) AR and HMD VR, though some studies also provided these through devices such as the HoloLens or as part of a CAVE system.

Of the studies examined for this review, 55.4% discussed (self-developed) prototypes, while 44.6% used (pre-existing) applications. The most prominently-mentioned engine for prototypes was Unity, with % (of 77 studies) using it. Concerning the ones that used applications (62 of 139), more than half discussed VE application Second Life (%), while open-source VEs OpenSimulator and Open Wonderland were used in smaller numbers (% and %, respectively). In regard to the intended function of systems used, the majority of articles described a strictly educational one (58.3%) and revolved around implementing these systems in educational contexts as well as using them to facilitate collaborative learning. Studies that used systems to both educate and entertain (22.3%) tended to focus on game-based learning and serious games, though some cases also discussed video games originally not intended for educational purposes (e.g., World of Warcraft ( Kong and Kwok, 2013 ), Minecraft ( Mørch et al., 2019 )). When training purposes were mentioned (17.3%), this often indicated the use of VEs to train specific expertises, such as liver surgery or aircraft inspection. Rare cases where a system was used for therapeutic purposes (just 2.2%) included use of VRCL to teach social skills to patients with autism ( Ke and Lee, 2016 ) or to train physical activities amongst elderly ( Arlati et al., 2019 ).

Motivation behind studies’ choices for the size of collaboration differed between experimental reasons (e.g., a limited number of participants), pedagogical reasons (e.g., using pairs to better stimulate personal social interaction between members compared to larger groups) and reasons related to the systems (e.g., limited hardware availability). Small groups proved to be the most used group size, with 37.4% describing groups of between three and nine members. Pairs were used in 22.3% of studies. Motivations behind pairs included focus on expert-novice interaction and system capabilities (e.g., support for two users maximum). Articles that described larger groups (ten or more members) generally had entire classes of learners interact with system (15.1%).

Apart from a small number of studies that did not provide sufficient information on the matter, virtual embodiment of the users was featured prominently. In cases where physical attributes were virtually represented by (imagery of) tools (18.0%), the VRCL environment was often implemented for specific training of certain expertises. In general, partial virtual embodiment appears in first person, HMD VR (for example, when only the user’s hands are made visible); while scarce (3.6%), studies that displayed partial virtual embodiment provided some interesting examples outside of HMD VR. Examples of partial embodiment included a detailed 3D face to focus on emotional and social expressions ( Cheng and Ye, 2010 ) and using controllable, flat-surfaced rectangles in a 3D environment on which users’ real-life faces were projected via webcam ( Nikolic and Nicholls, 2018 ). Full-body embodiment proved to be the most popular, with 67.6% of studies using systems that provide users complete (full-body) virtual representation. To a degree, the relatively high number of studies that present full-body embodiment can be explained by the systems that were implemented; applications such as Open Simulator and Second Life provide users with customizable avatars, making a full-body virtual embodiment a default feature. In some cases, however, studies specifically examined the effects of virtual embodiment, such as Gerhard et al. (2001) examining possible influences of different avatars on users’ sense of presence. On the topic of user influence on VEs, a little more than half the studies (53.2%) used systems that allowed (some degree of) virtual object manipulation, whereas approximately a quarter of the studies (26.6%) also provided users the tools to manipulate actual content of the VRCL environment. In 16.5% of studies, the system only allowed users to be visibly present inside the VE, while only 3.6% did not provide sufficient information on the matter.

3.3 Evaluation

For the third dimension, the selected articles were analyzed on how they evaluated applying VRCL. Articles frequently concentrated on evaluation of the system(s), with a higher number of them using self-report evaluation methods. Study design of the studies shows a similar result: pre-experimental study design (typically used for preliminary testing of systems) was regularly implemented, with surveys being a popular method of collecting data. While the number of participants was diverse, roughly half of studies reviewed used a sample size between 1 and 25 participants. The majority of articles discussed positive outcomes, whereas only a small amount featured negative results. Detailed results are displayed in Table 3 .

TABLE 3 . Distribution of Evaluation-related attributes.

The majority of studies focused on evaluating a system’s effectiveness when using it in educational settings (71.2%). These studies concentrated on the system’s capacity to support collaboration between learners. Other topics of discussion were student interest in the system and how the system can facilitate learning. Whenever studies examined processes (34.5%), evaluation would be centered around attempts to understand how group interaction materializes in these environments. This included how learners resolve social conflicts ( Cheong et al., 2015 ) and examining how co-presence (e.g., Kong and Kwok, 2013 ) and PBL take shape in VRCL environments. 35.3% of articles discussed learning outcomes after participants interacted with the system. The few situations where the above three attributes did not apply (3.6%) included a study that aimed to develop design guidelines ( Economou et al., 2001 ) and a study primarily interested in the teacher’s role when learners interact with VEs ( Lattemann and Stieglitz, 2012 ).

Most studies collected self-reported data from their participants (85.6%), while over half used behavioral methods to obtain tracking and observational data (59.0%). Articles that reported on knowledge- and/or performance-based assessments (20.9% of studies) often used pre- and post-tests to acquire their data, while only one appeared to use physiological data, tracking participants’ heart rate (0.7%). A notable number of articles (79.9%) implemented pre-experimental design in their studies. Some of these were case studies, applying VEs to educational settings (e.g., Terzidou et al., 2012 ), while others performed pilot studies to establish a first impression of the effects of a system on specific pedagogical situations (e.g., examining how VE-based application OpenSimulator influences Transactive Memory Systems amongst learners ( Kleanthous et al., 2016 )). Quasi-experimental- (13.7%) and true experimental designs (5.8%) were used scarcely, while only 2 out of 139 studies (1.4%) performed an experiment with single-subject design. With respect to non-experimental and descriptive designs, 84.9% of studies implemented a survey-based design, whereas a little over half used observational designs to collect data (56.1%). In some cases, comparative and correlation designs were implemented (7.9% and 15.8%, respectively).

Table 3 also reveals that approximately half of the studies sampled between 1 and 25 participants (53.2%), while around a quarter (26.6%) used a sample size between 26 and 50 participants. For 13.7% of articles, between 51 and 100 participants were used, whereas only 6.5% discussed using more than 100 participants for collecting data. In terms of outcomes, around half of the studies concluded that their system(s) seemed positive and promising (53.2%), while 17.3% draw positive conclusions based on significant outcomes from statistical hypothesis testing. Negative outcomes were scarce, with only 2.2% of the studies reporting negative results. Mixed outcomes were reported for 7.2% of the studies, whereas 20.1% discussed results that were inconclusive, showed no effect or reported outcomes on which positive and negative effects are not applicable.

4 Qualitative results

In general, the literature reviewed for this paper shows a positive attitude towards the use of VR to support and enhance CL. However, the results quickly make it apparent that the methods of applying VR to educational fields to support and enhance CL can vary greatly amongst the studies examined here. In order to acquire a general understanding how these studies have attempted to support and enhance CL using VR, this section will discuss qualitative results established. The rest of this section will be divided into sub-sections, each focusing on discussing results related to one of the four research questions of this literature review.

4.1 Skills and competences trained with VRCL

A number of elements can be identified regarding skills and competences trained with VRCL. Based on the skills and competences discussed in the reviewed literature, five categories were established for this study with the intention to provide a concise overview. These categories, including examples of each category, can be viewed in Table 4 .

TABLE 4 . Skills focused on in the reviewed literature.

For the types of skills and competences shown in Table 4 to be trained effectively, a VRCL environment requires a number of features that support the learners in learning these abilities. Based on the information provided by the reviewed literature, nine required features and design parameters of VRCL can be identified. First, virtual embodiment plays an important role in how learners view themselves and each other inside the VE, impacting learning outcomes and collaborative behavior by providing a sense of awareness and belonging ( Edirisingha et al., 2009 ; McArdle and Bertolotto, 2012 ). Second, efficient communicational tools are essential for effective collaboration: verbal (audio) communication is crucial ( Economou et al., 2001 ; De Pace et al., 2019 ), though additional modalities such as haptic technology can further enhance collaboration ( Moll and Pysander, 2013 ). Third, usability and accessibility should be taken into consideration: VRCL systems should be accessible to all levels of technical skills as differences negatively affect group cohesion and learning between group members (Y. Chang et al., 2016 ; Denoyelles and Kyeong-Ju Seo, 2012 ). Fourth, learners’ perceived usefulness of the VE also affects group cohesion; factors such as awareness, presence and social presence appear to significantly influence this perceived usefulness ( Denoyelles and Kyeong-Ju Seo, 2012 ; Yeh et al., 2012 ). Fifth, the ability to interact with elements inside the VE are considered key: to optimize learning outcomes, learners must have the option to manipulate elements inside the VE (e.g., virtual objects or virtual tools) in a seemingly natural and intuitive way ( Vrellis et al., 2010 ; Bower et al., 2017 ). Sixth, academic efficacy can be achieved if tasks inside the VE are designed around its educational, collaborative objectives, especially when designed for equal input from all learners in a group ( Wang et al., 2014 ; Nisiotis and Kleanthous, 2019 ). Seventh, educators should be ready to provide support, motivation and moderation of collaboration while learners interact inside the VE ( Lattemann and Stieglitz, 2012 ; Bower et al., 2017 ). However, the eighth feature, a level of autonomy, is equally important for each individual learner, not just in terms of independence from the educators, but more importantly from each other, as this allows them to provide different points of views as well as to explore multiple representations, thus improving CL ( Hwang and Hu, 2013 ). Ninth, implementation of VRCL should make sure to primarily support socialization inside the VE, as underestimating the importance of socialization might lead to features of VR obstructing rather than facilitating CL ( Chang et al., 2009 ).

Surprisingly, only a small number of the literature reviewed focused on goals related to the affective domain (7.9%). With some calling VR the “ultimate empathy machine” ( Rueda and Lara, 2020 , p.6), the medium’s ability to induce emotions has been prominently discussed and studied. Not only has VR been shown to indeed be capable of enhancing empathy amongst users ( Herrera et al., 2018 ), with some even arguing it to be more effective than traditional empathy-shaping methods ( Liu, 2020 ), studies have also suggested it to be an effective tool to offer a uniquely different level of understanding ( de la Peña et al., 2010 ). This would suggest that VR’s ability to create a better understanding of different group members’ points of view could in turn support collaboration between learners.

Similarly, even less literature reviewed focused on goals related to the psychomotor domain (5.0%). Prior studies have been positive and hopeful regarding VR to expand the possibilities of physical training ( Pastel et al., 2020 ). Interestingly, technical features such as positional tracking even seem to be effective in predicting psychomotor outcomes ( Moore et al., 2021 ), which could prove useful for domains that specifically focus on expert-novice training in primarily physical tasks (e.g., certain types of engineering). However, positional tracking, not unlike psychomotor outcomes, is only discussed sparingly (11.5%) in the literature reviewed.

An interesting observation in relation to the evaluation methods used in the scientific literature is that only 1 out of 139 articles used physiological measures. As suggested by research, physiological synchrony between group members can serve as an effective indicator for the quality of interpersonal interaction between them (with a higher physiological synchrony correlating with a higher interaction level) ( Liu et al., 2021 ). Furthermore, physiological measurements can be used to identify multiple predictors related to education and training, including the quality of collaboration between group members ( Dich et al., 2018 ). Additionally, visualizing physiological results of each member of a group to the others in real-time during collaboration has shown to have a positive effect on the empathy levels and cohesion of the group, further suggesting how collaboration between learners could benefit from physiological measures ( Tan et al., 2014 ). Considering VR’s visual characteristics as well as research arguing that physical signals such as electroencephalogram (EEG) can conveniently and unobtrusively be tracked during use of HMD VR ( Tremmel et al., 2019 ), future research on VRCL could prove fruitful in terms of training collaborative skills and competences via use of physiological-based information.

4.2 Disciplines focused on regarding VRCL

When looking at the most prominently-featured domains in the literature reviewed (as shown in Figure 4 ), examining what motivated researchers to study VRCL in the field of 1) education, 2) computer science, robotics and informatics and 3) social sciences can provide an understanding of VRCL’s role in these different disciplines.

FIGURE 4 . Results of the educational disciplines focused on in the reviewed literature.

For the field of education, some studies focus on the potential behind VRCL, intending to discover what it can mean for the development of cognitive and technical skills ( Franco and De Deus Lopes, 2009 ). Other studies focus on possible learning gains, examining how knowledge gained in VEs transfers to the real world (i.e., how learners apply outcomes in VEs to situations in actual reality) or attempting to facilitate this transfer by implementing elements of both ( Carron et al., 2013 ). In certain cases, articles specifically examine VEs’ effects on collaboration and how VR can be used to reinforce CL (e.g., Tüzün et al., 2019 ), whereas others aim to determine if existing educational paradigms such as constructivism can be applied to VRCL environments and, if so, how that affects group knowledge gain between learners ( Girvan and Savage, 2010 ). Together, these studies present a general motivation to discover what VRCL can mean for education and where its potential may lie.

For computer science, robotics and informatics, use of VRCL can be summarized in two motivations: 1) innovate these domains and 2) create a learning community. In the first case, researchers intend to utilize the affordances VRCL environments have to offer to further advance fields such as computer science, which have been criticized in the past for using two-dimensional learning platforms and oral-based teaching methods ( Pellas, 2014 ). With VEs, educators can provide learners realistic yet illusionary worlds that are flexible, customizable and even allow for detailed statistics on learners’ performance ( Champsas et al., 2012 ). In the second case, reviewed articles vocalize a desire to use VRCL to provide learners purposeful collaborative activities that create a sense of belonging to a learning community, using aspects such as awareness, presence and different methods of communication to motivate learners in these fields to work together closely ( De Lucia et al., 2009 ).

In similar fashion, social studies appears to be interested in how socialization between learners is manifested inside VRCL (e.g., Edirisingha et al., 2009 ). Some articles go further, studying how VRCL can support socialization: Molka-Danielsen and Brask (2014) suggest that presence, awareness and belonging allow for communication, negotiation and trust between learners, elements deemed necessary for completing collaborative tasks. Other studies focus on specific characteristics of socialization, such as how gender could affect social interaction and group cohesion inside VEs ( Denoyelles and Kyeong-Ju Seo, 2012 ). Collectively, these articles show a desire to understand how elements related to socialization transfer to VRCL, as well as how these environments can sustain and even enhance those elements.

4.3 Systems developed and/or established for VRCL

The results related to systems used show that there is quite a disparity between use of HMD VR and that of non-HMD VR. Almost 80% of systems implemented non-HMD VR, with AR/MR and HMD VR implemented far less frequently (16.5% and 7.2%, respectively, as illustrated in Figure 5 ). Almost 90% of studies described the use of flat-surface monitors and displays, which, when compared to the 10.8% of studies that used HMD devices, further highlights the low use of HMD VR in the literature reviewed (see Figure 6 ).

FIGURE 5 . Results of the degree of virtuality of systems discussed in the reviewed literature.

FIGURE 6 . Results of the hardware used in the reviewed literature.

The lack of representation of HMD VR in these articles is somewhat surprising, considering this type of virtuality and hardware is commonly associated with the medium of VR ( Dixon, 2006 ; Bonner and Reinders, 2018 ; Jing et al., 2018 ). The statement that research into application of VR to the field of education lacks a focus on HMD VR, however, is not uncommon ( Sousa Santos et al., 2009 ; Scavarelli et al., 2021 ), thus begging the question: why is it underrepresented in the reviewed literature?

One possible explanation could be that HMD VR is known to be difficult to apply to educational settings because of its high costs ( Olmos et al., 2018 ). Some of the articles analyzed for this review were published in the late 90s; while HMD VR technology was already available in those times, devices were more expensive and less technologically advanced compared to the technology that is available now ( Mehrfard et al., 2019 ; Wang et al., 2022 ). Furthermore, the technical skills necessary to implement VR properly in educational settings can prove challenging ( Jensen and Konradsen, 2018 ). Since collaboration involves multiple people, difficulties related to accessibility could be more severe when applying VR to a larger group of learners. Another possible reason is the health risks associated with the technology: HMD VR is often connected to motion sickness and cybersickness ( Olmos et al., 2018 ; Yoon et al., 2020 ). A third reason refers to the general lack of pedagogy on the topic of HMD VR: while the medium’s potential for education is often discussed, general guidelines as to how it should be applied efficiently to educational settings ( Cook et al., 2019 ; Zheng et al., 2019 ) as well as an understanding of how learning mechanisms operate inside VR environments ( Smith, 2019 ) are missing. Naturally, the small size of research done on VR and CL exacerbates this lack even further when specifically discussing VRCL. A possible fourth reason that is more closely tied to this particular literature review is that, despite its popularity in research, HMD VR appears to still lack empirical evidence of its educational value ( Sousa Santos et al., 2009 ; Makransky et al., 2019 ; Radianti et al., 2020 ), which, considering this review’s focus on empirically-based knowledge, could explain its scarcity.

The low representation of HMD VR and high representation of non-HMD VR could be related to the ongoing discussion about what defines VR and how it differs from VEs, as discussed in-depth by Girvan, (2018) . Girvan argues that some use terms synonymously with VR and/or VEs, while others use these same terms to classify different types of VEs, thus creating a fragmented understanding of what these are (and what they are not). Girvan’s point is reflected in the reviewed literature of this paper: while some studies identify Second Life as a “virtual environment” or “virtual world” (e.g., Terzidou et al., 2012 ), others refer to it as “virtual reality” (e.g., Sulbaran and Jones, 2012 ). To prevent further confusion with technologies with similar technical features, Girvan suggests to conceptualize VEs as ‘shared, simulated spaces which are inhabited and shaped by their inhabitants, who are represented as avatars [that] mediate our experience of this space as (…) we interact with others, with whom we construct a shared understanding of the world at that time’. VR, then, should be defined as ‘a technical system through which a user or multiple users can experience [such] a simulated environment’ ( Girvan, 2018 ).

Apart from causing a fragmented understanding of the terms in the literature, different interpretations of VR and VEs also lead to HMD VR and non-HMD VR being described as one and the same thing under the moniker of “virtual reality”. Though this may seem a trivial dispute about labels, treating these two types as identical will lead to misconceptions regarding both, as HMD and non-HMD VR contain different benefits and limitations when applied to education. While some studies showed no differences between the two in terms of specific learning outcomes (e.g., spatial- ( Srivastava et al., 2019 ) and language learning (J. Y. Jeong et al., 2018 )), other research highlighted several differences between HMD and non-HMD. Compared to non-HMD, HMD VR has shown to provide a much higher sense of embodiment, which in turn is hypothesized to lead to higher performances, in particular in psychomotor skills ( Juliano et al., 2020 ; Saldana et al., 2020 ). Similarly, HMD VR appeared superior to computer screens in terms of arousal, engagement and motivation in learners ( Makransky and Lilleholt, 2018 ). In contrast, however, Makransky et al. (2019) reported overloads and distractions caused by HMD VR, leading to poorer learning outcomes compared to non-HMD, a sentiment shared by Parong and Mayer (2021) , who described HMD VR to cause high affective and cognitive distractions. Amati and McNeill (2012) even argue that the difference between HMD and non-HMD VR (and in particular how the two are interacted with by users) have severe implications for teaching and practice.

With all of the above in mind, the low representation of HMD VR in the literature examined for this review can be interpreted in two ways. On the one hand, the underutilization underlines that HMD VR is not being used to its full potential and could very well hold much more promise for the field of education and CL. On the other hand, the low use of HMD VR could suggest that implementation of HMD VR in education and/or CL is, in fact, not worth the trouble it brings with it. Whether HMD VR is a benefit or a burden, then, arguably depends on three important elements: 1) the goals (i.e., what skills and/or competences are supposed to be trained), 2) the setting (i.e., the disciplines and fields to which it is applied), and 3) the affordances of VRCL (and to what degree these conform to the goals and setting).

4.4 Empirical knowledge established regarding VRCL

When summarizing the outcomes of the 139 articles, 70% of the studies reviewed displayed a positive attitude towards the application of VRCL to education. While a relatively low number (approximately 25%) presented statistically significant outcomes, this does illustrate a strong optimism amongst those studying VRCL environments in different fields of education as described in prior literature reviews on the topic. This could also explain the high number of studies that deployed pre-experimental study designs: with VRCL being a relatively new addition to the world of CSCL, as well as one that continues to rapidly advance because of the technology behind it, many seem enthusiastic and eager to see what promises VRCL holds when used in different fields and with different types of learners.

Regarding affordances discussed in the reviewed literature, several features are identified. First, VRCL appears an efficient tool to engage learners and to motivate them to study and learn. The ability to customize VRCL environments and their content provides learners more personalized experiences that better suit their personalities and attitudes, thereby enhancing the motivation to learn on both an individual and group level ( Arlati et al., 2019 ). Furthermore, VRCL’s immersive qualities tend to make the experiences more engaging for learners, encouraging them to engage in presentations and demonstrations as well as to communicate and collaborate with each other ( Avanzato, 2018 ).

The second affordance identified VRCL as a great tool for distance learning and remote collaboration. VEs provide a method for learners and educators to work together and collaborate despite distances. In comparison to other media, however, VRCL brings with it a high sense of immediacy (i.e., ‘verbal and non-verbal behaviors that give a sense of reduction of physical and psychological distance between the communicators’), which in turn presents an increased perception of learning ( Edirisingha et al., 2009 ). Additionally, VRCL’s immersive qualities and high presence allow for environments capable of simulating training as preparation for real-life experiences ( Al-Hatem et al., 2018 ) that simultaneously promote active participation and social interaction ( Mystakidis et al., 2017 ) in a setting that feels personal despite distances between learners ( Desai et al., 2017 ). In certain cases, such as education for learners with physical disabilities, learners and educators even considered connectivity to be more accessible and easier than real-life equivalents ( Aydogan and Aras, 2019 ), illustrating that VRCL environments can potentially go beyond simply being a replacement. To effectively support the distance learning and remote collaboration, however, design of the VEs should focus on providing learners a sense of 1) presence, 2) awareness and 3) belonging to the group ( Molka-Danielsen and Brask, 2014 ).

Thirdly, the literature reviewed suggests that VRCL environments are effective spaces to support multi- and interdisciplinary learning and collaboration. The ability to customize VEs, adapting to suit users’ needs, prevents them from being restricted to just a single specific subject field. This in turn allows educators to change the environments to accommodate many different subject fields and topics so as to make sure that learners from different backgrounds can collaborate with each other undisturbed ( Bilyatdinova et al., 2016 ). Moreover, it seems that VRCL environments made some of the literature studies reviewed realize the importance of interdisciplinary collaboration in the learning process ( Franco et al., 2006 ; Nadolny et al., 2013 ).

The fourth affordance identified might be an unsurprising but nonetheless important one: VRCL seems to be a tool for the development of social skills. While identity construction and projection through virtual embodiments can be complex for learners (depending on their technical skills), VRCL is found to facilitate social presence and foster socialization ( Edirisingha et al., 2009 ). VRCL’s customizability allows learners to integrate personal preferences and identity expressions into processes inside the environment (e.g., through their virtual embodiments), in turn mediating identity and norm construction for real-life social settings ( Ke and Lee, 2016 ). Vital social skills, such as the ability to identify and manipulate basic emotional states, can be taught and trained using VEs, improving learners’ socialization, communication skills and emotional intelligence ( López-Faican and Jaen, 2020 ). Learners’ prior experience with VEs, however, should not be underestimated, as a difference in familiarity with VRCL environments has been shown to impact collaboration ( Bluemink et al., 2010 ).

Fifth, VEs appear fitting for CL-related learning paradigms and educational approaches. Some studies specifically focus on examining to what degree VRCL environments are applicable to paradigms such as constructivism, socio-constructivism and constructionism (e.g., Girvan and Savage, 2010 ; Pellas et al., 2013 ; Abdullah et al., 2019 ), concluding that these indeed go well together. Other studies, however, focus on theories and methods commonly associated with these paradigms. In particular, experiential learning and PBL seem appropriate for VRCL environments. VEs allow for safe, consequence-free learning for exploring, experiencing and practicing without any real-life risks ( Cheong et al., 2015 ; Le et al., 2015 ), making it suitable for experiential learning. Moreover, VRCL’s immersive qualities seem to support and even elevate experiential learning strategies such as roleplay and improvisation, providing learners close to real-world experiences in a controlled environment ( Jarmon et al., 2008 ; Ashley et al., 2014 ). In the case of PBL, each individual learner can use different tools inside VRCL environments to illustrate and represent ideas and suggestions to the rest of the group. Considering that VEs seem great tools for conceptual learning because of their customizability and visual nature ( Brna and Aspin, 1998 ; Griol et al., 2014 ), learners can use these features to explain their point of view in ways that they otherwise could not. As a result, learners appear to become more active and effective in sharing ideas, joint problem solving and the co-construction of mental models when working in groups inside VRCL environments ( Rogers, 2011 ; Hwang and Hu, 2013 ).

Returning to the topic of disparity between HMD and non-HMD VR represented in the reviewed literature, as well as both being discussed as one and the same “Virtual Reality”, an important question to ask is whether the affordances identified here are transferable between the two. HMD and non-HMD VR differ in several ways: they are interacted with differently, face different obstacles when applied to education and appear to have different learning outcomes based on different educational settings.

With the definitions of VEs and VR as given by Girvan (2018) as a frame of reference, however, an answer can be given regarding the transferability of these affordances between HMD and non-HMD VR. Both HMD and non-HMD VR should be considered tools, technical systems through which users can virtually enter VEs, i.e., shared simulated spaces in which they can interact with the environment as well as each other. As such, the affordances described in this paper do not revolve around the tools used, but that which they provide access to: the VRCL environments. Simultaneously, which tool is used to access these VRCL environments can in turn affect both the interaction and the outcome of users’ experiences with VEs. For example, HMD VR might offer more effective development of social skills compared to non-HMD VR, considering the former provides a higher sense of embodiment and, in extension, more intuitive and expansive methods of expression. If, however, cognitive learning outcomes are the most important educational objective, non-HMD VR could be a better option, considering HMD VR’s tendency to cause affective and cognitive distractions. This, then, reflects the aforementioned statement regarding HMD VR being a benefit or a burden. While affordances of VRCL environments apply to both HMD and non-HMD VR, the effect of these affordances depend on 1) the goals, 2) the setting and 3) which affordances of VRCL are most vital to the first two elements. As such, the choice between non-HMD VR and HMD VR should be made depending on those three elements.

5 Conclusion and future research

With current research on the topic being scarce while the demand for remote collaboration and distance learning keeps increasing, this literature review intends to study how VR has been (and can be) used to support and enhance CL. To achieve this, it attempts to answer four research questions regarding prior research on VRCL: what skills and competences have been trained with VRCL and what does VRCL provide in these scenarios? To what educational domains has VRCL been applied and why? What systems have been used for VRCL? And what empirical knowledge has been established regarding VRCL?

This paper identifies five types of skills and competences commonly trained with the use of VRCL. Furthermore, a number of features and design principles are identified in terms of what these environments should offer for these skills to be developed. Educational fields and domains appear to be interested in VRCL because of a desire to innovate, to form communities, to support remote collaboration and to enhance socialization skills of learners. In terms of technology, systems used for VRCL-related purposes appear to predominantly focus on monitor-based (non-HMD) VR and mouse-and-keyboard controls, contrasting what VR is commonly associated with (e.g., HMD VR, specialized controls involving gaze control and positional tracking). This study perceives a general optimism present in the literature reviewed regarding the use of VR to support and enhance CL in learners. Additionally, a number of affordances of VRCL are described, though it is of importance to note that these affordances could differ in strength depending on which type of VR (i.e., non-HMD or HMD) is used.

While the literature on VRCL reviewed for this paper is diverse, it suggests that Virtual Reality can be an effective tool for supporting and enhancing Collaborative Learning. This diversity, however, also highlights that pedagogies of VRCL are lacking, with studies showing many different and contrasting approaches to applying VR to their respective fields for the support of CL. In order to see VR become more adopted as an educational tool for collaborative purposes, pedagogies should be clearly structured, highlighting similarities and differences in regards to both the technologies used and the domains they are used in. As such, this paper proposes a number of suggestions for future research. First, the difference between hardware used in the literature reviewed and the state-of-the-art of VR suggests that further examination of differences between non-HMD and HMD VRCL, both in terms of affordances as well as challenges and obstacles, could lead to a better understanding of VRCL’s potential. Second, despite the advantages VR has for development in affective and psychomotor skills, the scientific literature on VRCL shows only minor focus on these domains. This study argues that CL would benefit from both these domains being featured more prominently and as such encourages more research into these matters. Third, this paper suggests that research into VRCL focuses on using study designs and evaluation methods that are less frequently (or barely) featured in the reviewed literature. While the repeated and dominant use of pre-experimental study design is understandably meant to identify the potential behind the technology, the domain of VRCL (and, in extension, research on VR in education) would benefit from more true experimental design. Additionally, considering that the use of physiological data for evaluation methods appears to be unexplored terrain, this paper suggests that future research into VRCL implements these types of methods.

Author contributions

NvM: Main author VvW: Co-author and coder W-PB: Co-author and supervisor MS: Co-author and supervisor. All authors contributed to the article and approved the submitted version.

This project has been funded by the Leiden-Delft-Erasmus Centre for Education and Learning (LDE-CEL).

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The supplementary material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/frvir.2023.1159905/full#supplementary-material

Supplementary Appendix A | List of all articles included.

Supplementary Appendix B1 | Results of agreement between first author and five additional coders on in- and exclusion criteria.

Supplementary Appendix C | Explanation/motivation behind Taxonomy.

Supplementary Appendix D1 | Results of agreement between first author and coder on use of taxonomy’s first category (Education).

Supplementary Appendix D2 | Results of agreement between first author and coder on use of taxonomy’s second category (System).

Supplementary Appendix D3 | Results of agreement between first author and coder on use of taxonomy’s third category (Evaluation).

Abdullah, J., Mohd-Isa, W. N., and Samsudin, M. A. (2019). Virtual reality to improve group work skill and self-directed learning in problem-based learning narratives. Virtual Real. 23 (4), 461–471. doi:10.1007/s10055-019-00381-1

CrossRef Full Text | Google Scholar

Al-Hatem, A. I., Masood, M., and Al-Samarraie, H. (2018). Fostering student nurses’ self-regulated learning with the Second Life environment: An empirical study. J. Inf. Technol. Educ. Res. 17, 285–307. doi:10.28945/4110

Amati, M., and McNeill, M. (2012). Learning from and through virtual worlds: A pilot study of second life. J. Educ. Built Environ. 7 (1), 39–55. doi:10.11120/jebe.2012.07010039

Arlati, S., Colombo, V., Spoladore, D., Greci, L., Pedroli, E., Serino, S., et al. (2019). A social virtual reality-based application for the physical and cognitive training of the elderly at home. Sensors Switz. 19 (2), 261. doi:10.3390/s19020261

Ashley, C., Kibbe, S., and Thornton, S. (2014). Experiential learning in second life: An application in retail management. Atl. Mark. J. 3 (2), 94–112. http://conf.vwbpe.org/index.php/VWBPE/11/paper/view/132 .

Google Scholar

Avanzato, R. (2018). Virtual technology to support student engagement and collaboration in online engineering courses. Comput. Educ. J. 9 (4). https://www.scopus.com/inward/record.uri?eid=2-s2.0-85058935867&partnerID=40&md5=c38a848de46d17a869b0307c6046d06e .

Aydogan, H., and Aras, F. (2019). Design, simulation and virtual implementation of a novel fundamental programmable logic controllers laboratory in a 3D virtual world. Int. J. Electr. Eng. Educ. 59, 266–281. doi:10.1177/0020720919856249

Bilyatdinova, A., Karsakov, A., Bezgodov, A., and Dukhanov, A. (2016). Virtual environment for creative and collaborative learning. Adv. Intelligent Syst. Comput. 416, 371–381. doi:10.1007/978-3-319-27478-2_26

Bloom, B. S., Engelhart, M. D., Furst, E. J., Hill, W. H., and Krathwohl, D. R. (1956). “The classification of educational goals,” in Taxonomy of educational objectives (New York, NY: David McKay ), 62–197.

Bluemink, J., Hämäläinen, R., Manninen, T., and Järvelä, S. (2010). Group-level analysis on multiplayer game collaboration: How do the individuals shape the group interaction? Interact. Learn. Environ. 18 (4), 365–383. doi:10.1080/10494820802602444

Bonner, E., and Reinders, H. (2018). Augmented and virtual reality in the language classroom: Practical ideas. Teach. Engl. Technol. 18 (3), 33–53. http://www.tewtjournal.org .

Bouta, H., and Retalis, S. (2013). Enhancing primary school children collaborative learning experiences in maths via a 3D virtual environment. Educ. Inf. Technol. 18 (4), 571–596. doi:10.1007/s10639-012-9198-8

Bower, M., Lee, M. J. W., and Dalgarno, B. (2017). Collaborative learning across physical and virtual worlds: Factors supporting and constraining learners in a blended reality environment. Br. J. Educ. Technol. 48 (2), 407–430. doi:10.1111/bjet.12435

Brna, P., and Aspin, R. (1998). Collaboration in a virtual world: Support for conceptual learning? Educ. Inf. Technol. 3 (3–4), 247–259. doi:10.1023/a:1009649631868

Carron, T., Pernelle, P., and Talbot, S. (2013). “Issues of learning games: From virtual to real,” in IADIS international conference on cognition and exploratory learning in digital age, CELDA (Texas, TX: International Assn for Development of the Information Society (IADIS) ), 133–140.

Champsas, I., Leftheris, I., Tsiatsos, T., Terzidou, T., and Mavridis, A. (2012). “Open games: A framework for implementing 3D collaborative educational games in opensim,” in Proceedings of the European conference on games-based learning (Cork, Ireland: European Conference on Games Based Learning (ECGBL) ).

Chang, V., Gütl, C., Kopeinik, S., and Williams, R. (2009). Evaluation of collaborative learning settings in 3D virtual worlds. Int. J. Emerg. Technol. Learn. 4, 6–17. doi:10.3991/ijet.v4s3.1112

Chang, Y., Aziz, E. S. S., Zhang, Z., Zhang, M., and Esche, S. K. (2016). Usability evaluation of a virtual educational laboratory platform. Comput. Educ. J. 16 (1), 24–36. doi:10.18260/p.24973

Chavez, B., and Bayona, S. (2018). “Virtual reality in the learning process,” in Advances in intelligent systems and computing (Midtown Manhattan, New York City: Springer International Publishing ), 746. doi:10.1007/978-3-319-77712-2_129

Cheng, Y., and Ye, J. (2010). Exploring the social competence of students with autism spectrum conditions in a collaborative virtual learning environment - the pilot study. Comput. Educ. 54 (4), 1068–1077. doi:10.1016/j.compedu.2009.10.011

Cheong, Y. G., Khaled, R., HolmgȦrd, C., and Yannakakis, G. N. (2015). “Serious games for teaching conflict resolution: Modeling conflict dynamics,” in Conflict and multimodal communication: Social research and machine intelligence (Cham, Switzerland: Springer ), 449–475. doi:10.1007/978-3-319-14081-0_21

Cook, M., Lischer-Katz, Z., Hall, N., Hardesty, J., Johnson, J., McDonald, R., et al. (2019). Challenges and strategies for educational virtual reality: Results of an expert-led forum on 3D/VR technologies across academic institutions. Inf. Technol. Libr. 38 (4), 25–48. doi:10.6017/ital.v38i4.11075

Dalgarno, B., and Lee, M. J. W. (2010). What are the learning affordances of 3-D virtual environments? Br. J. Educ. Technol. 41 (1), 10–32. doi:10.1111/j.1467-8535.2009.01038.x

Daphne, E., William, M. L., Steve, P. R., and Adrian, W. J. (2000). “CVE technology development based an real world application and user needs,” in Proceedings IEEE 9th International Workshops on Enabling Technologies: Infrastructure for Collaborative Enterprises (WET ICE 2000) , Gaithersburg, MD, USA , 14-16 June 2000 . doi:10.1109/ENABL.2000.883698

de Back, T. T., Tinga, A. M., Nguyen, P., and Louwerse, M. M. (2020). Benefits of immersive collaborative learning in CAVE-based virtual reality. Int. J. Educ. Technol. High. Educ. 17 (1), 51. doi:10.1186/s41239-020-00228-9

de la Peña, N., Weil, P., Llobera, J., Giannopoulos, E., Pomés, A., Spanlang, B., et al. (2010). Immersive journalism: Immersive virtual reality for the first-person experience of news. Presence Teleoperators Virtual Environ. 19 (4), 291–301. doi:10.1162/PRES_a_00005

De Lucia, A., Francese, R., Passero, I., and Tortora, G. (2009). Development and evaluation of a system enhancing second Life to support synchronous role-based collaborative learning. Softw. - Pract. Exp. 39 (12), 1025–1054. doi:10.1002/spe.926

De Pace, F., Manuri, F., Sanna, A., and Zappia, D. (2019). A comparison between two different approaches for a collaborative mixed-virtual environment in industrial maintenance. Front. Robotics AI 6, 18–14. doi:10.3389/frobt.2019.00018

Denoyelles, A., and Kyeong-Ju Seo, K. (2012). Inspiring equal contribution and opportunity in a 3d multi-user virtual environment: Bringing together men gamers and women non-gamers in Second Life®. Comput. Educ. , 58(1), 21–29. doi:10.1016/j.compedu.2011.07.007

Desai, K., Belmonte, U. H. H., Jin, R., Prabhakaran, B., Diehl, P., Ramirez, V. A., et al. (2017). “Experiences with multi-modal collaborative virtual laboratory (MMCVL),” in Proceedings - 2017 IEEE 3rd International Conference on Multimedia Big Data, BigMM , Laguna Hills, CA, USA , 19-21 April 2017 , 376–383. doi:10.1109/BigMM.2017.62

Dich, Y., Reilly, J., and Schneider, B. (2018). “Using physiological synchrony as an indicator of collaboration quality, task performance and learning,” in Lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics) (Midtown Manhattan, New York City: Springer International Publishing ). doi:10.1007/978-3-319-93843-1_8

Dixon, S. (2006). A history of virtual reality in performance. Int. J. Perform. Arts Digital Media 2 (1), 23–54. doi:10.1386/padm.2.1.23/1

Economou, D., Mitchell, W., Pettifer, S., Cook, J., and Marsh, J. (2001). “User centred virtual actor technology,” in Proceedings VAST 2001 virtual reality, archeology, and cultural heritage (New York, NY: Association for Computing Machinery ), 323–331. doi:10.1145/584993.585052

Edirisingha, P., Nie, M., Pluciennik, M., and Young, R. (2009). Socialisation for learning at a distance in a 3-D multi-user virtual environment. Br. J. Educ. Technol. 40 (3), 458–479. doi:10.1111/j.1467-8535.2009.00962.x

Franco, J. F., Da Cruz, S. R., and Lopes, R. D. D. (2006). “Computer graphics, interactive technologies and collaborative learning synergy supporting individuals’ skills development,” in ACM SIGGRAPH 2006 educators program, SIGGRAPH ’06 (New York, NY: Association for Computing Machinery ). doi:10.1145/1179295.1179338

Franco, J. F., and De Deus Lopes, R. (2009). “Three-dimensional digital enviroments and computer graphics influencing K-12 individuals’ digital literacy development and interdisciplinary lifelong learning,” in ACM SIGGRAPH ASIA 2009 educators program, SIGGRAPH ASIA ’09 (New York, NY: Association for Computing Machinery ). doi:10.1145/1666611.1666626

Freina, L., and Ott, M. (2015). “A literature review on immersive virtual reality in education: State of the art and perspectives,” in Proceedings of ELearning and Software for Education (ELSE) , Bucharest, Romania , April 23--24, 2015 .

Gerhard, M., Moore, D. J., and Hobbs, D. J. (2001). Continuous presence in collaborative virtual environments: Towards a hybrid avatar-agent model for user representation. Lect. Notes Comput. Sci. 2190, 137–155. doi:10.1007/3-540-44812-8_12

Girvan, C., and Savage, T. (2010). Identifying an appropriate pedagogy for virtual worlds: A communal constructivism case study. Comput. Educ. 55 (1), 342–349. doi:10.1016/j.compedu.2010.01.020

Girvan, C. (2018). What is a virtual world? Definition and classification. Educ. Technol. Res. Dev. 66 (5), 1087–1100. doi:10.1007/s11423-018-9577-y

Godin, J. J., and Pridmore, J. (2019). Using design thinking strategies and virtual reality in global virtual teams. Issues Inf. Syst. 20 (4), 104–106. doi:10.48009/4_iis_2019_104-106

Griol, D., Molina, J. M., and Callejas, Z. (2014). An approach to develop intelligent learning environments by means of immersive virtual worlds. J. Ambient Intell. Smart Environ. 6 (2), 237–255. doi:10.3233/AIS-140255

Herrera, F., Bailenson, J., Weisz, E., Ogle, E., and Zak, J. (2018). Building long-term empathy: A large-scale comparison of traditional and virtual reality perspective-taking. PLoS ONE 13, e0204494. doi:10.1371/journal.pone.0204494

PubMed Abstract | CrossRef Full Text | Google Scholar

Hwang, W. Y., and Hu, S. S. (2013). Analysis of peer learning behaviors using multiple representations in virtual reality and their impacts on geometry problem solving. Comput. Educ. 62, 308–319. doi:10.1016/j.compedu.2012.10.005

Jackson, R. L., Taylor, W., and Winn, W. (1999). “Peer collaboration and virtual environments: A preliminary investigation of multi-participant virtual reality applied in science education,” in Proceedings of the ACM symposium on applied computing (New York, NY: Association for Computing Machinery ), 121–125.

Jarmon, L., Traphagan, T., and Mayrath, M. (2008). Understanding project-based learning in Second Life with a pedagogy, training, and assessment trio. Educ. Media Int. 45 (3), 157–176. doi:10.1080/09523980802283889

Jensen, L., and Konradsen, F. (2018). A review of the use of virtual reality head-mounted displays in education and training. Educ. Inf. Technol. 23 (4), 1515–1529. doi:10.1007/s10639-017-9676-0

Jeong, H., and Hmelo-Silver, C. E. (2016). Seven affordances of computer-supported collaborative learning: How to support collaborative learning? How can technologies help? Educ. Psychol. 51 (2), 247–265. doi:10.1080/00461520.2016.1158654

Jeong, J. Y., Seo, S. J., Han, Y. J., Kim, S. J., and Jeong, H. (2018). “Learning English conversations in an immersive virtual reality environment,” in Icce 2018 - 26th international conference on computers in education, workshop proceedings (Philippines: Asia-Pacific Society for Computers in Education ), 638–643.

Jing, Y., Behzad, M., Jeff, H. P., Sarah Farukhi, A., S Khizer, K., Hinton, R. J., et al. (2018). Vital roles of β-catenin in trans-differentiation of chondrocytes to bone cells. Int. J. Ophthalmol. Clin. Res. 5 (2), 1–9. doi:10.7150/ijbs.23165

Joyner, D. A., Mavilakandy, A., Mittal, I., Kutnick, D., and Macintyre, B. (2021). Content - neutral immersive environments for cultivating scalable camaraderie . New York, NY: Association for Computing Machinery , 47–58.

Juliano, J. M., Spicer, R. P., Vourvopoulos, A., Lefebvre, S., Jann, K., Ard, T., et al. (2020). Embodiment is related to better performance on a brain–computer interface in immersive virtual reality: A pilot study. Sensors Switz. 20 (4), 1204. doi:10.3390/s20041204

Ke, F., and Lee, S. (2016). Virtual reality based collaborative design by children with high-functioning autism: Design-based flexibility, identity, and norm construction. Interact. Learn. Environ. 24 (7), 1511–1533. doi:10.1080/10494820.2015.1040421

Kleanthous, S., Michael, M., Samaras, G., and Christodoulou, E. (2016). “Transactive memory in task-driven 3d virtual world teams,” in ACM international conference proceeding series (New York, NY: Association for Computing Machinery ), 23–27. doi:10.1145/2971485.2996728

Kong, J. S. L., and Kwok, R. C. W. (2013). “Knowledge conversion in massive peers: A preliminary study on mutualistic co-presence,” in Proceedings - pacific asia conference on information systems, PACIS 2013 (Korea: Pacific Asia Conference on Information Systems (PACIS) ). https://www.scopus.com/inward/record.uri?eid=2-s2.0-84928485362&partnerID=40&md5=9a1b86ea4622e938ec393f4b968a7fc6 .

Kwon, C. (2019). Verification of the possibility and effectiveness of experiential learning using HMD-based immersive VR technologies. Virtual Real. 23 (1), 101–118. doi:10.1007/s10055-018-0364-1

Lattemann, C., and Stieglitz, S. (2012). “Challenges for lecturers in virtual worlds,” in Ecis 2012 - proceedings of the 20th European conference on information systems (Barcelona, Spain: European Conference on Information Systems (ECIS) ). https://www.scopus.com/inward/record.uri?eid=2-s2.0-84905717967&partnerID=40&md5=3fda0e424f60827726a346d39465e313 .

Le, Q. T., Pedro, A., and Park, C. S. (2015). A social virtual reality based construction safety education system for experiential learning. J. Intelligent Robotic Syst. Theory Appl. 79 (3–4), 487–506. doi:10.1007/s10846-014-0112-z

Li, H., Chan, G., and Skitmore, M. (2012). Multiuser virtual safety training system for tower crane dismantlement. J. Comput. Civ. Eng. 26 (5), 638–647. doi:10.1061/(asce)cp.1943-5487.0000170(asce)cp.1943-5487.0000170

Liu, Y., Wang, T., Wang, K., and Zhang, Y. (2021). Collaborative learning quality classification through physiological synchrony recorded by wearable biosensors. Front. Psychol. 12, 674369. doi:10.3389/fpsyg.2021.674369

Liu, Y. (2020). The application of virtual reality in empathy establishment: Foresee the future. Proc. Int. Conf. Comput. Intell. Appl. ICCIA 2020, 188–193. doi:10.1109/ICCIA49625.2020.00043

López-Faican, L., and Jaen, J. (2020). EmoFindAR: Evaluation of a mobile multiplayer augmented reality game for primary school children. Comput. Educ. 149, 103814. doi:10.1016/j.compedu.2020.103814

Makransky, G., and Lilleholt, L. (2018). A structural equation modeling investigation of the emotional value of immersive virtual reality in education. Educ. Technol. Res. Dev. 66 (5), 1141–1164. doi:10.1007/s11423-018-9581-2

Makransky, G., Terkildsen, T. S., and Mayer, R. E. (2019). Adding immersive virtual reality to a science lab simulation causes more presence but less learning. Learn. Instr. 60, 225–236. doi:10.1016/j.learninstruc.2017.12.007

McArdle, G., and Bertolotto, M. (2012). Assessing the application of three-dimensional collaborative technologies within an e-learning environment. Interact. Learn. Environ. 20 (1), 57–75. doi:10.1080/10494821003714749

McHugh, M. L. (2012). Interrater reliability: The kappa statistic. Biochem. Medica 22 (3), 276–282. https://hrcak.srce.hr/89395 . doi:10.11613/bm.2012.031

Mehrfard, A., Fotouhi, J., Taylor, G., Forster, T., Navab, N., and Fuerst, B. (2019). A comparative analysis of virtual reality head-mounted display systems . arXiv . http://arxiv.org/abs/1912.02913 .

Merchant, Z., Goetz, E. T., Cifuentes, L., Keeney-Kennicutt, W., and Davis, T. J. (2014). Effectiveness of virtual reality-based instruction on students’ learning outcomes in K-12 and higher education: A meta-analysis. Comput. Educ. 70, 29–40. doi:10.1016/j.compedu.2013.07.033

Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., and Group, T. P. (2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLoS Med. 6 (7), e1000097. doi:10.1371/journal.pmed.1000097

Molka-Danielsen, J., and Brask, O. D. (2014). Designing virtual collaboration environment for distance learners: Exploring socialization as a basis for collaboration. Lect. Notes Bus. Inf. Process. 186, 74–89. doi:10.1007/978-3-319-09546-2_6

Moll, J., and Pysander, E. L. S. (2013). A haptic tool for group work on geometrical concepts engaging blind and sighted pupils. ACM Trans. Accessible Comput. 4 (4), 1–37. doi:10.1145/2493171.2493172

Moore, A. G., McMahan, R. P., and Ruozzi, N. (2021). Exploration of feature representations for predicting learning and retention outcomes in a VR training scenario. Big Data Cognitive Comput. 5 (3), 29. doi:10.3390/bdcc5030029

Mørch, A. I., Mifsud, L., and Eie, S. (2019). Developing a model of collaborative learning with minecraft for social studies classrooms using role-play theory and practice. Computer-Supported Collab. Learn. Conf. CSCL 1, 272–279.

Motejlek, J., and Alpay, E. (2019). “A taxonomy for virtual and augmented reality in education,” in Proceedings of the 46th SEFI annual conference 2018: Creativity, innovation and entrepreneurship for engineering education excellence ( arXiv ), 1089–1100.

Muhammad Nur Affendy, N., and Ajune Wanis, I. (2019). “A review on collaborative learning environment across virtual and augmented reality technology,” in IOP Conference Series: Materials Science and Engineering 5(1). doi:10.1088/1757-899X/551/1/012050

Mystakidis, S., Berki, E., and Valtanen, J. (2017). Toward successfully integrating mini learning games into social virtual reality environments – recommendations for improving open and distance learning. Edulearn17 Proc. 1, 968–977. doi:10.21125/edulearn.2017.1203

Nadolny, L., Woolfrey, J., Pierlott, M., and Kahn, S. (2013). SciEthics Interactive: Science and ethics learning in a virtual environment. Educ. Technol. Res. Dev. 61 (6), 979–999. doi:10.1007/s11423-013-9319-0

Nikolic, S., and Nicholls, B. (2018). Exploring student interest of online peer assisted learning using mixed-reality technology . Cham, Switzerland: Springer , 715, 48–54. doi:10.1007/978-3-319-73210-7_6

Nisiotis, L., and Kleanthous, S. (2019). “The relationship between students’ engagement and the development of transactive memory systems in MuvE: An experience report,” in Annual conference on innovation and technology in computer science education, ITiCSE, 71–77 (New York, NY: Association for Computing Machinery ). doi:10.1145/3304221.3319743

Nuñez, M., Quirós, R., Nuñez, I., Carda, J. B., and Camahort, E. (2008). Collaborative augmented reality for inorganic chemistry education. WSEAS Int. Conf. Proc. Math. Comput. Sci. Eng. 5, 271–277.

Olmos, E., Cavalcanti, J. F., Soler, J.-L., Contero, M., and Alcañiz, M. (2018). Mobile virtual reality: A promising technology to change the way we learn and teach (Singapore: Springer ), 95–106. doi:10.1007/978-981-10-6144-8_6

Parong, J., and Mayer, R. E. (2021). Cognitive and affective processes for learning science in immersive virtual reality. J. Comput. Assisted Learn. 37 (1), 226–241. doi:10.1111/jcal.12482

Pastel, S., Chen, C. H., Petri, K., and Witte, K. (2020). Effects of body visualization on performance in head-mounted display virtual reality. PLoS ONE 15, 0239226. doi:10.1371/journal.pone.0239226

Pellas, N. (2014). Harnessing and evaluating open sim for the implementation of an Inquiry-Based Collaborative Learning (Ib[C]L) script in computer science: Preliminary findings from a case study in higher education. Open Source Technol. Concepts, Methodol. Tools, Appl. 3–4, 1223–1246. doi:10.4018/978-1-4666-7230-7.ch059

Pellas, N., Peroutseas, E., and Kazanidis, I. (2013). “Virtual communities of inquiry (VCoI) for learning basic algorithmic structures with open simulator and Scratch4OS: A case study from the secondary education in Greece,” in ACM international conference proceeding series (New York, NY: Association for Computing Machinery ), 187–194. doi:10.1145/2490257.2490278

Radianti, J., Majchrzak, T. A., Fromm, J., and Wohlgenannt, I. (2020). A systematic review of immersive virtual reality applications for higher education: Design elements, lessons learned, and research agenda. Comput. Educ. 147, 103778. doi:10.1016/j.compedu.2019.103778

Rebelo, F., Noriega, P., Duarte, E., and Soares, M. (2012). Using virtual reality to assess user experience. Hum. Factors 54 (6), 964–982. doi:10.1177/0018720812465006

Rogers, L. (2011). Developing simulations in multi-user virtual environments to enhance healthcare education. Br. J. Educ. Technol. 42 (4), 608–615. doi:10.1111/j.1467-8535.2010.01057.x

Rueda, J., and Lara, F. (2020). Virtual reality and empathy enhancement: Ethical aspects. Front. Robotics AI 7, 506984. doi:10.3389/frobt.2020.506984

Saldana, D., Neureither, M., Schmiesing, A., Jahng, E., Kysh, L., Roll, S. C., et al. (2020). Applications of head-mounted displays for virtual reality in adult physical rehabilitation: A scoping review. Am. J. Occup. Ther. 74 (5), 7405205060p1–7405205060p15. doi:10.5014/ajot.2020.041442

Scavarelli, A., Arya, A., and Teather, R. J. (2021). Virtual reality and augmented reality in social learning spaces: A literature review. Virtual Real. 25 (1), 257–277. doi:10.1007/s10055-020-00444-8

Schreiber, J., and Asnerly-Self, K. (2011). Educational research. The interrelationship of questions, sampling, design and analysis . Hoboken, New Jersey: John Wiley & Sons, Inc .

Shawky, D., Badawi, A., Said, T., and Hozayin, R. (2014). “Affordances of computer-supported collaborative learning platforms: A systematic review,” in Proceedings of 2014 international conference on interactive collaborative learning, ICL (New York, NY: Institute of Electrical and Electronics Engineers (IEEE) ), 633–651. doi:10.1109/ICL.2014.7017846

Shi, Y., Dua, J., Ahn, C. R., and Ragan, E. (2019). Impact assessment of reinforced learning methods on construction workers’ fall risk behavior using virtual reality. Automation Constr. 104, 197–214. doi:10.1016/j.autcon.2019.04.015

Sims, R., Chang, B., Bennett, V., Krishnan, A., Aboubakar, A., Coman, G., et al. (2022). “Step into my mind palace: Exploration of a collaborative paragogy tool in VR,” in Proceedings of 2022 8th international conference of the immersive learning research network, iLRN 2022. Immersive learning research network (New York, NY: Institute of Electrical and Electronics Engineers (IEEE) ). doi:10.23919/iLRN55037.2022.9815936

Smith, S. A. (2019). Virtual reality in episodic memory research: A review. Psychonomic Bull. Rev. 26 (4), 1213–1237. doi:10.3758/s13423-019-01605-w

Sousa Santos, B., Dias, P., Pimentel, A., Baggerman, J. W., Ferreira, C., Silva, S., et al. (2009). Head-mounted display versus desktop for 3D navigation in virtual reality: A user study. Multimedia Tools Appl. 41 (1), 161–181. doi:10.1007/s11042-008-0223-2

Srivastava, P., Rimzhim, A., Vijay, P., Singh, S., and Chandra, S. (2019). Desktop VR is better than non-ambulatory HMD VR for spatial learning. Front. Robotics AI 6, 50–15. doi:10.3389/frobt.2019.00050

Stahl, G., Koschmann, T., and Suthers, D. (2006). “Computer-supported collaborative learning: An historical perspective,” in Cambridge handbook of the learning sciences . Editor R. K. Sawyer (Cambridge: Cambridge University Press ), 409–426. http://gerrystahl.net/cscl/CSCL_Chinese_traditional.pdf .

Sulbaran, T., and Jones, L. F. (2012). “Utilizing a collaborative virtual reality environment as a training tool for construction students,” in ASEE annual conference and exposition, conference proceedings (Texas, TX: American Society for Engineering Education (ASEE) ). doi:10.18260/1-2--22208

Tan, C. S. S., Luyten, K., van Den Bergh, J., Schoning, J., and Coninx, K. (2014). The role of physiological cues during remote collaboration. Presence Teleoperators Virtual Environ. 23 (1), 90–107. doi:10.1162/PRES_a_00168

Terzidou, T., Tsiatsos, T., Dae, A., Samaras, O., and Chasanidou, A. (2012). “Utilizing virtual worlds for game based learning: Grafica, a 3D educational game in second life,” in Proceedings of the 12th IEEE international conference on advanced learning technologies, ICALT (New York, NY: Institute of Electrical and Electronics Engineers (IEEE) ), 624–628. doi:10.1109/ICALT.2012.55

Tovar, D. F., Jonker, V., and Hürst, W. (2020). Virtual reality and augmented reality in education: A review . Utrecht, Netherlands: Utrecht University .

Tremmel, C., Herff, C., Sato, T., Rechowicz, K., Yamani, Y., and Krusienski, D. J. (2019). Estimating cognitive workload in an interactive virtual reality environment using EEG. Front. Hum. Neurosci. 13, 401–412. doi:10.3389/fnhum.2019.00401

Tüzün, H., Bilgiç, H. G., and Elçi, S. Y. (2019). The effects of 3D multi-user virtual environments on collaborative learning and social presence. Int. Electron. J. Elem. Educ. 11 (3), 221–231. doi:10.26822/iejee.2019349247

Vallance, M., Ibayashi, K., and Goto, Y. (2015). Engineering active learning in 3D virtual worlds. Commun. Comput. Inf. Sci. 533, 268–282. doi:10.1007/978-3-319-22629-3_22

Vrellis, I., Papachristos, N. M., Bellou, J., Avouris, N., and Mikropoulos, T. A. (2010). Designing a collaborative learning activity in second life: An exploratory study in physics. Proc. - 10th IEEE Int. Conf. Adv. Learn. Technol. ICALT 2010, 210–214. doi:10.1109/ICALT.2010.65

Wang, C. J., Low, A. S. L., and Woo, C. S. (2014). “3D system for learning gains in the lower secondary normal (technical) stream,” in 3D Immersive and Interactive Learning (Singapore: Springer ), 93–103. doi:10.1007/978-981-4021-90-6_7

Wang, A., Thompson, M., Roy, D., Pan, K., Perry, J., Tan, P., et al. (2022). Iterative user and expert feedback in the design of an educational virtual reality biology game. Interact. Learn. Environ. 30 (4), 677–694. doi:10.1080/10494820.2019.1678489

Yeh, S. C., Hwang, W. Y., Wang, J. L., and Chen, Y. R. (2012). “Effects of multi-symbols on enhancing virtual reality based collaborative task,” in Lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics) (Berlin, Germany: Springer Berlin Heidelberg ), Vol. 7220, 101–111. doi:10.1007/978-3-642-31439-1_10

Yoon, H. J., Kim, J., Park, S. W., and Heo, H. (2020). Influence of virtual reality on visual parameters: Immersive versus non-immersive mode. BMC Ophthalmol. 20 (1), 200–208. doi:10.1186/s12886-020-01471-4

Zheng, L., Xie, T., and Liu, G. (2019). “Affordances of virtual reality for collaborative learning,” in Proceedings - international joint conference on information, media and engineering, ICIME 2018 (New York, NY: Institute of Electrical and Electronics Engineers (IEEE) ), 6–10. doi:10.1109/ICIME.2018.00011

Keywords: virtual reality, collaborative learning, virtual reality education, collaborative virtual environment, virtual reality and collaborative learning, collaborative virtual reality, collaborative virtual reality systems, educational technologies

Citation: van der Meer N, van der Werf V, Brinkman W-P and Specht M (2023) Virtual reality and collaborative learning: a systematic literature review. Front. Virtual Real. 4:1159905. doi: 10.3389/frvir.2023.1159905

Received: 06 February 2023; Accepted: 02 May 2023; Published: 19 May 2023.

Reviewed by:

Copyright © 2023 van der Meer, van der Werf, Brinkman and Specht. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Nesse van der Meer, [email protected]

Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.

A Literature Review on Immersive Virtual Reality in Education: State Of The Art and Perspectives

• Conference: eLearning and Software for Education (eLSE)
• At: Bucharest (Romania)

• Italian National Research Council

Abstract and Figures

Discover the world's research

• 25+ million members
• 160+ million publication pages
• 2.3+ billion citations

• Junaid Qadir

• Yuzheng Chen

• Fidel Romero

• Raghad Alfaisal

• Pranas Jaruševičius
• Lukas Paulauskas

• Evlalia Touloudi

• Vasileios T. Stavrou
• Yannis Theodorakis
• Educ Inform Tech

• Cansu ŞAHİN KÖLEMEN
• Blessing Foluso Adeoye
• Adesoji A. Oni

• Guillermo Vasquez de Velasco

• Michael Jones
• M. Jeannette Lawler
• E. G. Hintz
• Mallory Trullender
• R.D. Webster

• Ziyang Zhang

• Recruit researchers
• Join for free
• Login Email Tip: Most researchers use their institutional email address as their ResearchGate login Password Forgot password? Keep me logged in Log in or Continue with Google Welcome back! Please log in. Email · Hint Tip: Most researchers use their institutional email address as their ResearchGate login Password Forgot password? Keep me logged in Log in or Continue with Google No account? Sign up

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Springer Nature - PMC COVID-19 Collection

Systematic literature review and bibliometric analysis on virtual reality and education

Mario a. rojas-sánchez.

1 Costa Rica Institute of Technology, Cartago, Costa Rica

Pedro R. Palos-Sánchez

2 Department of Financial Economics and Operations, University of Sevilla, Seville, Spain

3 NECE-UBI Research Unit in Business Sciences, University of Beira Interior (UBI), Covilhã, Portugal

José A. Folgado-Fernández

4 Departamento de Economía Financiera y Contabilidad, Universidad de Extremadura, Badajoz, España

Associated Data

Data sets used and/or analyzed during the current study are available from the corresponding author under reasonable request.

The objective of this study is to identify and analyze the scientific literature with a bibliometric analysis to find the main topics, authors, sources, most cited articles, and countries in the literature on virtual reality in education. Another aim is to understand the conceptual, intellectual, and social structure of the literature on the subject and identify the knowledge base of the use of VR in education and whether it is commonly used and integrated into teaching–learning processes. To do this, articles indexed in the Main Collections of the Web of Science, Scopus and Lens were analyzed for the period 2010 to 2021. The research results are presented in two parts: the first is a quantitative analysis that provides an overview of virtual reality (VR) technology used in the educational field, with tables, graphs, and maps, highlighting the main performance indicators for the production of articles and their citation. The results obtained found a total of 718 articles of which the following were analyzed 273 published articles. The second stage consisted of an inductive type of analysis that found six major groups in the cited articles, which are instruction and learning using VR, VR learning environments, use of VR in different fields of knowledge, learning processes using VR applications or games, learning processes employing simulation, and topics published during the Covid-19 pandemic. Another important aspect to mention is that VR is used in many different areas of education, but until the beginning of the pandemic the use of this so-called “disruptive process” came mainly from students, Institutions were reluctant and slow to accept and include VR in the teaching–learning processes.

Introduction

The knowledge society recognizes that Education Institutions are a fundamental part of the globalization process, where the use of information and communication technologies (ICT) improve students' attitudes towards learning (Lazar & Panisoara, 2018 ). The concept of education refers to the process of facilitating learning, acquiring knowledge, skills or positive values with the aim of preparing students for life, work and citizenship (Kamińska et al., 2019 ). Virtual platforms often simulate the classroom and can provide a safe environment for testing experiments that can be dangerous in real life (Tzanavari & Tsapatsoulis, 2010 ). This revolutionizes learning processes, although professional training and scientific research are required, to facilitate innovative processes and develop new knowledge to meet the challenges of modern world demands (Castillo, 2010 ). The use of digital technologies has increased at all academic levels with educators adopting them in order to improve the learning experience of their students (McGovern et al., 2019 ).

Learning about sciences cannot always be fully implemented in classrooms, so it can be useful to use other options (Buehl, 2017 ; Folgado-Fernández et al., 2020 ). For example, scientific experiment in which physical risks would exist for students or use of very expensive scientific-technological material. In these cases, VR could simulate this environment and e-learning conditions. VR is suited to this as it consists of using a 3D environment that has been generated by a computer where the user can navigate and interact, achieving real-time simulation with a part, or all, of a user's senses (Guttentag, 2010 ). Another definition considers virtual reality as an immersive and interactive three-dimensional computer-generated environment in which interaction can occur on multiple sensory channels such as touch and position (Brey, 2014 ).

VR has been widely investigated in many fields such as sports (Vignais et al., 2015 ), tourism (Tussyadiah et al., 2018 ), virtual stores (Bonetti et al., 2018 ; Mann et al., 2015 ), healthcare from surgery simulation to nursing applications (Beyer-Berjot et al., 2016 ; Fagan et al., 2012 ), the military for flight simulation and training (Mihelj et al., 2014 ) and of course, in education (Zhang et al., 2017 ). Applications in education can be found for geography (Lv et al., 2017 ), nature sciences (Palos-Sanchez et al., 2022 ), mathematics (Xu & Ke, 2016 ), robotics (Román-Ibáñez et al., 2018 ), construction safety (Pham et al., 2018 ), medical training and assessment (Lövquist et al., 2012 ), physical education (Gómez-García et al., 2018 ) and many others.

Compared to traditional education, using virtual simulation-based training provides safety, cost savings and efficiency because less time is required for training (Shen et al., 2019a , b ). Educators' acceptance of VR in the classroom has been successfully investigated by (Hussin et al., 2011 ). The behavioral intentions for using VR in learning was analyzed (Shen et al., 2019a , b ) and also its acceptance by surgeons using VR for training (Hen, 2019 ).

The aim of this study is to identify and analyze the scientific literature with a bibliometric review to find the main topics, authors, sources, most cited articles and countries, as well as to know the conceptual, intellectual and social structure and identify the knowledge base of VR in education, and whether it is commonly used and integrated into teaching–learning processes. To reach the objectives, the articles in the scientific production indexed in the Web of Science and Scopus Main Collection were consulted, analyzing the articles and the emerging trends in research in articles published between January 1, 2010, and July 31, 2021.

This study analyzes relevant data from previous research to answer the following research questions (RQ) (Table ​ (Table1 1 ):

Research questions

This article is organized as follows: the first part introduces the study, including the objectives and the research questions. A second section presents a review of the literature on bibliometric analysis. The methodology used is defined in the third section, indicating the search procedures used to identify the literature on VR in education. The fourth section presents the results, and these are discussed in the fifth section. Finally, in the sixth section, the conclusions are drawn, and future lines of research are suggested.

Review of the literature on bibliometric analysis

This study uses a bibliometric analysis, which is a term that was coined by Pritchard ( 1969 ) who stated that it can be applied in all studies that aim to quantify the process of written communication (Gokhale et al., 2020 ).

Bibliometric analysis is an approach that uses a set of quantitative methods to measure, track, and analyze scholarly literature (Roemer & Borchardt, 2015 ). It identifies the publications by authors, the most prominent journals, as well as the methodologies used and the conclusions obtained (Durán Sánchez et al., 2014 ).

Metadata gives an overview of any field of research (Milian et al., 2019 ). Bibliometric methods involve a large volume of bibliographic material and have been used for the analysis of different topics (Blanco-Mesa et al., 2017 ), Journals (Martínez-López et al., 2018 ), Countries (Mas-Tur et al., 2019 ) and others.

The scientific literature contains important bibliometric analyses such as that by Huang et al. ( 2016 ), who performed a retrospective bibliometric analysis of articles about rehabilitation medicine using VR technology. The conclusion was that VR technology was one of the most popular technological advances. The results found a rapid growth in the production of articles in recent years.

Methodology

In this study, the selected dataset is analyzed using a quantitative exploration with a bibliometric study that identifies and analyzes the literature on VR in education to provide a map of the knowledge structure (Álvarez-García et al., 2019 ). A performance analysis and scientific mapping is done in the first part of the study. The scientific or bibliometric mapping provides a representation of how disciplines, fields, specialties, individual papers, and authors are related to each other (Small, 1999 ). The recommendations of (Cobo et al., 2011 ) were used to produce the maps to know the research topics and the different structures in the dataset.

A second analysis was also performed, ordering the articles in descending order according to the number of citations. The recommendations of (Heradio et al., 2016 ) were used to complete this part of the study. Figure  1 shows the methodology used. This figure shows different steps. For example, data cleaning was performed after applied inclusion criteria (see Table ​ Table2—Search 2 —Search string) or indicator elaboration permits sorted in relevance different articles.

Methodology used. Methodology according to (Danvila-del-Valle et al., 2019 )

Documents obtained from searches in the databases

Articles that constitute a representative sample of international scientific activity published in scientific journals were analyzed (Durán-Sánchez et al., 2018 ; Velasco et al., 2011 ). Therefore, meetings papers, editorials, books, chapters, proceedings, news, and other types of documents found in the databases were excluded.

Identification of sources

Data was gathered from journal articles indexed in the Web of Science Core Collection. This database was selected because of three criteria:

• It has quality indexes such as JCR.
• It covers a long time period.
• It allows a considerable number of stored references to be downloaded simultaneously.

The presence of these characteristics is sufficient to justify its use (Durán-Sánchez et al., 2019 ).

The Scopus multidisciplinary bibliographic database was also used to find information in articles from scientific journals (ASJC) classified into an organized hierarchy of fields and subfields (Hassan et al., 2019 ). This database was selected because of three criteria:

• It has quality indexes such as SJR.
• It provides approximately 20% more coverage than the Web of Science.
• Simultaneous downloading of a considerable number of references is allowed (Falagas et al., 2008 ).

We used the Lens database and academic meta-search engine, compatible with the biblioshiny software used for data analysis. The search was limited to articles containing the keyword “virtual reality” in the title. This is included in quotation marks to obtain all documents containing that combination of words in the document title and also containing the possible combinations with the terms educat* (to obtain words that start with that word but may have different endings). The terms learn*, teach* class* and student* were also used to obtain articles with titles containing words related to learning, teaching, class or classroom, students. innovat* and Covid* were also included to obtain articles about innovation and the teaching process during the Covid pandemic.

Study selection criteria

The search was performed in English to obtain the largest number of documents in the dataset on VR in education. Table ​ Table2 2 shows the initial data obtained with the proposed search strings. The inclusion criteria applied was document type: only articles were selected, language: English and years of publication: 2010–2021(July). The exclusion criteria were to exclude the field of Medicine. This study has not included specialized areas of medicine to get an overview of as many applications as possible in Education. It is particularly valuable for researchers, as information is presented about present and future lines of research which investigate the usefulness of VR in periods of crisis and confinement such as the one, we have just faced evidenced by the sudden leap in the use of technologies, and therefore virtuality, in platforms, applications, games and videos.

Data analysis process

The documents included in this analysis contain bibliographic information obtained after a manual review of the 436 relevant documents found in WoS, 584 found in Scopus and the 251 in Lens. 553 duplicate documents were eliminated, and the names of the authors and journals were normalized, which resulted in 718 documents unified in an.xlsx file. This whole process is summarised in the three stages of search strategies shown in Fig.  2 .

Search strategy summary. Note: Search strategy for the articles

Biblioshiny software was used for the analysis of the data. It is a tool which analyzes all the data identified in the body of literature and identifies the main themes (Huber, 2002 ). The application provides a web interface for the Bibliometrix software version 3.0 (Aria & Cuccurullo, 2017 ) and provides the data in graphical format, if desired, to visualize the statistics. In this study, the graphs describe the information about VR in the educational field over the time period chosen for the study. Figure  2 summarizes the search strategy used.

SLR articles on VR and education

Tables ​ Tables3 3 and ​ and4 4 show some summarized systematic literature reviews (SLR) on virtual reality and education. There has recently been an increase in the number of documents, possibly due to current circumstances like the Covid pandemic and teleworking, which means that technological innovation in education systems has greater prominence than before.

Summary of related systematic literature reviews

Main information

Main statistical indicators

The use of ICTs in education is commonplace, with VR no exception and often included in the teaching–learning processes. The evolution in the productivity of articles over the period analyzed clearly shows a rapid growth from 2015 to 2021, as shown in Fig.  3 .

Annual production of articles. Note: Annual productivity of virtual reality in education. Dotted line is the exponential trend line

This growth is due to the development of specific content in VR, with more and more sectors involved such as: real estate, locomotion, security, and even education itself with the new e-Learning systems.

Performance analysis

According to Heradio et al. ( 2016 ) the main procedure for research performance evaluation is citation analysis, which means, the more citations of an article, the greater its influence in that field. The h-index is considered a suitable measure of the quantity and impact of the scientific output of the publications of a researcher.

Overview of the analyzed data set

The information from the analyzed data is summarized in descriptive statistics presented in Table ​ Table4. 4 . Considering the results obtained, we can say that RV is a topic of great academic interest as evidenced by the number of papers (718) and the more than ten average citations per article.

The average number of annual citations are presented in Fig.  4 , while Fig.  5 provides an overview of the trends in the knowledge structure of the use of VR in education.

Average annual citations per year. Note: Average total number of citation per year

Trend tropics. Journals with most productivity and impact according to the h-index

Figure  4 shows that the highest average number of citations per year were in 2010 with 6.9 citations per year and in 2014 with 7.3. Contrary to what the authors expected due to the important advances and changes in the market for VR, there were only 2.55 citations in 2016 continuing with little growth until 2018 and maintaining lower averages thereafter. 2020 and the current, available data for 2021 does not show an impact on citations due to the covid-19 situation.

Figure  5 shows that the main topics of the trends from 2010 to 2017 were about using simulation as a learning tool to obtain greater student attention. 2018 continued integrating technology into the teaching process with topics such as the virtual community of players of 2nd life with customizable avatars that allow players to enjoy a second life. This uses voice text messaging with people from different places and countries and integrates visualization tools. In 2019 education included the design of technological environments. 2020 showed an increase in the literature consulted about e-learning using tools such as virtual or augmented reality. This could possibly be because of the changes in education methods imposed by the Covid-19 pandemic and the change from presential to virtual teaching, which was made abruptly in some cases. It should be noted that 6 months into 2021 the trend is towards platforms that can be used to teach or attend lessons, i.e., information technologies and user acceptance of these take on greater importance and there is also increasing interest in artificial intelligence with deep learning that uses machine learning processes such as speech recognition or automated translation.

The journal with the most impact in this study is Computers & Education with an h-index of 16. This means that a number, h, of publications of the journal have been cited h times. An h-index of 16 implies that this number of publications have been cited at least 16 times. Table ​ Table4 4 shows the journals ordered by the number of documents published, as well as the impact measured with the h-index. We can say that the selected journals contain 207 articles in total, of which 40% correspond to 3 publications: International Journal of Emerging Technologies in Learning, Computers & Education and Virtual Reality (Table ​ (Table5 5 ).

Magazines with most productivity and impact

Relevant information on the journals included in the dataset

Authors with most impact according to h-index

The authors with the highest productivity are shown in Fig.  6 and can be seen to be Chen W., Chen Y. and Lee J. Figure  6 orders the authors according to impact where it can be seen that Chen Y. and Jong M. have the highest impact with an h-index of 7, that is, each author has 7 papers with at least 7 citations each, which means that the author has been included in at least 49 publications (Fig.  7 ).

Most relevant authors

Impact of the authors

The most active authors in the last four years have been Chen W., Lee J., Kim, J., Ly Y. and Makransky G., as shown in Fig.  8 .

Productivity of the main authors over the period of time studied

Chen W. primarily studies problem solving in the classroom using VR technology, to assist in cognitive processing and knowledge transfer to the students. On the other hand, Lee J., studies the adaptation of the three-dimensional visualization made possible by immersive virtual reality.

Other authors, such as Kim J., jointly approach the analysis of VR and augmented reality (AR) whith the current skin electronics are summarized as one of the most promising device solutions for future VR/AR devices.

Ly Yan approaches the study of VR based simulation in hospital settings, that facilitates the acquisition of skills without compromising patient safety. Finally, Makransky's research deals with various aspects of RV in education, such as an important role in education by increasing student engagement and motivation.

The main affiliations of institutions can be seen in Fig.  9 , which shows National Taiwan Normal University as being the most productive with 17 papers published in the analyzed dataset. In second position is Chinese University Hong Kong with 12 papers and in third position is Texas Aandm University with 11 papers.

Most relevant affiliations

Main documents and most frequently used words in the dataset

Table ​ Table6 6 shows the documents with the largest number of citations in this study. The authors of the document with most citations were (Merchant et al., 2014 ) with 452 citations and, in second place, (Huang et al., 2010 ) with 236 citations, both of which were published in the Computer & Education journal. In third place was (Jensen & Konradsen, 2018 ) with 196 citations in the Telematics and Informatics journal. When an article has many citations, it influences the researchers who develop the area under investigation (Rodríguez & Navarro, 2008 ).

Most globally cited documents in the dataset

The words which occur with the highest frequency in the dataset can be seen in Fig.  10 . The first four words are related to the terms contained in the search strings, but the frequency and hierarchy follows the occurrences of the words “e-learning”, “environments”, “augmented reality”, “technology”, “simulation” and “learning systems”. This highlights the technological component of the field of education. The coincidence of keywords represents the knowledge structure of the literature (Cheng et al., 2018 ).

Most relevant words

Scientific mapping analysis

A similarity measure known as the strength of association was used to construct the bibliometric maps (Cobo et al., 2011 ; Van Eck & Waltman, 2007 ). This allows a variety of scientific maps to be prepared which show the structural and dynamic aspects of the data obtained from the scientific research (Börner et al., 2003 ).

According to Cobo et al. ( 2011 ) the maps show the evolution of a field of research and the conceptual structure of the field can be found from the co-occurrence. Co-citation and bibliographic coupling allow us to analyze the intellectual structure of a field of scientific research and the social structure can be found by analyzing the authors, also known as co-authorship analysis, as well as the data found from the author's affiliations such as the organization or country.

Main topics in keywords plus according to factor analysis

Figure  11 shows a two-dimensional graph formed by the topic words in Keywords Plus of the cited papers. A multiple correspondence analysis can be used to summarize big data with multiple variables in a low-dimensional space, creating a two-dimensional map where the words near the central point of the group have received a lot of attention in recent years and those near the edge are topics which have been used less in research or have been incorporated into other topics (Xie et al., 2020 ).

Factor map cluster analysis of high-frequency key words . Note: Factor analysis, keyword map, number of terms: 50, number of clusters: 2, label size: 12, number of words: 500

The first cluster covers words related to VR display devices which are used by users of virtual reality educational products. When it is not clear which device to use in a curriculum, the relevant constituent components of immersive technologies which differentiate their roles must be considered. An example is for the two common modes of virtual reality displays, head-mounted display (HMD) and desktop computer (DT) which may affect spatial learning (Srivastava et al., 2019 ). On the one hand desktop-based VR has higher installation costs, while mobile device-based virtual reality cannot produce the same environment quality due to the limited processing power. A result of the lower environment quality has, in some cases, caused higher rates of nausea and blurred vision (Moro et al., 2017 ).

A person can interact in an environment created with VR in a seemingly real or physical way by using special electronic equipment, such as a helmet with a screen inside it or gloves equipped with sensors (Katsioloudis et al., 2017 ). Jensen and Konradsen ( 2018 ) identify situations where HMDs are useful for cognitive skill acquisition, such as remembering and understanding spatial and visual information, psychomotor skills like head movement, visual or observational exploration and affective skills for emotional control and response to stressful or difficult situations.

The second group covers VR applied to education, which includes topics such as technologies that allow the visualization of situations, which receive more attention from students and motivate them. Examples of this type of technology are virtual communities, educational games, interactive learning environments, educational technologies that improve the teaching process, online learning, user experience, immersive learning, immersive virtual reality and deep learning. There is a wide variety of possibilities, most of which are immersive, using helmets, games or applications that provide an interactive learning experience for students.

Learning environments using animation and multimedia highlight a change in VR learning which is more immersive, simulating the real world with 3D models that provide an interactive environment and reinforce the feeling of immersion. Using this technology, educators combine theory and instruction methods that allow intelligent use of these environments (Huang et al., 2010 ). There are many ways to create these environments with equipment like VR helmets for experiential learning in a virtual space (Kwon, 2019 ) and new and improved environments, such as the PILE System that integrates video capture technology into the classroom where interaction is made through physical movements (Yang et al., 2010 ). As technology advances, better graphics and virtually animated actors or avatars can be used. These improve the applications by being more motivating and enjoyable, even though the applications become more complex which may prevent a novice learner from learning effectively (Kartiko et al., 2010 ).

VR applications enable potential learning. Authors such as Johnson-Glenberg, ( 2018 ) explore applications of educational theory which design classes using immersive virtual reality with two unique attributes of VR, which are making the student feel present in any given situation and to be able to use gestures and perform manipulations in three dimensions. For decades the primary interfaces of educational technology have been the mouse and keyboard, but now highly immersive environments can enhance learning and affect the way content is retained and encoded.

Games are useful in educational technology with many examples available. Some of these are used to train students in safety through role-playing and social interaction (Palos-Sanchez et al., 2018 ), which allows students to understand the causes of accidents and inspect risks in an immersive environment provided by the game (Le et al., 2015 ). Interaction was found to play an important role in understanding mathematics and geometry with problem solving. A whiteboard and a virtual tool were used to solve problems individually or in pairs. Group learning was found to be more effective, although the results of the groups were different as the difficulty of the problems were varied (Hwang & Hu, 2013 ).

Articles were found concerning online learning which shares digital content and technological tools for e-learning and virtual reality learning. One of these articles compares techniques such as email, attachments, shared use of Web interfaces and a VR engine which provides a virtual interface. The results indicate that users completed their workflow 50% faster with the VR option (Lampert et al., 2018 ). Another application that marked a change in e-learning is an innovative tool for young adults with mild cognitive impairments. It is an immersive virtual reality game called "In Your Eyes" that focuses on skills related to spatial perspective involving all five senses which shows that an immersive world can be an excellent training method (Freina et al., 2016 ).

Co-occurrence network mapping

Bibliometric mapping of the keywords used by the author was done to gain a thorough understanding of the conceptual structure. Apart from the Keyword Virtual Reality itself, it highlights those related to education, e-Learning and Students. The co-occurrence analysis is shown in Fig.  12 .

Keywords Plus co-occurrence network

Keyword co-occurrence analysis is an effective tool for understanding knowledge structures and research trends. This makes it easier to understand primary and secondary publications (Altınay Ozdemir & Goktas, 2021 ). In this figure, one should start by distinguishing nodes by their size. This represents the number of documents, while the line between two nodes represents a link between the two groups. A link means a co-occurrence between the two keywords (Guo et al., 2019 ). If the line is short the link is strong and vice versa.

In this bibliometric analysis we mainly distinguish the following keywords: 'virtual reality', 'e-learning' and 'students' in a first cluster. Each cluster represents a keyword and shows the most linked and repeated keywords in the publications. All clusters have a different colour. In Fig.  12 a distinction is made between the red colour for this cluster and the blue colour showing the following main keywords: 'Education', 'Technology', 'augmented reality', 'performance', 'simulation' and 'environments'. Because this bibliometric study found few papers, the number of co-occurrence links between keywords was not excessive. As Fig.  12 shows, two groups had a stronger relationship: 'Education Technology for simulation environments with augmented reality' and 'Virtual reality for e-learning systems'.

Productivity mapping of items by country

The countries or regions with the highest document productivity in this study are the Republic of China with 273, United States of America with 242 documents, followed by South Korea with 57 and Spain with 50, as shown in Fig.  13 . The high productivity of the United States is consistent with the bibliometric mapping for an analysis of studies on foreign language teaching in early childhood education by (Yilmaz et al., 2019 ) as well as the work of other authors (Hernández-Torrano & Kuzhabekova, 2019 ). China is a world power in VR technology and Chinese universities have been concerned to increase research in this field to meet the challenges posed by VR technologies. The main reason is that Chinese people are very prone to adopt emerging technologies, we can say that it is an important virtual reality market in the world.

Productivity of items by country

In Fig.  14 we see how the main keywords, students, virtual reality, technology, e-learning, have a greater relationship with the countries of China and the USA, as well as the universities in the last column, which reflects a greater scientific production in topics related to technology applied to education.

Three fields plot

VR technology in the educational field is opening up space through e-learning, game-based learning to mobile learning, going from simulation, machine learning to Deep Learning, where immersive virtual reality is part of the topics present as shown in Fig.  15 .

Thematic evolution

In Fig.  15 we can see the thematic evolution through a Sankey energy diffluence diagram, which is a specific type of flow diagram. In this paper, based on the Sankey diagram, we visualise the thematic evolution over time in the field of VR and Education research. This figure helps us to understand the temporal evolution of the conditions in which the different topics in the field of Virtual Reality applied to Education have been flowing. In this Fig.  15 we can clarify quantitative information such as thematic flow, direction of thematic flow and conversion relationships (Soundararajan et al., 2014 ).

Collaboration between countries mapping

This map gives an improved understanding of the social structure, not only of authors, but also of the countries to which they belong. Figure  16 shows collaborative relationships between China, USA, Korea, and Canada, as well as Germany, Denmark and the United Kingdom, along with others, the collaboration between the USA and China are the most important.

Collaboration between countries

Second stage analysis

This section briefly summarizes the most cited articles, and they are classified into categories, in the literature few authors provide us with a panoramic view of virtual reality technology applied to the educational field. In contrast to other authors (Zappatore et al. ( 2015 ) who perform an eminently quantitative approach in their analyses, this paper follows the line of Heradio et al. ( 2016 ) by providing a dual quantitative–qualitative approach. Thus, our analysis is not limited to counting articles, authors or journals, but describes and comments on the most relevant data for the RV community (Bardakci et al., 2022 ; Kushairi & Ahmi, 2021 ). The articles were listed in descending order according to the number of citations to find the main topics addressed and describe the documents that are considered the most important. All papers which had citations were classified and grouped into six categories: (a) papers about VR-based instruction and learning, (b) papers studying VR learning environments, (c) papers presenting the use of VR in different fields of knowledge, (d) papers describing learning processes that use VR applications, devices or games, (e) papers on research about learning processes using simulation, and (f) topics published during the Covid-19 pandemic.

VR-based instruction and learning

Among the most outstanding papers are the following: Merchant et al. ( 2014 ), Lee et al. ( 2010 ), Makransky and Lilleholt ( 2018 ) and Jensen and Konradsen, ( 2018 ). Jensen and Konradsen, ( 2018 ) seek to update knowledge on the use of head-mounted displays (HMD) in Education and training. The study identifies the acquisition of skills such as cognitive skills, i.e., remembering, understanding information, spatial and visual knowledge; as well as visual exploration or observation, among the most important are the affective skills related to control and emotional response to stressful or difficult situations. These learning tools enhance learning and are very useful in the educational field.

The most cited paper is Merchant et al. ( 2014 ). This work performs a meta-analysis that investigates the effectiveness of reality-based virtual instruction on learning outcomes. In order to do this, the authors researched the overall effect and impact of selected instructional design principles of VR technology-based instruction such as, games, simulation, virtual worlds, in higher education settings and as a result found that using games has a greater effect on learning than simulations and virtual worlds.

The study by Lee et al. ( 2010 ) examined how desktop VR (VR) enhances learning, finding that VR features have an indirect effect on learning. The learning experience was individually measured by psychological factors, such as presence, motivation, cognitive gains, control, and active learning, as well as reflective thinking which all affected the learning outcomes when using the desktop VR-based learning environment. Further research investigated how spatial ability and learning style enable instructional designers and VR software developers to improve learning effectiveness and therefore increase the amount the software is used. According to Makransky and Lilleholt ( 2018 ) much remains to be discovered about the impact and use of immersive VR in e-learning tools that impact students' emotional processes while learning.

Many authors investigate VR as a tool in learning processes. An example is Huang and Liaw, ( 2018 ) who explored how virtual reality technology actively focuses on the learner's interactive learning processes and attempts to reduce the gap between learner knowledge and real-life experience. Alfalah ( 2018 ) examined perceptions when using VR as a tool for education confirming that teachers and students are willing to use VR.

Allcoat and von Mühlenen ( 2018 ) assigned students into three groups who taught with different methods, 1) traditional book learning, 2) virtual reality learning and 3) video. The students were tested for their knowledge of the subject being taught before and after the classes, finding that participants in the virtual reality group showed better recall performance and more positive emotions than the other groups.

Hewawalpita et al. ( 2018 ) explored an improved configuration of massive open online courses. Two groups were used, one group were students who had already taken the traditional course and the second group started from scratch and were given virtual reality content. The results showed that the second group had significantly better performance and it was concluded that interactive learning content can be designed for the different learning needs of students.

Wang, ( 2018 ) proposed a distance learning virtual reality experiment with computers and VR technology and found practical reasons to promote the development of distance learning using computer.

In view of the works analyzed in the context of VR-based Instruction and Learning, we can say that the VR-based learning process is a useful tool for the educator, as it can replicate or complement traditional teaching methods. It is a fully effective concept even using basic forms, such as VR glasses or smartphones. This method is suitable for classroom teaching, distance learning, self-learning and other educational environments, and allows the simulation of scenarios that enrich teaching, even those dangerous experiments that cannot be reproduced in reality.

These papers conclude that it is essential to design useful and learner-friendly VR learning tasks and activities in order to improve learning outcomes. These activities should be adapted to learners with different learning styles and special abilities. Although this is a novel experience, the gap in these research models lies in the need for new longitudinal studies to verify whether improvements in teaching processes are maintained over time. It is necessary to identify differences in teaching processes, such as context, sample, duration, cultural background or learning programs with different content.

There is a need to deepen and explore the influence of virtual reality on the relationship between motivation and learning performance. That is, it is desirable to know whether students' disappointment can have a negative influence on their learning, using qualitative and quantitative methods in the study of prolonged periods.

VR learning environments

Geng et al. ( 2021 ) explore the pedagogical potential of Interactive Spherical Virtual reality based on video in geographic education, considering the perspective of teachers, they were given an introduction to this technology to know the acceptance, creation and experience, it was intended that teachers know the potential of this technology for teaching and learning purposes, the main concerns were the technological integration in pedagogy, they find that they need more professional development to design and refine this methodology. Perhaps it is more change adversity that is reflected in the need to ask for more training in the use of technology.

VR learning environments are explained by Huang et al. ( 2010 ) who indicate that there is a shift in learning from conventional multimedia to a more immersive, interactive, intuitive and exciting VR learning environment. It combines positive pedagogy and the use of technological innovations that are immersive and trigger the imagination of the learner.

Fowler, 2015 tried to give a more pedagogical description of adopting learning in three-dimensional (3-D) virtual learning environments (VLE) using a "design for learning" perspective that is useful for those who design learning activities in 3D VLEs, but considers that the risk of high-fidelity 3D VLEs is that using them to create virtual classrooms that "feel" and look like real classrooms means that they miss the opportunity to create pedagogically new and innovative learning environments.

Yang et al. ( 2010 ) investigated designing and developing a physically interactive learning environment. This was a PILE system that integrated VR video capture technology in a classroom. The group using the system showed a significant difference in pre-test and post-test knowledge. Makransky and Petersen ( 2019 ) believe that VR has the potential to enrich students' educational experiences. The authors investigated the affective and cognitive factors that play a role in learning when using desktop virtual reality simulation and concluded that learners can benefit from desktop virtual reality simulation in which emphasis is given to effective virtual reality features with a high level of usability.

The works analyzed emphasize that previous experiences with virtual reality in education have improved significantly. Although in the beginning they only used a mouse and keyboard as input devices, the benefits and educational effectiveness of 3D virtual learning and new virtual tools such as the PILE system, which allows students to interact with objects on the screen through physical movements, are gradually emerging.

Although technology has accompanied the teaching process exponentially in recent years, replacing traditional whiteboards with smart boards and VR elements, the gap in these research models lies in the need for new research is needed to explore the variables that may affect learning outcomes when using VR simulations. The described works suggest that it would be convenient to explore aspects such as duration, users' prior knowledge or dual cognitive/affective component.

The study of individual student differences, the long-term implications for knowledge acquisition, the frequent use of technology outside of teaching, the ease of use of different VR tools and the willingness of teachers to implement new VR-based utilities are also considered. This analysis evidences the importance of VR-based simulation processes, especially in areas of knowledge development that teachers deem necessary, allowing a balance between the cost and benefit of the experiences obtained.

Use of VR in various fields of knowledge

Osti et al. ( 2021 ) They seek to train construction workers using a novel VR system, this simulated a virtual training site, implemented a 3D training video with a VR head-mounted display, and compared it with a second group shown simple 2-D instructional video training, the first group presented better results in terms of retention, task performance, learning speed and participation. The practical application of VR as a teaching and learning tool is remarkable.

Schmidt and Glaser ( 2021 ) investigated the use of virtual reality by individuals with autism using 360-degree video modeling and headset-based virtual reality to investigate skills acquirement in adults on the autism spectrum in order to promote safety and the appropriate use of public transport. The results suggest a very positive learning experience and that the intervention is feasible and relevant for the unique needs of the target population.

Vélaz et al. ( 2014 ) studied the influence of interaction technology on the learning process when performing assembly tasks and learning processes using games and VR applications for industrial education. Sampaio et al. ( 2013 ) investigated the use of VR in civil engineering education by using it as a tool to create interactive applications as part of research work with students in which VR applications were developed for use in the construction industry.

A study by Eaves et al. ( 2011 ) determined the effects of two variations of real-time VR and feedback when learning a complex dance movement. Crocetta et al. ( 2018 ) presented and described a VR software package that helps in the rehabilitation of people living with disabilities. The findings of the study suggest that motor skills could be influenced differently depending on the environment and interface in which the software is used.

Learning with VR apps, devices or games

Chen and Hsu ( 2020 ) used a VR game-based English mobile learning application to investigate the effectiveness in English learning from a cognitive and psychological perspective, finding that interaction with the virtual reality application and the challenges of a game-based design allow students to enter the flow state easily and enhance their motivation to learn.

Authors M. Zhang et al. ( 2018 ) studied recent developments in game-based VR educational laboratories. According to the author there are several inherent disadvantages of VR that prevent its widespread deployment in the educational field such as unrealistic representation, lack of customization and flexibility, financial feasibility and the physical and psychological discomfort of users.

Sood and Singh ( 2018 ) considered that educational games for electroencephalography (EEG) can be widely used to improve the cognitive and learning skills of students. This can be achieved with the combination of VR and computing that provides accessible e-learning education worldwide. Psotka ( 2013 ) indicated that new technology such as VR and educational games can often disrupt established practices and are therefore considered disruptive technologies. The author believes however that they are appropriate for education and training today but have not been accepted in education due to changing social lifestyles.

Learning processes using simulation

Makransky et al. ( 2020 ) investigated the value of using immersive virtual reality (IVR) laboratory simulations in science education in two studies. The first study used an IVR laboratory safety simulation with pre- and post-test design. The second study compared the value of using IVR simulation and video simulation for learning the topic of DNA analysis. The results show that in both groups there were significant gains in self-efficacy and physical outcome expectations, but the increase in career aspirations and personal outcome expectations did not reach statistical significance.

Hsu et al. ( 2016 ) considered that visual simulation technologies have received considerable attention in learning. A vehicle driving simulation system was created to assist novice drivers in practicing their skills by considering various environmental driving factors that may be encountered while traveling. Dubovi et al. ( 2017 ) evaluated the effectiveness of VR learning simulation in pharmacology for higher education students requiring special skills to learn about medications and the procedure for administering them. The results revealed higher conceptual and procedural knowledge than with solely lecture-based learning.

Topics published during the Covid-19 pandemic

Wu et al. ( 2021 ) They use an immersive virtual reality approach based on video, they developed a landscape architecture VR learning system, due to the fact that during the Covid-19 pandemic the fields are closed, in addition, online education lacks the necessary scenarios for the courses taught during the pandemic, so better results and learning attitudes are achieved than students not subjected to this VR system. The importance of VR as a learning tool is evidenced in the face of the limitation of a real environment, here technology becomes an important ally.

Paszkiewicz et al. ( 2021 ) presented an educational process for Industry 4.0 that included the design, creation, implementation and evaluation of individual courses implemented in a virtual reality environment, identifying significant advantages and disadvantages of VR-based education. The development and implementation of appropriate courses in the virtual reality environment was found to reduce costs and increase the safety and efficiency of activities.

Yerden & Akkuş, ( 2020 ) examined the effects of the use of a Virtual Reality Supported Remote Access Laboratory (VRRALAB) system using remote access and virtual reality technologies on students' learning experience. The interactive use of a real device with a VR-supported remote access laboratory environment does not have any risks for novice users. The results indicate that remote access labs using virtual reality are likely to increase learning quality and student satisfaction levels.

Taçgın, ( 2020 ) investigated the characteristics of an immersive virtual reality learning environment (IVRLE) by evaluating perceived simulation effectiveness for student learning, attitude, and confidence by using gesture interaction to teach preoperative surgical procedures and concepts to undergraduate nursing students. Well-designed and targeted IVRLE was found to help to improve students' confidence in practical skills. Wang, ( 2020 ) applied virtual reality techniques in modular teaching to construct virtual simulation teaching resources and built two teaching modules that are visual, interactive, scalable, upgradable and optimizable. The results of the research suggest a new method of modular teaching and are a useful reference.

The results showed that the production of documents on VR in education has increased since 2015, possibly due to the increase in interest in virtual reality technology. There were important changes in the field in 2014 with the introduction of the Oculus Rift Frame. The interest in VR is reflected in the number of publications in 2018 and has been increasing with the changes in education systems due to the Covid-19 pandemic. The productivity in the last two years has been much higher than before.

The growth is then constant from 2015 onwards, increasing 10 times by 2020. There was a high number of publications in 2019 and a much higher number in 2020, possibly due to the new interest in VR and the impact of the Covid-19 pandemic. Higher production is expected for VR in Education in 2021. The recent growth is consistent with the results of the study on VR and motivation in the educational field (Soto et al., 2020 ) and with the work in the field of rehabilitation (Huang et al., 2016 ).

Figure  14 shows that China leads the number of publications with 273 articles, but interestingly the journals with the highest number of publications are the German International Journal of Emerging Technologies in Learning and Computers & Education from the UK. These journals also had the highest impact in the studied dataset. The difference of China as the leader of publications and the publishing journals may be due to the papers contained in the selection process. From the total of 298 journals and 718 papers that make up the dataset, there are 72 papers that were written by a single author. Most publications were written by an average of 2.95 authors, which means that from 1939 authors there are 1867 authors with papers with multiple authors. These papers are widely cited, with an average number of 10.03 citations per paper over a 10-year period and an annual average number of 2.49 citations per paper. Interestingly, most citations were in 2010 with an average annual number of 6.97 citations per paper, 2014 with 7.13 citations per paper and 2018 with 5.99. The data for the Covid-19 pandemic period, which started in 2020, is clearly shown in Fig.  4 . The two most cited papers were (Merchant et al., 2014 ) with 574 citations in volume 70 of the Computers & Education journal and (Huang et al., 2010 ) with 321 citations in volume 55 of the same journal.

Figure  7 presents the authors with the highest impact after analyzing values of the h-index. In this study they were found to be Chen Y. and Jong M. both with an h-index value of 7. Figure  6 shows the authors Chen W., Chen Y. and Lee J., who are considered very productive because they have 8 publications.

Co-occurrence was used to identify the conceptual structure by analyzing the words in the Keywords Plus of cited articles. The words that appear most often were found to be virtual reality with 107 occurrences, students with 70, education with 66, learning with 58 occurrences, and environments with 44. All these words are closely related to the search topics although technology, performance, simulation, design and learning systems were also found. This information is shown in Fig.  10 .

The words covering the topics in the Keywords Plus of the documents were analyzed by means of a factorial map and a cluster analysis, as shown in Fig.  9 . Two dimensions were used, the first of which covers topics related to VR visualization devices by users of virtual reality educational products, and the second dimension covers VR uses in education. Words related to visualization, attention and motivation of students, the use of serious games, interactive learning environments, online learning and user experience to strengthen the use of VR in education were all found in this analysis stage.

Co-citation was used to analyze the intellectual structure. The authors of the documents were analyzed, and Lee was found to have a co-citation relationship Merchant, Dalgarno, Smith, Mikropoulos, Davis and Bouman, while a second group of co-citation authors was formed by Chen, Wang, Huang, Zhang, Chang, Yang and Lin. This information is shown in Fig.  11 .

An inductive analysis found that the most cited papers were into the major categories of virtual reality-based instruction and learning, learning environments using VR, VR for teaching learning processes, instructional design and VR as a tool that enhances learning processes.

VR learning environments contain different systems where VR is applied in education, ranging from conventional multimedia to a physically interactive learning environment.

Research was found that investigated the use of VR in different fields of knowledge and various areas of education such as civil engineering, production lines and rehabilitation.

Learning processes incorporating the use of VR applications, devices or games are considered disruptive technologies that alter the usual teaching, cognitive and learning practices of students. However, the combination of VR with computing was seen to provide effective e-learning.

Articles were also found in the dataset about research using simulation models, for example driving vehicles, assembly lines, operations in the medical field. These simulation models used three-dimensional technology for the teaching process.

The research topics published during the Covid-19 pandemic examined the new trend in the field of education caused by the abrupt change from face-to-face teaching to virtual, online teaching. Research topics included deep learning and machine learning using artificial intelligence as alternatives to everyday teaching technologies and platforms. These included robot teleoperation, remote access laboratories, Virtual Reality-Based Cognitive Telerehabilitation Systems, Machine Learning Predictions, Immersive Virtual Reality learning environments, intelligent virtual reality technology, distance learning classrooms using machine learning and virtual reality, Virtual Reality-Interactive Classroom, Student Orientation in Distance Education Programs, Virtual Reality and BIM Methodology for Teaching–Learning Improvement, virtual reality-based gaming instruction and virtual reality for students' adaptive learning among others.

The six categories of the second analysis showed that improvements in VR learning processes have occurred in recent years and important advances have been made in the application and use of this technology. However, even though the engagement and motivation of students can be improved by using this technology, there still remains a lot to be discovered about the use of e-learning tools. VR learning environments have advanced significantly from multimedia to interactive environments with desktop and immersive VR processes, 360-degree tours and various applications and games. Education has generally adopted these changes slowly with most attention coming from the new generations of students. VR technology has existed for many years but has not been adopted as quickly as would be expected by educational centers, where it is seen as a disruptive process and is more commonly used by students for recreational purposes. The results of this study show that despite the multiple applications of VR in different fields of education, there is little evidence of incorporating them in teaching–learning processes. Educators have, however, quickly resorted to the use of technology platforms to teach because of the sudden change from face-to-face to virtual, online classes due to the Covid-19 pandemic. With the many available applications of VR in education and given that students have accepted the use of VR for recreational purposes, it can be incorporated as part of educational and training courses and demonstrations, as well as being useful for evaluating students.

Finally, the quality assessment of this research work was evaluated applying contrasted methods, following recommendations proposed by Ramey and Rao ( 2011 ) and Rao and Ramey ( 2011 ). Similarly, the quality of the research work was evaluated by incorporating bibliometric criteria, especially in the automation of the whole process, by Pulsiri and Vatananan-Thesenvitz ( 2018 ). External validity was contrasted with the criteria of dos Santos Rocha and Fantinato ( 2013 ) and Nguyen-Duc et al. ( 2015 ).

Conclusions

Bibliometric analysis is a valuable tool that can be used to reveal the evolution of the articles contained in the dataset and answer research questions.

The conceptual structure shows that the most-used words are related to the search terms, although some words were found to be used with higher frequency. These were virtual reality, students, education, e learning, teaching, while the main topics investigated range from basic ones such as virtual reality and education to technology, teaching, visualization, student motivation and attention, e-learning, learning system, deep learning, and immersive virtual reality.

Recent publications show an increasing interest in VR, but rather than delving deeper into this technology other technological options are being explored with combinations of VR and other technology and systems. The pandemic improved our abilities to adapt and innovate with virtuality, along with teleworking and virtual teams (Garro-Abarca et al., 2020 , 2021 ). There has been a lot of research into tools that allow educators to improve and even reinvent teaching processes. Virtual reality applications are interactive and immersive with telepresence and education might reconsider its opinion of this technology and take advantage of it to make classes more enjoyable with the new normality we are living.

One of the contributions of this study has been to confirm the progress of VR technology. In the future, educational centers will be able to solve many challenges, such as learning by experiencing and interacting with an environment, instead of passively receiving the information to be assimilated. VR favors the motivation and involvement of students and educational staff in general, in addition to increasing the speed of learning. b. The frontier and future of VR learning environments. On the other hand, the frontier and future of VR learning environments, the article predicts that the increasing diffusion of virtual reality learning environments forces to create limits. One of the limits is based on the capacity of these didactic tools to improve their effectiveness at a training level compared to other more traditional methods. In addition, other aspects such as user privacy and equal opportunities for all students must be taken into consideration.

Thus, the trends of type and use of VR in different fields of knowledge will be directed towards the inclusion of experiential VR models, adapted to each field of knowledge. The aim is to create learning vehicles that enhance the acquisition of knowledge by students in specific fields of knowledge. In this context, the training of teachers in the use of these technologies within a clear educational framework that is adaptable to the different academic disciplines will be increasingly important. In this sense, the future of VR in education will depend to a large extent on the motivation generated in students, and on being able to identify the necessary characteristics to achieve an adequate level of learning through VR. The push for VR-based education, brought about by the Covid-19 pandemic, is expected to make this style of learning one of the educational preferences in the future, especially for self-learning and teaching in certain areas where VR technology is more developed.

This study was limited to investigating the use of virtual reality technology in education. Articles dealing with the medical field were not selected as primary data sources because the focus of research was on the application of VR in teaching and learning processes in secondary and university education, as well as other applications in conferences or training processes. However, articles were included from multidisciplinary areas that include research in the medical field to broaden the spectrum of practical applications of this technology.

Suggestions for future bibliometric analyses include the evolution of other subject areas, using new search terms that allow other articles related to the field of education to be included for a broader analysis of the metadata and investigating other present and future lines of research.

Acknowledgements

Not applicable.

Authors' contributions

All authors read and approved the final manuscript.

Open Access funding provided thanks to the CRUE-CSIC agreement with Springer Nature.

Data availability

Declarations.

The authors declare that they have no conflicting interests.

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

• Alfalah SF. Perceptions toward adopting virtual reality as a teaching aid in information technology. Education and Information Technologies. 2018; 23 (6):2633–2653. doi: 10.1007/s10639-018-9734-2. [ CrossRef ] [ Google Scholar ]
• Allcoat, D., & von Mühlenen, A. (2018). Learning in virtual reality: Effects on performance, emotion and engagement. Research in Learning Technology , 26 . 10.25304/rlt.v26.2140
• Altınay Ozdemir MA, Goktas LS. Research trends on digital detox holiday: A bibliometric analysis, 2012–2020. Tourism & Management Studies. 2021; 17 (3):21–35. doi: 10.18089/tms.2021.170302. [ CrossRef ] [ Google Scholar ]
• Álvarez-García J, Maldonado-Erazo CP, Río-Rama D, Cruz M, Castellano-Álvarez FJ. Cultural Heritage and Tourism Basis for Regional Development: Mapping of Scientific Coverage. Sustainability. 2019; 11 (21):6034. doi: 10.3390/su11216034. [ CrossRef ] [ Google Scholar ]
• Aria M, Cuccurullo C. bibliometrix: An R-tool for comprehensive science mapping analysis. Journal of Informetrics. 2017; 11 (4):959–975. doi: 10.1016/j.joi.2017.08.007. [ CrossRef ] [ Google Scholar ]
• Bardakci S, Soylu MY, Akkoyunlu B, et al. Collaborations, concepts, and citations in educational technology: A trend study via bibliographic mapping. Education and Information Technologies. 2022; 27 :4321–4346. doi: 10.1007/s10639-021-10785-9. [ CrossRef ] [ Google Scholar ]
• Beyer-Berjot L, Berdah S, Hashimoto DA, Darzi A, Aggarwal R. A virtual reality training curriculum for laparoscopic colorectal surgery. Journal of Surgical Education. 2016; 73 (6):932–941. doi: 10.1016/j.jsurg.2016.05.012. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Blanco-Mesa F, Merigó JM, Gil-Lafuente AM. Fuzzy decision making: A bibliometric-based review. Journal of Intelligent & Fuzzy Systems. 2017; 32 (3):2033–2050. doi: 10.3233/JIFS-161640. [ CrossRef ] [ Google Scholar ]
• Bonetti, F., Warnaby, G., & Quinn, L. (2018). Augmented reality and virtual reality in physical and online retailing: A review, synthesis and research agenda. In Augmented reality and virtual reality (pp. 119–132). Springer. 10.1007/978-3-319-64027-3_9
• Börner K, Chen C, Boyack KW. Visualizing knowledge domains. Annual Review of Information Science and Technology. 2003; 37 (1):179–255. doi: 10.1002/aris.1440370106. [ CrossRef ] [ Google Scholar ]
• Brey, P. (2014). Virtual reality and computer simulation. In Ethics and emerging technologies (pp. 315–332). Palgrave Macmillan. 10.1057/9781137349088_21
• Buehl D. Classroom strategies for interactive learning. Stenhouse Publishers; 2017. [ Google Scholar ]
• Calabuig-Moreno F, González-Serrano MH, Fombona J, García-Tascón M. The emergence of technology in physical education: A general bibliometric analysis with a focus on virtual and augmented reality. Sustainability. 2020; 12 (7):2728. doi: 10.3390/su12072728. [ CrossRef ] [ Google Scholar ]
• Castillo HGC. The triple helix model as a means for university-business linkage. National Journal of Administration. 2010; 1 (1):85–94. doi: 10.22458/rna.v1i1.286. [ CrossRef ] [ Google Scholar ]
• Chen YL, Hsu CC. Self-regulated mobile game-based English learning in a virtual reality environment. Computers & Education. 2020; 154 :103910. doi: 10.1016/j.compedu.2020.103910. [ CrossRef ] [ Google Scholar ]
• Cheng F-F, Huang Y-W, Yu H-C, Wu C-S. Mapping knowledge structure by keyword co-occurrence and social network analysis. Library Hi Tech. 2018; 36 (4):636–650. doi: 10.1108/lht-01-2018-0004. [ CrossRef ] [ Google Scholar ]
• Cobo MJ, López-Herrera AG, Herrera-Viedma E, Herrera F. Science mapping software tools: Review, analysis, and cooperative study among tools. Journal of the American Society for Information Science and Technology. 2011; 62 (7):1382–1402. doi: 10.1002/asi.21525. [ CrossRef ] [ Google Scholar ]
• Crocetta TB, Araújo LV, Guarnieri R, Massetti T, Ferreira FHIB, De Abreu LC, Mello Monteiro CB. Virtual reality software package for implementing motor learning and rehabilitation experiments. Virtual Reality. 2018; 22 (3):199–209. doi: 10.1007/s10055-017-0323-2. [ CrossRef ] [ Google Scholar ]
• Danvila-del-Valle I, Estévez-Mendoza C, Lara FJ. Human resources training: A bibliometric analysis. Journal of Business Research. 2019; 101 :627–636. doi: 10.1016/j.jbusres.2019.02.026. [ CrossRef ] [ Google Scholar ]
• Di Natale AF, Repetto C, Riva G, Villani D. Immersive virtual reality in K-12 and higher education: A 10-year systematic review of empirical research. British Journal of Educational Technology. 2020; 51 (6):2006–2033. doi: 10.1111/bjet.13030. [ CrossRef ] [ Google Scholar ]
• dos Santos Rocha R, Fantinato M. The use of software product lines for business process management: A systematic literature review. Information and Software Technology. 2013; 55 (8):1355–1373. doi: 10.1016/j.infsof.2013.02.007. [ CrossRef ] [ Google Scholar ]
• Dubovi I, Levy ST, Dagan E. Now I know how! The learning process of medication administration among nursing students with non-immersive desktop virtual reality simulation. Computers & Education. 2017; 113 :16–27. doi: 10.1016/j.compedu.2017.05.009. [ CrossRef ] [ Google Scholar ]
• Durán Sánchez, A., Álvarez-García, J., Río-Rama, D., & Cruz, M. (2014). Active tourism research: A literature review. ROTUR, 8 , 62–76.
• Durán-Sánchez A, Álvarez-García J, Río-Rama D, Cruz M, Oliveira C. Religious tourism and pilgrimage: Bibliometric overview. Religions. 2018; 9 (9):249. doi: 10.3390/rel9090249. [ CrossRef ] [ Google Scholar ]
• Durán-Sánchez A, Álvarez-García J, Río-Rama MDL, Rosado- Cebrián B. Science mapping of the knowledge base on tourism innovation. Sustainability. 2019; 11 (12):3352. doi: 10.3390/su11123352. [ CrossRef ] [ Google Scholar ]
• Eaves DL, Breslin G, Van Schaik P, Robinson E, Spears IR. The short-term effects of real-time virtual reality feedback on motor learning in dance. Presence: Teleoperators and Virtual Environments. 2011; 20 (1):62–77. doi: 10.1162/pres_a_00035. [ CrossRef ] [ Google Scholar ]
• Fagan M, Kilmon C, Pandey V. Exploring the adoption of a virtual reality simulation: The role of perceived ease of use, perceived usefulness and personal innovativeness. Campus-Wide Information Systems. 2012; 29 (2):117–127. doi: 10.1108/10650741211212368. [ CrossRef ] [ Google Scholar ]
• Falagas ME, Pitsouni EI, Malietzis GA, Pappas G. Comparison of PubMed, Scopus, web of science, and Google scholar: Strengths and weaknesses. The FASEB Journal. 2008; 22 (2):338–342. doi: 10.1096/fj.07-9492lsf. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Folgado-Fernández JA, Palos-Sánchez PR, Camacho MA. Motivaciones, formación y planificación del trabajo en equipo para entornos de aprendizaje virtual. Interciencia. 2020; 45 (2):102–109. [ Google Scholar ]
• Fowler C. Virtual reality and learning: Where is the pedagogy? British Journal of Educational Technology. 2015; 46 (2):412–422. doi: 10.1111/bjet.12135. [ CrossRef ] [ Google Scholar ]
• Freina, L., Bottino, R., & Tavella, M. (2016). From e-learning to VR-learning: An example of learning in an immersive virtual world. Journal of E-Learning and Knowledge Society , 12 (2). https://www.learntechlib.org/p/173465/
• Garro-Abarca V, Palos-Sanchez P, Aguayo-Camacho M. Virtual teams in times of pandemic: Factors that influence performance. Frontiers in Psychology. 2021; 12 :232. doi: 10.3389/fpsyg.2021.624637. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Garro-Abarca VM, Palos-Sanchez PR, Rus-Arias E. Working in virtual teams: A systematic literature review and a bibliometric analysis. IEEE Access. 2020; 8 :168923–168940. doi: 10.1109/ACCESS.2020.3023546. [ CrossRef ] [ Google Scholar ]
• Geng J, Chai CS, Jong MSY, Luk ETH. Understanding the pedagogical potential of Interactive Spherical Video-based Virtual Reality from the teachers’ perspective through the ACE framework. Interactive Learning Environments. 2021; 29 (4):618–633. doi: 10.1080/10494820.2019.1593200. [ CrossRef ] [ Google Scholar ]
• Gokhale A, Mulay P, Pramod D, Kulkarni R. A bibliometric analysis of digital image forensics. Science & Technology Libraries. 2020; 1 :18. doi: 10.1080/0194262x.2020.1714529. [ CrossRef ] [ Google Scholar ]
• Gómez-García, M., Trujillo-Torres, J. M., Aznar-Díaz, I., & Cáceres-Reche, M. P. (2018). Augment reality and virtual reality for the improvement of spatial competences in Physical Education . 10.14198/jhse.2018.13.proc2.03
• Guo Y-M, Huang Z-L, Guo J, Li H, Guo X-R, Nkeli MJ. Bibliometric analysis on smart cities research. Sustainability. 2019; 11 (13):3606. doi: 10.3390/su11133606. [ CrossRef ] [ Google Scholar ]
• Guttentag DA. Virtual reality: Applications and implications for tourism. Tourism Management. 2010; 31 (5):637–651. doi: 10.1016/j.tourman.2009.07.003. [ CrossRef ] [ Google Scholar ]
• Hassan SU, Visvizi A, Waheed H. The ‘who’and the ‘what’in international migration research: Data-driven analysis of Scopus-indexed scientific literature. Behaviour & Information Technology. 2019; 38 (9):924–939. doi: 10.1080/0144929x.2019.1583282. [ CrossRef ] [ Google Scholar ]
• Hen, L. B. (2019). Exploring Surgeon’s Acceptance of Virtual Reality Headset for Training. In Augmented Reality and Virtual Reality (pp. 291–304). Springer. 10.1007/978-3-030-06246-0_21
• Heradio R, De La Torre L, Galan D, Cabrerizo FJ, Herrera-Viedma E, Dormido S. Virtual and remote labs in education: A bibliometric analysis. Computers & Education. 2016; 98 :14–38. doi: 10.1016/j.compedu.2016.03.010. [ CrossRef ] [ Google Scholar ]
• Hernández-Torrano, D., & Kuzhabekova, A. (2019). The state and development of research in the field of gifted education over 60 years: A bibliometric study of four gifted education journals (1957–2017). High Ability Studies , 1–23. 10.1080/13598139.2019.1601071
• Hewawalpita S, Herath S, Perera I, Meedeniya D. effective learning content offering in MOOCs with virtual reality-an exploratory study on learner experience. Journal of Universal Computer Science. 2018; 24 (2):129–148. [ Google Scholar ]
• Hsu, K. S., Jiang, J. F., Wei, H. Y., & Lee, T. H. (2016). Application of the environmental sensation learning vehicle simulation platform in virtual reality. Eurasia Journal of Mathematics, Science & Technology Education , 12 (5). 10.12973/eurasia.2016.1525a
• Huang, H. M., & Liaw, S. S. (2018). An analysis of learners’ intentions toward virtual reality learning based on constructivist and technology acceptance approaches. International Review of Research in Open and Distributed Learning , 19 (1). 10.19173/irrodl.v19i1.2503
• Huang HM, Rauch U, Liaw SS. Investigating learners’ attitudes toward virtual reality learning environments: Based on a constructivist approach. Computers & Education. 2010; 55 (3):1171–1182. doi: 10.1016/j.compedu.2010.05.014. [ CrossRef ] [ Google Scholar ]
• Huang Y, Huang Q, Ali S, Zhai X, Bi X, Liu R. Rehabilitation using virtual reality technology: A bibliometric analysis, 1996–2015. Scientometrics. 2016; 109 (3):1547–1559. doi: 10.1007/s11192-016-2117-9. [ CrossRef ] [ Google Scholar ]
• Huber JC. A new model that generates Lotka’s law. Journal of the American Society for Information Science and Technology. 2002; 53 (3):209–219. doi: 10.1002/asi.10025. [ CrossRef ] [ Google Scholar ]
• Hussin, N. H., Jaafar, J., & Downe, A. G. (2011). Assessing educators’ acceptance of Virtual Reality (VR) in the classroom using the Unified Theory of Acceptance and Use of Technology (UTAUT. International Visual Informatics Conference , 216–225. 10.1007/978-3-642-25191-7_21
• Huttar CM, BrintzenhofeSzoc K. Virtual reality and computer simulation in social work education: A systematic review. Journal of Social Work Education. 2020; 56 (1):131–141. doi: 10.1080/10437797.2019.1648221. [ CrossRef ] [ Google Scholar ]
• Hwang WY, Hu SS. Analysis of peer learning behaviors using multiple representations in virtual reality and their impacts on geometry problem solving. Computers & Education. 2013; 62 :308–319. doi: 10.1016/j.compedu.2012.10.005. [ CrossRef ] [ Google Scholar ]
• Jensen L, Konradsen F. A review of the use of virtual reality head-mounted displays in education and training. Education and Information Technologies. 2018; 23 (4):1515–1529. doi: 10.1007/s10639-017-9676-0. [ CrossRef ] [ Google Scholar ]
• Johnson-Glenberg MC. Immersive VR and education: Embodied design principles that include gesture and hand controls. Frontiers in Robotics and AI. 2018; 5 :81. doi: 10.3389/frobt.2018.00081. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Kamińska D, Sapiński T, Wiak S, Tikk T, Haamer RE, Avots E, Anbarjafari G. Virtual reality and its applications in education: Survey. Information. 2019; 10 (10):318. doi: 10.3390/info10100318. [ CrossRef ] [ Google Scholar ]
• Kartiko I, Kavakli M, Cheng K. Learning science in a virtual reality application: The impacts of animated-virtual actors’ visual complexity. Computers & Education. 2010; 55 (2):881–891. doi: 10.1016/j.compedu.2010.03.019. [ CrossRef ] [ Google Scholar ]
• Katsioloudis, P., Jones, M., & Jovanovic, V. (2017). Use of virtual reality head-mounted displays for engineering technology students and implications on spatial visualization ability as measured through rotational view drawings. The Engineering Design Graphics Journal , 81 (1). Winter
• Kushairi, N., & Ahmi, A. (2021). Flipped classroom in the second decade of the Millenia: A bibliometrics analysis with Lotka’s law. Education and Information Technologies , 26, 4401–4431. 10.1007/s10639-021-10457-8 [ PMC free article ] [ PubMed ]
• Kwon C. Verification of the possibility and effectiveness of experiential learning using HMD-based immersive VR technologies. Virtual Reality. 2019; 23 (1):101–118. doi: 10.1007/s10055-018-0364-1. [ CrossRef ] [ Google Scholar ]
• Lampert B, Pongracz A, Sipos J, Vehrer A, Horvath I. MaxWhere VR-learning improves effectiveness over clasiccal tools of e-learning. Acta Polytechnica Hungarica. 2018; 15 (3):125–147. doi: 10.12700/aph.15.3.2018.3.8. [ CrossRef ] [ Google Scholar ]
• Lazar I, Panisoara IO. Understanding the role of modern technologies in education: A scoping review protocol. Psychreg J. Psychol. 2018; 2 :74–86. [ Google Scholar ]
• Le QT, Pedro A, Park CS. A social virtual reality based construction safety education system for experiential learning. Journal of Intelligent & Robotic Systems. 2015; 79 (3–4):487–506. doi: 10.1007/s10846-014-0112-z. [ CrossRef ] [ Google Scholar ]
• Lee EAL, Wong KW, Fung CC. How does desktop virtual reality enhance learning outcomes? A structural equation modeling approach. Computers & Education. 2010; 55 (4):1424–1442. doi: 10.1016/j.compedu.2010.06.006. [ CrossRef ] [ Google Scholar ]
• Lövquist E, Shorten G, Aboulafia A. Virtual reality-based medical training and assessment: The multidisciplinary relationship between clinicians, educators and developers. Medical Teacher. 2012; 34 (1):59–64. doi: 10.3109/0142159x.2011.600359. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Lv Z, Li X, Li W. Virtual reality geographical interactive scene semantics research for immersive geography learning. Neurocomputing. 2017; 254 :71–78. doi: 10.1016/j.neucom.2016.07.078. [ CrossRef ] [ Google Scholar ]
• Maas MJ, Hughes JM. Virtual, augmented and mixed reality in K–12 education: A review of the literature. Technology, Pedagogy and Education. 2020; 29 (2):231–249. doi: 10.1080/1475939x.2020.1737210. [ CrossRef ] [ Google Scholar ]
• Makransky G, Lilleholt L. A structural equation modeling investigation of the emotional value of immersive virtual reality in education. Educational Technology Research and Development. 2018; 66 (5):1141–1164. doi: 10.1007/s11423-018-9581-2. [ CrossRef ] [ Google Scholar ]
• Makransky G, Petersen GB. Investigating the process of learning with desktop virtual reality: A structural equation modeling approach. Computers & Education. 2019; 134 :15–30. doi: 10.1016/j.compedu.2019.02.002. [ CrossRef ] [ Google Scholar ]
• Makransky G, Petersen GB, Klingenberg S. Can an immersive virtual reality simulation increase students’ interest and career aspirations in science? British Journal of Educational Technology. 2020; 51 (6):2079–2097. doi: 10.1111/bjet.12954. [ CrossRef ] [ Google Scholar ]
• Mann, M. K., Liu-Thompkins, Y., Watson, G. S., & Papelis, Y. E. (2015). A multidisciplinary examination of 3D virtual shopping environments: Effects on consumer perceptual and physiological responses. In Ideas in marketing: Finding the new and polishing the old (pp. 752–755). Springer. 10.1007/978-3-319-10951-0_277
• Martínez-López FJ, Merigó JM, Valenzuela-Fernández L, Nicolás C. Fifty years of the european journal of marketing: A bibliometric analysis. European Journal of Marketing. 2018; 52 (1/2):439–468. doi: 10.1108/EJM-11-2017-0853. [ CrossRef ] [ Google Scholar ]
• Mas-Tur A, Modak NM, Merigó JM, Roig-Tierno N, Geraci M, Capecchi V. Half a century of quality & quantity: A bibliometric review. Quality & Quantity. 2019; 53 (2):981–1020. doi: 10.1007/s11135-018-0799-1. [ CrossRef ] [ Google Scholar ]
• McGovern E, Moreira G, Luna-Nevarez C. An application of virtual reality in education: Can this technology enhance the quality of students’ learning experience? Journal of Education for Business. 2019; 95 (7):490–496. doi: 10.1080/08832323.2019.1703096. [ CrossRef ] [ Google Scholar ]
• Merchant, Z., Goetz, E. T., Keeney-Kennicutt, W., Kwok, O. M., Cifuentes, L., & Davis, T. J. (2012). The learner characteristics, features of desktop 3D virtual reality environments, and college chemistry instruction: A structural equation modeling analysis. Computers & Education, 59 (2), 551–568. 10.1016/j.compedu.2012.02.004
• Merchant Z, Goetz ET, Cifuentes L, Keeney-Kennicutt W, Davis TJ. Effectiveness of virtual reality-based instruction on students’ learning outcomes in K-12 and higher education: A meta- analysis. Computers & Education. 2014; 70 :29–40. doi: 10.1016/j.compedu.2013.07.033. [ CrossRef ] [ Google Scholar ]
• Mihelj, M., Novak, D., & Beguš, S. (2014). Virtual reality technology and applications . 10.1007/978-94-007-6910-6
• Milian EZ, Spinola MDM, Carvalho MM. Fintechs: A literature review and research agenda. Electronic Commerce Research and Applications. 2019; 34 :100833. doi: 10.1016/j.elerap.2019.100833. [ CrossRef ] [ Google Scholar ]
• Moro, C., Štromberga, Z., & Stirling, A. (2017). Virtualisation devices for student learning: Comparison between desktop-based (Oculus Rift) and mobile-based (Gear VR) virtual reality in medical and health science education. Australasian Journal of Educational Technology, 33 (6). 10.14742/ajet.3840
• Mystakidis S, Berki E, Valtanen JP. Deep and meaningful e-learning with social virtual reality environments in higher education: A systematic literature review. Applied Sciences. 2021; 11 (5):2412. doi: 10.3390/app11052412. [ CrossRef ] [ Google Scholar ]
• Nguyen-Duc A, Cruzes DS, Conradi R. The impact of global dispersion on coordination, team performance and software quality–a systematic literature review. Information and Software Technology. 2015; 57 :277–294. doi: 10.1016/j.infsof.2014.06.002. [ CrossRef ] [ Google Scholar ]
• Osti F, de Amicis R, Sanchez CA, Tilt AB, Prather E, Liverani A. A VR training system for learning and skills development for construction workers. Virtual Reality. 2021; 25 (2):523–538. doi: 10.1007/s10055-020-00470-6. [ CrossRef ] [ Google Scholar ]
• Palos-Sanchez, P. R., Saura, J. R., & Debasa, F. (2018). The influence of social networks on the development of recruitment actions that favor user interface design and conversions in mobile applications powered by linked data. Mobile Information Systems, 2018.
• Palos-Sanchez, P. R., Folgado-Fernandez, J. A., & Rojas-Sanchez, M. (2022). Virtual reality technology: Analysis based on text and opinion mining. Mathematical Biosciences and Engineering, 19 (8), 7856–7885. 10.3934/mbe.2022367 [ PubMed ]
• Paszkiewicz A, Salach M, Dymora P, Bolanowski M, Budzik G, Kubiak P. Methodology of implementing virtual reality in education for industry 4.0. Sustainability. 2021; 13 (9):5049. doi: 10.3390/su13095049. [ CrossRef ] [ Google Scholar ]
• Pham HC, Dao N, Pedro A, Le QT, Hussain R, Cho S, Park CSK. Virtual field trip for mobile construction safety education using 360-degree panoramic virtual reality. International Journal of Engineering Education. 2018; 34 :1174–1191. [ Google Scholar ]
• Pritchard A. Statistical bibliography or bibliometrics. Journal of Documentation. 1969; 25 (4):348–349. doi: 10.1108/eb026482. [ CrossRef ] [ Google Scholar ]
• Psotka, J. (2013). Educational games and virtual reality as disruptive technologies.   Educational Technology and Society 16 (2), 69–80 https://www.jstor.org/stable/jeductechsoci.16.2.69
• Pulsiri, N., & Vatananan-Thesenvitz, R. (2018). Improving systematic literature review with automation and bibliometrics. In 2018 Portland International Conference on Management of Engineering and Technology (PICMET) (pp. 1–8). IEEE.
• Radianti J, Majchrzak TA, Fromm J, Wohlgenannt I. A systematic review of immersive virtual reality applications for higher education: Design elements, lessons learned, and research agenda. Computers & Education. 2020; 147 :103778. doi: 10.1016/j.compedu.2019.103778. [ CrossRef ] [ Google Scholar ]
• Ramey, J., & Rao, P. G. (2011). The systematic literature review as a research genre. In 2011 IEEE International Professional Communication Conference (pp. 1–7). IEEE.
• Rao, P. G., & Ramey, J. (2011). Use of mobile phones by non-literate and semi-literate people: A systematic literature review. In 2011 IEEE International Professional Communication Conference (pp. 1–10). IEEE.
• Rodríguez ARR, Navarro JR. Intellectual basis of entrepreneurship research: A biometric study. European Journal of Management and Business Economics. 2008; 17 (1):13–38. [ Google Scholar ]
• Roemer RC, Borchardt R. Meaningful metrics: A 21st century librarian’s guide to bibliometrics, altmetrics, and research impact. Association of College and Research Libraries; 2015. [ Google Scholar ]
• Román-Ibáñez V, Pujol-López FA, Mora-Mora H, Pertegal-Felices ML, Jimeno-Morenilla A. A low-cost immersive virtual reality system for teaching robotic manipulators programming. Sustainability. 2018; 10 (4):1102. doi: 10.3390/su10041102. [ CrossRef ] [ Google Scholar ]
• Sampaio A, Rosario D, Gomes A, Santos J. Virtual reality applied on civil engineering education: Construction activity supported on interactive models. International Journal of Engineering Education. 2013; 29 (6):1331–1347. [ Google Scholar ]
• Schmidt, M., & Glaser, N. (2021). Investigating the usability and learner experience of a virtual reality adaptive skills intervention for adults with autism spectrum disorder. Educational Technology Research and Development , 1–35. 10.1007/s11423-021-10005-8
• Shen CW, Ho JT, Ly PTM, Kuo TC. Behavioural intentions of using virtual reality in learning: Perspectives of acceptance of information technology and learning style. Virtual Reality. 2019; 23 (3):313–324. doi: 10.1007/s10055-018-0348-1. [ CrossRef ] [ Google Scholar ]
• Shen H, Zhang J, Yang B, Jia B. Development of an educational virtual reality training system for marine engineers. Computer Applications in Engineering Education. 2019; 27 (3):580–602. doi: 10.1002/cae.22099. [ CrossRef ] [ Google Scholar ]
• Small H. Visualizing science by citation mapping. Journal of the American Society for Information Science. 1999; 50 (9):799–813. doi: 10.1002/(sici)1097-4571(1999)50:9. [ CrossRef ] [ Google Scholar ]
• Sood SK, Singh KD. An Optical-Fog assisted EEG-based virtual reality framework for enhancing E-learning through educational games. Computer Applications in Engineering Education. 2018; 26 (5):1565–1576. doi: 10.1002/cae.21965. [ CrossRef ] [ Google Scholar ]
• Soto, M. N. C., Navas-Parejo, M. R., & Guerrero, A. J. M. (2020). Virtual reality and motivation in the educational context: Bibliometric study of the last twenty years from Scopus . 10.17163/alt.v15n1.2020.04
• Soundararajan K, Ho H, Su B. Sankey diagram framework for energy and exergy flows. Applied Energy. 2014; 136 :1035–1042. doi: 10.1016/j.apenergy.2014.08.070. [ CrossRef ] [ Google Scholar ]
• Srivastava P, Rimzhim A, Vijay P, Singh S, Chandra S. Desktop VR is better than non-ambulatory HMD VR for spatial learning. Frontiers in Robotics and AI. 2019; 6 :50. doi: 10.3389/frobt.2019.00050. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Taçgın Z. The perceived effectiveness regarding Immersive Virtual Reality learning environments changes by the prior knowledge of learners. Education and Information Technologies. 2020; 25 (4):2791–2809. doi: 10.1007/s10639-019-10088-0. [ CrossRef ] [ Google Scholar ]
• Tussyadiah IP, Jung TH, Dieck MC. Embodiment of wearable augmented reality technology in tourism experiences. Journal of Travel Research. 2018; 57 (5):597–611. doi: 10.1177/0047287517709090. [ CrossRef ] [ Google Scholar ]
• Tzanavari A, Tsapatsoulis N. Affective, interactive, and cognitive methods for e-learning design: Creating an optimal education experience. Information Science Reference. 2010 doi: 10.4018/978-1-60566-940-3. [ CrossRef ] [ Google Scholar ]
• Van Eck NJ, Waltman L. Bibliometric mapping of the computational intelligence field. International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems. 2007; 15 (05):625–645. doi: 10.1142/s0218488507004911. [ CrossRef ] [ Google Scholar ]
• Velasco CAB, Parra VFG, García CQ. Evolution of the literature on family business as a scientific discipline. Cuadernos De Economía y Dirección De La Empresa. 2011; 14 (2):78–90. doi: 10.1016/j.cede.2011.02.004. [ CrossRef ] [ Google Scholar ]
• Vélaz, Y., Rodríguez Arce, J., Gutiérrez, T., Lozano-Rodero, A., & Suescun, A. (2014). The influence of interaction technology on the learning of assembly tasks using virtual reality. Journal of Computing and Information Science in Engineering , 14 (4). 10.1115/1.4028588
• Vignais N, Kulpa R, Brault S, Presse D, Bideau B. Which technology to investigate visual perception in sport: Video vs Virtual Reality. Human Movement Science. 2015; 39 :12–26. doi: 10.1016/j.humov.2014.10.006. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Wang F. Computer distance virtual experiment teaching application based on virtual reality technology. International Journal of Emerging Technologies in Learning (IJET) 2018; 13 (04):83–94. doi: 10.3991/ijet.v13i04.8472. [ CrossRef ] [ Google Scholar ]
• Wang Y. Application of virtual reality technique in the construction of modular teaching resources. International Journal of Emerging Technologies in Learning (IJET) 2020; 15 (10):126–139. doi: 10.3991/ijet.v15i10.14129. [ CrossRef ] [ Google Scholar ]
• Wu WL, Hsu Y, Yang QF, Chen JJ. A spherical video-based immersive virtual reality learning system to support landscape architecture students’ learning performance during the COVID-19 era. Land. 2021; 10 (6):561. doi: 10.3390/land10060561. [ CrossRef ] [ Google Scholar ]
• Xie H, Zhang Y, Wu Z, Lv T. A bibliometric analysis on land degradation: Current status, development, and future directions. Land. 2020; 9 (1):28. doi: 10.3390/land9010028. [ CrossRef ] [ Google Scholar ]
• Xu X, Ke F. Designing a virtual-reality-based, gamelike math learning environment. American Journal of Distance Education. 2016; 30 (1):27–38. doi: 10.1080/08923647.2016.1119621. [ CrossRef ] [ Google Scholar ]
• Yang JC, Chen CH, Jeng MC. Integrating video-capture virtual reality technology into a physically interactive learning environment for English learning. Computers & Education. 2010; 55 (3):1346–1356. doi: 10.1016/j.compedu.2010.06.005. [ CrossRef ] [ Google Scholar ]
• Yerden, A., & Akkuş, N. (2020). Virtual reality remote access laboratory for teaching programmable logic controller topics.
• Yilmaz RM, Topu FB, Takkaç Tulgar A. An examination of the studies on foreign language teaching in pre-school education: A bibliometric mapping analysis. Computer Assisted Language Learning. 2019 doi: 10.1080/09588221.2019.1681465. [ CrossRef ] [ Google Scholar ]
• Zappatore, M., Longo, A., & Bochicchio, M. A. (2015). The bibliographic reference collection GRC2014 for the Online Laboratory Research community. In Proceedings of 2015 12th International Conference on Remote Engineering and Virtual Instrumentation (REV) (pp. 24–31). IEEE).
• Zhang M, Zhang Z, Chang Y, Aziz ES, Esche S, Chassapis C. Recent developments in game-based virtual reality educational laboratories using the microsoft kinect. International Journal of Emerging Technologies in Learning (IJET) 2018; 13 (1):138–159. doi: 10.3991/ijet.v13i01.7773. [ CrossRef ] [ Google Scholar ]
• Zhang X, Jiang S, Pablos P, Lytras MD, Sun Y. How virtual reality affects perceived learning effectiveness: A task– technology fit perspective. Behaviour y Information Technology. 2017; 36 (5):548–556. doi: 10.1080/0144929x.2016.1268647. [ CrossRef ] [ Google Scholar ]

Information

• Author Services

Initiatives

You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.

All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .

Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.

Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.

Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.

Original Submission Date Received: .

• Active Journals
• Find a Journal
• Proceedings Series
• For Authors
• For Reviewers
• For Editors
• For Librarians
• For Publishers
• For Societies
• For Conference Organizers
• Open Access Policy
• Institutional Open Access Program
• Special Issues Guidelines
• Editorial Process
• Research and Publication Ethics
• Article Processing Charges
• Testimonials
• Preprints.org
• SciProfiles
• Encyclopedia

Article Menu

• Subscribe SciFeed
• Recommended Articles
• Google Scholar
• on Google Scholar
• Table of Contents

Find support for a specific problem in the support section of our website.

Please let us know what you think of our products and services.

Visit our dedicated information section to learn more about MDPI.

JSmol Viewer

Virtual reality for training in assembly and disassembly tasks: a systematic literature review.

1. Background and Motivation

1.1. challenges in assembly and disassembly tasks, 1.2. training strategies, 1.3. virtual reality technologies for training, 1.4. main contribution and paper structure, 2. materials and methods, 2.1. systematic literature review.

• Main bibliometric characteristics of papers whereby the articles were classified according to information about the authors, the year of publication, and the type of document.
• Content analysis, as better described in the next section.

2.2. Content Analysis

• Type of task and research environment: The case studies were classified concerning the assembly or disassembly activities and the type of environment in which they were carried out (laboratory/company plant).
• Technology used: The different technologies for the development of VR used in the selected cases were identified and classified. This was essential to be able to identify the level of immersion reproduced in the case study. Level of immersion is the ability to abstract the user from the real world, using projection screens or helmets. The advancement of information technologies today allows us to navigate photo-realistic environments in real time, interacting with the objects present in them. When talking about VR, it is good to immediately make a distinction between non-immersive VR and immersive VR: in the former, the user simply finds himself in front of a monitor, which acts as a window onto the three-dimensional world with which to interact through special joysticks. The resulting effect is different from that obtained with an immersive VR, in which the effects that the user perceives are much more engaging and capable of making the user live in a completely new reality.
• Sample experience with technology and tasks: Pre-learning processes were identified for each case study, which aimed to increase participants’ confidence in both the technology used for the study and the task to be performed.
• Learning strategy: For each article, the devices used, the methods of comparison, the number of tasks performed, and the number of attempts allowed were defined.
• Operator performance: Results on objective metrics such as training time; task completion time; and number of errors and subjective ones, such as perceived workload, usability, and other qualitative evaluations, were analyzed.

3. SLR Results

3.1. selected papers, 3.2. content analysis, 3.2.1. type of case study, 3.2.2. level of immersion.

• High, characterized by the use of HMDs (Head Mounted Displays), wire gloves, as well as tracking devices for head and eye movements;
• Medium, characterized by the use of stereoscopic viewers and movement-tracking devices;
• Low, characterized by the use of a mouse, keyboard, 2D desktop PC, and headset.

3.2.3. Previous Experience and Pre-Learning

• Experience with VR technology , which could influence the time to complete the training because the operator appears to be already comfortable with the new technology;
• Experience with the task , which could result in effects on both the number of errors and task completion time, because experienced operators, while undergoing traditional training processes, could achieve better performance even without the aid of VR technologies.

3.2.4. Evaluation of VR Impact

• time required for training;
• completion time of the assigned task(s);
• number of errors.
• perceived workload;
• qualitative evaluation.
• physical tutors, i.e., people experienced in the specific task [ 25 , 54 , 59 ];
• paper-printed instructions, i.e., manuals with step-by-step instructions [ 20 , 37 , 38 , 39 , 41 , 56 , 62 , 64 ];
• video tutorials, depicting the steps needed to assemble/disassemble the object [ 41 , 43 , 47 , 56 , 59 , 61 ];
• audio recordings, describing the steps needed to assemble/disassemble the object [ 63 ];
• conventional guidance, i.e., on-the-job training [ 36 , 49 ].

3.3. Benefits and Limitations

• Reducing risks of accidents: VR makes it possible to simulate difficult working conditions, failures, or dangerous/emergency conditions that cannot be reproduced with real equipment, allowing the user to acquire skills and knowledge previously impossible. Through VR, every aspect of the components involved in complex operations can be explored, allowing an infinite number of repetitions. Furthermore, the use of VR can significantly reduce the risks associated with work-related injuries, quality defects, and financial losses [ 8 ]. VR-based systems may be more expensive, but they provide risk-free and injury-free environments for teaching and training [ 45 ].
• Improved learning and skill retention: Physical exploration of simulated space and time facilitates learning, knowledge, and memorization, while experimental practice helps to understand complex themes, concepts, and theories. According to Edgar Dale, in fact, a person remembers 10 per cent of what he or she learns when reading, 20 per cent when listening, and as much as 90 per cent when performing a determined action [ 65 ]. VR provides a platform for ‘learning by doing’, rather than learning by seeing, hearing, or observing, and this, therefore, explains how VR technologies allow to improve the effectiveness of training and extend knowledge retention time. VR supports active learning and the practice of repeated tasks because it offers the opportunity to examine all the details of the parts involved in a complex operation, and, above all, it offers the opportunity to perform the desired number of repetitions and applications without worrying about the damage that results from any mistakes as these have no consequences in real life. This can only have a positive effect on learning effectiveness, as also underlined in the case study [ 39 ].
• Increased staff motivation: As VR is still considered an innovative technology, immersive training is less frustrating and sometimes even more fun than a classic treatment. The possibility offered to the user to immerse and interact with a virtual world and the use of gaming techniques and features (such as the division into levels and the realization of intermediate drill games [ 39 , 44 , 46 , 48 , 56 ]) significantly improves the operator’s attention. Again, VR presents an intuitive approach because movement in a virtual environment resembles real-world actions. This makes it possible to enter the training scenario more quickly, keeping the user’s attention on the learning content. These aspects, therefore, increase the attention and motivation of the staff and improve the learning process which will be, as mentioned, effective and efficient.
• Enlarged availability: Because it has no physical limits, the virtual environment offers paradoxically unlimited spaces in which many individuals can participate in training activities simultaneously, thus stimulating collaboration and meeting the current needs of increasingly global organizations.
• Easier onboarding: The extended availability thus enables a mass training process at the same time, which in turn speeds up onboarding, i.e., the return on investment on new employees within a company. In addition, for companies that need to train employees who are geographically dispersed or who prefer to work remotely, this mode eliminates the need to organize in-person training.
• Reduced costs: The adoption of VR technologies for personnel training results in cost savings for the company due to several aspects. Firstly, the possibility of interacting with virtual equipment avoids the need to stop the production process to illustrate the structure and operation of the various machines and tools to workers, which represents a significant cost saving. Secondly, again thanks to virtual replicas, real parts are no longer required, avoiding the potential damage not only to expensive equipment due to the inexperience of the trainees, but also to the parts themselves.
• Tracking: It is essential for a quality VR experience to have stable and accurate tracking of the user’s body in virtual space. Tracking should be accurate to within 1 mm and without ‘jitter’. Common limitations include optical occlusions, limited space, and tracking accuracy.
• Simulation software: This introduces several sources of error in VR interactions, such as usability problems, rendering, scene lighting, and collision detection.
• Visualization: Devices such as HMDs have limitations in the field of view, low latency, and limited frame rate and resolution, which may affect the user experience.
• User factors: The user’s level of training and physiological limitations, such as tremor or vision problems, can affect the overall performance of the VR system.

4. Discussion and Conclusions

4.1. preliminary guidelines for defining vr training strategies, 4.2. research gaps and future research agenda, author contributions, data availability statement, conflicts of interest.

• El Maraghy, W.; El Mar, H. A new engineering design paradigm-The quadruple bottom line. Procedia CIRP 2014 , 21 , 18–26. [ Google Scholar ] [ CrossRef ]
• Falck, A.-C.; Örtengren, R.; Rosenqvist, M.; Söderberg, R. Proactive assessment of basic complexity in manual assembly: Development of a tool to predict and control operator-induced quality errors. Int. J. Prod. Res. 2017 , 55 , 4248–4260. [ Google Scholar ] [ CrossRef ]
• Oh, K.; Kim, D.; Hong, Y.S. Measurement of assembly system complexity based on the task differences induced from product variety. In Proceedings of the ASME Design Engineering Technical Conference, Boston, MA, USA, 2–5 August 2015; pp. 1–9. [ Google Scholar ] [ CrossRef ]
• Latos, B.A.; Kalantar, P.; Burgert, F.; Arend, M.; Nitsch, V.; Przybysz, P.M.; Mütze-Niewöhner, S. Development, implementation and evaluation of a complexity measure for the work of assembly teams in one-piece-flow assembly systems employing simulation studies. In Proceedings of the 32nd Annual European Simulation and Modelling Conference 2018, ESM 2018, Ghent, Belgium, 24–26 October 2018; pp. 88–94. [ Google Scholar ]
• Alkan, B.; Vera, D.; Ahmad, M.; Ahmad, B.; Harrison, R. A Model for Complexity Assessment in Manual Assembly Operations Through Predetermined Motion Time Systems. Procedia CIRP 2016 , 44 , 429–434. [ Google Scholar ] [ CrossRef ]
• BS 8887-220:2010 ; Specifies Requirements for the Process of Remanufacture. European Standards: Plzen, Czech Republic, 2010.
• Available online: https://ec.europa.eu/clima/eu-action/european-green-deal/european-climate-law_en (accessed on 1 March 2024).
• Caterino, M.; Fera, M.; Macchiaroli, R.; Pham, D.T. Cloud remanufacturing: Remanufacturing enhanced through cloud technologies. J. Manuf. Syst 2022 , 64 , 133–148. [ Google Scholar ] [ CrossRef ]
• Zhou, Z.; Liu, J.; Pham, D.T.; Xu, W.; Ramirez, F.J.; Ji, C.; Liu, Q. Disassembly sequence planning: Recent developments and future trends. Proc. Inst. Mech. Eng. Part B J. Eng. Manuf. 2019 , 233 , 1450–1471. [ Google Scholar ] [ CrossRef ]
• Wang, Y.; Lan, F.; Liu, J.; Huang, J.; Su, S.; Ji, C.; Pham, D.T.; Xu, W.; Liu, Q.; Zhou, Z. Interlocking problems in disassembly sequence planning. Int. J. Prod. Res. 2021 , 59 , 4723–4735. [ Google Scholar ] [ CrossRef ]
• Xu, C.; Wei, H.; Guo, X.; Liu, S.; Qi, L.; Zhao, Z. Human-Robot Collaboration Multi-objective Disassembly Line Balancing Subject to Task Failure via Multi-objective Artificial Bee Colony Algorithm. IFAC-PapersOnLine 2020 , 53 , 1–6. [ Google Scholar ] [ CrossRef ]
• Xu, W.; Tang, Q.; Liu, J.; Liu, Z.; Zhou, Z.; Pham, D.T. Disassembly sequence planning using discrete Bees algorithm for human-robot collaboration in remanufacturing. Robot. Comput. Manuf. 2020 , 62 , 101860. [ Google Scholar ] [ CrossRef ]
• Ramírez, F.J.; Aledo, J.A.; Gamez, J.A.; Pham, D.T. Economic modelling of robotic disassembly in end-of-life product recovery for remanufacturing. Comput. Ind. Eng 2020 , 142 , 106339. [ Google Scholar ] [ CrossRef ]
• Pimminger, S.; Kurschl, W.; Panholzer, L.; Schönböck, J. Exploring the Learnability of Assembly Tasks Using Digital Work Instructions in a Smart Factory. Procedia CIRP 2021 , 104 , 696–701. [ Google Scholar ] [ CrossRef ]
• Butzer, S.; Schötz, S.; Steinhilper, R. Remanufacturing Process Capability Maturity Model. Procedia Manuf. 2017 , 8 , 715–722. [ Google Scholar ] [ CrossRef ]
• Le, S.T.Q.; Huynh, V.N. An evaluation model for task complexity in production lines. Int. J. Product. Perform. Manag. 2023 , 72 , 3162–3183. [ Google Scholar ] [ CrossRef ]
• Falck, A.-C.; Tarrar, M.; Mattsson, S.; Andersson, L.; Rosenqvist, M.; Söderberg, R. Assessment of manual assembly complexity: A theoretical and empirical comparison of two methods. Int. J. Prod. Res 2017 , 55 , 7237–7250. [ Google Scholar ] [ CrossRef ]
• Gunasekara, H.; Gamage, J.; Punchihewa, H. Remanufacture for sustainability: Barriers and solutions to promote automotive remanufacturing. Procedia Manuf. 2020 , 43 , 606–613. [ Google Scholar ] [ CrossRef ]
• Maier, M.; Tropschuh, B.; Teubner, S.; Reinhart, G. Method for designing the training process in manual assembly. ZWF Z. Fuer Wirtsch. Fabr. 2020 , 115 , 682–686. [ Google Scholar ]
• Hoedt, S.; Claeys, A.; Van Landeghem, H.; Cottyn, J. The evaluation of an elementary virtual training system for manual assembly. Int. J. Prod. Res. 2017 , 55 , 7496–7508. [ Google Scholar ] [ CrossRef ]
• Cannas, V.G.; Pero, M.; Pozzi, R.; Rossi, T. Complexity reduction and kaizen events to balance manual assembly lines: An application in the field. Int. J. Prod. Res. 2018 , 56 , 3914–3931. [ Google Scholar ] [ CrossRef ]
• Duan, F.; Zhang, Z.; Gao, Q.; Arai, T. Verification of the effect of an assembly skill transfer method on cognition skills. IEEE Trans. Cogn. Dev. Syst. 2016 , 8 , 73–83. [ Google Scholar ] [ CrossRef ]
• Lim, J.T.A.; Hoffmann, E.R.; Hoffmann, E.R. Strategies in performing a manual assembly task. Int. J. Ind. Ergon. 2014 , 50 , 121–129. [ Google Scholar ] [ CrossRef ]
• Haber, J.; Xu, H.; Priya, K. Harnessing virtual reality for management training: A longitudinal study. Organ. Manag. J. 2023 , 20 , 93–106. [ Google Scholar ] [ CrossRef ]
• Wolfartsberger, J.; Zimmermann, R.; Obermeier, G.; Niedermayr, D. Analyzing the potential of virtual reality-supported training for industrial assembly tasks. Comput. Ind. 2023 , 147 , 103838. [ Google Scholar ] [ CrossRef ]
• Jenab, K.; Moslehpour, S.; Khoury, S. Virtual maintenance, reality, and systems: A review. Int. J. Electr. Comput. Eng. 2016 , 6 , 2698–2707. [ Google Scholar ] [ CrossRef ]
• Mak, S.L.; Tang, F.W.F.; Li, C.H.; Lee, G.T.W.; Chiu, W.H. A review on development and application of virtual reality (Vr) training platform for testing, inspection and certification industry. Int. J. Inf. Educ. Technol. 2020 , 10 , 926–931. [ Google Scholar ] [ CrossRef ]
• Akinola, Y.M.; Agbonifo, O.C.; Sarumi, O.A. Virtual Reality as a tool for learning: The past, present and the prospect. J. Appl. Learn. Teach. 2020 , 3 , 51–58. [ Google Scholar ] [ CrossRef ]
• Barkokebas, R.; Ritter, C.; Sirbu, V.; Li, X.; Al-Hussein, M. Application of virtual reality in task training in the construction manufacturing industry. In Proceedings of the 36th International Symposium on Automation and Robotics in Construction, ISARC 2019, Banff, AB, Canada, 21–24 May 2019; pp. 796–803. [ Google Scholar ] [ CrossRef ]
• Pallavicini, F.; Toniazzi, N.; Argenton, L.; Aceti, L.; Mantovani, F. Developing effective virtual reality training for military forces and emergency operators: From technology to human factors. In Proceedings of the 14th International Conference on Modeling and Applied Simulation, MAS, Bergeggi, Italy, 21–23 September 2015; pp. 206–210. [ Google Scholar ]
• Watson, W.L.; House, S.; Hart, R.; Abbas, J.; Asthana, S.; Gaudl, S.E. A Case Study Using Virtual Reality to Prime Knowledge for Procedural Medical Training ; Springer Nature: Cham, Switzerland, 2023; Volume 14145, pp. 189–208. [ Google Scholar ] [ CrossRef ]
• Ho, N.; Wong, P.-M.; Chua, M.; Chui, C.-K. Virtual reality training for assembly of hybrid medical devices. Multimed. Tools Appl. 2018 , 77 , 30651–30682. [ Google Scholar ] [ CrossRef ]
• Bayonne, E.; Marin-Garcia, J.A.; Alfalla-Luque, R. Partial least squares (PLS) in operations management research: Insights from a systematic literature review. J. Ind. Eng. Manag 2020 , 13 , 565–597. [ Google Scholar ] [ CrossRef ]
• Di Pasquale, V.; Miranda, S.; Neumann, W.P.; Setayesh, A. Human reliability in manual assembly systems: A Systematic Literature Review. IFAC-PapersOnLine 2018 , 51 , 675–680. [ Google Scholar ] [ CrossRef ]
• Boletsis, C.; Karahasanovic, A. Immersive technologies in retail: Practices of augmented and virtual reality. In Proceedings of the 4th International Conference on Computer-Human Interaction Research and Applications, INSTICC, Budapest, Hungary, 5–6 November 2020. [ Google Scholar ] [ CrossRef ]
• Wei, L. Development of Virtual disassembly and assembly platform for marine air compressor. J. Phys. Conf. Ser. 2021 , 1884 , 012011. [ Google Scholar ] [ CrossRef ]
• Sportillo, D.; Avveduto, G.; Tecchia, F.; Carrozzino, M. Training in VR: A preliminary study on learning assembly/disassembly sequences. In Proceedings of the Augmented and Virtual Reality: Second International Conference, AVR 2015, Lecce, Italy, 31 August–3 September 2015; Volume 9254, pp. 332–343. [ Google Scholar ] [ CrossRef ]
• Langley, A.; Lawson, G.; Hermawati, S.; D’Cruz, M.; Apold, J.; Arlt, F.; Mura, K. Establishing the Usability of a Virtual Training System for Assembly Operations within the Automotive Industry. Hum. Factors Ergon. Manuf. 2016 , 26 , 667–679. [ Google Scholar ] [ CrossRef ]
• Ulmer, J.; Braun, S.; Cheng, C.-T.; Dowey, S.; Wollert, J. Gamification of virtual reality assembly training: Effects of a combined point and level system on motivation and training results. Int. J. Hum.-Comput. Stud. 2021 , 165 , 102854. [ Google Scholar ] [ CrossRef ]
• Vélaz, Y.; Lozano-Rodero, A.; Suescun, A.; Gutiérrez, T. Natural and hybrid bimanual interaction for virtual assembly tasks. Virtual Real. 2014 , 18 , 161–171. [ Google Scholar ] [ CrossRef ]
• Horejsi, P.; Novikov, K.; Simon, M. A smart factory in a smart city: Virtual and augmented reality in a smart assembly line. IEEE Access 2020 , 8 , 94330–94340. [ Google Scholar ] [ CrossRef ]
• Gallegos-Nieto, E.; Medellín-Castillo, H.I.; González-Badillo, G.; Lim, T. Virtual training of assembly tasks using virtual reality techniques and haptic systems. In Proceedings of the ASME International Mechanical Engineering Congress and Exposition, Montreal, QC, Canada, 14–20 November 2014; Volume 46445, p. V02BT02A063. [ Google Scholar ] [ CrossRef ]
• Vélaz, Y.; Arce, J.R.; Gutiérrez, T.; Lozano-Rodero, A.; Suescun, A. The influence of interaction technology on the learning of assembly tasks using virtual reality. J. Comput. Inf. Sci. Eng. 2014 , 14 , 041007. [ Google Scholar ] [ CrossRef ]
• Garbaya, S.; Romano, D.M.; Hattar, G. Gamification of assembly planning in virtual environment. Assem. Autom. 2019 , 39 , 931–943. [ Google Scholar ] [ CrossRef ]
• Farr, A.; Pietschmann, L.; Zürcher, P.; Bohné, T. Skill retention after desktop and head-mounted-display virtual reality training. Exp. Results 2023 , 4 , e2. [ Google Scholar ] [ CrossRef ]
• Ordaz, N.; Romero, D.; Gorecky, D.; Siller, H.R. Serious Games and Virtual Simulator for Automotive Manufacturing Education & Training. Procedia Comput. Sci. 2015 , 75 , 267–274. [ Google Scholar ] [ CrossRef ]
• Shen, S.; Chen, H.-T.; Raffe, W.; Leong, T.W. Effects of Level of Immersion on Virtual Training Transfer of Bimanual Assembly Tasks. Front. Virtual Real. 2021 , 2 , 597487. [ Google Scholar ] [ CrossRef ]
• Palmas, F.; Labode, D.; Plecher, D.A.; Klinker, G. Comparison of a gamified and non-gamified virtual reality training assembly task. In Proceedings of the 2019 11th International Conference on Virtual Worlds and Games for Serious Applications (VS-Games), Vienna, Austria, 4–6 September 2019. [ Google Scholar ] [ CrossRef ]
• Kalkan, M.K.; Karabulut, E.; Höke, G. Effect of Virtual Reality-Based Training on Complex Industrial Assembly Task Performance. Arab. J. Sci. Eng. 2021 , 46 , 12697–12708. [ Google Scholar ] [ CrossRef ]
• Rodrigues, P.; Coelho, H.; Melo, M.; Bessa, M. Virtual Reality For Training: A Computer Assembly Application. In Proceedings of the 2022 International Conference on Graphics and Interaction (ICGI), Aveiro, Portugal, 3–4 November 2022; pp. 1–6. [ Google Scholar ] [ CrossRef ]
• Han, Y.; Chen, L.; Zhang, Z.; Liu, H. Usability study of auditory feedback and visual feedback in an immersive virtual assembly system. In Proceedings of the 2020 International Conference on Intelligent Computing, Automation and Systems (ICICAS), Chongqing, China, 11–13 December 2020; pp. 395–399. [ Google Scholar ] [ CrossRef ]
• Kloiber, S.; Settgast, V.; Schinko, C.; Weinzerl, M.; Schreck, T.; Preiner, R. A System for Collaborative Assembly Simulation and User Performance Analysis. In Proceedings of the 2021 International Conference on Cyberworlds (CW), Caen, France, 28–30 September 2021; pp. 93–100. [ Google Scholar ] [ CrossRef ]
• Niedermayr, D.; Wolfartsberger, J.; Maurer, M. Virtual Reality for Industrial Assembly Training: The Impact of Tool Interaction Realism on Learning Outcomes. In Proceedings of the 2023 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), Sydney, Australia, 16–20 October 2023; pp. 183–190. [ Google Scholar ] [ CrossRef ]
• Zigart, T.; Stürzl, F.; Niedermayr, D.; Wolfartsberger, J.; Sorko, S.R. Comparative Evaluation of Virtual Reality and In-Person Onboarding for Assembly Trainings in Manufacturing. In Proceedings of the 2023 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), Sydney, Australia, 16–20 October 2023; pp. 167–174. [ Google Scholar ] [ CrossRef ]
• De Moura, D.Y.; Sadagic, A. The effects of stereopsis and immersion on bimanual assembly tasks in a virtual reality system. In Proceedings of the 26th IEEE Conference on Virtual Reality and 3D User Interfaces (VR), Osaka, Japan, 23–27 March 2019; pp. 286–294. [ Google Scholar ] [ CrossRef ]
• Murcia-Lopez, M.; Steed, A. A comparison of virtual and physical training transfer of bimanual assembly tasks. IEEE Trans. Vis. Comput. Graph. 2018 , 24 , 1574–1583. [ Google Scholar ] [ CrossRef ]
• Dwivedi, P.; Cline, D.; Joe, C.; Etemadpour, R. Manual assembly training in virtual environments. In Proceedings of the 18th International Conference on Advanced Learning Technologies (ICALT), Mumbai, India, 9–13 July 2018; pp. 395–399. [ Google Scholar ] [ CrossRef ]
• Manuri, F.; Decataldo, F.; Sanna, A.; Brizzi, P. A Comparison of Two Interaction Paradigms for Training Low Cost Automation Assembly in Virtual Environments. Information 2023 , 14 , 340. [ Google Scholar ] [ CrossRef ]
• Koumaditis, K.; Chinello, F.; Mitkidis, P.; Karg, S.T. Effectiveness of Virtual vs. Physical Training: The Case of Assembly Tasks, Trainer’s Verbal Assistance, and Task Complexity. IEEE Comput. Graph. Appl. 2020 , 40 , 41–56. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Grajewski, D.; Hamrol, A. Low-cost VR system for interactive education of manual assembly procedure. Interact. Learn. Environ. 2023 , 31 , 68–86. [ Google Scholar ] [ CrossRef ]
• Gavish, N.; Gutiérrez, T.; Webel, S.; Rodríguez, J.; Peveri, M.; Bockholt, U.; Tecchia, F. Evaluating virtual reality and augmented reality training for industrial maintenance and assembly tasks. Interact. Learn. Environ. 2015 , 23 , 778–798. [ Google Scholar ] [ CrossRef ]
• Abidi, M.H.; Al-Ahmari, A.; Ahmad, A.; Ameen, W.; Alkhalefah, H. Assessment of virtual reality-based manufacturing assembly training system. Int. J. Adv. Manuf. Technol. 2019 , 105 , 3743–3759. [ Google Scholar ] [ CrossRef ]
• Vix Kemanji, K.; Mpwadina, R.; Meixner, G. Virtual Reality Assembly of Physical Parts: The Impact of Interaction Interface Techniques on Usability and Performance. In Proceedings of the Virtual, Augmented and Mixed Reality Applications in Education, Aviation and Industry—14th International Conference, VAMR 2022 Held as Part of the 24th HCI International Conference, HCII 2022, Virtual Event, 26 June–1 July 2022; Proceedings, Part II. Volume 13318. [ Google Scholar ] [ CrossRef ]
• Daling, L.M.; Tenbrock, M.; Isenhardt, I.; Schlittmeier, S.J. Assemble it like this!—Is AR- or VR-based training an effective alternative to video-based training in manual assembly? Appl. Ergon. 2023 , 110 , 104021. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Dwyer, F. Edgar Dale’s Cone of experience: A quasi-experimental analysis. Int. J. Instr. Media 2010 , 37 , 431–438. [ Google Scholar ]
• Numfu, M.; Riel, A.; Noel, F. Virtual reality based digital chain for maintenance training. Procedia CIRP 2019 , 84 , 1069–1074. [ Google Scholar ] [ CrossRef ]
• Kuipers, N.; Kolbeinsson, A.; Thorvald, P. Appropriate assembly instruction modes: Factors to consider. Adv. Transdiscipl. Eng. 2021 , 15 , 27–32. [ Google Scholar ] [ CrossRef ]
• Marraffino, M.D.; Johnson, C.I.; Garibaldi, A.E. Virtual Reality is Better Than Desktop for Training a Spatial Knowledge Task, but Not for Everyone. In Proceedings of the 14th International Conference, VAMR 2022, Held as Part of the 24th HCI International Conference, HCII 2022, Virtual Event, 26 June–1 July 2022; Proceedings, Part I. Volume 13317, pp. 212–223. [ Google Scholar ] [ CrossRef ]
• Li, C.; Yeom, S.; Dermoudy, J.; de Salas, K. Cognitive Load Measurement in the Impact of VR Intervention in Learning. In Proceedings of the 2022 International Conference on Advanced Learning Technologies (ICALT), Bucharest, Romania, 1–4 July 2022; pp. 325–329. [ Google Scholar ] [ CrossRef ]
• Young, K.; Dermoudy, J.; Lewis, I.; Young, K.; Dermoudy, J.; Lewis, I. A Digital Future in Virtual Reality—Insights for Training. 2020. Available online: https://aisel.aisnet.org/acis2020/26/ (accessed on 12 April 2023).
• Lee, C.-M.; Woo, W.-S.; Roh, Y.-H. Remanufacturing: Trends and issues. Int. J. Precis. Eng. Manuf. Technol. 2017 , 4 , 113–125. [ Google Scholar ] [ CrossRef ]
• Matsumoto, M.; Yang, S.; Martinsen, K.; Kainuma, Y. Trends and research challenges in remanufacturing. Int. J. Precis. Eng. Manuf. Technol. 2016 , 3 , 129–142. [ Google Scholar ] [ CrossRef ]
• Pollard, K.A.; Oiknine, A.H.; Files, B.T.; Sinatra, A.M.; Patton, D.; Ericson, M.; Thomas, J.; Khooshabeh, P. Level of immersion affects spatial learning in virtual environments: Results of a three-condition within-subjects study with long intersession intervals. Virtual Real. 2020 , 24 , 783–796. [ Google Scholar ] [ CrossRef ]
• de Back, T.T.; van Hoef, R.; Tinga, A.M.; Louwerse, M.M. Presence is Key: Unlocking Performance Benefits of Immersive Virtual Reality. In Proceedings of the 40th Annual Meeting of the Cognitive Science Society: Changing Minds, CogSci 2018, Madison, WI, USA, 25–28 July 2018; Volume 1, pp. 1602–1607. [ Google Scholar ]
• Martirosov, S.; Bureš, M.; Zítka, T. Cyber sickness in low-immersive, semi-immersive, and fully immersive virtual reality. Virtual Real. 2022 , 26 , 15–32. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Gutierrez-Maldonado, J.; Andres-Pueyo, A.; Jarne, A.; Talarn, A.; Ferrer, M.; Achotegui, J. Virtual reality for training diagnostic skills in anorexia nervosa: A usability assessment. In Virtual, Augmented and Mixed Reality: Proceedings of the 9th International Conference, VAMR 2017, Held as Part of HCI International 2017, Vancouver, BC, Canada, 9–14 July 2017 ; Springer International Publishing: Cham, Switzerland, 2017; Volume 10280, pp. 239–247. [ Google Scholar ] [ CrossRef ]

Click here to enlarge figure

Share and Cite

Di Pasquale, V.; Cutolo, P.; Esposito, C.; Franco, B.; Iannone, R.; Miranda, S. Virtual Reality for Training in Assembly and Disassembly Tasks: A Systematic Literature Review. Machines 2024 , 12 , 528. https://doi.org/10.3390/machines12080528

Di Pasquale V, Cutolo P, Esposito C, Franco B, Iannone R, Miranda S. Virtual Reality for Training in Assembly and Disassembly Tasks: A Systematic Literature Review. Machines . 2024; 12(8):528. https://doi.org/10.3390/machines12080528

Di Pasquale, Valentina, Paolo Cutolo, Carmen Esposito, Benedetta Franco, Raffaele Iannone, and Salvatore Miranda. 2024. "Virtual Reality for Training in Assembly and Disassembly Tasks: A Systematic Literature Review" Machines 12, no. 8: 528. https://doi.org/10.3390/machines12080528

Article Metrics

Article access statistics, further information, mdpi initiatives, follow mdpi.

Subscribe to receive issue release notifications and newsletters from MDPI journals

The Potential of Virtual Reality to Improve Diagnostic Assessment by Boosting Autism Spectrum Disorder Traits: A Systematic Review

• Published: 05 August 2024

Cite this article

• Mariangela Cerasuolo   ORCID: orcid.org/0000-0002-4178-4695 1 , 2 , 3 ,
• Stefania De Marco 1 , 4 ,
• Raffaele Nappo 1 , 4 ,
• Roberta Simeoli 1 , 5 &
• Angelo Rega 1 , 6  

Explore all metrics

While studies examining the effectiveness of virtual reality (VR) systems in autism spectrum disorder (ASD) intervention have seen significant growth, research on their application as tools to improve assessment and diagnosis remains limited. This systematic review explores the potential of VR systems in speeding-up and enhancing the assessment process for ASD.

We conducted a systematic search of peer-reviewed research to identify studies that compared characteristics of autistic and neurotypical participants performing tasks in virtual environments. Pubmed and IEE Xplore databases were searched and screened using predetermined keywords and inclusion criteria related to ASD and virtual reality, resulting in the inclusion of 20 studies.

Studies reviewed revealed that VR technologies may serve as a booster of ASD “traits” that might otherwise go unnoticed when using traditional tools. Specifically, results indicated that ASD individuals exhibited distinct behavioral nuances compared to typically developing participants across four main domains: communication and social interaction skills, cognitive functioning and neurological pattern, sensory and physiological processing, and motor behavior and body movements. Also, recent studies analyzed here underscored the potential of integrating machine learning with VR technologies to enhance accuracy in identifying ASD based on motor behavior, eye gaze, and electrodermal activity.

Conclusions

The integration of VR technologies can complement traditional tools in ASD diagnosis, providing more objective and reliable assessment within a controlled, ecological, and motivating virtual environment. In addition, the reviewed literature suggests machine learning models combined with VR technologies may support phenotypic diagnosis, offering a more refined classification of ASD subgroups within immersive virtual contexts.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save.

• Get 10 units per month
• Download Article/Chapter or eBook
• 1 Unit = 1 Article or 1 Chapter
• Cancel anytime

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Similar content being viewed by others

Virtual and Augmented Reality in Social Skills Interventions for Individuals with Autism Spectrum Disorder: A Scoping Review

Cognitive rehabilitation in people with autism spectrum disorder: a systematic review of emerging virtual reality-based approaches

The Application of Extended Reality in Treating Children with Autism Spectrum Disorder

Adams, L., Simonoff, E., Tierney, K., Hollocks, M. J., Brewster, A., Watson, J., & Valmaggia, L. (2022). Developing a user-informed intervention study of a virtual reality therapy for social anxiety in autistic adolescents. Design for Health, 6 (1), 114–133. https://doi.org/10.1080/24735132.2022.2062151

Article   Google Scholar  

Al-Beltagi, M. (2021). Autism medical comorbidities. World Journal of Clinical Pediatrics, 10 (3), 15–28. https://doi.org/10.5409/wjcp.v10.i3.15

Article   PubMed   PubMed Central   Google Scholar  

Alcañiz, M., Chicchi-Giglioli, I. A., Carrasco-Ribelles, L. A., Marín-Morales, J., Minissi, M. E., Teruel-García, G., Sirera, M., & Abad, L. (2022). Eye gaze as a biomarker in the recognition of autism spectrum disorder using virtual reality and machine learning: A proof of concept for diagnosis. Autism Research : Official Journal of the International Society for Autism Research, 15 (1), 131–145. https://doi.org/10.1002/aur.2636

Article   PubMed   Google Scholar  

Alcañiz Raya, M., Chicchi Giglioli, I. A., Marín-Morales, J., Higuera-Trujillo, J. L., Olmos, E., Minissi, M. E., Teruel Garcia, G., Sirera, M., & Abad, L. (2020a). Application of supervised machine learning for behavioral biomarkers of autism spectrum disorder based on electrodermal activity and virtual reality. Frontiers in Human Neuroscience, 14 , 90. https://doi.org/10.3389/fnhum.2020.00090

Alcañiz Raya, M., Marín-Morales, J., Minissi, M. E., Teruel Garcia, G., Abad, L., & Chicchi Giglioli, I. A. (2020b). Machine Learning and virtual reality on body movements’ behaviors to classify children with autism spectrum disorder. Journal of Clinical Medicine , 9 (5). https://doi.org/10.3390/jcm9051260

American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders: DSM-5. American Psychiatric Association.

Book   Google Scholar  

Anzulewicz, A., Sobota, K., & Delafield-Butt, J. T. (2016). Toward the autism motor signature: Gesture patterns during smart tablet gameplay identify children with autism. Scientific Reports, 6 (1), 1. https://doi.org/10.1038/srep31107

Arthur, T., Harris, D., Buckingham, G., Brosnan, M., Wilson, M., Williams, G., & Vine, S. (2021). An examination of active inference in autistic adults using immersive virtual reality. Scientific Reports, 11 (1), 20377. https://doi.org/10.1038/s41598-021-99864-y

Ashwood, K. L., Gillan, N., Horder, J., Hayward, H., Woodhouse, E., McEwen, F. S., Findon, J., Eklund, H., Spain, D., Wilson, C. E., Cadman, T., Young, S., Stoencheva, V., Murphy, C. M., Robertson, D., Charman, T., Bolton, P., Glaser, K., Asherson, P., … Murphy, D. G. (2016). Predicting the diagnosis of autism in adults using the autism-spectrum quotient (AQ) questionnaire. Psychological Medicine , 46 (12), 2595–2604. https://doi.org/10.1017/S0033291716001082

Baniasadi, T., Ayyoubzadeh, S. M., & Mohammadzadeh, N. (2020). Challenges and practical considerations in applying virtual reality in medical education and treatment. Oman Medical Journal, 35 (3), e125. https://doi.org/10.5001/omj.2020.43

Bekele, E., Crittendon, J., Zheng, Z., Swanson, A., Weitlauf, A., Warren, Z., & Sarkar, N. (2014). Assessing the utility of a virtual environment for enhancing facial affect recognition in adolescents with autism. Journal of Autism and Developmental Disorders, 44 (7), 1641–1650. https://doi.org/10.1007/s10803-014-2035-8

Bell, I. H., Nicholas, J., Alvarez-Jimenez, M., Thompson, A., & Valmaggia, L. (2020). Virtual reality as a clinical tool in mental health research and practice. Dialogues in Clinical Neuroscience, 22 (2), 169–177. https://doi.org/10.31887/DCNS.2020.22.2/lvalmaggia

van Bennekom, M. J., de Koning, P. P., & Denys, D. (2017). Virtual reality objectifies the diagnosis of psychiatric disorders: A literature review. Frontiers in Psychiatry , 8 . https://doi.org/10.3389/fpsyt.2017.00163

Biffi, E., Costantini, C., Ceccarelli, S. B., Cesareo, A., Marzocchi, G. M., Nobile, M., Molteni, M., & Crippa, A. (2018). Gait pattern and motor performance during discrete gait perturbation in children with autism spectrum disorders. Frontiers in Psychology, 9 , 2530. https://doi.org/10.3389/fpsyg.2018.02530

Brown, C., & Dunn, W. (2022). Adolescent-adult sensory profile: User’s manual . Psychological Corporation.

Google Scholar  

Bugnariu, N., de Weerd, C., Young, C., Longnecker, R., Garver, C., van Loon, E., Rockenbach, K., & Patterson, R. M. (2013). Motor function in children with autism spectrum disorders. International Conference on Virtual Rehabilitation (ICVR), 2013 , 51–56. https://doi.org/10.1109/ICVR.2013.6662080

Burdea, G. C., & Coiffet, P. (2003). Virtual reality technology . John Wiley & Sons.

Burke, S. L., Li, T., Grudzien, A., & Garcia, S. (2021). Brief report: Improving employment interview self-efficacy among adults with autism and other developmental disabilities using virtual interactive training agents (ViTA). Journal of Autism and Developmental Disorders, 51 (2), 741–748. https://doi.org/10.1007/s10803-020-04571-8

Cerasuolo, M., Simeoli, R., Nappo, R., Gallucci, M., Iovino, L., Frolli, A., & Rega, A. (2022). Examining predictors of different ABA treatments: A systematic review. Behavioral Sciences (basel, Switzerland), 12 (8), 267. https://doi.org/10.3390/bs12080267

Chen, Y., Fei, X., Wu, T., Li, H., Xiong, N., Shen, R., Wang, Y., Liang, A., & Wang, H. (2022). The relationship between motor development and social adaptability in autism spectrum disorder. Frontiers in Psychiatry, 13 , 1044848. https://doi.org/10.3389/fpsyt.2022.1044848

Cipresso, P., Giglioli, I. A. C., Raya, M. A., & Riva, G. (2018). The past, present, and future of virtual and augmented reality research: A network and cluster analysis of the literature. Frontiers in Psychology , 9 https://doi.org/10.3389/fpsyg.2018.02086

Dechsling, A., Shic, F., Zhang, D., Marschik, P. B., Esposito, G., Orm, S., Sütterlin, S., Kalandadze, T., Øien, R. A., & Nordahl-Hansen, A. (2021). Virtual reality and naturalistic developmental behavioral interventions for children with autism spectrum disorder. Research in Developmental Disabilities, 111 , 103885. https://doi.org/10.1016/j.ridd.2021.103885

Dechsling, A., Sütterlin, S., & Nordahl-Hansen, A. (2020). Acceptability and normative considerations in research on autism spectrum disorders and virtual reality. In D. D. Schmorrow & C. M. Fidopiastis (Eds.), Augmented cognition. Human cognition and behavior (pp. 161–170). Springer International Publishing. https://doi.org/10.1007/978-3-030-50439-7_11

Dudley, J., Yin, L., Garaj, V., & Kristensson, P. O. (2023). Inclusive immersion: A review of efforts to improve accessibility in virtual reality, augmented reality and the metaverse. Virtual Reality, 27 (4), 2989–3020. https://doi.org/10.1007/s10055-023-00850-8

Duffield, T. C., Parsons, T. D., Landry, A., Karam, S., Otero, T., Mastel, S., & Hall, T. A. (2018). Virtual environments as an assessment modality with pediatric ASD populations: A brief report. Child Neuropsychology: A Journal on Normal and Abnormal Development in Childhood and Adolescence, 24 (8), 1129–1136. https://doi.org/10.1080/09297049.2017.1375473

Emmelkamp, P. M. G., & Meyerbröker, K. (2021). Virtual reality therapy in mental health. Annual Review of Clinical Psychology, 17 , 495–519. https://doi.org/10.1146/annurev-clinpsy-081219-115923

Escobedo, L., Tentori, M., Quintana, E., Favela, J., & Garcia-Rosas, D. (2014). Using augmented reality to help children with autism stay focused. IEEE Pervasive Computing, 13 (1), 38–46. https://doi.org/10.1109/MPRV.2014.19

Fears, N. E., Templin, T. N., Sherrod, G. M., Bugnariu, N. L., Patterson, R. M., & Miller, H. L. (2023). Autistic children use less efficient goal-directed whole body movements compared to neurotypical development. Journal of Autism and Developmental Disorders, 53 (7), 2806–2817. https://doi.org/10.1007/s10803-022-05523-0

Feng, S., Wang, X., Wang, Q., Fang, J., Wu, Y., Yi, L., & Wei, K. (2018). The uncanny valley effect in typically developing children and its absence in children with autism spectrum disorders. PLoS ONE, 13 (11), e0206343. https://doi.org/10.1371/journal.pone.0206343

Forbes, P. A. G., Pan, X., & de C. Hamilton, A. F. (2016). Reduced Mimicry to virtual reality avatars in autism spectrum disorder. Journal of Autism and Developmental Disorders, 46 (12), 3788–3797. https://doi.org/10.1007/s10803-016-2930-2

Freeman, D., Reeve, S., Robinson, A., Ehlers, A., Clark, D., Spanlang, B., & Slater, M. (2017). Virtual reality in the assessment, understanding, and treatment of mental health disorders. Psychological Medicine, 47 (14), 2393–2400. https://doi.org/10.1017/S003329171700040X

Frolli, A., Savarese, G., Di Carmine, F., Bosco, A., Saviano, E., Rega, A., Carotenuto, M., & Ricci, M. C. (2022). Children on the autism spectrum and the use of virtual reality for supporting social skills. Children (basel, Switzerland), 9 (2), 181. https://doi.org/10.3390/children9020181

Greene, R. K., Vasile, I., Bradbury, K. R., Olsen, A., & Duvall, S. W. (2022). Autism Diagnostic Observation Schedule (ADOS-2) elevations in a clinical sample of children and adolescents who do not have autism: Phenotypic profiles of false positives. The Clinical Neuropsychologist, 36 (5), 943–959. https://doi.org/10.1080/13854046.2021.1942220

Hamad, A., & Jia, B. (2022). How virtual reality technology has changed our lives: An overview of the current and potential applications and limitations. International Journal of Environmental Research and Public Health, 19 (18), 11278. https://doi.org/10.3390/ijerph191811278

Hopkins, I. M., Gower, M. W., Perez, T. A., Smith, D. S., Amthor, F. R., Casey Wimsatt, F., & Biasini, F. J. (2011). Avatar assistant: Improving social skills in students with an ASD through a computer-based intervention. Journal of Autism and Developmental Disorders, 41 (11), 1543–1555. https://doi.org/10.1007/s10803-011-1179-z

Hossain, M. M., Khan, N., Sultana, A., Ma, P., McKyer, E. L. J., Ahmed, H. U., & Purohit, N. (2020). Prevalence of comorbid psychiatric disorders among people with autism spectrum disorder: An umbrella review of systematic reviews and meta-analyses. Psychiatry Research, 287 , 112922. https://doi.org/10.1016/j.psychres.2020.112922

Hours, C., Recasens, C., & Baleyte, J.-M. (2022). ASD and ADHD comorbidity: What are we talking about? Frontiers in Psychiatry, 13 , 837424. https://doi.org/10.3389/fpsyt.2022.837424

Junaidi, A. R., Alamsyah, Y., Hidayah, O., & Mulyawati, N. W. (2020). Development of virtual reality content to improve social skills in children with low function autism. 2020 6th International Conference on Education and Technology (ICET) , 115–119. https://doi.org/10.1109/ICET51153.2020.9276607

Junaidi, A. R., Dewantoro, D. A., Yuwono, J., Irvan, M., Alamsyah, Y., & Mulyawati, N. W. (2021). The acceptance level of low functioning autism while using virtual reality head-mounted display. 2021 7th International Conference on Education and Technology (ICET) , 199–203. https://doi.org/10.1109/ICET53279.2021.9575117

Karami, B., Koushki, R., Arabgol, F., Rahmani, M., & Vahabie, A.-H. (2021). Effectiveness of virtual/augmented reality-based therapeutic interventions on individuals with autism spectrum disorder: A comprehensive meta-analysis. Frontiers in Psychiatry, 12 , 665326. https://doi.org/10.3389/fpsyt.2021.665326

Ke, F., Moon, J., & Sokolikj, Z. (2022). Virtual reality–based social skills training for children with autism spectrum disorder. Journal of Special Education Technology, 37 (1), 49–62. https://doi.org/10.1177/0162643420945603

Kent, R., & Simonoff, E. (2017). Chapter 2—Prevalence of anxiety in autism spectrum disorders. In C. M. Kerns, P. Renno, E. A. Storch, P. C. Kendall, & J. J. Wood (Eds.), Anxiety in Children and Adolescents with Autism Spectrum Disorder (pp. 5–32). Academic Press. https://doi.org/10.1016/B978-0-12-805122-1.00002-8

Kim, K., Rosenthal, M. Z., Gwaltney, M., Jarrold, W., Hatt, N., McIntyre, N., Swain, L., Solomon, M., & Mundy, P. (2015). A virtual joy-stick study of emotional responses and social motivation in children with autism spectrum disorder. Journal of Autism and Developmental Disorders, 45 (12), 3891–3899. https://doi.org/10.1007/s10803-014-2036-7

Koegel, L. K., Singh, A. K., & Koegel, R. L. (2010). Improving motivation for academics in children with autism. Journal of Autism and Developmental Disorders, 40 (9), 1057–1066. https://doi.org/10.1007/s10803-010-0962-6

Koirala, A., Yu, Z., Schiltz, H., Van Hecke, A., Armstrong, B., & Zheng, Z. (2021). A preliminary exploration of virtual reality-based visual and touch sensory processing assessment for adolescents with autism spectrum disorder. IEEE Transactions on Neural Systems and Rehabilitation Engineering : A Publication of the IEEE Engineering in Medicine and Biology Society, 29 , 619–628. https://doi.org/10.1109/TNSRE.2021.3064148

Kourtesis, P., Kouklari, E.-C., Roussos, P., Mantas, V., Papanikolaou, K., Skaloumbakas, C., & Pehlivanidis, A. (2023). Virtual reality training of social skills in adults with autism spectrum disorder: An examination of acceptability, usability, user experience, social skills, and executive functions. Behavioral Sciences (basel, Switzerland), 13 (4), 336. https://doi.org/10.3390/bs13040336

Kumazaki, H., Yoshikawa, Y., Muramatsu, T., Haraguchi, H., Fujisato, H., Sakai, K., Matsumoto, Y., Ishiguro, H., Sumiyoshi, T., & Mimura, M. (2021). Group-based online job interview training program using virtual robot for individuals with autism spectrum disorders. Frontiers in Psychiatry, 12 , 704564. https://doi.org/10.3389/fpsyt.2021.704564

Lamash, L., Klinger, E., & Josman, N. (2017). Using a virtual supermarket to promote independent functioning among adolescents with Autism Spectrum Disorder. International Conference on Virtual Rehabilitation (ICVR), 2017 , 1–7. https://doi.org/10.1109/ICVR.2017.8007467

Lee, C. M., Altschuler, M. R., Esler, A. N., Burrows, C. A., & Hudock, R. L. (2023). Why are only some children with autism spectrum disorder misclassified by the social communication questionnaire? An empirical investigation of individual differences in sensitivity and specificity in a clinic-referred sample. Journal of Neurodevelopmental Disorders, 15 (1), 28. https://doi.org/10.1186/s11689-023-09497-7

Li, C., Belter, M., Liu, J., & Lukosch, H. (2023). Immersive virtual reality enabled interventions for autism spectrum disorder: A systematic review and meta-analysis. Electronics, 12 (11), 11. https://doi.org/10.3390/electronics12112497

Lin, Y., Gu, Y., Xu, Y., Hou, S., Ding, R., & Ni, S. (2022). Autistic spectrum traits detection and early screening A machine learning based eye movement study. Journal of Child and Adolescent Psychiatric Nursing : Official Publication of the Association of Child and Adolescent Psychiatric Nurses, Inc , 35 (1), 83–92 https://doi.org/10.1111/jcap.12346

Liu, X., Wu, Q., Zhao, W., & Luo, X. (2017). Technology-facilitated diagnosis and treatment of individuals with autism spectrum disorder: An engineering perspective. Applied Sciences, 7 (10), 10. https://doi.org/10.3390/app7101051

Lord, C., Rutter, M., & Le Couteur, A. (1994). Autism diagnostic interview-revised: A revised version of a diagnostic interview for caregivers of individuals with possible pervasive developmental disorders. Journal of Autism and Developmental Disorders, 24 (5), 659–685. https://doi.org/10.1007/BF02172145

Lord, C., Luyster, R. J., Gotham, K., & Guthrie, W. (2012). Autism diagnostic observation schedule, second edition (ADOS-2) manual (part II): Toddler module . Western Psychological Services.

Malihi, M., Nguyen, J., Cardy, R. E., Eldon, S., Petta, C., & Kushki, A. (2020). Data-driven discovery of predictors of virtual reality safety and sense of presence for children with autism spectrum disorder: A pilot study. Frontiers in Psychiatry , 11 https://doi.org/10.3389/fpsyt.2020.00669

Maskey, M., Lowry, J., Rodgers, J., McConachie, H., & Parr, J. R. (2014). Reducing specific phobia/fear in young people with autism spectrum disorders (ASDs) through a virtual reality environment intervention. PLoS ONE, 9 (7), e100374. https://doi.org/10.1371/journal.pone.0100374

Maskey, M., Rodgers, J., Ingham, B., Freeston, M., Evans, G., Labus, M., & Parr, J. R. (2019). Using virtual reality environments to augment cognitive behavioral therapy for fears and phobias in autistic adults. Autism in Adulthood, 1 (2), 134–145. https://doi.org/10.1089/aut.2018.0019

Mesa-Gresa, P., Gil-Gómez, H., Lozano-Quilis, J.-A., & Gil-Gómez, J.-A. (2018). Effectiveness of virtual reality for children and adolescents with autism spectrum disorder: An evidence-based systematic review. Sensors (basel, Switzerland), 18 (8), 2486. https://doi.org/10.3390/s18082486

Milano, N., Simeoli, R., Rega, A., & Marocco, D. (2023). A deep learning latent variable model to identify children with autism through motor abnormalities. Frontiers in Psychology , 14 . https://doi.org/10.3389/fpsyg.2023.1194760

Miller, H. L., & Bugnariu, N. L. (2016). Level of immersion in virtual environments impacts the ability to assess and teach social skills in autism spectrum disorder. Cyberpsychology, Behavior and Social Networking, 19 (4), 246–256. https://doi.org/10.1089/cyber.2014.0682

Miller, H. L., Templin, T. N., Fears, N. E., Sherrod, G. M., Patterson, R. M., & Bugnariu, N. L. (2023). Movement smoothness during dynamic postural control to a static target differs between autistic and neurotypical children. Gait & Posture, 99 , 76–82. https://doi.org/10.1016/j.gaitpost.2022.10.015

Minissi, M. E., Gómez-Zaragozá, L., Marín-Morales, J., Mantovani, F., Sirera, M., Abad, L., Cervera-Torres, S., Gómez-García, S., Chicchi Giglioli, I. A., & Alcañiz, M. (2023). The whole-body motor skills of children with autism spectrum disorder taking goal-directed actions in virtual reality. Frontiers in Psychology, 14 , 1140731. https://doi.org/10.3389/fpsyg.2023.1140731

Minissi, M. E., Altozano, A., Marín-Morales, J., Chicchi Giglioli, I. A., Mantovani, F., & Alcañiz, M. (2024). Biosignal comparison for autism assessment using machine learning models and virtual reality. Computers in Biology and Medicine, 171 , 108194. https://doi.org/10.1016/j.compbiomed.2024.108194

Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., PRISMA Group. (2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLoS Medicine, 6 (7), 1000097. https://doi.org/10.1371/journal.pmed.1000097

Mouga, S., Duarte, I. C., Café, C., Sousa, D., Duque, F., Oliveira, G., & Castelo-Branco, M. (2022). Parahippocampal deactivation and hyperactivation of central executive, saliency and social cognition networks in autism spectrum disorder. Journal of Neurodevelopmental Disorders, 14 (1), 9. https://doi.org/10.1186/s11689-022-09417-1

Newbutt, N., Glaser, N., Francois, M. S., Schmidt, M., & Cobb, S. (2023). How are autistic people involved in the design of extended reality technologies? A systematic literature review. Journal of Autism and Developmental Disorders . https://doi.org/10.1007/s10803-023-06130-3

Ouzzani, M., Hammady, H., Fedorowicz, Z., & Elmagarmid, A. (2016). Rayyan—A web and mobile app for systematic reviews. Systematic Reviews, 5 (1), 210. https://doi.org/10.1186/s13643-016-0384-4

Parsons, S. (2016). Authenticity in virtual reality for assessment and intervention in autism: A conceptual review. Educational Research Review, 19 , 138–157. https://doi.org/10.1016/j.edurev.2016.08.001

Parsons, T. D., & Carlew, A. R. (2016). Bimodal virtual reality stroop for assessing distractor inhibition in autism spectrum disorders. Journal of Autism and Developmental Disorders, 46 (4), 1255–1267. https://doi.org/10.1007/s10803-015-2663-7

Quintero, J., Baldiris, S., Rubira, R., Cerón, J., & Velez, G. (2019). Augmented reality in educational inclusion. A systematic review on the last decade. Frontiers in Psychology , 10 . https://doi.org/10.3389/fpsyg.2019.01835

Raajan, N. R., Shiva, G., Mithun, P., & Vijayabhaskar, P. V. M. (2014). A review on: Augmented reality technologies, systems and applications. Journal of Applied Sciences, 14 (14), 1485–1495. https://doi.org/10.3923/jas.2014.1485.1495

Robles, M., Namdarian, N., Otto, J., Wassiljew, E., Navab, N., Falter-Wagner, C. M., & Roth, D. (2022). A virtual reality based system for the screening and classification of autism. IEEE Transactions on Visualization and Computer Graphics, 28 (5), 2168–2178. https://doi.org/10.1109/TVCG.2022.3150489

Roth, D., Jording, M., Schmee, T., Kullmann, P., Navab, N., & Vogeley, K. (2020). Towards computer aided diagnosis of autism spectrum disorder using virtual environments. IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR), 2020 , 115–122. https://doi.org/10.1109/AIVR50618.2020.00029

Russo-Ponsaran, N., McKown, C., Johnson, J., Russo, J., Crossman, J., & Reife, I. (2018). Virtual environment for social information processing: Assessment of children with and without autism spectrum disorders. Autism Research : Official Journal of the International Society for Autism Research, 11 (2), 305–317. https://doi.org/10.1002/aur.1889

Satu, P., Minna, L., & Satu, S. (2023). Immersive VR assessment and intervention research of individuals with neurodevelopmental disorders is dominated by ASD and ADHD: A Scoping Review. Review Journal of Autism and Developmental Disorders . https://doi.org/10.1007/s40489-023-00377-3

Serret, S., Hun, S., Iakimova, G., Lozada, J., Anastassova, M., Santos, A., Vesperini, S., & Askenazy, F. (2014). Facing the challenge of teaching emotions to individuals with low- and high-functioning autism using a new Serious game: A pilot study. Molecular Autism, 5 , 37. https://doi.org/10.1186/2040-2392-5-37

Shahab, M., Taheri, A., Hosseini, S. R., Mokhtari, M., Meghdari, A., Alemi, M., Pouretemad, H., Shariati, A., & Pour, A. G. (2017). Social virtual reality robot (V2R): A Novel concept for education and rehabilitation of children with autism. 2017 5th RSI International Conference on Robotics and Mechatronics (ICRoM) , 82–87. https://doi.org/10.1109/ICRoM.2017.8466148

Silva, R. M., Carvalho, D., Martins, P., & Rocha, T. (2023). Systematic literature review of the use of virtual reality in the inclusion of children with autism spectrum disorders (ASD). Innovative Technologies and Learning: 6th International Conference, ICITL 2023 Porto, Portugal, August 28–30 Proceedings 2023 501 509 https://doi.org/10.1007/978-3-031-40113-8_49

Simeoli, R., Rega, A., Cerasuolo, M., Nappo, R., & Marocco, D. (2024). Using machine learning for motion analysis to early detect autism spectrum disorder: A systematic review. Review Journal of Autism and Developmental Disorders . https://doi.org/10.1007/s40489-024-00435-4

Simeoli, R., Arnucci, M., Rega, A., & Marocco, D. (2020). Movement detection software to enhance autism assessment processes. Proceedings of the 7th International Conference on Movement and Computing , 1–4. https://doi.org/10.1145/3401956.3404246

Slater, M., & Wilbur, S. (1997). A framework for immersive virtual environments (FIVE): Speculations on the role of presence in virtual environments. Presence Teleoperators and Virtual Environments, 6 (6), 603–616. https://doi.org/10.1162/pres.1997.6.6.603

Smith, M. J., Ginger, E. J., Wright, K., Wright, M. A., Taylor, J. L., Humm, L. B., Olsen, D. E., Bell, M. D., & Fleming, M. F. (2014). Virtual reality job interview training in adults with autism spectrum disorder. Journal of Autism and Developmental Disorders, 44 (10), 2450–2463. https://doi.org/10.1007/s10803-014-2113-y

Thabtah, F., & Peebles, D. (2019). Early autism screening: A comprehensive review. International Journal of Environmental Research and Public Health, 16 (18), 3502. https://doi.org/10.3390/ijerph16183502

Torres, E. B., Brincker, M., Isenhower, R. W., Yanovich, P., Stigler, K. A., Nurnberger, J. I., Metaxas, D. N., & José, J. V. (2013). Autism: The micro-movement perspective. Frontiers in Integrative Neuroscience, 7 , 32. https://doi.org/10.3389/fnint.2013.00032

Valencia, K., Rusu, C., Quiñones, D., & Jamet, E. (2019). The impact of technology on people with autism spectrum disorder: A systematic literature review. Sensors (basel, Switzerland), 19 (20), 4485. https://doi.org/10.3390/s19204485

Valori, I., McKenna-Plumley, P. E., Bayramova, R., & Farroni, T. (2021). Perception and motion in real and virtual environments: A narrative review of autism spectrum disorders. Frontiers in Psychology , 12 . https://doi.org/10.3389/fpsyg.2021.708229

Vismara, L. A., & Lyons, G. L. (2007). Using perseverative interests to elicit joint attention behaviors in young children with autism: Theoretical and clinical implications for understanding motivation. Journal of Positive Behavior Interventions, 9 (4), 214–228. https://doi.org/10.1177/10983007070090040401

Wade, J., Bian, D., Fan, J., Zhang, L., Swanson, A., Sarkar, M., Weitlauf, A., Warren, Z., & Sarkar, N. (2015). A virtual reality driving environment for training safe gaze patterns: Application in individuals with ASD. In M. Antona & C. Stephanidis (Eds.), Universal Access in Human-Computer Interaction. Access to Learning, Health and Well-Being (pp. 689–697). Springer International Publishing. https://doi.org/10.1007/978-3-319-20684-4_66

Wedyan, M., Aljumaily, A., & Dorgham, O. (2020). The use of augmented reality in the diagnosis and treatment of autistic children A review and a new system. Multimedia Tools and Applications, 79 (25), 18245–18291. https://doi.org/10.1007/s11042-020-08647-6

World Health Organization. (2019). International statistical classification of diseases and related health problems .

Yuan, S. N. V., & Ip, H. H. S. (2018). Using virtual reality to train emotional and social skills in children with autism spectrum disorder. London Journal of Primary Care, 10 (4), 110–112. https://doi.org/10.1080/17571472.2018.1483000

Zhang, L., Weitlauf, A. S., Amat, A. Z., Swanson, A., Warren, Z. E., & Sarkar, N. (2020). Assessing social communication and collaboration in autism spectrum disorder using intelligent collaborative virtual environments. Journal of Autism and Developmental Disorders, 50 (1), 199–211. https://doi.org/10.1007/s10803-019-04246-z

Zhang, M., Ding, H., Naumceska, M., & Zhang, Y. (2022). Virtual reality technology as an educational and intervention tool for children with autism spectrum disorder: Current Perspectives and Future Directions. Behavioral Sciences, 12 (5), 5. https://doi.org/10.3390/bs12050138

Zhao, Z., Tang, H., Zhang, X., Qu, X., Hu, X., & Lu, J. (2021). Classification of children with autism and typical development using eye-tracking data from face-to-face conversations: Machine learning model development and performance evaluation. Journal of Medical Internet Research, 23 (8), e29328. https://doi.org/10.2196/29328

Zhao, J., Zhang, X., Lu, Y., Wu, X., Zhou, F., Yang, S., Wang, L., Wu, X., & Fei, F. (2022). Virtual reality technology enhances the cognitive and social communication of children with autism spectrum disorder. Frontiers in Public Health, 10 , 1029392. https://doi.org/10.3389/fpubh.2022.1029392

Download references

Author information

Authors and affiliations.

Neapolisanit S.R.L. Rehabilitation Center, 80044, Ottaviano, Italy

Mariangela Cerasuolo, Stefania De Marco, Raffaele Nappo, Roberta Simeoli & Angelo Rega

Associazione Italiana Per L’Assistenza Spastici Onlus Sez Di Cicciano, 80033, Cicciano, Italy

Mariangela Cerasuolo

Faculty of Medicine, University of Ostrava, 70103, Ostrava, Czech Republic

Department of Psychology, University of Campania “Luigi Vanvitelli”, Caserta, Italy

Stefania De Marco & Raffaele Nappo

Department of Humanistic Studies, University of Naples Federico II, Naples, Italy

Roberta Simeoli

Department of Psychology and Educational Sciences, Pegaso Telematic University, Naples, Italy

Angelo Rega

You can also search for this author in PubMed   Google Scholar

Contributions

M. C. and A. R. contributed to the conceptualization of this manuscript. The first and last authors (M. C. and A. R.) also performed the literature search and data analysis. The first author (M. C.) prepared the first draft of the work. S. d. M., R. N., R. S., and A. R. contributed to methodology, editing, and revision of the work. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Mariangela Cerasuolo .

Ethics declarations

Ethics statement.

This research was approved by the Institutional Review Board at Neapolisanit s.r.l., Clinic Centre, Ottaviano, Italy.

Informed Consent

Not applicable.

Conflict of Interest

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Cerasuolo, M., De Marco, S., Nappo, R. et al. The Potential of Virtual Reality to Improve Diagnostic Assessment by Boosting Autism Spectrum Disorder Traits: A Systematic Review. Adv Neurodev Disord (2024). https://doi.org/10.1007/s41252-024-00413-1

Download citation

Accepted : 22 July 2024

Published : 05 August 2024

DOI : https://doi.org/10.1007/s41252-024-00413-1

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Virtual reality
• Autism spectrum disorder
• Machine learning
• Find a journal
• Publish with us
• Track your research

• Utility Menu

GA4 Tracking code

A Literature Review on Immersive Virtual Reality in Education: State Of The Art and Perspectives

A recent paper from Freina & Ott s urveys the literature on outcomes for using virtual reality in education, especially in higher education environments, finding that it “increases the learner’s involvement and motivation while

Into Practice Categories

• Into Practice Resources (256) Apply Into Practice Resources filter
• Into Practice Research (169) Apply Into Practice Research filter
• discussion (64) Apply discussion filter
• collaborative learning (54) Apply collaborative learning filter
• feedback (46) Apply feedback filter
• experiential learning (35) Apply experiential learning filter
• learning management system (33) Apply learning management system filter
• active learning (29) Apply active learning filter
• online learning (26) Apply online learning filter
• blended approaches (21) Apply blended approaches filter
• classroom contracts (20) Apply classroom contracts filter
• learning goals (19) Apply learning goals filter
• course transformation (18) Apply course transformation filter
• peer instruction (18) Apply peer instruction filter
• object learning (16) Apply object learning filter
• assessment (15) Apply assessment filter
• case-based learning (14) Apply case-based learning filter
• multimedia (14) Apply multimedia filter
• museums (14) Apply museums filter
• engaged scholarship (11) Apply engaged scholarship filter
• interdisciplinary (10) Apply interdisciplinary filter
• group work (9) Apply group work filter
• guest speakers (9) Apply guest speakers filter
• learning by making (9) Apply learning by making filter
• backward design (8) Apply backward design filter
• libraries (8) Apply libraries filter
• research assignments (8) Apply research assignments filter
• storytelling (8) Apply storytelling filter
• syllabus design (8) Apply syllabus design filter
• community events (7) Apply community events filter
• devices (7) Apply devices filter
• classrooms and space (5) Apply classrooms and space filter
• lecture (5) Apply lecture filter
• teaching empathy (4) Apply teaching empathy filter
• literature-based learning (3) Apply literature-based learning filter
• teaching fellows (3) Apply teaching fellows filter
• data (2) Apply data filter
• leadership (1) Apply leadership filter

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here .

Loading metrics

Open Access

Peer-reviewed

Research Article

Exploring user acceptance of online virtual reality exhibition technologies: A case study of Liangzhu Museum

Roles Conceptualization, Data curation, Methodology, Software, Visualization, Writing – original draft

Affiliation Department of Environment Design, College of Design, Jiaxing University, Jiaxing, Zhejiang, China

Roles Data curation, Formal analysis, Funding acquisition, Investigation, Project administration, Resources, Supervision, Validation, Writing – original draft, Writing – review & editing

* E-mail: [email protected]

Affiliation Department of Animation and Digital Media, College of Arts, Wuyi University, Wuyishan, Fujian, China

• Jia Li, 

• Published: August 1, 2024
• https://doi.org/10.1371/journal.pone.0308267
• Peer Review
• Reader Comments

Museums increasingly rely on cutting-edge digital technologies to attract visitors. Understanding the intricate factors influencing user acceptance of these technologies is, however, crucial for their effective use. This study therefore proposes a model, grounded in the technology acceptance model, to investigate user acceptance of online virtual reality (VR) museum exhibitions. Leveraging the online VR exhibition at Liangzhu Museum as a case study, data were collected from 313 participants and analyzed using partial least squares structural equation modeling (PLS-SEM) with Smart PLS. Semi-structured interviews with 15 individuals were conducted to complement the quantitative findings. The results reveal that factors such as interactivity, immersion, and presence positively influenced users’ intrinsic technological beliefs (perceived ease of use, perceived enjoyment, and perceived usefulness), ultimately affecting their willingness to use and intention to visit on-site. Notably, immersion had a direct positive effect on perceived usefulness. There is a pressing need to leverage digital and web technologies to cater to the increasingly complex and diverse needs of online visitors, and emphasizing navigational performance in online VR exhibitions is also paramount for enhancing the overall user experience.

Citation: Li J, Lv C (2024) Exploring user acceptance of online virtual reality exhibition technologies: A case study of Liangzhu Museum. PLoS ONE 19(8): e0308267. https://doi.org/10.1371/journal.pone.0308267

Editor: Anis Eliyana, Universitas Airlangga, INDONESIA

Received: May 8, 2024; Accepted: July 15, 2024; Published: August 1, 2024

Copyright: © 2024 Li, Lv. This is an open access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: The data are available from Figshare (doi: https://doi.org/10.6084/m9.figshare.26181599.v3 ), and all relevant data are within the manuscript and its Supporting Information files.

Funding: This study was funded by Fujian Province Social Science Foundation Project approval number FJ2021C106. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Competing interests: The authors have declared that no competing interests exist.

Introduction

The development of the internet and digital technologies has revolutionized people’s lifestyles, which has prompted museums to reassess their relationship with their audience [ 1 ]. With the widespread adoption of consumer-grade digital technologies and personal smart devices, individuals are increasingly reliant on mediums such as smartphones and computers to access information [ 2 , 3 ]. Considering this trend, the traditional reliance of museums on physical visits is being called into question [ 4 ]. To tackle this challenge, many museums have proactively embraced the latest online digital technologies and collaborated with experts across various fields to develop diverse digital resources, including virtual reality (VR) [ 5 ]. These innovative digital technologies and virtual environments have emerged as crucial tools for enhancing museums’ competitiveness and attracting new visitors [ 6 ]. By effectively harnessing online digital resources such as social media and virtual exhibitions, the dynamics of the relationship between museums and individuals, as well as between museums and society, are undergoing significant transformations [ 7 ]. However, the integration of digital technology into museums elicits a multitude of complex factors that can influence their success while posing challenges for assessing associated risks and leading to indecision, particularly among resource-limited museums [ 8 ].

The recent COVID-19 pandemic accelerated digital transformation in the museum sector while also exposing deficiencies in digital resources. The pandemic resulted in a reduction of physical visitors to museums and the closure of many museums, which prompted these institutions to expedite the development of online digital resources [ 9 – 11 ]. However, digital resources in museums still have limitations. One significant reason is the lack of close connection between museums’ digital resources and their users [ 12 ]. The instability and complexity of digital technology can often lead many museums to outsource the development of digital resources to technical personnel and experts, which can result in a disconnect between the new technologies in the museums and the actual perceptions and needs of users [ 13 ]. Due to a lack of understanding among users of digital resources, many museums have found it challenging to determine which digitization methods are most effective [ 9 ]. Consequently, compared to the continuous development and integration of digital technologies in museums, in-depth user research and feedback mechanisms are still lacking, which may affect the effectiveness of the systems employed and the user experience [ 14 , 15 ].

Within the context outlined above, the present study poses the following research question:

RQ: What are the primary factors affecting users’ acceptance of online virtual exhibition technology in museums?

This study sought to achieve the following main objectives to promote the sustainable development of museum digital resources:

• Based on the technology acceptance model (TAM), we established a structural equation model for visitors’ acceptance of museum online virtual reality exhibitions and propose research hypotheses.
• Taking the online VR exhibition adopted by the Liangzhu Museum as an example, we analyzed the factors influencing visitors’ acceptance of this technology using structural equation modeling (SEM) and evaluated the relationships between them.
• We then supplemented and extended the quantitative research by conducting interview studies.

The rest of this article consists of six major parts. The first part, the literature review, compiles the background information on current research in relevant fields through a review of literature related to the research topic. The second part presents the research model, the research hypotheses, and the main variables in the research model. The third part presents the methodology, primarily instrument development, research materials, data collection, and analysis tools. The fourth part covers the research findings, analyzes the collected data, and validates the effectiveness of the research model. The fifth part discusses quantitative and qualitative research results. The sixth part provides the research conclusion and study limitations.

Literature review

Vr exhibitions in museums.

VR, also known as a virtual environment, originated in the United States in the 1960s. In the decades since its inception, VR has played a significant role in enhancing educational and entertainment experiences [ 16 ]. Conceptually, VR traces its roots back to the popularity of panoramas in Europe during the 19th century, according to Nedelcu [ 17 ]. Notably, Byerly [ 18 ] suggested that panoramas emerged from the Victorian fascination with tourism, offering an experience of “being in two places at once” while emphasizing a perceptual shift from knowingly experiencing an illusion to feeling it as reality. Thus, in some respects, VR exhibitions in museums can be considered the panoramas of the digital age [ 19 ].

The introduction of VR (in the modern sense) into museums began in the mid-1990s [ 20 ]. As the cost of 3D mapping software decreased, the barrier to creating virtual environments fell significantly, which facilitated the integration of VR technology into the cultural heritage sector [ 21 ]. Museum VR exhibitions have appeared in various forms, ranging from large-scale cave automatic virtual environments (CAVEs) to simple multimedia displays and applications [ 20 ]. The significance of VR exhibitions for museums lies in overcoming temporal and spatial constraints on visitors [ 20 ]. These constraints include limitations imposed by the museum’s physical space, difficulties in reproducing vanished or inaccessible heritage, and restrictions on interacting with fragile and endangered exhibits[ 22 , 23 ].

In recent years, VR based on panoramic photography has garnered attention from researchers and developers of museum VR exhibitions due to its simplicity and effectiveness [ 24 ]. Montagud and Orero [ 25 ] argued that, although panoramic-based VR cannot offer the same level of 3D interaction as fully virtual environments, it can directly capture real-world scenes to provide a strong sense of immersion.

Museum VR exhibition user study

Early research has primarily focused on the general issues and methods of integrating VR into museum contexts, but systematic user studies are lacking. For instance, Lepouras and Charitos [ 26 ] identified visitor demands for VR exhibitions based on the quality of visitor experiences (e.g., immersion, display modes, resolution) and provided essential insights for related design and development. In a study of an immersive VR exhibition system for cultural heritage, Roussou [ 27 ] suggested that VR has the potential to offer museum visitors high-quality visual aesthetics and interactive experiences for information cognition, but the high application and maintenance costs cannot be ignored. Lepouras and Vassilakis [ 28 ] conducted a small-scale informal user evaluation study to test a prototype desktop-level VR exhibition system using low-cost 3D gaming technology. Carrozzino and Bergamasco [ 29 ] argued that VR stimulates visitors’ senses through images, sounds, and other information in museums while allowing nonprofessional users to obtain information effectively from museum exhibitions.

With the continuous development of VR technology, its potential application in museums has been continuously explored, and user research has become more systematic in recent years. Izzo [ 30 ] highlighted the advantages of VR in terms of information richness and the customization of experience based on information and communication technology attributes. Su and Teng [ 31 ] conducted a user study on cross-object user interfaces in museum VR exhibition environments. One significance of this study is to demonstrate that user research findings for museum VR exhibitions may differ from the expectations of designers and developers, thus highlighting the necessity of user research. Robbani and Rosmansyah [ 32 ] argued that online VR based on panoramic photography, in conditions where physical visitation is not possible, can still enhance museum visitor experiences in terms of education, entertainment, immersion, overall experience, and visit intensity.

Museum VR exhibition TAM study

Earlier studies have employed TAM to assess user acceptance of VR technology in museums [ 33 ]. Huang and Backman [ 34 ] further expanded the TAM by adding three exogenous latent variables: computer self-efficacy, personal innovativeness, and media richness. They demonstrated the potential of digital virtual museum technology for enhancing experiences and learning. Although they repeatedly emphasized that VR technology is an integral part of virtual museums, they did not explicitly specify the type of technology they used for their case study.

Recent studies applying TAM to museum VR exhibition technology have further expanded the boundaries of research in this field and made it more adaptable to different museum research contexts. Hammady, Ma, and Strathearn [ 35 ] measured the acceptance of mixed reality (MR) technology for visualizing historical information in museums using TAM. However, the limited sample size and informal prototypes they used could restrict the validity of their findings. Wu et al. [ 36 ] used an extended TAM to investigate the factors influencing users’ adoption of online digital virtual exhibitions in costume museums; however, their study sampled scholars and students from the fashion design field, who might not be representative of the general public’s perspective. Iftikhar, Khan, and Pasanchay [ 37 ] assessed the acceptance of VR technology in tourism-related activities by people with disabilities based on the TAM theory. Wen, Sotiriadis, and Shen [ 38 ] identified key factors for visitors’ acceptance and adoption of on-site VR technology in cultural heritage museums. However, their study, along with the aforementioned studies, lacks qualitative research to supplement the quantitative findings, which might lead to the omission of some key factors and details. As museums are a type of tourism resource, their research results have limited reference value. Therefore, given the increasing number of museums offering online VR exhibitions, research on user acceptance of this technology is necessary [ 39 ]. In summary, there is still a lack of mixed-methods research combining quantitative and qualitative approaches based on TAM theory, and specifically on the acceptance of VR exhibition technology in online museums.

Research model

TAM is a theoretical framework developed by Davis and Bagozzi [ 40 ] that has been widely employed in the study of user acceptance of new technologies. It is based on the theory of reasoned action (TRA), and the classic TAM comprises three main components: external variables, internal beliefs, and behavioral intention [ 41 , 42 ] ( Fig 1 ). Exogenous latent variables are associated with different users, technologies, and tasks. Technological beliefs include perceived ease of use and perceived usefulness, while user behavioral intention encompasses attitudes toward use, behavioral intention to use, and actual system use. Model analysis based on the TAM theoretical structure makes it possible to test the rationality of TAM variables and explore the causal relationships between variables, ultimately aiding in the explanation and prediction of user acceptance of new technological systems. TAM has more recently been widely applied in the study of various new technologies, including smart educational technology, the metaverse, and artificial intelligence, demonstrating its flexibility and effectiveness [ 43 – 48 ].

• PPT PowerPoint slide
• PNG larger image
• TIFF original image

https://doi.org/10.1371/journal.pone.0308267.g001

External variables

Many studies have augmented external variables to adapt to different research contexts [ 49 ]. These variables are typically associated with specific tasks, types of technology, and user groups [ 50 ]. Drawing upon relevant theoretical literature, this study proposes three external variables: interactivity, immersion, and presence.

Interactivity is a technological characteristic of VR. It is defined as the ability for users to engage actively with the content of the VR system [ 51 ]. Human interaction with information technology includes both interactions with people and with information [ 52 ]. In the context of this study, interactivity primarily refers to interaction with information, which can lead to positive responses toward information technology and its beliefs [ 52 ]. VR interactivity has been conceptualized as a characteristic that allows individuals to exercise control and engage in synchronous communication with the system [ 53 ]. A study on the impact of vividness and interactivity on VR experiences confirmed the importance of interactivity in VR experiences [ 51 ]. This study suggests that enhancing VR interactivity can enhance users’ experience of media richness and further influence their future usage behavior. High levels of interactivity can, it has been suggested [ 54 ], enhance the sense of control over VR and lead to greater perceived enjoyment. The results of regression analyses indicate that high interactivity has a significantly positive impact on information acquisition and enjoyment of VR interfaces. Based on the above theories and experiences, this study considers interactivity as an important external variable for studying the acceptance of VR exhibition technology in museums. We therefore formulated the following hypotheses:

• H1 : Interactivity has a positive impact on perceived ease of use.
• H2 : Interactivity has a positive impact on perceived enjoyment.
• H3 : Interactivity has a positive impact on perceived usefulness.

Immersion can be understood as the user’s lack of awareness of time and the real world while using a system, as well as the sense of involvement in and environmental feeling during tasks [ 55 ]. Jennett and Cox [ 55 ] emphasized the difference between immersion and presence, noting that an immersive experience is an experience of time, while presence is a state of mind. Bafadhal and Hendrawan [ 56 ] defined the VR immersive experience as a user’s state of sensation and interaction in a virtual environment that is related to continuous sensory stimulation by VR system sensors; the higher the immersion, the more positive the user’s attitude toward VR. Argyriou and Economou [ 57 ] suggested that immersion in VR based on panoramic photography involves the realism of the virtual environment, the degree of disconnection from the real world, and the perception of time. Furthermore, Vishwakarma and Mukherjee [ 58 ] argued that higher immersion can promote users’ willingness to use VR systems and further enhance their perceived usefulness by increasing the enjoyment of use. Based on the above theories and experiences, this study considers immersion as an important external variable affecting users’ VR experiences, and formulated the following research hypotheses:

• H4 : Immersion has a positive impact on perceived ease of use.
• H5 : Immersion has a positive impact on perceived enjoyment.
• H6 : Immersion has a positive impact on perceived usefulness.

Presence can be understood as the feeling of “being there” in a virtual environment, and researchers have suggested that VR presence depends on the degree of isolation from real space in the virtual environment [ 59 , 60 ]. Schubert [ 61 ] described presence in virtual environments as an unconscious feedback process of spatial perception, while Wu and Lai [ 62 ] also emphasized the spatial aspect of presence. This study posits that presence is a core concept based on panoramic photography VR systems that can replace or simulate a certain real-life environment; it is closely related to spatial intention. Research has indicated that a strong sense of presence can enhance users’ emotional involvement in VR, predict their willingness to use the system, and reduce resistance to difficulties [ 63 ]. Tsai [ 60 ] pointed out that enhancing presence is an important method for improving visitors’ cognition, emotions, and conceptual imagery related to the scene. In the context of on-site visits, studies in the field of online VR destination marketing have identified the influential role of presence in enhancing individuals’ awareness of destination attributes [ 64 , 65 ]. These studies affirm that VR presence can create positive user intentions toward destinations and promote their real-life visits. Based on the above theories and experiences, this study considers presence as an important external variable influencing visitors’ acceptance of museum VR technology. The following research hypotheses were therefore formulated:

• H7 : Presence has a positive impact on perceived ease of use.
• H8 : Presence has a positive impact on perceived enjoyment.
• H9 : Presence has a positive impact on perceived usefulness.

Internal belief variables

Perceived enjoyment is an internal belief variable developed by Davis and Bagozzi [ 66 ] based on the original TAM framework. Contrasting with perceived usefulness, which examines users’ extrinsic motivation for using a new system, perceived enjoyment assesses users’ internal motivation [ 66 ]. Extrinsic motivation refers to activities that contribute to increasing or adding additional value beyond the use of the system itself, such as acquiring information, enhancing skills, or increasing income. In contrast, perceived enjoyment refers to the degree of pleasure and joy experienced during the use of a new system, without any other foreseeable additional consequences. Davis and Bagozzi [ 66 ] argued that many new systems are rejected or considered unsatisfactory not because they fail to improve work performance or are difficult to use, but because they overlook the inherent enjoyment of system use. Explaining users’ willingness and behavior to use new technology systems by perceived enjoyment is thus different from perceived ease of use and perceived usefulness.

Perceived enjoyment is one of the most widely applied variables in studies exploring the acceptance of new technology based on TAM [ 43 , 48 ]. Some studies have examined the relationship between visitors’ perceived enjoyment, perceived ease of use, perceived usefulness, and intention to use digital exhibition technologies in museums [ 35 , 67 ]; other studies have further compared their differential impacts on intention to use [ 44 , 48 , 68 ]. This demonstrates the high potential of digital virtual technologies in creating a pleasant and enjoyable museum experience [ 69 ]. Immersive and interactive digital virtual technologies generate more positive emotions and stimulate behavior [ 36 ]. However, the impact of perceived enjoyment has been controversial [ 33 ], so further exploration is needed.

Based on the above theories and experiences, the following hypotheses were developed:

• H10 : Perceived enjoyment has a positive impact on perceived ease of use.
• H11 : Perceived enjoyment has a positive impact on perceived usefulness.
• H12 : Perceived enjoyment has a positive impact on intention to use.

According to TAM, an individual’s acceptance of new technology systems depends on the perceived usefulness and ease of use of those systems [ 70 ]. TAM assumes that users’ acceptance of new technology systems is determined by their intention to use, which is influenced by their attitude toward use. This attitude is shaped by the basic technological beliefs of perceived ease of use and perceived usefulness [ 71 , 72 ]. Perceived usefulness assesses the extent to which individuals subjectively believe that using a new technology system will benefit performance. Perceived ease of use assesses the effort individuals subjectively believe is required to achieve a goal using the new technology system [ 41 ]. According to Davis, perceived usefulness typically involves three items: job effectiveness, productivity and time savings, and importance. In contrast, perceived ease of use typically involves three items: physical effort, mental effort, and ease of learning. Many studies have, however, adjusted the scales for these concepts based on specific situations. For example, in a TAM study on cultural heritage VR conducted by Jung and Nguyen [ 73 ], perceived usefulness was emphasized in relation to the acquisition of information about cultural heritage sites. Based on the above theories and experiences, the following hypotheses were developed:

• H13 : Perceived ease of use has a positive impact on intention to use.
• H14 : Perceived ease of use has a positive impact on perceived usefulness.
• H15 : Perceived usefulness has a positive impact on intention to use.

Behavior intention variables

This study draws on the research of Jung and Lee [ 74 ] and Hammady and Ma [ 35 ] to revise and integrate the impact of technological beliefs on users’ attitudes, intentions, and behaviors into the intention to use, thereby reducing the behavioral intention construct in the research model. Promoting on-site visits is one of the main impacts of online VR technology [ 75 ]. Because online virtual experiences can only provide a small part of the on-site visit experience [ 76 ], virtual museums can be seen as a means to attract visitors to the physical museum site [ 77 ]. This point is of higher value in efforts to revitalize social vitality in the post-pandemic era [ 76 ]. This study therefore incorporated the tendency to visit actual sites into the behavioral intention construct and developed the following hypothesis:

• H16: Intention of use has a positive impact on the tendency to visit actual sites.

The proposed model in this study, based on these hypotheses, is illustrated in the Fig 2 .

https://doi.org/10.1371/journal.pone.0308267.g002

Methodology

Instrument development.

This study designed a research questionnaire based on the model described above and combined with results of previous research. The questionnaire consisted of three parts: a demographic questionnaire, scale items, and an interview. Demographic variables included age, gender, and education level. The items in the scale questionnaire were all derived from published academic literature and modified according to the background of this study ( Table 1 ). Items in the scale questionnaire were rated on a 5-point Likert scale, with response options ranging from strongly disagree to strongly agree . The interview portion was semi-structured, with questions designed based on the research model. Both the scale and interview questionnaire items were reviewed by three experts in the relevant field, and adjustments were made based on their feedback. The identities of the three experts are shown in the Table 2 .

https://doi.org/10.1371/journal.pone.0308267.t001

https://doi.org/10.1371/journal.pone.0308267.t002

Research materials

The online VR exhibition of the Liangzhu Museum was used as a case study, and its users were the research subjects. Located in Hangzhou, Zhejiang Province, China, Liangzhu Museum is a cultural heritage museum showcasing local Liangzhu cultural heritage in the form of archeological finds. Its main exhibits include various artifacts such as jade, pottery, stone tools, and lacquerware, as well as ancient human remains, plant and animal fossils, architectural and environmental models, and images of reconstructions.

The VR exhibition technology adopted by Liangzhu Museum uses an online VR exhibition system based on 360° panoramic photography. The permanent exhibitions of the museum were captured and produced through 360° photography, which allowed a simulated on-site visit experience in VR. Visitors can use their smartphones or computers to access the VR exhibition via the museum’s official website ( https://www.lzmuseum.cn/vr/index.html ). This VR system possesses the following functionalities:

• Simulated on-site visit: Users can switch between exhibition theme areas on their smartphone or computer screen to visit the exhibition. They can interact with the exhibit by, for example, zooming in or rotating the perspective. There are 63 theme areas available for exploration.
• Audio guide: Automatic voiceovers play in certain exhibition theme areas, which provide an overall introduction to different exhibition themes. There are 19 audio guide points.
• Video playback: Videos are played on virtual screens within the VR environment. These provide further information about specific theme areas. There is one video playback point.

The visual content of the VR exhibition consists of the permanent exhibitions of Liangzhu Museum, with different exhibition halls and theme areas represented by thumbnail icons at the bottom of the interface. Visitors can follow the on-site visit sequence by clicking on the guiding arrows in the picture, or they can switch to different theme areas by clicking on the thumbnail icons at the bottom. Background music plays throughout the visit.

Data collection

Data were obtained from participants through scale-based questionnaires and semi-structured interviews. The target population of this study consists of ordinary users in China. All participants were recruited through social media platforms, ensuring that participation in the study was open to anyone. The recruitment period began on January 4, 2023, and ended on December 10, 2023. A mobile phone–based online questionnaire was employed to apply the scale items. Prior to completing the questionnaire, participants were required to use their mobile phones to explore the VR exhibition of the Liangzhu Museum and locate a specific designated exhibit, as shown in Fig 3 . Once found, participants were free to explore the VR exhibition until they decided to end the session. We randomly selected a certain number of participants and asked if they would be willing to participate in further interviews.

https://doi.org/10.1371/journal.pone.0308267.g003

Random sampling was utilized to contact a certain number of participants through social media, asking them if they were willing to participate in further interviews. If the participants agreed, they proceeded to the interview phase. Participants who completed all scale items were eligible to participate in a lottery draw. Participants who completed the interviews received an additional gift valued at approximately US$7.

Over a period of approximately 12 months, a total of 347 questionnaires were collected, of which 314 were deemed valid. The sample size thus met the minimum requirements for SEM, with each estimated parameter having between 10 and 20 cases [ 79 ]. A total of 15 participants were involved in interviews (9 females and 6 males). All participants and interviewees participated anonymously after providing informed consent, and recordings were made with the interviewees’ consent.

Analysis tools

This study used the partial least squares (PLS) method to analyze the data, with Smart PLS 2.0 as the analysis software. PLS is a computational method for structural equation modeling (SEM)—that is, PLS-SEM. According to relevant statistical data, there has been a trend of significant growth in the use of PLS in empirical research, which indicates that it is increasingly a mainstream method for SEM analysis [ 80 ]. The advantages of its low requirements for residual distribution and smaller sample size have contributed to its increasing application in recent years [ 81 ]. The sample size requirements for PLS analysis are generally based on the 10 times rule, which suggests that the minimum sample size should be 10 times the total number of paths involving exogenous and endogenous variables in the model [ 82 – 84 ]. In this study, the model comprised 16 paths, so the minimum sample size required was no less than 160.

Ethical statement

The study was approved by the Academic Ethics Committee of Jiaxing University. The study complied with IRB principles. Prior to formally completing the questionnaire, all participants, including the guardians of minors, were duly informed regarding the purpose, content, data usage, risks, and benefits of the study, and all participants provided their electronic consent. All tests were done with the participants’ consent, and all questionnaires and interviews were completed anonymously.

Sample description

This study collected a total of 313 samples, including 182 females and 131 males, resulting in a female-to-male ratio of 1.38:1. The age group of 30–39 had the highest representation, with 170 individuals, accounting for 54.31% of the total sample. The next most represented age group was 18–29, with 107 individuals, comprising 34.19% of the total sample. The remaining age groups were as follows: 40–49 (25 individuals, 7.99% of the total sample), 50–59 (10 individuals, 3.19% of the total sample), and under 18 (1 individual, 0.32% of the total sample). The majority of participants held a university degree (247 individuals, 78.91% of the total sample), followed by those with other professional certificates (30 individuals, 9.58% of the total sample) and those with postgraduate degrees (26 individuals, 8.31% of the total sample). The lowest educational attainment was high school or below, with only 10 individuals, comprising 3.19% of the total sample.

Measurement model

We used factor analysis to evaluate the convergent and discriminant validity of the model. Convergent validity refers to the degree of association among observed variables within each construct in a research model to assess the rationality of item design within constructs [ 83 ]. Using the PLS algorithm, Cronbach’s α coefficient, composite reliability (CR), and average variance extracted (AVE) were computed to test the convergent validity of each construct, thus measuring the internal consistency within constructs. Constructs were considered to have high internal consistency when the Cronbach’s α coefficient was greater than 0.7, CR was greater than 0.7, and AVE was greater than 0.5. Table 3 presents the results of the convergent validity calculations for each construct. Most constructs had Cronbach’s α, CR, and AVE values exceeding the acceptable thresholds (Cronbach’s α, CR > 0.7, AVE > 0.5). Although the Cronbach’s α for interactivity is 0.687, which falls slightly below the 0.7 threshold, it still meets the acceptable range of 0.6–0.7 and is close to 0.7 [ 85 ]. The results thus indicate good internal consistency within the constructs.

https://doi.org/10.1371/journal.pone.0308267.t003

To further test the validity of the model, we employed cross-loadings and the Fornell–Larcker criterion to examine discriminant validity. It is generally considered that within the cross-leading of each construct, if the loading of each observed variable is greater than that of the other observed variables and exceeds 0.7, then the construct can be considered to have good discriminant validity. The Fornell–Larcker criterion assesses the discriminant validity between latent constructs by comparing the square root of the AVE of a construct with its correlations with other constructs [ 86 ]. The results for the cross-loadings and Fornell–Larcker criterion are presented in the Tables 4 and 5 . The results indicate that the constructs demonstrate good discriminant validity.

https://doi.org/10.1371/journal.pone.0308267.t004

https://doi.org/10.1371/journal.pone.0308267.t005

Structural model

This study employed bootstrapping to conduct structural model analysis of the research model to examine the causal relationships between latent constructs [ 83 ]. Through bootstrapping, the t-values and p-values of the paths were obtained to test the significance of path coefficients between latent constructs and validate the hypotheses. The computed results and decisions are summarized in Table 6 . The new relationships among variables in the research model are depicted in Fig 4 .

https://doi.org/10.1371/journal.pone.0308267.g004

https://doi.org/10.1371/journal.pone.0308267.t006

Interview results

External variables..

For the interactivity construct, participants generally provided positive evaluations of the VR exhibition overall, but also raised some specific issues. Most participants indicated that it was “okay” (VF1, VM7, VM9), “not bad” (VF2, VF11), or “alright” (VM10). The issues raised by participants mainly focused on the perception of orientation and the diversity of interactions within the VR environment. For example, some participants (VF1, VM4, VM9, VF15) found it easy to become disoriented or lost during the visit, while others (VM4, VF5) felt that the interaction with exhibits and artifacts was too simple and wanted more interactive elements.

Although most evaluations of immersion were positive, many participants also pointed out existing issues. Some participants mentioned that immersion was affected by the small size of the phone screen or unclear content, which resulted in evaluations such as “average” (VF1), “not good enough” (VF9), and “occasionally there” (VF12). Some participants also provided an “average” evaluation (VM4, M9) due to issues with orientation perception. It is worth noting that some participants believed that the VR exhibition’s music, voiceovers, and visual effects were good and enhanced the sense of immersion (VF5, VF11, VF14).

Most participants gave positive evaluations to presence. Many participants expressed feelings of “being there” (VF1, VF2, VF3, M9, VF12, VM14) or that the sense of presence was “very obvious” (VF6) and “like being there” (VF8, VF2). A significant number of participants attributed the good sense of presence to the realism of the VR visuals (VF1, VF3, VF12, VF13, VF2). However, some participants mentioned differences between the VR experience and the real world due to screen distortion (VM4) or lack of realism (VM7). Some participants also felt that presence was reduced due to a lack of interaction and diverse exhibition formats (VF5, VF15).

Internal belief variables.

All participants provided positive evaluations for the perceived usefulness construct. Almost all participants believed that the VR exhibition was useful and gave responses explicitly indicating its usefulness. However, some participants pointed out that the images and text were not clear enough and required extra effort to view (VF1, VF12). Others felt that the information provided by the VR was not only insufficient but also lacked firsthand data (VF6).

Most participants perceived the VR exhibition to be simple in terms of perceived ease of use. However, it is worth noting that some participants believed that it took some time to adapt before finding it simple (e.g., VF2, VM4, VM3.123, VF11). Some participants also mentioned that the small text of the user interface or the lack of prompts could affect perceived ease of use (VF3). Issues with orientation perception could also cause usability issues (VF8, VM9).

Most participants gave positive evaluations to the perceived enjoyment construct. Some participants mentioned that learning from the VR exhibition brought them joy (VF11). Interestingly, some participants thought that the VR music effects were excellent and enhanced the perceived enjoyment (VF2, VF3, VF13, VM8); however, there were also participants who felt the opposite (VF9). VF3 and M10 suggested that implementing gaming features in the VR arcade of the museum could further enhance enjoyment. It is also worth noting that perceived enjoyment of VR may decrease over time with prolonged experience (VF4, VM8, VF15).

Behavior intention variables.

In the intention of use construct, participants expressed varying degrees of willingness to use similar technologies in the future. When asked about their future use of similar technology, some participants gave very clear positive responses. However, there were also participants who believed that it would depend on the situation (VM5, VF11, VF15)—for example, they would use it before doing homework or traveling (VF12).

For the tendency to visit actual sites construct, although most participants admitted to having the intention to visit physically, the reasons varied. Some participants thought it was due to the lack of clarity in the VR content (VF1) or insufficient information that led to the desire for a physical visit (VM4, VM6, VF12). Others became interested in a physical visit due to their positive experience with the VR exhibition (VF3, VM7, VF13, VM14). Some participants were originally interested in museum visits and this led to their intention to pursue a physical visit (VF5). Additionally, some participants considered physical tourism due to the lifting of COVID-19 restrictions (VF5, VF11).

Overall assessment.

The majority of participants provided positive evaluations regarding their overall satisfaction with VR. Many participants gave moderately positive overall ratings. However, some participants considered the current exhibition format to be relatively monotonous and thus provided average evaluations (VF15).

Extension of existing research

This study found that the interactivity experience of museums’ online VR exhibition positively influenced visitors’ perceived enjoyment, perceived usefulness, and perceived ease of use. The results of this study confirm the significant role of interactivity in enhancing visitor acceptance of museums’ online VR exhibition technology. The findings supported hypotheses H1, H2, and H3. This research thus corroborates and extends existing research findings, such as those from studies suggesting that interactivity can enhance users’ perception of the richness of VR media and emphasize users’ ability to control their experience [ 53 , 87 ].

The immersion of the VR exhibition was positively correlated with perceived usefulness but did not positively influence the perceived ease of use and perceived enjoyment of the system. The immersive quality of VR can be understood as the extent to which the virtual environment provides users with a sensory experience close to objective reality that allows users to temporarily forget about the real world and the passage of time during the experience. According to the results of this study, hypothesis H6 was confirmed. This is consistent with some prior research conclusions; for example, Vallade and Kaufmann [ 88 ] suggested that if participants perceive the VR environment as real, they should consider it a high-quality experience, which affects users’ perception of system usefulness.

On the other hand, hypotheses H4 and H5 were not supported by the results. This indicates that enhancing the immersion of museum online VR systems did not positively influence visitors’ perceived ease of use and perceived enjoyment. This result is inconsistent with some existing research findings. For example, Vishwakarma and Mukherjee [ 58 ] suggested that a high level of immersion enables people to enjoy the online VR experience and increases their willingness to further use such technology. The analysis in this study suggested that the following reasons may lead to the lack of support for hypothesis H5:

• The immersion of VR is determined by the extent to which VR systems stimulate users’ senses to approximate the real objective environment. The difficulty of VR use is related to the complexity of its logic. A sense of immersion is obtained from the sensory experience of directly stimulating the senses, which is quite different from the rational experience of the VR usage logic, so there may be no direct relationship between the two.
• According to the interview results, the perception of enjoyment in VR use is subjective and unstable. This may affect the relationship between immersion and enjoyment. For example, participant VF15 suggested that the perception of enjoyment in experiencing VR diminishes as the experience time increases. Some participants (VF5, VF11, VF14) thought that music was one of the main things affecting immersion; however, according to the interviews, participants’ evaluations of the music elements in VR were inconsistent. For example, participant VM9 explicitly stated a dislike for the existing VR background music. These reasons may lead to the lack of support for hypothesis H5.

The results of this study indicate that the sense of presence in the online VR experience positively influenced visitors’ perceived ease of use, perceived enjoyment, and perceived usefulness. The sense of presence involves users’ subjective psychological state of perceiving themselves as being present in a virtual environment, which can be briefly described as the degree to which users feel immersed in VR. This can be assessed by the extent of visitors’ transition between the real world and the digital virtual world [ 49 – 51 ]. The results of this study supported hypotheses H7, H8, and H9. These conclusions also corroborate existing research findings [ 88 – 90 ].

The results of this study suggest that perceived enjoyment positively influences perceived ease of use, perceived usefulness, and intention of use in VR exhibition. Perceived enjoyment refers to the degree to which individuals feel happy when experiencing museum online VR exhibition technology, and this perception is unrelated to utility [ 91 ]. The results of this study supported hypotheses H10, H11, and H12. These conclusions are similar to existing research findings [ 92 , 93 ]; however, in contrast to most studies that suggest a positive influence of perceived ease of use on perceived enjoyment, this study found that perceived enjoyment also has a positive relationship with perceived ease of use. For example, some research suggests that perceived ease of use is an important predictor of people’s acceptance of new digital technology systems and directly affects their perception of system enjoyment [ 76 ]. This point has also been confirmed by many other studies [ 34 , 94 ].

New findings of this study

The design of online VR exhibition content in museums needs to consider the needs of various media and diverse audience groups. Although VR provides panoramic guidance composed of thumbnail images of different venues and exhibition areas, some respondents raised concerns about the small text size and incomplete display, which may affect their overall experience (VF3, VM14). One possible reason for this situation is that the VR test in this study primarily used smartphones as the main playback medium, which resulted in limited screen size. Comparing the playback effects between smartphones and computers, that VR text displayed on computers was indeed clearer and more complete. However, targeting smartphones as a platform has become a trend as smartphones have become the mainstream way for the general public to access information. Designers and managers of online VR museums should thus fully consider the compatibility between online VR exhibitions and smartphone display characteristics; they should develop graphic and textual content suitable for playback on smartphones.

Another aspect is the need to consider the differences in needs among elderly people, those with poor vision, and visitors who do not speak Chinese in the context of aging societies and globalization. Although the current experiment did not involve elderly people or those with poor vision, a considerable number of respondents (VF1, VF3, VM14) still mentioned that reading information was difficult due to small font sizes or graphics. The universality of VR is thus an important issue that cannot be ignored in the design, development, and deployment of VR applications.

The potential of digital and network technologies needs to be further leveraged to enrich the visitor experience in museum VR exhibitions. Online VR exhibitions are novel digital technologies that integrate video, music, and narration to provide visitors with a richer audiovisual experience. This advantage was confirmed in interviews with respondents VF3, VM4, VM10, VF11, and VF13. However, many respondents also indicated that the online VR exhibition of the Liangzhu Museum did not fully meet their needs. For example, there was a lack of variety in the interaction between visitors and museum artifacts. Some respondents pointed out that, while VR offers gaming spaces, the current exhibition lacked gaming features (VF3, VM10), and there was limited interaction with exhibits (M7, F12). Additionally, it is worth noting that there were differing opinions about the background music for the museum’s online VR exhibition (VF2, VF13, VF9), which indicates the complexity of visitor demands. To meet the needs of different visitors, it is necessary to increase the diversity and selectivity of exhibition content. Managers and developers should fully utilize the advantages of digital and network technologies to provide diversified services. These measures may include but are not limited to:

• Providing hyperlinks to expand on artifacts, displaying more detailed and professional academic information to meet the needs of visitors with academic interests;
• Providing more interactive and game designs to meet entertainment needs;
• Providing more video, image, and text-based information to meet learning needs; and
• Integrating the exhibition with social software or media to meet visitors’ social needs.

Good navigational performance ensures that visitors can always understand their location and orientation, which is an factor for improving the acceptance of museum VR exhibitions. According to descriptions from respondents VF1, VM4, VM9, and VF15, it was easy to become disoriented and even experience circling in the virtual environment, which greatly affected their visitor experience. The analysis suggested that one reason for this could be the inconsistency between the change in screen perspective and the expected direction indicated by the VR guidance arrow after clicking on the arrow. Another reason could be the lack of a navigation system design adapted to smartphone screens; the existing navigation system icons and text were too small and incomplete on the phone, which made them easy to ignore. Future VR design and development or improvement should strengthen the design of spatial orientation and guidance systems.

Research implications

Theoretical implications.

This study expanded the applicability and explanatory power of the traditional TAM by incorporating variables such as interactivity, immersion, and presence. Through a combination of quantitative and qualitative analyses, this study revealed how the experience of users of online VR museum exhibitions influences their intrinsic technological beliefs, thereby affecting their intention to use the technology and visit the museum. This extension not only enhances our understanding of user acceptance of online VR museum exhibition technology, but also fills a research gap in this field. This study also emphasized the need to consider different media and diverse user groups in VR content design, which can provide new insights for future research on the role of different device platforms in VR experiences. These findings offer new theoretical perspectives and empirical support for the further exploration of user experience with museum digital resources in academia.

Practical implications

The research findings provide guidance for museums in formulating digital strategies and enhancing user experiences. First, museums should focus on improving the interactivity, immersion, and presence of online VR exhibitions to increase user acceptance of the technology and their willingness to use it. Second, curators, exhibition designers, and other decision-makers should enhance the inclusivity and diversity of VR exhibitions to meet the needs of elderly individuals, those with visual impairments, and visitors who speak different languages. This can be achieved by offering interactive game designs and rich multimedia content to improve user experiences. Additionally, emphasizing the importance of navigability in online VR exhibitions can help enhance the overall user experience. This has practical significance for designing and developing online VR exhibitions based on 360° panoramic photography, especially in large and complex museum spaces. Through these improvements, museums can not only expand their audience reach and promote cultural dissemination, but they can also enhance the educational effectiveness of online exhibitions, enabling visitors to understand and experience cultural heritage more profoundly in a virtual environment.

Conclusions and limitations

This study took the VR exhibition technology of Liangzhu Museum as an example and established a TAM-based research model after a literature review. The classic TAM was adjusted by adding exogenous latent variables, internal belief variables, and behavioral intention variables. The exogenous latent variables were interactivity, immersion, and presence. The internal belief variables were perceived usefulness, perceived enjoyment, and perceived ease of use, while the behavioral intention variables were intention of use and tendency to visit actual sites. The online VR exhibition of Liangzhu Museum was then used as a case study to verify the hypotheses about the variables and their relationships proposed in the model. Except for the lack of a positive impact of interactivity on perceived ease of use and perceived enjoyment, all other research hypotheses were confirmed. The research model demonstrated good internal consistency and predictive power. Additionally, navigability may be a new variable influencing visitors’ acceptance of museum VR exhibitions.

Although museum online VR exhibitions have the advantage of digital technology, they still need to fully consider the differentiated needs of users. The interviews revealed that there are significant differences in visitors’ demands for VR exhibitions in museums. While digital and network technologies expand the boundaries of museum visitors, they also increase the difficulty of understanding them. As Lester [ 95 ] pointed out, virtual exhibition visitors are far more diverse than traditional museum visitors; these differences may stem from their abilities, perceptions, and demands related to museums and digital virtual exhibitions, as well as the complexity of deeper cultural and social backgrounds. Understanding these complexities has long-term significance for promoting the sustainable development of museums, especially in the context of social diversification, democratization, and economic recovery.

Subsequent research should provide more case studies of different types of museums. Liangzhu Museum is a medium to large cultural heritage museum that primarily display local archaeological artifacts; it represents just one of the many types of museums in the world. Whether the conclusions of this study are applicable to other types and sizes of museums requires more interactive confirmation through different case results by subsequent researchers. More studies on different types of visitors are also needed. As mentioned, virtual exhibition visitors have diverse differences. Subsequent research should consider samples of different types of visitors, including families, tour groups, the elderly, children, and people with different physical and mental abilities. Subsequent research needs to understand these different visitor types more meticulously and deeply, particularly when confronted by VR exhibitions, to further enhance visitors’ acceptance of this technology.

Supporting information

S1 data. data for statistical analysis..

https://doi.org/10.1371/journal.pone.0308267.s001

• View Article
• Google Scholar
• 4. Burton C, Scott C. Museums: Challenges for the 21st century. In: Sandell R, Janes RR, editors. Museum management and marketing. 1st ed: Routledge; 2007. p. 49–66.
• PubMed/NCBI
• 13. Giannini T, Bowen JP. Museums and digitalism. In: Giannini T, Bowen JP, editors. Museums and Digital Culture. 1st ed: Springer; 2019. p. 27–46.
• 15. Barbieri L, Bruno F, Mollo F, Muzzupappa M. User-centered design of a Virtual Museum system: a case study. In: Eynard B, Nigrelli V, Oliveri S, Peris-Fajarnes G, Rizzuti S editors. Advances on Mechanics, Design Engineering and Manufacturing: Proceedings of the International Joint Conference on Mechanics, Design Engineering & Advanced Manufacturing (JCM 2016). Cham: Springer; 2017. pp. 155–65.
• 22. Gaitatzes A, Christopoulos D, Voulgari A, editors. 6th International Conference on Virtual Systems and Multimedia, Ogaki, Japan. 6th International Conference on Virtual Systems and Multimedia; 2000 3–6 Oct.; Ogaki, Japan.
• 23. Jung T, tom Dieck MC, Lee H, Chung N. Effects of virtual reality and augmented reality on visitor experiences in museum. In: Inversini A, Schegg R editors. Information and Communication Technologies in Tourism 2016. Cham: Springer. 2016. pp. 621–35.
• 26. Lepouras G, Charitos D, Vassilakis C, Charissi A, Halatsi L, editors. Building a VR-Museum in a Museum. Proceedings of Virtual Reality International Conference; 2001 May 16–18; Laval, France. 2001.
• 27. Roussou M, editor. Immersive interactive virtual reality in the museum. Proceedings of the Trends in Leisure Entertainment; 2001 Jun; London, UK. 2001.
• 42. Zhang P, Li N, editors. Love at first sight or sustained effect? The role of perceived affective quality on users’ cognitive reactions to information technology. Proceedings of the International Conference on Information Systems (ICIS’04); 2004 Dec 12–15; Washington, D.C., USA. 2004.
• 49. Alomary A, Woollard J, editors. How is technology accepted by users? A review of technology acceptance models and theories. Proceedings of the 5th International Conference on 4E; 2015 Nov 21; London, UK. 2015.
• 56. Bafadhal AS, Hendrawan MR. Exploring the immersion and telepresence in gamified virtual tourism experience toward tourist’s behaviour. In: Supriono B, Zauhar S, Suhadak , Astuti ES, Sumartono , Sujarwoto , et al., editors. Annual International Conference of Business and Public Administration (AICoBPA 2018); Malang, Indonesia. Paris: Atlantis Press; 2019. p. 53–6.
• 65. Huang Y-C, Backman SJ, Backman KF. The impacts of virtual experiences on people’s travel intentions. In: Gretzel U, Law R, Fuchs M, editors. Information and Communication Technologies in Tourism 2010; 2010 Feb 10–12; lugano, Switzerland. Vienna: Springer; 2010. p. 555–66.
• 68. Lee H, Chung N, Koo C, editors. Moderating Effects of Distrust and Social Influence on Aesthetic Experience of Augmented Reality: Motivation—Opportunity—Ability Model Perspective. Proceedings of the 17th International Conference on Electronic Commerce 2015; 2015 Aug 3–5; Seoul, Republic of Korea. New York: Association for Computing Machinery; 2015.
• 92. Diefenbach S, Kolb N, Hassenzahl M, editors. The’hedonic’in human-computer interaction: history, contributions, and future research directions. Proceedings of the 2014 conference on Designing interactive systems; 2014 Jun 21–25; Vancouver, Canada. New York: Machinery.

The impact of near-peer virtual agents on computer science attitudes and collaborative dialogue

New citation alert added.

This alert has been successfully added and will be sent to:

You will be notified whenever a record that you have chosen has been cited.

To manage your alert preferences, click on the button below.

New Citation Alert!

Please log in to your account

Information & Contributors

Bibliometrics & citations, view options, recommendations, accessible multimodal-interaction platform for computer-supported collaborative learning system.

In order to facilitate learning in the context of social interactions, a number of studies have proposed computer-supported collaborative learning (CSCL) systems by using information and communication technologies. The basic design of a CSCL system is ...

Inferring interaction to support collaborative learning in 3D virtual environments through the user's avatar Non-Verbal Communication

When a group of students, placed around a shared workspace and working on a predefined task, is observed, the teacher can intuitively, to a certain extent, understand the collaboration that is taking place within the group without listening to the ...

Designing for collaborative learning in immersive virtual reality: a systematic literature review

Immersive learning technologies such as virtual reality have long been deemed as the next generation of digital learning environments. There is a limited number of studies addressing how immersive technologies can be designed, applied, and studied ...

Information

Published in.

Elsevier Science Publishers B. V.

Netherlands

Publication History

Author tags.

• Virtual learning companions
• Elementary school
• Computer science learning
• Collaborative learning
• Research-article

Contributors

Other metrics, bibliometrics, article metrics.

• 0 Total Citations
• 0 Total Downloads
• Downloads (Last 12 months) 0
• Downloads (Last 6 weeks) 0

View options

Login options.

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Share this publication link.

Copying failed.

Share on social media

Affiliations, export citations.

• Please download or close your previous search result export first before starting a new bulk export. Preview is not available. By clicking download, a status dialog will open to start the export process. The process may take a few minutes but once it finishes a file will be downloadable from your browser. You may continue to browse the DL while the export process is in progress. Download
• Download citation
• Copy citation

We are preparing your search results for download ...

We will inform you here when the file is ready.

Your file of search results citations is now ready.

Your search export query has expired. Please try again.

• DOI: 10.33394/ijete.v1i2.12316
• Corpus ID: 271750288

Physics Learning Technology for Sustainable Development Goals (SDGs): A Literature Study

• S. Prayogi , N. Verawati
• Published in International Journal of… 23 July 2024
• Physics, Education, Computer Science
• International Journal of Ethnoscience and Technology in Education

Figures and Tables from this paper

30 References

Utilization of technology in physics education: a literature review and implications for the future physics learning.

• Highly Influential

SUSTAINABLE DEVELOPMENT GOALS AND THEIR IMPLEMENTATION IN PHYSICS

Utilizing ai for physics problem solving: a literature review and chatgpt experience, revealing the true potential and prospects of augmented reality in education, utilisation of augmented reality technology in physics education: a bibliometric analysis and its impact on sustainable development goals (sdgs), improving the learning-teaching process through adaptive learning strategy, addressing the pisa 2022 results: a call for reinvigorating indonesia's education system, renewable energy learning project in physics classroom: achieving education for sustainable development, the effect of problem based learning model assisted by phet simulation on understanding physics concepts, digitization of learning through virtual laboratory in assessing students' science process skills on mechanical wave material, related papers.

Showing 1 through 3 of 0 Related Papers

Main navigation

• Continuing Nursing Education
• News & Events

Argerie Tsimicalis champions virtual reality in children's healthcare

• Add to calendar
• Tweet Widget

When Argerie Tsimicalis, RN, PhD, was tasked with bringing virtual reality (VR) to the Shriners Hospitals for Children® - Canada (SHC-Canada), she jumped at the chance to work collaboratively with former Chief of Staff, Dr. Reggie Hamdy and Director of Nursing and Patient Care Services, Ms. Kelly Thorstad. Together, they explored with the healthcare team how best to use this technology to alleviate pain and anxiety associated with medical procedures.

An associate professor at the Ingram School of Nursing (ISoN) and associate member of McGill’s Department of Oncology and Faculty of Dental Medicine and Oral Health Sciences, Prof. Tsimicalis is a Fonds de recherche du Québec – Santé Junior 2 research scholar and a nurse scientist at SHC-Canada in Montreal. As she explains, “When a child feels pain, that increases their fear around medical procedures such as cast removals, pin removals, and intravenous inserts. We found that virtual reality is an effective non-pharmacological tool that distracts children, reducing their pain and distress and allowing them to see the hospital as a safe space.”

The work to bring VR to SHC-Canada began in earnest in 2018, where Drs. Hamdy and Tsimicalis co-supervised master’s research student Sofia Addad in McGill’s Experimental Surgery program. Ms. Addad, conducted a systematic literature review of the use of VR in children’s health care, and tested the feasibility of using VR during medical procedures at SHC-Canda. Alongside, Prof. Tsimicalis partnered with Prof Sylvie Le May, a child health pain expert based at CHU Sainte Justine and the Université de Montréal, Together, they were awarded a CIHR grant for a multi-site, pilot randomized control trial on the use of VR during removal of pins and sutures.

Professor Tsimicalis’ team also was awarded an infrastructure grant to survey clinicians in Quebec regarding the use of VR in children’s healthcare settings and provide training and resources to support the implementation of VR into practice. The team has expanded, creating a hub of scientists, trainees, and advocates, where new projects have ensued. The team has welcomed PhD students conducting clinical trials studying the use of VR mindfulness programs depicting nature scenes to ease the perioperative anxiety for children as well as testing the feasibility of incorporating VR for dental, anesthesia and surgical interventions.

Recognized internationally for its expertise in treating rare and painful bone conditions such as osteogenesis imperfecta (OI), SHC-Canada encourages experimentation with innovative technologies. “It’s a magical place where support for new ideas and creativity flourish - a perfect fit for my own creative bent,” notes Professor Tsimicalis. Storytelling features prominently in her research, which uses different media such as comic books, songs and humorous videos as teaching tools. She is particularly proud of her work as an editor for The Dream Machine , a novel about a 16-year-old girl living with OI who witnesses her younger sister fracturing her leg in a downhill skiing accident.

On April 26, 2024, with the support of the Réseau de recherche en santé buccodentaire et osseuse, Professor Tsimicalis hosted a Virtual Reality Workshop at SHC-Canada featuring leading experts in VR and implementation science. Ninety-five healthcare professionals attended the event, which included presentations by Prof. Tsimicalis, Guillaume Fontaine, Sylvie Le May, and Stéphane Bouchard.

Given that implementation of new technologies is a complicated process requiring support from a wide variety of stakeholders, Prof. Tsimicalis continues to work diligently on drafting policies, procedures and resources for the successful integration of VR SHC-Canada. “I’m a firm believer in interprofessional collaboration,” she asserts. “From the beginning, we engaged everyone from top-down to bottom-up, from doctors, nurses and allied health professionals right through administrative, technical and kitchen staff.”

Getting buy-in from patients, families and the general public has been critical to the success of VR implementation at SHC-Canada. To that end, Professor Tsimicalis has pursued creative ways to provide parents and children with information about VR in easily digestible formats. For example, she published two children’s books showcasing the use of VR, one of which was put together with the help of McGill nursing alumni Miranda Harington, and spearheaded the creation of a cartoon to educate children about non-pharmacological approaches to pain management.

“VR has been a gamechanger for our patients,” concludes Prof. Tsimicalis. “I’m excited to see what the future will bring.”

Department and University Information

Ingram school of nursing.