Pre-operative anxiety is prevalent among pediatric patients undergoing surgical procedures, estimated to be as high as 75% (Fronk & Billick, 2020; Getahun et al., 2020). Anxiety is manifested through post-operative complications and behavior changes, reduced wellbeing, and increased healthcare costs (Agbayani et al., 2020; Kain et al., 1996, 1999, 2006; Mathias et al., 2023). Clinicians use sedative premedication and non-pharmacological interventions to alleviate pre-operative anxiety in pediatric patients, including clown and music therapy, puppet and therapeutic play, pre-operative preparation videos, and video games for distraction (Dionigi et al., 2014; Fortier & Kain, 2015; Huang et al., 2021; Liguori et al., 2016; Markova et al., 2021; Wright et al., 2007). The role of parents, as well as certified child life specialists (CCLSs) and other healthcare professionals, in providing emotional support and addressing children’s concerns cannot be understated (Getahun et al., 2020; Kain et al., 1996; West et al., 2020; Wright et al., 2007). Non-pharmacological interventions are gaining in popularity as studies demonstrate their efficacy in reducing pre-operative anxiety and the need for pharmacological interventions (Agbayani et al., 2020; Mathias et al., 2023). Pharmacological interventions require additional monitoring to minimize health risks and cannot be administered outside of clinical settings like some non-pharmacological interventions (Heikal & Stuart, 2020; Wang et al., 2022). Despite their potential, differing resources across hospital systems make it hard to scale non-pharmacological interventions. Variations in their delivery make it challenging for clinicians to deliver them with fidelity so they can be evaluated and standardized across hospitals.
Virtual reality (VR), a computer-generated environment that users view through a 3D display and interact with via hand tracking or controllers, is a nascent medium for non-pharmacological interventions. VR is promising for children due to its acceptability and potential to enhance learning in this population (Liu et al., 2020). Like pre-operative preparation videos, clinicians can standardize VR content and delivery. Additionally, VR immerses users in virtual environments and allows them to interact with simulated objects and characters to create a sense of presence that is not possible through 2D visualizations (Weber et al., 2021). VR-induced premost sence addresses anxiety through two mechanisms. Virtual environments with entertaining content distract patients from feelings of anxiety and pain in the real world (Burns-Nader & Goldstein, 2023; Chao et al., 2023; Spiegel, 2020; Teh et al., 2024). Virtual environments that mimic real-life stress-inducing scenarios, similar to exposure therapy (Boeldt et al., 2019), address anxiety and the fear of the unknown by acclimating patients to what they will face in the real world (Rizzo et al., 2009; Spiegel, 2020). To date, clinicians have mainly used VR to distract patients from feelings of anxiety in the hospital, especially pediatric patients (Ridout et al., 2021).
This study contributes to an emerging field of VR research, helping patients reduce their pre-operative anxiety by virtually experiencing and acclimating to perioperative environments (Eijlers et al., 2019; Gold et al., 2021; Ryu et al., 2017). The study evaluates the feasibility and acceptability of a VR app that provides a gamified walkthrough of the perioperative environment for pediatric patients before surgery as the first step in a process to evaluate the app’s impact on pre-operative anxiety. All authors, except Nguyen, developed the app to ensure it met specifications for pediatric patients, by appealing to children through animal characters, for example, and by adding enough realism to simulate pre-operative procedures. The developers also wanted the app to be publicly available via Meta Quest through their App Lab store and accessible by clinicians and patients alike. This paper focuses on the results of a pilot study that evaluated the feasibility and acceptability of the VR app with pediatric patients before surgery and their parents in a hospital. Most prior research has only reported feedback from pediatric patients. Birckhead et al. (2019) discuss the development and evaluation of VR healthcare applications through three types of studies that begins with content development informed by providers and patients (i.e., VR1 studies) and is followed by pilot testing to assess feasibility and acceptability (i.e., VR2 trials). Authors discuss the app development process and results of the pilot study in line with a VR1 study and VR2 trial, respectively. It was hypothesized that the pilot study would demonstrate the app to be feasible and acceptable based on prior VR research with pediatric patients discussed above.
Method
Overview
The research team included a CCLS (Serrano), a pediatric surgeon (Wagner), two pediatric hospitalists (Liu and Newcomer), and a behavioral researcher (Comulada). Human-centered design principles were used to develop Surgery Prep, a VR app that provides pediatric patients with a walkthrough of the perioperative environment to reduce their pre-operative anxiety. Three groups of stakeholders informed app development as described below: clinicians, VR app developers, and children. Development was followed by a single-arm pilot study to evaluate the feasibility and acceptability of Surgery Prep with pediatric patients scheduled for surgery at the University of California, Los Angeles (UCLA) Santa Monica Medical Center, a hospital that is located in Santa Monica, California and is part of the UCLA Health healthcare system.
Surgery Prep App Development
First, clinical study team members (Serrano, Wagner, Liu, and Newcomer) and an additional CCLS listed in the acknowledgements (Pike) specified virtual environments and activities users would engage in throughout the walkthrough as summarized in Table 1. The team first decided on hospital scenes that included a pre-operative area, the operating room (OR), and a recovery area like Eijlers et al. (2019). Inclusion of a recovery area was motivated by anecdotal evidence. The CCLS felt it was important to help the user understand they will wake up after surgery, a concern that pediatric patients had voiced to the CCLS during their clinical practice. The team also specified key background sounds that patients would likely hear, such as the beeps from a vital signs machine and the background chatter of the surgical team in the OR.
The next decision was whether to use a first-person perspective like Eijlers et al. (2019) or a third-person perspective like Gold et al. (2021) and Ryu et al. (2017). The study team selected a third-person perspective for two reasons. First, a third-person perspective reduced the potential for VR sickness, a type of motion sickness. VR sickness is caused by a mismatch between the user’s perceived motion in the virtual world and actual motion (e.g., if the user is virtually walking while standing still) (Chang et al., 2020). A third-person perspective accommodated scenarios where the user did not have to move except to rotate their head to look around so that the actual and perceived movements in the virtual experience were aligned. Second, a third-person perspective enabled the use of gamification. The experience centered on a non-human character instead of the user. This allowed the user to interact with the character to prepare them for surgery as a form of medical play that gives children a sense of agency and provides experiential learning to help reduce anxiety and improve knowledge retention (Rashid et al., 2021).
Last, clinicians selected activities aligned with common pre-operative procedures and those they felt were troubling for pediatric patients, like the placement of an induction mask over their face to administer anesthesia. Activities matched those featured in other VR perioperative walkthroughs for pediatric patients (Eijlers et al., 2019; Ryu et al., 2017).
Once clinicians specified virtual scenes and activities, the behavioral researcher led the development of Surgery Prep in collaboration with the study team and VR developers. The researcher met with four girls and three boys within the eligible age range for patient participation (i.e., five to 14-years old) as stakeholders who were not part of the study. The researcher selected the children as a convenience sample from the children of parents within the researcher’s social network to solicit ideas for the gamification elements, including what the non-human character would look like. In consultation with child stakeholders, the researcher selected a narwhal as the non-human character for its appeal across cultures and genders and popularity (Bates, 2023; Charry et al., 2021; McLeish, 2013; Sulc, 2023). A sea creature theme was maintained for other characters in the VR experience, such as an octopus as an anesthesiologist.
The researcher storyboarded activities in each virtual scene and met with VR developers to finalize VR content based on their expertise regarding what would be visually appealing and realistic to program. For example, developers used existing characters and virtual environments (i.e., assets) rather than creating them to optimize development time and reduce costs. The developers iteratively updated the walkthrough after receiving additional input from clinicians and children. For example, the original OR asset contained patches of red that were recolored to align with common medical practice to avoid using red colors in the OR. Activities in the original iteration required users to press trigger buttons on the hand controllers. This was hard to reach for children with smaller hands, so developers reconfigured activities so that users could press any buttons on the hand controllers to engage with the activities. Developers built the app through the Unity game engine development platform for the Meta Quest headset.
Pediatric Patients
The study team aimed to recruit 15 pediatric patients at the UCLA Santa Monica Medical Center to pilot Surgery Prep. A CCLS who was part of the study team (Serrano) was the study coordinator and conducted all recruitment and pilot study activities. Recruitment began in February 2024. Eligible patients were five to 14 years old, English speaking, scheduled for a surgical procedure requiring anesthesia, able to view the VR experience through a VR headset and use hand controllers to navigate the VR experience. The eligible age range was chosen to align with prior VR studies in pediatric patient populations that demonstrated VR headset tolerability in children as young as 4 to 6 years old (Dehghan et al., 2019; Eijlers et al., 2019; Gold et al., 2021; Tychsen & Foeller, 2020). Exclusion criteria included seizure disorders, developmental delays precluding the ability to interact with the application, or other underlying medical disorders determined by the study coordinator to prevent the patient’s ability to safely participate in the trial. This study was approved by the UCLA Institutional Review Board (IRB#23-000336).
Study Procedures
The study coordinator referenced a schedule of upcoming surgeries and approached patients in the eligible age range awaiting same-day surgery in the pre-operative area. The coordinator invited patients to participate in the study in the presence of a parent or legal guardian, hereafter referred to as a parent. The coordinator then sought parental permission for interested patients to participate in the study and scheduled a one-time visit in a designated hospital room. At the visit, the study coordinator provided information about the study and VR app to patients and their parents. The study coordinator also conducted chart reviews and informally observed mental and physical states of the patients to check for developmental or physical limitations that could prevent them from wearing a VR headset and using VR hand controllers to participate in the study. The study coordinator approached 18 patients to participate in the study and three declined to participate after hearing about the study. One patient declined participation because they did not like video games, and the other two patients were focused on their upcoming surgeries and not interested in participating in a study. Fifteen eligible patients who wished to participate provided assent and their parents provided written consent. The study coordinator then handed parents an iPad to complete an electronic (Qualtrics, Provo, UT) survey and report sociodemographic and background characteristics about themselves and their children enrolled in the study. Next, the study coordinator showed patients how to use the VR hardware, the Meta Quest 2 (Figure 1), helped them put on the VR headset, and invited patients to pilot test Surgery Prep for up to 10 minutes. The study team designated a conservative 10-minute time limit to reduce the possibility of cybersickness that Palmisano et al. (2020) reported for most of their participants playing VR video games after 10 to 15 minutes. Other investigators have permitted longer durations for healthcare applications like Surgery Prep (e.g., 25 to 30 minutes for Doc McStuffins; Gold et al., 2021). Afterwards, the study coordinator administered electronic surveys to the parents and patients on an iPad to assess their experience using the VR app. All of the parents and nine of the 15 patients ages 10 and older filled out the survey on their own. The coordinator read survey questions aloud to younger patients and assisted them in filling out the survey. At the conclusion of the study, the coordinator provided traditional child life preparatory services to all patients that were tailored to the needs of each patient by an informal developmental assessment.
Measures
Sociodemographic and Background Characteristics
Parents reported their gender identity as a “Man,” “Woman,” or “Other” based on pre-specified categories, ethnicity, race independent of ethnicity, and age. They also reported on characteristics of their children in the study: sex assigned at birth, age, the number of hospitalizations in the past 12 months prior to the admission for the upcoming surgery, the number of days in the hospital for the upcoming surgery, and the type of scheduled surgery.
Feasibility
Feasibility was determined by the ability to meet recruitment numbers, a minimum number of refusals to participate in the study, and low attrition. The study team anticipated attrition to occur from technical issues, either due to the VR equipment or Wifi connectivity, and adverse events associated with VR use (e.g., eye strain, dizziness, and nausea). The study coordinator tracked technical issues and adverse events based on their own observations, and reports from patients and their parents through open-ended survey questions they responded to after patients pilot tested Surgery Prep. There were four open-ended questions for parents: “What do you think your child liked about playing Surgery Prep?” “What do you think your child disliked about playing Surgery Prep?” “What suggestions or ideas do you think your child has for improving Surgery Prep?” and “Do you have any comments?” Similarly, there were four open-ended questions for patients: “What did you like about Surgery Prep?” “What did you dislike about Surgery Prep?” “What suggestions or ideas do you have for improving Surgery Prep?” and “Do you have anything else you would like to tell me about your experience using Surgery Prep?”
Acceptability
The study coordinator administered the 10-item System Usability Scale adapted for children between the ages of seven to 11 years old (SUS; Putnam et al., 2020) to patients as part of the electronic survey after they pilot tested Surgery Prep. Following standard practice, the study team calculated an SUS summary score that ranges from zero to 100 with scores of 68 and higher indicating good to excellent VR app usability and acceptability. The study coordinator also administered a single question used to calculate the Net Promoter Score (NPS) as a measure of a user’s satisfaction with a product (Reichheld, 2003). The question asked patients to rate how likely they would be to recommend Surgery Prep to a friend before their surgery. They were shown a scale from zero to 10 with sad and happy faces anchoring zero and 10, respectively. The NPS is the difference between the percentage of patients with scores of 9 and 10 (i.e., promoters) and the percentage of patients with scores of six or less (i.e., detractors) and ranges from -100 to 100. The study team presents the NPS and individual scores to discuss them in the context of open-ended question responses, along with SUS scores for individual patients.
Analysis
Descriptive statistics summarize parent and patient sociodemographic characteristics and background information, including means and standard deviations for continuous measures and proportions and sample sizes for categorical measures. Ninety-five percent confidence intervals for the SUS and NPS question estimate the uncertainty of the mean scores relative to standard benchmarks. The study team conducted all quantitative analyses using R software version 4.2.2 (R Core Team, 2022). They used a mixed-methods approach in two steps to analyze qualitative and quantitative feasibility and acceptability data and to evaluate the consistency of the results. First, two authors (Serrano and Comulada) independently transcribed open-ended question responses and organized qualitative content into themes about the benefits and limitations of Surgery Prep. The two authors then compared generated themes and discussed findings with the other authors to reach consensus. Second, authors compared individual open-ended question responses to the SUS and NPS question responses to check for alignment between the qualitative and quantitative findings.
Results
Table 2 shows the demographic characteristics and surgery details of the patients enrolled in the study and their parents. Almost two-thirds of the parents were women (60.0%) and the remainder were men (40.0%). Most parents reported a Hispanic ethnicity (73.3%) and were between the ages of 35 to 44 years old (73.3%). Sex at birth was female for 60.0% of the patients and the other 40% were male. The median age for patients was 11 years old and ranged from seven to 14 years old. All but one patient had no prior hospitalizations in the past 12 months. Scheduled surgeries for the current admission covered a range of elective procedures as indicated in Table 2.
As promising indicators of feasibility, we met our recruitment goal with a minimum number of patients declining to participate (n = 15 of 18 patients approached). All enrolled patients completed the study. No patients reported dizziness or nausea, symptoms sometimes accompanying VR use. Only one patient, age 13, reported eye strain after study completion (“my eyes were blurry after”). Technical issues were also minimal. One patient commented about connectivity problem but was still able to complete the study (“when I moved to another room it stopped working”).
Qualitative data and quantitative measures indicate a reasonably high level of acceptability. The average SUS of 72.2 (95% CI = 64.0 – 80.3) is above the cut off for a reasonable level of app usability and acceptability. Similarly, the NPS of 67 (= 100 x (11/15 – 1/15)) indicates a high level of satisfaction; the average of the individual scores was 8.7 (95% CI = 7.3 – 10.2). These scores match the range of feedback. There were numerous positive comments from parents like one indicating “it should be available to all kids.” Other parent comments suggested that Surgery Prep could help patients psychologically prepare for surgery (e.g., “nice idea for patients to know what to expect”) and distract them (e.g., “something to pass the time when he waited to go to surgery”). Parents also liked the novelty of Surgery Prep (e.g., “playing something new”) and its gamification elements (e.g., “it was fun” and “the characters”). This sentiment was echoed by patients (e.g., “using VR is cool,” “liked helping the animals in surgery,” and “fun getting ready for my surgery”).
Examining qualitative data associated with the four lowest system usability scores (37.5, 42.5, 62.5, and 65) added insight for future improvements. Dislikes most consistently pertained to the fit and heaviness of the headset for seven of the 15 patients based on parent (e.g., “the headset was too heavy”) and patient comments (e.g., “the headset kept moving so it was blurry”); three of the seven patients were seven years old, the youngest patients in the study. Another hardware issue based on two parent comments was difficulties “manipulating the remotes [hand controllers]” and that “it was hard to do with one hand since her other arm was in a cast.” Dislikes about the headset or hand controllers matched two of the three patients with the lowest individual NPS scores (both 7). The parent of a 14-year-old patient expressed that Surgery Prep was “more for younger kids,” and the parent of a 12-year-old patient indicated that “she wanted to do more.” Similarly, an 11-year-old patient stated wishing they “could do more activities that are harder.”
Discussion
The results of this study demonstrated the feasibility and acceptability of a VR perioperative walkthrough for pediatric patients designed to help pediatric patients psychologically prepare for surgery. Psychological preparation is an increasing focus for VR application development that compliments the deployment of SnowWorld© and other VR games to reduce patients’ anxiety and pain during medical procedures through distraction (Li et al., 2011). It has been a natural progression for researchers to expand the scope of VR applications beyond games to include gamified software applications that aim to psychologically prepare patients for medical procedures and reduce anxiety through patient education (Shepherd et al., 2022). Study findings underscore the importance of gamification in designing patient education software for pediatric patients. Patients in our study liked that the VR walkthrough was like playing and commented positively on the animals and interactivity, similar to Gold et al. (2021). Parents also noted the experience to be fun and commented on the characters.
Most of the parents liked the gamified elements and the benefit of distraction., but there were also parent comments related to psychological preparation such as being able to see what it will look like in the OR, and therefore better supporting patients during the surgical experience. This suggests that the walkthrough had a degree of balance between gamification and patient education that could help pediatric patients psychologically prepare for surgery.
The study incorporated the Meta Quest 2 headset, relative to prior studies that used a VR headset that acts as a smartphone holder to position the smartphone in front of the patients’ eyes, such as the Samsung VR gear (Ryu et al., 2017; Tennant et al., 2020) or the Google Daydream View VR (Gold et al., 2021). The tradeoff was a more immersive experience for participants but also subjected them to a heavier headset relative to studies using a headset housing a smartphone. This was reflected in parent and patient comments of the headset being too heavy. A study by Eijlers et al. (2019) incorporated the HTC Vive, a headset slightly heavier than the Quest 2, with a patient population in a similar age range. It is not possible to tell if there were headset weight issues because the study did not report on hardware usability or tolerability. They only reported quantitative anxiety, pain, and behavioral measures. Survey questions in this study did not ask about tolerability either, which was a missed opportunity. However, the measurement of tolerability is typically anchored by standardized scales to measure cybersickness. The open-ended questions administered in this study allowed parents and patients to provide feedback on aspects of tolerability that may have been missed through standardized scales. This highlights the importance of including open-ended questions in VR studies to more comprehensively solicit feedback than pre-specified quantitative measures alone.
Despite the role of technology in this study, the human element mattered most as exhibited by the CCLS who informed the design of the VR walkthrough, similar to the role they play in developing videos and other informational resources (Chicas et al., 2023). A CCLS also implemented the study with patients and their parents. VR is a useful tool to create educational experiences that supplement information provided by CCLS and other clinicians, mainly information about the medical procedure itself. A CCLS provides emotional support and information that is more difficult for VR applications to impart on patients but also central to psychological preparation for surgery, like information about the sensations patients will feel during procedures and how to cope with procedures (National Clinical Guideline Centre, 2010).
Limitations
Study limitations center on factors that impede the generalizability of the findings for other hospitals. The CCLS who implemented this study was technologically savvy, had their own VR hardware, and was versed in its use with patients. In general, VR uptake is still low in clinical settings and health professionals have limited familiarity with VR applications (Kouijzer et al., 2023; Shiner et al., 2024). The feasibility of VR implementation may be lower in other hospitals relative to our study. Costs to develop and modify VR applications for different hospitals and procedures is another limitation. Surgery Prep developers used existing assets (e.g., an existing virtual operating room) with minimal modifications to reduce development costs, but development was still cost-prohibitive for typical child life budgets. Another limitation is that development was informed by a convenience sample of child stakeholders without surgical experience. This helped reduce the study timeline, but also reduced the utility of stakeholder input to optimize the VR experience for patients. The study sample was balanced on sex assigned at birth and the types of surgeries that pediatric patients are likely to be admitted for in other hospitals. However, the study sample was mostly Hispanic and not balanced on race and ethnicity. This was beneficial in setting a path for interventions in non-White populations but also introduced a potential language barrier. The study recruited English-speaking patients because Surgery Prep only displayed and verbalized narrative text in the English language. Findings may have been more favorable if we accommodated the Spanish language and cultural factors in the VR experience, something we will explore in future versions of the VR software. This serves as a reminder that it is important for VR offerings to reflect the cultural diversity of the intended patient population. Additionally, the age distribution of the sample did not include pediatric patients as young as five years old as originally intended. Otherwise, issues some patients had with the heaviness and fit of the VR hardware, as well as difficulties manipulating hand controllers, may have been exacerbated in younger pediatric patients. Three of the seven patients experiencing issues with the weight or fit of the headset were the youngest patients in the study. The content of the VR walkthrough may have had greater appeal for younger patients as suggested by comments, so it’s hard to know how the inclusion of younger patients would have impacted the findings on feasibility and acceptability.
Recommendations and Implications
This study builds upon prior VR studies that have demonstrated the feasibility and acceptability of VR applications in hospital settings with pediatric and adult patients. A novel aspect of the study relative to other VR studies that have been implemented by researchers is that it was implemented solely by a CCLS without researcher support. The CCLS downloaded the VR app onto their own Meta Quest VR hardware and conducted the study. This showcases the scalability of VR offerings through CCLSs in clinical settings, once infrastructure is in place. The expansion of VR hospital infrastructure is taking place through burgeoning hospital VR programs, like the Chariot Program (Caruso et al., 2020), and charities that donate VR equipment to hospitals, like the Child’s Play Charity and Starlight Children’s Foundation.
Next steps will be to further develop Surgery Prep and conduct randomized controlled trials to evaluate the efficacy of Surgery Prep for its intended purpose – a psychological preparation tool to help pediatric patients reduce pre-operative anxiety before surgery. Considering well-known factors that impact pre-operative anxiety, including patient age, previous medical encounters, and neurodiversity, an area for further Surgery Prep development will be to allow CCLSs to alter characters, environments, and activities to increase acceptability and tolerability across different pediatric patient populations. For example, visual, auditory, and olfactory sensory overload in the perioperative environment can be problematic for pediatric patients with autism spectrum disorder (Whippey et al., 2019). Autistic-friendly options that let users alter sounds and colors can reduce overstimulation (Dealessandri, 2023). Research is also needed to establish the optimal mix of gamification and patient education to reduce pre-operative anxiety. This study offered an interactive experience like Gold et al. (2021) in contrast to a passive virtual reality tour like Ryu et al. (2017). It also opted for a third-person experience centered on a non-human character like Gold et al. (2021) and Ryu et al. (2017) versus a first-person perspective like Eijlers et al. (2019). More studies are needed to alter these types of gamification elements and compare reductions in anxiety levels.
As VR technology advances, future offerings may supplement visual and auditory information with sensations patients will feel during procedures to more fully prepare them for upcoming procedures. For example, researchers are developing VR surgery simulators where neurosurgery trainees wear haptic gloves to create vibrations in their hands that mimic feelings of touching real surgical equipment in virtual space (Boutin et al., 2024). Haptic sensors could create procedural sensations for patients, like the placement of a pulse oximeter on a finger.
Conclusion
This study found that a VR app designed to help pediatric patients psychologically prepare for surgery was feasible for deployment by CCLSs in a hospital and acceptable by patients and their parents. Parent feedback suggested that the VR experience may serve two purposes, to psychologically prepare pediatric patients for surgery and distract them while they await surgery in a hospital. Both purposes may alleviate anxiety, which can be evaluated through randomized controlled trials.
Acknowledgment
We acknowledge Megan Pike, CCLS, at UCLA Mattel Children’s Hospital, the virtual reality developers at Saritasa, and children who informed the development of Surgery Prep alongside co-authors for this article. We acknowledge Sam Silverstein, Jeremy Paczos, and Laughter is the Cure for their contributions in providing the VR headsets to help put this project in motion. We also acknowledge the child life team, other clinicians and staff at the UCLA Santa Monica Medical Center who supported this study. We appreciate the pediatric patients and parents who participated.
Disclosures
Drs. Comulada and Wagner co-founded the company that created Surgery Prep. None of the other authors have anything to declare.