Acceptability, Feasibility, and Quality of Telehealth for Adolescent Health Care Delivery During the COVID-19 Pandemic: Cross-sectional Study of Patient and Family Experiences

Introduction

With the rapid shift to video visits during the COVID-19 pandemic, adolescents, who are typically digital natives, have been key consumers of technology-delivered health care [1]. Prior to COVID-19, telehealth was seen as a potential tool to increase access to care and reduce health disparities for adolescents, but geographic restrictions and limited reimbursement led to low utilization [2]. Widespread adoption of telehealth was facilitated by emergency waivers issued by the Centers for Medicare and Medicaid Services, which allowed for geographic flexibility and expanded reimbursement, and the proliferation of Health Insurance Portability and Accountability Act compliant videoconferencing platforms [3,4]. Most commercial insurers quickly followed in relaxing telehealth restrictions to keep pace.

With the rapid transition to telehealth during the COVID-19 pandemic, data gathering of end-user acceptability of telehealth has lagged. The crisis conditions of the pandemic resulted in minimal opportunity for stakeholder input and design from adolescents and young adults (AYA) and their families. Even prior to the pandemic, there were limited data on the acceptability of telehealth for adolescents, and existing studies were mostly confined to mental and sexual health care [5-7], thus neglecting other areas of adolescent health care delivery, including gender-affirming care and management of eating disorders. Although recent systematic reviews demonstrate acceptability of telehealth for a variety of pediatric and adult conditions and modest effect sizes for effectiveness for telemedicine management of pediatric conditions, including asthma, attention deficit hyperactivity disorder, and depression, the acceptability of video-delivered care for a broad sample of adolescent health conditions remains unknown [1,8].

Telehealth for adolescent care presents unique use case challenges. Adolescent Medicine service providers navigate additional confidentiality barriers, frequently need to integrate mental health care into visits, and often practice within interdisciplinary care teams, including psychologists, nutritionists, and social workers. Additional protections are needed to maintain confidentiality during adolescent enrollment within electronic health portals and telehealth applications while still allowing for parent and caregiver (caregiver) proxy access to essential health care information [9]. For example, caution is needed to assure that sensitive test results (such as pregnancy testing) are not released to parents through portals without adolescent consent and that confidential telehealth visits for sexual health services are not “visible” to parents. Early analyses have demonstrated successful adoption of telehealth, with high uptake rates for adolescent health care over periods of just days to weeks [9,10]. However, separate from adoption metrics, acceptability and feasibility assessments are essential to assure that telehealth is delivered with equivalent confidentiality protections to in-person care. The importance of confidential care has been amplified by the pandemic, given the rising rates of mental health conditions that necessitate additional privacy protections for both data collection and treatment delivery [11]. Early data from Adolescent Medicine providers demonstrate challenges to ensuring privacy and confidentiality, despite use of headphones, platform chat functions, and yes/no history-taking questions [12]. Providers have also noted that some patients from lower socioeconomic status households experienced greater difficulty securing private space owing to more crowded living arrangements, thereby presenting a potential challenge to equity [12].

Concerns surrounding widespread implementation of telehealth for adolescents remain, including threats to quality of care across the Institute of Medicine (IOM) dimensions of health care quality: safety, effectiveness, timeliness, efficiency, equity, and patient-centeredness [13,14]. Although telehealth has the potential to increase the reach of health care, early data show that it may paradoxically worsen health disparities owing to differential access to wireless internet, private spaces for visits, and mobile devices across race and socioeconomic status [15,16]. An additional area for concern is patient safety, as the lack of hands-on physical examinations and standardized collection of vital signs could lead to errors in diagnosis [17].

The perspectives of both patients and caregivers are critical for assessing the acceptability of telehealth for adolescents. The American Academy of Pediatrics Supporting Pediatric Research in Outcomes and Utilization of Telehealth (SPROUT) research network developed the SPROUT Telehealth Evaluation and Measurement (STEM) framework, a mechanism to evaluate perspectives on telehealth across stakeholders [18]. The Experience branch of the STEM framework emphasizes the need to understand patient and caregiver perspectives on multiple visit aspects, including overall satisfaction, communication quality, and impact on family routines [18]. We therefore sought to examine patient and caregiver attitudes toward telehealth in an Adolescent Medicine subspecialty clinic system. Our primary aim was to determine the acceptability, feasibility, and quality of telehealth for delivery of adolescent health care among patients and caregivers. A secondary aim sought to evaluate the agreement between patient and caregiver responses on acceptability measures.

Methods

We conducted a cross-sectional web-based survey to assess attitudes toward telehealth in AYA and parents and caregivers (caregivers).

Settings and Participants

Participants or their dependents received care within an Adolescent Medicine subspecialty clinic, within a large academic pediatric hospital network in the Philadelphia area. The clinic provides contraceptive and gynecologic services, gender-affirming care, HIV treatment and prevention, and management of eating disorders for AYA. The clinic transitioned from 100% in-person visits to majority synchronous video visits starting March 2020 owing to the COVID-19 pandemic [10]. The telehealth platform allowed for a multiple user interface, and visits were attended by multiple clinical team members, including registered dieticians, social workers, psychologists, and interpreters as needed. Patients aged ≥13 years who completed a video visit from May-July 2020 were eligible for enrollment. Caregivers were eligible if their child <18 years of age completed a video visit during the study period or if they accompanied their child 18 years or older in a video visit (ie, the patient did not attend the visit independently). Patients and caregivers could participate independent of each other, and the data therefore do not represent patient/caregiver dyads. Potential participants were called before their telehealth visit by study staff, had contact information confirmed, and were informed about the survey. After visits were completed, links to research electronic data capture–based surveys were sent via text message or email to the participant and, separately, their caregiver, per inclusion criteria.

Measures

The 32-item (AYA) and 29-item (caregiver) web-based surveys assessed telehealth acceptability and feasibility. Survey items were adapted from previously validated scales, and items were selected using a modified Delphi procedure with experts from Adolescent Medicine, psychology, and informatics [19]. Telehealth acceptability was measured on a 5-point Likert scale comparing telehealth to in-person care with respect to provider-patient communication, convenience, privacy, and achieving goals of care. Feasibility was assessed via questions regarding technical difficulties with visits. Additional independent measures were included for AYA and caregivers, respectively. AYA surveys included items addressing ability to find a private space for the visit and whether there were opportunities to speak with their provider alone. Caregivers provided both their own and their child’s demographic information and completed an additional question on their perceptions of how well their child’s concerns were addressed at telehealth visits compared to in-person visits. In order to capture additional perspectives on telehealth that may not have been captured in our measures, both surveys contained 3 open-ended questions: (1) what are the disadvantages of telehealth compared to in-person visits? (2) what are the advantages of telehealth compared to in-person visits? and (3) please let us know any additional areas in which you felt telehealth was different from in-person visits.

Quantitative Analysis

Demographic characteristics of patients and caregivers were assessed via descriptive statistics, including means, medians, and standard deviations. For items comparing in-person to telehealth visits, we assessed noninferiority of telehealth to in-person care by dichotomizing responses into 2 categories: telehealth better or the same as in-person and telehealth worse than in-person. As our primary aim was to assess acceptability for both caregivers and adolescents and to identify areas for optimization to assure joint acceptability, we compared proportions of each population rating telehealth as noninferior to in-person care by using chi-squared and Fisher exact test. All analyses were completed in Stata 15 (College Station, TX, StataCorp LP).

Qualitative Analysis

Three independent coders qualitatively analyzed responses to the open-ended questions in the survey regarding telehealth advantages and disadvantages. The primary (AWP) and secondary (PM, HLF) coders reviewed the open-text survey responses by using a semiquantitative spreadsheet approach, which captured the descriptions and frequencies of themes. The patient and, separately, caregiver-specific responses were independently double-coded using deductive thematic analysis to identify themes unique to the patient and caregiver experiences. The coding team developed an initial codebook of themes based on consensus with each coder and then they separately applied the codebook themes to the entirety of the open-text survey data. Any coding discrepancies were resolved by consensus. To ground findings within an existing health care quality framework, the primary coder categorized the final themes according to the IOM dimensions of health care quality: safety, effectiveness, timeliness, efficiency, equity, and patient-centeredness [13,14]. All procedures were reviewed and deemed by the Institutional Review Board to be exempt as quality improvement.

Results

Quantitative Data

In May-June 2020, 268 and 563 surveys were deployed to unique AYA patients and caregivers, respectively, with a response rate of 20.5% (55/268) for AYA and 21.8% (123/563) for caregivers. The majority of the patient and caregiver respondents were White cisgender females (Table 1). The race and sex distributions of patient survey respondents were representative of the patient population seen by the clinic during spring 2019. The most common visit reasons were eating disorders (18/55, 33% patients, 52/123, 42.3% caregivers) and gynecology/reproductive health (18/55, 33% patients, 44/123, 35.8% caregivers).

The majority of the visits were conducted by physician providers (Table 2). Most AYA and caregivers used a smartphone with a Wi-Fi connection for their telehealth visit. With respect to confidentiality, nearly all AYA (54/55, 98%) were able to identify a private space for their visit (Table 2) and 36 out of 55 AYA (65%) spoke to a provider alone during their telehealth visit (Table 2). Of the 19 AYA who did not speak to their provider alone, 3 (16%) wanted to do so (Table 2).

With regards to acceptability (Table 3), the majority of AYA and caregivers rated telehealth as noninferior to in-person visits with respect to privacy, communication, managing medication questions, and discussing test results, mood, and mental health. A significantly higher proportion of AYA compared to caregivers felt telehealth was inferior to in-person care with respect to privacy (11/51, 22% vs 3/118, 2.5%, respectively, P<.001). There were no other significant differences between AYA and caregivers in the acceptability ratings across domains.

With respect to feasibility, 39 out of 123 (31.7%) caregivers and 14 out of 55 AYA (25%) reported technical difficulties with telehealth, including difficulty accessing the patient portal. However, 104 out 123 caregivers (84.5%) and 49 out of 55 AYA (89%) reported that the technology system was easy to use, and 97 out of 123 caregivers (78.8%) and 38 out of 55 AYA (69%) reported that video visits improved efficiency of care, including time saved, compared to in-person visits (Table 2).

Qualitative Data

Nearly half (n=26) of the 55 patients (47%) and 86 of the 123 caregivers (69.9%) completed the open-ended questions. The demographics of the patient and caregiver responses to the open-ended questions were reflective of the total survey population. The sample was largely White (19/26, 73% AYA; 75/86, 87% of caregivers) cisgender females (19/26, 73% AYA; 75/86, 87% of caregivers). Emergent themes within the IOM quality framework and exemplar quotes are shown in Table 4.

The most frequently cited advantage of telehealth compared to in-person visits was within the IOM dimension of Timeliness. Both patients and caregivers indicated that time was saved from no commute or in-person waiting room time and reported financial savings from less work missed and no transportation costs. The second most common theme was “Improved access to care for vulnerable populations” within the Equity IOM domain. Patients and caregivers described telehealth as expanding access to people who may experience a variety of challenges with attending in-person Adolescent Medicine visits. Patients also discussed how telehealth improved equity in care delivery, including, but not limited to, reducing misgendering patients by clinic staff.

With respect to the disadvantages of telehealth, the most common theme was “Limitations in scope of practice” within the Effectiveness IOM domain. Patients and caregivers discussed that the lack of hands-on physical examination and laboratory testing, which were felt to be essential for the delivery of evidence-based care, could lead to decreased quality of care. Patients and caregivers also frequently endorsed challenges to patient-centeredness, particularly in communication and building rapport. With respect to Equity, one caregiver described the financial burden of telehealth owing to the challenges with a limited scope of practice, where caregivers may be required to pay for separately for both a telehealth visit and an in-person laboratory visit to meet the health needs of their child.

Caregiver responses differed qualitatively from patient open-ended responses in 2 ways. First, caregivers placed greater emphasis on the importance of preexisting provider-patient relationships in successfully creating a comfortable visit environment via telehealth. Second, caregivers more commonly framed telehealth as advantageous with respect to patient-centeredness, including comfort, provider-patient communication, and engagement in visits.

Discussion

Principal Findings

Within an Adolescent Medicine clinic, we found high acceptability of telehealth among both patients and caregivers. The majority of the patients and caregivers reported that telehealth visits were easy to use and saved time and they expressed willingness to participate in another telehealth visit. Key areas for optimization in telehealth implementation included improving technical problems, which may limit uptake, and ensuring adequate confidentiality standards for AYA in the video visit setting. Although >85% of respondents found the telehealth system easy to use, a quarter of the patients and nearly a third of caregivers reported experiencing at least one technical issue during their telehealth visit. The most common issue across both groups was malfunctioning of the audio component in the video visit. The analysis of telehealth satisfaction among pediatric neurology service providers during the COVID-19 pandemic similarly revealed high levels of satisfaction, despite nearly 40% encountering technical challenges, and the providers surveyed also reported that audio problems were the most common [16]. In order to optimize telehealth quality, it will be essential to resolve technical issues impacting communication of clinical information and treatment recommendations. As technology continues to rapidly evolve, health systems likely need to “go back to the drawing board” to conduct more extensive usability testing on their systems. The user-centered design process is typically part of scale-up of new mobile health interventions but was bypassed owing to the urgency of the pandemic. Periods of respite between COVID surges may provide an opportunity to refine the user experience. Lastly, as health systems optimize their technology, consideration should be given to integrating remote patient monitoring options such as heart rate monitors, actigraphy, and pulse oximetry to augment video history and examination findings [20-22]. These digital tools also hold promise as health-promoting interventions in their own right. Remote patient monitoring strategies with real-time patient feedback may improve disease self-management and treatment adherence in conditions such as asthma and diabetes for adolescents.

Privacy was the only acceptability measure in which we found divergence between caregivers and adolescents. A significantly higher proportion of patients rated telehealth as inferior to in-person care for privacy. This finding suggests that AYA perceptions of visit privacy may be more complex than the simple ability to identify a private space for the visit, which >98% of patients were able to do. Prior research efforts with Adolescent Medicine providers have identified several strategies for optimizing privacy and confidentiality during telehealth visits. For at-home visits in which patients have access to adequate technology and space for the visit, these include the use of headphones, yes/no history-taking questions, use of chat functions, and using background white noise to lessen the chance that others in the household will overhear [12,23]. In efforts to improve telehealth privacy, special attention should be paid to adolescents who lack stable housing, private space, or consistent access to technology. These include creating dedicated patient telehealth “drop-in” kiosks stocked with computers or tablets and soundproof space at essential locations that may remain open in a public health crisis, such as pharmacies, primary care clinics, or schools. In addition, models from the Veterans Administration have demonstrated that delivery of tablets to unstably housed individuals is a feasible strategy for maintaining access to telehealth for vulnerable populations [12,15,23-25].

The high acceptability and convenience of telehealth reported by AYA patients and caregivers point to potential benefits of integrating telehealth visits in adolescent care in the future years. However, the future of telehealth in the United States remains uncertain. In April 2021 and July 2021, the US Department of Health and Human Services renewed the declaration of the COVID-19 pandemic as a public health emergency for an additional 90 days [26]. Under this renewal, the blanket waivers issued by the Centers for Medicare and Medicaid Services to increase geographic flexibility and expanded reimbursement remained in effect [3]. In the absence of a further renewal, however, many of these waivers may no longer apply, making telehealth far less feasible. Some commercial insurers began withdrawing additional provisions, allowing for expanded telehealth reimbursement in fall 2020, with more following in winter and spring 2021. Given broad state discretion, telehealth policy for Medicaid and Children’s Health Insurance Policy is also in flux across states. High acceptability of telehealth suggests that the integration of telehealth as an additional care delivery mode may be highly beneficial. In addition, given the increasing rates of adolescent mental health diagnoses, suicidal ideation, and suicide attempts during the pandemic [27-29], telemedicine will be an essential means of delivering evidence-based mental health care to youth, given the dearth of available in-person services [1]. Whether our health system can rise to this challenge will depend on the continuation of policies that, by lessening geographic restrictions and achieving parity with in-person visit reimbursement rates, enabled widespread telehealth use.

Our analysis has several limitations. The survey response rate was low, and therefore, may not provide a complete picture of patient and caregiver experiences with telehealth. Surveys were sent to patients attending visits during May-June 2020, when COVID-19 cases were rapidly rising in the United States. Many patients and caregivers were experiencing abrupt changes to their routines and additional stressors during these months, which may have limited the response rate. However, the response rate of the patients compared to that of the caregivers was approximately the same, and the patient race and sex demographic distribution did not differ significantly from patients seen in the clinic for in-person visits during spring 2019. The majority of the respondents were White, non-Hispanic, cisgender females, and therefore, our results may not be generalizable to other populations. Previous analyses of telehealth during COVID-19 in both pediatric and adult populations have demonstrated racial and socioeconomic disparities in telehealth utilization, with non-White patients, Latinx patients, and patients with low median household incomes having both lower overall utilization and utilizing audio only visits more often than audio plus video [16,30-32]. These studies provide an early signal that rapid introduction of telehealth, in many instances, has led to the unintended consequence of widening the equity gap in health care delivery. Our telehealth platform was designed to allow multiple users, including interpreters, to attend visits. This multiuser interface may not be generalizable to less-resourced health systems, and thus, attention should be paid in future research to capture the experiences of populations with limited English proficiency in a diversity of health systems. Understanding and addressing emerging health disparities and evaluating telehealth acceptability among marginalized groups will be crucial in any future implementation of telehealth.

Conclusions

Widespread telehealth adoption in response to the COVID-19 pandemic altered health care delivery during 2020 and 2021. We demonstrate high acceptability of telehealth by AYA and caregivers of AYA, a population for which very little was previously known about the acceptability and feasibility of the use of telehealth. Our data support the importance of maintaining reimbursements for telehealth as a strategy for adolescent health care delivery. Future research addressing telehealth in adolescents should focus on ensuring equity, optimizing the end-user experience, and improving confidentiality protections.