This study was a collaborative effort that included clinician researchers across multiple service delivery divisions, including IPC, developmental behavioral pediatrics (DBP), and general pediatrics. This study took place in a culturally and ethnically diverse primary care clinic at a Midwest academic medical center. Most families served within the clinic depend on publicly funded insurance and face many barriers to care (ie, financial, social, logistical, and cultural). The setting included the use of a patient portal that allowed families to receive notifications about appointments, access clinical care through video visits, schedule and reschedule appointments, send and receive messages to and from pediatric care providers, complete behavioral health questionnaires before their visits, request their medical record for mental health appointments, submit release of information forms for outside medical and school providers to talk with third parties (eg, schools and mental health clinics), and access resources (eg, patient education materials and community mental health resources). A qualitative research design was used involving focus groups to explore participants’ perceptions and experiences, allowing for in-depth discussions and identification of shared themes and insights. We used the Consolidated Criteria for Reporting Qualitative Research (COREQ) guidelines to ensure a rigorous qualitative methodology [24]. Data were coded, analyzed, and interpreted using the Consolidated Framework for Implementation Research (CFIR) [25] to provide a structured approach for identifying key factors influencing the implementation of the patient portal across both internal and external systems of care.

This study was overseen by the University of Michigan Institutional Review Board (HUM00215154) and was deemed “not regulated.” Caregivers provided verbal consent upon scheduling the focus groups. At this time, they were informed that this was an optional qualitative improvement project and that they could leave or withdraw from the study at any time. Researchers deidentified transcripts; the transcripts were housed in a secure, protected health information drive; and any documents containing protected health information were password protected. Caregivers were also notified that their information would be deidentified when the findings were published. Participants received a US $60 Visa gift card for taking part in this study.

The CFIR was used to categorize responses relevant to implementation in combination with grounded theory methods to allow novel themes to emerge. After each focus group, the research team met as a group and independently read the transcripts to discuss major themes that emerged from each relevant CFIR domain. A coding tree was developed within the CFIR domains, with 2 themes per domain. The research team consisted of 3 physicians (MD), 2 licensed psychologists (PhD), and 1 research assistant. Two of the physicians were developmental behavioral pediatricians, and 1 was a primary care pediatrician. The 2 PhD psychologists were integrated behavioral health psychologists. The research assistant was a PhD candidate in clinical psychology. All members of the research team identified as women, were from a diverse range of backgrounds (Asian, Black, Middle Eastern, and White), worked at an academic medical center, and had experience working on qualitative research within IPC. The participants knew that the researchers consisted of physicians and psychologists within an academic medical center. The focus group transcripts were read by all 6 researchers to create themes and identify example quotes. Two researchers (one of the physicians and one of the psychologists) returned to the transcripts to code for themes iteratively. Following each focus group discussion, we discussed the conceptual depth of themes and agreed through collaborative consensus. After completing 4 focus group interviews, it was decided that adequate conceptual depth and data saturation were achieved. Data were managed in Microsoft Excel. The outer setting domain involves how well the institution connects and communicates with outside referrals and agencies. For this study, many barriers were identified in the outer setting, including structural barriers to accessing the portal and portal resources. The inner setting domain details how well the institution connects and communicates within the organization. Many strengths were identified in the inner setting, including benefits and opportunities for portal use. The characteristics of individuals domain included aspects of caregivers’ own preferences or personality that aided or hindered portal use and their preferences for communicating with pediatric care providers through the portal. Many structural barriers existed for caregivers’ participation in this study, contributing to a small sample size. However, in the tradition of qualitative research, high information power was achieved through the narrow aim, dense specificity, use of an applied theory, strong dialogue, and case analysis [26].

One in-person and 3 virtual 90-minute focus groups were hosted involving 7 caregivers of children 2 to 7 years of age. One focus group only had 1 caregiver attend, but the other 3 had multiple caregivers in attendance. All 7 caregivers were female. The children receiving behavioral health services were of Black (n=3), White (n=3), and other (n=1) races. (see Table 1 for demographic information). Participants provided feedback to the research team about clinic changes that they would like to see.

Caregivers generally described positive experiences with the portal, which helped them better access their children’s health care and connect with their behavioral health care clinicians. Caregiver access and comfort with their children’s patient portal was revealed as the most influential variable to their behavioral health care experience. Caregivers who could access their children’s EHR portal shared that they liked the ability to schedule and see appointments, receive appointment reminders, and communicate with pediatric care providers. Conversely, parents who identified portal access as a barrier reported difficulties using technology and limited access due to their children’s age and/or their caregiver status. Overall, 6 themes were identified, which we categorized into the outer setting, inner setting, and characteristics of individuals domains of the CFIR (Table 2).

In the outer setting, themes identified were “lags and barriers to care” and “connections with outside medical and behavioral health providers.” Lags and barriers to care emerged as a concept that referred to systemic factors shaping caregiver access to care. Caregivers generally viewed the portal favorably to reduce the lags and barriers to care, such as allowing for easy scheduling and rescheduling of appointments, but some families were not provided with equitable access, especially those providing care for foster children. Some of the lags and barriers to care were mentioned 11 times during the focus groups and included long waits for the call center, no portal access for foster parents, and delays in care when parents needed to reschedule appointments. The lags and barriers to care were perceived as critical. For example, a caregiver stated the following:

As represented by the theme “connections with outside mental and behavioral health providers,” caregivers reported a strong desire for care coordination between pediatric care providers using the portal and included difficulties and lengthy intake processes with outside referrals as major barriers to care. Outside medical and behavioral health providers included Early On, applied behavior analysis therapy, behavioral therapy, community mental health, and other outside behavioral or developmental services. One caregiver wished there was a “universal” portal system:

Connections with outside medical and behavioral health providers were mentioned 6 times during the focus groups.

In the inner setting, the themes identified were “putting the ‘human’ in ‘human-centered design’” and “virtual access to information.” The “putting the ‘human’ in ‘human-centered design’” theme refers to the various ways in which caregivers can use the portal as a tool for medical information, accessing pediatric care providers, and coordinating their children’s care. This theme included statements about how the portal is an effective tool and ways in which it could be more tailored to meet families’ needs. One caregiver even noted the following:

The theme of putting the “human” in “human-centered design” was identified 8 times throughout the focus groups. When 1 caregiver had difficulty getting ahold of someone to help them log into the portal, they mentioned missing behavioral health care visits. Virtual access to information refers to the overarching ways in which families seek information online to inform medical decisions. It was identified 12 times during the groups and included how caregivers access information about their children’s health care, including information sent through the portal, such as websites with behavioral health information, and a desire for behavioral health providers to send behavioral health parent training videos. Without information provided through visits or the portal, parents stated that they would refer to social media.

Themes identified in the characteristics of individuals domain were “flexibility” and “personalization.” Flexibility was conceptualized as the desire to have a responsive medical system that suits each unique family system and was mentioned 6 times during the focus groups. Some parents preferred visits to be in person, whereas others preferred them to be virtual. One caregiver mentioned that it was very challenging to bring her child to an in-person behavioral health visit, whereas another caregiver mentioned that it was difficult to get her child to sit still for a virtual behavioral health visit. Personalization was conceptualized as the caregiver’s need for tailored and individualized care for their child and was mentioned 10 times during the focus groups. One caregiver noted the following:

Whatever method of communication (in person, virtual, phone, or portal message), parents want personalized care that fits their children and families. Caregivers seek flexibility and personalization in their care, and the portal can either help facilitate these goals or hinder them depending on how it is designed and used.

To our knowledge, this is the first study of its kind to use the CFIR to better understand caregiver-reported facilitators and barriers surrounding accessing and using patient care portals in the context of pediatric IPC. The purpose of this investigation was to gather information for the development of a streamlined behavioral health toolkit that will guide institutional policies and procedures surrounding the use of patient-facing EHR tools to improve patient care. The current study is the first to our knowledge to examine caregiver perspectives on the use of a patient EHR portal for pediatric IPC, finding that caregivers believe it is a useful tool but there continue to be access issues to the portal.

Overall, the caregivers included in this study viewed the patient care portal as a positive space to participate in care for their children, in particular, helping meet needs within the context of the health system (inner setting). Structural barriers and inequities in caregiver access to electronic behavioral health records were also reported across participants, particularly surrounding care coordination and the need for more tailored and individualized care.

Using the family voice in our study design allowed for the identification of human-centered design principles to inform potential improvements in the patient and caregiver experience. Caregiver-reported positive interactions with the portal fell primarily within the inner setting domain of the CFIR, which suggests that, at the individual patient level, the portal aids in effective communication between caregivers and pediatric care providers. Specifically, the patient care portal was helpful in aiding caregivers’ scheduling of appointments and their access to clinical care services such as video visits. Families also reported accessing health information on social media and through other open online sources, suggesting that future portal integration of health care information from trusted resources may be an area that families may find beneficial. Ultimately, families perceived the patient portal as an effective bridge to health care access. However, families often differed in their preference for virtual vs in-person care and communication of behavioral health needs.

A structural barrier commonly reported by participants was the problem of lengthy wait times to be able to schedule or reschedule behavioral health services when having to go through call centers. The patient portal, however, was perceived as something that helped increase access to care and reduce wait times. Prior work using time use diaries has found that low-income families are more likely to spend an additional 12 minutes per day waiting for services compared with their higher-income counterparts [26]. One potential mechanism might include a dearth of medical and behavioral health providers accepting Medicaid [27], which limits caregiver access to medical care overall. In this study, families indicated that the patient portal was one way to bypass long waits in the call center and that it was the more ideal way to reschedule care. Patient portals may be an important way to accommodate the time constraints of low-income families, whose employment may not have substantial flexibility to address scheduling and make phone calls during regular clinic hours.

Another area of potential future innovation that should be considered is developing capabilities to allow for communication across pediatric care providers and institutions. Families noted that systems-based communication between pediatric care providers and early intervention or community mental health was lacking, but a portal system that was universal and not institution specific could facilitate warm handoffs and referral monitoring between disparate systems. For example, the patient portal could display referral status across systems to create continuity and personalization of care, which families valued and desired.

Finally, an important barrier identified by caregivers was the lack of universal accessibility of the patient portal to foster parents. The number of children in foster care has increased from approximately 397,000 in 2012 to 442,000 in 2017 [28]. Children in foster care experience increased health problems compared to the general population and may require more extensive communication with their pediatric care providers [29]. For this population, there may be multiple encounters with the health care system in many settings and with many caregivers, leading to fragmented care [30] and complicating the communication of critical health information [31]. There is promise in leveraging the patient portal to create a more streamlined system to detangle health care delivery among children in foster care. Recognizing this gap in service delivery, Texas; Cincinnati, Ohio; and Allegheny County, Pennsylvania, have passed laws or created data systems that improve health care communication for children in foster care. Lessons learned from these regional efforts could be used to inform how health systems can incorporate more inclusive policies to allow foster parents to have portal access for services and features that can facilitate patient care while maintaining protection and privacy of patients in state custody.

Our study had several limitations. As a single-center study, our work may not be generalizable to other populations using different EHR portal systems or in other states with different regulations regarding foster care or privacy. In addition, data were qualitative and included a small number of participants. Patient perspectives may not reflect the views of other populations. Strengths of this study include the specificity of behavioral health information and inclusion of a low-income and racialized population, who experience greater barriers to health care and access. Future research should explore caregiver education on features and functions of the patient portal, as well as how policy changes can help caregivers better access behavioral health information, provide a more positive experience regarding scheduling appointments, and be more inclusive.

Using in-depth qualitative methods, we found that a patient portal is an effective and well-received tool for patients accessing behavioral health information. Despite positive perceptions, ways in which these tools can be personalized and reach beyond single institutions were identified as potential areas of improvement. It was also evident that disparities remain in health IT access that require systemic work to ensure that all populations have access to high-quality care, streamlined communication, and behavioral health information.