Failure to recruit a sufficient number of participants is a common barrier to the successful and timely completion of research studies [1]. Insufficient accrual of participants may require additional resources to achieve target enrollment, and failure to meet enrollment goals may result in underpowered studies [2,3].

Electronic patient portals are web-based applications owned and administered by health care institutions that allow patients to access their electronic health records (EHRs). In the past two decades, a growing number of patients have used this technology to manage their health care and communicate with their providers [4-6]. Estimates of patient portal use vary by study, subpopulation, and measured outcome, but reports range from 25.8% to 84.1%, and longitudinal analyses indicate that utilization rates are growing [7-16]. In a national US sample, Turner et al [17] found that 24.9% of participants reported using at least one patient portal tool in 2017, compared to only 12.6% in 2011. As adoption becomes widely accepted, researchers have recognized the opportunity to use EHR data to identify eligible research cohorts and send recruitment invitations to potential participants via the patient portal [18,19]. Direct messaging through patient portals enables a study to efficiently contact eligible patients and facilitates low-touch, low-cost outreach to large numbers of patients, an approach that is particularly advantageous for studies with large target sample sizes. In addition, once a system of electronic recruitment is established, the process of identification, prescreening, and outreach can be automated and repeated.

Studies using patient portal research invitations for recruitment report a wide range of study enrollment rates, with 1.8% to 24.7% of those who received an invitation consenting to participate [20-27]. A summary of 14 studies that sent research recruitment invitations through the patient portal at a single medical center found that condition-specific studies had higher response and enrollment rates compared to general health studies [23]. The ADAPTABLE (Aspirin Dosing: A Patient-Centric Trial Assessing Benefits and Long Term Effectiveness) study, a pragmatic trial with a recruitment target of 15,000 participants, reported enrollment rates from four different modes of outreach: patient portal, email, mailed letter, and in-person communication with the research coordinator. Although in-person recruitment had the highest enrollment rates of all the modes, patient portal, and email outreach yielded the most overall study enrollment because they allowed the study team to approach many more potential participants than did the other modes [24]. A recent study comparing in-person, email, and patient portal recruitment of adults from primary care and bariatric clinics also found that electronic forms of outreach resulted in the most overall study participants in spite of lower recruitment efficiency compared to in-person recruitment [27].

Despite the advantages of using patient portals to recruit for research, they remain primarily clinical tools, and using them to send research invitations may run the risk of decreasing patients’ trust in the health care system or utilization of the platform for clinical purposes. However, there is some evidence that patients find recruitment through patient portals acceptable. Plante et al [25] reported only 2 complaints and 1 request to unsubscribe from future messages in a study that sent 6896 invitations, and Gleason et al [28] noted that most patients reported research recruitment to be an acceptable use of patient portals in a satisfaction survey from a study that sent 1303 invitations. Patients who find patient portal recruitment to be unacceptable, however, may not open a message, send a complaint, or complete a satisfaction survey. Thus an in-depth understanding of factors influencing acceptability of patient portal recruitment remains to be determined.

Demographic disparities between patient portal users and nonusers present a major barrier to the recruitment of a representative study sample. Studies have shown that patient portal nonusers are more likely to be racial/ethnic minorities, older, male, low socioeconomic status, low health literacy, and live in a rural area [8,23,29-31]. Patient portal recruitment may, however, decrease study population disparities that result from certain demographic groups being approached for research participation less frequently in clinic settings [32,33]. There is some evidence that clinicians as gatekeepers may contribute to the under-representation of certain populations, particularly among patients with minority backgrounds [33]. Mass electronic invitations may be a universal recruitment outreach approach reflective of demographic variation. Some studies have recommended that patient portal recruitment be one part of a comprehensive outreach approach, including approaches that specifically target traditionally under-represented groups [34].

In this article, we describe our use of invitations sent through the Epic EHR and patient portal (MyChart) within UNC Health (UNCH). At UNCH, the patient portal is branded my UNC Chart. We used my UNC Chart to recruit for Early Check, a research study offering screening to all newborns in the state of North Carolina for a panel of genetic conditions. With a target recruitment rate of over 10,000 newborns per year, an online consent process that does not require contact with a research coordinator, and broad eligibility criteria, Early Check is a study for which recruitment messaging through patient portals is a good fit. Additionally, the target populations for recruitment outreach are pregnant women and mothers of newborns; this group is relatively younger and female, both groups which have been shown to be more likely to open a patient portal account and use patient portals to manage their health [15]. Two recent studies reported a rate of patient portal utilization between 34% and 72% of pregnant patients [21,35]. One study recruited pregnant women to a research study through a patient portal and found 34% of pregnant patients used their patient portal, and when invited to their study, 11% consented and completed their questionnaire [21].

Since Early Check began recruitment in October 2018, the primary outreach method has been personalized direct mail letters and emails on letterhead from the North Carolina Department of Health and Human Services, a study partner, sent postnatally to all women with a listed mailing or email address in the North Carolina newborn screening records. A social media outreach campaign was piloted from March to September 2019. We resumed social media advertising on Facebook and Instagram on April 1, 2020. An evaluation of the direct mail outreach impact on study enrollment showed that approximately 4% of all women who were sent a recruitment letter enrolled their newborn in the study, and the enrollment rate among women who also received a recruitment email was approximately 5% [36]. An analysis of the social media campaign from March 2019 to September 2019 showed that paid ads on social media resulted in 3.5 additional daily enrollments in the study for each day ads were run [37]. To further increase outreach to eligible participants, we used my UNC Chart to send recruitment invitations to pregnant women whose newborns would be eligible for Early Check.

In this article, we describe the use of a patient portal to recruit research participants for Early Check and report on characteristics of mothers who received and opened a research recruitment message and enrolled their newborns in the study. We addressed two research questions:

To address these questions, we examined data on 4510 UNCH patients who were invited to participate in Early Check through my UNC Chart between November 22, 2019, and March 5, 2020.

A collaboration between RTI International, the University of North Carolina at Chapel Hill, Wake Forest School of Medicine, Duke University, and the North Carolina State Laboratory of Public Health, Early Check is a large research study offering screening for a panel of conditions to all newborns in the state of North Carolina [38]. The panel includes fragile X syndrome (October 2018-current), spinal muscular atrophy (October 2018-March 2021), and Duchenne and related muscular dystrophies (November 2020-current). Newborns are eligible if they have a newborn screening in North Carolina and live in North Carolina or South Carolina. Newborns may be enrolled in the study by their mother or legally authorized representative in the event the mother is unavailable, between the start of the mother’s second trimester and when the newborn is one month old. During the phase of the study described herein, permission for the newborn to participate was completed entirely online without direct engagement with a research recruiter [39]. The Office of Human Research Ethics at the University of North Carolina at Chapel Hill serves as the central Institutional Review Board for Early Check (#18-0009), and they approved these activities.

The process of identifying women within UNCH to be sent an invitation to participate in Early Check via my UNC Chart began with a computable phenotype, a data query that “use[s] EHR data exclusively to describe clinical characteristics, events, and service patterns for a specific patient population [40].” UNCH’s enterprise data warehouse, the Carolina Data Warehouse for Health (CDWH), was queried using the computable phenotype to identify invitation recipients. UNCH’s appointment discharge paperwork provides patients with a unique ID that can be used to activate their my UNC Chart account. Women were invited if they had ever activated their my UNC Chart account, regardless of how recently they had logged into the account. The primary criteria in the computable phenotype were: a) having an active pregnancy “episode of care,” and b) being in the second or third trimester of pregnancy (ie, 13-42 weeks’ gestation).

In Epic@UNC, a pregnancy Episode of Care groups all prenatal encounters and diagnoses for a pregnancy. A pregnancy Episode of Care can be generated at any time during the pregnancy, although ideally, it is generated at the time a pregnancy is first confirmed or at the time a pregnant woman transfers her care from another health care system. Pregnancy Episodes of Care are designed to automatically resolve after delivery, specifically after any of the following: a) 48 weeks with no linked encounter; b) 364 days after the episode creation date; c) 84 days after the estimated delivery date in the patient’s medical record. An Episode of Care may also be manually resolved after the baby is born. Pregnancy Episodes of Care are not designed to resolve automatically when a woman loses her pregnancy; the Episode of Care must be manually resolved (eg, closed out) in the case of pregnancy loss.

Since a cohort based on active pregnancy Episodes of Care may unintentionally include some women who have lost their pregnancy, women were excluded from receiving an invitation if their health record showed any of a series of International Classification of Diseases 10th revision or current procedural terminology codes associated with elective or spontaneous abortion within 10 months of the date that the CDWH was queried. Women were also excluded if they had indicated in their communication preferences that UNCH was not permitted to contact them through my UNC Chart. Women were only sent one invitation, so they were also excluded from the cohort if they had already been sent an invitation for the study during the same pregnancy Episode of Care. The computable phenotype with inclusionary and exclusionary codes and a figure showing the text of the research invitation can be found in the Multimedia Appendices 1 and 2.

We consolidated data from four sources: (1) UNCH patient records with my UNC Chart invitation data; (2) ZIP code-level rural-urban commuting area (RUCA) approximation codes, from which we derived dichotomous urbanicity status; (3) newborn screening records gathered from the North Carolina State Laboratory of Public Health, to which Early Check mailing list data have been appended; and (4) enrollment information collected through the Early Check permission portal. After cleaning and standardizing the data, we iteratively matched records from my UNC Chart to the newborn screening and Early Check permission portal datasets using several combinations of data fields appearing in two or more sources, including phone number, email address, name, date of birth, and street address (including ZIP codes). We visually inspected the combined data set after each pass to find and update mismatched records or duplicates. The final data set used in the analysis contained one record per patient with variables derived from all four sources.

The main analyses presented in this report focus on all 4510 women living in North Carolina or South Carolina who had not yet enrolled in Early Check and were sent invitations via my UNC Chart from November 22, 2019, through March 5, 2020. To standardize results from batches of invitations sent on different dates, we set a 27-day window for recruitment and enrollment outcomes starting from the date participants were sent a my UNC Chart invitation. We selected a 27-day window to avoid any overlap with a social media ad campaign for Early Check that began April 1, 2020. We also compiled aggregate data for women with patient records in the UNCH system who did not have an active my UNC Chart account but would have otherwise met the eligibility criteria for an invitation. We used this aggregate data to estimate the proportion of Early Check-eligible UNCH patients who were reachable through my UNC Chart and to examine whether there are differences by age, race and ethnicity, or urbanicity between women who received an invitation and those who did not.

We conducted logistic regressions to test for differences in whether women enrolled their newborns in the study or opened the my UNC Chart invitation by outreach methods, urbanicity, race and ethnicity, and age. The model estimating enrollment included whether women opened the invitation as a predictor variable; all other regressors were the same in both models. Cases with missing values on one or more regressors were excluded by listwise deletion in the logistic regression models. In addition to reporting model estimates, we also present the predicted probabilities for significant categorical variables, which represent the rates of enrollment and invitation-opening within levels of those variables while controlling for other regressors in the models. To examine potential differences by urbanicity, race and ethnicity, and age between women who were sent invitations through my UNC Chart and patients in the UNCH system who were otherwise eligible but did not have an active my UNC Chart account, we conducted χ² tests of independence. We followed up significant χ² tests involving independent variables with more than 2 categories using two-sample z tests for the difference of proportions. For these pairwise comparisons, we used a Bonferroni-adjusted alpha level of .0085. We conducted all analyses using Stata Statistical Software (version16.0; StataCorp).

In total, 12,036 patients within the UNCH system fit the computable phenotype that would have made them eligible to receive an invitation from November 22, 2019, through March 5, 2020, but only 4510 out of 12,036 (37.5%) had an active my UNC Chart patient portal account. We compared the demographic characteristics of women to whom we sent a my UNC Chart invitation to those who did not receive an invitation because they did not have an active account. We found no significant differences by age (χ² [4, N=12,036]=3.51; P=.48) or urbanicity (χ²[2, N=12,036]=0.37; P=.54), but we did find differences by race/ethnicity (χ²[3, N = 12,036] = 180.99; P<.001, Cramér’s V = 0.12). A greater percentage of non-Hispanic White patients (2527/5852, 43.2%) had an active my UNC Chart account compared to non-Hispanic Black patients (932/2738, 34.0%, z=8.05; P<.001), Hispanic patients (531/1,916, 27.7%, z=12.03; P<.001), or non-Hispanic patients of any other race (520/1530, 34.0%, z=6.50; P<.001). Hispanic patients were significantly less likely to have an active my UNC Chart account than patients in any of the other three race/ethnicity groups. The full contingency table comparing whether women had an active my UNC Chart account by race and ethnicity is shown in Table 1.

Characteristics of North Carolina or South Carolina residents who were sent an invitation to participate in Early Check from November 22, 2019, through March 5, 2020, are presented in Table 2. The sample excludes 18 patients who had already enrolled their newborns in Early Check before their my UNC Chart invitation was sent. Invitations were sent in five batches, with approximately half of the recipients (2466/4510, 54.7%) included in the first mailing on November 22, 2019. Two-thirds of all recipients (3054/4510, 67.7%) logged into their my UNC Chart accounts and opened the invitation within 27 days of when it was sent. Among women who opened the invitation, it took them 2.45 days (SD 4.83) on average to do so; however, the distribution is positively skewed, with 68.6% (2094/4510) opening it within 24 hours. Relatively few women (357/4510, 7.9%) were sent a postnatal recruitment letter or a personalized email (24/4510, 0.5%) within 27 days of being sent the my UNC Chart invitation. This observation is not unexpected, given that the computable phenotype was designed to target women with an active pregnancy as early as the 13th week of gestation.

In all, 3.6% (161/4510) of women who received a my UNC Chart invitation enrolled their newborns in the study within 27 days. Excluding 8 women who enrolled their newborns in Early Check without opening the invitation, women took on average 3.92 days (SD 6.50) to enroll. Similar to the distribution for the time it took to open the invitation, the enrollment timing distribution was positively skewed, with 63.4% (97/4510) enrolling within 1 day of opening the invitation.

Our first research question examined whether and to what extent women who opened a research invitation sent to them through my UNC Chart were more likely to enroll in Early Check within 27 days of receiving the invitation. The overall logistic regression model predicting enrollment was significant (χ² [11, N=4507]=134.90; P<.001, R²_McFadden=.10). As shown in Table 3, the odds of enrolling among women who opened the invitation was nearly 9 times the odds of enrolling among women who did not open and who therefore did not view their invitation (SE 3.24, OR 8.86, 95% CI 4.33-18.13; P<.001). Expressed in terms of predicted probabilities holding everything else in the model constant, 4.88% of women who opened the invitation (SE 0.38 , 95% CI 4.13%-5.63%) enrolled their newborns in Early Check within 27 days of when it was sent compared to only 0.58% of women who did not open the invitation within that time frame (SE 0.02, 95% CI 0.18%-0.99%) and most likely became aware of the study through another outreach method Being sent a postnatal recruitment letter (P=.57 or a personalized email invitation (P=.53) did not have a significant additional impact on enrollment within the 27-day period.

Our second research question asked, in part, whether there are differences in enrollment by race/ethnicity, age, or urbanicity. Although we observed no significant differences in enrollment rates across age groups, race/ethnicity and urbanicity were both related to enrollment. The odds of enrolling for Black women who were sent a my UNC Chart invitation was 0.29 times the odds of White women (SE 0.09, OR 0.29, 95% CI 0.16-0.55; P<.001). Expressed in terms of predicted probabilities, whereas 4.49% of White women (SE 0.40, 95% CI 3.72%-5.27%) enrolled their newborns in Early Check within 27 days of when their invitations were sent, only 1.38% of Black women enrolled their newborns (SE 0.41, 95% CI 0.57%-2.19%). We found no other differences in enrollment by other race/ethnicity groups. Additionally, women with a home address in urban zip codes were more likely to enroll than women from rural zip codes (SE 0.97, OR 3.03, 95% CI 1.62-5.67; P=.001). Controlling for the other variables in the model and expressed in terms of predicted probabilities, 4.04% of urban women (SE 0.32, 95% CI 3.41%-4.67%) enrolled their newborns in Early Check compared to 1.40% of rural women (SE 0.42, 95% CI 0.57%-2.22%).

Our second research question also considers associations of race/ethnicity, age, and urbanicity on whether women opened the research invitation sent to them through my UNC Chart. As shown in Table 4, the logistic regression model that predicts opening the invitation was significant (χ²[10, N=4507]=62.38; P<.001, R²_McFadden=.01). Women who were sent a postnatal recruitment letter by mail within 27 days of a my UNC Chart invitation were significantly less likely to open the invitation (SE 0.09, OR 0.76, 95% CI 0.60-0.96; P=.02). Holding everything else constant and expressed in terms of predicted probabilities, 62.1% of women who were sent a postnatal recruitment letter opened their my UNC Chart invitations (SE 2.6, 95% CI 56.9%-67.2%) versus 68.2% of women who were not sent a recruitment letter (SE 0.7, 95% CI 66.8%-69.6%). Whether women were sent a postnatal personalized email within the 27-day timeframe was not significantly associated with opening the my UNC Chart invitation (P=.19), nor was urbanicity (P=.75). However, race/ethnicity and age were both significantly related to opening the invitation. Black women were significantly less likely than White women to open the invitation (SE 0.05, OR 0.67, 95% CI 0.57-0.78; P<.001), with 61.4% of Black women (SE 1.6, 95% CI 58.3%-64.5%) opening the invitation compared to 70.4% of White women (SE 0.9, 95% CI 68.6%-72.2%). Hispanic women were also less likely to open the invitation than were White women (SE 0.07, OR 0.73, 95% CI 0.60-0.89; P=.002), with an estimated 63.4% of Hispanic women opening their invitations. Lastly, opening the my UNC Chart invitation differed significantly by age. Compared to women under 20 years of age, women aged 25 to 29 years (SE 0.26, OR 1.51, 95% CI 1.08-2.10; P=.02), 30 to 34 years (SE 0.28, OR 1.67, 95% CI 1.20-2.33; P=.003), or 35 years or more (SE 0.25, OR 1.44, 95% CI 1.03-2.03; P=.35) were significantly more likely to open the invitation. The predicted probabilities of opening the my UNC Chart invitation by age are shown in Table 5.

We examined the utility of sending research invitations to pregnant women through a patient portal and whether opening an invitation was associated with enrollment in the study. We found an association between opening a patient portal research invitation and enrollment in the study but found disparities by race/ethnicity in having a my UNC Chart patient portal, opening the invitation, and enrolling in the study.

The use of EHR data to identify and contact eligible participants through their patient portal proved to be successful. The findings show that the my UNC Chart patient portal within UNCH could be used to send recruitment invitations to over 4500 pregnant women whose newborns would be eligible for Early Check over a period of approximately 15 weeks. These results demonstrate the efficiency of using patient portals to send recruitment invitations to large numbers of potential research participants, compared to the time and effort it would require to contact thousands of participants through other methods like phone or in-person recruitment. As such, patient portals are especially valuable for studies seeking to approach and enroll very large numbers of participants.

Despite contacting thousands of eligible women, those we contacted accounted for a minority (4510/12.036, 37.5%) of patients at UNCH who met the computable phenotype during the time period of this study; a majority of eligible women did not have active my UNC Chart accounts and thus could not receive a recruitment message. However, for a study like Early Check with broad eligibility criteria and for which patients will become newly eligible over time as women become pregnant, my UNC Chart still proved an efficient method to contact thousands of eligible women.

Overall, patient portal research invitations sent through my UNC Chart were associated with enrollment in the Early Check study among women who opened those invitations. We found that a majority of women who received a my UNC Chart research invitation opened it, and of those who opened it, 4.88%, expressed in predicted probability, enrolled their newborn. In comparison, a previous analysis of the other primary recruitment method for Early Check, postnatal letters and emails to new mothers showed an overall statewide enrollment rate of 4% [36]. For those women who were sent a postnatal letter in addition to the recruitment invitation through my UNC Chart, the receipt of the postnatal letter did not increase the odds of enrollment. We did not independently compare my UNC Chart recruitment with direct letters and emails.

The findings demonstrated disparities in the use of patient portals, opening of research invitations, and enrollment in the study by race/ethnicity. There were also disparities in enrollment by urban/rural home address and in opening research invitations by age. Black women and Hispanic women were less likely to open an Early Check recruitment invitation sent through my UNC Chart and were less likely to enroll in the study after opening the invitation compared to non-Hispanic White women. We also found disparities by race and ethnicity among women we had hoped to reach using my UNC Chart invitations because they did not have an active my UNC Chart account. Members of traditionally underrepresented racial and ethnic minority groups were less likely than non-Hispanic White women in our target audience to have an active my UNC Chart account. Hispanic women were least likely to be my UNC Chart users, a finding that may be partially due to the availability of my UNC Chart in English only.

In our analysis of my UNC Chart by age and rural/urban home address, we found that there was no difference by age or urbanicity in those who had an active my UNC Chart account. Age was not significantly associated with opening the message or enrolling in the study except for women less than 20 years of age who were less likely to open the invitation. We found that women from urban areas were significantly more likely to enroll their newborns in the study compared to women from rural areas. It is not clear why urban women were more likely to enroll their newborns in the study although proximity to academic medical institutions and research familiarity may play a role.

Our enrollment rate among women who received Early Check my UNC Chart patient portal research recruitment invitations was similar to other studies using patient portals for recruitment, including the ADAPTABLE study performed in the same health system using a similar messaging protocol (4.4%) and a review of 13 studies recruiting through the patient portal of a single health system (2.9%-3.4%) [20,23,24]. Some studies have reported higher enrollment rates using patient portals ranging from 7% to 38% [22,28,32,34]. Bower et al [21], a study that also recruited pregnant women through patient portals, had a higher enrollment rate (11%) compared to the enrollment rate we report here (161/4510, 3.6%). The reasons for the differing enrollment rates across these studies are unclear but may be partially due to the target study population, demographics of patient portal users at an institution, type of study, demand on participants, formatting of the message, and the timing of the invitation in relation to a scheduled medical appointment. More research is needed on the factors associated with successful recruitment through patient portals and on the acceptability of using patient portals to recruit for research, to identify those studies for which a patient portal recruitment approach is likely to be most productive and acceptable.

The findings of racial and ethnic disparities in the users of my UNC Chart, opening of the recruitment invitation, and enrollment in the study are consistent with the findings across other studies examining the use of patient portals for recruitment and the use of patient portals for clinical care [8,23,29,32,34]. It is important to recognize that patient portal recruitment approaches have limited reach and may compound the problem of underrepresentation in health research. Identifying barriers to patient portal use for clinical care and intervening with specific subgroups to address those barriers may improve the reach of patient portals and their utility in recruiting a diverse research sample [17]. In the meantime, research administrators should use patient portals as part of a broader recruitment strategy and not the sole recruitment method.

The study examined patient portal research invitations sent to pregnant women, and findings may have limited generalizability to other types of patients. Findings may also have limited generalizability to organizations that use a patient portal other than Epic MyChart. It is also a limitation of the study that we did not directly compare the effectiveness of recruitment to Early Check through my UNC Chart research invitations to recruitment through postnatal letters and emails. We were unable to conclude whether one of these recruitment approaches was superior in enrolling newborns in the Early Check study or whether one approach would have resulted in a more representative sample.

Patient portals are an effective way to recruit participants for research studies and are especially useful for studies with large target sample sizes. There remain substantial racial and ethnic disparities in the use of patient portals, the response to receipt of an invitation, and enrollment in the study.