Behavior parent training (PT) is considered the gold standard for supporting and strengthening parenting practices and parent-child relationships and for the prevention and treatment of child behavior problems [1,2]. Positive parenting practices are a protective factor in buffering the negative effects of childhood trauma or early adverse experiences [3,4]. Parenting practices are often informed by family of origin, community, and social connections, and parents often look to their community for parenting support and guidance and many community-based agencies offer evidence-based PT [3].

Unfortunately, community organizations delivering PT experienced major disruptions in the provision of services with an abrupt ending of all in-person services due to the onset of the COVID-19 pandemic and shelter-in-place orders. These disruptions exacerbated an existing access gap for many communities, particularly those who were providing in-person individual or group-based PT. In addition, while parents lost access to parenting supports and child activities (eg, group activities, child care, school, and playgrounds), they experienced a substantial burden of shelter-in-place requiring a balance of work and parenting responsibilities [5,6]. Partially due to social isolation, balancing of responsibilities, and limited access to information and resources, data suggest a surge in negative mental, emotional, and physical health outcomes for parents and children [5,7,8]. Therefore, resources and support to mitigate the effects of the pandemic were crucial during this time.

Many community organizations shifted delivery of in-person PT and other parent supportive services using technology (eg, using videoconferencing for groups and web-based programming) [9-12]. The purpose of this paper is to describe the transition of a group-based PT program (ie, the Chicago Parent Program [CPP]) to a hybrid delivery model using the web-based PT of the CPP (ezParent) paired with videoconferenced group sessions and implementation in a community-based parent support program. This study is the result of a community-academic partnership in which representatives from each coled the design, implementation, and interpretation of the data.

The CPP is for parents of children aged 2 to 8 years and designed as a strength-based approach to promote positive parenting skills (ie, warmth, positive engagement, and support), support optimal child social and emotional development, and enhance parent-child interactions [13-16]. The CPP is a 12-session, group-based program that was developed with input from an advisory board of Black and Hispanic parents in low-income neighborhoods and aims to be contextually and culturally relevant for families from diverse racial, ethnic, and income backgrounds. Currently, the third edition of the CPP is broadly used in community agencies, pediatric primary care, early childhood care and education centers, and schools [17].

ezParent is the web-based version of the CPP and was designed to deliver the core parenting skills and strategies in the CPP via 6 self-directed modules. Similar to the group-based delivery, ezParent content and delivery was informed by an advisory group of Black and Hispanic parents and designed to help parents develop positive and effective parenting skills and decrease physical punishment through the use of behavioral strategies (eg, routines and labeled praise), brief videos of parents using the strategies, activities, quizzes, and practice assignments [18]. The web-based delivery of the CPP was developed to increase access by addressing challenges for parents and providers related to in-person delivery, including logistic barriers (eg, work demands, scheduling, childcare, and transportation), access to in-person groups, and potential for stigma [18,19].

The Rochester Area Parenting Program (RAPP), a program initiative at the Children’s Institute Inc in Rochester, New York, began implementing the group-based CPP in 2018. From March 2018 through March 2020, RAPP supported 35 CPP groups in collaboration with 6 partnering community programs across 14 sites (eg, churches, Head Start, childcare centers, and a large urban school district). During this time, 396 parents participated in CPP groups led by 23 CPP-trained group leaders. A key community engagement parent empowerment and leadership initiative of RAPP is to identify parents who completed CPP groups and subsequently support these parents in completing CPP training to become group leaders; thus, 25% of CPP-trained group leaders were parent graduates of CPP groups. At the onset of the COVID-19 pandemic shelter-in-place orders (ie, March 2020), RAPP suspended 3 in-person CPP groups and canceled 2 groups that were scheduled to start and began exploring options for alternative delivery methods of CPP to assure continued service provision and support for the families they serve. The RAPP coordinators engaged the CPP developers to determine options for delivery. During this time, many in-person CPP groups nationally moved to web-based delivery using videoconferencing platforms and adapted aspects of group delivery (eg, role-play, group activities, and discussion format). This option was presented to the RAPP leadership team, a group that provides strategic leadership and input on RAPP programming. The RAPP leadership team is a community collaboration with RAPP administrators and agency directors, staff, parent graduates of RAPP programming, trained facilitators, and faculty experts from partnering community agencies, schools, and universities. The RAPP leadership team reached consensus that 2-hour videoconferenced groups over 12 weeks would be challenging for the parents they serve due to logistics and competing demands on parents’ time. In addition, the RAPP leadership team considered parent feedback from the group-based CPP indicating that parents highly valued peer and facilitator interaction and support. Therefore, RAPP worked closely with the CPP developers to develop a hybrid model of delivery where parents would complete ezParent modules independently and participate in four 1-hour videoconferenced group sessions led by 1 CPP-trained facilitator.

The purpose of this paper is to describe the implementation outcomes of the hybrid delivery model of self-administered, web-based PT (ie, ezParent) and group sessions during community-based dissemination. Specifically, we seek to evaluate the implementation outcomes, using the RE-AIM (Reach, Efficacy, Adoption, Implementation, and Maintenance) model, of delivering the ezParent program, and identify facilitators and barriers to implementation delivery to support the sustainability of program implementation.

The RE-AIM framework was designed to provide a model to enhance the quality, speed, and public health impact of efforts to translate research into practice using 5 dimensions: reach the intended target population; efficacy of the intervention and implementation strategies; adoption by trained facilitators and settings (agencies); implementation, including fidelity and consistency of delivery; and maintenance of intervention effects in individuals and implementation in settings over time [20,21]. In this evaluation, we will report on the reach, efficacy, adoption, and implementation components and present maintenance as part of the discussion.

This single-group, mixed methods evaluation examined the implementation outcomes using RE-AIM of 24 hybrid ezParent groups (n=240 parents) delivered by 13 CPP-trained facilitators in the northeastern United States. We report on survey and interview data from hybrid ezParent delivery between September 1, 2020, and August 1, 2022.

Exempt approval for this project was granted by the Ohio State University Institutional Review Board (number 2022E0128).

Although the change in delivery was reactive to the COVID-19 pandemic shutdown, the delivery modification was a thoughtful and engaged process occurring between April and August 2020. Modifications occurred as part of a collaborative effort between CPP and ezParent developers, RAPP coordinators, and the RAPP leadership team to assure congruence with underlying CPP theory and content and RAPP family needs with the goal of maximizing intervention fit and implementation success. Refer to Textbox 1 for a description of key team members involved in the implementation of hybrid ezParent. We describe the adaptation process in Table 1 using the last 4 of 8 categories of the Framework for Reporting Adaptations and Modifications–Expanded (FRAME) outline [22]. The FRAME provides a structure for reporting the delivery adaptation. The FRAME categories include what is changed or modified, at what level of delivery the modification is made, type or nature of context or content-level modifications, extent to which the modification is fidelity consistent, and the reasons for the modification (ie, the intent or goal of the change and contextual factors that influenced the decision). What did not change as part of the adaptation was the ezParent program and core CPP skills and strategies, participation incentives provided for parents from RAPP, and groups facilitated by trained CPP group leaders.

The hybrid ezParent program began with an introductory videoconferenced group session with the facilitator and parents. The purpose of this session was to introduce parents to the purpose of the ezParent program and the groups sessions (eg, to review content parents were learning, review key strategies, and discuss parents’ successes and challenges in using the strategies) and to prepare them for the next session. The purpose of subsequent group sessions was to clarify the content, promote social connection and engagement, and help keep parents engaged in using ezParent. At the introductory session, parents were instructed to complete module 1 and 2 over the next 2 weeks before the next group session. This schedule is repeated for modules 3 to 4 and 5 to 6, each followed by a group session (Figure 1).

The RAPP partnered with 6 community organizations (hereafter referred to as RAPP sites). These sites included a large Head Start and Early Head Start program serving over 1000 families; a child care center serving families of children aged 0 to 5 years; a community organization providing childcare, free education and comprehensive services to support Latino/a families; a community organization with the goal of optimizing an nurturing and supportive environment to empower women; a child care network providing early care and education services, including universal PreK; and the Rochester City School District Early Childhood Education Department, serving over 2800 children (aged 3-4 years) and their families.

The RAPP site managed parent recruitment to participate in the hybrid ezParent and used successful recruitment techniques used for other programming for the parents they serve. Promotional flyers were provided by RAPP coordinators and emails were sent to parents to provide information about the ezParent, provision of an android tablet, monetary incentives, and details on how to sign up. If there was in-person contact at the agency or school (eg, child drop-off or pickup), parents were provided with information regarding the program. All sites had a capacity of 10 parents per group. If more than 10 parents expressed interest, the sites kept a waiting list. In subsequent groups at the RAPP site, parents on the waitlist were invited first to participate.

Once the parents enrolled, they were invited to complete presurveys (refer to the Measures section) and received an android tablet computer that had been preloaded with the ezParent program, zoom access, and internet access for the duration of program participation. Parents kept their android tablet at the end of the program for their personal use. Parents were instructed that all surveys throughout the course of the intervention were voluntary. A waiver of consent was obtained for these data as the parent data were anonymous and aggregated. In addition, parents were given instructions related to their first videoconferenced introductory group and paper copies of the curriculum handouts. Parents received US $20 for each group session and US $20 for completion of each module of ezParent (ie, total possible US $200). These conditional cash incentives were provided using a debit card registered in the parents’ name. Conditional cash incentives have been shown to be an effective method for supporting engagement in programming [24,25]. The debit card was either sent to their address or hand delivered. Of note, using debit cards was selected by parents and the RAPP leadership team as the best and most effective way to compensate parents.

Facilitators participated in a 2-day CPP group leader training between 2018 and 2021. At the time of training, trainees completed a demographic survey that included their report of race, ethnicity, education, employment status, current position, and experience working with families of young children and leading groups with families. Either facilitators were paid a stipend or they facilitated the ezParent groups as part of their work responsibilities.

After each videoconferenced group session, facilitators were invited to assess their adherence to the session protocol using a 7-item (session dependent) measure adapted from the CPP fidelity checklist [28]. Facilitators were not required to complete fidelity forms as the RAPP leadership team were concerned about burden on facilitators. Facilitators indicated yes or no, depending on whether they performed the expected action during the group session. Examples of adherence items include review of group ground rules, review of module summary of important points, and discussion of the ezParent module practice assignment. In addition, facilitators self-reported the length of time of the group (in minutes), rated parents level of engagement (eg, the overall groups interest and active involvement in the discussion) during the group session on a 3-point scale (ie, high, mixed, and low), and assessed the extent parents seemed to understand the program strategies (ie, high, moderate, and low). Finally, facilitators could provide additional comments or feedback as an open-ended response.

As part of program implementation, the RAPP coordinators monitored the number of groups that could be supported by RAPP sites and parent participation. This was driven by funding constraints, and coordinators balanced program costs to increase the number of groups available to reach the maximum number of parents. Parents using the ezParent program could report technical issues related to ezParent through a link in the program or contact coordinators for other technical support needs. Data logs of technical issues were maintained by the coordinators. Finally, coordinators met with the facilitators on an as needed basis to meet specific training and implementation needs. Coordinators kept a log of any implementation adaptations reported during these meetings.

In spring 2022, 8 facilitators who were currently involved with RAPP and hybrid ezParent delivery at the time of the evaluation were invited to participate in a brief (<30 minutes) postprogram interview. The goal of the interview was to assess facilitators opinions regarding the hybrid delivery methods (ie, implementation in RE-AIM framework), use of the discussion guide to support the conduct of the group session, and evaluate the strengths and weaknesses of the implementation model. An interview guide was developed (Multimedia Appendix 1) and interviews were conducted by 2 of the authors (RP and SMB) who were independent of the RAPP. Interviews were not recorded; detailed notes were taken during the interview. In total, 6 participants (response rate: 6/8, 75%) completed the informed consent process and the phone interview. Participants received a US $25 gift card as a thank you for their time and input.

All statistical analyses were conducted using SPSS (version 29; IBM Corp) [29]. Descriptive statistics were used to describe the sample and the reach, adoption, and implementation components of the RE-AIM model.

A thematic analysis focused on identifying implementation outcomes aligned with the implementation component of the RE-AIM model. Specific implementation categories were identified a priori (ie, facilitator adherence to group session protocol, length of group sessions, and facilitator perception of group delivery and parent engagement). Two authors (RP and SMB) reviewed the data independently, then met to review quotes for consensus and alignment with the implementation variables. Data from the interviews were used in a convergent design [30] to corroborate and provide examples of the quantitative data.

Of the 23 facilitators who conducted in-person CPP groups sessions, 11 (48%) facilitated ezParent group sessions. Two additional trained facilitators ran ezParent sessions who had not previously conducted in-person groups. Overall, this was a diverse group of facilitators with 69% (9/13) identifying as Black, 17% (2/12) identifying as White, and 17% (2/12) identifying as American Indian (individuals could select all that apply). In addition, 17% (2/12) of the facilitators identified as Hispanic ethnicity. Of the 13 ezParent facilitators, 2 (15%) reported having a high school degree, 5 (39%) having an associate degree or some college, with the remaining of the sample having a bachelor’s degree (n=2, 15%) or a graduate degree (n=4, 31%). Facilitator (N=13) experience at the time of training varied with greater experience working with individual families (a lot: n=6, 46%; quite a bit/some: n=7, 54%) and less experience leading groups of families a lot; n=2, 15%; quite a bit/some; n=10, 77%; a little; n=1, 8%).

In total, 38% (n=5) of the 13 facilitators completed end of session adherence self-reports, representing 31 of the 96 group sessions (32.3%). In addition, 6 of the 8 current facilitators participated in the facilitator interviews. Refer to Table 4 for facilitator quantitative and qualitative themes for implementation from adherence self-reports and facilitator interviews. Overall, facilitators found the hybrid delivery easy to implement and reported high parent engagement and understanding of CPP strategies. Implementation technical issues were infrequent and related to issues with the parent tablet which were administratively resolved. Only 1 technical issue was reported for the ezParent program and was related to logging in. RAPP coordinators experienced technical issues related to the ezParent administrative dashboard that was developed for this project. Because of these technical issues, ezParent prepared monthly parent use reports for the RAPP coordinators to assure accurate use data for parent incentives.

Times of the group meetings were determined based on parent polls, most groups occurred on weekday evenings (ie, Monday through Thursday) between 5 PM and 7 PM, 1 group was held at lunch time (ie, noon to 1 PM) and 1 group was held on a Friday evening from 6 PM to 7 PM. No parents indicated wanting a weekend meeting time. Facilitators reported several adaptations to address challenges to hybrid delivery.

Three main challenges were identified by facilitators, including completing a group session in 1 hour, fostering parent engagement in the program, and building connection in the virtual environment. As 1 facilitator noted, “for in-person groups we had more bonding time, it was hard to build relationships virtually.” To address these challenges, several facilitators would stay on the videoconference an extra 30 minutes to provide time for the parents to continue sharing. A facilitator reported, “I would say I’m here to stay on and many would stay on to discuss, talk it out, vent, I let them. They need that. In person they would have the time they needed.” In addition, facilitators created text groups and provided text reminders for group sessions and to engage parents in completing the practice assignment with their children. As 1 facilitator noted “one thing I liked was the text messaging—once I started doing that—parents would thank me for reminding them—they liked that.”

Although the facilitators felt positive about the group session delivery, they reported needing to tailor facilitation methods for videoconferenced groups. This included creating strategies to keep parents’ attention and promoting group sharing, as one facilitator noted, “I had to get creative to keep everyone’s attention.” Strategies included, asking parents to keep videos on, calling parents by name, and responding to nonverbal behaviors. One facilitator reported “For the last session I did a round robin and asked each parent what their take was away from completing the modules. I also asked if they were practicing. Each parent shared what really stood out for them and what helped them the most.”

Despite the challenges, facilitators found the hybrid delivery convenient for them and for parents. Facilitators reported that the hybrid delivery was logistically easier for them compared with in-person groups. In addition, although many reported that there was some loss in group connection compared with in-person groups, facilitators agreed that the convenience and ability for parents to participate in the “comfort of their own homes” might outweigh the challenges. Additional benefits of delivering the hybrid program included managing childcare (for both parents and facilitators), not having to spend time in gathering food and beverages for in-person group meetings, and not needing to spend time driving to and from the group. In addition, facilitators reported that parents were highly engaged and would readily report what they were learning from the program and changing in their own lives and the group session helped parents learn from each other’s experiences.

The purpose of this paper was to describe the implementation outcomes of the hybrid delivery model of ezParent during community-based dissemination in response to the COVID-19 pandemic lockdown causing in-person intervention to be unavailable. Using the RE-AIM framework, we found that the ezParent program was successful in reaching a diverse sample from the city of Rochester, New York, resulting in high levels of parent satisfaction and improvements in confidence in managing their child’s behaviors. Our findings for parent satisfaction are consistent with prior research investigating general parent satisfaction with ezParent [31]. The improvements in parental confidence in managing their child’s behaviors within hybrid ezParent is consistent with prior research conducted with CPP [13].

Overall, the trained facilitators easily adopted the new delivery format and implemented the sessions with minimal difficulties. As the aim of launching hybrid ezParent was not to conduct research but support the community during the COVID-19 pandemic, we were unable to determine potential sustainability (ie, Maintenance) as proposed by the RE-AIM framework. While we were able to demonstrate success within the implementation of ezParent, it is imperative to identify facilitators and barriers to implementation delivery to support the sustainability of the implementation of the program using the hybrid delivery model.

Factors that eased ezParent implementation included the support and strength of the RAPP leadership team relating to decisive actions in implementing the new modality, the use of community-based facilitators who were familiar with the community which was represented in the group and in CPP delivery, flexibility of timing of the group sessions including the length of the meeting (60-90 minutes), parental preference in day and time of meetings, and ability to participate from anywhere. Group day and times were parent driven, not chosen by the facilitator or the community partner. One facilitator polled parents for the best time to meet and was overwhelmed and challenged by the variety of responses. We recommend community organizations provide parents with a few meeting times to lessen variability while providing an opportunity for parents to choose. Most groups were held between 5 PM and 7 PM on weekday evenings as parents identified this as the best timeframe. However, there were competing priorities during synchronous group sessions, including childcare or preparing dinner that may affect parent engagement. However, many parents were able to creatively address these competing priorities. For example, some parents arranged their meal break at work for the group session. We believe this flexibility contributed to the proportion of parents who attended at least one session (91.3%) and of those who engaged in all 4 group sessions (77.7%). Attendance in this sample was slightly below parent attendance rates recently reported for videoconferenced delivery of 12-session CPP (82.3% of the enrolled parents attended at least 1 group session) [32]. We calculated our engagement metrics (ezParent use and group attendance) for all parents who enrolled in the program regardless of attending group to assess program adoption.

In addition to providing access and flexibility for parent engagement, the RAPP facilitators enjoyed the flexibility in the internet-based groups and conserving efforts and resources to manage in-person meetings. However, increased flexibility seemed to come with a loss of the reported social connection that came with in-person groups. Therefore, facilitators used other tools, such as group text messaging to stay in contact with parents between group sessions and to help build and maintain a sense of community. Social connection and building community are vital components to successful group interventions and the pandemic gave rise to challenges in maintaining social connectedness. The facilitators’ use of digital tools to create, maintain, and enhance a sense of community within their groups were found to be useful ways to engage parents and create community. These results are consistent with Plesko et al [32], highlighting the challenges with moving from in-person CPP to web-based group meetings.

We recommend using a variety of communication means to engage parents in-between sessions to support completion of the ezParent module, support parent enactment and practice of parenting skills, and to remind parents of the next scheduled group sessions. Although not included in our hybrid delivery model, we suggest including a booster session 6 to 8 weeks after the last live session to determine if parents have any questions or barriers to practicing and implementing new behaviors to support their children. This will also aid in obtaining data for the Maintenance component of the RE-AIM framework.

While reflecting on the implementation and evaluation of hybrid ezParent, there are several lessons learned for future community-based dissemination. During year 1 of implementation, parents were sent a web-based link for the presurvey and anecdotal comments, which indicated that this was seen as impersonal and not particularly relevant for parents to complete. In year 2, we changed this to a paper survey being administered at the time of disbursement of the tablet. This change resulted in a more robust completion of the presurvey by parents. In addition, parents were not incentivized to complete the presurvey and postsurvey; therefore, we suggest that parent participants receive an incentive to complete surveys. This may improve response rates and allow for more rigorous evaluation of the program outcomes.

An ezParent administrative dashboard was developed for this project; however, RAPP coordinators reported multiple technical issues. On the basis of these concerns, revisions have been made to the dashboard and we anticipate more autonomy for RAPP coordinators to process desired reports accurately and efficiently. In addition, further refinements of the ezParent dashboard have been implemented based on these findings and others [33] to assure relevance to community organizations implementing ezParent. For example, the ezParent dashboard now allows lists of users to be created based on cohort as well as the main page having a summary of users. If the user needs more detailed information on a participant, they can easily access the information from the main page.

Our results indicate that using the hybrid ezParent is a feasible and effective way to engage parents. We also found that facilitators and parents enjoyed the flexibility that a web-based group session provides. Future research should comprise of more rigorous evaluation processes so that changes in the outcomes of hybrid ezParent participants can be explored more precisely. In addition, more research needs to be conducted on the processes surrounding implementation of the ezParent program and its dissemination.