Despite the potential of digital health programs (DHPs) to prevent and treat mental health conditions in young children, DHPs have yet to make a significant mental health impact at scale. Many DHPs have faced suboptimal engagement [1], which also tends to decrease over time (ie, “the law of attrition”) [2]. Engagement in DHPs is multifaceted, encompassing affective (eg, enjoyment), cognitive (eg, attention), and behavioral (eg, adherence to intervention prompts) components [3]. Moreover, interaction with a DHP’s user interface is distinct from the enactment of the DHP’s specific behavior change techniques, which is further distinguished from engagement in relevant health behavior changes [4]. Importantly, effective engagement is defined as the quantity and timing of energy investment into a stimulus or task that is required to bring about a desired outcome [3]. Understanding effective engagement across multiple dimensions is important to DHP development, dissemination, and evaluation.

Typical patterns of participation vary widely across DHPs, consumers, target conditions, and settings, and depend on how engagement is operationalized [5]. Thus, rather than assuming a rigid, “one-size-fits-all” approach, authors have called for research that focuses on the degree of improvement as a primary metric of engagement success while also exploring multifaceted patterns of involvement [5,6]. Meta-analytic work has shown that engagement in a DHP for mental health is positively associated with clinical improvement, regardless of the intervention focus or whether the intervention is guided or unguided [7]. In one anxiety-focused DHP, the Unwinding Anxiety mobile app, completing a higher proportion of modules, as well as specific behavior change techniques, was associated with greater improvements in mental health symptoms [8]. Few preventive, parent-mediated digital interventions for youth 7 years or younger have been rigorously evaluated [9]. Moreover, typical patterns of participation in these programs remain unstudied. Inclusion of consumer perspectives in intervention design, development, and dissemination may help to overcome challenges with program engagement, thereby enhancing outcomes [10]. Understanding consumer engagement in digital mental health interventions is critical, particularly for programs focused on young children.

In parent-mediated interventions, caregiver participation engagement (CPE) refers to active participation in a program, including follow-through with homework and behavior change plans [11]. A meta-analysis found that CPE in youth-focused prevention programs was associated with increases in positive parenting behaviors and decreases in child internalizing and externalizing symptoms [11]. Given that family engagement in mental health intervention programs continues to be a challenge [12], especially in prevention programs [13,14], and that the determinants of CPE are understudied [11], more attention is needed to identify the barriers and facilitators to effective involvement in family-focused behavioral health interventions.

In search of empirically informed targets for the prevention of anxiety, over 50 studies to date have linked higher error-related brain activity with anxiety, in both adult and pediatric samples [15-17]. This neural marker, the error-related negativity (ERN), is generated when mistakes are made on a task and can be recorded using an electroencephalogram. A larger ERN in children is related to higher anxiety. Furthermore, an elevated ERN in early childhood independently predicts the risk for clinical anxiety in later childhood [18]. The ERN has been associated with error sensitivity, or the tendency to react negatively to one’s own mistakes [19]. Concern over making mistakes is also a hallmark of maladaptive perfectionism [20]. Given these findings, our research team developed an intervention that aims to impact the ERN by targeting overly negative responses to making mistakes (ie, behavioral error sensitivity or perfectionism) in young children [21].

Families high in error sensitivity or perfectionism may face unique challenges in engaging with preventive DHPs. Perfectionistic families may not always seek psychological services; thus, understanding engagement in non–treatment-seeking perfectionistic families is important, consistent with a prevention or health-promotion perspective [22]. According to the social expectations model, parents with unreasonably high expectations for their children may have an overcontrolling parenting style, thereby increasing the risk of maladaptive child perfectionism [23]. Moreover, the social learning model (also [23]) suggests that parents pass on perfectionistic behaviors to their children via modeling. These hypotheses are supported by meta-analytic research, which has revealed that parental self-oriented perfectionism, expectations, and criticism positively relate to child perfectionism [24]. Maladaptive perfectionism may result from child traits that dynamically interact with parental factors, such as harsh or critical communication, authoritarian or controlling parenting, psychological control, perfectionistic modeling, and excessive performance feedback focusing on error avoidance [25-31]. Furthermore, behavioral genetics has shown that perfectionism is moderately heritable, meaning perfectionistic families likely share genetic diatheses [20].

Few interventions have targeted perfectionism in youth [32,33] (see [34] for a cursory review). Parent-mediated interventions for perfectionism should target theorized intergenerational mechanisms of children’s maladaptive perfectionism, such as setting excessively high standards, encouraging avoidance of mistakes, and modeling perfectionism [23,24,35]. Published recommendations on preventive interventions for perfectionism include adopting a positive psychology focus; providing a rationale and discussing risk; targeting maladaptive perfectionistic beliefs; providing cognitive reappraisal strategies; promoting a growth mindset, self-acceptance, and self-compassion; and engaging parents in reducing criticism, modeling calm responses to errors, and reiterating anti-perfectionistic messages [36].

In accordance with empirical findings, theory, and guidelines, a preventive DHP targeting error sensitivity in 5‐ to 7-year-old child-parent dyads has shown pilot efficacy [21]. A randomized controlled trial (RCT) of an expanded version of this program is underway.

Mixed methods approaches can elucidate engagement and implementation outcomes in parent-mediated digital interventions (eg, [37-43]). However, little is known about program participation in error-sensitive families. Likewise, there is limited research on family engagement in preventive interventions for anxiety or on DHPs with children under the age of 7 years. Given the increasing importance of DHPs in preventing and treating pediatric mental illness, it is critical to understand how families participate in and benefit from these programs, the factors that facilitate participation, and the facets of involvement that contribute most to program success (ie, effective engagement). The findings will enhance the potential effectiveness and reach of DHPs by advancing knowledge of typical family participation in preventive DHPs, clarifying the relationship between engagement and program outcomes, and contributing to a deeper understanding of the factors that influence program participation.

In this study, we examine family participation in a preventive DHP targeting error sensitivity in young children and their parents. We delineate patterns of engagement with Making Mistakes, including adherence within dyads; perceptions of program outcomes; and cognitive, affective, and behavioral facets of engagement. We also explore facilitators and barriers to family involvement. The research questions are as follows: (1) To what degree do dyads engage in Making Mistakes with fidelity, including program adherence and enactment of content learned? (2) How is participation perceived to impact family functioning? (3) What are facilitators and barriers to family engagement? (4) How do qualitative descriptions of engagement contextualize quantitative findings? This study builds on the emerging body of parent-mediated and digital mental health interventions by using mixed methods to explore families’ patterns of program participation. We incorporate robust qualitative methods to contextualize quantitative findings on program adherence and gain an in-depth understanding of participation. Understanding barriers and facilitators to engagement—and what constitutes effective engagement—may improve mental health outcomes for families by augmenting the impact of Making Mistakes and similar programs.

This mixed methods study included families who were randomized to the 3 active treatment arms of an ongoing RCT evaluating Making Mistakes, a web-based program targeting anxiety risk in 5‐ to 7-year-old children. Parent-child dyads were recruited from a community, nonclinical sample. In the larger study, dyads were interviewed about their experience, including program acceptability and appropriateness. Procedures and methods for the RCT can be found elsewhere ([34]; also see Multimedia Appendix 1).

Parent-child dyads were recruited for quantitative assessment as part of their participation in the larger RCT. Inclusion criteria were as follows: (1) elevated error sensitivity as measured by elevated scores on the Child Error Sensitivity Index, Child Error Sensitivity Index–Parent Report, or the Parent Sensitivity to Child Errors Index [19] and (2) completion of the baseline intervention module. Two dyads were excluded because they did not receive the weekly text message or email reminders as prescribed, due to clerical errors. An additional 7 dyads were excluded for not completing the entirety of the baseline intervention module, resulting in a final sample of 87 dyads for the quantitative assessment (n=30 with both parent and child in the active condition [P+/C+], 30 parent-active+child-control [P+/C–], and 27 child-active+parent-control [P–/C+]). Reported reasons for not completing the baseline module indicated barriers related to childcare requirements (ie, siblings), busy schedules, and factors potentially related to child age or developmental level (eg, child refusal to continue the study).

Families were eligible for qualitative interviews if they were randomized into 1 of 3 active intervention arms and completed at least a portion of the baseline module. The lead author (IAM) conducted all interviews, which were solicited across various phases of intervention completion. Random and convenience sampling were used for the first 10 interviewees. Strategic sampling was used for the remainder, which prioritized P+/C+ dyads. Sampling aimed to match or surpass the demographic diversity of the overall participant sample with respect to the timing of the interview vis-à-vis the phase of the study or intervention, race or ethnicity, gender and sexual orientation, and relationship status. Recruitment goals were based on data saturation, which often occurs between 10 and 17 interviews [44]. Data saturation was reached at 18 dyads (n=18 adults, 14 children). Additional information regarding qualitative data collection is reported elsewhere [45].

Study procedures were approved by Florida State University’s Institutional Review Board (STUDY00000605). Informed consent was obtained from parents, and assent was obtained from children. Participant information was kept private and confidential. We compensated participants with a US $25 gift card.

Adherence was reported descriptively for children and adults, including rates of overall and sustained adherence. A Welch’s ANOVA examined differences in adherence by intervention condition. To interpret the qualitative data, a standard thematic analysis informed by the framework method was performed [51,52] using the NVivo software (version 12.7.0; by Lumivero). This study relied on a codebook generated for a prior study, and data preprocessing is described elsewhere in further detail [45]. Summarily, the analysis used a thematic framework [53], data familiarization, iterative development of a coding scheme and coding, double-coding and coauthor consensus building, recoding, development of a thematic matrix, and interpretation of data via the themes. The thematic matrix was developed using a combination of deductive and inductive processes. Rigor and trustworthiness were established through the use of previously published inquiry methods, strategic sampling methods to achieve a diversity of perspectives, triangulation of multiple data types, consultation, integration of author positionality, negative case analysis, and member checking [47,54].

Our quantitative and qualitative results provided complementary insights. Although quantitative patterns revealed relatively low fidelity of adherence, this did not translate to poor engagement with the content. Importantly, barriers to high-fidelity participation were not always barriers to overall program success. Qualitative results highlighted diverse and idiosyncratic barriers and facilitators to engagement and suggested numerous positive outcomes of the program that, in turn, facilitated engagement.

Demographics are summarized in Table 1; additional demographics of the qualitative sample are reported in Table S1 in Multimedia Appendix 1. Parent and child overall adherence were strongly positively correlated (r₈₅=0.946, 95% CI 0.919-0.965; P<.001). The overall rate of adherence to the weekly modules was approximately 20%, with substantial variability; indeed, 43.7% (n=38) of the parents and 44.8% (n=39) of the children did not log into any of the weekly modules. Parent and child sustained adherence were also strongly positively correlated (r₈₅=0.949, P<.001, 95% CI 0.923-0.966). Table 2 shows adherence by parent and child (see Table S2 in Multimedia Appendix 1 for analogous data within the qualitative sample). Furthermore, adherence declined over time (Figure 1).

ANOVA assumptions were not met (skewness=1.98; kurtosis=3.15; nonhomogeneity of variance). Thus, a Welch ANOVA, with Games-Howell post hoc comparisons, was used to test for differences in average sustained adherence (ie, number of unique active weeks) by intervention condition (P+/C+: mean 2.37, SD 3.48; P+/C–: mean 2.77, SD 4.26; P–/C+: mean 1.56, SD 2.54). The test was not significant (F_2,54.84=1.05, ω²_random=−0.002, 95% CI −0.012 to 0.038; P=.36), indicating no differences in sustained adherence rates by condition. Post hoc comparisons revealed that sustained adherence in P+/C– was not significantly greater than in P+C+ (95% CI −2.02 to 2.82; P=.92, or P–/C+, 95% CI −1.01 to 3.43; P=.39); nor was there a difference between P–/C+ and P+C+ (95% CI −2.74 to 1.12; P=.57). (As a sensitivity analysis, we fit a population-averaged logistic model using generalized estimating equations with exchangeable working correlation to account for within-dyad dependence across weeks. The Condition × Week interaction was not significant (Wald χ²₂=4.36; P=.11), indicating that completion trajectories across modules did not differ by condition. This result converged with the primary Welch ANOVA findings.)

This study examined participation fidelity, family involvement, perceived outcomes, and barriers and facilitators to engagement in a web-based anxiety prevention program in parent-child dyads. Qualitative and quantitative data showed high levels of engagement in the initial module, albeit low levels of DHP adherence over the 6-month intervention period. Barriers and facilitators included competing demands and program credibility, respectively. Perceived benefits emerged as a key facilitator, resulting from engagement spanning cognitive, affective, and behavioral domains.

Participants completed, on average, one-fifth of the weekly modules, with engagement dropping off over time; however, the modal dyad completed zero weekly modules. Online video and paper journal completion generally coincided, though more families completed the videos, and a few completed journals without the videos. In contrast to the weekly modules, in-lab engagement in the baseline module activities was high. Differences in lab versus home participation align with research demonstrating the utility of human facilitation in promoting DHP usage [55]. DHPs often face low adherence, which may limit therapeutic benefits [56]. Additional work should clarify the relative importance of—and strategies to optimize—different levels of digital engagement, including mechanical program usage, cognitive engagement with program notifications, and behavioral enactment of nondigital tasks resulting from the program [3]. Our findings suggest that this type of DHP may have a strong impact when integrated with human facilitators at the outset, followed by software-delivered engagement strategies tailored to the individual family’s needs. Future research should determine the optimal forms of involvement needed to attain program benefits [57].

Numerous barriers and facilitators to family engagement were identified. Facilitators included programmatic factors, such as teaching specific content and strategies, rationale provision (ie, psychoeducation on error sensitivity), mobile-friendliness and navigability, regular reminders with repeated exposure to the content, participation in the in-lab baseline module, and approaching Making Mistakes as a shared family activity. Facilitators also included family characteristics, such as child enthusiasm for learning; perceived relevance to child or parent based on program descriptions of traits associated with error sensitivity (eg, shy, sensitive, socially anxious, reactive); belief in the program’s potential and interest in child psychology and parenting improvement; personal negative experiences with high error sensitivity; and buy-in from co-caregivers. Indeed, the broad applicability of Making Mistakes appeared to be a key driver of parent participants’ affective engagement, or emotional investment, with the program, which facilitated behavioral change. Moreover, early improvements in family functioning facilitated buy-in and continued engagement. Facilitators overlapped with those found in previous research, including perceived need, appealing content and delivery, openness to prevention and health promotion approaches, individual motivation and interest, and increased insight into health [38,58]. Our findings on facilitators and barriers shed light on potentially important factors influencing CPE within a DHP [11]. Additional strategies, such as personalization and gamification, may enhance overall program adherence and efficacy.

Both program-related and family-related barriers were identified. Regarding program-specific barriers, some families cited issues related to poor customizability of content and delivery, too many steps to access the content, repetitive or overly simple content, intervention dose (ie, weekly) perceived as daunting, and insufficient incentives to complete the weekly activities. Family-level barriers consisted of low perceived relevance due to parent or child traits; multiple competing demands, especially work and childcare; single-parent status; and temporary stressors, such as moving, separation, or child psychosocial stressors. Barriers overlapped with those found in previous research (eg, [38,42]), including childcare demands, employment, low perceived need, difficulty with program navigability, and perceived burdensomeness. Many barriers were potentially related to social determinants of health and the limited sophistication of the program; therefore, just-in-time adaptive interventions could be developed to equitably address error sensitivity in at-risk families [59]. Of note, the sample only included study completers or partial completers, and thus barriers to initial participation were not captured (eg, see [60]). Overall, barriers and facilitators aligned with those predicted by the health belief model and the theory of planned behavior—such as perceived costs, threats, attitudes, and subjective norms—consistent with research demonstrating support for these models in parenting intervention engagement [61]. These findings also add to the growing body of evidence on engagement in youth mental health services [62,63]. However, the development and testing of theories of family engagement in DHPs will be needed to address barriers and optimize facilitators.

Abundant positive changes were highlighted via qualitative interviews. Commonly cited outcomes included direct intervention targets—including parent sensitivity to child errors and child error sensitivity—though broad changes to parenting and parent-child functioning were also described. Changes to beliefs or thinking (eg, reframing mistakes, increased parenting awareness) seemed to reflect evidence of cognitive engagement. These mental shifts were described as preceding behavioral changes. In addition, improvements in child emotional functioning appeared to be driven by changes in parental behavior, although perceived impacts were bidirectional. Participants described additional unanticipated outcomes, including benefits to siblings, other caregivers, friends, family, and community members. Indeed, these descriptions of wide-ranging improvements appeared to reflect a high level of behavioral engagement with Making Mistakes, even despite low rates of adherence to the weekly modules.

This apparent discrepancy—that is, abundant perceived positive outcomes and descriptions of strategies enacted, despite low weekly adherence—merits further attention. The findings support distinctions among aspects of behavioral engagement, such as using a DHP’s interface, enacting specific techniques learned, and making broader changes to health behaviors [4]. It is possible that the initial in-lab intervention module was an adequate dose for many participants, and subsequent program interfacing was not necessary for broader enactment of concepts learned. Alternatively, the simple weekly reminder texts and emails may have spurred continued improvements, even for those who did not complete the activities. The findings extend the literature on CPE within a DHP, suggesting that cognitive and affective engagement, as well as more “downstream” indices of behavioral engagement, may be more important than traditional metrics of DHP adherence. To better understand what constitutes effective CPE [3,11], future research may directly compare different behavioral engagement strategies and explicitly link them to outcomes of interest.

Qualitatively, families also commented on the positive impact of study involvement, including participation in dyadic games during the lab, as well as routine outcomes monitoring via error-sensitivity measures [34]. Future research should clarify the relative importance of such nonspecific program components, including measurement and observation procedures. Indeed, given the known benefits of measurement-based care in youth mental health services [64] as well as the potential for observer effects or reactivity in parent-child interaction research [65], participant reports of program-related changes to functioning should be interpreted cautiously.

Enhancements to program design and structure may help to overcome common program-related barriers to participation. Beyond well-established DHP design recommendations, such as customizable reminders and intervention frequency, gamification, human facilitation, ease of access, and appealing content and delivery (eg, [66-68]), this study suggests several additional strategies that may enhance engagement. Participants responded well to psychoeducation about error sensitivity, including its causes and consequences, which increased the perceived relevance of the program. Additionally, DHPs and caregiver-mediated programs may be enhanced through options to engage cocaregivers in the program. Moreover, the findings suggested that offering the initial in-lab module, along with parent-child games, and the opportunity to approach the program as a dyadic activity may have facilitated program engagement. Finally, regular reminders (nudges), including periodic assessments, may have driven family investment even in the absence of repeated doses of the intervention content. These potential strategies should be empirically validated in future research.

This study has multiple strengths, including integrated qualitative and quantitative data; demonstration of qualitative rigor and trustworthiness; inclusion of young children in data collection; multiple novel foci (ie, DHPs for 5 to ‐7-year-olds, error sensitivity, preventive parent-child program development); intervention targets derived from a neural marker of anxiety; multiple operationalizations of adherence; and a critical focus on multiple aspects of engagement, a common pitfall of DHPs and of caregiver-mediated interventions [56].

This study must also be interpreted in light of several limitations. Both intervention conditions were housed on the same website, and thus, a few participants accessed a video from the other condition. Thus, analyses focused on rates of adherence by condition should be interpreted with some caution. In addition, our measures of behavioral engagement were somewhat limited, including overall adherence and sustained engagement. Future research should incorporate more dynamic and multidimensional metrics, such as the amount of time spent with the intervention or implementing strategies learned; explore temporal patterns of program participation; or use ecological momentary assessment methods to capture behavioral, cognitive, and affective engagement over time. In addition, though diverse perspectives were represented among the qualitative interviews, external validity is limited by the sample’s makeup: mostly White, highly educated, multi-parent homes with relatively high household income. Thus, the results may not be fully generalizable to marginalized families facing additional participation barriers. Future studies must examine Making Mistakes engagement in diverse samples.

Preventive programs for young children and families may yield broad-ranging benefits, including transdiagnostic impacts beyond specific intervention targets. Our findings highlight the importance of qualitative assessment in trials of preventive programs and contribute to the sparse literature on prevention programs for young children [69]. The findings also contribute to the broader literature on perfectionism interventions for young people [36], digital health interventions [9,66] including those for preschool anxiety [70,71], and parent-based interventions for young children [72,73].

These findings support further evaluation of a low-intensity preventive intervention, Making Mistakes, which shows promise in promoting positive change within error-sensitive families. Making Mistakes included a single-session intervention (see [74]) in tandem with brief reminders or nudge interventions [75]. Future iterations of Making Mistakes or similar programs could benefit from further tailoring for very young or preliterate children, including the use of verbal recordings instead of written practice assignments. Finally, to meet the diverse needs of families with elevated error sensitivity (see [31]), modular DHPs could be developed to prioritize family preferences and accommodate additional areas of concern to the family [76]. Future research should evaluate the long-term impact of Making Mistakes and elucidate family engagement in diverse populations. Importantly, family-focused developmental models of perfectionism, including the social expectations, social learning, and anxious rearing models [23,24,31], should guide efforts to develop and implement engaging digital mental health interventions to prevent the development of pediatric anxiety disorders and cultivate resilience in families.