Approximately 6.5% of adolescents internationally experience anxiety [1]. Anxiety can become entrenched, lead to a lifelong struggle, and is linked to numerous other problems, such as reduced academic performance, poor self-esteem, social problems, substance abuse, depression, and attention and concentration difficulties [2,3]. Digital mental health (dMH) interventions are an effective treatment of choice for many adolescents with anxiety [4-6]. These digital interventions are highly accessible, flexible, affordable, require no waitlists, and reduce concerns about stigma [6].

While dMH interventions are highly accessible and efficacious, it is a concern that, when delivered on a self-directed basis without therapist support, many adolescents cease the program early or neglect to complete program activities [4]. For example, in a study on the dMH program, youthCOACH, which targets chronic medical conditions in youth aged 12 and 21 years using cognitive behavioral therapy (CBT), only 40% of adolescents adhered to the intervention (completed at least 80% of the seven modules) [7]. In another study on the BRAVE Self-Help program, an open-access CBT dMH program for child and adolescent anxiety, only 30% of the 4425 children and adolescents completed more than three sessions, with the greatest clinical improvements made by those users completing at least 6 of 10 sessions [4]. These findings are consistent with those of a recent systematic review and meta-analysis of 35 predominantly adult dMH interventions (both therapist-supported and self-help), which found that greater engagement was associated with stronger treatment gains [8]. Thus, those with lower engagement potentially fail to obtain the greatest clinical benefit.

Gan et al [8] suggested that variability in the operationalization of engagement is a limitation of the dMH field, and defined engagement as any objective indicator of intervention use (eg, sessions completed, time spent, or number of logins). Indeed, a broader inspection of the literature reveals that dMH researchers have not applied a universal definition of the construct of engagement, and the term is frequently used interchangeably with “adherence,” “compliance,” and “use” [9,10]. To date, the majority of research has focused on the amount of the intervention completed, such as the number of completed treatment sessions or activities (eg, March et al [4], Calear et al [11], and Fleming et al [12]) to capture the use of dMH interventions. However, it is now widely acknowledged that engagement is multifaceted and incorporates more than just the number of sessions completed [4,13,14]. Two key reviews by Yardley et al [14] and Perski et al [15] suggest that engagement should be conceptualized as including both use and user experience. Use refers to the amount of intervention content completed (ie, completed sessions and activities), frequency of session completion, duration spent on intervention, and depth of responses (ie, character or word count). In contrast, user experience refers to subjective, self-reported experiences (ie, attention, interest, and satisfaction with the program), and affect experienced during the program [14,15]. The clinical validity and reliability of these constructs are not yet known; however, the frameworks proposed by Yardley et al [14] and Perski et al [15] provide a useful and comprehensive structure to guide research and advance our understanding and measurement of the numerous potential facets of engagement in dMH interventions.

Given that higher program use (at least in terms of the number of sessions completed) has consistently been shown to be positively associated with treatment outcomes, but that many participants do not progress beyond a few sessions of dMH programs [8,16], it appears that understanding early engagement (EE) in the first few sessions could offer valuable insight and assist in identifying ways to enhance overall program completion. In a blended-care CBT intervention (combination of videoconference therapy sessions and personalized dMH activities), Wu et al [17] examined the predictors of program retention and found that engagement was poor if participants did not complete their assigned web-based activities early in treatment. Here, engagement was defined as the completion of both the first telehealth therapy session and assigned dMH activities after the session. In relation to self-help dMH programs, there is some research that has identified early “use” in dMH programs is related to program engagement, however, evidence is limited to engagement conceptualized as word count of activity responses [18]. For example, Wallert et al [18] found that writing more words in early dMH activities was related to higher rates of overall program adherence (completing three or more homework tasks). It is possible that greater effort or more in-depth use of content and tasks early in the program (better engagement) may facilitate interest, trigger symptom improvements, and sustained engagement over time. Given that more recent frameworks propose the conceptualization of engagement beyond the number of program sessions completed, there is a need to examine additional facets of engagement and determine whether different types of EE (eg, early completion of in-session and homework tasks, character count, subjective self-report measures, and affect experienced during the program) are important in facilitating program engagement in dMH interventions. This study focuses specifically on the program use component of engagement as it is typically routinely collected and operationalizes multiple measures of program use. It also focuses primarily on the role of EE as described above, and its relationship to overall engagement.

Much of the existing literature has examined factors associated with dMH engagement in adult populations. In a review of 83 adult dMH programs, Liverpool et al [19] found that both intervention-specific influences (such as program suitability, usability, and acceptability) and person-specific influences (such as motivation, opportunity and time availability, family support, and capability) influenced user engagement. Borghouts et al [20] further highlighted that social connectedness, participants feeling “in control” of their health, and having insight into their concerns, were also facilitators of dMH program engagement. With respect to adolescent programs, a review of dMH programs by Lehtimaki et al [21] found that female, those with lower baseline anxiety, and those with a longer history of mental health conditions were more likely to complete dMH interventions [21]. Similarly, March et al [4,16] found that male, those with higher baseline anxiety severity, and those living in nonmetropolitan regions showed lower levels of engagement (operationalized as the number of sessions completed) in a dMH program for anxiety. However, to date, there is very limited research specifically examining the individual facets of engagement (as suggested in the frameworks by Perski et al [15] and Yardley et al [14]) that predict program engagement with dMH interventions by adolescents.

This study examined the factors influencing program engagement, specifically, program use, in a dMH intervention for adolescent anxiety (BRAVE Self-Help). Extending on previous literature, engagement was conceptualized as being multifaceted. The first aim was to examine the effect of person-specific demographic factors (including gender, age, and residential location) on program engagement (sessions completed, in-session and homework tasks completed, depth (total character count), and average frequency (days) between sessions). The second aim was to examine the impact of EE on continued program engagement. EE was examined specifically in relation to program use, and conceptualized as the number of in-session tasks completed, homework tasks completed, and character count (depth of use), within the first two sessions of the program. In addition, previous research has found that participants completing a greater number of BRAVE sessions showed lower baseline anxiety severity [4] and were less likely to be classed in the high anxiety severity group when examining trajectories of change [16], suggesting that anxiety severity may be a relevant factor to consider. Therefore, this study further examined whether baseline anxiety severity moderated the relationship between demographic and EE factors and program engagement.

Participants were 2850 adolescents aged between 12 and 17 years who had registered for the BRAVE Self-Help program between July 2014 and May 2020. In line with the intended audience of BRAVE Self-Help, only participants demonstrating baseline anxiety severity at an elevated or clinical level on the Children’s Anxiety Scale 8-item (CAS-8; see below for details) [22] were included in this study. This study was restricted to adolescents who had completed at least 2 sessions of the program. This was in line with our conceptualization of EE in this study. To ensure adequate measurement of EE, data were required relating to in-session tasks completed (sessions 1 and 2), and between-session tasks (homework allocated in session 1 that was subsequently reported on in session 2). Thus, to provide a comprehensive indication of EE, we required data collected in sessions 1 and 2. Further, as per calls for research to more comprehensively examine engagement through multiple components (eg, Yardley et al [14] and Perski et al [15]), it was important to include 2 sessions in our definition of EE as many dMH programs focus only on introductory and education-based content in the first session (eg, March et al [16]) without the inclusion of many interactive activities. Therefore, examining only responses to session 1 would not allow sufficient examination of program use in the form of completed activities or use depth. Participants were excluded from the study if they were completing the program as part of a school curriculum or school-based intervention, in order to mitigate any other engagement strategies delivered to these participants (eg, mandatory attendance and homework completion and support or facilitation by a school professional).

Participants were self-referred or referred by others (eg, general practitioners, school counselors, and other mental health practitioners) to the open-access BRAVE Self-Help program. No incentives were provided for participating in the program. Participants provided informed consent prior to commencing the program, and parental consent was also obtained if the adolescent was younger than 16 years of age. Informed consent was also provided for their data to be used in future research. All data analyzed were deidentified. During the registration process, participants provided demographic information (see below) and completed the CAS-8 [22]. This study was part of a larger, ongoing effectiveness study of BRAVE Self-Help (human research ethics approval: ETH2023-0832; University of Southern Queensland; 2020/581; Griffith University).

This study used data extracted from BRAVE Self-Help for teenagers (aged 12 to 17 years), a dMH intervention for adolescents with anxiety [4]. BRAVE Self-Help is delivered in 10 web-based, interactive, weekly sessions of 30 to 60 minutes duration and includes cognitive-behavioral strategies such as psychoeducation, cognitive restructuring, physiological awareness, relaxation, graded exposure, problem-solving strategies, positive reinforcement, and guided imagery [23]. Homework tasks are allocated each week to consolidate and enhance learning from each session. Reminders to complete sessions are sent to participants via automated email. BRAVE Self-Help is completed without therapist support, and sessions can be completed at any time but must be completed in order. For a detailed description of the program, refer to March et al [4].

EE was defined as engagement within the first 2 sessions of the program. Similar to the operationalization of engagement by Wu et al [17] , EE in this study was defined as the number of tasks completed in sessions 1 and 2, the number of session 1 homework tasks completed (data collected in session 2), and depth of responses (character count) in sessions 1 and 2. Because homework tasks allocated in session 1 are completed between sessions and reported in session 2, it was essential that this study included session 2 in its conceptualization of EE.

Program engagement was operationalized as both overall program engagement (OE) and continued engagement (CE). OE was measured through the total number of program sessions completed, total number of program tasks completed, total number of homework tasks completed, total engagement depth (character count of responses typed into any program tasks), and average frequency of sessions (average days between sessions). CE was defined as engagement after session 2 (ie, the number of in-session and homework tasks, sessions completed, and depth of engagement from sessions 3 to 10 only). The number of prescribed tasks and homework tasks per session is detailed in Multimedia Appendix 1.

Data were analyzed using SPSS Statistics (version 27; IBM Corp). EE and OE were computed from raw program output data and are described in Table 1 and reported in Table 2 for the total sample, elevated, and clinical subgroups. Differences between the elevated and clinical groups for overall and early program engagement were analyzed using Cohen d.

For the first research question, 15 separate moderated regression analyses were conducted to examine whether baseline demographic factors predicted OE (one moderated regression per predictor and dependent variable) and whether baseline anxiety severity moderated these relationships. Demographic factors (age, gender, and location) acted as the independent variables, whilst total completed sessions, total tasks completed, total homework tasks completed, depth of responses, and frequency of session completion acted as the (OE) dependent variables.

For the second research question, 15 separate moderated regression analyses were conducted to examine whether EE factors predicted CE (one moderated regression per predictor and dependent variable), and whether baseline anxiety severity moderated these relationships. Early tasks, early depth, and early homework acted as the independent variables. The dependent CE variables were completed sessions, total tasks completed, total homework tasks completed, depth of responses, and frequency of session completion.

Anxiety severity acted as the moderator variable, with participants categorized as either elevated or clinical. Regression analyses were conducted separately for each predictor, given that previous research has shown certain demographic factors may contribute more to overall engagement than others [4,21] and models suggest that the multifaceted nature of engagement needs to be further understood [14,15]. As we were interested in understanding which EE factors had the strongest effects in predicting CE, we conducted separate rather than multivariate regression analyses. Given the large number of regression analyses conducted, a conservative Bonferroni correction of P<.001 was applied for all analyses.

A summary of participant baseline demographic and clinical characteristics across anxiety status is provided in Table 2.

As seen in Table 3, adolescents in the elevated group demonstrated significantly greater OE than those in the clinical group, as measured by total homework, program depth, and early depth. Adolescents in the elevated group also had a significantly higher frequency (days between sessions) than those in the clinical group, which may suggest that these adolescents may be completing the program more in line with program frequency recommendations (sessions should be 5 to 7 days apart).

Before completing moderated regressions, correlations were analyzed between all predictor and outcome variables, which are presented in Multimedia Appendix 2. Of the relationships of interest, greater age was significantly correlated with higher completed sessions, frequency, total homework, total tasks, and severity. Female gender was strongly associated with greater completed sessions and program depth. In relation to EE predictors, there was a significant correlation between higher early tasks and greater program depth, total homework, frequency, and total tasks. Greater early depth was also significantly positively associated with total homework, completed sessions, total tasks, and average frequency. Higher early homework was significantly and positively associated with program depth, total homework, total tasks, and frequency. Finally, lower anxiety severity was significantly correlated with greater program depth (CE), average frequency (CE), and program depth (OE).

Outcomes from moderated regressions were conducted to determine whether EE variables predicted CE, and if these relationships were moderated by anxiety severity.

This study analyzed the relationship between demographic and EE factors, and OE and CE, respectively, and assessed whether such relationships were moderated by baseline anxiety severity in a dMH self-help intervention for adolescent anxiety, BRAVE Self-Help. Importantly, this study examined the utility of conceptualizing engagement, specifically, program use, in multiple ways to better understand engagement in dMH interventions. Overall, the results highlighted that a higher level of EE in the first two sessions was associated with greater CE (intervention-specific factor). This was evident specifically when predicting CE program depth (characters written), but a trend was also observed for CE tasks completed. That is, early depth of program use (characters written in the first two sessions) and early tasks completed in the first two sessions were associated with greater CE in terms of program tasks and depth of responses. In addition, there were bivariate associations between multiple EE and CE variables, suggesting a clear relationship.

Additionally, contrary to expectations, the study failed to demonstrate that anxiety severity impacts engagement either directly or indirectly for most indicators of engagement, although a higher baseline level of anxiety was associated with fewer average days between sessions (frequency). This may suggest that adolescents with more severe anxiety are eager to seek further assistance and see improvements in their anxiety. Although the impact of anxiety severity on engagement was inconsistent in this study, previous research has found that higher anxiety severity is related to poorer program engagement for adolescents in dMH programs [4,16,20,21]. Variations in the way in which engagement has been measured across studies may account for the difference in findings in this study.

Importantly, person-specific demographic factors (age, gender, and location) showed little relevance in predicting program engagement in this study. Previous studies have reported mixed findings with respect to the impact of demographic variables on engagement in dMH programs [4,16,21]. Further research in relation to the relevance of person-specific factors in predicting program engagement should be undertaken once engagement can be measured more holistically.

The intervention-specific finding that EE was generally associated with CE is not surprising, yet an important research finding. Whilst previous research has identified that program engagement is associated with positive clinical outcomes [4,8], this study shows that engaging well early in the program, specifically early program use, is specifically associated with greater CE. Although the effects were small, our findings highlight that there is a need to identify ways to meaningfully engage adolescents in both in-session and homework tasks in the first two sessions of the program, to increase program engagement, and ultimately, the benefits obtained from dMH interventions.

There are few studies that have identified that EE, in terms of use, is important in predicting program engagement, but these studies did not capture the full breadth of engagement (ie, as proposed by Perski et al [15]) and were blended-care or therapist-supported interventions, rather than self-help. In a blended-care CBT intervention for adults with anxiety and depression, where participants had both face-to-face video therapy and digital activities (eg, thought monitoring and mood diaries), participants were around 10 times more likely to withdraw from the intervention if they had not completed the assigned dMH activities after the first session [17]. However, unlike this study, Wu et al [17] defined engagement purely as the number of sessions and web-based activities completed. Another study by Wallert et al [18] demonstrated the importance of early use factors in a therapist-supported dMH program, however, that was limited to early word count and homework tasks only. The study by Wu et al [17] found that higher word count in the first homework tasks was associated with greater adherence (measured in terms of completed homework activities). Interestingly, EE factors included in this study were not associated with completed sessions, but rather, what adolescents did in the program (ie, the amount of in-session and homework activities and character count). This may reflect that participants get what they want or need, without having to complete the entire program (ie, symptom relief or education about anxiety). Alternatively, these findings may suggest that we have not adequately measured all aspects of engagement that relate to treatment outcomes. These results are consistent with findings by both Alberts et al [26] and Yardley et al [14] who found that program engagement may not necessarily predict clinical outcomes. Overall, this study’s findings emphasize the importance of understanding more about EE, its relationship to program completion and outcomes, and the strategies to enhance it.

Contemporary models of engagement highlight the need to consider both user experience and use [14,15]. This study, whilst meaningful in highlighting the importance of EE (program use) in dMH programs for adolescents, did not capture all facets of engagement (eg, user experience), which may explain the small effect sizes found. There is a large variation in the way in which engagement is defined and measured across dMH programs [12], and engagement has not often been measured holistically. This study primarily focused on the use aspects of engagement rather than the experiential or affective elements. The assessment of user experience would allow the opportunity for a more nuanced understanding of engagement, where participants’ own subjective experience of the program is measured [14,15]. For example, this could include self-report questionnaires or interviews on engagement and experience with the program, or examination of the content and quality of activity responses rather than just character count [15], which may allow a richer understanding of the factors predicting OE, reasons for disengagement, and inform the development or refinement of dMH programs to achieve higher levels of engagement.

This study holds several strengths. First, the study had a large sample size of adolescents taking part in a real-world effectiveness study. The study further explored novel predictive factors of engagement in dMH programs, filling a significant gap in the literature regarding our understanding of adolescent engagement, particularly adolescent use, in these programs. The results were also analyzed using both a conservative and traditional significance level, adding merit to our findings. Despite its strengths, there were some limitations inherent to the study. First, this study did not consider the predictors of people who fail to take up BRAVE Self-Help, for example, factors at registration, or in the first session, such as participant perceptions of the content being valuable and helpful, time available to complete the program, and beliefs about the need for treatment. These were not the focus of this study; however, they should be the focus of future research, with specific adaptations to collect data of this nature. Furthermore, while this study considered engagement as multifaceted, the components of engagement analyzed in this study were limited to the available data that has been routinely collected within the BRAVE Self-Help platform, and thus, not all components of engagement as suggested in the engagement models of Perski et al [15] and Yardley et al [14] could be examined. Specifically, this study focused on understanding multiple measures of program use, but did not address user experience, which should be analyzed in future research. Further, whilst this study did examine the role of anxiety severity in influencing engagement, it did not examine whether changes in anxiety experienced beyond EE and over the course of treatment, had additional influence on OE. It is possible that as anxiety symptoms reduce, participants would be more or less likely to continue to engage in the program, and this should be examined in future research. Finally, whilst not possible to include all potential psychosocial factors in this study and analysis, engagement may have been influenced by other factors, such as satisfaction with the program, personality, socioeconomic status, and previous exposure to treatment.

This study highlights the need to better understand how to measure the construct of engagement and investigate the role of EE in supporting OE. It is recognized that facets of engagement may overlap and be interrelated (eg, completed tasks and completed sessions), and therefore, conducting a factor analysis may be useful in identifying whether these variables are discrete, and may assist in the empirical validation of proposed engagement models such as that of Perski et al [15] and Yardley et al [14]. Additionally, analysis of predictors of further potential facets of engagement, such as user experience as proposed by Perski et al [15] (including interest in the program and attention sustained within the program, affect, and satisfaction) will be useful in more broadly understanding the construct of engagement and its relationship to EE, anxiety severity, and outcome. Subsequent studies may also consider the use of digital linguistic tools, which may be useful in providing more information about whether sentiment and affect relate to engagement. Future research should also seek to understand how to enhance EE. This could be achieved through qualitative research with adolescents themselves, to better understand the unique needs of the user, and gain a detailed, nuanced account of how to enhance EE. Adding to our understanding will allow for dMH programs to be refined, tailored, and developed with such findings in mind. Taken together, the application of the findings of this study could improve engagement, and ultimately, potential clinical outcomes for adolescents with anxiety.

To date, there has been little research investigating the factors influencing engagement in dMH interventions for adolescents. The intervention-specific factor of poor EE, in terms of early program use, was found to be a barrier to CE in this study. Understanding how to better operationalize and measure engagement, along with identifying areas of focus to enhance EE, will help refine adolescent dMH interventions and aid in the design and development of future interventions. By doing so, user engagement can be enhanced and the benefit to the user of the program can be maximized.