Adolescence is marked by considerable emotional, social, and physical changes, including increasing autonomy from parents, greater influence from peers, and increased likelihood to engage in risk-taking behaviors [1]. Corresponding to this period of experimentation, harms associated with risky behaviors such as alcohol and other drug (AOD) use peak during adolescence and early adulthood [2-4]. Globally, alcohol, tobacco, and cannabis are the most commonly used drugs among adolescents [4]. This also holds true for Australia, where a 2017 national survey among secondary school students aged 12 to 17 years showed that 46% had consumed a full serve of alcohol, 14% had used cannabis, and 13% had smoked cigarettes in the past year [5]. Early initiation of AOD use is associated with a range of negative outcomes including poor school performance, truancy, school dropout, juvenile offending, and increased risk of drug dependence and mental illness during adulthood [6-9]. To interrupt this trajectory and reduce the harms associated with AOD use, effective strategies are needed to prevent the onset and escalation of their use.

Studies have identified a number of modifiable individual risk factors that are consistently associated with AOD use among adolescents [10]. They include peer AOD use, low self-efficacy to refuse alcohol or other drugs, poor school engagement and connectedness, and mental health disorders such as depression and attention-deficit/hyperactivity disorder [10-12]. Studies also highlight the importance of parents in influencing and preventing adolescents’ AOD use. Recent evidence shows an association between parental supply of alcohol and increased risk of alcohol-related harm during adolescence and early adulthood [13,14]. This has challenged the commonly held perception that allowing teenagers to drink alcohol under parental supervision protects them against alcohol-related harms. Similarly, less restrictive parental attitudes toward alcohol use are likely to lead to earlier and more frequent alcohol use and increased drunkenness among adolescents [15].

Prevention programs targeting individual and parental risk factors are effective in reducing AOD use among adolescents [16-22]. A number of prevention programs implemented during secondary school have consistently demonstrated effectiveness in reducing AOD use [18,21], and school-based delivery offers a number of advantages including tailoring of messages to students’ developmental level and universal delivery to achieve wide reach [22,23]. Internet- and computer-based AOD prevention programs offer additional benefits in terms of engaging youth and increasing implementation fidelity and have been found to be effective in reducing adolescent AOD use [17]. Despite the growing evidence base, only a small proportion of schools implement effective AOD prevention strategies. A 2003 review of 3 decades of AOD education studies concluded that worldwide, effective AOD prevention is not widely implemented [24]. More recently, our 2017 survey of Australian schoolteachers found that <25% of teachers had implemented evidence-based AOD education programs. The study identified lack of confidence, resources, time, and support from school; attitude of parents and students; and difficulty in communicating as the main barriers to implementing programs [25].

Parenting strategies that are consistently associated with delayed initiation of alcohol use include parental monitoring, limited access to alcohol, parent-child relationship quality, parental involvement, and communication [20,26]. However, studies show that parents do not always act in accordance with evidence-based parenting strategies, which may be related to lack of clear guidance or confidence [27,28]. Although parents actively seek information about illicit drugs and parenting practices to prevent AOD use, primarily from friends or the internet, they often report low-to-moderate confidence in communicating and influencing their children’s choices regarding AOD use [29]. Programs aimed at modifying parenting practices and promoting the use of effective prevention strategies have shown promise in reducing adolescent AOD use [30-32]. Honest communication between parents and adolescents, established on the basis of positive parent-adolescent relationship, has been associated with reduced alcohol use among adolescents [32-34]. This highlights the importance of engaging parents in efforts to reduce harms associated with adolescent AOD use.

It is critical that effective AOD prevention strategies are implemented consistently and widely to alleviate the considerable burden associated with AOD use. Therefore, teachers, school staff, and parents need to have easy access to evidence-based information, strategies, and programs that equip them to respond most effectively. The Positive Choices national AOD prevention initiative was funded by the Australian Government Department of Health to enhance access to and implementation of evidence-based AOD prevention strategies within school communities. The initiative responds to a call from school principals for support to implement evidence-based AOD prevention resources (Australian National Council on Drugs; 2013) and was developed iteratively with experts and end users. School staff, parents, and students provided input and feedback across two phases (for full details refer to the study by Stapinski et al [25]): (1) formative consultation to clarify scope and identify user needs and (2) review and feedback on a beta version of the website [35]. The final website was launched in December 2015 and provides web-based training, support, and access to a centralized database of evidence-based AOD prevention programs, recommendations, and resources [25]. The website emphasizes the importance of implementing resources that are supported by research evidence—only evidence-based resources are listed, and each resource page provides a rating to indicate the strength of the supporting evidence according to the Australian National Health and Medical Research Council’s evidence hierarchy [36]. To facilitate implementation, resources are categorized based on purpose and intended audience in a searchable database, with brief factsheets to guide users about when and how to implement evidence-based prevention strategies. A survey was conducted 8 months after the launch to determine the initial impact of Positive Choices. Among teachers who accessed the website, most found it useful and reported that they would continue using it, would recommend it to others, felt more comfortable discussing AOD use prevention following website access, and felt that their students were more engaged with AOD education since using the website [25]. When compared with a general teacher sample, teachers who used Positive Choices were more likely to consider the evidence base and effectiveness of AOD education resources before using them in class [25]. Despite highlighting these benefits associated with the use of Positive Choices, this initial evaluation did not reveal the specific factors or features that facilitated the dissemination of evidence-based AOD prevention strategies.

In this study, we extended this initial evaluation by conducting a more comprehensive examination of the effectiveness and impact of the Positive Choices initiative over a 3-year period, between 2017 and 2019. This is the first evaluation of a web-based health initiative that specifically promotes translation of evidence-based AOD prevention resources to school communities, parents, and youth. To guide this evaluation, we applied the reach, effectiveness, adoption, implementation, and maintenance (RE-AIM) framework, which was developed to facilitate comprehensive and rigorous evaluation of health promotion initiatives, spanning these 5 key dimensions. It has been widely applied to evaluate the implementation and real-world impact across a variety of settings, including educational settings [37-40]. Guided by this framework, this study assessed the impact of the Positive Choices initiative in increasing awareness and implementation of evidence-based AOD prevention strategies. The study aimed to evaluate the reach of the Positive Choices website, its effectiveness in improving access to and uptake of effective AOD prevention strategies, and its adoption in accessing evidence-based prevention strategies. Furthermore, implementation will be evaluated by users’ capacity to interact with the website, and maintenance will be evaluated by users’ intention to continue using the website for evidence-based strategies. The study also obtained feedback from end users to improve and optimize the Positive Choices website.

Similar to the effectiveness dimension, the adoption and implementation dimensions were assessed via the web-based evaluation surveys. Adoption was assessed by identifying barriers to and enablers of access and uptake of evidence-based prevention strategies by school staff and parents. A single question assessed which of the following characteristics of web-based AOD prevention resources or information were most valued by school staff: (1) evidence-based information, (2) resources that had been tested in schools and proven to prevent AOD use, (3) engaging website, (4) interactive website, (5) website that is easy to navigate and use, and (6) simple and easy-to-understand language. Similarly, enablers for parents and guardians were identified through the same items with the addition of the following two items that pertained specifically to parents: (1) parental strategies that have been proven to be effective and (2) website with advice from other parents.

Implementation was evaluated using the System Usability Scale (SUS) [41]. It is a standardized instrument used to measure the usability of products, software, apps, and websites. It provides participants with 10 usability-related statements on a 5-point Likert scale, ranging from “strongly disagree” to “strongly agree.” SUS scores range from 0 to 100, with score ≥85 representing exceptional usability, score between 50 and 70 representing good usability, and score <50 representing unacceptable usability [41,42]. As such, participants’ capacity to interact with the website to access and subsequently deliver evidence-based strategies was assessed using the SUS scores.

This dimension was assessed using the website evaluation surveys, as described previously. For the participating school staff and parents, the question assessed whether users intended to access Positive Choices in the future. It was conceptualized as follows: Did participants intend to maintain their use of the Positive Choices resources?

To assess school staff and parents’ general feedback about the Positive Choices website, the evaluation survey included the following open-text items: (1) Do you have any suggestions for improving the website? and (2) Do you have any final comments about the website?

The evaluation surveys were administered via Survey Monkey, and the data were exported to Stata (version 14; StataCorp), which was used to generate descriptive data, including mean scores and response frequencies. Data from the 2017 and 2019 evaluation surveys were merged to examine reach, effectiveness, adoption, implementation, and maintenance, using descriptive statistics, separately for school staff and parents. The SUS score was calculated by converting raw individual scores for each question to a number; these scores were summed to obtain the total score, 5 was subtracted from the total score of all odd-numbered questions, and 25 was subtracted from the total score of all even-numbered questions. Then, the total score of the new values was multiplied by 2.5 [43].

Data on community awareness were also collected via Survey Monkey and exported to Stata to generate descriptive data. The data were examined separately for school staff and parents at baseline, 6 months, and 12 months.

Inductive analysis was used to assess participants’ feedback about Positive Choices [44], using the qualitative data analysis software, NVivo (version 12; QSR International). Feedback from participants was examined by SN, who then developed a coding framework. Using the framework, common themes were identified independently by two coders (SN and TG).

The 2017 survey was approved by the Human Research Ethics Committee, University of New South Wales (project number HC12548), and the 2019 survey was approved by the Human Research Ethics Committee, University of Sydney (project number 2018/873).

From 2017 to 2019, a total of 200 participants completed the evaluation surveys, of which 73 (36.5%) participants were school staff and 127 (63.5%) were parents.

Adolescence is a time of increased susceptibility to engaging in risk-taking behaviors such as AOD use. Prevention strategies designed to target modifiable risk factors have been demonstrated to be effective in reducing AOD use and related harms among adolescents [16,18]. Despite this evidence, effective AOD prevention strategies are not widely implemented in schools or by parents. Positive Choices was developed to help overcome some of the barriers faced by school communities and parents and promote widespread implementation of evidence-based prevention. This study used the RE-AIM framework to evaluate the success of Positive Choices in increasing awareness and implementation of evidence-based AOD prevention practices among school staff and parents across the five domains: reach, efficacy, adoption, implementation, and maintenance.

A total of 5 years following its launch, this Australian AOD prevention website has reached >1.7 million users, and the page views have continued to grow. The user reach of Positive Choices has expanded beyond Australia, with 59% of users currently being international. Promotion via a national social media campaign proved to be an effective method for increasing awareness within the Australian community, with awareness increasing from 8% to 14% among school staff and from 15% to 22% among parents after the campaign. The findings suggest that sustained promotion efforts may be required to maintain high levels of awareness about Positive Choices. Even among participants who were not using Positive Choices, most reported using other web-based sources to obtain AOD education information and resources, thus supporting the value of the internet as a tool for disseminating health information and promoting evidence-based prevention strategies [45,46].

Although evidence shows that effective AOD prevention strategies are not commonly implemented by schools [24] or parents [27], most school staff and parents in this study reported that they were already implementing evidence-based AOD prevention strategies. Moreover, following a brief interaction with the website, most school staff and parents reported an intention to shift toward evidence-based practices in cases where they were not already following the recommended prevention strategies for young people. The exception was the recommendation that parents modify their own alcohol use to model appropriate behavior for their children, which may indicate that changing their own behavior is a significant challenge for parents, which requires additional attention and support. The sample intended to maintain their use of the Positive Choices website in the future to obtain evidence-based information and resources. The enablers that facilitated adoption and implementation were ease of availability and good website usability. Both user groups reported high level of confidence in communicating about AOD-related topics. Study participants’ suggestions for improvement pertained to some of these enablers (eg, improving navigation and making the website more engaging), and thus, have informed a recent website update.

The widespread use of web-based health promotion tools and websites by school staff and parents to access information and resources highlights the need for quality control measures. This will ensure that they are evidence-based, up to date, and engaging and use simple language. In addition, the websites themselves should be accessible and easy to use and navigate. Findings from robust evaluations will allow critical assessment of the benefits of such tools and websites, inform content, and inform website updates and developments to optimize their usability. Although there have been evaluations of mental health information websites [47,48], to the best of our knowledge, this is the first evaluation of a web-based health initiative that specifically promotes the translation of evidence-based AOD prevention resources for school communities, parents, and youth.

A limitation of the study is that the findings rely on users’ self-reported intentions to implement evidence-based strategies, rather than actual assessment of their subsequent behavior. Furthermore, the study design assessed participants’ feedback and behavioral intentions after they interacted with the website for a relatively short period. Future studies with a pre-post study design will enable a more comprehensive evaluation, including assessment of whether the website affected subsequent behaviors and implementation of evidence-based strategies by school staff and parents. Another limitation is that the targeted campaign used to recruit participants for the evaluation study may have resulted in a sample selection bias. Most of the surveyed school staff and parents in the current sample reported that they were already implementing evidence-based prevention strategies and were confident about discussing alcohol and drug use. These results contrast with previous evidence suggesting low confidence and implementation of evidence-based AOD prevention strategies among parents and in schools [24,27], and thus, may reflect that our recruitment strategies attracted a sample who were already interested in and aware of evidence-based AOD prevention approaches.

The findings from the evaluation of Positive Choices demonstrate that the website reached 1.7 million users, and it has the capacity to be an effective strategy for disseminating evidence-based AOD prevention information and resources. Furthermore, this evaluation highlights the importance of investing in ongoing promotion to maintain or enhance awareness of health websites. As the use and acceptability of health education websites increase, developers and health care and research teams should ensure that health websites developed in the future are easy to navigate, are engaging, use simple language, contain evidence-informed resources, and are supported by ongoing promotional activities. This study provides methodology and recommendations to guide future evaluations of web-based health tools to determine their effect on behavioral and health outcomes.