Published on in Vol 7 (2024)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/62919, first published .
Recruitment of Adolescents to Virtual Clinical Trials: Recruitment Results From the Health4Me Randomized Controlled Trial

Recruitment of Adolescents to Virtual Clinical Trials: Recruitment Results From the Health4Me Randomized Controlled Trial

Recruitment of Adolescents to Virtual Clinical Trials: Recruitment Results From the Health4Me Randomized Controlled Trial

1Susan Wakil School of Nursing and Midwifery, Faculty of Medicine and Health, The University of Sydney, Western Avenue, Camperdown, Australia

2InsideOut Institute for Eating Disorders, Faculty of Medicine and Health, The University of Sydney, Sydney, Australia

3Department of Endocrinology, Nepean Hospital, Nepean Blue Mountains Local Health District, Kingswood, Australia

4Department of Health Sciences, Faculty of Medicine, Health and Human Sciences, Macquarie University, Sydney, Australia

5The George Institute for Global Health, University of New South Wales, Sydney, Australia

6Institute for Evidence-Based Healthcare, Bond University, Robina, Australia

7Charles Perkins Centre, The University of Sydney, Camperdown, Australia

8see Acknowledgments

Corresponding Author:

Rebecca Raeside, BBiomedSc, MPH


Background: Preventive interventions are needed to provide targeted health support to adolescents to improve health behaviors. Engaging adolescents in preventive interventions remains a challenge, highlighting the need for innovative recruitment strategies. Given adolescents’ lives are intertwined with digital technologies, attention should be focused on these avenues for recruitment. The evolving nature of clinical trials, including the emergence of virtual clinical trials, requires new recruitment approaches, which must be evaluated.

Objective: This study aimed to examine the effectiveness and cost of various digital recruitment strategies for recruiting adolescents to a virtual clinical trial, evaluate the progression of participants from screening to enrollment, and explore factors associated with nonparticipation. This was conducted using data from the Health4Me Study, a preventive digital health intervention to improve physical activity and nutrition behaviors among adolescents aged 12 to 18 years.

Methods: Participants were recruited into the Health4Me Study via social media advertisements on various contemporary platforms, emails to schools, emails to contacts within known networks, and emails to relevant youth organizations. Data were collected from social media advertisements, screening, and recruitment logs. Data analysis included summary and descriptive statistics, as well as chi-square tests to explore factors associated with nonparticipation.

Results: From 2369 expressions of interest, 390 (16.4%) participants were enrolled. A total of 19 advertisements were placed on social media, and 385 promotional emails were sent to schools, contacts within known networks, and relevant youth organizations. Social media advertisements reached 408,077 unique accounts. Advertisements mostly reached those living in populous states in Australia (306,489/408,077, 75.11% of unique accounts) and those identifying as female (177,698/408,077, 43.55% of unique accounts). A total of 24.97% (101,907/408,077) of advertisements were delivered to accounts with uncategorized genders. The total cost per participant enrolled was Aus $3.89 (approximately US $2.58). Most participants (1980/2305, 85.90%) found out about this study through Instagram. Differences in screening characteristics between eligible participants who did and did not enroll were found to be statistically significant for gender (P=.02), with fewer males and more individuals reporting their gender as “other” enrolling than expected by chance alone. The recruitment method also differed (P<.001), with fewer participants enrolling through Instagram and more enrolling through other methods (eg, known networks or word of mouth) than expected by chance alone.

Conclusions: This study found that virtual clinical trial recruitment was found to be low-cost, with the potential to increase trial participation. Social media was the most effective recruitment method, reaching all states and territories, including hard-to-reach populations. Future action is needed to explore recruitment methods that are more effective for males and to build trust among adolescents regarding clinical trial recruitment via social media.

Trial Registration: Australia New Zealand Clinical Trials Registry ACTRN12622000949785; https://www.anzctr.org.au/Trial/Registration/TrialReview.aspx?id=383576&isReview=true

JMIR Pediatr Parent 2024;7:e62919

doi:10.2196/62919

Keywords



Adolescence is regarded as the second window of opportunity—a critical period to intervene and provide targeted support to improve health outcomes that have a profound impact on health and well-being throughout the life course [1]. Failure to invest in primary prevention among today’s adolescents will increase the burden of chronic diseases and the existing sizable total health expenditure of Aus $24 billion (approximately US $15.9 billion) on potentially avoidable risk factors [2]. It is important that high-quality public health interventions that focus on primary prevention of chronic diseases are tested with adolescents through clinical trials. However, challenges exist with engaging adolescents in preventive interventions, including their health system disengagement making them hard to access for delivering such interventions [3], a prevention lens not being appealing to adolescents [4], and their evolving need for autonomy in providing informed consent [5]. Innovative methods to engage adolescents within preventive interventions are needed that can overcome identified barriers.

Adolescents’ lives are increasingly intertwined with digital technologies such as mobile phones and the internet [6]. With that comes the opportunity to harness developments in digital technologies for innovative preventive interventions [7]. The use of digital methods for recruitment to clinical trials is increasing in popularity and they are particularly beneficial for recruitment to online clinical trials. Previous research has focused on comparing social media or other digital strategies to traditional in-person recruitment [8-13], and the use of digital tools for recruitment and retention of clinical trial participants [14,15]. While research shows that digital recruitment strategies are effective compared to traditional in-person recruitment, limited research is available to understand the efficiency of digital recruitment strategies alone and their impact on clinical trial participants and investigators (eg, helping investigators identify eligible trial participants) [14]. A previous review identified that Facebook (Meta) is effective for recruiting adolescent participants [16], and a cross-sectional study revealed that the use of Instagram (Instagram from Meta) and Snapchat (Snap Inc) may also be useful and cost-effective to recruit young people to surveys [17], but limited evidence is available for the use of these contemporary platforms for adolescent recruitment to clinical trials. As more social media platforms become available and others diminish in popularity, it is crucial that research is undertaken to understand their effectiveness for recruiting adolescents to research.

Additional complexities occur when there are no physical recruitment sites, otherwise known as remote [18], decentralized [19], or virtual [20] clinical trials, from hereon in called virtual clinical trials. Virtual clinical trials can leverage digital technologies for participant recruitment and retention, enabling online consent for participants, on-time data collection, and delivery of the intervention that is convenient for participants, as they do not have to travel to a physical site [21]. Virtual clinical trials among adolescent participants have the potential to overcome some of the previously identified barriers, including reaching those who are disengaged with the health system and reaching adolescents directly [22,23], allowing them autonomy in making decisions about their health [24], including providing informed consent (depending on ethics approvals). However, there is limited research to understand digital recruitment strategies for clinical trials among adolescents. Furthermore, it is also important to understand factors that may cause eligible participants not to engage in digital preventive interventions. Reporting will enable future research to tailor recruitment toward the most effective digital strategies and address factors that cause disengagement. Therefore, this study aimed (1) to examine the effectiveness and cost of various digital recruitment strategies for recruiting adolescents to a virtual clinical trial, (2) to evaluate the progression of participants from screening to enrollment, and (3) explore factors associated with nonparticipation.


Study Design

The Health4Me study was used as the context for this research. The full protocol is published elsewhere [25]. In brief, the Health4Me Study is a virtual clinical trial, based in Australia, of a community-based, 6-month text message intervention. The intervention aims to improve physical activity and nutrition behaviors among adolescents aged 12 to 18 years.

Ethical Considerations

Primary ethics approval was received from the University of Sydney Human Research Ethics Committee (2022/402), and the trial is registered at the Australia New Zealand Clinical Trials Registry (ANZCTR; ACTRN12622000949785; date registered: July 5, 2022).

Participants and Eligibility Criteria

Participants were eligible to take part in the Health4Me Study if they were (1) aged 12 to 18 years, (2) owned a mobile phone capable of sending and receiving text messages, (3) had an Australian mobile phone number, (4) had sufficient English proficiency to read text messages pitched at a 7th grade reading level, and (5) provided electronic consent (or from their parents or guardians if they were aged <14 years). Participants were excluded from this study if they (1) had a diagnosis of type 1 or type 2 diabetes mellitus, (2) had a previous or current diagnosis of an eating disorder or were at high risk for an eating disorder as assessed in screening, (3) weighed <25th centile for their age, (4) had recent rapid weight loss, (5) had a medical condition that would preclude providing informed consent or ability to comply with this study’s protocol, (6) were enrolled in an alternative randomized lifestyle management program, (7) were pregnant or planning to become pregnant during the 6-month intervention, and (8) were unable to read English at a 7th grade reading level. The eligibility criteria for the Health4Me Study have been published elsewhere [25].

Given the Health4Me Study was conducted virtually, several steps were embedded to ensure participants could safely enroll into this study. To complete screening procedures, the research team partnered with the InsideOut Institute for Eating Disorders, a team of researchers and clinician experts in eating disorders based at the University of Sydney. Potential participants first expressed interest to take part in this study by filling out the Expression of Interest (EOI) form on REDCap (Research Electronic Data Capture; Vanderbilt University), which included contact details and screening against the eligibility criteria, as well as screening for eating disorder risk using two validated questionnaires—InsideOut Institute Screener (IOI-S) and Eating Disorder Examination Questionnaire (EDE-Q) [26,27]. Study specific cut points were set for the IOI-S (≥16) and EDE-Q (>3 and any of behavioral questions 15‐18 endorsed ≥1). Potential participants first completed the IOI-S, if they scored below the cut point and met all inclusion criteria, they were sent the e-consent form. However, if participants scored above the cut point on the IOI-S, they were directed to complete an EDE-Q. If potential participants scored under the cut point on the EDE-Q, they were deemed eligible and sent the e-consent form. If a potential participant was detected to be above the cut point on the EDE-Q, they were referred to the InsideOut Institute for Eating Disorders for an assessment to determine suitability to participate by eating disorder expert clinicians (clinical psychologist or registered clinical psychology students with expertise in eating disorders) via phone call. If they received clearance from the eating disorder expert clinicians, they were sent the e-consent form and deemed eligible to enroll in this study. If they did not receive clearance, they were sent an email with various resources for eating disorder support. If a potential participant did not meet other inclusion criteria, they were sent an email explaining why they were ineligible. All participants provided informed e-consent (and from their parents or guardians if they were aged <14 years) before baseline measures were collected [25]. Participants were randomized once all baseline measures were complete.

Recruitment

The protocol was to enroll 390 participants—195 per arm—based on detecting a mean difference in moderate to vigorous physical activity minutes per day of 14.8 (control: 42.55 and intervention: 57.36) with an SD of 21.45 for control and 37.79 for intervention or a 13.37% difference in the proportion of appropriate vegetable consumption (control: 4.85% and intervention: 18.22%) with 90% power and accounting for 30% dropout. The Bonferroni adjusted significance level of 0.025 was used to account for two primary outcomes. The participant information statement detailed that participants would receive an Aus $30 (approximately US $19.90) gift voucher at the completion of all baseline assessments as a reimbursement for their time. A recent review has suggested that financial incentives can be provided to children appropriately, and few studies suggest incentives are inherently harmful [28]. Recruitment methods are detailed below.

Recruitment Methods

Overview

Recruitment ran from February 2023 to February 2024 using a range of methods including social media advertising on Facebook, Instagram, TikTok, and Twitter/X, emails to schools, emails to contacts within known networks, and emails to relevant youth organizations. A dedicated study website was also created to establish legitimacy.

Social Media Advertisements

Initially, study-dedicated Facebook and Instagram pages were established with this study’s logo, study contact information and detailed the purpose of this study. Posts were made on Instagram to establish this study as an authentic and active social media account. All content and this study logo were co-designed with adolescents [29]. All advertisements were created using ethics approved text and images on Meta Ads Manager, which simultaneously promoted advertisements on Facebook and Instagram or on TikTok for Business, which promoted advertisements on TikTok. Due to restrictions in advertising to people aged younger than 18 years [30], all advertisements were targeted only for people aged 13 to 18 years in Australia. Examples of the social media advertisements (images and text) are available in Multimedia Appendix 1. Advertisements on Meta were run for a maximum of 2 weeks, with a maximum budget of Aus $20 (approximately US $13.30) per day. The single advertisement on TikTok was run for 4 days, with a lifetime budget of Aus $50 (approximately US $33.20). A single post was made on Twitter/X by a member of the research team. All advertisements linked directly to this study EOI form, hosted on REDCap.

Emails

Emails were sent to schools, known networks and contacts of the research team and, relevant youth organizations, for example, headspace. Emails contained a link to this study’s website and this study’s REDCap page.

Study Website

Previous formative work by the research team revealed that adolescents desire online health information that is credible and reliable [31]. A study website was created to establish this study as legitimate and to build trust among adolescents. This study’s website contained this study’s logo, study contact information, detailed the purpose of this study, how to become involved (including a direct link to this study’s REDCap page), photographs and names of the key researchers and names of the wider research team. Potential participants could also access the full participant information sheet through this study’s website.

Data Sources

Social Media Advertisements

Data were available and collected from Meta Ads Manager. For each advertisement, data were collected on the number of days the advertisement ran, advertisement strategy used, reach, impressions, link clicks, cost per result, and total amount spent (Aus $). Deidentified advertisement audience demographic data included location, age, and gender. User’s location was based on their state or territory (New South Wales [NSW], Victoria, South Australia, Queensland, Western Australia, Northern Territory, Tasmania, and Australian Capital Territory). Age and gender data were based on what social media users disclose on their user profiles and were summarized according to Meta Ads Manager categories (age: 13‐17 or 18‐24 years; gender: male, female, or uncategorized). Data were available and collected from TikTok for Business. For each advertisement, data were collected on the number of days the advertisement ran, reach, impressions, link clicks, cost per result, and total amount spent (Aus $). Post analytics were available and collected from Twitter/X. For the single post, data were collected on likes, reposts, impressions, and link clicks.

Recruitment Log

A log was kept of all dates on which emails were sent to schools, known networks and contacts, and youth organizations. Data were also collected on the number of people who visited this study’s REDCap page each day to express interest and the number of enrollments. Detailed notes were kept on the log by the research team.

Screening Logs

A detailed log was kept of all participant inquiries. The secure online REDCap [32] database collected data, including age (12-14 years or 15-18 years), gender (male, female, other, or prefer not to say), high school attendance (yes or no), height and weight (for BMI calculations, categorized as underweight, healthy weight, above a healthy weight, or well above a healthy weight) [33,34], and recruitment method. The responses for recruitment method included (1) Facebook, (2) Instagram, (3) Twitter/X, (4) TikTok, (5) other social media platform, (6) headspace, (7) general practitioner or doctor, and (8) other. The screening log also contained details on eligibility and reasons for exclusion. A further screening log was also collected from the InsideOut Institute for Eating Disorders, which kept a detailed log of potential participants requiring screening for eating disorder risk. The secure online REDCap database allowed both the psychologists and research team to make comments. Potential participants were contacted a maximum of two times by the eating disorder expert clinicians. If contact was not established after two attempts, they were marked as ineligible and sent resources via email.

Data Analysis

Summary statistics regarding social media data are presented. Total costs (Aus $) are reported for social media advertisements, with the average cost calculated per participant eligible and per participant enrolled. Descriptive statistics for continuous measures, including counts and percentages for recruitment method, were used to summarize the breakdown of potential participants who inquired and participants screened by the InsideOut Institute for Eating Disorders.

To explore factors associated with nonparticipation, differences in screening characteristics between eligible participants who did and did not enroll in this study were compared using chi-square tests. The significance level was set at 5%. Characteristics included age, gender, BMI, high school attendance, and recruitment method. Adjusted standardized residuals (ASRs) were used to measure the strength of the difference between observed and expected values. Data were analyzed using IBM SPSS (version 29.0; IBM Corp).


Effectiveness and Cost of Recruitment Strategies

A total of 2369 entries were made to the EOI form. Of those, 2305 respondents completed the question asking how they heard about this study. Most (1980/2305, 85.90%) found out about this study through Instagram, followed by other (182/2305, 7.9%) and then Facebook (112/2305, 4.9%). The full sample size of 390 adolescents was reached in 12 months.

For the Health4Me Study, there were 17 advertisements run on Meta Ads Manager over 12 months. The length at which the advertisements were running for ranged from 2‐18 days, with advertisements running for a total of 146 days. Overall, advertisements reached a total of 408,077 unique Meta accounts and were viewed >2.3 million times. A cost-per-link-click strategy was employed in 16/17 advertisements, and one employed a cost-per-post-engagement strategy. Across 17 advertisements, 7211 link clicks were made. Advertisements mostly reached people in NSW, Victoria, and Queensland, accounting for 75.11% (306,489/408,077) of the audience. With regard to age, 96.89% (395,403/408,077) of the advertisement audience was 13‐17 years old. For gender, advertisements mostly reached females (43.55%, 177,698/408,077); however, a quarter (24.97%, 101,907/408,077) of the advertisements were delivered to accounts with uncategorized genders. One advertisement was run through TikTok for Business, which ran for 4 days, reached 8386 unique accounts, and was viewed 14,832 times, with 144 link clicks made. No further data were available. One advertisement was placed on Twitter/X, which was viewed a total of 1041 times and reposted 11 times, with 15 link clicks made.

The overall cost of social media advertisements run through Meta was Aus $1478.63 (approximately US $965.69). Cost-per-link-click ranged from Aus $0.03 to Aus $0.87 (approximately US $0.02 to US $0.54). The overall cost of the advertisement through TikTok was Aus $39.97 (approximately US $26.10), with cost-per-link-click at Aus $0.28 (approximately US $0.18). In total, Aus $1518.60 (approximately US $991.80) was spent on social media advertisements. Cost per eligible participant was Aus $1.64 (approximately US $1.09), and cost per participant enrolled was Aus $3.89 (approximately US $2.58). A full breakdown of all social media data is available in Tables 1 and 2.

Emails requesting inclusion in school communications to students were sent to 367 high schools across NSW. One school announced this study at their school assembly. One email was sent to our mailing list of young people who have expressed interest in future research. Emails were sent to 17 other known networks, contacts, and youth organizations. Of those, the research team was made aware that one shared in their general practitioner newsletter, and one shared within their local health district. All sharing through schools, known networks, contacts, and youth organizations was at no cost to the research team.

Table 1. Breakdown of Meta advertisements reach by state or territory, age, and gender.
Reach, n (%)
State or territory
 New South Wales123,543 (30.27)
 Victoria95,954 (23.51)
 Queensland86,992 (21.32)
 Western Australia44,808 (10.98)
 South Australia31,814 (7.8)
 Northern Territory5633 (1.38)
 Tasmania9858 (2.42)
 Australian Capital Territory5377 (1.32)
 Unknown4098 (1)
Age (years)
 13‐17395,403 (96.89)
 18‐2412,674 (3.11)
Gender
 Female177,698 (43.55)
 Male128,472 (31.48)
 Uncategorized101,907 (24.97)
Total408,077 (100)
Table 2. Breakdown of social media advertisements for the Health4Me study through Meta, TikTok, and Twitter/X.
Social media platform and advertisement start dateEnd dateDays advertisements live (n)Advertisement strategyaReachbImpressionscTotal cost (Aus $d)Daily budget (Aus $)Link clickse (n)Cost per result (Aus $)Post reactions (n)Post saves (n)Post shares (n)
Meta (Instagram and Facebook)
February 10, 2023February 28, 202318Post engagement14,12836,340112.9610680.03318451
March 2, 2023March 8, 20236Link clicks88,929138,40459.87105500.1128417
March 31, 2023April 15, 202316Link clicks70,992164,879115.66106750.1776519
April 28, 2023May 12, 202315Link clicks86,464237,153143.3107750.18106622
May 16, 2023May 28, 202813Link clicks93,409237,016123.73106410.197829
May 31, 2023June 14, 202315Link clicks117,056345,260144106800.2120819
June 15, 2023June 30, 202315Link clicks110,532307,182162.05107060.23127110
July 4, 2023July 18, 202314Link clicks96,816301,837143.73106640.226938
July 21, 2023July 25, 20234Link clicks72,945124,06380.86207310.111131323
July 28, 2023July 31, 20234Link clicks64,67299,32759.77204100.156678
September 1, 2023September 4, 20234Link clicks49,60076,72159.99203750.164169
October 13, 2023October 16, 20234Link clicks44,79572,06659.99204350.1451714
November 23, 2023November 25, 20233Link clicks18,48418,9954020720.56303
December 1, 2023December 3, 20233Link clicks35,27836,98941.1820850.48000
January 16, 2024January 21, 20246Link clicks818513,46362.74151030.61303
February 2, 2024February 5, 20244Link clicks15,31119,94044.3515510.871011
February 22, 2024February 23, 20242Link clicks114,365134,19624.45201900.136110
TotalsN/Af146N/A408,0772,363,8311478.63N/A7211N/A422462156
TikTok
August 25, 2023August 29, 20234N/A838614,83239.97101440.28700
Twitter
February 14, 2024N/AN/AN/AN/A10410015015011

aPost engagement: strategy aims to encourage users to like, share, comment on, or save the advertisement. Link clicks: strategy aims to encourage users to click on the advertisement URL link.

bNumber of times the advertisement is delivered to a unique account.

cNumber of times the advertisement is delivered in total (including being delivered multiple times to one account).

dThe conversion rate over the time of the study was approximately Aus $1=US $0.65.

eNumber of times the advertisement URL link was clicked.

fN/A: not applicable.

Progression of Participants From Screening to Enrollment

Figure 1 shows the progression of participants and dropout points from screening to enrollment. Of the 2369 entries on the EOI form, 655 were immediately excluded. Of the excluded entries, 41 did not have an Australian mobile number. These 41 entries were among the first 100 EOIs and were believed to be bots. After adding a reCAPTCHA to the REDCap screening survey, no further suspicious entries were received. Of the remaining, 462 entries were incomplete, 145 were duplicate entries, and 32 withdrew their EOI postscreening. In addition, 328 entries were ineligible, as they did not meet the inclusion criteria, with reasons for exclusion shown in Figure 1.

A total of 481 potential participants scored above the cut point on the IOI-S. A total of 292 participants were unable to be contacted to complete the EDE-Q (per protocol for further screening for eating disorder risk). A further 189 potential participants were referred to the InsideOut Institute for assessment from eating disorder expert clinicians. Of those, 104 were unable to be contacted by the eating disorder expert clinicians (and were therefore marked as ineligible), 38 were ineligible postcall as they were assessed as high risk. A further 25 potential participants withdrew at this step, and 22 potential participants were assessed as eligible by the psychologist and sent the e-consent form.

A total of 927 potential participants were eligible and sent the e-consent form. Of those, 506 were unable to be contacted and the e-consent form was never signed. The e-consent form was signed by 421 participants, and they were sent the baseline surveys, but 31 participants did not commence answering baseline questions, giving 390/2369 (16.4%) as participants enrolled in this study.

Figure 1. Progression of participants from screening to enrollment. AUS: Australian; ED: eating disorder.

Factors Associated With Nonparticipation

During screening, 537 potential participants were identified as eligible but did not proceed to enrollment, and 390 participants enrolled in the Health4Me Study. Chi-square tests indicated that there were statistically significant differences in screening characteristics between eligible participants who did and did not enroll in this study for gender and recruitment method. Gender differences were significant (χ23927=9.8, P=.02); ASRs indicated fewer males and more individuals reporting “other” enrolled than expected by chance alone. Additionally, the recruitment method was significant (χ23925=17.39, P<.001), ASRs indicated fewer participants enrolled through Instagram and more enrolled through other methods (eg, known networks or word of mouth) than expected by chance alone. No differences were observed for other screening characteristics. A full breakdown of screening characteristics between those who were eligible and did or did not enroll is available in Table 3.

Table 3. Comparison of screening characteristics between those who were eligible who did and did not enroll in the Health4Me Study.
Total eligible and not enrolled (n=537)Total enrolled (n=390)Chi-square (df)P value
Age (years)3.8 (1927).05
 12‐148041
 15‐18457349
Gender9.8 (3927).02a
 Male16692
 Female347274
 Other1117
 Prefer not to say137
BMIb2.1 (3878).55
 Underweight2116
 Healthy weight410279
 Above a healthy weight6051
 Well above a healthy weight2120
Attending high school0.5 (1927).82
 Yes464339
 No7351
Recruitment strategyc17.3 (3925)<.001a
 Facebook1822
 Instagram470301
 Other social media platformd45
 Othere4461

aStatistically significant.

bDue to being asked gender during screening and not sex assigned at birth, we are unable to accurately calculate BMI for those who listed their gender as “other” or “prefer not to say.”

cOne record missing from each for recruitment strategy. Total eligible and not enrolled (n=536), and total enrolled (n=389).

dDue to small numbers, categories of Twitter/X, TikTok, and other social media platform were combined.

eDue to small numbers, categories of headspace, general practitioner or doctor, and other were combined.


Principal Results

The Health4Me Study aimed at improving physical activity and nutrition behaviors among those aged 12 to 18 years. A total of 2369 EOIs were received, and 390/2369 (16.4%) participants were recruited in less than 12 months. Social media was the main source of recruitment. The research team did try to engage with schools, known networks, and relevant youth organizations via emails with limited success. Social media advertisements through Meta were effective, reaching 408,077 unique accounts across all states and territories in Australia. Overall, social media advertisements were low cost (Aus $3.89 per participant enrolled [approximately US $2.58]). From screening to enrollment, there were multiple points of dropout. Of the EOIs from potential participants who were eligible (927/2369, 39.1%), statistically significant differences were observed for those who did and did not enroll in terms of gender and recruitment method. Fewer males and more individuals reporting their gender as “other” enrolled than expected by chance alone. In addition, fewer individuals enrolled through Instagram and more enrolled through other methods (eg, known networks or word of mouth) than expected by chance alone.

Comparison With Prior Work

Virtual clinical trials have the potential to address challenges in traditional site-based recruitment and be cost-effective [20]. Yet, prevention programs among adolescents are known to have the lowest recruitment rates [35], and stakeholders have identified that a prevention lens may not be engaging for adolescents [4]. In the Health4Me Study, a digital preventive intervention, digital recruitment strategies that were employed were effective, recruiting 390 adolescents in less than 12 months. The Health4Me Study was guided by factors associated with successful recruitment from a previous virtual clinical trial [36], including (1) national recruitment, (2) self-referrals, (3) unmet need for trial intervention, (4) patient and public involvement, (5) regular monitoring and communication, and (6) reimbursement and early exclusion. In the Health4Me Study, a national sample was recruited and participants self-referred into this study. In addition, there are limited prevention programs currently available for adolescents [37], and the intervention and all advertising materials were co-designed with adolescents [29]. A small day-to-day research team was employed who communicated regularly through detailed screening and recruitment logs, and participants were reimbursed through online gift vouchers after completing all study activities at baseline and 6-month follow-up.

The costs reported in this study for social media advertising are lower per enrolled participant than what has previously been reported in reviews (approximately US $3-US $628) [9,38], however these studies mostly report on Facebook and compare social media to traditional in-person recruitment. Limited research is available reporting virtual clinical trial recruitment costs to recruit adolescents. A virtual clinical trial, which aimed to prevent and reduce cyberbullying among adolescents that used Instagram for study recruitment, found a higher consent rate than the Health4Me Study (24.4% vs 16.4%) yet had much higher social media advertisement costs (approximately US $19 versus approximately US $2.59 per enrolled participant) [39]. It is essential for future virtual clinical trials to report costs associated with recruitment to understand their cost-effectiveness for enrolling participants from the target population.

Virtual clinical trials allow remote access to research, potentially enhancing the diversity of participants, and recruiting from hard-to-reach populations [20]. In the Health4Me Study, it was observed that more individuals reporting their gender as “other” enrolled than expected by chance alone. This “other” category captures any gender other than male or female (eg, nonbinary or transgender). In another virtual clinical trial targeting cyberbullying found that nearly half of the participants recruited via Instagram identified as lesbian, gay, or bisexual [39]. It was also observed that less eligible males enrolled in the Health4Me Study than expected by chance alone. When looking at the social media advertisements, they reached less males overall. Evidence from large datasets demonstrates that females spend more time overall and more time per day on social media [40,41]. Additionally, another virtual clinical trial among an older population found that males were underrepresented [42]. Future efforts should be directed to identifying effective methods to recruit males to virtual clinical trials.

Another factor considered to attract hard-to-reach participants online is that those interested can self-refer into this study. A previous study, which aimed to assess effectiveness of online behavioral therapy for tics among young people, found that the majority of participants self-referred from online [36], enabling those who were not under the care of mental health clinicians to be included. The Health4Me Study is unique in that participants who are aged 15 years and older can consent themselves into this study, without the need for parent or guardian consent. This was approved by the ethics committee, with support from our youth advisory group as the Health4Me Study is a low-risk, preventive health intervention. This capacity to self-refer gives adolescents some autonomy around their health, especially given that preventive care is seldom given within primary care to this age group [43].

Within this study, less eligible adolescents enrolled through Instagram than were expected by chance alone, and more enrolled through other methods (eg, known networks or word of mouth) than what was expected by chance alone. Hypothesized reasons for this are around trust in health information that adolescents view online, adolescents being discouraged from sharing personal information online, and having poor knowledge and attitudes on clinical trials [44]. Previous reviews show that adolescents often distrust health information found online yet continue to engage with this information [45]. In terms of health information on social media, friends and networks are particularly important for gaining adolescents trust in this space [46]. Future studies planning to recruit adolescents through social media could explore the use of peer referral or endorsement from youth advisors or reputable organizations (eg, study sponsor) to gain an increased level of trust. In addition, adolescents are acutely aware of how their personal data are being used, and building trust and authenticity among this population is vital [31]. Partnerships with known youth health organizations and endorsement of the clinical trial through their own social media accounts may be useful to increase trust among adolescents.

Within the Health4Me Study, there were multiple points of dropout from screening to enrollment. The largest point of dropout was those who were eligible and sent the consent form but never responded. For all eligible participants, the research team sent the consent form twice via email, however after no response they were marked “unable to contact.” Though emails are a highly acceptable form of communication among adolescents [47], future efforts should be directed to streamlining processes of screening and consent for scalability of future trials. Previous studies have aimed to do this using mobile apps, for example, ResearchKit (Apple Inc) [48], which is an open-source software framework designed to streamline the process of screening and consenting participants into research studies. Evidence of success is available for research studies among adults [49,50], yet no outcomes are currently available among adolescents [51]. Other strategies for enhancing communication with adolescents could also be explored in the future (with appropriate ethical approval), such as text messaging and direct messaging on social media platforms.

The second highest rate of dropout among participants was those who required further screening for a potential eating disorder. Out of 2369 potential participants, 292 (12.3%) did not complete an EDE-Q and were therefore excluded, and 189 (8.0%) required clearance through a phone call with this study’s psychologist. When compared to the prevalence of eating disorders overall among Australian adolescents, this rate is lower than what has previously been reported (point prevalence of 22.2%) [52]. Thus, screening for eating disorders was not identified as a barrier to enrollment, rather an important safety precaution for potential identification of disordered eating among this population in a preventive intervention.

Limitations

Limitations in this study exist. First, this study is not representative of all adolescents due to inclusion criteria, which remove some groups. However, as this is a prevention intervention, the inclusion criteria aim to represent a large percentage of the adolescent population within Australia. Second, there are restrictions on advertising to adolescents via social media and changes are constantly occurring in this space. Though the inclusion criteria for age in this study was those aged 12 to 18 years, social media advertisements are unable to be targeted to adolescents aged younger than 13 years, as you can only establish a social media account if you are over 13 years. Advertisements were developed for distribution on Snapchat; however, advertising of clinical trials is not allowed on its platform. Therefore, recruitment of adolescents via social media is also a limitation to reaching adolescents aged <13 years. Third, adolescents report that they find recruitment via social media to be feasible and acceptable for recruitment and retention [53,54]. However, this was not assessed within this study as follow-ups are ongoing. Adolescent perceptions for using social media for recruitment will be assessed in the process evaluation for the Health4Me Study, by assessing retention rates and analyzing focus group data. The findings of this study require validation with studies among other adolescent populations and other types of interventions.

Conclusions

Within the Health4Me Study, it was observed that recruitment was most effective via social media, and this was low cost per participant enrolled. Throughout the screening to enrollment process, there were multiple points of dropout, and future efforts should be directed toward streamlining screening and enrollment processes for scalability of future trials. In addition, our results highlight the importance of building trust among clinical trials and health information generally among adolescents on social media for future success in recruiting adolescents via this digital strategy.

Acknowledgments

The team would like to acknowledge Jessie Zhou who assisted with participant screening and recruitment for the Health4Me Study. We would also like to acknowledge Dr Karen Spielman and Dr Kyra Sim from the Health4Me Steering Committee, and Teisha Schrimer from the Mid North Coast Local Health District, who were helpful in promoting this study among their known networks. We would also like to acknowledge Judith Fethney from the Susan Wakil School of Nursing and Midwifery for statistical support. The Health4Me Team is comprised of the following authors: Katharine Steinbeck, PhD—Specialty of Child and Adolescent Health, Westmead Clinical School, Faculty of Medicine and Health, The University of Sydney, Westmead, NSW, Australia; Gemma Figtree, PhD—Sydney Clinical School (Northern), Faculty of Medicine and Health, The University of Sydney, Sydney, NSW, Australia; Robyn Gallagher, PhD—Susan Wakil School of Nursing and Midwifery, Faculty of Medicine and Health, The University of Sydney, Sydney, NSW, Australia; Melissa Kang, PhD—General Practice Clinical School, Sydney Medical School, Faculty of Medicine and Health, The University of Sydney, Sydney, NSW, Australia; Karice Hyun, PhD—School of Health Sciences, Faculty of Medicine and Health, The University of Sydney, Sydney, NSW, Australia; Karen Spielman, MBBS (Hons)—InsideOut Institute, Faculty of Medicine and Health, University of Sydney, Sydney, NSW, Australia; Kyra A Sim, PhD—Metabolism & Obesity Service, Sydney Local Health District, Sydney, NSW, Australia; Tim Usherwood, MD—General Practice Clinical School, Sydney Medical School, Faculty of Medicine and Health, The University of Sydney, Sydney, NSW, Australia; Charlotte Hepse, MBBS (Hons)—The University of Notre Dame Australia, Sydney, NSW, Australia; Liliana Laranjo, PhD—Westmead Applied Research Centre, Faculty of Medicine and Health, The University of Sydney, Westmead, NSW, Australia; John Skinner, PhD—Djurali Centre for Aboriginal and Torres Strait Islander Health Research, Heart Research Institute, Sydney, NSW, Australia; Danielle Castles—Nepean Blue Mountains Local Health District, Penrith, NSW, Australia; Katrina E Champion, PhD—The Matilda Centre for Research in Mental Health and Substance Use, The University of Sydney, Sydney, NSW, Australia; Lauren A Gardner, PhD—The Matilda Centre for Research in Mental Health and Substance Use, The University of Sydney, Sydney, NSW, Australia. The Health4Me Study is funded by the Australian Government Department of Health Medical Research Future Fund Primary Care Grant (2006315). The funder has no role in this study's design; collection, analysis, and interpretation of data; or writing of this paper. RR is funded by a SOLVE-CHD Completion Scholarship. ART is funded by a University of Sydney Postgraduate Research Scholarship in Adolescent Health and Nutrition. JR is supported by an NHMRC Investigator Grant (Leadership Level 2 [GNT1143538]). SRP is supported by a National Heart Foundation Future Leader Fellowship (106646) and University of Sydney Horizon Fellowship.

Data Availability

The datasets generated or analyzed during this study are not publicly available due to ethical approvals that protect the data of participants who have taken part in the study.

Authors' Contributions

RR, S Maguire, KW, S Mihrshahi, MLH, JR, SRP, and The Health4Me Team conceptualized the study. RR, ART, S Barakat, SR and S Boulet conducted the investigation. RR conducted the formal analysis and wrote the initial draft. JR and SRP provided supervision. All authors reviewed and edited this paper. All authors approved the final version of this paper.

Conflicts of Interest

None declared.

Multimedia Appendix 1

Health4Me social media advertisement examples.

DOCX File, 1461 KB

  1. Patton GC, Sawyer SM, Santelli JS, et al. Our future: a Lancet commission on adolescent health and wellbeing. The Lancet. Jun 2016;387(10036):2423-2478. [CrossRef]
  2. Australian Institute of Health and Welfare. Health System Spending per Case of Disease and for Certain Risk Factors. AIHW; 2022.
  3. Sanci L, Steinbeck K, Jan S, et al. Access 3: young people and the health system in the digital age-final research report. Department of General Practice Westmead, The University of Sydney and the Australian Centre for Public and Population Health Research, The University of Technology Sydney, Australia; 2018.
  4. Raeside R, Todd A, Sim KA, et al. Accelerating implementation of adolescent digital health prevention programs: analysis of insights from Australian stakeholders. Front Public Health. 2024;12:1389739. [CrossRef] [Medline]
  5. Mathews B. Adolescent capacity to consent to participate in research: a review and analysis informed by law, human rights, ethics, and developmental science. Laws. 2023;12(1):2. [CrossRef]
  6. Dalessandro C. Recruitment tools for reaching millennials: the digital difference. Int J Qual Methods. 2018;17(1):1609406918774446. [CrossRef]
  7. Khoury MJ, Iademarco MF, Riley WT. Precision public health for the era of precision medicine. Am J Prev Med. Mar 2016;50(3):398-401. [CrossRef] [Medline]
  8. Brøgger-Mikkelsen M, Ali Z, Zibert JR, Andersen AD, Thomsen SF. Online patient recruitment in clinical trials: systematic review and meta-analysis. J Med Internet Res. Nov 4, 2020;22(11):e22179. [CrossRef] [Medline]
  9. Darmawan I, Bakker C, Brockman TA, Patten CA, Eder M. The role of social media in enhancing clinical trial recruitment: scoping review. J Med Internet Res. Oct 26, 2020;22(10):e22810. [CrossRef] [Medline]
  10. Rait MA, Prochaska JJ, Rubinstein ML. Recruitment of adolescents for a smoking study: use of traditional strategies and social media. Transl Behav Med. Sep 2015;5(3):254-259. [CrossRef] [Medline]
  11. Moreno MA, Waite A, Pumper M, Colburn T, Holm M, Mendoza J. Recruiting adolescent research participants: in-person compared to social media approaches. Cyberpsychol Behav Soc Netw. Jan 2017;20(1):64-67. [CrossRef] [Medline]
  12. Parker JN, Hunter AS, Bauermeister JA, Bonar EE, Carrico A, Stephenson R. Comparing social media and in-person recruitment: lessons learned from recruiting substance-using, sexual and gender minority adolescents and young adults for a randomized control trial. JMIR Public Health Surveill. Dec 1, 2021;7(12):e31657. [CrossRef] [Medline]
  13. Ellington M, Connelly J, Clayton P, et al. Use of Facebook, Instagram, and Twitter for recruiting healthy participants in nutrition-, physical activity-, or obesity-related studies: a systematic review. Am J Clin Nutr. Feb 9, 2022;115(2):514-533. [CrossRef] [Medline]
  14. Frampton GK, Shepherd J, Pickett K, Griffiths G, Wyatt JC. Digital tools for the recruitment and retention of participants in randomised controlled trials: a systematic map. Trials. Jun 5, 2020;21(1):478. [CrossRef] [Medline]
  15. Watson SE, Smith P, Snowden J, et al. Facilitators and barriers to pediatric clinical trial recruitment and retention in rural and community settings: a scoping review of the literature. Clin Transl Sci. Apr 2022;15(4):838-853. [CrossRef] [Medline]
  16. Amon KL, Campbell AJ, Hawke C, Steinbeck K. Facebook as a recruitment tool for adolescent health research: a systematic review. Acad Pediatr. 2014;14(5):439-447. [CrossRef] [Medline]
  17. Ford KL, Albritton T, Dunn TA, Crawford K, Neuwirth J, Bull S. Youth study recruitment using paid advertising on Instagram, Snapchat, and Facebook: cross-sectional survey study. JMIR Public Health Surveill. Oct 9, 2019;5(4):e14080. [CrossRef] [Medline]
  18. McDermott MM, Newman AB. Remote research and clinical trial integrity during and after the coronavirus pandemic. JAMA. May 18, 2021;325(19):1935-1936. [CrossRef] [Medline]
  19. Petrini C, Mannelli C, Riva L, Gainotti S, Gussoni G. Decentralized clinical trials (DCTs): a few ethical considerations. Front Public Health. 2022;10:1081150. [CrossRef] [Medline]
  20. Ng CE, Bowman S, Ling J, Bagshaw R, Birt A, Yiannakou Y. The future of clinical trials-is it virtual? Br Med Bull. Dec 11, 2023;148(1):42-57. [CrossRef] [Medline]
  21. National Academies of Sciences, Engineering, and Medicine. Virtual Clinical Trials: Challenges and Opportunities: Proceedings of a Workshop. National Academies Press (US); 2019. [CrossRef]
  22. Wong CA, Madanay F, Ozer EM, et al. Digital health technology to enhance adolescent and young adult clinical preventive services: affordances and challenges. J Adolesc Health. Aug 2020;67(2S):S24-S33. [CrossRef] [Medline]
  23. Van Eck K, Thakkar M, Matson PA, Hao L, Marcell AV. Adolescents’ patterns of well-care use over time: who stays connected. Am J Prev Med. May 2021;60(5):e221-e229. [CrossRef] [Medline]
  24. Brisson J, Ravitsky V, Williams-Jones B. “Fostering autonomy” for adolescents to access health services: a need for clarifications. J Adolesc Health. Jun 2021;68(6):1038-1039. [CrossRef] [Medline]
  25. Raeside R, Spielman K, Maguire S, et al. A healthy lifestyle text message intervention for adolescents: protocol for the Health4Me randomized controlled trial. BMC Public Health. Sep 23, 2022;22(1):1805. [CrossRef] [Medline]
  26. Bryant E, Miskovic-Wheatley J, Touyz SW, Crosby RD, Koreshe E, Maguire S. Identification of high risk and early stage eating disorders: first validation of a digital screening tool. J Eat Disord. Sep 6, 2021;9(1):109. [CrossRef] [Medline]
  27. Forsén Mantilla E, Birgegård A, Clinton D. Factor analysis of the adolescent version of the Eating Disorders Examination Questionnaire (EDE-Q): results from Swedish general population and clinical samples. J Eat Disord. 2017;5(1):19. [CrossRef] [Medline]
  28. Afkinich JL, Blachman-Demner DR. Providing incentives to youth participants in research: a literature review. J Empir Res Hum Res Ethics. Jul 2020;15(3):202-215. [CrossRef] [Medline]
  29. Raeside R, Todd A, Wardak S, et al. Striking the right balance: co-designing the Health4Me healthy lifestyle digital health intervention with adolescents. Res Involv Engagem. Dec 7, 2023;9(1):114. [CrossRef] [Medline]
  30. Meta. About advertising to teens. Meta. 2024. URL: https://www.facebook.com/business/help/229435355723442 [Accessed 2024-11-16]
  31. Raeside R, Jia SS, Redfern J, Partridge SR. Navigating the online world of lifestyle health information: qualitative study with adolescents. JMIR Pediatr Parent. Feb 11, 2022;5(1):e35165. [CrossRef] [Medline]
  32. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)--a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. Apr 2009;42(2):377-381. [CrossRef] [Medline]
  33. Cole TJ, Bellizzi MC, Flegal KM, Dietz WH. Establishing a standard definition for child overweight and obesity worldwide: international survey. BMJ. May 6, 2000;320(7244):1240-1243. [CrossRef] [Medline]
  34. Cole TJ, Lobstein T. Extended international (IOTF) body mass index cut-offs for thinness, overweight and obesity. Pediatr Obes. Aug 2012;7(4):284-294. [CrossRef] [Medline]
  35. Kilicel D, De Crescenzo F, Pontrelli G, Armando M. Participant recruitment issues in child and adolescent psychiatry clinical trials with a focus on prevention programs: a meta-analytic review of the literature. J Clin Med. Mar 16, 2023;12(6):2307. [CrossRef] [Medline]
  36. Hall CL, Sanderson C, Brown BJ, et al. Opportunities and challenges of delivering digital clinical trials: lessons learned from a randomised controlled trial of an online behavioural intervention for children and young people. Trials. Dec 9, 2020;21(1):1011. [CrossRef] [Medline]
  37. Partridge SR, Reece L, Sim KA, et al. An analysis of current obesity strategies for adolescents in NSW against best practice recommendations: Implications for researchers, policymakers and practitioners. Health Prom J Aust. Apr 2023;34(2):390-397. [CrossRef]
  38. Topolovec-Vranic J, Natarajan K. The use of social media in recruitment for medical research studies: a scoping review. J Med Internet Res. Nov 7, 2016;18(11):e286. [CrossRef] [Medline]
  39. Kutok ER, Doria N, Dunsiger S, et al. Feasibility and cost of using Instagram to recruit adolescents to a remote intervention. J Adolesc Health. Nov 2021;69(5):838-846. [CrossRef] [Medline]
  40. Twenge JM, Martin GN. Gender differences in associations between digital media use and psychological well-being: evidence from three large datasets. J Adolesc. Feb 2020;79:91-102. [CrossRef] [Medline]
  41. Social media usage by gender 2024. OBERLO. 2024. URL: https://www.oberlo.com/statistics/social-media-usage-by-gender [Accessed 2024-09-23]
  42. Fortunato MP, Girard A, Coratti S, et al. Investigating racial and gender disparities in virtual randomized clinical trial enrollment: Insights from the BE ACTIVE study. Am Heart J. Oct 2024;276:120-124. [CrossRef] [Medline]
  43. Irwin CE Jr, Adams SH, Park MJ, Newacheck PW. Preventive care for adolescents: few get visits and fewer get services. Pediatrics. Apr 2009;123(4):e565-e572. [CrossRef] [Medline]
  44. Brown DL, Cowdery JE, Jones TS, Langford A, Gammage C, Jacobs TL. Adolescent knowledge and attitudes related to clinical trials. Clin Trials. Jun 2015;12(3):212-214. [CrossRef] [Medline]
  45. Freeman JL, Caldwell PHY, Scott KM. The role of trust when adolescents search for and appraise online health information. J Pediatr. Jun 2020;221:215-223. [CrossRef] [Medline]
  46. Freeman JL, Caldwell PHY, Scott KM. How adolescents trust health information on social media: a systematic review. Acad Pediatr. 2023;23(4):703-719. [CrossRef] [Medline]
  47. Borus J, Weas S, Fleegler E, Kahn J, Chan E. Email isn’t just for old people: teen acceptability of internet communication with medical providers. Clin Pediatr (Phila). Dec 2016;55(14):1350-1353. [CrossRef] [Medline]
  48. ResearchKit and CareKit. Apple. 2016. URL: https://www.apple.com/au/researchkit/ [Accessed 2024-11-16]
  49. Bot BM, Suver C, Neto EC, et al. The mPower study, Parkinson disease mobile data collected using ResearchKit. Sci Data. Mar 3, 2016;3:160011. [CrossRef] [Medline]
  50. Chan YFY, Wang P, Rogers L, et al. The Asthma Mobile Health Study, a large-scale clinical observational study using ResearchKit. Nat Biotechnol. Apr 2017;35(4):354-362. [CrossRef] [Medline]
  51. Lalloo C, Pham Q, Cafazzo J, Stephenson E, Stinson J. A ResearchKit app to deliver paediatric electronic consent: protocol of an observational study in adolescents with arthritis. Contemp Clin Trials Commun. Mar 2020;17:100525. [CrossRef] [Medline]
  52. Mitchison D, Mond J, Bussey K, et al. DSM-5 full syndrome, other specified, and unspecified eating disorders in Australian adolescents: prevalence and clinical significance. Psychol Med. Apr 2020;50(6):981-990. [CrossRef] [Medline]
  53. Anderson E, Parslow R, Hollingworth W, et al. Recruiting adolescents with chronic fatigue syndrome/myalgic encephalomyelitis to internet-delivered therapy: internal pilot within a randomized controlled trial. J Med Internet Res. Aug 12, 2020;22(8):e17768. [CrossRef] [Medline]
  54. Wisk LE, Nelson EB, Magane KM, Weitzman ER. Clinical trial recruitment and retention of college students with type 1 diabetes via social media: an implementation case study. J Diabetes Sci Technol. May 2019;13(3):445-456. [CrossRef] [Medline]


ANZCTR: Australia New Zealand Clinical Trials Registry
ASR: adjusted standardized residual
EDE-Q: Eating Disorder Examination Questionnaire
EOI: Expression of Interest
IOI-S: InsideOut Institute Screener
NSW: New South Wales
REDCap: Research Electronic Data Capture


Edited by Sherif Badawy; submitted 04.06.24; peer-reviewed by Digby Krastins, Timothy Johnson; final revised version received 24.09.24; accepted 11.10.24; published 16.12.24.

Copyright

© Rebecca Raeside, Allyson R Todd, Sarah Barakat, Sean Rom, Stephanie Boulet, Sarah Maguire, Kathryn Williams, Seema Mihrshahi, Maree L Hackett, Julie Redfern, Stephanie R Partridge, The Health4Me Team. Originally published in JMIR Pediatrics and Parenting (https://pediatrics.jmir.org), 16.12.2024.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Pediatrics and Parenting, is properly cited. The complete bibliographic information, a link to the original publication on https://pediatrics.jmir.org, as well as this copyright and license information must be included.