Repeated Psychosocial Screening of High School Students Using YouthCHAT: Cohort Study

Introduction

Adolescence is a period of great change and new challenges, when young people aged 13-18 years may experience psychological challenges that result in either distress or disorder [1]. These challenges can have a short-term impact on their general health, academic functioning, and relationships, and a longer-term impact on their adult functioning [2-4]. Almost half of adult mental health issues originate during adolescence, and early intervention can make a difference to their long-term trajectories [5,6]. Young people often find it hard to seek help, partly due to limited health literacy and the stigma of attending mental health services [7,8]. Many clinicians have also been found to lack the resources to identify psychosocial problems due to a lack of time, a lack of confidence, and concerns about the over-identification of problems for which personal management skills and access to services may be limited [9]. Although routine screening for psychosocial problems has been recommended for many years [10], until recently, this has not been feasible, primarily due to the lack of a suitable instrument that could identify the range of psychosocial issues faced by young people. Instead, opportunistic screening has been undertaken in schools and clinical settings using cumbersome and psychometrically invalidated face-to-face assessments such as the HEEADSSS (home, education, eating, activities, drugs and alcohol, sexuality, suicide and depression, safety) interview [11].

In New Zealand, routine psychosocial screening of high school students is undertaken using the HEEADSSS assessment. However, due to financial constraints, this is restricted to Year-9 students (13 year olds) in low decile schools [12]. In a typical high school, one or two school nurses take 6-12 months to complete these assessments [12]. Although completion rates are reported to the New Zealand Ministry of Health, outcomes of these assessments are neither reported nor published.

Our research group in Auckland, New Zealand, developed an electronic psychosocial screener called YouthCHAT (Youth version, Case-finding and Help Assessment Tool) to be used with young people (aged between 13-25 years), based on the adult screener CHAT [13] and its electronic version, eCHAT [14]. It comprises 13 modules designed to align with the HEEADSSS interview and includes 3 validated screeners, for anxiety [the GAD-7 (7-item instrument for Generalized Anxiety Disorder)] [15]; depression [PHQ-A (Patient Health Questionnaire–Adolescents)] [16]; and substance misuse, such as smoking, drinking, and recreational drug use [SACS (Substances and Choices Scale)] [17]. Other domains covered by YouthCHAT are problematic gambling, general stresses, behavior problems, eating problems, exposure to abuse, sexual health, anger management problems, and physical activity [14]. For each positive screen within a domain, a help question appears, which asks participants if they would like help today or later. The help question provides an opportunity for young people and their health providers to further discuss any issues that young people may want to address, thereby enhancing mutual decision-making. YouthCHAT is designed to be accessed as a website by health providers who have secure access to it. Once a YouthCHAT screen is completed, a summary report is generated for the health provider to review [12].

A counter-balanced randomized trial comparing YouthCHAT and the HEEADSSS assessment was conducted in a low decile Auckland high school with 13-year-old (Year 9) students in 2017. The study was registered with the Australian New Zealand Clinical Trials Registry (ANZCTR) ACTRN12616001243404p and was approved by the New Zealand Northern Region Ethics Committee (16/CEN/137/AM03). The study found that YouthCHAT was twice as fast to complete as the HEEADSSS assessment, as effective at identifying key psychosocial problems, and acceptable to both students and school nurses [12].

Due to the emergence of psychosocial problems across the whole of adolescence, it is likely that single episode screening, particularly when focused at its onset, will fail to identify the majority of problems faced by young people. We have previously proposed routine school-based screening as part of an annual health check to maximize the chance of early identification of psychosocial problems and normalize psychosocial screening within the context of a holistic health assessment. While YouthCHAT has shown promise as a psychosocial screener, its suitability for repeated use within a high school or other settings is currently unclear. Therefore, we decided to use YouthCHAT to rescreen the same cohort of 14-year-old (Year 10) students who were previously screened with YouthCHAT at 13 years of age in 2017, in the same high school [12]. Due to concerns about the potential burden on health services from positively screened individuals, we also planned to explore the nature of onward clinical referrals required by the group in 2018.

This study was designed to (1) provide novel insights into the acceptability of repeated screenings of high school students using an electronic screener such as YouthCHAT; (2) provide information on differing rates of psychosocial problems between 2 consecutive years (to see if rates increase with age, as might be expected during adolescence, and to see if being positively screened or receiving early intervention in 2017 was associated with a reduction in the severity of symptoms in 2018); and (3) provide information about resulting clinical referrals.

Our specific aims were to (1) compare acceptability and detection rates with repeated YouthCHAT screenings of high school students when they were 13 years of age (Year 9) and 14 years of age (Year 10), and (2) examine where onward clinical referrals were provided to those with identified needs.

Methods

Participants

All Year-10 (14- to 15-year-old) students at a low decile high school in Auckland, New Zealand, who had previously taken part in a YouthCHAT study in 2017 in Year 9 (at 13 to 14 years of age) [12] were invited to participate in this study, following the provision of paired informed individual assent and parental consent. No students were excluded. A subset of 20 students was invited to participate in a series of focus groups, during which verbal and written feedback regarding their use of YouthCHAT was obtained.

Procedure and Instrumentation

Written informed consent from students and paired assent from parents were obtained using paper forms. As some students had left and others had arrived between 2017 and 2018, only a subgroup of the total sample consisted of paired participant data from the same students screened in both years. Differences in acceptability of repeated screenings and changes in rates of identified psychosocial problems were analyzed in this subgroup.

YouthCHAT screening was completed on a tablet device with a Wi-Fi connection to the YouthCHAT website in the school health center over an 8-month period, between April and November 2018. A series of 4 focus groups with between 5-10 participants each was conducted on school premises in November 2018 to obtain written and verbal feedback from a subset of students who had been invited to take part by the school nurse, based on their interest and availability.

Encrypted YouthCHAT results were manually extracted and securely stored on a University of Auckland server as per New Zealand Northern Region Ethics Committee (16/CEN/137/AM03) requirements. Student demographic and postscreening referral data were also manually extracted from Medtech (a locally utilized patient management software system) and securely stored on the University server. Written feedback from focus group participants was collected on paper forms and securely stored on University premises. Audiotaped feedback from focus group participants was transcribed by an agent who had signed a confidentiality agreement, and electronic audio-files were securely stored on the University server.

Outcomes

Psychosocial problems from YouthCHAT were measured by positive responses to any items. Substance misuse was measured by positive responses to A Stop Smoking In Schools Trial (ASSIST) [18] or the Substances and Choices Scale (SACS). Problems with eating were determined by positive responses to any items in the eating module. Mental health and distress were measured by positive screens to the PHQ-A or GAD-7 scale. Life stresses were measured by positive responses to questions regarding different stresses such as the following: issues at home, school, or work; money; or relationships with specific people in one's life. Sexual health was measured by positive responses to concerns regarding sexual orientation, risky sexual behavior, or exposure to unwanted sex. Safety was determined by positive responses to either the anger or abuse modules, and to being bullied or exposed to violence in the stress module. Physical inactivity was measured by negative responses to regular physical activity engagement.

Data Analysis

Data were analyzed using Microsoft Excel (version 15.0; Microsoft) and Statistical Package for the Social Sciences (SPSS; version 25; IBM Corp). The YouthCHAT screening was considered positive if any items were acknowledged within each module. The depression, anxiety, and substance misuse modules were further examined to ascertain the severity of the results (rated as subthreshold, mild, moderate, or severe, according to the developers of the PHQ-A, GAD-7, and SACS). Analysis of paired data (for those who completed YouthCHAT in both 2017 and 2018) was conducted using the McNemar test for categorical variables. Focus group data from 2017 and 2018 were treated as independent groups. Responses to items regarding YouthCHAT and its use were analyzed using chi-square or Fisher exact tests to compare categorical variables; the Fisher exact test was used where there were any expected cell counts less than 5 [19]. The Kruskal-Wallis test was used for continuous or ordinal variables.

Results

Description of Participants

Of 141 eligible students, 114 (80%) were screened in 2018; incomplete data for 1 screening provided a total sample size of 113 for analysis in 2018 and 129 (93%) out of 139 in 2017. None of the invited students declined to take part in the study. A subset of 97 students who completed YouthCHAT screenings in both years were compared. In regard to participant ethnicity, of the paired subset, 60 (62%) students were Pacific Island, 24 (25%) were Maori, 1 (1%) was New Zealand European, 1 (1%) was Middle Eastern/Latin American/African, and 11 (11%) were of other ethnicities. The number of men (50/97, 52%) and women (45/97, 46%) in the paired subset were relatively similar, with 2 students (2%) identifying as gender diverse (which was an additional option in the 2018 YouthCHAT screener). Of the 20 participants invited to take part in focus groups, 16 agreed to do so. The self-reported demographic details of the 16 students who attended the focus groups were as follows: 5 men (31%), 9 women (56%), and 1 gender-diverse individual (6%); 7 (44%) were of Maori descent, 5 (31%) were of Pacific Island descent, and 3 (19%) were of other ethnic descent. Demographic details for 1 participant are unknown as they did not complete the attendance sheet.

Time Taken to Complete YouthCHAT and Time Period Between Screens in Paired Subset

The average time taken to complete YouthCHAT in 2018 was 6 minutes and 32 seconds (range, 2 minutes and 13 seconds to 16 minutes and 45 seconds). This was similar to the 8 minutes and 57 seconds (range, 1 minute and 45 seconds to 54 minutes and 15 seconds) taken by students in 2017. The Wi-Fi connection was lost for some students in 2017, which may explain the outlier of 54 minutes and 15 seconds. The average time period between the YouthCHAT screens in 2017 and 2018 was 11.4 months (range 6-18 months), with the time periods varying for each student from the paired subset.

Overall Detection Rates and Comparison of Results in a Paired Subset Between 2017-2018

Overall rates of detection of psychosocial problems and a comparison of results between 2017 and 2018 are presented in Table 1. The table displays the number of students who were positive for psychosocial problems in both years and the number of students who remained positive for the particular psychosocial problem in the repeated screening. It also represents the number of new positive screens for psychosocial problems in the following year of YouthCHAT screening. There was a statistically significant reduction in the rate of perceived life stress (-13%; 95% CI -25.1% to -1.0%; P=.04), a statistically nonsignificant increase in the rate of substance misuse, and a statistically significant increase in the rate of depression (18.8%; 95% CI 6.8-30.7; P=.002) between the 2 years. According to the 2018 PHQ-A scores, of the 96 respondents, 25 students had subthreshold depression, 3 students had mild depression, 4 had moderate depression, 4 had moderately severe depression, and 1 had severe depression. Otherwise, rates of psychosocial problems between these screening events were broadly similar, with concerns about weight and eating, and behavior and anger, being the most reported issues. Of the 90 students who self-identified with anxiety via the GAD-7 in 2018, 7 had subthreshold anxiety, 11 had mild anxiety, 7 had moderate anxiety, and 1 had severe anxiety. In regard to SACS scores, 8 students had scores greater than 2, indicating they required further assessment; 3 had scores greater than 4, indicating clinically significant problems; and 3 had scores greater than 6, indicating serious problems. Of the 5 students who screened positive for depression in 2017 and 2018, 3 had changed from severe to moderately severe symptoms, 1 remained unchanged with mild symptoms, and 1 had changed from moderate to moderately severe symptoms. Of the 8 students who screened positive for anxiety in both 2017 and 2018, 3 had changed from severe to moderate/mild, and 6 remained unchanged with either mild or subthreshold symptoms.

Acceptability of Repeated Screening Using YouthCHAT

In 2018, the 16 students who took part in the focus group gave YouthCHAT a rating of 7.8 (range 5-10) on a point scale from “lame” to “awesome;” they also said it had been helpful for discussing psychosocial issues with their school nurse or doctor at a rating of 7.8 (range 5-10) on a 10-point scale from “not helpful at all” to “helpful.” Although many (10/16, 63%) denied disliking any YouthCHAT questions, 6 of the 16 students (38%) said they did not like answering questions related to sexual health, substance misuse, exercise, anger, gambling, and depression. The results of student rating scores from both 2017 (n=10) and 2018 (n=16) are presented in Tables 2 and 3; they were not paired, as different groups of students attended the focus groups in each year. Overall, there were no major differences in the acceptability of YouthCHAT from one year to the next. Although fewer students said they had time to think about their responses in 2018, there was a notable reduction in feelings of embarrassment about discussing results with the school nurse, and a similar number of students reported comfort with disclosing things they would otherwise have not mentioned in both years.

Onward Clinical Referrals From YouthCHAT Screening

Table 4 presents the number of students who were referred to internal school health staff, health services, and programs following the YouthCHAT screening in 2018. Onward clinical referrals were made by the school nurse based on the severity of the psychosocial issue (eg, a concerning score from the PHQ-A questionnaire) and whether school health staff were already seeing the student who screened positive for any psychosocial problems prior to the YouthCHAT screening. As evident from Table 4, most students with positive YouthCHAT screens who received onward clinical referrals following school nurse review of their results were seen by school-based practitioners, particularly nurses and counselors. Only a few required assistance from external community-based or specialist health services. All students referred for onward clinical referrals had positive YouthCHAT screens, and most of those with depression (12/113, 11%), anxiety (19/113, 17%), and substance misuse (11/113, 10%) had clinically significant levels of symptoms.

Discussion

Principal Findings

Our results demonstrate that YouthCHAT remains an acceptable, efficient, and effective HEEADSSS-aligned instrument for undertaking repeated psychosocial screenings with high school students. They also suggest that most identified problems can be managed within the school environment. Overall rates of identified issues at 14 years of age were not that different from those identified at 13 years of age, which may be unsurprising given the fact that most significant mental health issues emerge between midadolescence and early adulthood [20]. However, the relatively high rates of anxiety (21%, compared to 13% in a 15-year-old youths sample from a previous New Zealand study) and depression (31%, compared to 3% in 2017 and 6% from the same previous New Zealand study of 15-year-old adolescents [21]) in this study are interesting to note. These may partly be explained by the predominance of students of Pacific Island ethnicity and their known propensity to experience twice the rates of these issues as other New Zealand youth [22]. Reasons for higher rates of these conditions in this group are not fully understood but may bear some relationship to culturally mediated values [23], status incongruity [24], and nonculturally related sociodemographic factors (such as social deprivation) [25].

Many of those identified with depression (42%), anxiety (40%), substance misuse (60-83%), and eating concerns (99%) at 13 years of age also screened positive at 14 years of age. Although it might seem like screening did not make a difference to later rates of these problems, the truth is probably more complex. This was a relatively small sample, and most students who screened positive for depression and anxiety experienced a maintenance or improvement in symptoms, which, in the context of naturally increasing rates and severity of these conditions, may indicate that earlier intervention had some clinical effect. Certainly, larger, more detailed, and longitudinal studies are needed to more accurately evaluate the value of routine screening in reducing the long-term prevalence of psychosocial problems and their associated disability. The fact that most onward clinical referrals were undertaken by school nurses and counselors underscores the role of YouthCHAT in supporting early intervention within students' natural environments. It also assuages the concern that routine screening increases the risk of further burdening stretched specialist mental health services [12].

Limitations

This study was conducted at the same high school as a previous study so that results between both years could be compared. However, this remained a convenience sample, and the generalizability of study results to other high schools remains to be proved. The inclusion of primarily Pacific Island and Maori students, who comprised the bulk of students in the examined class, is both a strength and a weakness. Although they comprise 11% and 20% of the New Zealand population, respectively, Pacific Island and Maori youth have higher rates of psychological issues, including depression and suicide [22]. They are also usually harder to reach, accessing specialist services at lower rates than other ethnicities [26,27]. Due to funding limitations, feedback was not collected from school nurses or counselors regarding their satisfaction with receiving referrals following repeated YouthCHAT screenings.

Conclusions

This study demonstrates the acceptability and effectiveness of the repeated use of YouthCHAT for students. The fact that the majority of positive screens to YouthCHAT were manageable within the school health setting supports the feasibility of routine psychosocial screening in the school environment and the likelihood of earlier intervention. YouthCHAT is available for use by schools and can be easily implemented by school nurses and counselors for opportunistic and routine psychosocial screening. Routine screening via YouthCHAT may lead to reduced costs compared with lengthy face-to-face clinical assessments and more timely interventions for students within the school environment.

Recommendations for immediate research include trialing YouthCHAT with older (15- to 18-year-old) students within the same high school to examine age-related trends, and evaluating differing acceptability and rates of psychosocial problems between low and high decile, urban and rural, and English and Maori immersion schools. International studies, with or without culturally relevant adaptation, and longitudinal studies to gauge the impact of earlier intervention following routine screening, would also be worthwhile. For the moment, YouthCHAT remains the only electronic psychosocial screener with evidence of its acceptability and feasibility for repeated use in any type of setting.