Acceptance of a French e–Mental Health Information Website (CléPsy) for Families: A Web-Based Survey

Introduction

Childhood mental health (MH) issues concern up to 13.4% of children worldwide [1] and represent a major public health challenge [2]. However, parents’ and caregivers’ lack of knowledge in this area constitutes a major obstacle to their management of these issues [3]. Child and adolescent MH has substantially worsened in recent years, especially during the COVID-19 pandemic, notably in girls and vulnerable populations such as families with low income or children and adolescents with pre-existing MH difficulties [4].

The World Health Organization has promoted a shift from patient-centered health to family empowerment as a catalyst for health development especially in chronic diseases. This switch involved providing families with tools to understand their rights regarding health care and how to access health services [5]. For example, patient empowerment is known to enhance health literacy in people with diabetes through increasing expression, self-judgment, and critical thinking, especially communicative health literacy and critical health literacy that relies on interaction and communication skills [6]. Parent health literacy was also considered an essential element in minimizing childhood obesity in a meta-analysis [7]. In the specific case of MH, family empowerment is a mediator for positive changes by enabling better management of children’s difficulties and providing a greater sense of confidence in parental problem-solving abilities, facilitating the resolution of future MH challenges [8].

Health literacy encompasses knowledge that helps individuals achieve and maintain good health, including the identification of symptoms and the ability to understand and evaluate health-related information effectively. More specifically, MH literacy has been defined as “understanding how to obtain and maintain positive MH; understanding mental disorders and their treatments; decreasing stigma related to mental disorders; and enhancing help-seeking efficacy” [9]. Parents and caregivers are in demand of MH knowledge—especially those concerned by previous experiences with MH. However seeking help and information might be limited by fear, stigma, and mistrust of the sources of information [10]. Indeed, the lack of MH knowledge and the stigma attached to mental impairments constitute major obstacles to health care access [11]. When faced with psychiatric difficulties in their children, many parents express a lack of knowledge and the necessary skills to address these issues effectively [12]. Consequently, almost all parents (96%) turn to online sources to find answers to health-related questions in general, with MH and parenting being the most frequently searched topics [13]. However, a study examining French-speaking MH websites conducted by Buteau-Poulin et al [14] revealed that only 43% of dedicated websites provided scientifically reliable content. Among these websites, only 42% addressed autism, 45% addressed learning disorders, and merely 10% covered language disorders and behavioral problems. Furthermore, MH information websites often employ language that exceeds the average parent’s comprehension level [15].

Better functional and communicative health literacy was associated with reduced stigma and reluctance to seek MH support among adults [16]. Digital health interventions in particular have the potential to be an effective—and cost-efficient—method for increasing parental knowledge regarding MH difficulties in their children and adolescents. These interventions can assist parents in assessing valid information and understanding the organization of psychiatric services [17]. In line with this goal of supporting children and their families, we launched a website in March 2020, intended for families of children with MH disorders in the context of the first French lockdown during the COVID-19 pandemic. This website provides evidence-based knowledge on MH topics, practical tool kits, and know-how to deal with children’s everyday difficulties. CléPsy is divided into categories (general information, anxiety and affective disorders, attention-deficit/hyperactivity disorder [ADHD], intellectual development disorder, addictology, autism spectrum disorders, language/learning disorders, and eating disorders). Examples of pages include information about augmentative and adaptive communication for children with autism spectrum disorders and how to use it, what early signs of ADHD should parents be aware of, or the mechanisms and risk factors of eating disorders. Authors are identified on each page and sources are mentioned when needed.

The main purpose of this study was to evaluate users’ general acceptance of the design and the content of CléPsy, a website dedicated to helping families concerned with child MH through information and practical tools. A secondary objective was to evaluate the effect of users’ characteristics on general acceptance.

Methods

Creation of the Online Platform

The first version of the website [18] was created and released in March 2020 by the authors of this paper as a rapid response to the MH concerns raised by families in the context of the COVID-19 crisis. A later version (rebranded as CléPsy [19]) was designed by a team of professional web developers and made available as of January 2021. The developer team sought general feedback from families and other website users to guide their strategy but did not implement specific focus groups. Since then, the platform has offered varied content covering a large array of MH topics in French. The content of the website was produced by the multidisciplinary staff of the Center of Excellence for Neurodevelopmental Disorders (Robert Debré Hospital, Paris), submitted to a proofreading committee, and had to follow specific editorial standards based on the eEurope 2002 Quality Criteria for Health Related Websites [20].

Questionnaires and Study Participants

A self-directed survey was made available through various methods between May and July 2022. First, the survey was directly sent to the CléPsy mailing list newsletter subscribers, which is usually used to give information about new content. Second, the survey was shared on social networks (Facebook, Twitter, and LinkedIn). Finally, posters with a QR code leading to the survey were displayed in Robert Debré Hospital. This web-based survey verified the recommendations of CHERRIES (Checklist for Reporting Results of Internet E-Surveys) for web survey quality [21] and was adapted from the questionnaire developed by Tlach et al [22] for the German e-portal Psychenet [23] and translated to French. The open survey was displayed on Lime Survey, and information about the aim and length of the survey was given before participants were asked if they consented to answer. No incentive was used to promote answering, and IP addresses were used to avoid multiple responses by the same visitor. The questionnaire first explores the baseline characteristics of respondents: age, gender, location, and educational level, as well as respondents’ experience with MH. In a second acceptance/usability–related section, we used a 4-point Likert scale (1=disagree, 2=somewhat disagree, 3=somewhat agree, 4=agree) to explore four dimensions: “Perceived ease of use,” which refers to the readability, design, and accessibility of the website; “Perceived usefulness,” which refers more specifically to the content and how it helped participants; “Perceived trustworthiness,” which is about how much the information on the website seems trustworthy and up to date; and finally, “general attitude toward the website” had questions about the tendency to recommend the website to others or to revisit it.

Web Analytics

Google Analytics was used to determine the number of connection sessions since the launch of the website as well as demographic information about the visitors, including the countries and cities associated with the users, the frequency and recency of the sessions, and the top site contents viewed. Google Analytics is a widely used web analytics service offered by Google that tracks and reports website traffic.

Data Analysis

The collected data on user characteristics and acceptance were analyzed on JMP statistical software (V17.0; JMP Statistical Discovery LLC). Our analysis was based on a similar work conducted by Tlach et al [22]. Only complete answers were analyzed. Means, SDs, and frequency distributions were calculated for each item to quantify the response. A total score for each of the four dimensions was calculated by summing the Likert score of all items in this dimension. One-way ANOVAs were conducted for interval-scaled variables (total scores of the four dimensions of acceptance and usability) to explore the effects of distinct participant characteristics (gender, age, educational level, experience with mental disorders, frequency of use) on the acceptance and usability of the website. A P value <.05 was considered to be significant for all analyses. The significance level was not adjusted as the tests served to generate hypotheses.

Ethical Considerations

This study was submitted to and approved by the local Assistance Publique - Hopitaux de Paris ethics board (2021‐588).

Results

Overview

From May to July 2022, 53,911 pages were visited on CléPsy by 20,513 independent visitors. Among them, 693 (3.4%) accessed the web-based questionnaire and 614 (3%) consented to participate. Among these 614 respondents, half responded completely (n=317, 51.6%), of which almost all were female (n=297, 94%) with an average age of 46.2 (SD 10.1) years. We observed that 63.1% (n=200) of the complete respondents were MH professionals and 65.6% (n=208) were highly educated (master’s degree or a doctorate). Almost all respondents used the internet daily (n=298, 94%), and 49.2% (n=156) learned about CléPsy through social networks. Finally, 60.6% (n=192) of respondents visited CléPsy >5 times (Table 1).

Participants’ responses to the four dimensions are summarized in Table 2.

Perceived Ease of Use

Almost all participants (n=304, 96%) stated that the website was easy to use, with 90.2% (n=305) indicating that they could easily find the information important to them. Additionally, 96% (n=304) found the design appealing. None of the main participants’ characteristics had a specific effect on the perceived ease of use (Table 3).

Perceived Usefulness

The majority of respondents agreed or somewhat agreed with the statement “All in all, the website is useful for me” (n=308, 97.1%). A significant percentage agreed or somewhat agreed that the website facilitated discussions about MH with professionals (n=264, 83.3%) or with their relatives (n=260, 82.1%). According to the ANOVA, there was a significant effect between frequency of use and perceived usefulness (F₂=4.85; P=.008; Table 4).

Perceived Trust and General Attitude Towards Using CléPsy

The vast majority of respondents described the website as trustworthy (n=308, 97.3%) and up to date (n=304, 95.9%), and would recommend it to other people (n=308, 97.2%). None of the main participants’ characteristics had a specific effect on their attitude toward using CléPsy (Table 5). According to the ANOVA, there was a significant effect relating perceived trust with the educational level (F₆=3.03; P=.007) and the frequency of use (F₂=5.30; P=.006; Table 6).

Discussion

Principal Findings

In this web-based study, we investigated the acceptance of the design and content of a new website dedicated to children’s MH information with the aim of gaining a better understanding of active users and areas for improvement to reach target users more effectively.

Among complete responders (n=317), almost all agreed (n=152, 47.9%) or somewhat agreed (n=152, 47.9%) with the affirmation that the website was easy to use. Those results were in line with previous studies exploring the ergonomics of online MH information. When evaluating the Psychenet website, Tlach et al [22] found equivalent rates of perceived ease of use. Engagement and perceived ease of use were equivalent regarding the Together for Wellness website, developed during the COVID-19 pandemic [24]. A preference for English and higher behavioral changes during the pandemic were predictors for higher comfort in using the Together for Wellness website. Concerning the design of the CléPsy website, most participants evaluated the font, the colors, the presentation of information, the illustrations, and the accessibility of information positively. User experience (UX) is an important feature of online MH information, and it can be a major barrier for end users [25]. In our study, perceived ease of use did not differ with participants’ characteristics. This underlines the importance of working with UX and design professionals to create state-of-the-art internet navigation interfaces that are easy to use. This is especially true when implementing online health information.

The usefulness of the CléPsy website was largely appreciated by the respondents. This could be in line with an almost complete lack of accessible digital information about child and adolescent MH written in French. Furthermore, two-thirds of French people believe that digital tools can help them better understand health care strategies [26]. This is especially the case for MH information, which is one of the main topics researched by parents online when they are looking for pediatric health information [13]. Parents could also be more aware of their children’s difficulties with the increasing awareness of MH issues in children and adolescents, especially since the COVID-19 pandemic [27]. For example, the online search for ADHD information has been increasing for several years, which could partially be explained by destigmatization induced by media coverage [28]. The perceived usefulness had a positive influence on the user’s ability to discuss MH with health professionals (including their general practitioners) or family members. Indeed, as reported in a systematic review by Kubb and Foran [29], most parents expressed difficulties in talking to their physicians about health information found online. The main reasons expressed by parents for not talking to the physicians were a lack of time, a fear of disapproval, or difficulties in understanding technical medical terminology. Interestingly, physicians express reservations on the subject. In a qualitative study, Karatas et al [30] explored the main attitude of pediatricians confronted with eHealth-based questions. Most of them expressed concerns about inaccurate health information that could lead to detrimental decisions by families. They highlighted an increased length of consultations to answer the questions but acknowledged that it helped build a more collaborative relationship with parents. In our study, most respondents were MH professionals. This could reflect a need for regular updates on practices, especially regarding evidence-based interventions, even if the therapist’s behavior seems to change not only with information but also with specific training [31]. A quantitative study confirmed the positive and bidirectional influence of the perceived usefulness of eHealth information on patient-physician interaction and trust [32]. From the patient’s perspective, it enriches participation and cooperation through enhanced resource integration, and from the physician’s perspective, it can improve the patient’s understanding and the relationship.

The CléPsy website was considered trustworthy by 97.2% (n=308) of respondents, but ANOVAs found an effect of educational level on the perceived trust, contrary to the evaluation of Psychenet by Tlach et al [22]. Parents generally have difficulties evaluating the trustworthiness of their source of information when searching for child health information [33], but most of them would like health care professionals to help them identify reliable sources [34]. When parents from Switzerland were asked about their relationship to online health information seeking, most of them were skeptical about the correctness of the information, but one-third of them did not, or rarely, check the reliability of the website [35]. Interestingly, first-time parents and parents with high levels of education made higher use of web pages targeted at parents. Trust in online health information is predicted by sociodemographic factors such as higher income and educational level [36], but online information remained the least trusted source, behind media sources and interpersonal information.

Limitations

This study must be considered with regard to several limitations. First, contrary to our expectations regarding the main audience, the majority of respondents were MH professionals, whereas the website was created to address inquiries from families. Furthermore, a significant proportion of respondents had a high level of education (master’s or doctorate). This may be attributed to the distribution method of the questionnaire, in particular via professional social networks, and to the institutional credibility of the Center of Excellence for Neurodevelopmental Disorders in France. In our analysis, the characteristics of the respondents did not have any effect on their answers. Considering the accessibility and usability of health websites, the literature shows that primary users and MH professionals tend to have the same opinion [37]. MH professionals could also be interested in our website due to the necessity to get valid information about the state-of-the-art practice in MH. Indeed, an important part of the French psychology curriculum is dedicated to psychoanalysis. Future studies would need to consider the differences between these audiences. Moreover, the representativeness of the respondent must be interpreted with caution as the response rate was very low compared to the total number of users during the same period. Second, the CléPsy website was created initially to address families’ questions about their children’s MH at the onset of the COVID-19 pandemic and facilitate the transmission of relevant content. However, due to this unique timeline, the current version of the website insufficiently meets the standard framework that recommends the co-construction of health information websites together with families and other users [38]. Focus groups with families have since been held by our team along with the UX professionals, resulting in a qualitative evaluation of parents’ needs. Third, as expressed by Tlach et al [22], there is a lack of standardized instruments to evaluate the acceptability and usefulness of digital content, which limits comparison. However, the questionnaire was designed based on recent recommendations such as the CHERRIES checklist. Finally, adverse effects of online health information seeking such as cyberchondria were not evaluated and should be taken into account in further research [39].

Conclusion

Our study provides preliminary support for the acceptability and usefulness of a French MH information website dedicated to children and adolescents. Results show that users are in demand of trustworthy information on this matter. In the future, special attention should be given to the co-construction of such websites with concerned users to improve the pertinence, inclusiveness, and accessibility of the contents to a wide and varied public. Moreover, parental role and child diagnosis should be identified in future research.