Parents who have a child with heart failure need understandable and reliable knowledge. Approximately 11,000 to 14,000 annual pediatric hospitalizations in the United States are due to children’s heart failure, with 87% of all initial cases diagnosed after an exacerbation in heart failure symptoms requiring invasive, life-saving medical intervention [1]. Heart failure in children can invoke uncertainty, heighten stress levels, and impose a steep learning curve on parents.

Since the release of North American evidence-based guidelines [2,3], more children with heart failure have been surviving, and parents have been caring for them in the outpatient setting. Parents are in the unique position of being termed proxy health information seekers, as they require advanced and ongoing information to provide day-to-day management for their ill child [4,5]. Aside from their health care team, a source where parents rely heavily upon for health information to make daily decisions about their child’s care is the internet [5]. Parents who have children with chronic health conditions have identified they require adequate and appropriate information to care for their child [6]. However, despite the call for this necessary information among parents, the literature still suggests that they feel generally unsupported in their quest for health information [7]. Undoubtedly, this need exists for parents of children with heart failure given the scarce amount of literature on this topic.

Multimedia-based educational tools (eg, e-books, apps, videos, and whiteboard animations) posted on the internet are novel strategies that can fill this knowledge gap by providing easy to access educational content to parents and caregivers who need it [8]. These tools have the ability to creatively accentuate evidence-based health information, resulting in better uptake by parent audiences [9]. They positively influence learning styles by providing complex information that is palatable, relevant, and understandable [10]. Knowledge translation tools have been created for parent audiences in other contexts and have been shown to provide understandable, accessible, and evidence-based knowledge that helps improve care [11]. Multimedia-based knowledge translation tools have not been widely explored in the context of childhood heart failure. To date, there is currently no understanding of what web-based knowledge translation tools exist for parent audiences about pediatric heart failure and how they are rated in terms of how they communicate medical information to parent audiences.

Our research seeks to better understand what tools are currently available for parents who have a child with heart failure and to assess each tool’s ability to enhance their knowledge. Therefore, the aim of our study is to understand what publicly available educational tools are available to parents of children with heart failure on the web and app stores.

The environmental scan (ES) methodology is used to scan the scan the environment in an organized manner for gray information pertaining to a specific topic or context [12]. Our ES was conducted in 4 stages searching Canadian and US-based educational tools for children’s heart failure that targeted parental audiences in June 2020.

A multimedia tool was included if it (1) was developed in either Canada or the United States, (2) focused solely on children’s heart failure content, (3) was developed in English language, and (4) targeted a parent or caregiver audience. Tools were only included if they were from Canada and the United States to preserve the feasibility of the study. Duplicate tools were excluded. Given the anticipated limitation in tools, a date range was not applied to the internet search to maximize our findings.

Data collection occurred in four separate phases: (1) app search, (2) internet search, (3) Suitability Assessment of Materials (SAM) evaluation, and (4) key informant interviews. Key informant interviewers serve to augment the findings of the search as they will provide richer detail about each resource’s development process. Multimedia Appendix 1 outlines the screening process of the app and internet search.

As our study included a qualitative interview component with key informants, ethical approval was obtained from the University of Alberta Research Ethics Office (Pro00106559).

Two app stores (Apple App Store and Google Play) were searched using the broad layman search term heart failure by the primary researcher (CC). Using the same term, a second researcher (JB) used a web scraping search strategy to ensure comprehensiveness. Web scraping is the systematic process of using a web bot (or software agent) to produce more comprehensive search results [13]. Searches were limited in the advanced search function to either Canada or the United States, totaling 2 searches. Modeled after previous ES methods [14], only the first 50 apps from the Canadian and US search in each app store were archived for review. The primary reviewer (CC) complied all the internet and app results into Microsoft Excel spreadsheets for screening. Screening for eligibility was completed by the primary reviewer, and all data were verified independently by a second reviewer (HS) to ensure accuracy.

The primary reviewer (CC) performed the search using three broad laymen’s search terms in the all these words function of Google Advanced Search: (1) child heart failure, (2) pediatric heart failure, and (3) child heart failure guidelines. The primary reviewer ran each term separately for each country (eg, Canada or the United States), resulting in 6 separate searches. Other strategies to increase search results included disabling cookies and turning off personalization to help reduce search bias. Again, no date restriction was placed to maximize our search results. To keep the search feasible, the search was limited to English language tools and within Canada and the United States. Another reason to limit the search to any North American tools was to tailor our findings to apply to an educational tool we are developing about children’s heart failure that will be used in this area.

Similar to other ES methods [12], the first 100 webpage results from each search string were archived using screenshots and uploaded into a Microsoft Excel spreadsheet for screening by two reviewers (CC and HS). Adapted from methods used for data extraction and screening in systematic review processes to avoid data extraction errors [15], CC completed the initial screen and data extraction, and HS verified all data line by line. All discrepancies in data extraction and relevancy were flagged and discussed between the 2 reviewers, with no disagreements needing to be brought forth to the senior author (SDS). To increase the quality and accuracy in data collection, CC (an experienced pediatric heart failure clinician) educated the second reviewer about children’s heart failure [15]. All included internet educational tools were downloaded and examined in detail.

To prevent missing any relevant educational tools in the search process, the primary reviewer also consulted with a subject expert (JC) in the field of pediatric heart failure (eg, pediatric cardiologist) to review the list of screened websites, identifying any further relevant tools that may exist but were missed. An additional tool was identified by our subject expert (April 2020). This tool was posted to the web after the date of our initial search, and it was added to our relevant list of tools for health literacy appraisal.

Descriptive statistics and frequencies in Microsoft Excel were used to analyze the characteristics of both app and internet resources. A list of any relevant web-based tools in either the app or internet search was reported for the apps and web-based tools.

All relevant internet and app resources that met the inclusion criteria were downloaded in full and scored independently by two reviewers (CC and HS) using the SAM scoresheet. The SAM assessment is a validated tool, developed by experts in health education for adults, that assesses the readability, usability, and suitability of health information [16]. SAM evaluation can pinpoint specific strengths or deficiencies in educational materials or compare different education materials for specific patient populations and suggest areas of improvement or refinement [16]. This method of scoring has not been applied in the pediatric heart failure context but in other pediatric contexts [17,18]. The original SAM scoring tool was developed by Doak et al [16] and was subsequently adapted with permission by Smith [19]. The version developed by Doak et al [16] includes 6 domains with 22 subfactors. The updated version by Smith [19] includes the 6 domains but with only 21 subfactors, omitting the scope evaluation within the content section. The modified version by Smith [19] was used, as the scope of this assessment was already tailored to include only tools about children’s heart failure that specifically educated the parental audience.

A rating score was obtained from each assessment (not suitable=0, adequate=1, and superior=2). Scores were then transformed into percentages (percent ratings; eg, 0%-39%=not suitable material, 40%-69%=adequate material, and 70%-100%=superior material). A rating of not suitable would indicate that a web-based resource requires some refinement to make it more suitable for the intended audience, whereas a superior rating would indicate that no further refinements are needed and a high level of health literacy [16]. Resources were not excluded from inclusion based on their SAM score, but rather the assessment was completed to provide a sense of the overall scope and quality of educational content that is available to parents who care for a child with heart failure. To ensure minimal bias in the review process [20], CC and HS both independently SAM rated each relevant educational tool, and then scores were disclosed and discussed. Any highly discrepant scores (>10 points difference) were discussed in detail among reviewers to understand the large variability in scores (eg, errors in scoring). Given the subjective nature of this scoring tool, an average overall rating between both reviewers for each domain was generated, giving an average SAM score for each resource.

Key informant interviews were conducted to complement the SAM ratings and add depth about the characteristics, distribution, and development process of each tool. Interviews with key interviewees were conducted by CC who contacted organizations from the information provided on the webpage. To maximize the number of informant responses, 3 attempts were made to contact each key informant (n=17) either by phone or by the email provided on the tool’s webpage. This approach was modeled after the method developed by Dillman [21] for achieving responsiveness in the context of surveys. All interviews were conducted and recorded using the Zoom video conference platform [22]. All interviews were live coded to allow for context and meaning to be present in the results [23]. All participants provided written consent before the interview.

Data collection and analysis occurred iteratively, allowing for a more precise and purposeful process. The number of interviews achieved was not decided based on data saturation but on the positive responses accepting the invitation to participate in a qualitative interview by key informants who played an integral role in tool development.

Thematic analysis was used to synthesize and identify common themes among key informants described in the semistructured interviews. Thematic analysis was modeled after the study by Braun and Clarke [24]. Outlined in their approach are four key stages: familiarization with the data, initial coding, searching for categories among the initial open codes, and constructing final major themes that best represented the data. A data-driven inductive approach was used to link the developed codes and themes to the data themselves [25]. The interviewer became immersed in the data through repeated listening of the recorded video interviews with live coding into summary tables. Codes remained genuine as they stayed as close to the participants’ own words. Codes started more general and became more focused as they were grouped into categories and then major themes. All codes and videos were then re-examined to ensure consistency and accuracy of the interpretation.

A detailed flowchart outlining the screening process is presented in Multimedia Appendix 1. The screening and SAM ratings occurred over a 9-month period (July 2020 to March 2021).

The app search was conducted in July 2020. In total, 112 apps were retrieved, 89 from the layman search strategy, and 24 additional from the scraper method. Unfortunately, no apps met the inclusion criteria, highlighting a knowledge gap in this platform for parents and caregivers about children’s heart failure.

The internet search was completed in August 2020. A combined total of 575 websites were retrieved across 6 search terms. Screening of the 455 websites occurred between two screeners (CC and HS). Details of the included web-based pediatric heart failure tools are shown in Multimedia Appendix 2. A total of 16 relevant tools met the inclusion criteria, 13 from the United States and 3 from Canada. An additional relevant tool from the United States was identified following consultation with a subject expert (JC) in the field of children’s heart failure. This tool was not missed in the original search; it was developed and posted on the internet after August 2020. The most relevant internet tools were in the form of webpages (n=13) and handouts (n=3). The content for the relevant tools focused on a varying range of information (eg, general information, symptoms, treatment strategies, and testing).

The average overall SAM factor rating between the 2 reviewers ranged from a low suitability score of 38% (16/42) to a high score of 62% (26/42; Multimedia Appendix 3). The total possible SAM suitability scores were out of 42 (100%). No tool scored 100% (26/26). Overall, 15 tools’ ratings were in the adequate range (40%-69%), and 2 tools’ ratings were in the not suitable range (0%-39%). No tools scored within the superior range (70%-100%).

Each tool was scored individually according to each of the 6 SAM factors in each domain (eg, content, literacy demand, graphics, layout and typography, learning stimulation and motivation, and cultural appropriateness). Raw scores for each factor of the 17 tools were combined on each SAM factor, and a percentage score was calculated, demonstrating the overall current state of web-based tools included in this ES (Multimedia Appendix 4). Overall, most of the tools had a higher reading level than recommended, averaging over a ninth-grade reading level (13/17, 76%). Layout and type scores were all within the superior range—typography (17/17, 100%), layout (10.5/17, 62%), and subheadings (7/17, 41%). In contrast, all graphic scores were in the not suitable range—cover graphics (12.5/17, 74%), type of illustrations (10.5/17, 62%), relevance of graphics (10.5/17, 62%), graphic explanation (17/17, 100%), and graphic caption (16/17, 94%).

This is the first ES to conduct a search for internet sources for parent audiences relating to children’s heart failure within North America. First, our ES identified that no apps exist on this topic, highlighting a significant knowledge gap for parents who are trying to seek information from this digital platform. Furthermore, our research highlighted that 17 web-based tools about children’s heart failure exist tools and were assessed to be adequate using SAM percent ratings.

We have highlighted that a modest number of relevant educational tools exist from our internet search (n=17), with varying degrees of content and health literacy for parent audiences. Notably, 88% (15/17) of the tools found appeared to be developed by clinicians for parents, instead of having parents actively involved in the development process. There were 2 tools that involved parent recommendations and feedback from a parent advisory group but were not created using parent experience evidence. Given the complexity of health journeys for families who have a child with heart failure, there is a critical need and gap to develop a tool based on parents’ lived experience to help deliver tools that are relevant and applicable to parent needs.

Of the relevant internet-based parent tools, no tool scored in the superior range, highlighting that work in the area of health communication and literacy could be improved upon. Most tools scored lower in the summary and review subsection, literacy subsection, and overall graphics section. Improving on these key aspects will provide parents, especially parents with lower literacy skills, information that is easier to understand with improved repetition of key information. However, the field of pediatric cardiology poses its own unique challenge in that this field contains many words with >2 to 3 syllables (eg, echocardiogram or cardiomyopathy). This aspect undoubtedly played a factor that increased the reading level and reduced the score in many of the tools. One of the key informants in the qualitative interviews highlighted their process for dealing with complex medical language to ensure that language was consistent, well defined in simpler terms, and providing information in bite-size pieces for parents and caregivers to enhance their uptake.

In the category of graphics, SAM scores identified that major refinements are needed in this area. An explanation for this lower score was that all relevant tools were website based and it was difficult to score these tools in relation to a graphic cover as suggested in the tools’ instructions. The recommended illustrations that Doak et al [16] outlined in their tool stated that simple line drawings can promote realism without distracting the details. As noted, this tool was developed in 1996 and intended mostly for print materials, so reviewers acknowledge that technology has advanced to include more digitalized, web-based infographics that can be just as impactful as line drawings. Infographics in today’s educational materials involve simpler computer infographics with more vivid, crisp colors that attracts the reader’s attention. Perhaps, updating of the assessment tool to include those aspects would have scored a few of the tools in the superior range, as reviewers found some of the graphics to be well done.

It is imperative that we improve knowledge translation strategies to improve the health literacy of parents and caregivers who care for children with heart failure in the home environment. Knowledge translation strategies that include parents as cocreators bring their unique perspective or lived experience that will improve uptake and understanding, as families in similar contexts will likely share similar knowledge needs [26]. When parents have a lack of understanding toward treatments or health conditions, worse outcomes occur in children’s health [27]. One way to mitigate these poor outcomes is to have robust health information available on the internet, and in alternative formats, that is based on research knowledge and parental lived experience. This is done through the avenues of improving parental and caregiver knowledge bases to make better decisions, reducing parental stress levels and invoking improved conversations with their child’s health provider through questions [28,29]. In addition, when clinicians are armed with credible and effective sources of information that can be easily shared with their parent audience, better relationships will result as parents will have more confidence in their health care provider.

As all tools did not include evidence-based references to indicate that they were developed from peer-reviewed research, the average parent would have difficulty discerning if the material from any of the tools were credible or even evidence-based. Recent published literature has demonstrated that a large portion of parents who searched the internet had difficulty discerning if the literature they found was from a credible source; furthermore, they were not confident in bringing it to a trusted health care professional [29]. Even more troubling is the fact that parents will make health decisions based on the information they find on the internet [28], which may or may not be based on the most credible sources. Despite clinicians’ good intentions of simplifying information to parents by not citing the source of their information, diligently citing evidence where they derived the material may relieve the stress of parents trying to discern whether the tool is evidence-based material.

As this search was conducted in July 2020, it is possible that more tools are now available to parents and caregivers or that the current tools scored in our search have now been updated to reflect different or enhanced content. As we know, the internet and app stores are rapidly adding more content or updating existing materials daily. Our search was only a snapshot in time and would be difficult to replicate the same results.

A limitation of our search was the use of only a single search engine (Google) to provide results, which may have exposed our results to an element of search bias. Published literature on search engines, such as Google, has suggested that theories relating to filter bubbles or personalized algorithms can change results based on who searches for health information [30-32]. Although we took steps to reduce this bias, by deleting cookies and turning off personalization, this is not a perfect process. Another solution would be to use engines, in addition to Google, in the web-based search process to provide more robust results.

Although the scoring of health information was performed using a validated SAM instrument, limitations still remain. We noticed that updates to the tool may be required in some sections that scored lower (eg, type of illustration) as the tool gave a higher score for simple adult-appropriate line drawings that are not congruent with today’s color infographics that can be generated from graphic artists. Current graphics are now designed as colorful infographics and characters, rather than simple line drawings or sketches, which the authors thought to be distracting. Scoring methods indicate that line drawings provide the least amount of distraction [16]. There would have been an improvement in scores if the tool had been updated to include simple computer infographics that are now commonly designed in current educational materials.

In addition, the 2 reviewers found the culture section in the instrument very difficult to score. This was evidenced by the similarity and lack of variability of scores. Culture within the context of children’s heart failure was very difficult to define within the context of pediatric heart failure solely based on a web-based tool or handout. Perhaps more detailed instructions and a definition for culture could be provided, making it more user-friendly. We did find that most graphics included varying types of races and genders among the photos of parents and children. Perhaps if there were tools included in video format, culture scores would vary more as there would be an increased presence of tone and gender role presentations.

This ES sought to explore what multimedia educational information or tools existed on the internet and within app stores for parent audiences about children’s heart failure. From our search, we found 17 parent tools and no apps relating to children’s heart failure that were developed in Canada and the United States. This highlights a gap in knowledge for parents who prefer this type of web-based content for learning about this important topic. Using SAM scoring, most web-based tools scored overall in the adequate range, meaning that they were adequate to teach parents, but there are some key improvements, especially in reading level and graphics, that can be made to maximize their educational effectiveness. The qualitative interviews with key informants who developed the tools highlight three key themes: timely introductory knowledge, credible and trustworthy knowledge, and challenges and points in how organizations plan to evolve this knowledge in the future. Further research is required to evaluate the effectiveness of such parent-targeted tools and their impact on parents’ ability to learn and care for these children more confidently in the home setting.