Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Monday, March 11, 2019 at 4:00 PM to 4:30 PM EST. We apologize in advance for any inconvenience this may cause you.

Who will be affected?

Advertisement

Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Published on 07.06.19 in Vol 2, No 1 (2019): Jan-Jun

Preprints (earlier versions) of this paper are available at http://preprints.jmir.org/preprint/12524, first published Nov 07, 2018.

This paper is in the following e-collection/theme issue:

    Original Paper

    Online Information About Periviable Birth: Quality Assessment

    1Department of Obstetrics and Gynecology, Kalispell Regional Healthcare, Kalispell, MT, United States

    2Department of Obstetrics and Gynecology, Magee-Womens Hospital, University of Pittsburgh, Pittsburgh, PA, United States

    3Department of Obstetrics, Gynecology and Reproductive Sciences, Magee-Womens Research Institute, University of Pittsburgh, Pittsburgh, PA, United States

    Corresponding Author:

    Katherine P Himes, MD, MS

    Department of Obstetrics, Gynecology and Reproductive Sciences

    Magee-Womens Research Institute

    University of Pittsburgh

    300 Halket Street

    Pittsburgh, PA, 15228

    United States

    Phone: 1 412 973 2249

    Email:


    ABSTRACT

    Background: Over 20,000 parents in the United States face the challenge of participating in decisions about whether to use life support for their infants born on the cusp of viability every year. Clinicians must help families grasp complex medical information about their baby’s immediate prognosis as well as the risk for significant long-term morbidity. Patients faced with this decision want supplemental information and frequently seek medical information on the Internet. Empirical evidence about the quality of websites is lacking.

    Objective: We sought to evaluate the quality of online information available about periviable birth and treatment options for infants born at the cusp of viability.

    Methods: We read a counseling script to 20 pregnant participants that included information typically provided by perinatal and neonatal providers when periviable birth is imminent. The women were then asked to list terms they would use to search the Internet if they wanted additional information. Using these search terms, two reviewers evaluated the content of websites obtained via a Google search. We used two metrics to assess the quality of websites. The first was the DISCERN instrument, a validated questionnaire designed to assess the quality of patient-targeted health information for treatment choices. The second metric was the Essential Content Tool (ECT), a tool designed to address key components of counseling around periviable birth as outlined by professional organizations. DISCERN scores were classified as low quality if scores were 2, fair quality if scores were 3, and high quality if scores were 4 or higher. Scores of 6 or higher on the ECT were considered high quality. Interreviewer agreement was assessed by calculated kappa statistic.

    Results: A total of 97 websites were reviewed. Over half (57/97, 59%) were for-profit sites, news stories, or personal blogs; 28% (27/97) were government or medical sites; and 13% (13/97) were nonprofit or advocacy sites. The majority of sites scored poorly in DISCERN questions designed to assess the reliability of information presented as well as data regarding treatment choices. Only 7% (7/97) of the websites were high quality as defined by the DISCERN tool. The majority of sites did not address the essential content defined by the ECT. Importantly, only 18% of websites (17/97) indicated that there are often a number of reasonable approaches to newborn care when faced with periviable birth. Agreement was strong, with kappa ranging from .72 to .91.

    Conclusions: Most information about periviable birth found on the Internet using common search strategies is of low quality. News stories highlighting positive outcomes are disproportionately represented. Few websites discuss comfort care or how treatment decisions impact quality of life.

    JMIR Pediatr Parent 2019;2(1):e12524

    doi:10.2196/12524

    KEYWORDS



    Introduction

    While the rate of survival with or without neurodevelopmental impairment for infants born between 22 and 25 weeks has improved over time, there is considerable uncertainty regarding short- and long-term prognosis for infants born during the periviable period [1-8]. Periviable birth, defined as delivery occurring from 20 0/7 weeks to 25 6/7 weeks, complicates roughly 20,000 deliveries annually in the United States. Survival with neonatal interventions ranges from roughly 10% at 22 weeks gestation to over 60% at 25 completed weeks [1]. The risk of long-term neurodevelopmental impairment remains high for all periviable infants who survive [1-5]. Women and families must navigate this uncertainty to make time-sensitive and value-laden decisions regarding obstetrical interventions for fetal benefit and neonatal care when periviable birth is imminent. Women facing periviable birth have expressed a desire for supplemental information after provider counseling [9-13].

    Increasingly, patients are turning to the Internet for supplemental health information. Survey estimates suggest that over half of people in the United States search the Internet for health information [14]. While data specific to periviable birth are limited, studies of other preference-sensitive decisions suggest that both patients and providers support provision of supplemental information that presents the pros and cons of all reasonable treatment options. Health care providers remain skeptical about the value of the Internet as a source of unbiased supplemental information [11,15].

    The usefulness of the Internet as a source of supplemental health information depends on the quality of information easily available to patients. There are now a number of validated assessment checklists—the DISCERN tool, Journal of American Medical Association Benchmarks, and Health On the Net Foundation Principles—to determine the quality of online information [16-18]. The aim of this study was to evaluate data regarding the quality of online information about periviable birth.


    Methods

    Defining Internet Search Terms

    We recruited pregnant women at gestation of 37 weeks or more from the Magee-Womens Hospital, Pittsburgh, PA, outpatient obstetrical clinics to participate in this study during June 2016. Women were excluded if they had experienced a prior preterm birth or had experienced threatened preterm labor during their current pregnancy. We approached all women meeting inclusion criteria. Following informed consent, women were read a script describing a hypothetical situation in which they were faced with a preterm birth at 23-weeks’ gestation. The script included information on short- and long-term outcomes, risks and benefits of obstetrical interventions for neonatal benefit, as well as care options for the neonate after delivery. The script was developed by input from members of the division of Maternal Fetal Medicine and Newborn Medicine at the University of Pittsburgh Medical Center, Pittsburgh, PA. Participants were then given a written survey asking them to identify the search engine they used most frequently and how often they searched for health information online. They were then asked to list four search terms they would use to seek out additional information about periviable birth. They were also asked whether they would prefer to learn about (1) statistics regarding survival and outcomes for periviable neonates, (2) narrative descriptions about women who delivered in the periviable period and the outcomes for their neonates, or (3) both. Women were recruited until we reached saturation in themes of search terms. We did not recruit partners of pregnant women as they were not readily present at the clinical venues where we recruited participants.

    Website Identification

    Using the four most common search terms provided, two reviewers (AFH and CAZ) independently evaluated the informational content of websites found via a Google search. Additional websites listed by participants were WebMD and Wikipedia. Sites were excluded if they were non-English or if registration was required for access. Eligible websites were then independently reviewed on the same date. The first three pages of websites using the top four search terms were included in the evaluation, as marketing data demonstrate that only 1.6% of users click on links from the third page of results or beyond [19].

    Analysis of Website Quality

    Websites were classified into six main categories: (1) government, (2) scientific resources, (3) nonprofit and advocacy organizations, (4) news and media reports, (5) for-profit organizations, and (6) personal commentary (eg, personal blogs). Two metrics were used to evaluate the quality and content of the websites. The first was the DISCERN instrument, a validated questionnaire designed to assess the quality of patient-targeted health information for treatment choices [20]. We elected to use this tool because it is tailored to evaluate how well websites prepare patients and providers to engage in a shared decision-making process, the approach recommended to help families make decisions about neonatal care after periviable birth. Questions included in the DISCERN instrument fall into three main sections. Section 1 (Questions 1-8) addresses the reliability of the site and whether it can be trusted as a source of information about treatment choices. These questions address the sources of information used to compile the site, the website’s ability to provide accurate and impartial information, and areas of uncertainty regarding treatment choices. Section 2 (Questions 9-15) focuses on specific details of the information about treatment choices. These questions examine the risks and benefits of each treatment choice and how well the site supports shared decision making. Section 3 (Question 16) is a subjective assessment by the reviewer of the overall quality of the website. Each question on the DISCERN tool is scored on a scale of 1 (low quality/not addressed) to 5 (high quality/fully addressed). We considered scores of 4-5 to be high quality, 3 to be fair quality, and 1-2 to be poor quality sources of information.

    The DISCERN tool can be used for any health-related content area and, thus, is not specific to periviable birth. Therefore, a second metric was developed—the Essential Content Tool (ECT)—to address whether websites covered information defined as critical for decision making around periviable birth. This tool was developed in two phases. First, we extracted 21 key components of periviable counseling as outlined by both the American College of Obstetrics and Gynecology and the American Academy of Pediatrics statements about periviable birth [21,22]. Content validity was evaluated by querying four physician members of the divisions of Maternal Fetal Medicine and Newborn Medicine to identify the minimum number of constructs necessary to provide families with essential information about periviable birth decision making. This process identified 10 essential topics pertinent to periviable birth. These included information about short- and long-term neonatal morbidity and mortality at different gestational ages as well as obstetrical interventions for neonatal benefit and options for neonatal care. We also assessed whether information was presented in a preference-sensitive manner: there is no right or wrong decision. All 10 domains are outlined in Multimedia Appendix 1. Thus, the ECT contains a total of 10 content questions that were scored dichotomously (present or not present). A website was considered of good quality if it addressed at least six of the content domains and high quality if it addressed at least eight of the content domains.

    Statistical Analysis

    Descriptive statistics were calculated to characterize participants and summarize results of the DISCERN tool and the ECT. Interreviewer agreement was assessed by a calculated kappa statistic and descriptive statistics were performed. This study was approved by the University of Pittsburgh’s Institutional Review Board.


    Results

    Patient Population and Search Terms

    A total of 20 women were recruited for the first stage of the study. The median age of our patient population at the time of enrollment was 28 years (interquartile range [IQR] 25-30), 35% (7/20) had Medicaid insurance, 65% (13/20) had private insurance, and 50% (10/20) of the women were black. A total of 90% (18/20) of women had graduated high school. The median gestational age was 38.0 weeks (IQR 37.6-38.6) and 40% (8/20) were nulliparous. Of those surveyed, 85% (17/20) chose Google as their preferred search engine and 75% (15/20) reported they used the Internet to search for health information often or all the time.

    Patients were asked to provide four search terms they would use to search for supplemental information online regarding periviable birth. Overall, participants recorded a total of 54 search terms. The four most common search terms elicited were (1) preterm birth, (2) birth at 23 weeks, (3) long-term effects of preterm birth, and (4) chances of survival at 23 weeks. These four responses or minor variations of these responses (ie, long-term complications of preterm birth) accounted for 83% (45/54) of total search terms provided by participants. Other search terms recorded by participants included the following: recurrence risk of preterm birth, prevention of preterm birth, resources available for people with preterm infants, and care decisions made by families facing periviable birth. Patients overwhelmingly desired both statistical information about survival and prognosis as well as patient narratives about their experiences with periviable birth, with 90% of women indicating they wanted both types of information.

    Website Characteristics

    After exclusion criteria were applied, 97 unique websites were reviewed out of a total of 120 possible websites (80.8%). Of those queried, 59% (57/97) were for-profit sites, news stories, or personal blogs; 28% (27/97) were government, hospital or research institution, or medical journal sites; and 13% (13/97) were nonprofit or advocacy sites. Most websites were from the United States (75/97, 77%), followed by the United Kingdom (17/97, 18%), Australia (3/97, 3%), and New Zealand (2/97, 2%).

    Quality of Website Content: The DISCERN Tool

    The DISCERN tool evaluates the reliability of website information as well as the quality of information about treatment choices. The reviewers defined “treatment” as neonatal interventions with the goal of sustaining the life of the neonate. Therefore, when the DISCERN tool asks if the website “addresses what would happen if no treatment was used or if there are alternatives to treatment,” we interpreted this as the website referring to the option of comfort care and subsequent neonatal death.

    Overall, websites scored poorly. A website could score a total of 80 points on 16 questions, 40 points on questions addressing the reliability of information (Questions 1-8), and 35 points on the questions addressing treatment choices (Questions 9-15). High scores indicate high quality. The distribution of scores is shown in Figure 1. While website quality was poor overall (see Figure 1A) with a median overall score of 36 (IQR 30-44), websites fell markedly short at discussing treatment options (see Figure 1C). The median score for the questions addressing treatment choices was 8 (IQR 7-13).

    As indicated above (see Figure 1C), information on treatment choices was poor and this was particularly notable for questions addressing support for shared decision making. Only 10% (10/97) of the websites scored highly (4 or 5) on the question of whether it was clear there was more than one reasonable treatment choice or medical approach. A total of 68% of websites (66/97) scored poorly (<2) in acknowledging uncertainty around the best treatment options. Importantly, 16% (16/97) had evidence of strong bias indicating a completely unbalanced view of options available to patients experiencing a periviable birth. The bias was uniformly in favor of presenting the option of a trial of resuscitation and not presenting comfort care as an option.

    Figure 1. Distribution of DISCERN scores overall (A), with regard to reliability of information (B) and information on treatment choices (C).
    View this figure

    Website Content Using the Essential Content Tool

    The ECT provides a more granular assessment of the quality of information as it pertains to periviable birth. Consistent with the findings using the DISCERN tool, most websites lacked the essential content necessary for periviable decision making as defined by the ECT. Only 38 of the 97 sites (39%) reviewed morbidity and mortality statistics by gestational age, and these were often not reflective of the current literature [1]. Roughly a quarter (27/97, 28%) reviewed obstetrical interventions for neonatal benefit, such as administration of betamethasone or magnesium for neuroprotection. Interestingly, the content area most commonly addressed by the queried websites (59/97, 61%) was the potential long-term outcomes for periviable neonates (eg, cognitive impairment, cerebral palsy, deafness, blindness, feeding difficulties, and oxygen requirement). Consistent with our findings from the DISCERN tool, only 18% (17/97) of websites presented a full range of care options for periviable infants, including comfort care. Few websites (10/97, 10%) characterized decision making around periviable birth as preference-sensitive, indicating that there is often no clear right or wrong choice in deciding to pursue life-sustaining interventions versus comfort care for the neonate. The complete distribution of scores for ECT are included in Multimedia Appendix 1.

    Overall Website Quality

    The overall quality of website content was measured using the last question on the DISCERN tool and by the presence of six out of 10 essential domains as defined by the ECT. Only 7% (7/97) of the websites were thought to represent high-quality websites by both reviewers using the last question on the DISCERN tool, and 2% (2/97) were considered high quality according to the ECT. A total of 20% (19/97) of the websites were of good quality according to the ECT. Notably, over half (5/9, 56%) of the high-quality websites were found by our reviewers on page 2 or 3 of the search results. Additionally, 74% (14/19) of good-quality websites were from academic sources, governmental sources, or nonprofit organizations. Only 2 out of 19 (11%) of the good-quality websites according to the ECT were from a for-profit site. To illustrate high- and low-quality websites, we provided quotes demonstrative of the websites’ messages. We found that many websites, particularly news stories and personal blogs, focused on sensational stories that do not always represent the more likely outcomes (see Table 1).

    Interreviewer agreement was substantial with kappa ranging from .72 to .90 for the ECT questions and from .36 to .71 for the individual DISCERN questions. The stronger agreement within the ECT can be explained by the binary nature of the questions, while the DISCERN questions are answered using a 1-5 scale. When using a weighted kappa statistic to account for the degree of disagreement within the DISCERN questions, agreement was strong, with kappa ranging from .66 to .91 for the individual DISCERN questions.

    Table 1. Content from high- and low-quality websites.
    View this table

    Discussion

    Principal Findings

    The American College of Obstetricians and Gynecologists, the Society for Maternal Fetal Medicine, and the American Academy of Pediatrics all emphasize the need for shared decision making regarding interventions for infants born during the periviable period [21,22]. Given patients frequently access the Internet for supplemental health information when faced with medical decision making, we sought to provide data regarding the quality of online information as it pertains to periviable delivery [11,12]. Our work revealed that the overwhelming majority of websites do not address content considered essential for patient education about the difficult decisions surrounding periviable birth. Less than 20% of websites addressed comfort care as a treatment option in periviable neonates and, if mentioned, it was often a single sentence. Furthermore, only 10% of websites acknowledged that there is more than one reasonable approach to care—a frame that is critical to supporting shared decision making. Additionally, over 15% of websites were found to have strong bias in favor of neonatal resuscitation, potentially further eroding high preference-based decision making.

    Roughly one in 10 websites were considered high quality by the DISCERN tool. The websites receiving the highest scores were nonprofit sites and government sites (eg, Wikipedia and the Australian Perinatal Practice Guidelines). Of concern, the highest-quality websites were often difficult to find within our search, often listed on the second or third page of the results. Prior research has shown that 91% of Internet users do not go beyond the first page of search results [15]. The websites that Google ranks on the first page of their search results for any given search term are those considered the most relevant using a complex algorithm, which is constantly updated and revised. Therefore, it can be difficult to move high-quality sites to the front of the search results without search engine optimization consultants [26].

    Prior studies have investigated the quality of online information for other preference-sensitive decisions, including treatment for localized breast cancer and treatment options for prostate cancer [27,28]. Consistent with our results, these studies concluded that although many websites address these topics, very few provide essential information necessary to actively participate in decision making related to treatment options. These studies suggest that there is a potential for the Internet to provide valuable information for patients, but it is up to their health care providers to identify high-quality websites to guide their patients.

    Our study has several important strengths. First, ours is the first investigation to specifically investigate the quality of online information as it pertains to periviable birth. Additionally, the search terms we used were defined by patients and thus likely reflect the usual search strategies employed by the general population. Furthermore, we used two metrics to evaluate the individual websites—both a validated tool to assess the quality of information for preference-sensitive decisions as well as a content tool that specifically assessed information related to periviable birth. This approach allowed for a multidimensional evaluation of the websites that addressed both content-specific as well as big-picture concepts that support high-quality shared decision making. Finally, interviewer agreement was found to be strong after our analysis.

    Our study also has several important limitations. First, the Internet search terms used in our study were generated by pregnant women at term, rather than women facing imminent periviable birth. This approach was taken after extensive deliberation with the study team and review board. We weighed the risk of the added stress and anxiety that participating in this study would cause a woman facing periviable birth versus the minimal added benefit that would come from asking the intended population to generate the search terms for our study. At the conclusion of our discussions, we felt a hypothetical approach with term women would generate realistic and useful search terms and opted for this approach. Second, we attempted to recreate how the patients would search the Internet for additional information regarding treatment options and outcomes surrounding a periviable birth and thus only one search engine was used, which may limit the generalizability of our findings. This search engine, however, was overwhelmingly picked as the search engine of choice by our sample population and still provided a large amount of pilot data. Third, the review of the websites was completed on personal computers, as opposed to mobile phones, which are often the modality used by patients. Over two-thirds of Americans now own a mobile phone, and 62% of mobile phone owners use their mobile phone to get information about a health care condition [14]. In light of this, it is possible that the search engine may have organized websites differently, and the websites themselves may be navigated differently for mobile phone users. We used a validated questionnaire specifically designed to assess the quality of patient-targeted health information for treatment choices. There are other elements of website design that could meaningfully impact the user experience and patient education that we did not assess. These include ease of readability and use of illustrative graphics. Neither the ECT nor the DISCERN tool include patient perspectives in what they would define as a high-quality website, and we acknowledge that patients may desire different content within websites than physicians. We did not take into account the idea that parents have different values and the role of incorporating their values into decision making regarding periviable delivery, but these are important areas of future research. Finally, we did not consider the Internet self-efficacy or health literacy of our population. These are important areas for future investigation.

    Although not the major focus of our study, 90% of our survey population indicated that in addition to high-quality information about prognosis and treatment for periviable neonates, they also wanted to learn about other patients’ experiences with periviable birth and periviable decision making. This suggests that there is a patient desire for the inclusion of narratives in the educational component of preference-sensitive counseling and that providing these narratives in a balanced fashion may be important. It also speaks to potential gaps in counseling by health care providers. Understanding these gaps is an important research effort of our group as well as others [12]. We hope that the information generated by this pilot study will lead to a larger, comprehensive review of online resources available to patients facing a periviable birth.

    Conclusions

    Moving forward, the creation of an evidence-based Internet resource that addresses both short- and long-term neonatal outcomes, patients’ values, the importance of shared decision making, and the option for comfort care should be developed to help parents make treatment decisions when facing a periviable delivery.

    Acknowledgments

    All phases of this study were supported by a grant from the Beckwith Institute. The funding source had no role in study design or analysis of data.

    Conflicts of Interest

    None declared.

    Multimedia Appendix 1

    The 10 essential topics pertinent to periviable birth.

    PPTX File, 2MB

    References

    1. Younge N, Goldstein RF, Bann CM, Hintz SR, Patel RM, Smith PB, Eunice Kennedy Shriver National Institute of Child Health and Human Development Neonatal Research Network. Survival and neurodevelopmental outcomes among periviable infants. N Engl J Med 2017 Dec 16;376(7):617-628 [FREE Full text] [CrossRef] [Medline]
    2. Stoll BJ, Hansen NI, Bell EF, Shankaran S, Laptook AR, Walsh MC, Eunice Kennedy Shriver National Institute of Child Health and Human Development Neonatal Research Network. Neonatal outcomes of extremely preterm infants from the NICHD Neonatal Research Network. Pediatrics 2010 Sep;126(3):443-456 [FREE Full text] [CrossRef] [Medline]
    3. Hintz S, Kendrick D, Vohr B, Poole W, Higgins R, National Institute of Child Health and Human Development Neonatal Research Network. Changes in neurodevelopmental outcomes at 18 to 22 months' corrected age among infants of less than 25 weeks' gestational age born in 1993-1999. Pediatrics 2005 Jun;115(6):1645-1651. [CrossRef] [Medline]
    4. Hintz S, Kendrick D, Wilson-Costello D, Das A, Bell EF, Vohr BR, NICHD Neonatal Research Network. Early-childhood neurodevelopmental outcomes are not improving for infants born at <25 weeks' gestational age. Pediatrics 2011 Jan;127(1):62-70 [FREE Full text] [CrossRef] [Medline]
    5. Mercier CE, Dunn MS, Ferrelli KR, Howard DB, Soll RF, Vermont Oxford Network ELBW Infant Follow-Up Study Group. Neurodevelopmental outcome of extremely low birth weight infants from the Vermont Oxford network: 1998-2003. Neonatology 2010 Jun;97(4):329-338 [FREE Full text] [CrossRef] [Medline]
    6. Rysavy M, Li L, Bell E, Das A, Hintz SR, Stoll BJ, Eunice Kennedy Shriver National Institute of Child Health and Human Development Neonatal Research Network. Between-hospital variation in treatment and outcomes in extremely preterm infants. N Engl J Med 2015 May 07;372(19):1801-1811 [FREE Full text] [CrossRef] [Medline]
    7. Kusuda S, Fujimura M, Uchiyama A, Totsu S, Matsunami K, Neonatal Research Network‚ Japan. Trends in morbidity and mortality among very-low-birth-weight infants from 2003 to 2008 in Japan. Pediatr Res 2012 Nov;72(5):531-538 [FREE Full text] [CrossRef] [Medline]
    8. Stoll B, Hansen N, Bell E, Walsh MC, Carlo WA, Shankaran S, Eunice Kennedy Shriver National Institute of Child Health and Human Development Neonatal Research Network. Trends in care practices, morbidity, and mortality of extremely preterm neonates, 1993-2012. JAMA 2015 Sep 08;314(10):1039-1051 [FREE Full text] [CrossRef] [Medline]
    9. Kaempf J, Tomlinson S, Campbell B, Ferguson L, Stewart V. Counseling pregnant women who may deliver extremely premature infants: Medical care guidelines, family choices, and neonatal outcomes. Pediatrics 2009 Jun;123(6):1509-1515. [CrossRef] [Medline]
    10. Ohlinger J, Kantak A, Lavin J, Fofah O, Hagen E, Suresh G, et al. Evaluation and development of potentially better practices for perinatal and neonatal communication and collaboration. Pediatrics 2006 Nov;118 Suppl 2:S147-S152. [CrossRef] [Medline]
    11. Grobman W, Kavanaugh K, Moro T, DeRegnier R, Savage T. Providing advice to parents for women at acutely high risk of periviable delivery. Obstet Gynecol 2010 May;115(5):904-909 [FREE Full text] [CrossRef] [Medline]
    12. Tucker Edmonds B, Savage T, Kimura R, Kilpatrick S, Kuppermann M, Grobman W, et al. Prospective parents' perspectives on antenatal decision making for the anticipated birth of a periviable infant. J Matern Fetal Neonatal Med 2019 Mar;32(5):820-825. [CrossRef] [Medline]
    13. Raju T, Mercer B, Burchfield D, Joseph G. Periviable birth: Executive summary of a joint workshop by the Eunice Kennedy Shriver National Institute of Child Health and Human Development, Society for Maternal-Fetal Medicine, American Academy of Pediatrics, and American College of Obstetricians and Gynecologists. Obstet Gynecol 2014 May;123(5):1083-1096. [CrossRef] [Medline]
    14. Smith A. Pew Research Center. Washington, DC: Pew Internet & American Life Project; 2015 Apr 01. US smartphone use in 2015   URL: https://www.pewinternet.org/2015/04/01/us-smartphone-use-in-2015/ [accessed 2019-06-03] [WebCite Cache]
    15. van Deursen AJ. Internet skill-related problems in accessing online health information. Int J Med Inform 2012 Jan;81(1):61-72. [CrossRef] [Medline]
    16. Charnock D, Shepperd S. Learning to DISCERN online: Applying an appraisal tool to health websites in a workshop setting. Health Educ Res 2004 Aug;19(4):440-446. [CrossRef] [Medline]
    17. Silberg W, Lundberg G, Musacchio R. Assessing, controlling, and assuring the quality of medical information on the Internet: Caveant lector et viewor--Let the reader and viewer beware. JAMA 1997 Apr 16;277(15):1244-1245. [Medline]
    18. Boyer C, Selby M, Appel R. The Health On the Net Code of Conduct for medical and health web sites. Stud Health Technol Inform 1998;52 Pt 2:1163-1166. [Medline]
    19. Ghita ML, Loiz D, Petrescu P. Advanced Web Ranking. Craiova, Romania: Advanced Web Ranking; 2014. Google Organic CTR Study   URL: https://www.advancedwebranking.com/ctrstudy/ [accessed 2019-06-03] [WebCite Cache]
    20. Charnock D, Shepperd S, Needham G, Gann R. DISCERN: An instrument for judging the quality of written consumer health information on treatment choices. J Epidemiol Community Health 1999 Feb;53(2):105-111 [FREE Full text] [CrossRef] [Medline]
    21. Cummings J, Committee on Fetus and Newborn. Antenatal counseling regarding resuscitation and intensive care before 25 weeks of gestation. Pediatrics 2015 Sep;136(3):588-595 [FREE Full text] [CrossRef] [Medline]
    22. American College of Obstetricians and Gynecologists and the Society for Maternal–Fetal Medicine, Ecker JL, Kaimal A, Mercer BM, Blackwell SC, deRegnier RA, et al. #3: Periviable birth. Am J Obstet Gynecol 2015 Nov;213(5):604-614. [CrossRef] [Medline]
    23. South Australian Perinatal Practice Guidelines Workgroup. South Australian Perinatal Practice Guidelines. Parent Information: For Babies Born 23-24 Weeks. Adelaide, Australia: Department of Health, Government of South Australia; 2013 Sep 30.   URL: https:/​/www.​sahealth.sa.gov.au/​wps/​wcm/​connect/​d1024b0042ac3768a346b7ad100c470d/​Too+small+too+soon+-+babies+born+23%E2%80%9324+weeks.​pdf?MOD=AJPERES&CACHEID=d1024b0042ac3768a346b7ad100c470d [accessed 2019-05-22] [WebCite Cache]
    24. Gandhi A. Patient. 2014 Sep 25. Premature babies and their problems   URL: https://patient.info/doctor/premature-babies-and-their-problems [accessed 2019-05-22] [WebCite Cache]
    25. Institute of Medicine. In: Butler AS, Behrman RE, editors. Preterm Birth: Causes, Consequences, and Prevention. Washington, DC: National Academies Press; 2007.
    26. Yates E, Dixon L. PageRank as a method to rank biomedical literature by importance. Source Code Biol Med 2015;10:16 [FREE Full text] [CrossRef] [Medline]
    27. Bruce J, Tucholka J, Steffens N, Neuman H. Quality of online information to support patient decision-making in breast cancer surgery. J Surg Oncol 2015 Nov;112(6):575-580 [FREE Full text] [CrossRef] [Medline]
    28. Borgmann H, Wölm JH, Vallo S, Mager R, Huber J, Breyer J, et al. Prostate cancer on the Web: Expedient tool for patients' decision-making? J Cancer Educ 2017 Dec;32(1):135-140. [CrossRef] [Medline]


    Abbreviations

    ECT: Essential Content Tool
    IQR: interquartile range


    Edited by G Eysenbach; submitted 07.11.18; peer-reviewed by A Chung, J Jenkinson, J Taylor, B Van den Borne; comments to author 18.02.19; revised version received 29.03.19; accepted 26.04.19; published 07.06.19

    ©Adriane F Haragan, Carly A Zuwiala, Katherine P Himes. Originally published in JMIR Pediatrics and Parenting (http://pediatrics.jmir.org), 07.06.2019.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Pediatrics and Parenting, is properly cited. The complete bibliographic information, a link to the original publication on http://pediatrics.jmir.org, as well as this copyright and license information must be included.