Published on in Vol 8 (2025)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/75569, first published .
An Information App (e-TOP) to Support Parents’ Information Needs During the First Year at Home After Preterm Birth: Development and Usability Study

An Information App (e-TOP) to Support Parents’ Information Needs During the First Year at Home After Preterm Birth: Development and Usability Study

An Information App (e-TOP) to Support Parents’ Information Needs During the First Year at Home After Preterm Birth: Development and Usability Study

1Department of Rehabilitation, Amsterdam Reproduction and Development, Amsterdam UMC Location University of Amsterdam, Meibergdreef 9, Amsterdam, The Netherlands

2Centre of Expertise Urban Vitality, Faculty of Health, Sports and Physical Activity, Amsterdam University of Applied Sciences, Amsterdam, The Netherlands

3Department of Rehabilitation, Amsterdam University Medical Centers, Amsterdam, The Netherlands

4Faculty of Digital Media and Creative Industry, Amsterdam University of Applied Sciences, Amsterdam, The Netherlands

5Amsterdam University Medical Center, Emma Kinderziekenhuis, Amsterdam, The Netherlands

6Department of Neonatology, Amsterdam Reproduction and Development, Amsterdam UMC Location University of Amsterdam, Amsterdam, The Netherlands

Corresponding Author:

Monique Flierman, MSc


Background: Parents of preterm infants often face challenges in transitioning from hospital to home, requiring reliable and accessible information to support their caregiving. Mobile health interventions have the potential to supplement postdischarge education and empower parents by providing tailored, evidence-based information.

Objective: The primary objectives of this study were to develop an information app (e-TOP) using a participatory design approach, incorporating input from parents and health care professionals, and evaluate its usability.

Methods: A 2-phase, mixed methods design was used for this study. In Phase 1, the app was developed through iterative focus group discussions with parents of preterm infants, parents with limited health literacy, and TOP interventionists. In Phase 2, a usability study was conducted over 6 months with parents of preterm infants. Usability was assessed with a broad range of measurements: thinking aloud sessions, user engagement analytics, 2 questionnaires, the system usability scale (SUS), a customized satisfaction survey, open question sections, and semistructured interviews.

Results: The collaborative approach with end users and experts for the development led to a fully functional e-TOP app. Expert review and content validation ensured that information was clinically accurate, accessible, and relevant to parental needs. For the usability testing, a total of 58 families (116 participants) were recruited and 69 participants actively used the app. The median cumulative e-TOP usage per participant in 6 months (26 weeks) was 39 minutes (IQR 8.8‐53.0). The median number of actions was 64.0 (IQR 33.5‐88.0). The e-TOP app received a median SUS score of 75 (IQR 67.5‐80.0), indicating good usability. Participants rated their overall median satisfaction at 7.0 (IQR 7.0‐8.0) out of 10. While the app was perceived as useful for finding information on prematurity-specific topics, engagement declined over time. Qualitative feedback highlighted a need for improved navigation (eg, a search function), expanded content (eg, motor development and sensory processing), and more interactive features (eg, chat support and parental community forums).

Conclusions: The e-TOP app is a valuable digital resource that can supplement postdischarge care by providing tailored, evidence-based, and accessible information for parents of preterm infants. While usability scores were high, engagement trends and retrieved feedback suggest the need for enhanced retention strategies, such as push notifications, timeline-based navigation structure, interactive tools, and adding (practical) content. Future iterations to improve the inclusivity of the e-TOP app require strategies to engage fathers and parents with limited health literacy skills.

Trial Registration: ISRCTN Registry ISRCTN65709138; https://www.isrctn.com/ISRCTN65709138

JMIR Pediatr Parent 2025;8:e75569

doi:10.2196/75569

Keywords



Background

In the Netherlands, 6.9 % of the children are born prematurely (before 37 weeks of gestational age) [1]. Although advances in perinatal and neonatal care have significantly improved survival rates for preterm infants [2], preterm birth still accounts for 66% of all deaths occurring before or during the first month after birth [3]. Prematurity also has large adverse effects on children’s motor, cognitive, and socioemotional development [4]. Giving birth to a premature infant is also a disruptive start for parents [5]. Parents of preterm infants often experience a range of mental health problems, such as anxiety and depression [6]. The dependency on medical and health professionals can challenge the formation of a secure infant-parent relationship and parents’ self-efficacy in their ability to independently care for their infant [7]. The transition to home following discharge is known as challenging. Without the 24/7 access to health care professionals, parents of preterm infants feel unprepared for this new situation and lack confidence in their capacity to care for their infant [8]. Despite efforts to improve parents’ readiness for discharge and the availability of follow-up services, many parents of preterm infants continue to experience unmet needs for support and practical information after leaving the hospital [9,10]. As the infant’s care needs and developmental milestones evolve during the first year, the informational needs of parents of preterm infants also shift accordingly. Providing tailored information about the consequences of prematurity and the specific health, behavioral, and developmental needs of premature infants can strengthen parental self-efficacy, enhance their sense of security in caring for their infant, and potentially reduce health care use [11,12].

Search for Information

After returning home, parents of preterm infants search the internet for information on topics, such as infant crying, choice of formula, common illnesses, and developmental stages. While they use search engines, social media, or mobile health (mHealth) apps, finding appropriate and reliable information on the unique challenges of caring for a preterm infant at home remains difficult [8,9]. The use of mHealth could support a shift from time- and professional-dependent access to information toward a more autonomous approach, empowering parents to obtain information whenever and wherever they need it. However, for mHealth apps to be useful and effective, they must address the specific information needs of parents of preterm infants [13,14]. In addition, the app must be user-friendly [15] and the information accessible and easy to understand for parents across varying levels of education and health literacy [16].

mHealth Apps

mHealth education interventions for parents of premature infants were developed over the last decades. However, Richardson’s evaluation of mobile apps targeting parents of infants in the neonatal intensive care unit (NICU) revealed that many apps are lacking quality and credibility or do not contain content identified by parents as important [17]. In contrast, the study on the NICU2HOME app provides high-quality information on infant and self-care during and after the NICU stay and showed promising results in enhancing parental self-efficacy and discharge preparedness [18]. Despite the rich body of literature regarding parental needs for information after discharge, available apps targeting the postdischarge period are limited. A systematic app review of mobile information apps for parents of preterm infants after hospital discharge found that, while the overall quality of the identified apps was acceptable, they insufficiently addressed key postdischarge concerns, such as information on transitions in nutrition, sleep patterns, or developmental stages [19]. This lack of targeted content may contribute to the limited usage and adoption of these apps, a common pitfall when end users and stakeholders are not adequately involved in the design process [20]. Participatory design approaches, such as the design thinking method, ensure that user requirements are accurately captured from the perspective of the end users, thereby enhancing usability and facilitating effective implementation [21-24].

The Dutch Context: TOP Program

In the Netherlands, a 1-year home-based responsive parenting intervention (TOP program) is standard care for very preterm infants and their parents and is currently expanded for moderate to late preterm infants. Pediatric physical therapists with an additional 1-year TOP training and affiliated with the Dutch Expertise Center for Premature Infants (EOP) carry out the TOP program. The strength-based and process-oriented intervention consists of 7 key strategies to enhance parental understanding of the infants’ behavioral and developmental needs [25]. An important key strategy is to provide parents with information about their infant’s behavioral and developmental needs. To enhance parents’ knowledge and understanding of their infants’ needs, TOP interventionists provide verbal information transfer and an individualized written parental report with strength-based recommendations accompanied by photos taken during the home visit. Despite the benefits of face-to-face care, there is a need for a complementary mHealth intervention that enhances information retention, supports parental confidence in managing their infant’s care, and potentially reduces their reliance on health care professionals [9,14]. Therefore, we aim to complement and enhance the existing Dutch TOP program by addressing gaps in information retention and supporting professionals in delivering reliable information to parents. Before conducting more extensive research to measure its effectiveness, developing an information app that ensures that user requirements are effectively incorporated and is tailored to the specific needs of parents of preterm infants is crucial.

The aim of this study was two-fold: (1) to develop a postdischarge information app (e-TOP) for parents of premature infants and (2) to evaluate its usability.


Phase 1 Methods

Setting

The study was performed between October 2021 and November 2023 by the EOP, affiliated with the Amsterdam University Medical Center, in collaboration with the Amsterdam University of Applied Sciences.

Ethical Considerations

The study was approved by the Medical Ethical Committee of the Stichting Amsterdam UMC (NL78996.018.21). All participants received an information letter detailing the study’s objectives, timeline, content, target group, data management practices, and privacy considerations. Subsequently, informed consent forms were obtained. Participants were officially enrolled in the study upon providing written consent. Data were pseudonymized. Participants did not receive any compensation.

Participants

The user-centered design approach for the development of the prototype started with 3 focus groups, 1 with TOP interventionists and 2 with parents of preterm infants. A pragmatic sampling approach was used, drawing on available data from the EOP center to identify and invite participants (TOP interventionists) with relevant experience across different contexts [26]. For focus group 1, TOP interventionists with more than 5 years of experience, representing different geographic and socioeconomic regions, were approached. For the parent focus groups, participating TOP interventionists approached potential eligible and interested families within their caseload. Parents were included if they (1) participated in the TOP program for at least 3 months and (2) were able to understand and speak the Dutch language. Focus group 3 specifically involved parents of preterm infants with limited health literacy (LHL). These parents were identified and recruited by the TOP interventionist by using the following 2 additional criteria: (1) parents struggle with understanding and applying the information provided during the intervention or in the parent report and (2) fulfill the criteria for LHL according to the Dutch checklist to recognize LHL [27].

Focus Group Meetings

The script for the focus group meetings was developed collaboratively with the research team, guided by the 2 main objectives of the sessions (Multimedia Appendix 1). First, to verify the identified need for information on the 10 topics previously found in our qualitative research [13] and to enrich these with participants’ personal experiences. Second, to elicit preferences regarding content, topics, design, and navigation of the app.

To address the first objective, the research team formulated open-ended questions and prompts for each of the 10 topics. These were designed to stimulate reflection, encourage sharing of personal experiences, and allow for the addition of new themes. Examples and short scenarios were incorporated to help participants relate the topics to their own context.

For the second objective, the script included guided activities involving visual and textual materials, such as screenshots of existing parenting apps, information cards, mock-ups, and text samples. These were selected and designed by the research team to support discussion on specific design features (eg, tone of voice, level of detail, and visual presentation) and to facilitate comparison between different styles. See Figure 1; information cards and Figure 2; mock-ups.

Figure 1. Information cards.
Figure 2. Mock-up.

The draft script was reviewed in 2 rounds by the research team to ensure alignment with the study objectives, clarity of language, and feasibility within the planned 2-hour sessions. Minor adjustments were made following pilot testing with a small group of colleagues to refine the sequence of questions and ensure smooth transitions between topics.

We conducted 3 focus group sessions. In the first meeting, 8 female TOP interventionists participated. Five interventionists worked in an urban setting with a diverse population in a challenging socioeconomic environment.

In the second focus group, 9 parents (5 mothers and 4 fathers) participated. Their mean age was 35.6 years (SD 5.8). For 5 parents, it was their firstborn, and for the other 4 parents, it was their second child. Their infants’ mean gestational age was 30 weeks and 2 days (SD 19.0). In the third focus group meeting, 6 mothers with LHL participated with a mean age of 31.6 years (SD 2.2). For 5 mothers, it was their firstborn infant, and for 1 mother, her third child. Their infants’ mean gestational age was 29 weeks and 1 day (SD 21.8).

The three 2-hour online focus group meetings were organized in the evenings, which was indicated by participants as the most convenient time. The focus groups were facilitated by an independent female researcher, MM. She was not involved in the study design or analysis. The first author (MF) attended the focus groups to take field notes but did not lead the discussions. The 3 focus group meetings were audio-recorded, relistened to, and summarized. Findings were discussed within the research team.

Phase 1 Results

Overview

The results of the focus group discussions with TOP interventionists and parents provided valuable insights that shaped the development of the e-TOP app, directly influencing its content and design refinements. TOP interventionists recommended not to use the age-based developmental norms strictly, as individual variation among preterm infants is substantial. Instead of providing medical guidance, the app should direct parents to contact their general practitioner or pediatrician when necessary. Additional topics identified by participants were the digestive system, reflux, constipation, medication management, sensory overload and stimulation, parent-infant attachment, and bonding. Parents expressed a preference for clear, supportive, and actionable information. Information should have a layered content structure, allowing simplified messages and scientifically substantiated information. A summary of these findings is provided in Table 1. We used them as a guideline for the further development of the e-TOP app.

Table 1. Results of focus group meetings.
TopicsFocus group 1aFocus group 2bFocus group 3c
Participants’ associations with a digital information source.
  • Reliable and user-friendly digital source aligns well with the needs of young parents.
  • An app can offer practical guidance to support everyday care.
  • Given the limited experience of general practitioners and Dutch health services with prematurity, the app could also be a valuable source.
  • Participants expressed curiosity and recognized the usefulness of an app in addressing their concerns.
What questions do parents have?
  • Ten topics most frequently searched align with participants’ concerns, validating the app’s content. In addition, practical information on adequate sensory input and the use of slings is needed.
  • Different experiences with gavage feeding at home, a need for more tailored information.
  • What can I do with my premature infant?
  • Participants sought information on baby positioning, especially with the preferred head position.
  • In addition to the shown topics: reflux and gavage feeding tips. Information about constipation.
About which topics do parents seek additional information?
  • Information about constipation, intestinal cramps, reflux, or medication.
  • Feeding, sleep patterns, sensory overexcitability
  • Inconsolable crying.
  • Psychological support for anxiety.
  • Premature clothes.
  • Information about long-term effects of oxygen.
What support do parents need?
  • Finding psychological support for themselves.
  • Need for a readily accessible professional after discharge.
  • Information on available services, particularly for single moms.
Where do they look for information?
  • Communities including Facebook groups and consumer association websites.
  • Parents shared their favorite websites, but they are often not specific to premature infants.
  • Forums such as 24baby are used.
  • Other parents with experiences serve as key sources of information.
Presentation of information
  • Adaptability to literacy levels. Videos and animations in multiple languages for less literate parents.
  • Varied content preferences. Different preferences: some participants liked text-based information, others favored videos.
  • Preference for animation (visually oriented).
  • Practical demonstrations with videos for making the crib.
  • Interactive features such as a chat box for quick response.
  • Peer experiences. Some valued real-life success stories; others were not interested in fellow sufferers.
Navigation
  • A structured topic design, with an FAQ chapter.
  • Search button to improve content discoverability.
  • Clicking through for more detailed information.
Information cards
  • Reassuring and supportive tone is essential.
  • Normalizing and comforting tone of voice is important.
  • Concise messages can raise questions. The messages should not oversimplify.
  • Informative, necessity to be able to search further for more extensive information.
Text validation
  • Be aware of the tone of voice, clarity of the information.
  • Comforting information is important. An informal tone of voice is nice, it relates to parents and feels personal.
  • Easy-to-read text, supplemented by practical examples on what to do.
  • Information can be more casually read, like you talk to a trusted friend or relative.
  • It needs to be valid information.
Depth of information
  • Layered information structure. Parents need to be able to explore in-depth details or stay at a simpler level.
  • Different headings can help to make content more accessible.
  • Information should clearly explain the differences in care for premature infants.
Scope of information
  • Coverage for the first year, aligning with the TOP program.
  • Guidance on more complex cases should refer parents to the health care professionals. Potential integration with a hospital app could enhance usability.
  • Developmental milestones per week. Receiving updates about what your baby is doing in relation to other infants (corrected for prematurity)

aTOP interventionist.

bParents of preterm infants.

cParents of preterm infants with limited health literacy.

From Prototype to a Functional e-TOP App

To create content for the app, experts were recruited through the researchers’ network. Health care experts were selected based on their expertise regarding prematurity or related topics, such as feeding, sleep, or motor development. To address user needs, the following 4 steps were taken. First, the topics parents indicated in our prior research [13] were complemented with the findings of the focus group meetings. Experts drafted comprehensive information on these topics. Two members of the research team (MF and EM) finalized the content in 3 rounds of review and refinement. Second, the Dutch Expertise Center for reducing health inequities provided support to improve the readability, whereafter the text was simplified by the first 2 authors (MF and EM) to that of the reading ability to understand everyday health information (B1/B2). Key messages for each topic were formulated at level A2 for parents who prefer to read short sentences with familiar words[28]. In addition to text, a read-out-loud function was added. Third, the first 2 authors (MF and EM) included images and picture stories. Fourth, video content, either developed by health care experts from different fields or retrieved from the EOP library, was added. Videos consisted of interviews, demonstrations of parent-infant interaction, and explanations of infant behavioral signs. To clarify the content, videos had voice-overs and subtitles.

Simultaneously with creating content, we collaborated with a design company (ZIGT) to build a functional prototype of a web app accessible on smartphones, tablets, and desktop devices. The research team organized and posted all content leading to the final functional e-TOP app (see Figure 3).

Figure 3. e-TOP app.

Phase 2 Methods

Participants

For the evaluation of the e-TOP app, the sample was composed of parents with a very preterm born infant (below 320/7 gestation) and moderate preterm born infants (320/7-346/7 weeks of gestation). Parents of very preterm infants received the TOP program (routine care, reimbursed by the basic package of insurers), and parents of moderate preterm infants received a shortened TOP program in a study context [29]. In addition to the home visits by the TOP interventionists, all participants received access to the e-TOP app.

Procedures

TOP interventionists were recruited using an invitation email distributed by the EOP center. Fifteen TOP interventionists responded to be available for participation in the study. They received a 1-day training in the intervention and research protocol including the informed consent procedure. The enrollment took place between November 2022 and March 2023. TOP interventionists introduced the study to eligible parents within their caseload. The requirement for parents to take part in the study was to be able to read and speak Dutch. When interested, parents received an information letter from the study center. If they agreed to participate, an informed consent visit was scheduled. Upon signing the informed consent, both parents received a unique access code and an animation video explaining how to install the e-TOP app. In addition, at the start (T0), parents were invited via email to complete a questionnaire on sociodemographic information including age, educational level, employment status, and family structure. Parents had continuous access to the app for the duration of 6 months. During this period, Matomo analytics was used to collect user data and thinking aloud tests were performed. At the end of this period (T1), parents received a follow-up invitation to complete questionnaires assessing their experiences, and 10 participants were invited for an interview.

Usability

For a comprehensive evaluation of the usability of the app, we used qualitative and quantitative measurements. User data provided insights into objective usage engagement, while qualitative data highlighted usability needs and identified issues for further improvement [30].

Usability Testing: Thinking Aloud

Thinking aloud tests were used to gain insights into the ease of use and navigation experiences with the app [31,32]. While navigating the app, parents were encouraged to express their thoughts, frustrations, questions, and feedback in real time. The tests were conducted by 2 students from the faculty of health of the University of Applied Sciences Amsterdam, supervised by author VV.

For the sample technique, a stepwise approach was used to ensure that the final participant list for the thinking aloud session included a varied sample across different levels of app engagement. The selection process involved the following steps: (1) all families were initially screened for e-TOP app installation. (2) For each participant, the total app usage time (in seconds) was calculated. (3) Quartiles of app usage time were determined, and participants were divided into 4 usage groups. (4) Each group was randomly ordered using a randomization tool. (5) Three participants per group (n=12) were approached by email for participation.

Participants were asked to complete 3 tasks: (1) determine when your infant can pick up a small object, (2) identify what actions you can take as a parent when your child is anxious about touch, and (3) calculate your infant’s corrected age. After completing each task, participants rated the difficulty using the Single Ease Question [33], which scores task performance on a scale from 1 (very difficult) to 7 (very easy). Afterward, parents were asked questions regarding the content (eg, “Are there any topics you would like to add?”), usability (eg, “How was your experience with the e-TOP app?”), and comprehension (eg, “Did you understand the answers you found when searching for information?”). These sessions were screen-recorded, and audio transcripts were generated.

User Data

User data were monitored and analyzed with the web analytics tool Matomo [34]. Users were defined as those who accessed the e-TOP app more than once. The main metrics of interest were installs, visits, visit duration and change of usage over time, and visited topics.

To allow participants to provide immediate feedback on the app, the e-TOP app included a prompt at the end of each page asking, “Was this information helpful?” with 2 response options: yes or no. After selecting an option, a comment box appeared, allowing users to provide additional feedback.

Usability Questionnaires

The Dutch version of the system usability scale (SUS) was used [35]. The SUS is an easy-to-use, valid, and reliable questionnaire for usability assessment of mHealth apps. The scale consists of 10 statements, which can be completed within 10 minutes. All items are scored on a 5-point Likert scale from “strongly disagree” to “strongly agree.” Scores per item are converted to an overall SUS score, which ranges from 0 to 100, with higher scores indicating better usability. Overall scores from 0 to 50 indicate “not acceptable,” scores from 51 to 67 indicate a marginal level of usability, and scores from 68 to 100 indicate “acceptable to excellent” levels of usability [36].

To measure user experience and appropriateness of the content, a customized 10-item questionnaire was used. Seven statements were scored on a 5-point Likert scale from “strongly disagree” to “strongly agree.” Two statement examples are: “The e-TOP app helps me to find information about premature birth” and “The pictures and videos help me to understand the information.” Overall satisfaction with the app was scored on a numeric scale from 1 to 10 with 1 indicating very dissatisfied to 10 indicating extremely satisfied. Two open-response questions allowed participants to provide unrestricted and more subjective answers: “What information did you miss when using the e-TOP app?” and “Do you have suggestions to improve the e-TOP app?”

Interviews

The interview guide was developed in collaboration with the research team. Questions were formulated to address predefined aspects considered critical for evaluating the app’s usability. These aspects were derived from the study objectives and focused on identifying strengths, limitations, and opportunities for improvement.

To gain a more in-depth understanding of the experiences with the e-TOP app, a subgroup of participants (n=10) was invited for a semistructured interview with the first author (MF) to discuss their experiences with the e-TOP app in more detail. Purposeful sampling, including educational level, gender, and e-TOP app usage, was used to obtain a heterogeneous sample.

Interviews were scheduled at a time convenient for the parents (by phone or video call). The interview guide was set up to elicit direct, actionable feedback on app usage (Multimedia Appendix 2).

Data Analysis

Questionnaire data were collected using the Castor database (Electronic Data Capture, Ciwit BV, 2021). Descriptive statistics were used to describe the participants and the user data (installs, visits, visit duration and change of usage over time, and visited topics). Questionnaire data were checked for normal distributions using a graphical summary of data and descriptive statistics. The median score with IQR was used in case of skewed distributions.

For the qualitative data collected through thinking aloud sessions and interviews, a pragmatic thematic analysis approach was used [37]. For the thinking aloud analysis, initial open coding was analyzed and discussed within the project group and subsequently categorized into themes. Subthemes were then developed under each main theme to capture more specific insights. In addition, user comments and open questions from the questionnaires were evaluated and categorized by the first author (MF). Ultimately, the qualitative analysis resulted in 4 overarching themes: user experience, content, navigation, and functionality.

Phase 2 Results

Participants

A total of 69 families with a preterm infant were assessed for eligibility by 15 TOP interventionists. Of these, 58 eligible families, consisting of 58 fathers and 58 mothers, agreed to participate, signed informed consent, and received the access code for the e-TOP app. Five families (10 participants) withdrew from the study after signing informed consent and receiving the first questionnaires. At T0, 61 participants completed the demographic questionnaire (see Figure 4 for flow diagram). For participant characteristics, see Table 2.

Figure 4. Flow of participants. SUS: system usability scale.
Table 2. Participant characteristics.
Participant characteristicsValues, n (%)
Parent
Mother42 (69)
Father19 (31)
Family status
Firstborn child37 (61)
Family status of 2 parents61 (100)
Language
Dutch language for health care information60 (98)
Educationa
Lower2 (3)
Intermediate22 (36)
High37 (61)
Employment
Full-time35 (57)
Part-time25 (41)
No employment1 (2%)

aHighest educational level completed: lower level education refers to primary school, vocational education, lower of middle general secondary education. Intermediate: refers to higher secondary general education, preuniversity education. High: higher vocational or university.

Usability Results
Thinking Aloud

Ultimately 5 participants (3 fathers and 2 mothers) were willing to schedule an appointment to perform the Thinking aloud tasks. The ease of use was rated as follows: Task 1 had a mean score of 6.4 (SD 0.55), Task 2 a mean of 6.0 (SD 0.89), and Task 3 a mean of 6.0 (SD 1.22) on a scale from 1 to 7, indicating that all 3 tasks were easy to perform. Participants appreciated the app, the tone of voice, and the understandability of the information. Participants provided explicit and actionable recommendations regarding content and functionality. For example, regarding content, participants expressed a desire for more information on health issues and medication. They also expressed interest in instructional videos on techniques, such as holding and carrying their baby. Regarding functionality, a home button, a scroll arrow, and a search function were missed.

User Data

In total, 69 participants (47 mothers and 22 fathers) used the personal access code to log in. Due to ad blockers, tracking data were available for 63 participants. The median cumulative e-TOP usage per participant during 6 months (26 weeks) was 39 minutes (IQR 8.8‐53.0). The median number of actions was 64.0 (IQR 33.5‐88.0). Most actions and the largest time spent on the app were registered during the 2 weeks after receiving the access code. See Figures 5 and 6 for further details. Multiple participants visited all 10 topics during the first 2 weeks. The total amount of visits per topic declined after this period with another peak in visits between weeks 11 and 17. The most visited topics in the app were nutrition, sleep, and correcting age for prematurity (Table 3).

Figure 5. Time spent per week.
Figure 6. Actions per week.
Table 3. Visited topics on the e-TOP app.
TopicTotal number of visitsPeak visits (week)
Nutrition8117
Sleep796
Correcting age543
Understanding your baby494
Long-term consequences4414
Motor development4211
Returning to work334
Parenting254
Infant health234
Follow-up219

Usability Questionnaires

A total of 55 participants completed the SUS, yielding a median total SUS score of 75.0 (IQR 67.5‐80.0), indicating generally good usability. The app was rated positively for its ease of use, functional integration, and user confidence.

Participants gave the app an overall satisfaction of 7.0 (IQR 7.0‐8.0), on a scale of 1‐10. Content satisfaction was rated with a mean score of 3.7 (SD 0.89) and the app’s appearance was considered attractive, with a mean score of 3.6 (SD 0.83) on a scale from 1 to 5. Participants found the pictures and videos particularly helpful (mean 4.0, SD 0.56) and expressed that the app was helpful in addition to face-to-face care from the TOP interventionist (mean 3.5, SD 0.99).

Qualitative Data

Data collected through thinking aloud sessions, interviews, and written comments from the app’s comment box were categorized in 4 themes: user experience, content, navigation, and functionality (see Table 4).

Table 4. Integrated results matrix for quantitative and qualitative data.
ThemesQuantitative findings (questionnaires and thinking aloud)Qualitative findings (interviews and comment box)Meta-inference
User experience
  • Overall satisfaction: median 7.0/10 (IQR 7-8)
  • Visual attractiveness: mean 3.6/5
Users appreciated the depth and specificity of information and the app’s tone. However, they missed tailored responses and desired more interactive features.Parents appreciate the user experience, yet desire more individualized and interactive features.
Content
  • Content rating: mean 3.7/5 (SD 0.89)
  • Visual helpfulness: mean 4.0/10 (SD 0.56)
Content was viewed as highly relevant and unique to prematurity. Suggestions included more on motor development, practical care, and instructional videos.Content meets core needs, but parents want more concrete, in-depth, and visually supported information tailored to daily care.
Navigation
  • Median total SUSa score of 75.0 (IQR 67.5‐80.0)
Parents found the app easy to use and appreciated its tone; they missed a search button and clearer navigationUsability is high, but navigation challenges (especially in locating specific info) call for interface redesign and better structure.
Functionality
  • Task ease thinking aloud: mean 6.1/7 (SD 0.89)
  • Usage: median 39 minutes (IQR 8.8-53) over 6 months
Parents appreciated the functions in the app, but had some suggestions for improvements.Functionality is strong but could be elevated by improved orientation, content sharing, and age-specific organization (eg, timeline).

aSUS: system usability scale.

User Experience

Participants appreciated the specificity and depth of information provided yet expressed concerns about retrieving tailored answers to specific situations. As a participant in the interview stated, “The only thing I couldn’t find was whether it’s possible to swim with a premature baby and at what age that is safe”. The app facilitated users’ understanding of infant behavioral cues and sleep patterns. However, parents also preferred direct interaction with a TOP interventionist, indicating a desire for more personalized and human support. For example, a mother stated in the interview that “The TOP interventionist explained it to me personally, so I found that easier and more comfortable.” Suggestions were given to incorporate interactive features, such as chat support or a parent community, indicating a preference for increased interactivity and social connection within the app.

Content

The content was generally deemed useful and relevant to the specific needs of parents with premature infants. As 1 participant noted in the comment box, “It gives information you cannot find on the internet, all content is related to premature birth.” Suggestions for improvement emerged from both the interviews and the comment box. Requests for additional practical tips, along with more in-depth information on motor development and health-related concerns, highlighted opportunities to expand the scope of information. As a participant stated during the interview: “More information about motor development, like crawling, creeping, and sitting, would be helpful. Another parent expressed during the interview, "I would like there to be more concrete information, such as a chart with a baby weighing this much should drink this much.” Videos were highlighted as particularly effective for conveying complex concepts, such as sleep patterns and infant signals.

Navigation

While users found the landing pages clear and well-organized, they reported difficulties with lengthy chapters and locating specific information. As a participant during the interview reported: “Searching for specific information is more difficult if you don’t know under which chapter to search.” Another participant reported in the comment box: “The sub-chapters contain too much information and not so easy to find answers to your questions.” The need for enhanced navigation design, such as the inclusion of a search function, a home button, and scroll arrows, was suggested and could enhance information retrieval. A participant explained during the thinking aloud session: "If there was an arrow next to the app indicating that there’s more room to scroll down, you might have looked further.”

Functionality

A timeline-based structure reflecting the developmental stages of premature infants was suggested as a potential solution to improve content organization. One participant stated in the comment box: “It would also be helpful if there was a timeline showing what a child should do at each (corrected) age.” The ability to share information easily with family members, particularly regarding hygiene practices for visits, emerged as a key functional improvement.


Overview

This study aimed to develop and evaluate a digital information source (e-TOP) to address the specific information needs of parents caring for preterm infants during the first year postdischarge.

Development of the App

To develop the e-TOP app, a user-centered design approach was used. The feedback from the 3 focus groups with TOP interventionists and parents of premature infants, including parents with LHL, directly influenced the app’s content and design. The cocreation and iterative process between experts, the design company, and the research team was highly successful in shaping the app’s functionality and refining its design and content. This collaborative approach ensured that the final prototype of the e-TOP app was both user-friendly and rich in relevant, evidence-based information, tailored to meet the diverse needs of its users. This methodology aligns with established practices in mHealth app development, emphasizing the importance of participatory design including health professionals to meet user needs [21-24].

Parents with LHL were actively engaged during the focus group meetings in the developmental phase of the e-TOP app. They expressed a need for clear, actionable information, preferably content in multiple formats, with an accessible tone and simplified messages. The app therefore incorporated features, such as simplified language (Level 1), multimedia content, and “read-out-loud” functionality to accommodate parents with LHL, with videos and text at different reading levels. This preference is supported by research indicating that visual-based interventions, particularly videos, are effective in improving health literacy and comprehension of health-related information [38].

Usage and Usability

Participants engaged with the app for a median total of 39 minutes (IQR 8.8-53.0) during the 6-month study period. The number of recorded actions per participant had a median of 64.0 (IQR 33.5-88.0), but usage varied significantly between participants. Differences in purpose, target groups, and the lack of user engagement metrics make comparison to other parenting apps challenging, as also described in Li’s systematic review of commercial mHealth parenting apps [39]. The e-TOP app is designed as an additional information source; if we compare the results to educational health apps, longer but less frequent sessions are found [40]. This found gap calls for established standardized benchmarks for digital engagement in neonatal mHealth interventions.

While the app was perceived as useful for finding information on prematurity-specific topics, user data revealed a decline in engagement after the initial 2 weeks. These differences in intent to use versus lower nonusage attrition are found in other web-based parenting interventions [15]. The e-TOP engagement peaked at week 11 and week 16. Parents revisited the app at key developmental milestones, such as rolling over, reaching for toys, and the transition to solid foods. Integrating features such as search functionality, interactive (push notifications and chat function), and timeline-based navigation could enhance the e-TOP app’s long-term utility [41,42].

SUS distribution with a mean SUS score of 68 is considered a suitable usability for Digital Health apps [43]. The e-TOP app demonstrated acceptable-to-good usability with a SUS score of 75.0 (IQR 67.5-80.0). By complementing the SUS with a customized satisfaction survey and data collected through thinking-aloud sessions, interviews, and written comments from the app’s comment box, this study followed best practices of combining quantitative ratings with rich qualitative findings for the usability testing [30]. Participants provided actionable feedback across key areas. By integrating these improvements, the e-TOP app can enhance usability, ensuring it remains a valuable long-term resource for parents of preterm infants.

Limitations and Opportunities for Future Research

The total study period of 2 years and restricted finances necessitated pragmatic approaches to the evaluation of the usability. We used several qualitative and quantitative methods, but we had to make compromises impacting the study methodology, such as the reliance on a single author for qualitative data analysis which may introduce bias. Integrating multiple researchers into the thematic analysis process not only enriches the analysis through diverse perspectives but also enhances the credibility and trustworthiness of the research findings [44].

Although parents with LHL contributed to the development of the app, the study faced challenges in recruiting parents with LHL for the evaluation phase. Their limited representation in the evaluation phase underscores a broader challenge in engaging these parents in research [45]. Barriers such as complex consent processes and digital questionnaires have hindered participation in our study. Allowing parents to first establish a connection with the researchers or interventionists and using postponed consent is expected to promote inclusivity. Also, verbal consent or simplified forms could foster inclusivity. Although improved comprehension and trust may have positive effects on study enrollment, we emphasize the active involvement of the target group in the early stages of research projects when developing the study design, procedures, and the selection of measurements [46].

This study had a high attrition of the fathers; only 32% of the fathers used the app and 25% completed the usability questionnaires. In general, retaining participants in studies using mHealth apps has been described as challenging [47]. The finding of higher attrition among fathers aligns with the results of the study on the NICU2HOME app (Garfield [18]), in which high usability scores were similarly observed among mothers, but lower engagement among fathers was noted [10]. Xie [48] also found in her systematic review of digital interventions for fathers of infants from conception to 12 months postpartum that fathers have expressed the need and desire for access to relevant, accurate, and up-to-date information on infant care and challenges associated with new parenthood and called for interventions that target father-identified priorities [49]. These studies and our findings highlight the importance of gaining a deeper understanding of perceived barriers to engaging fathers.

The usability evaluation confirms that the app was responsive to the needs of parents with a premature infant. Given the evidence supporting user-centered iterative development, we strongly advocate for proceeding with the implementation of the e-TOP app, ensuring it remains a valuable and available resource while integrating continuous user feedback by using real-time analytics, for example, engagement metrics [50,51]. Future research should focus on assessing the effectiveness by evaluating how app usage translates into improved parental confidence and decision-making in caregiving with the goal of reducing dependency on health care professionals and fostering parental autonomy.

Conclusion

The e-TOP app has demonstrated strong potential to complement information transfer to the face-to-face care after discharge and to bridge the information gap for parents when they leave the hospital. While usability findings are promising, continued development, customization, and integration of the app with health care systems are necessary to maximize its long-term impact. By addressing user engagement, content expansion, accessibility, and health care integration, future versions of e-TOP can become an essential tool for empowering parents of preterm infants and improving neonatal postdischarge care.

Acknowledgments

We would like to thank all involved parents and TOP interventionists for providing us with valuable feedback and photo and video material. We thank the experts for their uncompensated commitment to provide the e-TOP app content. We especially would like to thank our facilitator for the focus group meetings, Marion Matthijssen (MM), and the Dutch parent association Care4neo. This research was co-financed by Regieorgaan SIA, part of the Netherlands Organization for Scientific Research (NWO).

Conflicts of Interest

None declared.

Multimedia Appendix 1

Focus group outline and key questions.

DOCX File, 28 KB

Multimedia Appendix 2

Interview guide for evaluating the e-TOP app.

DOCX File, 27 KB

  1. Preterm birth: kerncijfers nederlandse geboortezorg 2021 [Report in Dutch]. Perined; 2025. URL: https://assets.perined.nl/docs/a0978620-5d23-492f-a7f2-69fe7beff1a6.pdf [Accessed 2025-06-29]
  2. Preterm birth: fact sheet. World Health Organization. 2022. URL: https://www.who.int/ news-room/fact-sheets/detail/preterm-birth [Accessed 2025-06-29]
  3. Information about public health and care. VZinfo. 2025. URL: https://www.vzinfo.nl [Accessed 2025-06-29]
  4. Allotey J, Zamora J, Cheong-See F, et al. Cognitive, motor, behavioural and academic performances of children born preterm: a meta-analysis and systematic review involving 64 061 children. BJOG. Jan 2018;125(1):16-25. [CrossRef] [Medline]
  5. Raap E, Weille KL, Flierman M, Jeukens-Visser M. “The Attitude Is Essential” The experience of two mothers with differing educational backgrounds receiving post-discharge intervention after very preterm delivery. Stud Clin Soc Work. Apr 2, 2024;94(2):109-135. [CrossRef]
  6. Vriend E, Leemhuis A, Flierman M, van Schie P, Nollet F, Jeukens-Visser M. Mental health monitoring in parents after very preterm birth. Acta Paediatr. Nov 2021;110(11):2984-2993. [CrossRef] [Medline]
  7. George K, Rowe J, Barnes M, Kearney L, Feng GC. The Parenting Premmies Support Program: designing and developing a mobile health intervention for mothers of preterm infants. Cogent Soc Sci. Jan 1, 2021;7(1). [CrossRef]
  8. Green J, Fowler C, Petty J, Whiting L. The transition home of extremely premature babies: an integrative review. J Neonatal Nurs. Feb 2021;27(1):26-32. [CrossRef]
  9. Davis-Strauss SL, Johnson E, Lubbe W. Information and support needs of parents with premature infants: an integrative review. J Early Interv. Sep 2021;43(3):199-220. [CrossRef]
  10. Breivold K, Hjaelmhult E, Sjöström-Strand A, Hallström IK. Mothers’ experiences after coming home from the hospital with a moderately to late preterm infant - a qualitative study. Scand J Caring Sci. Sep 2019;33(3):632-640. [CrossRef] [Medline]
  11. Setiawan J, Mannix T, Sweet L. Understanding the effects of neonatal early discharge on parents: a literature review. J Perinat Neonatal Nurs. 2019;33(2):170-188. [CrossRef] [Medline]
  12. Alderdice F, Gargan P, McCall E, Franck L. Online information for parents caring for their premature baby at home: a focus group study and systematic web search. Health Expect. Aug 2018;21(4):741-751. [CrossRef] [Medline]
  13. Flierman M, Bossen D, de Boer R, et al. Parents’ information needs during the first year at home with their very premature born child; a qualitative study. PEC Innov. Dec 2024;4:100270. [CrossRef] [Medline]
  14. Taylor K, Havinga J, Moore B, Tumin D, Bear K. Social media as a source of medical information for parents of premature infants: a content analysis of prematurity-related Facebook groups. Am J Perinatol. Nov 2023;40(15):1629-1637. [CrossRef] [Medline]
  15. Virani A, Duffett-Leger L, Letourneau N. Parents’ use of mobile applications in the first year of parenthood: a narrative review of the literature. Health Technol. 2021;5:14-14. [CrossRef]
  16. Richardson A, Allen JA, Xiao H, Vallone D. Effects of race/ethnicity and socioeconomic status on health information-seeking, confidence, and trust. J Health Care Poor Underserved. Nov 2012;23(4):1477-1493. [CrossRef] [Medline]
  17. Richardson B, Dol J, Rutledge K, et al. Evaluation of mobile apps targeted to parents of infants in the neonatal intensive care unit: systematic app review. JMIR mHealth uHealth. Apr 15, 2019;7(4):e11620. [CrossRef] [Medline]
  18. Garfield CF, Kerrigan E, Christie R, Jackson KL, Lee YS. A mobile health intervention to support parenting self-efficacy in the neonatal intensive care unit from admission to home. J Pediatr. May 2022;244:92-100. [CrossRef] [Medline]
  19. Jeukens-Visser M, Flierman M, Giezeman R, Vijn V, Engelbert R, Bossen D. A systematic review evaluating mobile education and support applications for parents of preterm infants post-discharge. Presented at: Conference proceedings of the 2023 Congress of the World Association for Infant Mental Health - Scientific abstracts 200450; Jul 16-19, 2023; Dublin, Ireland.
  20. Noorbergen TJ, Adam MTP, Teubner T, Collins CE. Using co-design in mobile health system development: a qualitative study with experts in co-design and mobile health system development. JMIR mHealth uHealth. Nov 10, 2021;9(11):e27896. [CrossRef] [Medline]
  21. Roberts JP, Fisher TR, Trowbridge MJ, Bent C. A design thinking framework for healthcare management and innovation. Healthcare (Basel). Mar 2016;4(1):11-14. [CrossRef]
  22. Garne Holm K, Brødsgaard A, Zachariassen G, Smith AC, Clemensen J. Participatory design methods for the development of a clinical telehealth service for neonatal homecare. SAGE Open Med. 2017;5:2050312117731252. [CrossRef] [Medline]
  23. Schweitzer R, Schlögl S, Schweitzer M. Technology-supported behavior change-applying design thinking to mHealth application development. Eur J Investig Health Psychol Educ. Mar 7, 2024;14(3):584-608. [CrossRef] [Medline]
  24. Molina-Recio G, Molina-Luque R, Jiménez-García AM, Ventura-Puertos PE, Hernández-Reyes A, Romero-Saldaña M. Proposal for the user-centered design approach for health apps based on successful experiences: integrative review. JMIR mHealth uHealth. Apr 22, 2020;8(4):e14376. [CrossRef] [Medline]
  25. Jeukens-Visser M, Koldewijn K, van Wassenaer-Leemhuis AG, Flierman M, Nollet F, Wolf MJ. Development and nationwide implementation of a postdischarge responsive parenting intervention program for very preterm born children: the TOP program. Infant Ment Health J. May 2021;42(3):423-437. [CrossRef] [Medline]
  26. Palinkas LA, Horwitz SM, Green CA, Wisdom JP, Duan N, Hoagwood K. Purposeful sampling for qualitative data collection and analysis in mixed method implementation research. Adm Policy Ment Health. Sep 2015;42(5):533-544. [CrossRef] [Medline]
  27. Health and quality of care for everyone. Pharos. 2020. URL: https:/​/www.​pharos.nl/​wp-content/​uploads/​2018/​10/​Checklist-HERKENNEN-laaggeletterdheid-maart-2022.​pdf [Accessed 2025-10-01]
  28. The CEFR levels. Council of Europe. URL: https://www.coe.int/en/web/common-european-framework-reference-languages/level-descriptions [Accessed 2024-07-23]
  29. Flierman M, Möller EL, Engelbert RHH, van Kaam AH, Bossen D, Jeukens-Visser M. Feasibility of a Dutch post-discharge parenting intervention (TOP program) for moderate preterm born infants. Early Hum Dev. Nov 2024;198:106124. [CrossRef] [Medline]
  30. Inal Y, Wake JD, Guribye F, Nordgreen T. Usability evaluations of mobile mental health technologies: systematic review. J Med Internet Res. Jan 6, 2020;22(1):e15337. [CrossRef] [Medline]
  31. Güss CD. What is going through your mind? Thinking aloud as a method in cross-cultural psychology. Front Psychol. 2018;9:1292. [CrossRef] [Medline]
  32. Hasso Plattner Institute of Design at Stanford University Stanford School. Stanford School. URL: https://dschool.stanford.edu/ [Accessed 2024-12-03]
  33. Hodrien A, Fernando TA. Review of poststudy and posttask subjective questionnaires to guide assessment of system usability. J Usability Stud. 2021;16(3):203-232. [CrossRef]
  34. Web & app analytics for teams who demand accuracy. Matomo. URL: https://matomo.org/ [Accessed 2022-10-03]
  35. Ensink CJ, Keijsers NLW, Groen BE. Translation and validation of the System Usability Scale to a Dutch version: D-SUS. Disabil Rehabil. Jan 2024;46(2):395-400. [CrossRef] [Medline]
  36. Bangor A, Kortum PT, Miller JT. An empirical evaluation of the System Usability Scale. Int J Hum Comput Interact. Jul 29, 2008;24(6):574-594. [CrossRef]
  37. Ramanadhan S, Revette AC, Lee RM, Aveling EL. Pragmatic approaches to analyzing qualitative data for implementation science: an introduction. Implement Sci Commun. Jun 29, 2021;2(1):70. [CrossRef] [Medline]
  38. Galmarini E, Marciano L, Schulz PJ. The effectiveness of visual-based interventions on health literacy in health care: a systematic review and meta-analysis. BMC Health Serv Res. Jun 11, 2024;24(1):718. [CrossRef] [Medline]
  39. Li K, Magnuson KI, Beuley G, Davis L, Ryan-Pettes SR. Features, design, and adherence to evidence-based behavioral parenting principles in commercial mHealth parenting apps: systematic review. JMIR Pediatr Parent. Jun 1, 2023;6:e43626. [CrossRef] [Medline]
  40. Baumel A, Muench F, Edan S, Kane JM. Objective user engagement with mental Health apps: systematic search and panel-based usage analysis. J Med Internet Res. Sep 25, 2019;21(9):e14567. [CrossRef] [Medline]
  41. Krishnamurti T, Moon R, Richichi R, Berger R. Integrating infant safe sleep and breastfeeding education into an app in a novel approach to reaching high-risk populations: prospective observational study. JMIR Pediatr Parent. Jan 14, 2025;8:e65247. [CrossRef] [Medline]
  42. Amagai S, Pila S, Kaat AJ, Nowinski CJ, Gershon RC. Challenges in participant engagement and retention using mobile health apps: literature review. J Med Internet Res. Apr 26, 2022;24(4):e35120. [CrossRef] [Medline]
  43. Hyzy M, Bond R, Mulvenna M, et al. System Usability Scale benchmarking for digital health apps: meta-analysis. JMIR Mhealth Uhealth. Aug 18, 2022;10(8):e37290. [CrossRef] [Medline]
  44. Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. Jan 2006;3(2):77-101. [CrossRef]
  45. Bader M, Zheng L, Rao D, et al. Towards a more patient-centered clinical trial process: a systematic review of interventions incorporating health literacy best practices. Contemp Clin Trials. May 2022;116:106733. [CrossRef] [Medline]
  46. Smits DW, van Meeteren K, Klem M, Alsem M, Ketelaar M. Designing a tool to support patient and public involvement in research projects: the Involvement Matrix. Res Involv Engagem. 2020;6:30. [CrossRef] [Medline]
  47. Eysenbach G. The law of attrition. J Med Internet Res. Mar 31, 2005;7(1):e11. [CrossRef] [Medline]
  48. Xie EB, Jung JW, Kaur J, Benzies KM, Tomfohr-Madsen L, Keys E. Digital parenting interventions for fathers of infants from conception to the age of 12 months: systematic review of mixed methods studies. J Med Internet Res. Jul 26, 2023;25:e43219. [CrossRef] [Medline]
  49. Dick S, O’Connor Y, Thompson MJ, et al. Considerations for improved mobile health evaluation: retrospective qualitative investigation. JMIR mHealth uHealth. Jan 22, 2020;8(1):e12424. [CrossRef] [Medline]
  50. Lupton D, Pedersen S, Thomas GM. Parenting and digital media: from the early web to contemporary digital society. Sociol Compass. Aug 2016;10(8):730-743. [CrossRef]
  51. van Gemert-Pijnen J, Nijland N, van Limburg M, et al. A holistic framework to improve the uptake and impact of eHealth technologies. J Med Internet Res. Dec 5, 2011;13(4):e111. [CrossRef] [Medline]


EOP: Dutch Expertise Center for Premature Infants
LHL: limited health literacy
mHealth: mobile health
NICU: neonatal intensive care unit
SUS: system usability scale


Edited by Sherif Badawy; submitted 07.Apr.2025; peer-reviewed by Emos Tella, Lina Weinert; final revised version received 09.Aug.2025; accepted 10.Sep.2025; published 22.Oct.2025.

Copyright

© Monique Flierman, Eline L Möller, Vincent L Vijn, Raoul H Engelbert, Martine Jeukens-Visser, Anton H van Kaam, Daniel Bossen. Originally published in JMIR Pediatrics and Parenting (https://pediatrics.jmir.org), 22.Oct.2025.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Pediatrics and Parenting, is properly cited. The complete bibliographic information, a link to the original publication on https://pediatrics.jmir.org, as well as this copyright and license information must be included.