A cross-sectional study of German websites and mobile apps with breastfeeding content was conducted in July 2023. The App Store for iOS and Google Play Store for Android were searched for mobile apps. Bing.de and Google.de were searched for websites. We followed the STROBE (Strengthening the Reporting of Observational Studies in Epidemiology) reporting guideline, omitting items 13 and 15, which were not relevant for this study, as no study participants were recruited and no outcomes were measured [23].

The German App Store for iOS and Google Play Store for Android were screened with smartphones from July 19 to 20, 2023, by 2 reviewers (MZ, JP) using 9 search terms (see Table S1 in Multimedia Appendix 1).

Inclusion criteria for mobile apps were that mobile apps were offered free of charge, were in the German language, and contained information on breastfeeding. Exclusion criteria were restricted access, formatting as an electronic book (eg, e-books, news, magazines, podcasts, blogs, or word documents), games or gaming apps, and mobile apps whose main function was to monitor time or frequency of infant care tasks without providing educational information on breastfeeding.

Two reviewers (MZ, JP) independently screened all mobile apps for eligibility. For each search term, the first 10 mobile apps were selected, and features such as rating, costs, number of downloads, and characteristics of the mobile app were noted. A list of mobile apps was selected, duplicates were removed, lists for Android and iOS were jointly merged, and duplicates were removed again.

Three validated tools were used to evaluate mobile apps for quality, suitability, and readability. Each mobile app was reviewed independently by 2 reviewers (MZ, JP) and then evaluated using an evaluation form (see Multimedia Appendix 2).

To appraise the quality of mobile apps, the user version of the Mobile Application Rating Scale (uMARS) questionnaire in the original English language was used [24]. The uMARS is an established tool to assess the quality of mHealth apps rated by end users. It consists of 16 items, including 4 objective quality subscales (engagement, functionality, esthetics, and information), 1 subjective quality subscale, and a further subscale to measure the users’ perceived impact of the evaluated mobile app [25]. All items were rated on a 5-point Likert scale (1=inadequate to 5=excellent) [24]. See Multimedia Appendix 2 for further information on the specific questions asked.

Suitability of information was assessed with the Suitability Assessment of Materials (SAM) questionnaire, which was validated by students and health care providers from several cultures [26,27]. It rates health-related materials in 6 areas (content, literacy demand, graphics, layout and type, learning stimulation and motivation, and cultural appropriateness). There are 22 questions on a 3-point scale (0=not suitable, 1=adequate, 2=superior). For missing items, the instructions of the SAM were followed. For questions answered as not applicable, 2 points of the possible 44 points were subtracted. Suitability was rated based on scoring (0%‐39% suitable material, 40%‐69% adequate material, and ≥70% superior material). The suitability for a particular population was ranked from 0 (not recommended) to 10 (recommended without reservations) based on socioeconomic and cultural backgrounds of the population [27].

Readability was rated with the Flesch Index online tool, which provides a score from 0 to 100 (0‐30: hard to read, 30‐50: difficult to read, 50‐60: challenging, 60‐70 normal, 70‐80: easy to read, 80‐90: easily to read, 90‐100 very easy to read) [28]. As the Flesch Reading Ease test was developed for the English language, it needed to be adapted for the German language [29]. For the evaluation, 3 randomly chosen paragraphs were copied into the online tool and the average of the Flesch Reading Ease test results was used for evaluation.

Websites were identified using Google.de and Bing.de between July 13 and 17, 2023 [30,31]. Google Trends and Ubersuggest were used to identify appropriate search terms [32,33]. Google Trends is a tool to get information about how often users of Google enter explicit search terms. Ubersuggest is a tool to analyze keywords, find ideas for new keywords, and give insights into the ranking of the keywords.

A layperson was asked to suggest how to search for breastfeeding information and problems. These terms were used to find the most popular search terms for this topic on Ubersuggest and Google Trends for Germany. Twelve German search terms were used (see Table S2 in Multimedia Appendix 1). Before screening each search term individually, the search history and cookies were deleted to simulate the first search of a layperson. For each search term, the first 10 links were selected, as consumers rarely read beyond the first page of search results [34].

Two reviewers (MZ, JP) independently screened all websites for eligibility. Search terms used were identified and recorded for each website. Websites with only 2 or fewer of the selected search terms were excluded. The 2 lists from Google and Bing were compiled, and duplicates were removed. Inclusion criteria comprised use of the German language and breastfeeding content. Websites inaccessible because of dead or broken links, websites formatted as PDFs, commercial websites, social media platforms (eg, Facebook), and websites without breastfeeding content were excluded.

Three validated tools were used to evaluate websites for quality, suitability, and readability. Each website was reviewed independently by 2 reviewers (MZ, JP) and then evaluated using an evaluation form (see Multimedia Appendix 3).

To appraise the quality of information, the Health-Related Web Site Evaluation Form (HRWSEF) was used [35]. The HRWSEF is an evaluation instrument for health educators and clinicians to evaluate the appropriateness of health education websites for clients and patients. It is a 36-item tool including 9 topics (website information, content, accuracy, author, currency, audience, navigation, external links, and structure). Users judge (1=disagree, 2=agree, 0=N/A) whether the website is a good source of information for the patient or whether it should not be recommended. Instructions of the HRWSEF questionnaire were followed for missing information (questions which were not applicable were deleted from the total scoring). Therefore, the total number of points depended on the number of questions answered with “agree” or “disagree.” Quality was rated based on scoring (0%‐74.9% poor quality, 75%‐89.9% adequate quality, and ≥90% excellent quality) [35].

As for mobile apps, suitability of information was assessed with the SAM questionnaire, and readability was assessed using the Flesch Index online tool [27,28]. The readability of the selected website was evaluated by copying 3 links of a paragraph of the sites hit by the search to the online tool, and the average of the Flesch Reading Ease test results was used for evaluation.

To assess the coverage of information, 11 basic breastfeeding topics were used, based on the guidelines of the German Healthy Start–Young Family Network [36]. The websites and mobile apps were screened independently by 2 reviewers with technical nutrition and lactation expertise for coverage of the 11 topics (MZ, JP). If the 2 reviewers had a different outcome, the websites and mobile apps were screened again to achieve consensus. The 11 items are listed in the 2 standardized self-developed checklists (see Multimedia Appendix 3 for the apps and Multimedia Appendix 3 for the websites). All items on the checklists were dichotomously answered (yes/no). Then 11 items were summed and converted into percentages for descriptive statistics.

Interrater reliability (IRR) testing was undertaken between the 2 reviewers assessing mobile apps and websites by comparing scores from each rating tool. The IRR agreement was assessed using the Kendall coefficient of concordance (Kendall rank correlation coefficient) for ordinal variables [37].

The scores for mobile apps and webpages were summarized using boxplots for each index, where the scores for each index were transformed into percentages to plot all indexes in one plot. Medians and IQR (25th and 75th percentiles) were calculated. Kendall τ correlation was used to examine correlations between Google Play Store ratings, App Store ratings, and the different uMARS scores, Flesch Index, and SAM questionnaire. Values from the 2 app stores were summarized by calculating means. A 2-sided P value <.05 was considered statistically significant. All analyses were performed using R Studio version 4.3.2 (R Foundation for Statistical Computing, Vienna, Austria) [38].

The selection process describing the numbers of mobile apps identified at each stage of the selection process is displayed in Multimedia Appendix 1. A description of all included mobile apps is given in Table 1.

All included mobile apps were free of charge; however, for some mobile apps, upgrades were available that required a fee. The average user rating of mobile apps ranged from 3.3 to 4.7 (out of 5, with 1 being the lowest and 5 being the highest rating). The Google Play Store showed download numbers ranging from more than 5000 to more than 5 million, whereas no such information was available in the App Store for iOS. Six out of 8 mobile apps were found in the stores under the category “Medicine,” one under “Lifestyle,” and one under “Foods + Drinks.” One mobile app (Baby & Essen) was provided by a governmental agency; all others were commercial in nature.

Included mobile apps not only provided informational content but also other features (eg, 3 apps had a tracker for tracking diapers, breastfeeding meals, and sleeping).

For 3 of the included mobile apps, it was not clearly stated whether health professionals wrote the text. None of the identified mobile apps showed any certification to prove independent and correct health information or had a data protection certification.

The mobile app with governmental affiliation showed no advertisements, whereas all others were commercial in nature. Six commercial mobile apps showed advertisements for breastfeeding-related products (eg, pumps, creams, bottles and teats, pacifiers, formula, and supplements) or linked wording within the texts to websites for ordering products. One mobile app (Preglife) showed general family-related advertisements.

There were only 4 mobile apps offering content in a language other than German. The mobile app Baby+ is available in 18 other languages (eg, Turkish, Polish, Russian, and English), the mobile app Keleya: Still- & Beckenboden offers content in the English language as well, Medela in 11 other languages (eg, Russian and English), and Preglife in 9 other languages (eg, Polish and English).

The quality of the selected mobile apps using the uMARS index ranged from 3.4 to 4.6 (out of 5), with a median of 4.1 (out of 5) (83% [IQR 73%‐87%]). However, the mobile apps differed more in the perceived impact of the uMARS, ranging from 23% to 93%.

The suitability of information showed a median of 84% (IQR 70%‐89%), whereas all mobile apps reached at least half of the points in the suitability for population, and some reached the total of 10 points (Medela, Baby+). The median of the Flesch Index was 59% (IQR 53%‐68%), which is around half of possible points to reach (Figure S3 in Multimedia Appendix 1).

The ratings of the subjective uMARS index and the store rating by the users were correlated (Kendall τ=0.67; P=.03). There was no statistically significant correlation between any of the other scores. Detailed evaluation results of all included mobile apps are given in Table 2.

The selection process describing the numbers of websites identified at each stage of the selection process is displayed in Multimedia Appendix 1. A description of all included websites is given in Table 3.

Four websites referred to companies selling baby products (Medela, Lansinoh, Hipp, Windeln), 3 referred to parenting or health-related magazines (eg, Eltern, Apotheken Umschau, Windeln), and 1 referred to a media brand (Familie). Of the 4 informational websites, only 1 had a government affiliation (Kindergesundheit-Info). One website was an encyclopedia (Wikipedia), and 1 referred to a mobile app (Babelli).

Seven websites did not specify which professional group wrote the texts, and only 2 websites provided content written by breastfeeding experts. One of the included websites showed a quality logo (afgis), one the certification of the Stiftung Gesundheit, and one other showed both; thus, less than a quarter of the websites had a quality of information certification.

Eight websites showed advertisements for breastfeeding-related products (eg, pumps, creams, bottles and teats, pacifiers, formula and an infant flatulence product) with a direct link to order these products. Two websites showed no advertisements (Wikipedia, Kindergesundheit-Info), 2 websites showed general advertisements (Still-Lexikon, Apotheken-Umschau), and 1 showed advertisement for a breastfeeding book and consultation (Team Muttermilch).

The quality of the selected websites showed a median HRWSEF of 86% (IQR 83%‐89%). The suitability of information varied between 64% and 100%, with a median of 89% (IQR 78%‐94%). The suitability for the German population was rated between 5 and 10 out of a maximum of 10 points, with 5 websites being recommended to the German population without reservation (Familie, Team Muttermilch, Medela, Lansinoh, Apotheken Umschau). The median of the Flesch Index was 58% (IQR 47%‐61%), which is around half of the possible points to reach (Figure S4 in Multimedia Appendix 1). Detailed evaluation results of all included websites are given in Table 4.

The mobile apps reached at least half of the points in the coverage of information (except Baby Care Gesund & Schwanger), and only 1 mobile app reached the total points (Medela) (Table 2). The coverage of information for the analyzed websites ranged between 8 and 11 (out of 11 items), with almost one-third reaching the maximum total points (Table 4).

The results for coverage of information show that websites provided more information for many topics than mobile apps (Figure 1).

Three companies (Medela, Eltern, Hipp) offered both a website and a mobile app. Some differences between mobile app and website content were identified. Website content was of higher quality, suitability of information, and coverage of information, and was easier to read compared with the app from the same company (Table S4 in Multimedia Appendix 1).

IRR testing between the 2 reviewers assessing all selected mobile apps and websites showed a Kendall coefficient of concordance values of 0.92 for mobile apps and 0.90 for websites (P<.01) (Tables S5 and S6 in Multimedia Appendix 1).

This is the first cross-sectional study of mobile apps and websites with breastfeeding content in Germany. The 8 apps scored a median of 4.1 (out of 5) on the uMARS scale, indicating good quality. Three of 13 websites were rated as excellent (3/13, 23%), and 10 of 13 as adequate quality (10/13, 77%) as per the HRWSEF. Four of the 8 mobile apps (4/8, 50%) and 9 of 13 websites (9/13, 70%) were rated as difficult or challenging to read based on the Flesch Index. Websites provided more comprehensive information coverage based on the checklist, with a median of 9 of 11 (IQR 8‐11) topics covered. This was slightly more than mobile apps, which had a median 7.5 of 11 (IQR 6.5‐9.5) topics covered.

Only 1 mobile app and 1 website had a governmental affiliation, whereas all other websites and mobile apps were commercial in nature. Most mobile apps (6/8, 75%) and websites (8/13, 62%) displayed advertisements for breastfeeding-related products. Only a minority of mobile apps (4/8, 50%) and websites (2/13, 15%) offered content written exclusively by medical and allied health professionals. We found the number of available commercially developed mobile apps and websites written by nonprofessionals to be concerning, since they can potentially provide biased or inaccurate information to parents.

Overall, we found that German mobile apps with breastfeeding content were of average to good quality based on their mean uMARS quality scores. Findings from studies in other countries that reviewed infant-feeding and pregnancy mobile apps rated the majority as poor or moderate quality [17,20,52,53]. Only 1 review rated 16 parenting mobile apps as good or excellent [18].

Compared with those studies, our reviewer ratings were typically higher for items included in the uMARS objective score than the subjective score. According to our findings based on the subjective uMARS scale, we would not recommend 2 of the 8 mobile apps (2/8, 25%). In addition, we would only recommend 5 mobile apps (5/8, 63%) with reservations. Only 1 mobile app had a high recommendation score (Preglife; Table S3 in Multimedia Appendix 1). Our findings align with Cheng et al [20] and may reflect reviewer bias, as most apps (7/8, 88%) were commercial. Given that the reviewers were health professionals trained in recognizing the effects of advertisements on health behavior, their positionality may have influenced the scoring. Specifically, commercial apps may have received lower ratings because of the reviewers’ awareness of how marketing strategies can shape consumer choices and health outcomes. A review of Virani et al [18] found that commercial parenting apps were more popular than governmental, likely because they are more visually appealing and entertaining. Apps that offer additional features, such as trackers, receive higher user ratings—an increase of 27% for each added feature [54]. However, monitoring and tracking breastfeeding seems to be the least effective way to improve breastfeeding outcomes [55].

Three websites were rated as excellent, and 10 as adequate quality. The selected German websites were of much better quality than those reviewed in previous studies [52,56].

The majority of mobile apps and websites received high ratings for health information, while a smaller proportion was deemed adequate. Previous evaluations reported lower scores, indicating that the quality of these resources has presumably improved in recent years [20,52,56].

Germany is ethnically diverse, with about a quarter of the population having a migration background [57]. In contrast to a review of Australian apps and websites, adequate suitability for population was considered for about half of the reviewed mobile apps and websites in our study [52].

Three-quarters of the websites and half of the mobile apps in this study were considered challenging or difficult to read and understand. A high literacy level is required to understand the content of many mobile apps and websites [20,52,56]. A previous survey in Germany found that 23% of the interviewed persons (n=2902) said the health information provided verbally or written was difficult to understand [1]. For example, print materials and health mobile apps for weight loss could be improved by using simpler language [58]. Apps that are only available in German could at least be improved by using a lower level of literacy if they aim to reach non-native German speakers. Turkish, Polish, Russian, and Syrian people are the largest immigration groups in Germany [57]. The mobile app Baby+ is available in 18 other languages, which is likely the reason it has the most downloads.

Three of 8 apps and 7 of 13 websites provided insufficient information about the qualifications of their authors. In Germany, titles such as “breastfeeding counselor” or “breastfeeding expert” are not certified professions, whereas “International Board Certified Lactation Consultant” (IBCLC) is protected [59]. Among the websites and apps included in this study, only 3 websites and no mobile app were authored by IBCLCs. From the perspective of parents, Hughson et al [12] claimed that users prefer mobile apps that come from a trusted source, whereas Biviji et al [16] found little concern among the users about evidence-based content. Helpful online breastfeeding support can increase the likelihood of breastfeeding [60], but breastfeeding-related advice and information that is potentially produced by unqualified health professionals or lay persons or without clear authorship is concerning.

Content that is not evidence based is harmful because it may spread myths, outdated or inaccurate advice, or contradict current public health recommendations, potentially increasing parental confusion and negatively affecting breastfeeding rates. Additionally, there is a lack of standards for the development of health apps [61]

Our results match previous findings stating that the coverage of information in mobile apps and websites is poor and partially contrary to Infant Feeding Guidelines [19,20,52,53,56]. When screening the apps and websites, the uMARS referred to the quality of information asking “Is app content correct, well written, and relevant to the goal/topic of the app?” (see Multimedia Appendix 2, app evaluation form), and the HRWSEF referred to accuracy asking “The information is accurate” (disagree, agree, or not applicable; see Multimedia Appendix 3, website evaluation form). However, as the user instruction for these tools suggests browsing the website(s) for a few minutes and the app(s) for at least 10 minutes (see Multimedia Appendix 2 and Multimedia Appendix 3), it is not possible to view the complete content. The rating of these questions is therefore based on the content assessed. To highlight the importance of evidence-based information, based on the German guidelines of the German Healthy Start-Young Family Network by the Federal Center for Nutrition, the authors used the self-developed checklist for coverage of information.

Of the 13 websites reviewed, 8 linked directly to shops for breastfeeding products or breast-milk substitutes or displayed related advertising. One site (Babelli), which claims to be certified by the Health Foundation, linked to welcome gifts from companies producing breast-milk substitutes. Three apps directly referred to companies promoting their own products (Baby+, Hipp, Medela), and 3 others displayed advertisements for breastfeeding-related products or linked parts of their text to online shops. In our view, information intended to support breastfeeding should not include advertising for breast-milk substitutes. Nevertheless, manufacturers use digital platforms to combine breastfeeding information with marketing [62,63]. The International Code of Marketing of Breastmilk Substitutes of the World Health Organization [64], European Union legislation on infant formula [65], and German legislation on infant formula [66] all stipulate that there should be no advertising or other form of promotion of infant formula to the general public. The World Health Organization recently published technical guidance on regulatory measures to restrict digital marketing of breast-milk substitutes [67]. Until now, Germany has only adopted some of the provisions of The International Code; therefore, information on follow-on formula is permitted to be advertised to the general public [68].

Several certificates for quality of health-related websites exist in Germany, including Action Forum Health Information System (afgis) [69], Stiftung Gesundheit (Health Foundation) [70], and the medical search machine for German-speaking countries (Medisuch) [71]. The HONCode (Health on the Net Foundation) [72] was a reliable certification for ethical and qualitative standards of websites with health-related topics. Although it stopped its service in 2022, some websites still showed the certification at the time of our search. Only 3 websites were certified, and not even a quarter of the websites had any proof for correct medical information.

One strength of this study is that the quality appraisal of websites and mobile apps was performed using a variety of validated tools.

The scope of this study was limited to mobile apps and websites with breastfeeding information available in German. Therefore, the findings of this study are not generalizable to websites and mobile apps in other languages.

This study focused on free-of-charge mobile apps, which are used most often. It is plausible that higher quality information might be available on breastfeeding mobile apps that charge a fee. Some health insurance companies offer a mobile app for pregnancy or parenthood with breastfeeding content “free of charge,” but these are only available to their insured members through app stores. Our mobile app search did not include mobile apps provided by health insurance companies. Due to the possibility of using Google Trends and Ubersuggest, the selection of websites involved more differentiated search terms compared to mobile apps. This might have increased the likelihood of selection bias.

The reviewed apps and websites were generally of acceptable quality, but many included advertisements for breast-milk substitutes. Credible health authorities should improve visibility through effective keywords, regular updates, and content authored by qualified professionals. Independent certification for noncommercial, accurate health information is needed, and all content should follow relevant guidelines. Texts should be motivating, easy to read, and supported by summaries, visuals, and videos to aid users with limited time or literacy. To reflect diverse populations, providers should use inclusive images and offer translations into commonly spoken languages.

Features such as breastfeeding support, meal and sleep tracking, quizzes, recipes, milestone documentation, and chat functions may enhance user engagement and app ratings and could be integrated by public actors when designing evidence-based health apps.

In conclusion, although current evidence is insufficient to demonstrate sustained beneficial effects of breastfeeding promotion and support on breastfeeding rates through mobile apps alone [15], integration of evidence-based educational content in mobile apps together with personalized remote support nonetheless holds potential to improve breastfeeding outcomes [55,73].