Evaluation of an mHealth Intervention (Growin’ Up Healthy Jarjums) Designed With and for Aboriginal and Torres Strait Islander Mothers: Engagement and Acceptability Study

Introduction

Background

Aboriginal and Torres Strait Islander people are the oldest surviving culture in the world [1]. The health of Aboriginal and Torres Strait Islander people changed significantly upon colonization and has continued to be disrupted by subsequent policies [2]. Improving the health and lives of Aboriginal and Torres Strait Islander people is a national priority. Mothers and babies receiving the best possible care and support for a good start to life is 1 of the 12 health priorities of the National Aboriginal and Torres Strait Islander Health Plan 2013-2023 [3]. Providing access to culturally responsive health information and services is an important strategy for achieving this goal [3].

Improving health literacy provides a foundation for individuals and communities to take action to improve their own health [4]. There is limited evidence on effective health literacy programs for Aboriginal and Torres Strait Islander people [5]. A systematic review examining interventions for improving health literacy among Aboriginal and Torres Strait Islander people included 5 studies with the following interventions: exercise classes, nutrition and cooking workshops, discussions and role plays, presentations, other learning activities, incentives, and reduction in the cost of fresh and frozen produce and low-sugar beverages and education at the point of sale [5]. All the included studies demonstrated statistically significant improvements in at least 1 health literacy–related outcome measure, although it should be noted that study quality was compromised because of small sample sizes and poor attendance [5]. More rigorous trials are needed on health literacy programs designed and implemented by Aboriginal and Torres Strait Islander people for Aboriginal and Torres Strait Islander people.

An array of mobile technologies is available to find, share, and generate health information [6]. The major benefit of mobile health (mHealth) is its ability to reach a large number of consumers, including those who cannot attend health services. Aboriginal and Torres Strait Islander women have a high interest in using mHealth [7] but have different preferences for delivery as well as content. Evidence to date shows that Aboriginal and Torres Strait Islander people are frequent users of Facebook (Meta Platforms, Inc) [8-11] and SMS text messaging [12-14] and report high acceptability of, but low engagement with [15], apps, as is often the case universally [16]. Content that centers on culture and frames positive health messages has greater acceptability [9,10,17]. Furthermore, certain delivery mechanisms may be particularly engaging to mothers. A report on Australian women’s use of digital health found that women caring for infants and young children were more likely than other women to use social media and web-based forums to share and create health information [8], whereas other studies have found SMS text messaging to have high acceptability among mothers [10,11].

Available mHealth programs for Aboriginal and Torres Strait Islander mothers or their children are limited but growing, including an app, a website, and SMS text messaging on infant feeding [18]; SMS text messaging, videos, and multimedia messaging service for otitis media in children [13]; SMS text messaging, a phone call, Facebook, or an email for postpartum blood glucose screening [12]; a prototype app for social and emotional well-being during pregnancy [19]; and a mindfulness app for women and children of all ages [20]. In the gray literature, the authors are aware of the Deadly Tots app and interactive website on child development [21] and Facebook pages such as Stay Strong and Healthy page for health during pregnancy [21] and Yarn and Heal—Our way for Aboriginal women of all ages to connect and yarn [22]. It is important that we seek to advance mHealth solutions developed by and for Aboriginal and Torres Strait Islander women to promote digital inclusion and access to health information, particularly as it is known that cultural minorities are less likely to use mainstream web-based health technologies [6].

In 2019, we co-designed a multimodality mHealth program for Aboriginal and Torres Strait Islander women’s and children’s health [23]. The aim of the program was to improve health literacy and health behaviors as well as increase access to health services. Formative research with 31 women and 11 health professionals took place in 3 communities in New South Wales (NSW) and included focus groups with storyboards, card sorting, and design activities [23]. On the basis of the findings from the formative research, we developed a web-based prototype application, an SMS text message library, and a Facebook page, collectively called the Growin’ Up Healthy Jarjums (an Yugambeh word used on the East Coast of Australia meaning children) program.

Following a formative research phase, subsequent steps to develop and evaluate mHealth interventions include conducting a pilot study, a randomized control trial, and an evaluation of the implementation impact [24]. The purpose of the pilot study stage is to determine acceptability, improve and refine the intervention, and test the content and regimen early in the research process [24]. Refining the intervention is often an iterative process in each research phase and beyond [24]. Continual improvements to mHealth interventions are important, given the constant upgrades to technology and that long-term engagement with mHealth can be difficult to achieve [25]. We used a pilot study design to evaluate the acceptability of and engagement with the Growin’ Up Healthy Jarjums prototype program.

Objectives

The aims of this study were (1) to assess the engagement with the Growin’ Up Healthy Jarjums program among mothers (or other women) caring for Aboriginal and Torres Strait Islander children aged ≤5 years; (2) to assess the acceptability of the Growin’ Up Healthy Jarjums program among mothers (or other women) caring for Aboriginal and Torres Strait Islander children aged ≤5 years; and (3) to assess the acceptability of the Growin’ Up Healthy Jarjums program among health professionals and early educators.

Methods

Project Design

A 4-week pilot study of the Growin’ Up Healthy Jarjums mHealth program was undertaken with Aboriginal and Torres Strait Islander women caring for children aged <5 years. Health professionals and early educators from the participating services provided feedback on the intervention in focus groups. Details on the development of the Growin’ Up Healthy Jarjums mHealth program can be found elsewhere [23]. The Aboriginal Health and Medical Research Council (AH&MRC) Ethical Guidelines: Key Principles (2020) version 2.0 were used to guide the implementation of this pilot study [26].

Research Team

This research was governed by an Aboriginal advisory board in partnership with Aboriginal organizations (listed in the Acknowledgments section) and coled by a Kuku Yalanji and Lama investigator (KH), as well as 2 non-Indigenous investigators (BB and JM). In total, 3 team members were Aboriginal women from (or connected to) the communities where the research took place: a Gumbaynggirr woman (NS), Gomeroi woman in the Kamilaroi Nation (BL), and Worimi woman working in the Awabakal community (BH). The cultural identities of the remaining team members are Macedonian-Australian (BB), German-Australian (JM), Pakeha or European-New Zealand (RD), and European-Australian (SJP). The team has various professional backgrounds: 4 women with Aboriginal lived experience (KH, NS, BL, and BH), a behavioral scientist (BB), a pediatrician and academic (JM), a nurse and public health researcher (KH), an mHealth and public health researcher (RD), an early educator (BH), an Aboriginal health practitioner (NS and BL), and an occupational therapist and PhD candidate (SJP). All team members contributed to the conception of this study. Aboriginal researchers from the participating communities (NS, BH, and BL) led the implementation of the project to support cultural safety.

Participant Sampling

Women aged ≥16 years who were either mothers or primary carers of Aboriginal or Torres Strait Islander children aged birth to 5 years or were pregnant (≥30 weeks’ gestation), owned or regularly used a smartphone, and had accessed a participating service (an Aboriginal health service or NSW health service) were eligible to participate. Health professionals from the participating health services and early educators from the participating preschools of all cultural identities who worked with women or children were eligible.

Procedures

This study was conducted remotely from August 2020 to March 2021 using telephone, SMS text messages, and videoconferencing owing to COVID-19 restrictions. Participants were recruited from 3 regional locations in NSW, Australia. A total of 5 Aboriginal organizations (2 Aboriginal health services, 2 Aboriginal preschools, and 1 Aboriginal family and parenting corporation) and 3 NSW health sites participated. In total, 2 Aboriginal researchers (NS and BH) completed most of the recruitment, consent procedures, interviews, and communication with the participants and services in line with the AH&MRC Ethical Guidelines to ensure culturally safe, best practice research procedures (2.2.3, 2.3.3, 2.5.2, and 3.3.1) [26].

Women who participated in the co-design phase [23] were contacted via phone and invited to participate in the pilot study. Convenience snowball sampling was also used [27]. The Aboriginal researchers (BH, NS, and BL) used their personal networks to recruit additional participants. The participants were also asked whether they would like to recommend a friend or family member to the study. The participating health services also reached out to potential participants. Potential participants were screened for eligibility when contacted by the researcher via phone. The researcher explained the study and obtained informed consent. The participants were sent an SMS text message with a link to a baseline survey on REDCap (Research Electronic Data Capture; Vanderbilt University) before starting the pilot study. During the 4-week study period, the participants were given access to the intervention (Textbox 1). The participants were sent a link via an SMS text message to access the application, and where possible, the research team contacted the participants to check whether they were able to access the application, explain the use of the application, check whether they were receiving SMS text messages, and explain how to like the Facebook page. The participants were asked to access the application as often as they felt compelled to, that is, there was no required amount of time that women needed to spend on the application or other parts of the program. Following the Facebook page was optional. After 4 weeks, the participants were contacted via telephone for an interview. Semistructured interviews with a mixture of open- and closed-ended questions [28] were conducted by Aboriginal researchers (NS and BH) and a non-Indigenous PhD student (SJP). The interviews were 6 to 25 minutes in length. They were recorded and transcribed, and interview notes were taken as a backup to recordings [28]. The participants were reimbursed with a shopping voucher worth Aus $20 (US $30) at baseline, a shopping voucher worth Aus $10 (US $15) per week for the 4-week pilot study Aus $40 (US $60) in total to cover data use, and a shopping voucher worth Aus $20 (US $30) for participating in the follow-up interview. The interviews were completed between August and September 2020.

The professionals who participated in the co-design phase were contacted via phone and invited to participate in the pilot study. Where these professionals were no longer working at the service, other professionals known to the Aboriginal researchers were contacted to participate. A total of 2 focus groups were conducted in February and March 2021. Focus groups [28] were conducted rather than individual interviews as per the professionals’ preference. Consent was initially obtained over the phone and then again in person, videoconference, or email before starting the focus group. A brief survey was conducted at the start of the focus group. The professionals accessed the program during the focus group only (not during the 4-week pilot study). We were interested in the professional’s feedback on the content only, not in how they might engage with the program over 4 weeks, as they were not the target end users. It was important, however, to obtain feedback from professionals who routinely provide health information to mothers, as they may be instrumental to the implementation of the program, if the program is effective. One focus group was conducted in person (as COVID-19 restrictions had been lifted) and another over videoconference. The focus groups were 13 and 21 minutes in length. The professionals were not reimbursed.

Measures

Demographics and Cultural Characteristics

The survey completed by mothers was a 16-item survey including demographic, cultural, and socioeconomic items. The items were selected from a previous study [29], with all items having been tested with Aboriginal and Torres Strait Islander mothers previously. The survey completed by professionals comprised 5 items related to demographic and professional practice characteristics.

Engagement

Objective measures are common for measuring the engagement of applications [30]. The user activity metrics collected included the number of log-ins, number of page views, length of page view, and number of links used on the application. We used user activity metrics in combination with interview data to identify user typologies. Data collected for the videos included the number of plays in total and per video, number of videos watched in full (completions), duration watched (in seconds; mean seconds and percentage), and number of unique videos. Data collected for the Facebook page included the number of posts by administrators, number of page likes, number of comments, number of followers, and the reach of posts and videos. Data were collected on topics that the women chose to receive SMS text messages on and the number of women who opted out of receiving SMS text messages. User engagement was evaluated only for the women participants (end users), not for the professionals.

Acceptability

An interview schedule was adapted from a previous study on the acceptability of a culturally tailored SMS text messaging program for mothers [31]. The interview schedule included the following topics: usefulness of the program, cultural appropriateness of the program, ease of understanding, appropriateness of the program, relevance of the program, perceived impacts, and suggestions for improvements. A shortened and adapted version of the interview schedule was used with professionals, which included items on usefulness and cultural appropriateness.

Data Analysis

The interview data were analyzed and summarized using descriptive quantitative analyses including means, SDs, and proportions [32]. Qualitative comments were analyzed using a simple thematic analysis with predetermined codes based on the research areas, for example, cultural appropriateness [28]. One of the researchers (SJP) cleaned and coded the responses for each predetermined code. Then, 3 researchers (SJP, BH, and NS) reflected on and discussed the participant quotes to form a summary statement for each code and select representative quotes.

Ethics Approval

Human research ethics approval was received from the AH&MRC (1485/19) and University of Newcastle (H-2019-00760).

Results

Overview

A total of 47 participants were recruited for the study: 41 (87%) women and 6 (13%) health professionals. The average age of the women was 31 (SD 7.35) years. The women were from 15 different communities; Kamilaroi (12/41, 29%) and Gumbaynggirr (10/41, 24%) were the most common. Almost half of the women (20/41, 49%) in this study had participated in the co-design phase of the project. The demographic characteristics of the participants are presented in Tables 1 and 2.

User Engagement (n=41 Women)

App

Of the 41 women, 31 (76%) participants accessed the application. Among these 31 participants, there were 154 log-ins, with an average of 5 log-ins per person. Of these 31 women, 13 (42%) users scrolled the main page only, and the remaining 18 (58%) users moved past the main page by clicking on other pages. A total of 23% (7/31) of the users clicked on 10 website links.

A total of four user typologies were identified: (1) could not use the app, (2) obligated to use the app, (3) reviewers, and (4) researchers. The “couldn’t use the app” group included those who could not log into or download the application. The “obligated to use the app” group included those who used the app to provide feedback in a research context and probably would not use the application in a real-world setting. The “reviewer” group included those who logged in once or twice out of curiosity to see what the application included but did not consistently use the application. The “researcher” group included those who used the application more regularly; they were users who likely wanted more information than available in the SMS text messages or on the Facebook page. By analyzing the user activity metrics, we estimated that 20% (8/41) of the women were “researchers,” indicating that they may have long-term engagement with the application in the real world.

Videos

There were 48 plays of the 12 videos (Table 3). The number of plays ranged from 0 for “Milestones” and “Exercise” videos to 11 plays for “Sleeping” video. The number of unique viewers ranged from 1 to 5 per video. The highest number of unique viewers was for the “Feeling Good” video. Among the 48 plays, there were only 6 (13%) video completions. The mean viewing time was 38 seconds. The ability to obtain feedback on the video content may have been limited by the fact that the videos were not watched by most women. One reason for this was that to watch the videos on the Facebook page, the user needed to leave the Facebook page and watch them on an external host (Vimeo [Vimeo, Inc]). Another reason may have been that the videos were not clearly displayed in the application. Finally, the duration of the videos may have been too long.

Facebook Page

Facebook administrators (Aboriginal researchers BH and NS) posted 101 posts over the 4-week pilot study. The page received 49 page likes and 51 followers, indicating reach beyond the study participants. The post with the highest reach was a supportive and affirming cultural post, which reached 308 people and had 17 reactions, comments, or shares. The second most popular post was a competition post, which reached 58 people and had 23 reactions, comments, or shares. The videos posted on Facebook (n=12) had an average of 20 people reach but only 1 to 2 reactions, comments or shares or clicks to watch externally.

SMS Text Messages

No participants opted out of the SMS text messages. The participants selected 3 topics to receive SMS text messages on. In the order of popularity, the topics chosen were ears, eyes, and teeth (23/41, 56%); sleeping (19/41, 46%); exercising (17/41, 41%); feeding and eating (13/41, 32%); eating (12/41, 30%); women’s business (12/41, 30%); milestones (10/41, 24%); feeling good (10/41, 24%); vaccinations and medicines (6/41, 14%); and safe drinking (0/41, 0%).

Acceptability

Overview

Of the 41 women, 32 (78%) were interviewed at the end of the pilot study. All the 6 professionals were interviewed. There were 7 themes identified in the analysis related to the acceptability (Figure 1).

‎

Usefulness

Almost all women (30/32, 94%) reported that the Growin’ Up Healthy Jarjums program was useful. On a scale of 1 (a little useful) to 5 (extremely useful), the mean rating of usefulness was 3.9. Furthermore, 84% (27/32) of the women reported that the program was relevant to them.

All women (30/32, 94%) reported that they would recommend the program to other families; the reasons included the following: helpful to first-time mothers, younger mothers, and mothers without family and other supports; ease of having all information in 1 spot; connection with other mothers; provides a sense of community; visually appealing and representative of Aboriginal and Torres Strait Islander people; and an accessible place for women feeling too ashamed or isolated to go the hospital or physician to access reliable health information:

I reckon just the feeling of still being connected, and being supported. I think it’s a nice way, especially for mums with little, little kids, I reckon sometimes you feel pretty isolated, especially if you’re not working and stuff. I think it’s a nice way to still feel like someone’s looking out for you or thinking of you.
[Participant 6]

I think it’s a good tool for our community, especially the young ones that we’ve got who may not have anywhere else to go to find that information or who are too ashamed to ask. I think having it in a way that they can find it themselves in an easy format is a good thing.
[Participant 27]

All professionals (6/6, 100%) reported that the Growin’ Up Healthy Jarjums program was useful. On a scale of 1 (a little useful) to 5 (extremely useful), the mean rating of usefulness was 3.3. All the professionals reported that they would recommend the program.

The professionals reported that the program would be useful to families for different reasons. Some of the cited reasons are as follows: the program is relatable to mothers because of co-design methods; using Facebook will result in family and friends seeing the health information; “Storytime” would be good for families that have less access to books; Facebook page was welcoming, easy to access, and visually attractive; the app was easy to navigate and would be easy for mothers to look through while “on the go”; overall, the program has a good balance of content, including health information, affirmations, and what is on in community; the Facebook page may be a good place for women to get ideas from each other and chat about recommendations from the posts; the Facebook page would be a good place for women to connect and not feel isolated; and the content is largely positive, which is important:

I like, on the Facebook page, it gives you a sense of not being so isolated. The Facebook page is a really nice place for Aboriginal mums or families to not feel isolated if they’re being recommended all this stuff and things to do outside of their lives, and then there’s the affirmations. It’s probably a really nice place for them to be especially if they’re suffering any mental health or with any isolation in their own lives.
[Participant 6]

Cultural Appropriateness

All the women (32/32, 100%) reported that the program was culturally appropriate. They reported that the colors, graphics, and language used were culturally representative. Of the 32 women, 1 (3%) woman recommended using different languages depending on what community the program is intended for. Another woman who had no exposure to her culture said that it was a helpful way to learn about her culture:

I’m Aboriginal, but I only just learnt of my Aboriginality, so I wasn’t actually brought up that way [according to her culture]. So it was helpful for me to learn new terminologies and stuff like that.
[Participant 3]

All the professionals reported that the program was culturally appropriate. A total of 33% (2/6) of the professionals commented on the cultural appropriateness of the language in a positive light. Overall, 17% (1/6) questioned whether the language would be difficult for some women to understand. She emphasized the need to ensure that representatives from each community that the program would be used in be involved in developing the content as well as to administer the Facebook page to ensure that the program continues to be relevant to women from different communities:

You’ve got different lingos, different meanings, different sayings that’s going to grab attention [in different communities]. When I look through these text messages there are some words in here that I would think some of our women wouldn’t really understand. You actually need the women to do it, they’re very different to the workers. They’re the ones that are going to give you the right language.
[Participant 1]

It’s all culturally appropriate...the language is really nice. It’s really easy to read and interpret. It doesn’t have any of those big, yukky words that can be quite clinical.
[Professional 5]

Easy to Understand

All the women (32/32, 100%) reported that the content was easy to understand. They talked about the program being jargon free but not too simplified:

It was spoken to you normally, not like all the medical jargon, do you know what I mean? It was understandable and relatable. I think if it’s too technical sometimes, it gets overwhelming.
[Participant 20]

It wasn’t dumbed-down. Like some things that we give to Aboriginal families that we get, some of them are so simplified that makes people think that we’re stupid, but this wasn’t. It was easy to understand, but it didn’t make me feel stupid, didn’t make me feel bad.
[Participant 27]

Appropriate

Most women (28/32, 88%) reported that the activities and information were appropriate. However, of the 32 women, 1 (3%) woman suggested that the language in the SMS text messages could have been more professional rather than colloquial:

We followed what was on those messages each day. We made it a project with our kids because we thought we would see how it would go. My son likes looking at the pictures of it when they’ve had the Facebook competitions because it was all kids that he knew.
[Participant 27]

Relevance

Most women (27/32, 84%) reported that the program was relevant to them and their family. They said that it was good having other mothers to relate to and access to reliable health information to talk to family about:

Just sort of seeing other women with kids and stuff and sort of just having someone to relate to, somebody that’s a little bit more similar to me. So you don’t feel so alone in what you’re kind of going through. Things that you might think are silly or you’re a bit shamed to ask anyone. At least the messages address it and then you can see other women on the Facebook page.
[Participant 4]

I learnt some good stuff. I think the thing I like most about the program for us personally, it allowed me to have conversations with my partner about smoking. With the program I was able to say ‘hey I got a text about smoking,’ ‘do you know this’ it meant that it wasn’t just me making points. I sort of used the app as the conversation starter.
[Participant 18]

The women who did not find the content relevant to them (2/32, 6%) said that they already knew the information or felt that the information was targeted toward women caring for younger children. Of the 32 women, 2 (6%) women also made comments about some SMS text messages not always being relevant to their family:

Some of them [SMS] might not be relevant to me personally, but I think as a community they are. The messaging was consistent, and I think that’s really important.
[Participant 6]

My little boy is older now, he’s two. So I felt like a lot of the messages and stuff like that, was more around the newborn stuff. But in saying that, if I had a newborn still, then it would have been more relevant.
[Participant 10]

Perceived Impact

Of the 31 women who commented on the overall positive impact of the program, 22 (71%) women reported that the program had an overall positive impact on themselves and their family. Table 4 presents a summary of the perceived impacts. The most common perceived impact was feeling more supported (21/29, 72%), followed by improvements in knowledge or understanding of child health (13/26, 50%), eating habits (11/29, 38%), and exercise (11/29, 38%). Many women commented on the supportive and affirming aspects that the program provided, including the validation of how hard parenting can be; information that certain health problems, such as ear infections, are common in children (validating that it was not their fault); the feeling that a service cared about their child’s health; and the feeling that someone cared about them, which arose because of the reception of regular messages:

The favourite text of mine was the reminder that we all have rough days...And that it was okay, I thought you know what, yeah, I am going to take a breath right now, and it is all okay.
[Participant 10]

It was good to know that there was a service out there that did care I guess or had an interest in my son’s health.
[Participant 14]

Other positive impacts that the women discussed included getting their child’s ear health checked by a general practitioner, more play with children, spending time together as a family, taking children for hearing and vision tests, more exercise, taking care of themselves, talking to friends and family about quitting or reducing smoking, cooking with children, limiting alcohol, getting their child immunized, improved knowledge of contraception, and improved family eating:

Usually I’m the type where I walk to the park and then watch her play. When I read the messages, I’m like, I should actually try with her more, and be more active with her at the park.
[Participant 24]

I was drinking far too much, it was just a stress handling thing, because we did have a lot of problems, and it was difficult the first couple years. And so, getting that information, that really helped me to kind of kick that habit and to look at my own lifestyle and stuff.
[Participant 9]

Other women experienced stressful life situations at the time and had competing priorities, limiting the potential impacts of the program.

Suggestions for Improvements

The most common suggestion for improving the program was to make changes to the application to overcome technical challenges. Of 32 women, at least 6 (19%) experienced technical problems. Difficulty in downloading and saving the web-based application, rather than accessing an Android or iOS (Apple Inc) app, was the main difficulty. Almost half of the women (14/32, 44%) suggested improvements to the application, including making the web-based application an Android or iOS app so that it “looks” like an app, removing the step of saving to the application to home screen, implementing single log in, making navigation simpler, making the application accessible on all phone types, localizing the application to specific communities, and making the application more interactive:

I had to log back in and it would take me to the web page. It kept wanting me to resave it to my home screen, but it was already saved to my home screen. I wasn’t too sure what was going on there.
[Participant 30]

The next most common suggestion is related to the SMS text messages. A total of 59% (19/32) of the women reported that there was just the right number of SMS text messages, but 38% (12/32) of the women reported that the SMS text messages were too frequent. The women indicated the preferred timing of SMS text messages to be earlier in the day and a preference to have SMS text messages from the same sender phone number for ease of reviewing.

Other suggestions to improve the program overall included suggestion to ensure better tailoring to the child’s age as well as suggestions for alternate topics (eg, parenting, toilet training, separation, toddler development, mental development for boys, available services, preschool readiness, allergies, and resources and services specific to Aboriginal people), suggestion to provide less content about smoking, suggestion to provide more links to further information (eg, local mothers’ groups), and suggestion to provide to make the program more interactive:

If it is targeting under five, some of the information could be more around toilet training, difficulties with separation, entry to preschool, advice on services and stuff, like Koori stuff around.
[Participant 9]

In total, 6% (2/32) of the women talked about web-based groups: 1 woman suggested a Facebook group (rather than a Facebook page) and another suggested a chat group within the application (to discuss specific topics). Moreover, 6% (2/32) of other women talked about the importance of the continuation of the program and the longevity of Aboriginal and Torres Strait Islander health programs in general, as Aboriginal health programs often have short funding cycles and the community is left with a gap:

I would have benefited more with the Facebook group if it was an actual group created, because normally you get notifications and stuff when you’re in the group and it tells you who’s posted what. I probably would have had more interaction with that if that was a constant notification coming up.
[Participant 13]

Keep it going. We find sometimes that programs are really good and then they stop. When the funding runs out or it doesn’t get approved or whatever it stops and then that’s a gap.
[Participant 27]

The professionals suggested several ways to improve the program, including continued involvement of women in the development of the program to ensure that the language and content remain relevant and appropriate, including for families living in regional and remote areas. Another professional suggested that the content should be current and tailored to the age of the child. Another suggested a “search” feature in the application so that families could easily search for the health issue or topic that they are interested in (otherwise, mothers will likely Google health information, which can make it difficult to determine reliable sources). Another suggested more specific steps about how to manage certain illnesses and to include more common childhood illnesses. Another suggested that the application needs to be more interactive, for example, tailored specifically to the child’s age, with notifications for activities for that age group or milestones. One other professional suggested forums or private group chats so that discussions are not public, whereas another suggested providing grandparents and other family members access to the program.

Discussion

Principal Findings

Overall, the Growin’ Up Healthy Jarjums program was found to have high acceptability. The results indicate that women found the program to be useful, culturally appropriate, and easy to use, and most women reported positive impacts. None of the participants withdrew from the SMS text message portion of the program, which indicated the high acceptability of and engagement with this component. Engagement with the Facebook page was found to be higher than that with the application. Individual users preferred different modes (SMS text message, Facebook page, or application), indicating that a multimodal intervention increases reach. Importantly, this pilot study showed several ways to improve the program, including technical changes to the app.

Similar to other studies with Aboriginal and Torres Strait Islander people, the SMS text message component of this program appeared to have high acceptability and engagement [12-14]. This pilot study provided an additional opportunity to focus on mothers and examine the desired frequency of SMS text messages. The frequency of SMS text messages sent in mHealth trials is often 1 SMS text message per day, although it can vary from multiple SMS text messages per day to weekly SMS text messages [34]. In our study, women were sent 1 SMS text message per day for 5 days of the week over 4 weeks. Most women (19/32, 59%) reported that the frequency was just right, 38% (12/32) of the women reported that the SMS text messages were sent too often, and 3% (1/32) of the women reported that the SMS text messages sent were not enough. This finding is similar to that of another study that reported that 1 SMS text message per day was preferred by the majority (42%), whereas the remaining 58% preferred either more or less frequency [24]. Giving a choice of frequency of either 1 SMS text message per day or 3 SMS text messages per week may be more appealing to users.

The women reported that they liked having a choice of topics for SMS text messages, as this increased relevance. Some women commented on the core (requisite) topics, indicating (1) breathing well and (2) smoke-free families as being irrelevant to them and their families. SMS text messages on smoking cessation and child lung health were requisite based on the original focus of the intervention to promote child lung health, including smoking cessation. The focus on child lung health was based on the call for more culturally appropriate information on childhood coughs [35]. In addition, there is strong evidence that SMS text messages are effective for quitting smoking [36]; thus, we decided to keep the focus on child lung health, including smoking cessation, for the SMS text message portion of the pilot. Furthermore, many of the SMS text messages were targeting the first 2 years of life, as that was the age range in which we expected to recruit most children; however, many of the children were older, which meant that some of the information was not relevant, although most women commented that they could see how beneficial the information would have been when their children were young. Encouragingly, the number of women who reported the SMS text messages to be irrelevant was low (3/30, 10%), similar to another study on SMS text messages for new mothers (6/22, 21%) [31]; however, giving users the choice of all topics may be a more acceptable and useful approach allowing for better tailoring to end users’ health information needs and interests.

Many participants experienced technical challenges in accessing the app, with nearly one-fourth (10/41, 24%) having been unable to access it. The prototype application used in the pilot study was a web-based application. A web-based application is accessed through an internet browser, such as Google Chrome (Google LLC) or Firefox (Mozilla Foundation), and is essentially a website designed to look like an Android or iOS app. Android or iOS apps are downloaded from an app store and saved on the phone [37]. The benefits of web-based applications are that they are fast to build, they are cost-effective, and their content can be changed easily [37]. The benefits of Android or iOS apps are that they are faster than web-based applications; they can work without internet connection; and end users are generally more familiar with them, including with downloading and saving them [37]. Using a web-based application for this trial resulted in many women having difficulty logging in and saving the application on their home screen. The women also commented that it did not “look” like an application and that it was slow. Other application trials with Aboriginal and Torres Strait Islander people have largely used Android or iOS apps [15,38-41]. Of these studies, 1 was unable to collect use data for 34% (21/61) of the participants reporting flat batteries, connectivity issues, and other problems [41]. A second study reported technical difficulties for participants with using the “challenge” function and signing in and out, although it was noted that technical challenges did not significantly impact the use for many participants [15]. In a third study, an app was used by clients with a practitioner present, and technical difficulties were reported with an Android emulator to enable compatibility with Windows [40]. The fourth study used the same app as the previous study [40] with a different population; thus, it was also used by clients with a practitioner present, but no technical difficulties were reported [39]. Although many studies evaluating mHealth apps have reported technical challenges, it seems that using a web-based application may have resulted in more users experiencing technical challenges and a more substantial challenge of not being able to log in or save the application. With a large proportion of women having had difficulty accessing the application, most of the feedback on the application was centered on the technical challenges, and there was limited feedback on the content. However, it was useful to discover during this early phase that a web-based application is not feasible. An Android or iOS app will need to be considered before further evaluation with a small group of end users to provide detailed feedback on content and navigation.

In addition to considering the technical barriers to accessing applications, careful consideration must be given to long-term engagement [16]. In a longitudinal study examining the reasons for continued use of mHealth apps, two connected factors were described: (1) users’ assessment of the mHealth app (related to the technology and content) and (2) users’ persistence of health goals (ie, those who have higher persistence toward reaching their goals appear to have longer engagement with health apps) [16]. The authors concluded that long-term engagement with health apps occurs when there is high user assessment and high persistence toward health goals [16]. With improvements to the technical aspects of the Growin’ Up Healthy Jarjums application, we expect to see improvements in initial access and a small increase in long-term engagement, although it is evident that an application is not going to be engaging to all users. Health applications are suggested to be most engaging for users who are younger, are more educated, and have higher levels of eHealth literacy skills [42]. It is also suggested that the use of health applications can improve when support from a clinician or another medical professional is provided [43]. The findings from this pilot would also suggest that those experiencing distressing life situations may find it difficult to engage with a health application, similar to the findings that suggest mental health applications may be more suitable for those with less severe illness [43]. As health applications are likely to continue to improve as technology continually does, engagement with health applications will also improve. However, at this point in time, it seems apparent that mHealth tools should be provided in a range of delivery modes to increase reach, digital inclusivity, and equity.

One such delivery mode is social media. It has been established that Aboriginal and Torres Strait Islander people are avid users of social media, Facebook in particular [8-11], which is a key reason why a Facebook page was part of the Growin’ Up Healthy Jarjums program. The qualitative findings from our pilot suggest that the Facebook page had high acceptability. The women commonly reported that they valued the connection and seeing what other families were doing. It can be difficult to track engagement with Facebook using objective data because of privacy measures and the complexity of identifying whether users accessed the page as “observers,” rather than more active users, which can be done only by examining page likes, comments, shares, etc [10]. A qualitative study examining social media and health information sharing among Aboriginal and Torres Strait Islander people shed more light on how social media are used for health promotion by identifying six typologies: (1) observer, (2) post sharer, (3) positive supporter, (4) educator, (5) expert, and (6) influencer [10]. Although we do not have the data to compare all typologies with the previous study, our results indicate that mothers were more likely to be “observers,” with many women reporting the value of connection and seeing what other families were doing but not often commenting, liking, or sharing posts during the study. Posts that were shared or commented on were more likely to have been posts uploaded by Aboriginal organizations, posts affirming Aboriginal culture, or posts about competitions where a prize could be won. The previous study used a methodology different from the one used in this study, wherein they had participants monitor their social media accounts for health-related content and conducted weekly interviews to explore perspectives and actions on posts. Interestingly, the authors found that users moved between typologies depending on the health topic and how information was provided [10]. In future research on the Growin’ Up Healthy Jarjums Facebook page, it may be useful to use a similar methodology with a subset of participants to better understand what and how health information is shared among mothers with young children, as well as how this correlates with changes in health literacy offline; given the high acceptability of and engagement with Facebook among this group of end users, Facebook has great potential to improve health literacy.

Limitations

A limitation of this study was the need to conduct all recruitment and instruction of the program remotely using SMS text messages, links, and phone calls owing to COVID-19 restrictions. In the initial protocol, we proposed recruiting women, setting up the app, and providing instructions on how to use the application in person to reduce technical problems. Unfortunately, this was not possible, and women experienced a high number of technical problems. With a large proportion of women having had difficulty accessing the application, most of the feedback on the application was centered on the technical challenges, and there was limited feedback on the content; however, it was useful to discover during this early phase that a web-based application is not feasible. An initial in-person setup would be considered important for further use of the program.

Another limitation of this study is that generalization to other communities is limited. Aboriginal and Torres Strait Islander communities are made up of many diverse cultural and language groups [1]. Each community has a unique history, cultural practices, and health needs. The Growin’ Up Healthy Jarjums program would need to be adapted, including by making changes to language, images, and health advice, to ensure cultural safety and relevance to women from other communities.

Conclusions

This study demonstrates that the Growin’ Up Healthy Jarjums program was perceived as useful and culturally appropriate by users and health professionals. The SMS text messages had the highest engagement, followed by the Facebook page and then the application. This study identified suggestions for improving the application. A trial is needed to assess the effectiveness of the Growin’ Up Healthy Jarjums program at improving health outcomes.