This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Pediatrics and Parenting, is properly cited. The complete bibliographic information, a link to the original publication on http://pediatrics.jmir.org, as well as this copyright and license information must be included.
Adolescents and young adults with spina bifida (AYA-SBs) have unique user needs, given their variable and complex symptom profile. Owing to multiple barriers to prevention and intervention treatments for secondary conditions (eg, obesity), AYA-SBs may benefit from the use of behavioral intervention technologies (BITs). However, as BITs are often designed and tested with typically developing individuals, it is unclear if existing BITs may be usable for AYA-SBs.
This study aimed to evaluate the usability of a high-quality, publicly available, weight management–focused mobile BIT (smartphone app) for AYA-SBs.
Overall, 28 AYA-SBs attending a Young Men’s Christian Association–based summer camp completed 4 structured usability tasks using a weight management app designed for the general public called My Diet Coach (Bending Spoons). Learnability was measured by (1) time to complete task, (2) number of user errors, and (3) correct entry of data when requested by the app. Satisfaction and general usability were measured via self-reported questionnaires and qualitative feedback following interactions with the app.
The majority of the sample were able to complete the tasks, with increased completion rates and improved times on second attempts of the tasks (
AYA-SBs were able to learn how to complete specific tasks independently on a weight management app, but design changes consistent with previously proposed user needs were recommended. Rather than designing entirely new BITs, it may be possible to adapt existing technologies to personalize BITs for specific populations such as AYA-SBs.
Spina bifida (SB) is the most common congenital birth defect affecting the central nervous system and requires the management of both a complex medical treatment regimen and a variety of cognitive and psychosocial comorbidities [
Multiple emotional (eg, low motivation and depressive symptoms) and practical barriers (eg, ambulation status and transportation) to addressing obesity and healthy lifestyles have been identified for AYA-SBs [
Given their unique user needs, calls for personalizing BITs for people with disabilities have been made [
To accomplish the task of evaluating how well currently available BITs may serve this population, usability testing was utilized. Usability testing is the systematic observation of planned tasks by potential end users to improve the design of a product or technology [
The purpose of this study was to evaluate the usability attribute of learnability for a high-quality [
Participants were recruited from the Young Men’s Christian Association (YMCA)-sponsored Camp Independence during the 2018 summer sessions. Camp Independence is located in Illinois (USA) and is a sleep-away camp designed for AYA-SBs. Programming includes (1) a 1-week stay with similarly aged campers, (2) typical camp-based activities (eg, swimming) with accommodations for camper needs, and (3) camp-based interventions to promote medical and social independence [
The app that was selected for usability testing was the My Diet Coach app. My Diet Coach was selected as it is (1) highly rated for quality using the Mobile App Rating Scale [
This study was approved by the Loyola University Chicago institutional review board. Participants aged 18 years and older provided informed consent. Participants younger than 18 years provided informed assent, and their parents provided consent.
To assess the usability attribute of learnability, the participants were asked to complete specific tasks (eg, log a food item) with an app on a mobile device. These tasks were related to food intake and activity monitoring, which were selected because (1) they are common tasks for weight loss and management and (2) they could be evaluated relatively quickly, so as to not keep participants from their camp activities. Learnability was measured by (1) time to complete task, (2) number of user errors (tracked on a standardized paper measure by the moderator), and (3) correct entry of data when requested by the app. Improvements across these measurements were hypothesized to occur across attempts (eg, logging a food item attempt 1 versus attempt 2). Satisfaction and general usability were measured via self-reported questionnaires (please see Measures section).
Participants were brought to a private area of the main camp building to complete testing. Testing was conducted by research and graduate assistants with experience in interviewing AYA-SBs. A think-aloud protocol was used [
Now, please imagine that you want to eat healthier. You see this description of My Diet Coach on the Google Play store and decide to download it. Let’s pretend that you just ate one piece of pepperoni pizza for lunch. Please open the app, which is already on the home page, and select “the lightning bolt” to log the pizza you just ate. Feel free to say out loud what you are thinking while you do this. Tell me when you are finished.
This interaction was timed and audiotaped, and any observed user errors or alternative paths to complete the task were noted by the research or graduate assistant. If a participant stopped working on the task, they were prompted with “What’s going through your mind right now?”, followed by “Please do your best to complete the task and let me know when you are finished.” To avoid participant frustration, the task was ended if a user stopped working for 90 consecutive seconds. Once the task was completed, participants were asked to share their thinking with regard to alternative paths taken to complete the task. Participants completed 3 more tasks following this same methodology: (1) exercise (ie, entering a 30-min activity of their choice), (2) second food (ie, entering in eating an apple for a snack), and (3) second exercise (ie, entering a different 15-min activity of their choice). Finally, participants were asked open-ended questions about (1) their impressions of the app (eg, “What are your overall impressions of the logging features of My Diet Coach?” and “Is there anything that you feel is missing?”) and (2) the designs of technology more generally for AYA-SBs (eg, “How could technology work better for you in terms of managing your health?”).
The following traditional data collection methodologies that have been used in the testing of other apps [
All measures were administered following completion of the interactions with My Diet Coach. Participants were given the opportunity to answer questionnaires via paper and pencil or via an electronic version administered through Opinio [
Participants were asked to report the following information: age, sex, race/ethnicity, and SB characteristics, including type, shunt status, and lesion level. Full Scale Intelligence Quotient (FSIQ) was measured and collected for those who also participated in another camp-based study [
The System Usability Scale (SUS) is a 10-item self-reported instrument measuring a user’s rating of a product’s usability [
The After-Scenario Questionnaire (ASQ) is a 3-item self-reported instrument measuring a user’s satisfaction with a product [
The Health Questionnaire is a modified and abbreviated (17 out of 87 original items) version of the 1999 Youth Risk Behavior Survey by the Centers for Disease Control and Prevention (CDC) [
The demographic and SB characteristics (eg, type of SB), usability testing measurements (eg, time to complete and number of user errors), and questionnaire data were analyzed using descriptive statistics. The
A total of 29 participants agreed to participate; however, 1 participant only completed questionnaires as she reported feeling too overwhelmed from the card sorting task [
Demographic, spina bifida, and health characteristics (N=29).
Characteristics | Adolescents and young adults with spina bifida | |
Age (years), mean (SD); range | 18.11 (4.55); 13-30 | |
|
||
|
Male | 12 (41.4) |
|
Female | 17 (58.6) |
|
||
|
African American | 2 (6.9) |
|
Asian | 2 (6.9) |
|
Caucasian | 21 (72.4) |
|
Hispanic | 4 (13.8) |
|
Other | —a |
|
||
|
Myelomeningocele | 20 (69.0) |
|
Other | 9 (31.0) |
|
Meningocele | — |
|
Lipomeningocele | 1 (3.4) |
|
Occulta | — |
|
Unsure | 8 (27.6) |
|
||
|
Thoracic | 1 (3.4) |
|
Lumbar | 12 (41.4) |
|
Sacral | 4 (13.8) |
|
Unsure | 12 (41.4) |
Shunt present, n (%) | 23 (79.3) | |
Full Scale Intelligence Quotientb, mean (SD); range | 84.67 (19.51); 55-132 | |
|
||
|
Android | 10 (34.5) |
|
iOS | 18 (62.1) |
|
Did not report | 1 (3.4) |
Body mass indexc, mean (SD); range | 21.84 (4.19); 14.50-32.00 | |
|
||
|
Gain | 6 (21.4) |
|
Lose | 11 (39.3) |
|
Maintain | 9 (32.1) |
|
No attempts to change | 2 (7.1) |
aNot applicable.
bData missing for 5 participants because of not participating in the larger camp-related intervention.
cData missing for 7 participants because of not responding about height and/or weight.
To contextualize the usability outcomes within the sample’s health behaviors, participants reported their current frequency of healthy food consumption, physical activity, screen time, and sleep.
Health behavior frequencies.
Frequency of daily healthy food intake and screen time | No times | 1-3 times in past 7 days | 4-6 times in past 7 days | Once per day | Twice per day | 3 times per day | >4 times per day |
100% fruit juicea (%) | 17.9 | 35.7 | 10.7 | 21.4 | 7.1 | 7.1 | 0 |
Fruit (%) | 17.9 | 28.6 | 17.9 | 17.9 | 17.9 | 0 | 0 |
Vegetables (%) | 14.3 | 32.1 | 21.4 | 10.7 | 3.6 | 0 | 17.9 |
Milk (%) | 25.0 | 17.9 | 14.3 | 17.9 | 14.3 | 3.6 | 7.1 |
aFrequencies reported from adapted version of the Youth Risk Behavior Surveillance System, as reported in the study by Kolbe et al [
Frequency of physical activity.
Physical activity | 0 days | 1 day | 2 days | 3 days | 4 days | 5 days | 6 days | 7 days |
≥30 min of strenuous exercisea (%) | 7.1% | 10.7 | 14.3 | 17.9 | 10.7 | 14.3 | 3.6 | 21.4 |
≥30 min of nonstrenuous exercise (%) | 21.4 | 14.3 | 17.9 | 10.7 | 14.3 | 3.6 | 3.6 | 14.3 |
Strength exercises (%) | 53.6 | 7.1 | 14.3 | 3.6 | 10.7 | 3.6 | 3.6 | 3.6 |
aFrequencies reported from adapted version of the Youth Risk Behavior Surveillance System, as reported in the study by Kolbe et al [
Sleep quality.
Ability to fall asleep | Not at all | Very little | Moderately often | Very often | Almost always |
Trouble falling asleepa (%) | 26.9 | 42.3 | 15.4 | 7.7 | 7.7 |
Trouble staying asleepa (%) | 50.0 | 34.6 | 11.5 | 3.8 | 1.2 |
aTwo participants did not report their sleep information.
The majority of the sample reported using iOS (62%, 18/28) for their personal mobile devices; however, most of them denied using apps to help manage their SB in any way (75%, 21/28). Most participants completed the usability tasks on the Android mobile phone (86%, 24/28); however, 4 participants requested to use the iPad to complete testing because of vision or motor dexterity issues. Given the small sample size and uneven number in the groups, Mann-Whitney U tests were run to compare those who reported owning an iPhone with those who reported owning an Android on the task measurements (ie, completion and error rates and time to complete tasks) to ensure that the device platform did not impact the results. There was no evidence to suggest significant differences between those who owned an iPhone and an Android (
About two-thirds of participants (n=20) were able to independently complete a food entry on their first attempt (ie, pizza), with an increase in completion on the second attempt (ie, apple; n=22). For activity entries, about two-thirds of participants (n=20) were able to independently complete an entry on the first attempt, with another increase in completion on the second attempt (n=24). Significance testing was not performed on the completion rates because of lower than expected counts in at least one cell of the contingency table for both activities. The activity entries could be selected by the user, with the most frequently chosen activities being lifting weights, sled hockey, and walking/running. Most participants (86%) were able to come up with their own activities without suggestions from the research or graduate assistant moderating the sessions.
Participants significantly decreased their time to complete tasks across the 2 attempts for both food entries (mean time 2:00, SD 1:38 vs mean 1:02, SD :56;
The most common user errors involved (1) entering incomplete or inaccurate data (16 out of 112 attempted tasks, 14.3%), (2) being unsure of how to proceed to the next step without being able to recover and complete the task (ie, a
Participants completed the SUS and ASQ following their interactions with My Diet Coach to evaluate usability and satisfaction, respectively. SUS ratings for My Diet Coach were highly variable (range 2.50-100). The average SUS rating was 64.17 (SD 29.59; a
Given the wide range of AYA-SB participants (13-30), sensitivity analyses were conducted to explore any differences that were driven by age. Specifically, exploratory chi-square and
Although the usability tasks were audiotaped, the majority of participants did not engage in the suggested
Despite the variable usability ratings, the majority of participants stated that My Diet Coach would be useful for them (64%, 18/28). When queried as to why this app might be useful, the most common response was related to the app reminding and encouraging them to change their eating and activity choices. For example:
I could see how many calories I'm burning and consuming, and balance those. I definitely see me using in the future.
[This app] would help me make healthy choices.
It would help me realize I need more fruits and vegetables.
Although the current usability testing evaluated initial learnability, qualitative feedback also suggested that learnability might improve with long-term use (eg, “It was a little tricky at first. As I used it more, it became easier to use.” [13-year-old male]; “It seems pretty easy to use once you get used to it.” [18-year-old female]). Suggested improvements specific to AYA-SBs included (1) tutorials (eg, “Make it more self-explanatory. Have practice stuff, give step-by-step directions to enter stuff.” [14-year-old female] and “There’s a lot of information in it with no instructions.” [20-year-old female]), (2) fewer logging options and/or simplifying the design throughout (eg, “It’s too confusing with too many steps” [17-year-old female] and “Make it easier by not making it so heavy in content and choices.” [19-year-old female]), (3) more activity options for those who ambulate by wheelchair (eg, “It needs to add exercise for people in wheelchairs.” [23-year-old male]), and (4) notifications to prompt use (eg, “I need reminders [to do this] on my phone…Remind me to do my exercises and eat healthy.” [20-year-old female]).
The purpose of this study was to evaluate the usability attribute of learnability for My Diet Coach, a publicly available, high-quality app [
A user needs model for AYA-SBs was recently created for BITs aiming to improve self-management [
Although the current findings suggest that My Diet Coach is learnable for AYA-SBs, it also implicates personalizing existing elements and characteristics for AYA-SBs and related users (eg, youth with physical and/or motor disabilities) [
Given the variable levels of impairment in motor dexterity, coordination, hearing, vision, and visuospatial processing in people with SB [
The majority of participants were able to complete the tasks and stated that the app would be personally useful. However, questionnaire responses were not indicative of high usability and satisfaction, and the majority of the sample reported that they do not use apps to manage their SB (75%). These discrepancies may have multiple explanations. First, young people with SB have variable cognitive profiles, with many falling within the category of having a mild intellectual disability [
This study builds upon previous work establishing the importance and feasibility of conducting usability testing with AYA-SBs [
For clinicians and engineers designing BITs for pediatric and AYA populations, the findings of this study suggest that it may be possible to iterate from existing technologies to personalize apps for specific populations. Doing so may decrease both the financial and time burden associated with designing and building a new technology. However, the use of appropriate user-centered design principles and the use of formative usability testing is still critical [
After-Scenario Questionnaire
adolescents and young adults
adolescents and young adults with spina bifida
behavioral intervention technology
body mass index
Centers for Disease Control and Prevention
Full Scale Intelligence Quotient
spina bifida
System Usability Scale
Young Men’s Christian Association
This research was supported in part by a Research Support Grant from Loyola University Chicago and grants from the Kiwanis Neuroscience Research Foundation: Illinois-Eastern Iowa District of Kiwanis International, National Institute of Nursing Research and the Office of Behavioral and Social Sciences Research (R01 NR016235), National Institute of Child Health and Human Development (R01 HD048629), and the March of Dimes Birth Defects Foundation (12-FY13-271). The authors thank the YMCA-sponsored Camp Independence and the Chicago Healthy Adolescent Transition Study (CHATS) research team, without whom this research would not have been possible.
None declared.