Globally, more than 3 million children are currently living with the autoimmune disorder juvenile idiopathic arthritis (JIA) [1,2]. JIA is the broad term used to describe a heterogeneous group of 7 inflammatory disorders, all of which have an unknown origin that begins in children younger than 18 years [3,4]. The commonality between these disorders is joint inflammation and pain [5,6]. Prolonged exposure to this inflammation can cause serious widespread complications for a child, such as impaired growth [7,8], delayed pubertal development [8,9], premature cardiovascular disease [10], and organ damage [11]. Ongoing exposure to pain can induce permanent changes in the nervous system, increasing pain sensitivity and the continuance of persistent pain in adulthood [6,12]. Regrettably, there is no cure. Instead, a “treat to target“ approach is used to induce clinical remission or lower disease activity [13]. The aim is to normalize and preserve joint function, prevent growth retardation, maintain physical function, and avoid permanent disability [7,14], ultimately improving the child’s quality of life [15].

The aim of this study is to evaluate the usability and acceptability of InteractiveClinics among children (aged 10‐18 years) by determining if the intervention is (1) easy to use, (2) acceptable, and (3) useful [57,58].

This study was part of a cross-sectional study, following the World Health Organization’s 6-stage step-up approach to develop a digital health intervention. These steps start from monitoring the intervention’s functionality and fidelity through to evaluating feasibility, usability, efficacy, and effectiveness [58], allowing improvements to be put into place after each stage of testing and improving the quality of the intervention [59].

This study focuses on evaluating InteractiveClinics’ usability to support the development of a user-centered design, because the success of an intervention is dependent on whether intended end users engage with the intervention [58]. For 2 weeks, children and parents gained hands-on experience using InteractiveClinics and then completed a postintervention survey. To ensure detailed analysis, survey questions were based on a quantitative descriptive and qualitative design, to invite participants to answer questions in their own words [60].

As part of this pediatric cross-sectional study, children were recruited through 2 pediatric rheumatology outpatient clinics within 2 tertiary children’s hospitals in Australia. The eligibility criteria included an age range of 10 to 18 years, a diagnosis of JIA, and good comprehension of the English language. The exclusion criteria included a cognitive impairment, physical disability, or visual impairment that would affect the child’s ability to understand or use smart technology.

Ethics approval for this study was granted by the Hunter New England Research Ethics Committee (approval no: 2019/ETH01035). To ensure informed consent, all potential participants were provided with an information sheet, explaining the study’s purpose, expectations, and how all data collected will be deidentified. Additionally, all study participants were informed that they could withdraw from the study at any time, without discrimination, by simply not completing the survey. In total, 12 children and 12 parents agreed to participate in the study, provided signed consent, and completed the anonymous survey between September and November 2022 (Table 1).

InteractiveClinics aims to motivate children to take their medication, record their pain, and increase their participation in physical activity. Personalized notifications were sent daily to the smartwatch and phone, to prompt and record medication adherence (Figure 1) and complete a pain level assessment within the app (Figure 2). Physical activity was automatically recorded by simply wearing the watch.

InteractiveClinics presents these 3 key areas of monitoring as 3 modules—medication adherence, pain level, and physical activity level—which can be monitored daily, weekly, or monthly by the child within the app (Figures 1 and 3) or on a secure, password-locked, web-based platform by the child, parents/caregiver, pediatric rheumatologist, and health care team (Figure 4).

Pain levels were recorded on the validated electronic visual analog scale (eVas) module [61-63]. eVas uses a simple horizontal line with defined pain limits. The left end point indicates “without pain,” and the right end point indicates “worst possible pain” (Figure 2). This reporting scale has been found to be highly reliable and consistent with the original paper-based visual analog scale [61-63].

The materials supplied to support participation in the intervention were as follows: a smartwatch (Apple Watch series 3, with a water-resistance rating of 50 meters, to support low-intensity activities such as swimming [64]), a smartphone (Apple iPhone, SE, 2016, loaded with Aus $30 [US $20.01] worth of pre-paid credit), the InteractiveClinics app (preset on the iPhone), and a personal password (to access the data collected on the secure web-based platform).

The persuasive influences used to promote the adoption of the intervention were as follows [43]: training (for 15 minutes on how to use the smartwatch, InteractiveClinics app, and web-based platform), an instruction manual, ongoing technical support, ongoing human communication (researcher contact details [SB] were placed in the phone), goals set by the user (for the physical activity module), personal reminders (daily personal notification sent to the watch and phone, at a time preselected by child, to prompt and record medication administration and pain level assessment), and a reactive feedback loop (for the physical activity module).

A more detailed description of InteractiveClinics has been provided in an earlier feasibility study [59].

InteractiveClinics’ usability and acceptability were measured through a postintervention survey. Two surveys were developed to collect feedback from children and their parents after they used the digital intervention for 2 weeks. Survey questions were adapted from the System Usability Scale [65,66], to ensure the survey questions encompassed the unique multimodal approach being used by InteractiveClinics. Further, to enable younger children to complete the survey without help from their parents, age-appropriate language was used.

For the parents, survey responses were based on the 1-to-5 Likert scale (1=strongly disagree, 2=disagree, 3=unsure, 4=agree, and 5=strongly agree). For children, survey responses were based on the visual face scale [66].

Written comments were also encouraged by children and parents throughout the survey, in a free-text format, to identify any unmet inclusions; to illustrate important points; and, importantly, to facilitate a user-friendly design.

The response rate to the survey among children was 100% (12/12). Most children (9/12, 75%) also completed the survey without any help from their parents. The results of the quantitative section of the survey are presented in Table 2.

The response rate to the survey among parents was 100% (12/12). The results of the quantitative section of the survey are presented in Table 3.

Overall, most children found all the information on InteractiveClinics easy to understand (9/12, 75%). However, only 5 children reported that the information collected met their needs (5/12, 41.7%), and 4 children reported that it would support their doctor with their care (4/12, 33.3%).

In contrast, most parents felt InteractiveClinics was appropriate to address their child’s needs (10/12, 83.3%). They found the information useful (8/12, 66.7%) in supporting their understanding of the effects JIA had on their child (8/12, 66.7%) and would use InteractiveClinics again (6/12, 50%). Importantly, 7 parents reported that InteractiveClinics would support their child in their next pediatric rheumatology consultation (7/10, 58.3%).

Summing up the feedback on all key areas in InteractiveClinics, participants wanted a unique in-app experience (Figure 5).

This study sought to evaluate the usability and acceptability of InteractiveClinics for children (aged 10 to 18 years) living with JIA. A unique feature of InteractiveClinics is the multimodal approach it uses to support the 3 key management areas in JIA treatment: medication adherence, pain, and the retention of physical activity. InteractiveClinics was supported through a commercial smartwatch, customized phone app, and web-based platform.

This study exposed both the strengths and weaknesses of InteractiveClinics. Most children and their parents liked the watch and web-based platform, finding it easy to learn and simple to use. They were satisfied with the pain and physical activity module, and no adverse events were reported. However, there was also the need for improvements. Children identified usability and acceptability barriers that hindered uptake, particularly in the phone app. This finding is important because user feedback can be vastly different from how the intervention was planned to be used [70]. To improve InteractiveClinics, children wanted a unique in-app experience through more personalization, simplification, flexibility, conferment, comment fields, graphical illustrations, a breakdown of metrics, and interconnections.

The importance of personalization is that it provides children with choice, autonomy, and ownership when using technology [71]. This is achieved by enabling children to customize the app features to suit their needs [72] and providing more feedback, on an individual level [73].

Parents also requested more simplification by removing all the buttons that were not relevant to their child. Other studies have also requested removing any complexities. DeForte et al [74], for example, reported children only wanted simple buttons. This would also support a child with a disability or low level of literacy, improving their ability to navigate the app [75].

In addition, the app needed to be more flexible. Children wanted to be able to change their medication response times within the app, earlier or later than the previously selected notification times given to the research team (SB). Rather than being reliant on the research team to update their medications, they wanted to adjust the response times to work around their activities. Interestingly, a systematic review of medication apps for adults, reported that the main components that foster medication adherence were reminders and importantly, medication-tracking histories [76]. Therefore, future changes will be made to InteractiveClinics to provide users control over their notification reminders, to ensure an accurate medication history is recorded.

Further, children wanted more from the pain module; instead of the numerical scale confirming their pain recording, they wanted more exciting conferment. In contrast, children with cancer pain in a phone app study requested the numerical scale instead of the pediatric face scale using smiley faces [77]. Therefore, it was not surprising then when children suggested more engaging gamification. Yet, very few studies have considered gamification mechanics as part of their pain assessment, despite the evidence suggesting they are liked and can increase motivation [78]. Indeed, this is certainly an area in need of further research.

There was also an overwhelming response from parents to include comment fields to record additional information such as medication-induced side effects. This is important because a recent study identified that two-thirds of children with JIA, within 1 year of diagnosis, experience side effects that have an impact on their life [79]. Understandably, concerns about these side effects can result in the termination of their treatment, posing the risk of increasing disease activity [80]. Therefore, it is imperative for InteractiveClinics to collect detailed input from children on what occurs between pediatric rheumatology appointments, to gain a more comprehensive overview of disease activity and adverse events [81], enabling timely re-evaluation and mitigation strategies to be put into place, rather than merely relying on the assessment of pain and active joint count at each visit [82]. Permitting suitable changes to treatment such as prescribing folate to reduce methotrexate-induced nausea [17,83,84] or brain fog [85], changing the route of administration [17,84], splitting doses, or altering the rate of absorbency [20] can reduce the risk of disease burden and further polypharmacy in adulthood [19].

For the physical activity module, children reported wanting more graphical illustrations representing their physical activity. According to the World Health Organization, children between the ages of 10 to 18 years are in a phase of life between childhood and adulthood where health behavior can be greatly influenced. Therefore, they are in need of age-appropriate comprehensive information [86], as they are active agents in their own development [87]. A similar study that used an activity tracker, app, and web-based profile also reported the need for more detailed feedback. Children, instead of a similar traffic light system, wanted the see the actual step count [88] (in other words, the breakdown of metrics).

To improve the app and web-based platforms’ usefulness, children also wanted to see interconnections between the medication adherence module, pain level module, and physical activity module to, for example, determine the impact pain or medications may have on physical activity levels. This is important because each of these modifiable risk factors can affect another [15,49]. Demonstrating a correlation between this information could also be useful for the pediatric rheumatology team to better understand disease progression or remission, therefore facilitating adjustments to treatment accordingly, and also to avoid the impact JIA can have on development, physical function, and health-related quality of life [15,49].

An important clinical finding in this study was parental endorsement. Parents, in contrast to children, felt InteractiveClinics was useful in addressing their child’s problems by helping them understand the effects JIA had on their children. This is an important finding because parents’ narratives of their child’s condition often remain unmet [89]. By gaining self-awareness, the parent may gain a more insightful understanding of their child’s status on treatment that may be interfering with their progress [89]. In addition, parents then examine their own views and behaviors that can also contribute either positively or negatively to their child’s health outcomes, because children are observational learners [90]. For example, a parent’s fear and catastrophizing can result in protective behaviors and avoidance of treatments, therefore impairing their child’s functional ability [90], suggesting the importance of digital health care in educating and empowering parents.

Overall, the children and parents included in this study, through their own lived experience, were incredible collaborators in improving the usability of InteractiveClinics, greatly extending our understanding of the unique needs of children with JIA. This is important, as there is criticism toward the current “treat to target” approach used in JIA management. Children and their parents are often not included when formulating treatment plans [82]. Yet, their goals of treatment are often different from those of the Pediatric Rheumatology teams because they are focused more on the present, rather than the long impact of the disease [82], emphasizing the importance digital health care can have in supporting child-centered and family-centered care [91].

There are several limitations that need to be considered when interpreting this study’s findings. First, this study only recruited a small convenience sample of 12 children and 12 parents, limiting generalizability. Although a small sample size (>10 participants) is typically used for usability testing[58], the suggestive inclusion to improve InteractiveClinics may not be representative of all children with JIA. Therefore, to overcome sampling bias, a larger, more diverse participant sample is needed across different demographic and geographical locations in the next stage of testing.

Also, due to the nature of this usability and acceptability study, consenting participants were actively prompted to be critical and provide written feedback on how the proposed digital health intervention could be improved. This may have limited the provision of any positive attributes.

In addition, the written feedback they provided may have not been as detailed as expected for thematic analysis. Patterns did emerge, and data saturation was reached; however, there was still a risk of research bias [69], so all authors internally reviewed and rereviewed emerging codes and themes against the original text during all stages of analysis.

Further research is now needed to examine the potential challenges and limitations of incorporating InteractiveClinics into clinical practice. First, this research needs to focus on device access and digital literacy, because this study supplied all the equipment needed to participate and provided ongoing technical support. Second, the level of engagement and length of adherence among participants using the intervention needs to be considered, to clearly understand whether digital health care is an effective and sustainable intervention to support chronic disease management for children and their families.

A comparative analysis is also needed to compare InteractiveClinics to other digital health interventions targeting pediatric chronic disease. This would help with positioning the intervention within the broader digital health landscape and identifying any unique benefits that may be offered over existing tools, in order to find supportive and effective digital health solutions.

Interestingly, for JIA, 2 recent systematic reviews identified no similar digital health interventions that have used a multimodal approach to support chronic disease management [43,92]. In fact, most interventions were still at an early stage of development [43], and heterogeneity exists, making it difficult to compare their effectiveness [92]. Instead, the findings of these reviews helped to identify 3 specific areas that are needed in JIA management—symptom monitoring, physical activity promotion, and self-management development [92]—which were used to support the development of the 3 modules included in InteractiveClinics: pain level, physical activity level, and medication adherence. It is also important to note that no studies directly targeted medication adherence [43,92], yet early aggressive pharmacological treatment and the monitoring of side effects are keystones in JIA treatment [91].

Now, the future direction of InteractiveClinics is to use the feedback gained from the children and parents within this study to improve the usability of the intervention. The World Health Organization clearly reinforces the importance of doing this before commencing costly trials [58]. Then, the next step is to test the intervention’s efficacy and effectiveness [58]. This will be achieved through conducting a pilot randomized controlled trial, which will remove the sampling bias identified in this study and determine the intervention’s effectiveness on health outcomes for children with JIA and children living with other chronic conditions.

Most children and their parents liked using the smartwatch and web-based platform; they found it easy to learn and simple to use. They were also satisfied with the pain and physical activity modules. However, usability and acceptability barriers that hindered uptake were identified in the phone app and medication module. Children sought a unique in-app experience, and their suggestive improvements included more personalization within the app; simplification by removing all nonrelevant links; flexibility in response times; improved conferment through gamification; additional comment fields for the input of more data, such as medication side effects or pain-related symptoms; more detailed graphical illustrations of the physical activity module, including a breakdown of metrics; and importantly, interconnections between the modules, because medication adherence, pain levels, and physical activity can each influence the other. Further research is now needed to ensure these inclusions are combined with standardized comprehensive assessments and evidence-based behavior change strategies to promote user adoption and advancement of new, validated digital health interventions in pediatric rheumatology clinical care.