Electronic Medical Record Data Missingness and Interruption in Antiretroviral Therapy Among Adults and Children Living With HIV in Haiti: Retrospective Longitudinal Study

Introduction

Despite improvements in HIV testing, care, and treatment and reduced HIV incidence over the last 3 decades, Haiti has the largest population of people living with HIV in the Caribbean, with an estimated 1.8% of the population (150,000 persons) having received a positive HIV diagnosis, including nearly 6000 children (aged 0-14 years) living with HIV [1]. Children living with HIV often have lower rates of HIV diagnosis, treatment, and viral load suppression [2]. In Haiti, children living with HIV fare worse across all steps of the care cascade, with only 63% knowing their HIV status (compared to 85% overall), 63% on treatment (compared to 85% overall), and 48% virally suppressed (compared to 73% overall) [1]. Reviews of patient records in Haiti revealed that children living with HIV were significantly less likely to initiate antiretroviral therapy (ART) in a timely manner as compared to adults, and once initiated, were less likely than adults to be retained in ART treatment [3,4].

Electronic medical records (EMRs) can improve HIV patient care and outcomes in multiple ways, including (1) directly informing individual patient care, such as tracking clinical outcomes, ART adherence and retention, as well as patient follow-up; and (2) promoting provider compliance with treatment and care guidelines [5-13]. However, these benefits are largely dependent on providers having access to high-quality data (ie, reliable, timely, and nonmissing data) [14-16]. In the context of EMRs, data missingness is both an element of quality of care (vis-à-vis noncompliance with reporting guidelines) and can itself lead to lower quality of care, as missing data cannot be used to inform clinical decision-making [17]. However, despite the importance of data quality in the value proposition of EMRs, the evidence base exploring the association between data missingness and patient outcomes is limited, especially in resource-limited settings. Although many studies of EMRs include both data quality and patient outcomes as indicators of interest, a direct association between the two is rarely assessed. In addition, no studies reviewed for this paper assessed this relationship by age cohort.

We hypothesize that data missingness will be associated with greater interruption in treatment (IIT) and that this relationship may be larger among children living with HIV. We used ART patient data extracted from the iSanté EMR system to assess the association between age group, data missingness, and IIT.

Methods

Study Design

This was a retrospective longitudinal study using patient-level routine EMR data.

Data Source

We used patient-level clinical and pharmacy data extracted from iSanté—the most widely used EMR in Haiti, which covers over 1.8 million primary care patients and more than 200,000 unique records for people living with HIV [18,19]. iSanté records include data on key HIV care cascade processes (eg, clinical history) and electronic pharmacy data (eg, ART dispensing and continuation).

Sample

This analysis included data from 50 facilities and covered individuals who initiated ART between June 2016 and December 2021. Our analyses had a number of data exclusions. A total of 24 facilities were excluded from the analysis due to being prison-based facilities, having >20% of records entered more than 90 days after the visit date, or having a mean number of prescription records per patient less than 5 (suggesting data record input issues). Data before June 2016 were excluded to account for changes in treatment patterns following adoption of the test-and-treat approach to HIV care in mid-2016. Patient records included in the analysis were restricted to patients who initiated ART at least n+2 months before the data extraction date (end of July 2022) to allow for sufficient follow-up time to observe the outcome and to account for any delay in the entry of patient files, where n refers to the 6- or 12-month IIT outcome (eg, for the 6 months outcome, data were restricted to those who had initiated ART before December 2021). Individuals without date of birth data (n=1174, 3.2%) were excluded as correct age group categorization was essential for the analysis. Additionally, to better assess the relationship between data missingness at intake and IIT status 6 or 12 months after initiating ART, patients who completed their intake visit more than 3 months prior to initiating ART were excluded from the analysis (n=4083, 11.5%). A CONSORT (Consolidated Standards of Reporting Trials) flow diagram can be found in Multimedia Appendix 1 [20].

Conceptual Model

Figure 1 shows our proposed causal model, which is situated within the Donabedian framework [21] for quality of care, as modified by Coyle and Battles [22] to include medical antecedents. The Donabedian framework divides care into 3 primary components: structure (ie, the context in which care is delivered), process (ie, actual service delivery), and outcome. In the context of HIV care and EMRs, these can be understood as the facility or organizational context in which HIV care is delivered as well as the system aspects of EMRs (eg, accessibility and usability); the provision of HIV care, including the use of EMRs to both document and inform care; and HIV outcomes (eg, IIT). The various pathways in our conceptual model were justified through the published literature [2,4-13,23-38].

‎

Analysis

Outcome Variable

The primary outcomes of interest were IIT at the current facility at 6 and 12 months post ART initiation, defined as being more than 28 days late in picking up ART medication as of the dates 6 or 12 months after initiating ART. This definition for IIT status has been used in prior research in Haiti and other settings [39-41].

Covariates

Age groups were categorized as pediatric (0-14 years) and adult (>15 years) as of the time of ART initiation, following the age definition used to define pediatric care in Haiti (<15 years).

Data Missingness

Data missingness was defined as an indicator not being collected during the patient’s intake visit. Assessment of missingness was restricted to indicators that were shared between both pediatric and adult intake forms, indicators that were clinically meaningful for HIV care, and where missingness could be differentiated from the absence of that issue (eg, the headache symptomology field may be missing due to a patient not presenting with a headache or due to the provider failing to document that issue, whereas the World Health Organization [WHO] HIV stage indicator is expected to be completed for all patients) [42]. Within these stipulations, we assessed missingness for weight, current WHO HIV stage, and current tuberculosis (TB) diagnosis. As the importance and impact of missingness for particular indicators may vary by age group (eg, routine documentation of weight is generally of higher importance for pediatric patients), missingness was analyzed as individual binary outcomes (defined as missing or nonmissing) as well as through a composite score indicator (the number of indicators of interest missing), which was analyzed as both a continuous and categorical outcome.

Models

Associations between data missingness (exposure) and interruption in ART treatment at a patient’s current facility (outcome) were assessed through marginal parameters from multilevel modified Poisson models with robust error variances and random intercepts for the facility to account for clustering. Modified Poisson models have been shown to provide unbiased estimates of the risk ratio, important for nonrare binary outcomes where odds ratios estimated through logistic regression will overestimate the risk ratio and potentially lead to improper interpretation of the results [43,44]. Patient sex, facility type, ownership, patient volume, and duration of iSanté use were included as fixed effects to control for potential confounding. Models were stratified by age group to understand the relationship between data missingness and IIT status within each age group. Additional models were run with the age group as an interaction term with the continuous composite indicator to assess the statistical significance of the age group as an effect modifier in the association between missingness and IIT status.

Ethical Considerations

The secondary use of deidentified patient data from the iSanté EMR was approved by the University of Washington Human Subjects Division as nonengaged research (STUDY00016591 “Patient Risk Profiles for Interruption in Treatment among People Living with HIV in Haiti: Leveraging Health Information Systems and Prediction Models to Identify Patients at High Risk”). The research was also reviewed and approved by the Haiti Ministry of Public Health and Population’s National Bioethics Committee (reference number 2223-26).

Results

Primary Findings

In total, data were drawn from 50 facilities across 9 departments (of 10 total) in Haiti and comprised 31,457 patient records for people living with HIV. Of these, 30,151 (95.8%) were adult patients and 1306 (4.2%) were pediatric patients. The majority of patients (n=19,544, 62.1%) were female and received care at health centers (n=19,051, 60.6%) or hospitals (n=9883, 31.4%). The median duration of iSanté use at each health facility was 17.5 (IQR 15.8-18.3) years, and the median monthly patient volume was 348 (IQR 172-544). Table 1 further details participant and facility characteristics.

IIT status and indicator missingness are detailed in Table 2. Across all age groups, the proportion of patients who experienced IIT at 6 and 12 months post ART initiation were 23.8% (n=7477) and 29.3% (n=9222), respectively. Overall, the weight indicator had the highest level of missingness, with 5365 (17.1%) patient records missing weight data, while TB diagnosis had the lowest (n=1417, 4.5%). Both IIT status and data missingness were higher among pediatric patients. Pediatric patients were more likely than adult patients to be IIT at both 6 months (n=431, 33.0% vs n=7046, 23.4%; P<.001) and 12 months (n=551, 42.2% vs n=8671, 28.8%; P<.001). Only 55.5% (n=725) of pediatric patient records had no indicators of interest missing, compared to 74.1% (n=22,339) of adult patient records. Pediatric patient records were also more likely to have at least 3 (3.1%) indicators missing compared to adult records (n=401, 1.3%; P<.001). Variation in missingness across age groups was greatest for the WHO HIV stage, with 32.5% (n=425) of pediatric records missing this indicator compared to 11.1% (n=3355) of adult records (P<.001).

Results from models exploring the association between IIT status and the composite missingness score as a continuous variable are shown in Figure 2 (full multivariable regression results can be found in Multimedia Appendix 2). Statistically significant associations were observed between higher values of the composite missingness score and a greater likelihood of experiencing IIT at both 6 and 12 months among pediatric patients. However, no such association was observed among adult patients for either outcome. Among pediatric patients, each additional indicator missing was associated with a 1.34 times greater likelihood of experiencing IIT at 6 months post ART initiation (95% CI 1.08-1.66; P=.008) and 1.24 times greater likelihood of experiencing IIT at 12 months (95% CI 1.05-1.46; P=.01). Our interaction models (not shown) revealed that the relationship between the composite score indicator and IIT status was statistically significantly larger among pediatric patients compared to adult patients at both 6 months, where pediatric patients had a 25% greater risk of experiencing IIT for each additional missing element compared to adult patients (95% CI 1.02-1.53; P=.03), and 12 months, where pediatric patients had an 18% greater risk of experiencing IIT for each additional missing element compared to adult patients (95% CI 1.01-1.38; P=.04).

‎

We also assessed the composite missingness score as a categorical variable (Figure 3) to understand the estimates of excess risk in the absence of the assumption of a linear relationship between missingness and IIT status (full multivariable regression results can be found in Multimedia Appendix 2). Compared to pediatric patients with 0 missing indicators, pediatric patients with 1, 2, or 3 missing indicators were 1.59 (95% CI 1.26-2.01; P<.001), 1.74 (95% CI 1.02-2.97; P=.04), and 2.25 (95% CI 1.43-3.56; P=.001) times more likely to experience IIT at 6 months, respectively. At 12 months, pediatric patients with 1, 2, or 3 missing indicators were 1.54 (95% CI 1.34-1.78; P<.001), 1.34 (95% CI 0.82-2.20; P=.24), and 1.75 (95% CI 1.08-2.85; P=.02) times more likely to experience IIT, respectively, although this association was no longer significant for those with 2 indicators missing. Among adult patients, compared to patients with 0 indicators missing, having all 3 indicators missing was associated with being 1.32 times more likely to experience IIT at 6 months (95% CI 1.03-1.70; P=.03), while having 3 indicators missing was not associated with IIT at 12 months, and having 1 or 2 indicators missing was not associated with IIT at either 6 or 12 months.

‎

For the individual missingness indicators (Multimedia Appendix 2), only the WHO HIV stage indicator was associated with IIT status among pediatric patients, where pediatric patients with missing WHO HIV stage data on their intake form were 2.17 times more likely to experience IIT at 6 months (95% CI 1.79-2.64; P<.001) and 1.79 times more likely to experience IIT at 12 months (95% CI 1.54-2.08; P<.001), as compared to pediatric patients with nonmissing WHO HIV stage data. Missingness for the WHO stage data among adult patients and missingness for weight and TB status among either age group were not associated with IIT status at either 6 or 12 months.

Sensitivity Analyses

We hypothesized that providers at facilities with a lower proportion of pediatric patients may be less familiar with pediatric care, and therefore, less compliant with treatment and reporting guidelines, which could potentially impact the relationship between data missingness and IIT. However, neither missingness nor IIT status showed a significant association with the proportion of pediatric patients at a given facility, and an interaction model did not show any difference in the relationship between data missingness and IIT status by the proportion of pediatric patients (data are not shown). We additionally explored a more granular definition for age groups 0-9, 10-14, 15-19, 20-24, and >25 years. In age group–stratified models, the association between the continuous composite data missingness score and IIT status at 6 months was only statistically significant for the 0-9 age group, which showed a positive association between greater missingness and likelihood of experiencing IIT (data are not shown).

Discussion

Principal Findings

In this retrospective longitudinal study of patient record data drawn from the iSanté EMR system in Haiti, we found that both data missingness and interruption in ART treatment were higher for pediatric patients compared to adult patients; nearly one-third of pediatric patients had IIT at 6 months compared to just over one-fifth of adults, and nearly half of pediatric patients had missing values for indicators of interest on their intake forms compared to just over one-quarter of adult patients. Data missingness showed a substantial and significant association with greater IIT, with adult patients being 30% more likely and pediatric patients more than twice as likely to have IIT at 6 months when all 3 indicators of interest were missing. The relationship between missingness and IIT status was stronger and more consistent among pediatric patients; pediatric patients showed statistically significantly greater likelihood of experiencing IIT at 6 and 12 months for the composite score indicator both overall (continuous) and across all levels of missingness (categorical), while for adult patients this relationship was only significant at 6 months and for the highest level of missingness in the categorical analysis. Individual indicator missingness showed little association with IIT status, except for the WHO HIV stage among pediatric patients. Within the modified Donabedian quality of care framework, our results show a link between the process of care provision (vis-à-vis compliance with reporting guidelines and data use for clinical decision-making) and patient outcomes (IIT status) after adjusting for structural elements (ie, facility characteristics), with the association being highly dependent on medical care antecedents (ie, patient age group) [21,22].

There is a rich evidence base showing the potential impact of EMR use on HIV service provision and quality of care by promoting adherence to care guidelines, enabling higher quality patient data, improving provider efficiency, and informing patient care, tracking, and follow-up [5-13]. The benefits of EMRs, however, are largely predicated on providers having access to quality data (ie, reliable, timely, and nonmissing) to inform their work, and there is a growing evidence base on the importance and impact of patient record quality (electronic or otherwise) on quality of care, care engagement, or health outcomes [14-16,29,30]. Particularly relevant to this analysis, one study of more than 6000 patient records collected from the National Clinical Audit for Rheumatoid and Early Inflammatory Arthritis found that missing baseline patient data was significantly associated with the odds of timely initiation of treatment being halved [29]. In a qualitative study of health care professionals in South Africa, participants reported viewing data quality as an critical element in the provision of quality health care services, including how poor EMR information integrity can lead “to errors that endanger patient safety or decrease the quality of care” [45]. A systematic review found that data missingness was a commonly cited barrier to the use of EMRs to inform population health efforts [12]. In another systematic review, Albagmi [11] found that EMRs were associated with both better documentation and higher quality of care, although a direct causal relationship between data quality and quality of care was not directly assessed. This limitation is common to much of the literature on EMR data quality; many studies of EMR implementation include both data quality and quality of care indicators as outcomes or indicators of interest, but few directly assess the relationship between data quality and quality of care or patient outcomes. Our results, therefore, contribute to this limited evidence base, providing evidence that EMR data quality is associated with interruption in ART treatment.

Although we have established a temporal sequence for the relationship between data quality and IIT status, the absence of measurements for other elements of quality of care makes it impossible to discern whether the observed association was due to poor data quality itself or data quality as a proxy for broader quality of care. Data quality could be a marker of lower provider competence, poorer supplies and infrastructure at the health facility, higher provider-patient ratio, lower contact time between providers and patients, or other phenomena associated with IIT status. Further research is necessary to understand the role data missingness plays in care provision.

Our finding that overall missingness was higher among pediatric patients and that the association between missingness and IIT status was stronger among pediatric patients supports our hypothesis that there may be differential quality of care among pediatric patients leading to poorer retention in care. Pediatric populations living with HIV have unique care needs, and poorer engagement for pediatric patients across the HIV care cascade is a multifaceted issue, involving behavioral, psychosocial, pharmacokinetic, and structural factors [17,46]. The literature has identified a number of key barriers to pediatric ART adherence, including stigma among caregivers to seek or continue care for children; lack of education or training for caregivers on caring for a child living with HIV; complexities inherent to a patient-caregiver-provider relationship; limited patient agency due to age and patient-caregiver power structures; patient-led treatment refusal, sometimes due to a lack of palatable formulations for younger patients; and lack of providers trained in pediatric HIV care or family-based service delivery [17,47,48]. Relevant to this analysis, prior research has shown direct links between quality of care, care engagement, and patient outcomes for this population. In their analysis of children living with HIV in Nigeria, Ojikutu et al [37] found that higher quality care—measured as a composite score exploring TB screening, adherence measurement and counseling, CD4 and weight documentation, and medication prescription—was significantly associated with a lower likelihood of pediatric and adolescent patients being lost to follow-up and mortality. Improving the quality of care for children living with HIV, including better patient record quality, is necessary to address the gaps in HIV testing and treatment among children living with HIV.

Limitations

At present, it is not possible to track patients between facilities within our analysis data. As such, it is not possible to distinguish patients who transferred to a new facility but remained on ART and those who interrupted or fully discontinued treatment. As a result, our ART retention outcome was defined as an IIT at a patient’s current facility rather than interruption overall. This outcome still fits within our causal model, with lower quality of care being feasibly associated with either an actual IIT or transfer to another facility for higher quality care, and it still represents a meaningful proxy indicator for clinical outcomes, as facility transfer may be associated with ART treatment gaps or discontinuation. Patient transfers are also not a limitation specific to this study; a systematic review of ART retention studies found that nearly 20% of patients classified as lost to follow-up had actually self-transferred to another facility [49].

Additionally, our results may be confounded if the missingness of the indicators is associated with the values of that indicator as well as our outcome. For example, if a higher WHO HIV stage is associated with both a greater likelihood of being missing and a greater likelihood of IIT, the observed association may be due to the latent WHO HIV stage rather than the data missingness. Of note, although integrated with iSanté, pharmacy data used to calculate the IIT outcome variables are collected through different mechanisms and staff. This includes greater data quality oversight, in part due to their inclusion in routine President’s Emergency Plan for AIDS Relief (PEPFAR) monitoring, evaluation, and reporting. Therefore, we do not anticipate that misclassification of the IIT outcome due to missing pharmacy data will be highly correlated with our exposure (missingness among indicators of interest), and thus, it will not present a substantial risk of bias. Finally, we were not able to assess associations with clinical outcomes (eg, viral suppression) due to data availability limitations.

Strengths

This was an observational study, and therefore, it could not assess a causal relationship between data missingness and IIT; however, our hypothesis is strengthened by the robust sample size and analytical design; strong association observed between missingness and IIT status; a dose-response relationship wherein greater missingness was associated with greater likelihood of a patient having IIT; and established temporal sequence, as the intake data are completed prior to ART initiation and the IIT outcomes.

Conclusions

Our analysis showed that both patient record data missingness for key indicators and interruption in ART treatment were common among patients, with nearly one-quarter of patients having IIT at 6 months and more than one-quarter of patients missing at least 1 indicator of interest in their patient record. Both IIT status and data missingness were more common among pediatric patients. Greater data missingness was associated with a higher likelihood of being IIT at 6 and 12 months for both pediatric and adult patients, although the association was stronger and more consistent among pediatric patients. Our findings motivate further research into the mechanisms by which EMR data quality impacts the quality of care and patient outcomes, particularly among children living with HIV. Additionally, efforts to improve both EMR data quality and quality of care should consider prioritizing pediatric patients.