Improving Care Management in Attention-Deficit/Hyperactivity Disorder: An RCT

We conducted a prospective randomized comparative effectiveness trial involving 11 primary care pediatric practices affiliated with a large children’s hospital from March 10, 2016, to April 12, 2019. The practices included 5 urban practices and 6 suburban practices that used a common electronic health record (EHR) and an ADHD-specific patient portal known as the ADHD Care Assistant.^18,19  Five of the practices had colocated behavioral health services. We recruited the practices using letters of invitation and in-person presentations. The study was approved by the Institutional Review Board at the Children’s Hospital of Philadelphia and was registered at Clinicaltrials.gov (NCT02716324) before recruitment of participants.

Children were eligible for the study if they received care at a participating practice, had an ADHD diagnosis code (International Classification of Diseases, Ninth Revision [ICD-9] code 314) recorded at an ambulatory visit in the past year, and were aged 5 through 12 years old. Children were excluded if they had a diagnosis of autism spectrum disorder (ICD-9 code 299), conduct disorder (ICD-9 code 312), psychosis (ICD-9 code 298), bipolar disorder (ICD-9 code 296) or suicidal ideation or intent (ICD-9 codes E950.0–E958.9) in the past 12 months. Lists of potentially eligible children were identified from EHR records and were verified by primary care clinicians at the practices. A random sample of at least 300 eligible children stratified by practice, sex, and age group (5–7 vs 8–12 years old) were selected to achieve a representative sample. Parents of selected children were mailed a recruitment letter and a stamped self-addressed postcard to opt out. Families who did not opt out within 2 weeks were called to screen for eligibility, provide study information, and schedule an enrollment visit. We randomly selected additional children from the same strata as those who declined participation, were ineligible, or were unable to be contacted. Children were consented and randomly assigned 1:1 within strata to the 2 groups by the study biostatistician using a random number generator.

We sought to compare the effectiveness of an ADHD portal embedded in the EHR (portal alone) with the portal combined with an ADHD care manager (care management + portal). The ADHD portal, known as the ADHD Care Assistant, was designed to (1) collect and share patient and family treatment preferences and goals with a clinician; (2) trend ADHD symptoms, performance impairment ratings, medication side effects, treatment receipt, and medication side effects by using electronically submitted parent and teacher reports; (3) provide a repository of ADHD educational materials; and (4) support information sharing between parents and teachers (Fig 1). The ADHD Care Assistant enabled clinicians to send survey-driven ADHD symptom rating scales to a child’s parent and teacher via e-mail. Data collected through this system were displayed directly to the clinician within the EHR. Parents were provided an opportunity to view teacher reports, and teachers could view parent-submitted reports with parental consent.²⁰  The frequency of e-mails from the portal varied from biweekly to every 3 months at the discretion of the clinician in consultation with the family. Parent and teacher usage of the ADHD Care Assistant during study participation was extracted from the EHR.

ADHD care managers were bachelor’s-trained individuals who were responsible for communicating information and facilitating coordination of care. The care managers confirmed family treatment preferences and goals, provided education on the treatment of ADHD and associated conditions, monitored attainment of parent-directed goals and emerging issues, and provided resources and assistance with concerns for patients with ADHD and families. The care managers sought to contact families, teachers, and clinicians by phone, text message, or e-mail every 3 months or sooner if problems arose. The care managers completed a fidelity checklist after each encounter to assess self-reported task completion (0, not completed; 1, partially completed; 2, fully completed) and summarized the sessions as a telephone encounter in the EHR.

We collected demographic information (child age, child sex, child race and/or ethnicity, child Supplemental Security Income [SSI] status, parental education level, urban or suburban residence, free or reduced lunch status, and school type [public, charter, or private]) at baseline. In addition, we geocoded participant home addresses and used American Community Survey tract-level data to obtain median family neighborhood income levels. ADHD medication (stimulants, α-agonists, and atomoxetine) fills and dates for each child during the study period were determined by abstraction from the EHR.

Change in ADHD symptoms was the primary outcome. We used the ADHD symptom subscale of the Vanderbilt Parent Rating Scale (VPRS) to assess ADHD symptoms.²¹  The VPRS has been shown to have excellent internal consistency (α = .90–.94) and concurrent validity (r = 0.79) in relation to diagnostic interviews. The VPRS includes 18 items that correspond to the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition symptom criteria and has a 4-point Likert response category (never [0] to very often [3]) for each item. We summed items on the VPRS to obtain an overall symptom score ranging from 0 to 54, with lower scores indicating fewer ADHD symptoms.

Changes in goal attainment and PROs were secondary outcomes. We identified family treatment goals through use of the ADHD Preference and Goals Instrument, a parent-reported 46-item tool that queries parents on their treatment preferences (medications and/or behavioral therapy) and goals for treatment.²²  A primary goal for each family was identified and categorized as academic, behavioral, or relational by consensus of the study team. Goal attainment scaling (GAS) was used to allow parents to rate the degree to which their goal was met over time and was scaled from 0 (no change) to 6 (goal completely met).^23–25  PRO measures were completed by a parent proxy and by children aged ≥8 for school performance, student engagement, peer relationships, family belonging, and teacher connectedness. These PRO measures consisted of Patient-Reported Outcomes Measurement Information System short forms and Healthy Pathway scales, which are brief, reliable, and precise measures of patient-reported health status.^26–29  Higher GAS and PRO scores indicate better outcomes.

Outcome measures were collected at baseline and at 3-month intervals for 9 to 12 months after enrollment by using Research Electronic Data Capture e-mail surveys. Research staff, who were blinded to participants’ study arms, contacted nonresponders at each study visit by phone to complete the surveys. In addition, we obtained VPRS scores from the ADHD portal, if available within 30 days of the corresponding Research Electronic Data Capture survey, for nonresponders.

We employed imputation methods for missing items for 2 of the outcomes (VPRS and PRO measures). If a participant had 1 or 2 items missing within 1 of the 2 domains on the VPRS or any of the PRO measures, the average of the nonmissing items for the domain or PRO measure was assigned to obtain a score. No between–study arm differences between pre- and postimputation VPRS and PRO mean scores were observed for cases and controls. VPRS subdomains and PRO measures with >2 missing items were not imputed. We elected not to impute missing GAS scores.

To determine differences in outcomes between groups, we followed the standard of an intention-to-treat repeated-measures longitudinal analysis. To check on the adequacy of randomization, patient characteristics were compared between groups. To assess bivariate associations between study arm and outcome at each of the study visits, t tests were used for ADHD symptoms, parent and child PRO measures, and goal attainment. To assess differences in outcomes, we used linear mixed-effects models to account for missing outcome values under a missing-at-random assumption. We examined results from analogous generalized estimating equations models to examine the robustness of our conclusions to model selection.^30,31  Intervention-by-time interactions for each outcome were used to represent the adjusted difference in outcome measures between the 2 groups over time. Models were adjusted for seasonality (fall, winter, spring, and summer), child age (5–7 and 8–12 years), child sex, child race and/or ethnicity (white, Black, Hispanic, or other), free or reduced school lunch, metropolitan status (urban or suburban), child SSI status, parent education level (up to high school, some college, or college degree), school type (public, private, or charter), median neighborhood income, and ADHD medication status. To account for clustering, clinic site was included in each model as a random effect. With a sample of 300, we had power of 0.87 to detect a difference of 2.5 points on the VPRS scores between groups, assuming α = .05, 80% follow-up, and moderate correlation (r = 0.6) over 9 months. The difference of 2.5 points represented a small clinically meaningful effect size from our pilot study. We conducted a sensitivity analyses to examine the impact of intervention dosage (ie, number of care management sessions) on ADHD symptoms. All analyses were conducted by using Stata 15 statistical software (Stata Corp, College Station, TX).

We identified 3118 potential participants aged 5 to 12 years old with ADHD from the 11 practices (Fig 2). Of these, we randomly selected and stratified 960 children. We sent 875 recruitment letters, but 572 were not enrolled for the following reasons: 112 declined participation, 174 were ineligible, and 286 were unable to be reached. Thus, we enrolled and randomly assigned 303 eligible children: 154 to care management + portal and 149 to portal alone. Of these, 273 (90.1%) completed the study, as defined by completing the final VPRS: 143 (93%) in care management + portal and 130 (87%) in portal alone.

After randomization, children in both study arms had similar sociodemographic characteristics (Table 1). The average age was 8.5 years old, with most (91%) ≥8 years of age. Two-thirds were male. Children were racially and socioeconomically diverse, with 51% residing in urban locations. More than half qualified for free or reduced school lunch (54%), and more than half attended public schools (60%). As expected, a slight majority (53%) of children reported receiving an ADHD medication at baseline.

Over the course of the study period, 206 (68%) study participants used the ADHD portal to complete a parent-reported VPRS (Table 2). The average number of VPRSs completed was similar in both arms: 2.3 in care management + portal and 2.2 in portal alone. Nearly one-third of participants (30%) had a teacher use the patient portal to complete a Vanderbilt Teacher Rating Scale. In care management + portal, most parents (96%) had at least 1 care management session. One-third of children (34%) had a teacher engage with the care managers. When care manager fidelity checklists were examined, 66% of parent and 63% of teacher sessions were rated as fully completed on all relevant items (data not shown) and the rest as partially completed or not relevant.

Table 3 shows the primary outcome, mean VPRS scores, by study visit and group. All 303 participants completed study visit 1, 258 (85%) completed study visit 2, 236 (78%) completed study visit 3, and 273 (90%) completed study visit 4. Mean VPRS scores decreased over time, indicating clinical improvement in ADHD symptoms, but there were no statistically significant differences in mean scores between groups at any study visit. GAS, parent proxy–reported PRO, and child-reported PRO scores did not change over time and did not differ between groups at any study visit (Supplemental Tables 5 through 7).

To assess changes between groups in VPRS scores, results from the previously described model are presented in Table 4. The intervention-by-time interaction in the full model was not significant (β = .00; 95% confidence interval [CI] −.01 to .01), indicating no difference between groups in changes in VPRS scores over time. In models without the interaction term, time (days) was significant, suggesting that VPRS scores decreased an average of 0.015 points per day or roughly 4 points over the course of the 9-month intervention period for both groups, a clinically meaningful improvement. Urban and medication status were also both significant, indicating that children residing in urban compared with suburban residences had VPRS scores that were greater and children on ADHD medications, as opposed to no medications, had VPRS scores that were lower. There were no adverse effects from either intervention identified, and interactions of intervention by race or income were not significant, suggesting no heterogeneity of treatment effects. Results from the generalized estimating equations model were consistent with results from the linear mixed-effects model.

We conducted a sensitivity analysis to determine if there was a dose-response in which greater engagement among care management + portal participants resulted in greater declines in VPRS scores. After adjustment for seasonality, we found that those participants who received ≥2 care management sessions experienced statistically significantly greater decreases in VPRS scores (−4.7; 95% CI −8.0 to −1.4) than those with 1 or 0 sessions. There were no demographic differences by the number of care management sessions attended. This nonrandomized result suggests that greater engagement of participants within the care management intervention resulted in greater symptom improvement.

In this comparative effectiveness study of 2 communication strategies, an electronic patient portal combined with a care manager versus a portal alone, we found no difference over time in primary or secondary outcomes between the 2 groups. We observed an overall improvement in ADHD symptoms over time in both groups but little to no change in goal attainment or PROs at any time point. This suggests that care management did not improve ADHD symptoms over and above that of the patient portal alone.

Our finding that children in both groups showed improvement in ADHD symptoms over time is consistent with previous studies employing patient portals for ADHD. Epstein et al¹³  found that community pediatricians using an ADHD portal were significantly more likely to collect information from parents and teachers than those using usual care. Nagykaldi et al³²  found that the use of a portal focused on preventive care resulted in increased patient activation and greater patient-centered care, and users were more likely to receive needed preventive care.

There was variable engagement by parents with the care manager. It is not entirely clear why parents did not consistently engage to a greater extent with the care manager than with the portal as we had postulated. Our results revealed modest engagement by parents and teachers with care managers. Care managers were instructed to engage with parents and teachers virtually at least every 3 months; however, care managers found it challenging to contact parents and teachers to schedule sessions. In some instances, care managers were unable to identify contact information for teachers despite calls to schools. Given the virtual nature of the care management intervention in this study, face-to-face contact was lacking and may have contributed to the inconsistent engagement. In a study of community health workers, those workers who provided face-to-face communication saw more beneficial effects on the quality of care and a reduction in hospital days.³³  In addition, systematic reviews of collaborative care trials among adults with depression have consistently demonstrated that care management involving face-to-face contact is associated with modest but sustained improvement in depression outcomes (standardized mean difference 0.25; 95% CI 0.18 to 0.32) compared with usual care.^34,35  Our preplanned sensitivity analysis revealed that those with a greater number of care management sessions had greater reductions in ADHD symptoms than those with fewer sessions, suggesting the importance of family engagement for this intervention to be effective.

There were several limitations to our study findings. First, our study was conducted in a single geographic area within an integrated pediatric health care system. Results may not be generalizable to other geographic areas or other health care systems. Second, our care managers primarily employed electronic means of communication, which may have limited engagement, as discussed above. Third, we did not enroll teachers in our study because of challenges in connecting with and obtaining formal approval from schools. This limitation did not permit us to interview teachers to discern their perceptions of the interventions. Finally, we lacked a no intervention control group, because use of the electronic portal was considered standard of care at our institution during the study period. This limited our ability to formally test benefits of the portal over no portal care.

In this comparative effectiveness study, we found no evidence that care management combined with a patient portal produced patient outcomes different from those of a patient portal alone among school-aged children with ADHD. Both groups demonstrated similar reductions in ADHD symptoms over time. Overall, there was variable engagement by parents with care managers, which likely limited its impact. Those families with greater care management engagement, as demonstrated by the number of sessions attended, showed greater reductions in ADHD symptoms over time than those with less. There was no heterogeneity of treatment effects as a function of race and/or ethnicity or household income. In future studies, researchers investigating the effects of care management should consider methods (eg, face-to-face meetings) to better engage families and teachers.

We thank the Pediatric Research Consortium (PeRC) at the Children’s Hospital of Philadelphia and all participating practices and clinicians for their support of this study. We also thank Stephanie Liu, MS, for her assistance with data collection.

ADHD

attention-deficit/hyperactivity disorder

CI

confidence interval

EHR

electronic health record

GAS

goal attainment scaling

ICD-9

International Classification of Diseases, Ninth Revision

PRO

patient-reported outcome

SDM

shared decision-making

SSI

Supplemental Security Income

VPRS

Vanderbilt Parent Rating Scale