Teaching Parents to Be Responsive: A Network Meta-analysis

A protocol for this study was developed and registered with the International Prospective Register of Systematic Reviews (CRD42018100100). A search strategy (detailed in Section 1 of the Supplemental Information) was developed in consultation with a librarian and experts in developmental psychology and population health; the strategy was then reviewed by another librarian using the Peer Review of Electronic Search Strategies checklist.²²  We searched in PsycINFO, Medline, the Cumulative Index to Nursing and Allied Health Literature, Social Work Abstracts, the Education Resources Information Center, ABI/INFORM, and the Cochrane Central Register of Controlled Trials from the inception of each of these databases to October 22, 2020. To identify additional studies, we scanned the reference lists of both included studies and relevant reviews retrieved in our search. The initial intent was to assess effective methods for teaching general populations of adults to be empathetic and responsive to what others are thinking and feeling. However, because of the number of studies identified and to ensure homogeneity of networks, we subdivided the analyses. The current article is focused on studies training parents only.

We included randomized trials of training programs teaching adults how to be more responsive toward their children. Studies had to be available in English (the language of the authors, a limitation because of coding capacities) and report on a quantitative measure of parents’ ability to understand what their children were thinking or feeling and respond in an appropriate (ie, sensitive, responsive, and empathic) manner. When data were not reported in a way that could be converted into an effect size, we contacted the author (20 contacts and 8 responses). We excluded studies in which the reported prevalence of any mental illness (eg, depression, anxiety, addiction, schizophrenia, and/or autism spectrum disorder) among parents was >25% (details are provided in Section 2 of the Supplemental Information). We applied this criterion to limit our findings to the general population, in which rates of psychopathology are ∼20%,²³  because responsivity learning may be dampened or different in clinical populations; indeed, parents facing the challenges of depression are, on average, less responsive and show smaller gains from responsivity training.^24,25  We also only included studies in which researchers assessed responsivity toward children in early childhood (ie, average age of <6 years) because this is the developmental period during which developmental trajectories are the most malleable and responsivity is most strongly associated with children’s later cognitive and socioemotional outcomes.^5,18 

We convened a group of 5 parenting researchers, adult educators, and clinical psychologists to conduct a qualitative consensus-based categorization procedure.²⁶  All participants first familiarized themselves with existing meta-analyses of instructional methods.^19,20  Then they reviewed the same 10 articles in which researchers describe different parenting interventions, defined their instructional approaches, independently categorized them, and discussed discrepancies to identify a final consensus. This resulted in coding based on 5 types of instructional methods: (1) didactic learning (reading about or being told how to be responsive), (2) observation (watching examples of responsive interactions), (3) reflection (reflecting on and discussing one’s own experience with responsivity), (4) rehearsal (practicing being responsive during interactions), and (5) feedback (receiving feedback on one’s display of responsivity). These methods were coded on the basis of their presence or absence in the program description; unfortunately, most articles did not provide sufficient detail to evaluate the relative intensity of, or time spent using, these different instructional approaches. We also distinguished 3 types of control groups, as per common practice in network meta analyses²⁷ : (1) those who received no training (eg, waitlist); (2) those who were receiving standard care (ie, treatment as usual), such as neonatal care or services to support a child with a developmental delay; and (3) those who were active controls, receiving training similar in format to the intervention group but covering other topics in child development (eg, nutrition or health) and not responsivity.

The first author trained advanced undergraduate and graduate student research assistants on each phase of study selection and data extraction; assistants had to obtain an interrater reliability >80% on a sample of articles to pass the training. Once trained, 2 research assistants independently screened all abstracts and full texts, abstracted data, and appraised study quality using the Cochrane risk-of-bias tool.²⁸  Coders extracted moderators, including characteristics of the study (eg, year and publication type), program (eg, individual or group delivery, facilitator, dose, and inclusion of home visits or video feedback), and sample (eg, age, education, income, and child health) using a predefined data extraction sheet, according to the definitions outlined in Section 2 of the Supplemental Information. Although all material was double coded and all discrepancies were resolved by the first author, it is notable that coder agreement was high (93% for abstract screening and 86% for both full-text review and data extraction).

For studies that reported multiple outcomes, we selected the most methodologically rigorous measures of responsivity, choosing reliable observational outcomes over parent-reported outcomes. Although observational measures are not without their limitations,²⁹  they are the most sensitive to change after intervention and provide the best prediction of child outcomes.^30,31  When multiple outcomes of the same methodologic quality (eg, multiple observational measures of responsivity) were reported in a given study, we calculated a pooled effect size across these measures.³²  For 3-arm trials in which 2 of the 3 interventions were functionally equivalent (ie, minor difference in content and format of delivery), we calculated a pooled effect size for those 2 interventions relative to the third. Details of the measures and study arms used for each comparison are available in Section 3 of the Supplemental Information.

We used Comprehensive Meta-Analysis software to calculate the standardized mean difference (Cohen’s d) effect sizes and standard errors. Then, we evaluated the assumptions of network meta-analysis. We assessed transitivity by examining the comparability of the distributions of study, program, and sample characteristics across comparisons. We assessed consistency globally using the design-by-treatment interaction model and locally using the loop-specific method.^33,34  If assumptions were met, we conducted a random-effects network meta-analysis using the network and network graphs packages in Stata (version 16; Stata Corp, College Station, TX), assuming a common within-network between-study variance (τ²). The efficacy of different interventions was ranked by using surface under the cumulative ranking (SUCRA) curve values.³⁵  To assess reporting bias, we plotted a comparison-adjusted funnel plot.³⁶  Sensitivity analyses were conducted, excluding outliers (defined as having a standardized mean difference 3 SDs above or below the mean) and studies with a high risk of bias. We also conducted meta-regressions for variables identified as potentially unevenly distributed on the basis of transitivity analyses.

In addition, we conducted a random-effects pairwise meta-analysis in Comprehensive Meta-Analysis using the subset of studies that had a control group. For three-arm studies in which 2 of the interventions were not equivalent, we used comparisons between the most intensive intervention group and control group. We assessed for statistical heterogeneity with the I² statistic and explored it using mixed-effects model meta-regressions.²⁸  We assessed for publication bias using the funnel plot and Egger’s test.²⁸ 

After screening 10 725 citations, 1637 articles were assessed for eligibility, and ultimately 119 articles in which researchers reported on data from 120 unique samples and 12 376 participants were included (Fig 1). A summary of the characteristics of these studies is reported in Table 1 (see Section 3 of the Supplemental Information for details). Most studies were conducted from 2010 to 2020 in North America or Europe; participants were predominantly white mothers with less than a university education. Most programs were delivered one-on-one by professionals, with a median duration of 8 sessions or 11.5 weeks. The majority of studies exhibited a low risk of bias in all categories except allocation concealment, for which 65% had an unclear risk (see Section 4 of the Supplemental Information for details).

A total of 16 different combinations of instructional methods or types of control groups were used across the studies, with 35 unique direct comparisons. The network geometry is presented in Fig 2. The most frequent type of intervention included all instructional methods, whereas there were relatively fewer studies of interventions in which researchers used only 1 or 2 approaches. The network meta-analysis revealed no evidence of inconsistency (global: χ²[19] = 11.03; P = .92; local: all P >.10) but some heterogeneity (τ² = 0.21).

Figure 3 displays the average effect size for each combination of instructional methods compared with the pure control groups with no training. In addition, a table listing the effect sizes for all possible two-way comparisons between different combinations of instructional methods is available in Section 5 of the Supplemental Information. Several combinations of instructional methods were significantly more effective than all 3 types of control groups, with medium to large effect sizes: programs using didactic, observation, rehearsal, and feedback (d = 0.89 to 1.07); all 5 instructional methods (d = 0.68 to 0.86); didactic, observation, and reflection (d = 0.63 to 0.81); didactic, observation, reflection, and rehearsal (d = 0.54 to 0.72); and didactic, reflection, rehearsal, and feedback (d = 0.31 to 0.50). Relative efficacy ranking, based on the SUCRA curves, revealed that the most effective types of program were those that included: (1) didactic, observation, rehearsal, and feedback, (2) all 5 instructional methods, and (3) didactic, observation, and reflection. The SUCRA values and curves are presented in Section 5 of the Supplemental Information.

The comparison-adjusted funnel plot (Section 6 of the Supplemental Information) suggested that small studies may exaggerate the effectiveness of less intensive interventions, although this was heavily influenced by a few outliers. In the sensitivity analysis without outliers, we identified the same top 3 combinations of instructional methods. In the analysis with only low risk-of-bias studies, we identified programs using all instructional methods as being the most effective and those without feedback being more effective than those without reflection (see Section 6 of the Supplemental Information for details). Program rankings also differed somewhat in adjusted versus unadjusted models (see Section 6 of the Supplemental Information). Nonetheless, programs that used didactic instruction, observation, rehearsal, and feedback ranked in the top 3 in 83% of network meta-regression analyses, whereas those using all instructional methods were ranked in the top 3 75% of the time. In all scenarios, at least 1 of these types of programs was in the top 3 and in many cases both were in the top 3, underscoring the consistent superiority of programs using these combinations of instructional approaches.

The majority of studies included a control group (113 samples involving 11 693 parents) and could be included in the pairwise meta-analysis. A forest plot for these studies is available in Section 7 of the Supplemental Information. Overall, there was a medium effect size of 0.56 (95% confidence interval [CI]: 0.47 to 0.65; without 3 outliers: d = 0.52; 95% CI: 0.43 to 0.60).³⁷  Although there was evidence of publication bias (Egger’s test P < .0001; funnel plot in Section 8 of the Supplemental Information), Duval and Tweedie’s trim-and-fill procedure did not suggest trimming any studies or produce a different estimate.

Because there was substantial heterogeneity (I² = 79.52; P < .001), we tested whether this could be explained by study, program, or sample characteristics. The results of the meta-regressions are presented in Table 2. Programs that were more effective required participants to complete homework assignments (d = 0.84; 95% CI: 0.66 to 1.02, compared with d = 0.40; 95% CI: 0.31 to 0.49), were facilitated by a researcher (d = 0.89; 95% CI: 0.66 to 1.12), rather than a different type of facilitator (d ranging from 0.20 to 0.53), and were focused narrowly on teaching responsivity (d = 0.72; 95% CI: 0.57 to 0.87), rather than covering a broad range of unrelated topics (d = 0.40; 95% CI: 0.30 to 0.50). Programs in which the participants’ children were preschoolers were also more effective (d = 0.87; 95% CI: 0.64 to 1.10), on average, than those in which children were infants (d = 0.47; 95% CI: 0.34 to 0.60) or spanned multiple age categories (d = 0.31; 95% CI: 0.07 to 0.54). Finally, program effects were stronger for participants whose children did not have any physical impairments (d = 0.62; 95% CI: 0.51 to 0.73), compared with those whose children were physical impaired (d = 0.35; 95% CI: 0.21 to 0.49). In a multivariate regression considering all moderators that were significant in the individual analyses, only use of homework (P = .006), type of facilitator (P = .007), and program scope (P = .02) remained statistically significant predictors of program effect size. Together, these 3 variables explained 40% of the variance in program effectiveness. Sensitivity analyses were also conducted, replicating all meta-regressions without the 3 outlier studies (see Section 8 in the Supplemental Information). In these analyses, significant effects for child age and physical impairment were not replicated (P = .12 and .07, respectively). However, there was an effect of program length whereby programs that lasted >1 year were significantly less effective (d = 0.17; 95% CI: −0.008 to 0.35) than those taking place within 1 year (d ranging from 0.54 to 0.66).

We also used meta-regression to test for the effects of individual instructional methods. Programs that had caregivers observe examples of responsivity were significantly more effective (d = 0.70; 95% CI: 0.57 to 0.83), on average, than those that did not (d = 0.33; 95% CI: 0.23 to 0.43), even when outlier studies were removed. Programs were not consistently more or less effective whether they included didactic, reflection, rehearsal, or feedback methods.

Enhancing parental responsivity is one way our society can invest in improving the mental and physical well-being of future generations and reduce intergenerational inequalities.¹³  The current review was conducted to help maximize the efficiency of efforts to enhance parental responsivity at a population level. The 3 most effective types of programs identified in the network meta-analysis all included didactic and observation activities, with rehearsal and feedback being an added feature of the single most effective category of programs. Across all sensitivity analyses, programs using all 5 instructional methods (didactic, observation, reflection, rehearsal, and feedback) were consistently identified as the most or second most effective and had a large effect size. Taken together, these results suggest that it is valuable to include all instructional methods when designing training programs; however, in cases in which reflection, rehearsal, or feedback are not feasible or too costly omitting one or a few of these formats should not come at a high cost to parent learning. In contrast, observational teaching should not be omitted, as the results of the pairwise meta-analysis suggest that one of the best ways for parents to learn how to be responsive caregivers is to observe good examples of responsive caregiving, an important and novel finding for program designers.

The pairwise meta-analysis also revealed 3 aspects of program design robustly associated with above-average effect sizes. First, in line with previous research,¹⁶  programs in which parents were only instructed on responsive caregiving were more effective than those with a broader scope. This replicated finding underscores the principle that when programs try to do everything, it is hard to do any one thing well. Second, programs in which homework was assigned to participants were significantly more effective than those in which it was not. Indeed, in the present meta-analysis, this aspect of program design individually accounted for 18% of the variance in program effectiveness. Although there is evidence to suggest that participants who complete more homework in parent training programs demonstrate stronger outcomes,³⁸  existing research has not compared the effectiveness of programs that do or do not assign homework. The current findings imply that including such assignments is valuable and that responsivity is a skill that is best learnt through consistent practice. Finally, the programs that were most effective tended to be facilitated by researchers. Past reviews have found mixed evidence about the relative effectiveness of professionals, compared with nonprofessionals, as parenting program facilitators; however, these reviews have all categorized researchers as professionals.^16,39  The current results suggest that researchers may be particularly effective at supporting parents to be more responsive. Although there is little literature to draw on in explaining this result, we speculate this distinction could be attributable to researchers’ enhanced understanding of key concepts related to responsivity and ability to convey a narrow, focused message regarding a program’s learning goals, thereby "teaching to the test."

This review also highlighted several program characteristics that do not enhance the learning of responsivity among parents without elevated levels of psychopathology: program dosage (sessions, hours, weeks, and frequency), individual versus group delivery, and the use of home visits or video feedback. In other words, long, individualized training programs may not help parents become substantially more responsive than short, group programs do. Furthermore, results suggest responsivity training programs benefit parents similarly, regardless of their level of education and income. Although these findings are in contrast with our hypotheses and findings from some older reviews,^16,39,40  because researchers in these previous reviews assessed the effects of programs with different foci (eg, attachment or disruptive behavior) and included quasi-experimental studies as well as studies with clinical populations (eg, depressed mothers), direct comparisons cannot be made. Furthermore, these past meta-analyses are not representative of the abundance of research that has emerged since 2005. Indeed, in a more recent review, researchers have called into question the notion that parent and child characteristics moderate the effects of parenting programs.⁴¹  On the basis of the current results, we conclude that, although one-on-one training using personalized video feedback or home visits undoubtedly helps parents learn to be more responsive, these labor-intensive programs may not be absolutely necessary to elevate responsivity among parents in the general population. That being said, there may be other benefits of individualized programs that were not captured in the current analysis; for example, such programs may be necessary to reach certain parents.⁴² 

In this study, we used network meta-analysis to compare the relative effectiveness of various combinations of instructional methods in parent responsivity training programs. We summarized evidence from >100 randomized trials, the majority of which had a low risk of bias, and results were robust to the removal of outliers and studies with a high risk of bias. However, all research was published in English and conducted predominantly with white mothers in Western populations; as there was also evidence of publication bias, our search could have been strengthened through greater identification of international and unpublished data sources. Nonetheless, because the gap in research on interventions for fathers and caregivers from diverse backgrounds was also identified in a different recent review,⁴³  this limitation was likely not a consequence of our search strategy or inclusion criteria and, moving forward, the academic community will need to conduct more research with fathers and ethnically diverse populations for us to better optimize program design for these populations. It is also important to note that our results are only applicable to parents without identified levels of psychopathology and this criterion led to the exclusion of some more vulnerable and diverse samples. Given that responsivity inversely correlates with psychopathology in parents,²⁴  it will be important for researchers in future studies to establish whether optimal instructional methods differ for clinical and general populations. Finally, we also identified a gap in the reporting of details of program design and delivery as well as a shortage of follow-up data. Increasing the availability of such data would allow us to assess the relative efficacy of different instructional methods on the basis of the amount of time spent using each method and determine if certain instructional methods more effectively support sustained behavior change, compared with short-term learning.

The current findings provide several actionable recommendations for health professionals, policy makers, and program developers looking to improve children’s developmental trajectories. Specifically, results suggest that using expensive methods, such as year-long individual home visits, to help parents in the general population be more responsive toward their young children is not necessary. Developing short, focused programs led by researchers that include opportunities for parents to learn about and observe examples of responsive parent-child interactions with some take-home assignments between sessions could be just as or even more effective. Such recommendations may be applied to improving population health across low-, middle-, and high-income countries, during routine encounters with health professionals or using low-resource group programs implemented in health care or community settings. These results may make it easier to support more parents without using more resources, ultimately improving the lives of more children.

CI

confidence interval

SUCRA

surface under the cumulative ranking