Evidence to guide treatment of pediatric medium-chain acyl-coenzyme A dehydrogenase (MCAD) deficiency and phenylketonuria (PKU) is fragmented because of large variability in outcome selection and measurement. Our goal was to develop core outcome sets (COSs) for these diseases to facilitate meaningful future evidence generation and enhance the capacity to compare and synthesize findings across studies.
Parents and/or caregivers, health professionals, and health policy advisors completed a Delphi survey and participated in a consensus workshop to select core outcomes from candidate lists of outcomes for MCAD deficiency and PKU. Delphi participants rated the importance of outcomes on a nine-point scale (1–3: not important, 4–6: important but not critical, 7–9: critical). Candidate outcomes were progressively narrowed down over 3 survey rounds. At the workshop, participants evaluated the remaining candidate outcomes using an adapted nominal technique, open discussion, and voting. After the workshop, we finalized the COSs and recommended measurement instruments for each outcome.
There were 85, 61, and 53 participants across 3 Delphi rounds, respectively. The candidate core outcome lists were narrowed down to 20 outcomes per disease to be discussed at the consensus workshop. Voting by 18 workshop participants led to COSs composed of 8 and 9 outcomes for MCAD deficiency and PKU, respectively, with measurement recommendations.
These are the first known pediatric COSs for MCAD deficiency and PKU. Adoption in future studies will help to ensure best use of limited research resources to ultimately improve care for children with these rare diseases.
Evidence to guide treatment of pediatric medium-chain acyl-coenzyme A dehydrogenase deficiency and phenylketonuria is fragmented due in part to variability in outcome measurement and reporting. Core outcome sets (COSs) can help address this gap.
We present the first COSs developed for 2 pediatric inherited metabolic diseases, medium-chain acyl-coenzyme A dehydrogenase deficiency and phenylketonuria. We also recommend measurement instruments for each included outcome. These COSs can facilitate comparison and synthesis across studies of treatment effectiveness.
Inherited metabolic diseases (IMDs) are a group of >1000 individually rare, single-gene disorders frequently diagnosed in early childhood.1 Children with IMDs often have disproportionately high health care needs and may experience clinical manifestations that vary depending on the specific disease, from acute life-threatening episodes to chronic multisystem sequelae.2,3 New IMD therapies are rapidly emerging,4 resulting in an urgent need to generate robust evidence to guide care decisions, at the bedside and health system level.5 Although the gold standard for primary research evaluating treatments, randomized clinical trials are frequently challenging to implement in rare disease settings.6 In those rare disease trials that are conducted, researchers often exclude patient-centered outcomes, thereby neglecting patient and caregiver priorities.7 Furthermore, outcomes and outcome measurement instruments (OMIs) vary greatly across studies, making it difficult to synthesize and compare results.8,9
In our recent review of outcomes reported in previous studies of 2 of the most common IMDs in children, medium-chain acyl-coenzyme A dehydrogenase (MCAD) deficiency and phenylketonuria (PKU), we identified substantial heterogeneity9 : 83 and 97 unique outcomes were reported across studies of MCAD deficiency and PKU, respectively, with 33% (MCAD deficiency) and 16% (PKU) of outcomes being reported in a single study. These findings suggest the potential value of developing core outcome sets (COSs). COSs specify a small number (typically 6–9) of standard, meaningful outcomes recommended for all future studies in a given disease area.8 Developing COSs for MCAD deficiency and PKU would assist future research by promoting greater harmonization of outcome selection and measurement and encouraging uptake of the outcomes considered the most meaningful from stakeholder perspectives.9 The goal of this study was to develop COSs for future studies of children (≤12 years of age) with MCAD deficiency or PKU.
Methods
Protocol and Study Design
Our study protocol10 was developed by using the Core Outcome Measures in Effectiveness Trials handbook.8 In brief, we used a two-stage, multistakeholder consensus process for each disease: (1) a multiround Delphi survey to narrow down the list of candidate outcomes and (2) an in-person workshop to review and vote on a final list of core outcomes. We held additional consultations to finalize the selection of recommended OMIs.
This article is reported in accordance with the COS Standard for Reporting statement11 (Supplemental Information).
Ethical Considerations
The Delphi survey received research ethics board approval from the University of Ottawa, Children’s Hospital of Eastern Ontario, and all other participating centers, either directly or as part of a preexisting cohort study.
Patient Partnership and Engagement
Our study was codesigned with 2 patient partner investigators (M.S. and N.P.) who have lived experience with rare diseases. These partners led a patient engagement strategy, including involvement of a family advisory forum (FAF) composed of 7 caregivers of children diagnosed with IMDs. FAF members were recruited by clinician investigators at 2 centers (Ottawa and Vancouver) and were provided with training and resources to contribute meaningfully. Patient partners and FAF members contributed to all study documents provided to parent and/or caregiver participants for the Delphi survey and participated in the consensus workshop.
Delphi Consensus Survey
The goal of the Delphi survey was to narrow down the list of candidate outcomes identified in our previous review9 to a maximum of 20 outcomes per disease for consideration at the consensus workshop. This number was chosen to support a balance between the breadth of discussion and feasibility of finalizing the COSs at the workshop.
Eligibility and Recruitment
To ensure broad input, we identified 3 Canadian expert groups to participate in the Delphi survey: parents or caregivers, health professionals, and health policy advisors. We recruited parents and caregivers of children aged 12 years or younger with MCAD deficiency or PKU from participants in a Canadian cohort study who had received care at 1 of 12 pediatric metabolic centers and consented to be contacted.12 Eligible parents and caregivers were contacted by telephone (up to 3 attempts) to obtain e-mail addresses for those interested in the Delphi survey. Health professionals with experience caring for children with MCAD deficiency or PKU were identified via the investigators’ professional networks and/or were forwarded an invitation by the Garrod Association, a Canadian organization focused on IMD care. Policy advisors, who did not need to have experience specific to MCAD deficiency or PKU to participate, were identified through the Web sites of Canadian federal or provincial health policy organizations. For health professionals and policy advisors, we also used snowball sampling, asking participants to identify other potentially eligible individuals. All potential participants in all expert groups were invited to participate in the survey by e-mail, with up to 3 e-mail attempts for each survey round.
Survey Implementation and Analysis
The initial list of candidate outcomes was derived from our previous literature review9 (Supplemental Information). Using Research Electronic Data Capture (REDCap) tools hosted at the Children’s Hospital of Eastern Ontario Research Institute,13,14 we designed an Internet-based Delphi questionnaire for each disease. The questionnaire presented participants with each candidate outcome and its definition and was available in English and French. Participants were asked to rate the importance of measuring the outcome in future studies evaluating treatments for that disease on a nine-point scale (1–3: not important; 4–6: important but not critical; 7–9: critical). Consensus was defined a priori as follows: at least 70% of respondents in an expert group rated the outcome as “critical” and <15% rated it as “not important.”15 Parents and caregivers only rated outcomes pertaining to their child’s diagnosed disease. Health professionals and health policy advisors rated outcomes for both diseases (Fig 1).
Overview of study processes, participants, and outcomes. a Five outcomes merged to create 2 outcomes for a net loss of 3 outcomes. b Eleven outcomes merged to create 5 outcomes for a net loss of 6 outcomes.
Overview of study processes, participants, and outcomes. a Five outcomes merged to create 2 outcomes for a net loss of 3 outcomes. b Eleven outcomes merged to create 5 outcomes for a net loss of 6 outcomes.
In round 1 of the Delphi survey, participants rated the importance of all candidate outcomes and could suggest additional outcomes (Supplemental Information). After review and analysis of the round 1 results, we combined health professionals and policy advisors into a single expert group because of the low number of policy advisor responses. In round 2, participants were asked to re-rate each outcome after reviewing both their own rating from round 1 and a graphical summary showing the distribution of round 1 ratings for each of the 2 expert groups. We also asked participants to rate the new outcomes suggested during round 1.
Because of the large number of outcomes reaching consensus in round 2, to reduce participant burden, we decided to reduce the number of outcomes included in round 3 to only those that reached consensus in both expert groups in round 2. Once again, respondents re-rated each outcome after reviewing both their own scores and a summary of the group-level round 2 results. At the end of the round 3 questionnaire, participants were presented with the list of outcomes they had personally rated as critically important (ie, assigned a score of 7–9) and ranked up to 10 outcomes from the most to least important.
Because of the large number of outcomes reaching consensus, additional post hoc steps were taken to reduce the number of outcomes for consideration at the workshop. Specifically, for both diseases, a working group (B.K.P., J.I., M.T.G., and P.C.) reviewed outcomes reaching consensus in ≥1 expert groups and merged those with highly overlapping concepts into a single composite outcome. For example, for MCAD deficiency, metabolic decompensation, hypoglycemia, and encephalopathy were combined into a single outcome, metabolic decompensation and associated symptoms. For PKU, we also used the round 3 Delphi results to reduce the number of outcomes to those rated or ranked most highly (Supplemental Information).
Multistakeholder Consensus Workshop
The goal of the one-day workshop, held in Ottawa, was to reach consensus on a final set of 6 to 9 core outcomes16,17 for each disorder. We also aimed for each COS to include outcomes across 5 a priori established core areas18,19 : death (relevant to MCAD deficiency only), growth and development, life impact, pathophysiological manifestations, and resource use.
Investigators (clinicians, patient partners, methodologists, or health system experts) and FAF members were invited to attend the workshop in person or by webinar. Delphi survey respondents were invited to join by webinar. Workshop participants reviewed results of the literature review9 and Delphi survey. Outcomes were then discussed separately for each disease by using the following process, moderated by an experienced chair (M.O.): (1) in an adapted nominal group technique, each in-person participant was allotted 1 minute to provide their opinion about the most important outcomes; (2) the webinar facilitator (S.N.) summarized input from online participants; and (3) we held an open discussion. After the open discussion, workshop participants attending in person voted anonymously on each outcome using electronic real-time voting. For each outcome, participants could assign a score of 1 (must be in the COS), 2 (strong contender for the outcome set), or 3 (can be left out of the COS). Participants were instructed to assign a score of 1 to a maximum of 3 outcomes per disease (ie, identify their top 3 outcomes) and encouraged to consider balance across core areas. Immediately after anonymous voting, outcomes were ranked according to the proportion of respondents who scored the outcome as 1, 2, or 3. On the basis of these results, participants identified a suggested set of outcomes for inclusion in each COS. After the workshop, we sought further feedback by e-mail from all study investigators, FAF members, and Delphi survey participants.
Selection of OMIs
After the consensus workshop, a working group (M.P., K.T., B.K.P., J.I., M.T.G., P.C., and S.S.) refined definitions of the outcomes in each COS and identified potential OMIs. Although this method was straightforward for unambiguous outcomes measurable in a standard way through clinical or laboratory assessment (eg, blood phenylalanine concentration for PKU), OMIs for neuropsychological and patient- and caregiver-reported outcomes required more discussion. For these outcomes, we identified and reviewed potential OMIs from the findings of our previous review9 using a multistep process. First, we considered the OMI’s relevance to the core outcome and to MCAD deficiency or PKU. Second, we evaluated the feasibility of administering the OMI, considering the length, applicable age range, mode of administration (self, caregiver, or interviewer reported), qualifications required, availability in different languages, accessibility, and cost. Third, guided by the Consensus-Based Standard for the Selection of Health Status Measurement Instruments checklist (COSMIN),20 we reviewed published reports regarding the OMI’s validity, reliability, and responsiveness to change. The working group made recommendations regarding ≥1 strong candidate OMIs for each outcome and sought e-mail feedback from all study investigators. FAF members were invited to participate in the final selection of OMIs for patient and caregiver-reported outcomes by completing a team-developed feedback form (Supplemental Information).
Results
Delphi Consensus Survey
We invited 355 experts (parents and caregivers, clinicians, and health policy advisors) to participate in the Delphi survey, of whom 226 expressed interest and were sent a survey link (Fig 1). There were 85 participants in round 1 (38% of those who expressed interest), 61 of 85 (72%) in round 2, and 53 of 61 (87%) in round 3 (Table 1). Retention from round 1 to round 3 was the lowest for health professionals and policy advisors. In round 1, the majority of parent and/or caregiver respondents were female (90%), aged 31 to 40 years (52%), and college or university educated (90%). A majority of clinician and policy advisor respondents were female (71%), physicians (68%), and had >10 years of experience in their field (71%). Only 1 policy advisor completed the survey.
Delphi Survey Participant Characteristics
| Round 1, n (%) | Round 2, n (%) | Round 3, n (%) | |
|---|---|---|---|
| Parents or caregivers of children with MCAD deficiency or PKU | n = 54 | n = 42 | n = 37 |
| Sex, female | 45 (90) | 37 (90) | 32 (94) |
| Age, y | |||
| 18–30 | 2 (4) | 2 (5) | 2 (6) |
| 31–40 | 26 (52) | 21 (51) | 18 (50) |
| >40 | 22 (44) | 18 (44) | 16 (44) |
| No. children with MCAD deficiency or PKU | |||
| 1 | 37 (76) | 30 (75) | 13 (76) |
| ≥2 | 12 (24) | 10 (25) | 4 (24) |
| Highest level of education completed | |||
| Some or all of high school | 5 (10) | 4 (10) | 4 (11) |
| Technical or vocational training | 2 (4) | 2 (5) | 2 (5) |
| College, university, or graduate school | 43 (86) | 35 (85) | 30 (83) |
| Marital status (married or common law) | 47 (96) | 38 (95) | 34 (97) |
| Health care providers and/or policy advisors | n = 31 | n = 19 | n = 16 |
| Sex, female | 22 (71) | 13 (68) | 12 (75) |
| Age, y | |||
| 18–40 | 6 (19) | 3 (16) | 2 (13) |
| 41–50 | 16 (52) | 11 (58) | 9 (56) |
| >50 | 9 (29) | 5 (26) | 5 (33) |
| Experience, y | |||
| 1–5 | 3 (10) | 2 (11) | 2 (13) |
| 6–10 | 6 (19) | 3 (16) | 2 (13) |
| >10 | 22 (71) | 14 (74) | 12 (75) |
| Canadian province or territory of work | |||
| Western (British Columbia, Alberta, or Manitoba) | 8 (27) | 5 (26) | 5 (31) |
| Central (Ontario or Quebec) | 18 (60) | 11 (58) | 9 (56) |
| Eastern (Nova Scotia) | 4 (13) | 3 (16) | 2 (13) |
| Health care provider or system advisor position | |||
| Physician | 21 (68) | 13 (68) | 10 (63) |
| Other health care provider (eg, nurse, dietitian, psychologist, or genetic counselor) | 9 (29) | 5 (26) | 5 (31) |
| Health technology assessment or knowledge synthesis | 1 (3) | 1 (5) | 1 (6) |
| Round 1, n (%) | Round 2, n (%) | Round 3, n (%) | |
|---|---|---|---|
| Parents or caregivers of children with MCAD deficiency or PKU | n = 54 | n = 42 | n = 37 |
| Sex, female | 45 (90) | 37 (90) | 32 (94) |
| Age, y | |||
| 18–30 | 2 (4) | 2 (5) | 2 (6) |
| 31–40 | 26 (52) | 21 (51) | 18 (50) |
| >40 | 22 (44) | 18 (44) | 16 (44) |
| No. children with MCAD deficiency or PKU | |||
| 1 | 37 (76) | 30 (75) | 13 (76) |
| ≥2 | 12 (24) | 10 (25) | 4 (24) |
| Highest level of education completed | |||
| Some or all of high school | 5 (10) | 4 (10) | 4 (11) |
| Technical or vocational training | 2 (4) | 2 (5) | 2 (5) |
| College, university, or graduate school | 43 (86) | 35 (85) | 30 (83) |
| Marital status (married or common law) | 47 (96) | 38 (95) | 34 (97) |
| Health care providers and/or policy advisors | n = 31 | n = 19 | n = 16 |
| Sex, female | 22 (71) | 13 (68) | 12 (75) |
| Age, y | |||
| 18–40 | 6 (19) | 3 (16) | 2 (13) |
| 41–50 | 16 (52) | 11 (58) | 9 (56) |
| >50 | 9 (29) | 5 (26) | 5 (33) |
| Experience, y | |||
| 1–5 | 3 (10) | 2 (11) | 2 (13) |
| 6–10 | 6 (19) | 3 (16) | 2 (13) |
| >10 | 22 (71) | 14 (74) | 12 (75) |
| Canadian province or territory of work | |||
| Western (British Columbia, Alberta, or Manitoba) | 8 (27) | 5 (26) | 5 (31) |
| Central (Ontario or Quebec) | 18 (60) | 11 (58) | 9 (56) |
| Eastern (Nova Scotia) | 4 (13) | 3 (16) | 2 (13) |
| Health care provider or system advisor position | |||
| Physician | 21 (68) | 13 (68) | 10 (63) |
| Other health care provider (eg, nurse, dietitian, psychologist, or genetic counselor) | 9 (29) | 5 (26) | 5 (31) |
| Health technology assessment or knowledge synthesis | 1 (3) | 1 (5) | 1 (6) |
Some respondents did not complete demographic questions (up to a maximum of 4 missing responses across items), so the denominators for each proportion may be slightly different.
During round 2, 12 (MCAD deficiency) and 6 (PKU) new outcomes were added on the basis of round 1 suggestions. From a starting point of 83 (MCAD deficiency) and 97 (PKU) candidate outcomes, after 2 rounds of the Delphi survey, we excluded 56 (MCAD deficiency) and 62 (PKU) outcomes that did not reach consensus in at least 1 of the 2 expert groups (Fig 1). After the third round, we excluded an additional 16 (MCAD deficiency) and 5 (PKU) outcomes. After merging conceptually overlapping outcomes, 20 MCAD deficiency and 30 PKU outcomes remained. We used the survey results to further reduce the PKU outcomes (Supplemental Information) so that 20 outcomes per disease went forward to the workshop (Fig 1; Tables 2 and 3).
Consensus Workshop Voting Results for MCAD Deficiency COS
| MCAD Deficiency Outcome | Total Votes | 1: Outcome Must Be in the Outcome Set, % | 2: Outcome Is a Strong Contender for the Outcome Set, % | 3: Outcome Can Be Left Out of the Outcome Set in Favor of Higher Priority Outcomes, % |
|---|---|---|---|---|
| 1. Metabolic decompensation and its associated complications | 17 | 88 | 12 | 0 |
| 2. Child quality of life | 17 | 82 | 12 | 6 |
| 3. Death | 17 | 65 | 29 | 6 |
| 4. Overall child development | 17 | 35 | 35 | 29 |
| 5. Fasting | 17 | 18 | 65 | 18 |
| 6. Parental experiences with illness care and prevention | 17 | 18 | 59 | 24 |
| 7. Emergency department use | 17 | 18 | 59 | 24 |
| 8. Access to care | 17 | 12 | 65 | 24 |
| 9. Adherence to prescribed or recommended treatment or management strategy | 17 | 6 | 59 | 35 |
| 10. Chronic sequalae of an acute event | 17 | 6 | 59 | 35 |
| 11. Supplementation with rapidly available carbohydrates during acute illness | 17 | 6 | 35 | 59 |
| 12. Age at treatment initiation | 17 | 6 | 18 | 76 |
| 13. Caregiver perceptions of feeling informed about MCAD management | 17 | 0 | 47 | 53 |
| 14. Overall clinician- assessed health status of child | 17 | 0 | 41 | 59 |
| 15. Seizures | 17 | 0 | 41 | 59 |
| 16. Sick day plan | 17 | 0 | 35 | 65 |
| 17. Achievement of treatment goals | 17 | 0 | 35 | 65 |
| 18. Possession or use of an emergency card or letter | 17 | 0 | 29 | 71 |
| 19. Infant feeding difficulties | 17 | 0 | 29 | 71 |
| 20. Carnitine supplementation | 17 | 0 | 12 | 88 |
| MCAD Deficiency Outcome | Total Votes | 1: Outcome Must Be in the Outcome Set, % | 2: Outcome Is a Strong Contender for the Outcome Set, % | 3: Outcome Can Be Left Out of the Outcome Set in Favor of Higher Priority Outcomes, % |
|---|---|---|---|---|
| 1. Metabolic decompensation and its associated complications | 17 | 88 | 12 | 0 |
| 2. Child quality of life | 17 | 82 | 12 | 6 |
| 3. Death | 17 | 65 | 29 | 6 |
| 4. Overall child development | 17 | 35 | 35 | 29 |
| 5. Fasting | 17 | 18 | 65 | 18 |
| 6. Parental experiences with illness care and prevention | 17 | 18 | 59 | 24 |
| 7. Emergency department use | 17 | 18 | 59 | 24 |
| 8. Access to care | 17 | 12 | 65 | 24 |
| 9. Adherence to prescribed or recommended treatment or management strategy | 17 | 6 | 59 | 35 |
| 10. Chronic sequalae of an acute event | 17 | 6 | 59 | 35 |
| 11. Supplementation with rapidly available carbohydrates during acute illness | 17 | 6 | 35 | 59 |
| 12. Age at treatment initiation | 17 | 6 | 18 | 76 |
| 13. Caregiver perceptions of feeling informed about MCAD management | 17 | 0 | 47 | 53 |
| 14. Overall clinician- assessed health status of child | 17 | 0 | 41 | 59 |
| 15. Seizures | 17 | 0 | 41 | 59 |
| 16. Sick day plan | 17 | 0 | 35 | 65 |
| 17. Achievement of treatment goals | 17 | 0 | 35 | 65 |
| 18. Possession or use of an emergency card or letter | 17 | 0 | 29 | 71 |
| 19. Infant feeding difficulties | 17 | 0 | 29 | 71 |
| 20. Carnitine supplementation | 17 | 0 | 12 | 88 |
Consensus Workshop Voting Results for the PKU COS
| PKU Outcome | Total Votes | 1: Outcome Must Be in the Outcome Set, % | 2: Outcome Is a Strong Contender for the Outcome Set, % | 3: Outcome Can Be Left Out of the Outcome Set in Favor of Higher Priority Outcomes, % |
|---|---|---|---|---|
| 1. Child quality of life, including psychosocial and social well-being | 18 | 89 | 11 | 0 |
| 2. Phenylalanine concentration in the blood or other tissues | 18 | 78 | 22 | 0 |
| 3. Cognition and intelligence or IQ | 18 | 39 | 61 | 0 |
| 4. Overall child development and functioning | 18 | 33 | 44 | 22 |
| 5. Executive functioning | 18 | 28 | 67 | 6 |
| 6. Costs of care | 18 | 22 | 39 | 39 |
| 7. Phenylalanine tolerance | 18 | 17 | 61 | 22 |
| 8. Adherence to PKU diet | 18 | 17 | 50 | 33 |
| 9. Achievement of treatment goals | 18 | 17 | 50 | 33 |
| 10. Child experiences and confidence with disease management, including diet | 18 | 11 | 56 | 33 |
| 11. Liberalization of PKU diet | 18 | 6 | 56 | 39 |
| 12. Dietary intake of phenylalanine | 18 | 6 | 50 | 44 |
| 13. Internalizing mental health or mood disorders and associated symptoms | 18 | 6 | 33 | 61 |
| 14. Behavior problems and externalizing mental health or behavior disorders | 18 | 0 | 50 | 50 |
| 15. School achievement and learning | 18 | 0 | 39 | 61 |
| 16. ADHD or ADHD-like symptoms | 18 | 0 | 33 | 67 |
| 17. Dietary intake of protein | 18 | 0 | 33 | 67 |
| 18. Prescriptions or use of medications to manage PKU | 18 | 0 | 28 | 72 |
| 19. Dietary intake of medical foods or formula, modified low-protein foods, and protein substitutes | 18 | 0 | 28 | 72 |
| 20. Age at treatment initiation | 18 | 0 | 11 | 89 |
| PKU Outcome | Total Votes | 1: Outcome Must Be in the Outcome Set, % | 2: Outcome Is a Strong Contender for the Outcome Set, % | 3: Outcome Can Be Left Out of the Outcome Set in Favor of Higher Priority Outcomes, % |
|---|---|---|---|---|
| 1. Child quality of life, including psychosocial and social well-being | 18 | 89 | 11 | 0 |
| 2. Phenylalanine concentration in the blood or other tissues | 18 | 78 | 22 | 0 |
| 3. Cognition and intelligence or IQ | 18 | 39 | 61 | 0 |
| 4. Overall child development and functioning | 18 | 33 | 44 | 22 |
| 5. Executive functioning | 18 | 28 | 67 | 6 |
| 6. Costs of care | 18 | 22 | 39 | 39 |
| 7. Phenylalanine tolerance | 18 | 17 | 61 | 22 |
| 8. Adherence to PKU diet | 18 | 17 | 50 | 33 |
| 9. Achievement of treatment goals | 18 | 17 | 50 | 33 |
| 10. Child experiences and confidence with disease management, including diet | 18 | 11 | 56 | 33 |
| 11. Liberalization of PKU diet | 18 | 6 | 56 | 39 |
| 12. Dietary intake of phenylalanine | 18 | 6 | 50 | 44 |
| 13. Internalizing mental health or mood disorders and associated symptoms | 18 | 6 | 33 | 61 |
| 14. Behavior problems and externalizing mental health or behavior disorders | 18 | 0 | 50 | 50 |
| 15. School achievement and learning | 18 | 0 | 39 | 61 |
| 16. ADHD or ADHD-like symptoms | 18 | 0 | 33 | 67 |
| 17. Dietary intake of protein | 18 | 0 | 33 | 67 |
| 18. Prescriptions or use of medications to manage PKU | 18 | 0 | 28 | 72 |
| 19. Dietary intake of medical foods or formula, modified low-protein foods, and protein substitutes | 18 | 0 | 28 | 72 |
| 20. Age at treatment initiation | 18 | 0 | 11 | 89 |
ADHD, attention-deficit/hyperactivity disorder.
Consensus Workshop
A total of 6 patient or family member investigators and advisors, 7 metabolic clinician investigators, and 5 methodologists or health systems investigators participated in person at the consensus workshop. Six individuals participated via webinar. Seventeen in-person participants voted on the outcomes to be included in the MCAD deficiency COS, reaching agreement on 7 outcomes for inclusion and 12 for exclusion (Table 2). One outcome, access to care, required further discussion after the workshop and was eventually included. Eighteen in-person participants voted on the outcomes to be included in the PKU COS, reaching agreement on 9 outcomes for inclusion and 10 for exclusion (Table 3). One outcome, child experiences and confidence with disease management including diet, required further discussion after the workshop and was eventually included. Two outcomes (adherence to PKU diet and achievement of treatment goals) were merged after the workshop on the basis of agreement that treatment goals closely center on dietary management. The final COSs included 8 outcomes for MCAD deficiency and 9 for PKU (Tables 4 and 5).
MCAD Deficiency COS and Measurement Recommendations
| Core Area | Outcome | Definition | Recommended OMIs and Operationalization |
|---|---|---|---|
| Death | Death | The end of a person’s life | Administrative data |
| Growth and development | Overall child development | Process of change across a combination of physical, emotional, cognitive, social, and language areas over the course of childhood | Vineland Adaptive Behavior Scales, Third Edition29 |
| Fasting | Not eating or drinking for a period of time | (1) The agreed or prescribed safe fasting period (ie, a shared understanding between the clinician and patient or caregiver on the amount of time the child can safely fast before being at risk for metabolic decompensation) and (2) the maximum overnight fasting period when the patient is well (ie, does not have an intercurrent illness), as reported by patients or parents | |
| Life impact | Child quality of life | A general concept of well-being that refers to a combination of various life aspects (eg, health, emotional, and social), as it relates to MCAD deficiency, and is assessed from the child’s perspective | Pediatric Quality of Life Inventory30 |
| Parental experiences with illness care and prevention | How parents think and feel about their involvement and observations related to taking care of a child with MCAD deficiency, including how they feel about strategies for preventing acute crises | Pediatric Inventory for Parents31,32 | |
| Pathophysiological manifestations | Metabolic decompensation | A serious health condition caused by low blood sugar and the buildup of toxic substances in the blood, often triggered by things like illness or fasting in people with MCAD deficiency, and is also called a metabolic crisis | The presence of hypoglycemia and/or encephalopathy with at least 1 other biochemical marker showing an abnormal value |
| Resource use | Access to care | The extent or ease with which patients or communities are able to use appropriate health services in proportion to their needs, with a specific focus on health system capacity and the resources required to deliver care | The operational definition is likely to be context and/or study-dependent |
| Emergency department use | Measures related to the frequency or nature of visits to the emergency department at the hospital | Use parent report and/or administrative data |
| Core Area | Outcome | Definition | Recommended OMIs and Operationalization |
|---|---|---|---|
| Death | Death | The end of a person’s life | Administrative data |
| Growth and development | Overall child development | Process of change across a combination of physical, emotional, cognitive, social, and language areas over the course of childhood | Vineland Adaptive Behavior Scales, Third Edition29 |
| Fasting | Not eating or drinking for a period of time | (1) The agreed or prescribed safe fasting period (ie, a shared understanding between the clinician and patient or caregiver on the amount of time the child can safely fast before being at risk for metabolic decompensation) and (2) the maximum overnight fasting period when the patient is well (ie, does not have an intercurrent illness), as reported by patients or parents | |
| Life impact | Child quality of life | A general concept of well-being that refers to a combination of various life aspects (eg, health, emotional, and social), as it relates to MCAD deficiency, and is assessed from the child’s perspective | Pediatric Quality of Life Inventory30 |
| Parental experiences with illness care and prevention | How parents think and feel about their involvement and observations related to taking care of a child with MCAD deficiency, including how they feel about strategies for preventing acute crises | Pediatric Inventory for Parents31,32 | |
| Pathophysiological manifestations | Metabolic decompensation | A serious health condition caused by low blood sugar and the buildup of toxic substances in the blood, often triggered by things like illness or fasting in people with MCAD deficiency, and is also called a metabolic crisis | The presence of hypoglycemia and/or encephalopathy with at least 1 other biochemical marker showing an abnormal value |
| Resource use | Access to care | The extent or ease with which patients or communities are able to use appropriate health services in proportion to their needs, with a specific focus on health system capacity and the resources required to deliver care | The operational definition is likely to be context and/or study-dependent |
| Emergency department use | Measures related to the frequency or nature of visits to the emergency department at the hospital | Use parent report and/or administrative data |
See Supplemental Information for further information regarding selection of potential OMIs.
PKU COS and Measurement Recommendations
| Core Area | Outcome | Definition | Recommended OMI(s) and Operationalization |
|---|---|---|---|
| Growth and development | Cognition and intelligence and/or IQ (for children aged ≥4 y) | A child’s learning abilities related to the activity and development of the brain | Stanford-Binet Intelligence Scales, Fifth Edition, routing tests33 ; Raven’s Progressive Matrices34,35,a |
| Overall child development and functioning | Process of change across a combination of physical, emotional, cognitive, social, and language areas over the course of childhood, including ability to perform age-appropriate daily activities and functions | Vineland Adaptive Behavior Scales, Third Edition29 | |
| Executive Functioning (for children aged ≥6 y) | Deliberate, conscious control over one’s own thoughts, actions, and emotions | Behavior Rating Inventory of Executive Functioning, Second Edition36 ; selected Cambridge Neuropsychological Test Automated Battery tasks37 ; and selected Amsterdam Neuropsychological Tasks38,a | |
| Life impact | Child quality of life, including psychosocial and social well-being | A general concept of well-being that refers to a combination of various life aspects (eg, health, emotional, and social), as it relates to PKU, and is assessed from the child’s perspective | Pediatric Quality of Life Inventory30 |
| Phenylalanine tolerance | Indicators related to the amount of phenylalanine a person can take in | Define phenylalanine tolerance as the maximum phenylalanine intake (milligrams per day) that keeps blood phenylalanine concentration within the treatment range of 120–360 μmol/L as recommended by ACMG guidelines39,40 | |
| Adherence to PKU treatment plan | The extent to which someone sticks to a special diet (eg, formula, medical food, or low-protein foods) or other aspects of the treatment plan prescribed specifically for someone with PKU | (1) Actual versus required number of blood samples sent for analysis; (2) actual versus prescribed phenylalanine intake (by using 3-d food diaries, diet app, or 24-h recall); and (3) if the child is taking Kuvan, self-reported compliance with Kuvan prescription | |
| Child understanding of and self-efficacy with management of PKU (as appropriate for age) | Refers to a child’s awareness, attitudes, and health beliefs regarding PKU and their confidence in implementing the behavior necessary to achieve desired health outcomes | Pediatric Rating of Chronic Illness Self-Efficacy41 | |
| Pathophysiological manifestations | Phenylalanine concentration in the blood or other tissues | How much phenylalanine there is in the blood or other tissues and whether the level is consistent with the targeted level | Blood phenylalanine should be reported in micromoles per liter |
| Resource use | Costs of care | The financial costs of health care services | Direct costs of care that are associated with the treatment being evaluated |
| Core Area | Outcome | Definition | Recommended OMI(s) and Operationalization |
|---|---|---|---|
| Growth and development | Cognition and intelligence and/or IQ (for children aged ≥4 y) | A child’s learning abilities related to the activity and development of the brain | Stanford-Binet Intelligence Scales, Fifth Edition, routing tests33 ; Raven’s Progressive Matrices34,35,a |
| Overall child development and functioning | Process of change across a combination of physical, emotional, cognitive, social, and language areas over the course of childhood, including ability to perform age-appropriate daily activities and functions | Vineland Adaptive Behavior Scales, Third Edition29 | |
| Executive Functioning (for children aged ≥6 y) | Deliberate, conscious control over one’s own thoughts, actions, and emotions | Behavior Rating Inventory of Executive Functioning, Second Edition36 ; selected Cambridge Neuropsychological Test Automated Battery tasks37 ; and selected Amsterdam Neuropsychological Tasks38,a | |
| Life impact | Child quality of life, including psychosocial and social well-being | A general concept of well-being that refers to a combination of various life aspects (eg, health, emotional, and social), as it relates to PKU, and is assessed from the child’s perspective | Pediatric Quality of Life Inventory30 |
| Phenylalanine tolerance | Indicators related to the amount of phenylalanine a person can take in | Define phenylalanine tolerance as the maximum phenylalanine intake (milligrams per day) that keeps blood phenylalanine concentration within the treatment range of 120–360 μmol/L as recommended by ACMG guidelines39,40 | |
| Adherence to PKU treatment plan | The extent to which someone sticks to a special diet (eg, formula, medical food, or low-protein foods) or other aspects of the treatment plan prescribed specifically for someone with PKU | (1) Actual versus required number of blood samples sent for analysis; (2) actual versus prescribed phenylalanine intake (by using 3-d food diaries, diet app, or 24-h recall); and (3) if the child is taking Kuvan, self-reported compliance with Kuvan prescription | |
| Child understanding of and self-efficacy with management of PKU (as appropriate for age) | Refers to a child’s awareness, attitudes, and health beliefs regarding PKU and their confidence in implementing the behavior necessary to achieve desired health outcomes | Pediatric Rating of Chronic Illness Self-Efficacy41 | |
| Pathophysiological manifestations | Phenylalanine concentration in the blood or other tissues | How much phenylalanine there is in the blood or other tissues and whether the level is consistent with the targeted level | Blood phenylalanine should be reported in micromoles per liter |
| Resource use | Costs of care | The financial costs of health care services | Direct costs of care that are associated with the treatment being evaluated |
See Supplemental Information for further information regarding selection of potential OMIs. ACMG, The American College of Medical Genetics and Genomics.
Multiple OMIs are recommended for certain outcomes in recognition that they may provide complementary information (eg, different measures of executive functioning) and/or because we recognize potential resource and language constraints (eg, measures of cognition and intelligence). We also acknowledge that it may be difficult to achieve harmonization in measures of cognition and intelligence, given that the feasibility of tool implementation may depend on the clinical or research context. Please refer to the Supplemental Information for additional information.
OMI Selection
Recommendations for OMIs were drafted and revised after review by study investigators and FAF members (Supplemental Information; Tables 4 and 5). For some outcomes (eg, cognition and intelligence), we recommended some flexibility in OMI selection to accommodate potential implementation constraints and/or because different OMIs may provide complementary or context-dependent information.
Discussion
Patients and caregivers, health professionals, methodologists, and health system experts reached consensus on pediatric COSs for MCAD deficiency (8 outcomes) and PKU (9 outcomes). These outcomes span established core areas,18,19 including growth and development, life impact, pathophysiological manifestations, resource use, and, for MCAD deficiency, death. Some outcomes are common across the 2 IMDs (eg, child quality of life and overall child development) and are included in other published pediatric COSs as well,21–24 facilitating comparisons of outcomes across studies of children with chronic health conditions. Other core outcomes are specific to the particular manifestations of the individual diseases. For example, MCAD deficiency carries a risk of acute life-threatening episodes. Thus, death, metabolic decompensation, and emergency department use are part of the COS. PKU has important chronic neuropsychological manifestations that may be related to metabolic control by diet and other interventions, so cognition and intelligence, executive functioning, and indicators of adherence to treatment are core outcomes. With the inclusion of these disease-specific outcomes, we corroborate the findings of our previous review, whereby death was the most frequently reported outcome in previous studies related to pediatric MCAD deficiency, with acute metabolic decompensation also commonly reported, and blood phenylalanine was the most frequently reported outcome in studies of pediatric PKU.9 Our previous review identified a scarcity of patient- or family-centered and patient- or family-reported outcomes in previous studies of PKU and inconsistency in the selection and reporting of such outcomes for both MCAD deficiency and PKU.9 With the inclusion of outcomes in the life impact core area of our COSs, such as child quality of life (both conditions), parental experiences with illness care and prevention (MCAD deficiency), and child understanding of and self-efficacy with disease management (PKU), we address this gap and help to ensure that, in future research, researchers incorporate outcomes that are meaningful to patients and families. The transferability of our COSs to rarer IMDs that have similar features to MCAD deficiency or PKU (eg, those that are characterized by risk of acute crises or that require strict dietary control) requires further investigation.
To further standardize outcome data collection and analysis, we have recommended OMIs for measuring these core outcomes in future studies. This is particularly important given that lack of agreed on valid OMIs has been identified as a barrier to COS uptake.25 Promoting uptake of our COSs by researchers and policy decision-makers is a critical next step. In addition to incorporating OMIs, strategies recommended for increasing COS uptake include the following: the involvement of funders and journal editors in encouraging COS use,26,27 promoting awareness of COSs (eg, through social media, patient organizations, and/or outcome repositories),25,28 integrating outcomes into existing infrastructure (eg, registries),27 and broad stakeholder engagement and ownership.27 With respect to engagement, as part of our knowledge translation strategy, we will seek feedback from the international community through publication and presentation to audiences that include professional and patient organizations outside of Canada, which may result in changes to the outcomes or OMIs. We are also planning the implementation of the COSs in Canadian pediatric centers and will publish our experience, including barriers and facilitators to implementation. Finally, these COSs should be updated as new evidence emerges in this rapidly evolving field.
This study has several strengths. The methods were guided by a published protocol10 based on established COS methodology. Active collaboration with patient partners and FAF members ensured meaningful input and relevance. However, there were also important limitations. We did not include the perspectives of children with MCAD deficiency or PKU; although the COSs are designed for children aged ≤12 years, in future research, researchers may elicit age-appropriate feedback from children, and, if necessary, the COSs could be updated. Only 1 policy advisor participated in the Delphi survey, representing an important group from which to seek further future feedback, given the central role of regulators and payers with respect to trial end points considered in policy decision-making. All study participants were Canadian, and, as noted, the COSs will benefit from further feedback we seek from researchers, health care professionals, and patients in the international community. Finally, although harmonized measurement of the outcomes across research studies and treatment centers is feasible for clinical and laboratory-based OMIs and those that involve self-complete questionnaires, it may be challenging to adopt standardized OMIs for cognitive tests, which frequently require specific qualifications for administration, are costly, and may not be available in multiple languages. In such cases, we have recommended multiple OMIs to add flexibility. Although we acknowledge that resource constraints may be a barrier to measuring some outcomes, the rigorous multistakeholder consensus approach we used underscores the importance of the selected outcomes for monitoring the health of children with MCAD deficiency and PKU.
Conclusions
We present the first COSs for 2 pediatric IMDs, MCAD deficiency and PKU. Implementation of these COSs in future studies will help to make the best use of limited rare disease research resources because harmonized outcome selection and reporting facilitates comparison and synthesis across studies. Our emphasis on patient and family-centered outcomes also addresses a key research gap and will help ensure that, in future studies, researchers incorporate meaningful end points.
Acknowledgments
We thank the members of the FAF, who contributed their expertise through lived experience to all aspects of this study. We also thank all participants in the Delphi survey and those who participated in the consensus workshop. We also thank all investigators that belong to the Canadian Inherited Metabolic Diseases Research Network who participated in foundational work that led to this study.
Mr Pugliese acquired the data, analyzed and interpreted the data, and drafted the article; Ms Tingley designed the study, acquired the data, analyzed and interpreted the data, and drafted the article; Ms Chow, Mr Rahman, Ms Tessier, Mr Iverson, and Ms Lamoureux acquired the data, analyzed and interpreted the data, and critically revised the article; Ms Pallone and Ms Smith conceptualized and designed the study, co-led the patient engagement strategy as patient partners, analyzed and interpreted the data, and critically revised the article; Drs Chakraborty, Mitchell, Stockler, Rockman-Greenberg, and Schulze conceptualized and designed the study, acquired the data, analyzed and interpreted the data, and critically revised the article; Drs Geraghty and M. Potter acquired the data, analyzed and interpreted the data, and critically revised the article; Drs Irwin and Butcher analyzed and interpreted the data and critically revised the article; Drs Nicholls, Offringa, Clifford, Coyle, Korngut, A. MacKenzie, and Taljaard conceptualized and designed the study, analyzed and interpreted the data, and critically revised the article; Drs Dyack, Jain Ghai, J. MacKenzie, Prasad, Sparkes, Trakadis, and Walia acquired the data, interpreted the data, and critically revised the article; Dr Hutton, Ms Skidmore, Ms Duddy, and Drs Karp and Kronick interpreted the data and critically revised the article; Ms Paik and Ms Tao acquired the data and critically revised the article; Dr B. K. Potter conceptualized and designed the study, acquired the data, analyzed and interpreted the data, and drafted the article; and all authors approved the final manuscript as submitted and agree to be accountable for all aspects of the work.
This trial has been registered with PROSPERO (https://www.crd.york.ac.uk/prospero/) (identifier CRD42017073524).
FUNDING: Funded by the Canadian Institutes of Health Research (grant 151614). The views expressed in this article are those of the authors and do not necessarily reflect those of the Canadian Institutes of Health Research or the Government of Canada.
Comments