Vikki Gale

Purpose Establishing the comprehensibility of patient-reported outcome measures (PROMs) in quality of life research is essential. Cognitive interviews are recommended as a ‘gold standard’ for evaluating comprehensibility among adult populations but are not routinely used with young children (≤7 years). The current study therefore aimed to evaluate the feasibility of cognitive interviewing using traditional and adapted methods with children aged 6-7 years to evaluate PROM item comprehensibility. Methods Fourteen children (6-7 years) with a range of diagnosed health conditions participated in individual cognitive interviews. Each child answered six mock PROM items (physical, psychological, and social health-related quality of life domains) and concurrent verbal probes were used to evaluate item comprehensibility. Interviews were audio recorded and transcribed verbatim. Transcripts were analysed using a novel Comprehensibility Continuum which coded the extent of alignment between children’s explanations of items and intended meanings. Results Cognitive interviews were successful; extent of comprehensibility could be determined for 83/84 (99%) item discussions. Most items were comprehensible, with children describing the intended item meaning for 74/84 (88%) items evidenced by contextual examples and/or de-contextual definitions in children’s responses to verbal probes. Three items (‘walk’, ‘sad’, and ‘made fun of’) were identified as requiring further testing and/or refinement, where a lower percentage of discussions contained evidence of intended item meaning. Conclusion Despite previous uncertainty, this study demonstrates how methodological challenges can be addressed to enable young children’s participation in cognitive interviews evaluating item comprehensibility, ultimately contributing to the accurate measurement of young children’s health outcomes in healthcare and research.

Background Qualitative research during the development/testing of Patient Reported Outcome Measures (PROMs) is recommended to support content validity. However, it is unclear if and how young children (≤ 7 years) can be involved in this research because of their unique cognitive needs. Objectives Here we investigate the involvement of children (≤ 7 years) in qualitative research for PROM development/testing. This review aimed to identify (1) which stages of qualitative PROM development children ≤ 7 years had been involved in, (2) which subjective health concepts had been explored within qualitative PROM development with this age group, and (3) which qualitative methods had been reported and how these compared with existing methodological recommendations. Methods This scoping review systematically searched three electronic databases (searches re-run prior to final analysis on 29 June 2022) with no date restrictions. Included studies had samples of at least 75% aged ≤ 7 years or reported distinct qualitative methods for children ≤ 7 years in primary qualitative research to support concept elicitation or PROM development/testing. Articles not in English and PROMs that did not enable children ≤ 7 years to self-report were excluded. Data on study type, subjective health and qualitative methods were extracted and synthesised descriptively. Methods were compared with recommendations from guidance. Results Of 19 included studies, 15 reported concept elicitation research and 4 reported cognitive interviewing. Most explored quality of life (QoL)/health-related quality of life (HRQoL). Some concept elicitation studies reported that creative/participatory activities had supported children’s engagement, but results and reporting detail varied considerably across studies. Cognitive interviewing studies reported less methodological detail and fewer methods adapted for young children compared with concept elicitation studies. They were limited in scope regarding assessments of content validity, mostly focussing on clarity while relevance and comprehensiveness were explored less. Discussion Creative/participatory activities may be beneficial in concept elicitation research with children ≤ 7 years, but future research needs to explore what contributes to the success of young children’s involvement and how researchers can adopt flexible methods. Cognitive interviews with young children are limited in frequency, scope and reported methodological detail, potentially impacting PROM content validity for this age group. Without detailed reporting, it is not possible to determine the feasibility and usefulness of children’s (≤ 7 years) involvement in qualitative research to support PROM development and assessment.

Purpose Evidence of comprehensibility is frequently required during the development of patient reported outcome measures (PROMs); the respondent’s interpretation of PROM items needs to align with intended meanings. Cognitive interviews are recommended for investigating PROM comprehensibility, yet guidance for analysis is lacking. Consequently, the quality and trustworthiness of cognitive interview data and analysis is threatened, as there is no clear procedure detailing how analysts can systematically, and consistently, identify evidence that respondent interpretations align/misalign with intended meanings. Methods This paper presents a novel, structured approach to comprehensibility analysis - the ‘Comprehensibility Continuum’ – that builds upon existing cognitive interview guidance. Results The Comprehensibility Continuum comprises a structured rating scale to code depth of alignment between intended item meaning and respondent interpretation and consists of five main stages: before cognitive interviews are conducted, researchers must (1) Define intended meanings of PROM items; and (2) Determine comprehensibility thresholds for both participant- and item-level. After conducting interviews, they (3) Prepare data by transcribing interviews ‘intelligent’ verbatim; (4) Code transcripts using the Comprehensibility Continuum scale in iterative sets, assigning an overall code for each item at participant-level; and (5) Compare participant-level codes across all participants to determine overall item comprehensibility, such that decisions can be made to retain, modify, or remove items. Conclusion Quality in qualitative data analysis is achieved through rigorous methods that are clearly described and justified. Given insufficiency in guidelines, cognitive interviewers must reflect on how best to demonstrate PROM comprehensibility systematically and consistently from interview data, and the Comprehensibility Continuum method offers a potential solution.

Background Recommendations suggest that children need to be ≥8 years-old to participate in concept elicitation (CE) and cognitive interviewing (CI) when developing patient reported outcome measures (PROMs). However, these recommendations have not been subject to thorough scrutiny and recent evidence suggests that younger children may be enabled to participate. This study audited current opinions of PROM developers regarding the feasibility of conducting CE and CI research with children. Methodology An online survey was developed to capture PROM developers’ perspectives, recruited from existing networks (UK PROMs, International Society for Quality of Life Research) and outcomes research groups from English-speaking countries between August-November 2024. Survey questions explored the ages from which developers considered it feasible to include children in CE and CI research, their previous experiences conducting CE/CI research with children, and respondents’ background experiences with children. Results were analysed descriptively, and exploratory comparisons were made based on developers’ characteristics. Results Fifty-eight responses were analysed. The mean youngest ages considered feasible to include children in CE and CI research were 6.66 years and 7.36 years, respectively. The mean youngest ages respondents reported involving children in CE and CI research in practice were 7.67 years and 8.13 years, respectively. Concern that children would have insufficient cognitive and/or linguistic skills was the most often endorsed reason for considering the involvement of younger children to be infeasible. Respondents who had recent parental experience with younger children tended to consider it feasible to include children from younger ages. Those who had conducted CI with children considered it feasible to include children in CI from younger ages. Opposingly, those who had conducted CE with children considered it less feasible to include younger children in CE research. Conclusions In-line with established precedent, PROM developers included children from ~8 years-old in CE and CI research, while in principle considering it feasible to include younger ages. Reasons for including (or not including) certain age groups in CE and CI research need critical evaluation and PROM developers may wish to consider ways in which more inclusive opportunities for younger children can be provided.