
==== Front
BMJ Open
BMJ Open
bmjopen
bmjopen
BMJ Open
2044-6055
BMJ Publishing Group BMA House, Tavistock Square, London, WC1H 9JR

39260865
10.1136/bmjopen-2024-084935
bmjopen-2024-084935
Protocol
Oncology
1717
1506
Data collection methods for patient-reported outcome measures in cancer randomised controlled trials: a protocol for a rapid scoping review
https://twitter.com/JLehmann_IBK
http://orcid.org/0000-0002-4670-7517
Lehmann Jens 1jens.lehmann@i-med.ac.at

Krepper Daniela 1daniela.krepper@i-med.ac.at

Pe Madeline 2madeline.pe@eortc.org

Kuliś Dagmara 2dagmara.kulis@eortc.org

Giesinger Johannes M 1johannes.giesinger@i-med.ac.at

Sztankay Monika 1Monika.Sztankay@tirol-kliniken.at

Kern Scottie 3skern@c-path.org

Fitzsimmons Deborah 4d.fitzsimmons@swansea.ac.uk

Holzner Bernhard 1bernhard.holzner@tirol-kliniken.at

on behalf of the EORTC Quality of Life Group
1 University Hospital of Psychiatry II, Medical University of Innsbruck, Innsbruck, Austria
2 European Organisation for Research and Treatment of Cancer, Brussels, Belgium
3 Electronic Clinical Outcome Assessment Consortium, Critical Path Institute, Tucson, Arizona, USA
4 Swansea Centre for Health Economics, Swansea University, Swansea, UK
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

BH owns intellectual property rights to the software CHES, a software for electronic data capture. JL reports consultancy for Evaluation Software Development Gmbh. All remaining authors declare no conflicts of interest.

Dr; jens.lehmann@i-med.ac.at
2024
10 9 2024
14 9 e08493501 2 2024
16 8 2024
Copyright © Author(s) (or their employer(s)) 2024. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.
2024
https://creativecommons.org/licenses/by-nc/4.0/ This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

Abstract

Background

There are different modes and ways to assess patient-reported outcomes (PROs) in clinical trials. However, there is little systematic information on how often different modes of assessment (MOA) are used in cancer clinical trials and how exactly assessments are conducted. The goal of this scoping review is to gain an understanding of the MOA and data management of PROs in cancer randomised controlled trials (RCTs) and the reporting quality thereof.

Methods and analysis

This scoping review protocol follows the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. Relevant trials will be identified via their indexed publications. We will search PubMed for RCTs conducted in cancer populations that evaluate a biomedical treatment with a PRO endpoint. Trials with publications published between January 2019 and November 2023 will be included. Two independent reviewers will review the references for both the abstract and full-text screening. We will extract data from the publications from a trial and the trial protocol if a protocol can be traced. Data will be summarised at the trial level. We will focus on a descriptive analysis of the MOA of PROs and on the relative frequencies of the different MOA. We will also evaluate the quality of reporting for the relevant SPIRIT and CONSORT guidelines that refer to the assessment of PROs in trials. Due to the scoping nature of our review, we will not perform a dedicated quality assessment of all trials.

Ethics and dissemination

The proposed review is based on secondary, published data. Hence, no ethics review is necessary. The review is part of an ongoing project on the use of electronic data capture methods in cancer clinical trials. The findings from the review will support the project and contribute to synthesising guidance to ultimately improve the (electronic) measurement of patient-reported outcome measures in clinical trials.

Patient Reported Outcome Measures
ONCOLOGY
Clinical trials
EORTC Quality of Life Group 008-2023
==== Body
pmcStrengths and limitations of this study

This scoping review protocol describes the methodology for a comprehensive exploration of how patient-reported outcome measures (PROMs) are assessed in cancer randomised controlled trials published on PubMed, with a particular emphasis on the mode of assessment (MOA) and the quality of reporting.

The synthesis of how different MOAs are used in trials may further our understanding of how PROMs are assessed in trials and which trial characteristics are associated with different MOAs.

The variability in how MOAs are reported (or not reported) may prevent quantitative analyses.

Background

Patient-reported outcomes (PROs) are increasingly assessed electronically in clinical trials, often referred to as ePRO data capture. Compared with traditional paper-based assessment, ePRO assessment holds the promise of improved data quality, more control over the correct assessment and timing of patient-reported outcome measures (PROMs) and shorter turnaround times, as well as improved audit capabilities.15 Consequently, ePROs are the preferred assessment method as recommended by the Centre for Medical Technology Policy,6 in the European Society for Medical Oncology practice guidelines,7 and are discussed by regulatory bodies like the US Food and Drug Administration (FDA) and the European Medicines Agency.8 However, there are also potential drawbacks to ePRO data capture that may limit usefulness, including limited computer literacy in some populations, problems if the ePRO software is difficult to use for patients or study personnel or network dependency to function.

There is little systematic information on how often ePRO assessments are being used in cancer clinical trials and how exactly assessments are conducted. Trials may use paper-pencil PROMs, electronic PROMs through ePRO data capture (which can also include interactive voice response systems) or even a mixture of both. Systematic information on the mode of assessment (MOA) and data management of PROMs in clinical trials is missing to inform the ongoing discussions about how PROMs should or should not be used in different scenarios (eg, 914).

Information about data management and the MOA of PROMs are recommended reporting items in guidelines for trial protocols (Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) and Guidelines for Inclusion of Patient-Reported Outcomes in Clinical Trial Protocols (SPIRIT-PRO))[15,16,15 16 as well as trial publications (Consolidated Standards of Reporting Trials (CONSORT) and Reporting of patient-reported outcomes in randomised trials: the CONSORT-PRO extension17 18). However, such information is often not sufficiently reported in protocols and publications, and there is ample evidence demonstrating a slow uptake of conformity with reporting guidelines in the literature in certain areas.19 20

The goal of this scoping review is to develop a more comprehensive picture of the MOA and management of data derived from PROs in cancer randomised controlled trials (RCTs) and the reporting quality thereof. Using the scoping review methodology to explore existing literature, we aim to fill the current knowledge gap regarding the prevalence of different practices of PROM assessment in cancer clinical trials. Additionally, we seek to identify areas where adherence to established reporting guidelines, that is, SPIRIT-PRO and CONSORT-PRO, can be improved.

Methods

This paper is a protocol for a scoping review. The protocol is reported in accordance with the Preferred Reporting Items for Systematic reviews and Meta Analyses extension for Scoping Reviews checklist.21 Our approach is adapted from the framework originally proposed by Arksey and O'Malley,22 which has received widespread recognition and refinement in the methodological literature.2325

A scoping review aims to explore and chart the fundamental concepts within a research field, along with the primary sources and various types of available evidence. Consequently, scoping reviews often investigate overarching research questions, and they do not engage in formal quality assessment. This review aims to gain a comprehensive picture of the MOA of PROMs and PRO data management in cancer RCTs and the reporting thereof. Hence, the methodology of a scoping review is followed to answer our research questions. Due to the nature of the research questions asked (see below), we will not conduct a systematic assessment of trial quality (eg, risk of bias). We argue that the reporting of the MOA of PROMs in trials is unlikely to be linked to a trial’s risk of bias and hence that this kind of assessment is unlikely to be relevant to our scope. However, the literature search approach is still systematic.

Identifying the research questions

Research question 1: What is the frequency of different MOA used to assess PROMs in cancer RCTs?

This review aims to provide an overview of the MOA of PROMs in cancer RCTs in light of existing guidance and regulatory recommendations. For example, the FDA generally recommends electronic source data capture, which includes PROMs, to harmonise and modernise clinical trials.26 Hence, the first research question is descriptive and focuses on the frequency of different MOA for PROMs among trials that report assessing PROMs.

Research question 2: Are there study characteristics that are linked to different MOA?

Second, we aim to evaluate if any specific study characteristics are associated with different types of MOA. For example, multicentre and international studies might be more prone to rely on a centralised ePRO data collection. Similarly, it might be too costly to implement ePRO data collection for academic trials which commonly have less funding compared with industry trials.

Research question 3: To what level does the reporting of data management and MOA conform to the CONSORT and SPIRIT recommendations?

Third, we aim to assess the reporting conformity of publications and protocols to the relevant EQUATOR guidelines in terms of data management and the MOA. For protocols, these are the SPIRIT guideline15 and the SPIRIT-PRO extension.16 For publications, these are the CONSORT guideline17 and the CONSORT-PRO extension.18 The relevant items are listed in the data extraction sheet in online supplemental material 1.

Research question 4: Do any trials report using PRO-based alerts during the trial to support patients’ clinical care?

Finally, we aim to identify whether any trials using electronic PROM assessment used the PRO data during the trial. In other words, whether PROM data were actively reviewed by healthcare professionals involved in the treatment of the study patient, for example, using alerts. There are good (ethical) arguments for not only using PRO data for analyses after the trial but for also monitoring PRO data during trials to maximise patient safety and well-being.27 28 However, we estimate that the number of trials using these advanced levels of PROM-based clinical alerts will be few.

Study selection

We will search for RCTs published between January 2019 and November 2023 (date of protocol inception). Two independent reviewers will decide on the eligibility of references on two subsequent levels (ie, level 1: abstract screening and level 2: full-text screening) using the literature software DistillerSR.29 Inconsistent ratings will be resolved by the two raters. In case of persistent disagreement, a third reviewer will be consulted. The following inclusion and exclusion criteria will be applied to decide on the eligibility of trial publications.

The inclusion criteria were as follows:

Study design: RCT.

Overall sample size (randomised patients) >50.

Cancer population (bladder, breast, colorectal, gynaecological, lung and prostate).

Biomedical treatment evaluated (eg, chemotherapy, radiotherapy, surgery, targeted therapy, immunotherapy and hormonal therapy), therefore, excluding psychosocial, educational and complementary medicine and exercise and behavioural interventions.

At least one PRO endpoint (primary, secondary or exploratory) using any European Organization for Research and Treatment of Cancer (EORTC) or Functional Assessment of Chronic Illness Therapy (FACIT) questionnaire.

Language: English.

The exclusion criteria were as follows:

Mixed population (ie, no mix of cancer diagnoses within one trial).

Screening study.

Survivorship study.

No full text available.

We focus on the abovementioned six tumour groups as they are the most common solid tumour types per incidence.30 Further, we only include studies with a PRO endpoint using either an EORTC or a FACIT questionnaire, as they are, by far, the two most common questionnaires used in cancer clinical trials.31 The search strategy and search strings were adapted from the PROMOTION registry (https://promotion.gimema.it/, 32); see online supplemental material 2 for the complete search string.

Data sources

Relevant trials will be identified via their publications. As outlined in the Study selection section, we will search for trials with a PRO endpoint. To keep the overall workload manageable, only trial publications indexed in PubMed will be considered. PubMed coverage of studies for lung, colorectal and gynaecological cancers is over 80% and even at 90% for breast cancer.33

Detailed information on the MOA and data management is unlikely to be frequently found in trial publications only (eg, due to word count limitations). Moreover, the authors may sometimes only report that ‘a questionnaire was assessed’ but not report the MOA as paper-based assessment is often perceived as the norm. Trial protocols might be more explicit in describing the MOA and data management. Hence, we will undertake reasonable efforts to source trial protocols. For each trial that is included in the analysis, we check the following sources to trace a protocol:

Searching Clinicaltrials.gov and EduraCT for an uploaded protocol.

If not obtained in the previous step: simple web search of the trial using the trial acronym and the registration number where available (eg, Open Science Framework and other trial databases).

If not obtained in the previous step: contacting the corresponding author of the trial’s primary publication (once).

Extracting and charting the data

We will extract data from both the publications from a trial, as well as the protocol if a protocol can be traced, again applying a double-review procedure in DistillerSR.29 Where feasible and of relevance, we will also distinguish between the origin of the data (from publication(s) or protocol). The table in online supplemental material 1 gives an overview of the information that will be assessed at the trial level.

Trial protocols will be charted according to relevant items regarding data collection and management from the SPIRIT checklist and the SPIRIT-PRO extension. To evaluate the quality of reporting with reference to the CONSORT and CONSORT-PRO, we will jointly refer to all publications from a single trial. If there are multiple publications from a single trial, we will not evaluate CONSORT conformity for each publication but rather consider the sum of all information extracted from all publications. While this approach may slightly overestimate CONSORT conformity for trials that have a lot of different publications, we argue that this approach is more true to the nature of the research question (ie, not all publications from a trial will focus on PROs).

Summarising and reporting the results

As outlined before, we will compare data on a trial level based on the information from all publications and the protocol of a single trial. The collected information will then be compared with regard to the research questions. Specifically, we will focus on a descriptive analysis of the MOA of PROMs (ie, how are PROMs assessed in trials that have information on their MOA) and on the relative frequencies of the different MOA and associated trial characteristics (ie, research questions 1 and 2). We will also evaluate the quality of reporting for the relevant SPIRIT and CONSORT guidelines that refer to the assessment of PROMs in trials (research question 3; see online supplemental material 1, table 1). Generally, we expect that the detailed reporting of the MOA will be the exception, rather than the norm, even in published trial protocols. Considering that the CONSORT and CONSORT-PRO guidelines were published in 2010 and 2013, respectively,16 17 this might indicate a lack of uptake of the MOA-related items or a perceived lack of importance by researchers. However, research shows that differences between MOA should be carefully considered, especially when mixing or comparing different MOA in the same trial.3 9 11 Hence, it would be important to increase awareness of (1) a potential lack of sufficient MOA reporting and (2) the challenges and benefits associated with different MOA in trials. Moreover, we will explore whether any trials report on using ePRO-based alerts during the trial (research question 4).

Ethics and dissemination of results

No ethics approval was sought for this study as the study is based on published literature only. This review is part of an ongoing EORTC project on the use of electronic data capture methods in cancer clinical trials. The findings from the review will be reviewed and discussed by a multidisciplinary expert committee to synthesise guidance to ultimately improve the (electronic) measurement of PROMs in clinical trials. Input from different stakeholders involved in the setup and use of ePRO solutions for trials will be sought to get further insight into the data.

Finally, the findings from the review will be presented at international meetings and published in a peer-reviewed journal following the PRIMSA-ScR guidelines.21

Patient and public involvement

Patients or the public were not involved in the design, conduct or reporting of this protocol. Patient (and other trial stakeholders) will however be involved in expert groups discussing the review’s findings and following works to draft recommendations around electronic data capture.

Limitations

First, a potential limitation is that our search strategy only includes trials with publications that include mentioning of PROs (or relevant synonyms). Possibly, there might be some trials that included a PRO in the protocol only but did not include any mention of PROs in any of the trial’s publications, and we would not find those trials with our search strategy. While a systematic review of trial protocols and publications between 2014 and 2017 found that about 38% of trials did not publish their PRO results,19 most trial publications commonly include at least the definitions of primary and secondary endpoints, thus allowing us to include them in our search.

Second, although restricting the search to a single database and studies from 2019 to 2023 helps keep the review workload manageable, it may also cause us to miss some potentially suitable publications.

Finally, focusing on trials that report some information on their PRO MOA might introduce sampling bias by picking trials with available information. However, it is unclear in what direction this may introduce bias (ie, in the direction of more or less electronic data capture).

supplementary material

10.1136/bmjopen-2024-084935 online supplemental material 1

10.1136/bmjopen-2024-084935 online supplemental material 2

Acknowledgements

We thank Sonya Eremenco (Critical Path Institute) for input on the protocol and review strategy.

Review Process File
10 09 2024

Funding: Funding for this study was provided by the EORTC Quality of Life Group (grant number 008-2023). The EORTC Quality of Life Group business model involves charges for commercial companies using EORTC instruments. Academic use of EORTC instruments is free of charge.

Prepub: Prepublication history and additional supplemental material for this paper are available online. To view these files, please visit the journal online (https://doi.org/10.1136/bmjopen-2024-084935).

Provenance and peer review: Not commissioned; externally peer reviewed.

Patient consent for publication: Not applicable.

Patient and public involvement: Patients and/or the public were not involved in the design, conduct, reporting or dissemination plans of this research.
==== Refs
References

1 Yu H Yu Q Nie Y et al Data Quality of Longitudinally Collected Patient-Reported Outcomes After Thoracic Surgery: Comparison of Paper- and Web-Based Assessments J Med Internet Res 2021 23 e28915 10.2196/28915 34751657
2 Zbrozek A Hebert J Gogates G et al Validation of electronic systems to collect patient-reported outcome (PRO) data-recommendations for clinical trial teams: report of the ISPOR ePRO systems validation good research practices task force V Health 2013 16 480 9 10.1016/j.jval.2013.04.002
3 Coons SJ Eremenco S Lundy JJ et al Capturing Patient-Reported Outcome (PRO) Data Electronically: The Past, Present, and Promise of ePRO Measurement in Clinical Trials Patient 2015 8 301 9 10.1007/s40271-014-0090-z 25300613
4 Mercieca-Bebber R Palmer MJ Brundage M et al Design, implementation and reporting strategies to reduce the instance and impact of missing patient-reported outcome (PRO) data: a systematic review BMJ Open 2016 6 e010938 10.1136/bmjopen-2015-010938
5 Stone AA Shiffman S Schwartz JE et al Patient non-compliance with paper diaries BMJ 2002 324 1193 4 10.1136/bmj.324.7347.1193 12016186
6 Basch E Abernethy AP Mullins CD et al Recommendations for Incorporating Patient-Reported Outcomes Into Clinical Comparative Effectiveness Research in Adult Oncology JCO 2012 30 4249 55 10.1200/JCO.2012.42.5967
7 Di Maio M Basch E Denis F et al The role of patient-reported outcome measures in the continuum of cancer clinical care: ESMO Clinical Practice Guideline Ann Oncol 2022 33 878 92 10.1016/j.annonc.2022.04.007 35462007
8 European Medicines Agency Appendix 2 to the guideline on the evaluation of anticancer medicinal products in man: the use of patient-reported outcome (PRO) measures in oncology studies 2016 Available https://www.ema.europa.eu/en/documents/other/appendix-2-guideline-evaluation-anticancer-medicinal-products-man_en.pdf
9 Hudgens S Kern S Barsdorf AI et al Best Practice Recommendations for Electronic Patient-Reported Outcome Dataset Structure and Standardization to Support Drug Development V Health 2023 26 1242 8 10.1016/j.jval.2023.02.011
10 Mowlem FD Elash CA Dumais KM et al Best Practices for the Electronic Implementation and Migration of Patient-Reported Outcome Measures V Health 2024 27 79 94 10.1016/j.jval.2023.10.007
11 Eremenco S Coons SJ Paty J et al PRO Data Collection in Clinical Trials Using Mixed Modes: Report of the ISPOR PRO Mixed Modes Good Research Practices Task Force V Health 2014 17 501 16 10.1016/j.jval.2014.06.005
12 Howry C Elash CA Crescioni M et al Best Practices for Avoiding Paper Backup When Implementing Electronic Approaches to Patient-Reported Outcome Data Collection in Clinical Trials Ther Innov Regul Sci 2019 53 441 5 10.1177/2168479018785160 30244593
13 Ly JJ Crescioni M Eremenco S et al Training on the Use of Technology to Collect Patient-Reported Outcome Data Electronically in Clinical Trials: Best Practice Recommendations from the ePRO Consortium Ther Innov Regul Sci 2019 53 431 40 10.1177/2168479018796206 30221997
14 Bodart S Byrom B Crescioni M et al Perceived Burden of Completion of Patient-Reported Outcome Measures in Clinical Trials: Results of a Preliminary Study Ther Innov Regul Sci 2019 53 318 23 10.1177/2168479018788053 30058368
15 Chan A-W Tetzlaff JM Altman DG et al SPIRIT 2013 statement: defining standard protocol items for clinical trials Ann Intern Med 2013 158 200 7 10.7326/0003-4819-158-3-201302050-00583 23295957
16 Calvert M King M Mercieca-Bebber R et al SPIRIT-PRO Extension explanation and elaboration: guidelines for inclusion of patient-reported outcomes in protocols of clinical trials BMJ Open 2021 11 e045105 10.1136/bmjopen-2020-045105
17 Moher D Hopewell S Schulz KF et al CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials BMJ 2010 340 c869 10.1136/bmj.c869 20332511
18 Calvert M Blazeby J Altman DG et al Reporting of Patient-Reported Outcomes in Randomized Trials JAMA 2013 309 814 10.1001/jama.2013.879 23443445
19 Kyte D Retzer A Ahmed K et al Systematic Evaluation of Patient-Reported Outcome Protocol Content and Reporting in Cancer Trials J Natl Cancer Inst 2019 111 1170 8 10.1093/jnci/djz038 30959516
20 Krepper D Giesinger JM Dirven L et al Information about missing patient-reported outcome data in breast cancer trials is frequently not documented: a scoping review J Clin Epidemiol 2023 162 1 9 10.1016/j.jclinepi.2023.07.012 37517504
21 Tricco AC Lillie E Zarin W et al PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation Ann Intern Med 2018 169 467 73 10.7326/M18-0850 30178033
22 Arksey H O’Malley L Scoping studies: towards a methodological framework Int J Soc Res Methodol 2005 8 19 32 10.1080/1364557032000119616
23 Peters MDJ Godfrey CM Khalil H et al Guidance for conducting systematic scoping reviews Int J Evid Based Healthc 2015 13 141 6 10.1097/XEB.0000000000000050 26134548
24 Levac D Colquhoun H O’Brien KK Scoping studies: advancing the methodology Implement Sci 2010 5 69 10.1186/1748-5908-5-69 20854677
25 Colquhoun HL Levac D O’Brien KK et al Scoping reviews: time for clarity in definition, methods, and reporting J Clin Epidemiol 2014 67 1291 4 10.1016/j.jclinepi.2014.03.013 25034198
26 U.S. Food and Drug Administration Guidance for industry: electronic source data in clinical investigations 2013 Available https://www.fda.gov/media/85183/download
27 Kyte D Draper H Calvert M Patient-reported outcome alerts: ethical and logistical considerations in clinical trials JAMA 2013 310 1229 30 10.1001/jama.2013.277222 24065005
28 Cruz Rivera S Aiyegbusi OL Ives J et al Ethical Considerations for the Inclusion of Patient-Reported Outcomes in Clinical Research: The PRO Ethics Guidelines JAMA 2022 327 1910 9 10.1001/jama.2022.6421 35579638
29 DistillerSR Inc DistillerSR n.d. Available https://www.distillersr.com
30 Siegel RL Miller KD Fuchs HE et al Cancer statistics, 2022 CA Cancer J Clin 2022 72 7 33 10.3322/caac.21708 35020204
31 Giesinger JM Efficace F Aaronson N et al Past and Current Practice of Patient-Reported Outcome Measurement in Randomized Cancer Clinical Trials: A Systematic Review V Health 2021 24 585 91 10.1016/j.jval.2020.11.004
32 Efficace F Rees J Fayers P et al Overcoming barriers to the implementation of patient-reported outcomes in cancer clinical trials: the PROMOTION Registry Health Qual Life Outcomes 2014 12 86 10.1186/1477-7525-12-86 24902767
33 Frandsen TF Eriksen MB Hammer DMG et al PubMed coverage varied across specialties and over time: a large-scale study of included studies in Cochrane reviews J Clin Epidemiol 2019 112 59 66 10.1016/j.jclinepi.2019.04.015 31051247
