
==== Front
HRB Open Res
HRB Open Res
HRB Open Research
2515-4826
F1000 Research Limited London, UK

10.12688/hrbopenres.13763.2
Research Note
Articles
Development and Initial Implementation of a Clinical Monitoring Strategy in a Non-regulated Trial: a research note from the ReStOre II Trial
[version 2; peer review: 2 approved, 1 approved with reservations]

O'Neill Linda Methodology Project Administration Writing – Original Draft Preparation Writing – Review & Editing https://orcid.org/0000-0002-0109-9650
12
Murphy Fiona Project Administration Writing – Original Draft Preparation Writing – Review & Editing https://orcid.org/0009-0006-8796-1104
123
Reidy Derval Project Administration Writing – Review & Editing 4
Poisson Camille Methodology Project Administration Writing – Review & Editing https://orcid.org/0009-0005-5853-2127
45
Hussey Juliette Funding Acquisition Methodology Project Administration Writing – Review & Editing 12
Guinan Emer Conceptualization Methodology Writing – Review & Editing https://orcid.org/0000-0001-9715-146X
a26
1 School of Medicine, Discipline of Physiotherapy, Trinity College, Dublin, Ireland
2 Trinity St James's Cancer Institute, Trinity College, St James's Hospital, Dublin, Ireland
3 School of Pharmacy and Pharmaceutical Sciences, Trinity College, Dublin, Ireland
4 Wellcome-Health Research Board Clinical Research Facility, Trinity College, St James’s Hospital, Dublin, Ireland
5 Research Innovation, Trinity College, Trinity College Dublin, the University of Dublin, Dublin, Ireland
6 Trinity Innovation & Enterprise, Trinity College, Hartford, Connecticut, USA
a emguinan@tcd.ie
No competing interests were disclosed.

6 12 2023
2023
6 464 12 2023
Copyright: © 2023 O'Neill L et al.
2023
https://creativecommons.org/licenses/by/4.0/ This is an open access article distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Background

Data and Safety Monitoring is integral to quality assurance of clinical trials. Although monitoring is a core legal component of regulated clinical trials, non-regulated trials are not mandated to incorporate monitoring. Consequently, the monitoring process has been underutilised and underreported in this setting. This research report outlines the development and plans for implementing a bespoke Clinical Monitoring Strategy within the ‘ Rehabilitation Strategies Following Oesophagogastric and Hepatopancreaticobiliary Cancer (ReStOre II) Trial’, a non-regulated trial comparing a 12-week multidisciplinary programme of rehabilitation to standard care in a cohort of 120 cancer survivors.

Methods

This research note provides a detailed overview of the ReStOre II Clinical Monitoring Strategy and describes the development of the strategy pre and post awarding of the grant. The strategy consists of the establishment and implementation of a comprehensive trial governance structure, inclusive of a Trial Management Group, Trial Steering Committee Meeting, and Independent Data Monitoring Committee. In addition, external trial monitoring by the Clinical Research Facility at St James’s Hospital. Three monitoring visits will be conducted during the trial; i) site initiation visit, ii) interim monitoring visit, and iii) close our visit.

Results

The Clinical Monitoring Strategy has been finalised and is currently being implemented within the ReStOre II Trial. Two site initiation visits and one interim monitoring visit have been completed to date.

Conclusion

This research note provides a template for implementation of a Clinical Monitoring Strategy in a non-regulated clinical trial.

Registration

ReStOre II Trial: https://clinicaltrials.gov/ct2/show/NCT03958019

Monitoring
Non-regulated trial
Independent Data Monitoring Committee
Health Research BoardHRB-DIFA-2018-009 Health Research Board Ireland [DIFA-FA-2018-009]. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Revised Amendments from Version 1

Minor changes were made to version 1 of the manuscript including updating reference #3 to the latest version, adding the names of the Independent Data Monitoring Committee to the Acknowledgements and some changes to the layout of the methodology section to ensure consistency of terminology and headings. Some minor edits were made to the text to keep the manuscript within word count. We thank the reviewers for their thorough and considered review of Version 1 of this manuscript.
==== Body
pmcIntroduction

Monitoring is a core strategy to ensure best practice, participant safety, integrity, internal validity and overall quality assurance of clinical trials 1– 3 , and is a key aspect of the International Conference of Harmonisation Good Clinical Practice (ICH GCP), which provides a standard for the conduct of clinical trials 4 . In Ireland, regulated clinical trials falls under the remit of the Health Product Regulation Authority (HPRA), and typically involve an Investigational Medicinal Product (IMP), regulated under the IMP regulatory framework (SI 190/2004), and therefore are legally required to comply with ICH GCP and the EU Clinical Trial Regulation EU#536/2014 or the EU Medical Device Regulation 2017/745 5 . Non-regulated trials are not legally required to comply with these regulations. The Declaration of Helsinki is a statement of ethical principles for medical research involving human subjects which is applicable to both regulated and non-regulated research, and for clinical trials is transposed into law through ICH GCP.

Trial monitoring involves independent review of trial documentation to ensure data is accurate and complete and verifiable against source documentation 6 , procedures are compliant with the approved protocol, and that the trial is being conducted in line with ICH GCP 4 . Trial monitoring can involve independent data monitoring committees (IDMC), independent statistical analysis, centralised monitoring, site monitoring, and endpoint adjudication 2, 3 . External data monitoring may be responsive to metrics set at the outset of the trial 7 , or routine. Responsive data monitoring may be just as effective as routine monitoring with fewer financial implications 8 , although certain vulnerable populations, (e.g. paediatrics) may benefit from external data monitoring as standard, with cost merely a secondary consideration 9 . Others argue that external data monitoring should not be done routinely. Instead, investigators should pay attention to the core principles of trial monitoring, and design rigorous monitoring procedures based on risk appropriate for each trial 10 . External data monitoring is essential for phased clinical trials evaluating pharmacological agents wherein safety evaluation is a core feature 11 . Since the Food & Drug Administration and the European Medicines Agency released guidelines in the 2000s 12, 13 , a handful of studies have demonstrated efficacy for external data monitoring. Indeed, one review attributed benefits in trial recruitment rates and protocol compliance to monitoring 3 .

The practicalities and efficacy of implementing monitoring strategies in non-regulated clinical trials is poorly documented. It stands to reason that the quality of trials involving complex interventions (e.g., behavioural, psychological, nursing, nutritional, exercise and rehabilitation), where ICH-GCP is not a legal requirement, may benefit from standardised monitoring procedures. The Rehabilitation Strategies Following Oesophagogastric and Hepatopancreaticobiliary Cancer (ReStOre II) trial is a two-armed randomised controlled trial (RCT), funded by a Health Research Board (HRB) Definitive Intervention and Feasibility Award (DIFA). The trial builds on findings from a HRB Health Research Award (2014–2017) which characterised unmet physical, nutritional and psychosocial needs in upper gastrointestinal cancer survivors 14– 18 , leading to the development, feasibility testing and piloting of a tailored 12-week multidisciplinary rehabilitation programme 19– 22 . The ReStOre II definitive RCT will examine if the 12-week programme can improve functional capacity and health related quality-of-life in 120 survivors of oesophagogastric and hepatopancreaticobiliary cancer in comparison to usual care 23 . Recruitment occurs at St James’s Hospital and St Vincent’s University Hospital, Dublin, with all trial assessments and intervention visits completed at the St James’s Hospital Clinical Research Facility (SJH-CRF). The trial was due to begin recruitment early 2020 but this was delayed significantly due to COVID-19. The trial opened in Spring 2022 and is due for completion in Quarter 4 (Q4) 2024.

Aim of research report

As a non-regulated trial, a novel aspect of ReStOre II is the inclusion of a bespoke Clinical Monitoring Strategy. The aim of this research report is to outline the development and initial implementation of the ReStOre II Clinical Monitoring Strategy.

Methods

Pre-award Planning for a Clinical Monitoring Strategy

The ReStOre II Trial is funded through a HRB DIFA Award (HRB-DIFA-2018-009). The grant application included a Clinical Monitoring Strategy as a commitment to quality assurance.

The proposed Clinical Monitoring Strategy included:

1) A Trial Governance Structure: 1. Trial Management Group

2. Trial Steering Committee

3. IDMC

2) A Trial Monitoring Plan

1. Site initiation visit

2. Interim monitoring visit

3. Close out visit

The monitoring proposal was developed in consultation with the Quality and Regulatory Affairs (QRA) Department in the SJH-CRF. As ReStOre II would not be investigating an IMP or any medical devices beyond the remit of their marketing authorisation, it was classified as a non-regulated trial which did not require an application to the HPRA. Accordingly, it was anticipated that three monitoring visits (Site Initiation Visit, Interim Monitoring Visit, and Close Out Visit) would likely be sufficient to ensure quality control, estimated to require 15 days of monitoring. This proposal was included in the infrastructural agreement with the SJH-CRF as part of the grant application and appropriate budget allocated.

Post Award Development of the Clinical Monitoring Strategy

The HRB-DIFA award was awarded to one of the authors (JH) in 2018. Upon commencement of the award on 1 st May 2019, the details of the Clinical Monitoring Strategy were developed and consistent with the pre-award proposal, this included a Trial Governance Structure and a Trial Monitoring Plan.

Developing Trial Governance Structures. A governance structure comprising three trial management committees for project management and oversight were established. The first version of the Terms of Reference (TOR) for each committee was finalised on the 29 th August 2019.

The Trial Management Group TOR outlined that the committee would consist monthly for the first six months of the trial, and quarterly thereafter, with additional meetings scheduled as required. Membership of the ReStOre II Trial Management Group consists of the Principal Investigator (PI), Co-applicants x3, Project Manager, Trial Statistician, and the Research Team (research assistants, research nurses). The Trial Management Group has responsibility for the day-to-day running of the trial, initial development of the study protocol, the review and approval of subsequent protocol amendments and the submission of reports to the Trial Steering Committee.

The Trial Steering Committee TOR outlined that the committee would provide independent oversight and supervision to the trial and ensure that the trial is conducted in accordance with Research Governance guidelines. The ReStOre II Trial Steering Committee would consists of n=4 Independent Members, n=1 patient representative, the PI, n=2 Co-applicants, and the Project Manager. The Trial Steering Committee would be chaired by an Independent Member and meets biannually. The committee has responsibility for the formal approval of the trial protocol, monitoring of trial progress against anticipated timelines and targets, and monitoring of adherence to the trial protocol and patient safety through consideration of reports from the IDMC.

Finally, the TOR for the IDMC outlined that the committee would consist of a panel of experienced researchers, who are independent of the trial, with complementary expertise in clinical exercise oncology (n=1), medical statistics (n=1), dietetics (n=1), upper gastrointestinal cancer surgery (n=1) and quality and regulatory affairs (n=1). The IDMC would meet biannually and have collective responsibility for monitoring the progress of the trial from the start to the end of data collection, scrutinising recruitment, randomisation, retention, protocol compliance and adherence and assessing adverse events. The IDMC would review the monitoring reports produced by the SJH-CRF following planned visits outlined in the Trial Monitoring Plan and may make recommendations to the Trial Steering Committee based on their review of the monitoring reports.

Developing the Trial Monitoring Plan. The ReStOre II research team worked with the SJH-CRF QRA Department to develop the Trial Monitoring Plan. This commenced with an overall study risk assessment and a second risk assessment specifically to evaluate the level of monitoring required.

An overall study risk assessment was completed according to standard QRA Department procedures and deemed RESTORE II to be a low-risk trial. Subsequently, the QRA Department used the SCTO Risk Based Monitoring Score Calculator 24 . The SCTO calculator determines the monitoring strategy for a clinical trial in consideration of the level of risk across twenty-three risk factors organised into seven categories. The SCTO risk-based monitoring score calculator classified ReStOre II as a low-risk trial (see Extended Data 25 ) and accordingly the QRA Department confirmed that the three monitoring visits, originally envisaged at pre-award stage (site initiation visit, interim monitoring visit, and close out visit) would be adequate. A Trial Monitoring Plan version 1 (2020) and version 2 (2023) were established by the QRA Department in line with this low-risk status.

All monitoring visits were planned to be conducted by a member of the SJH-CRF QRA Department who is independent of the ReStOre II trial team but performs sponsor delegated tasks from Trinity College Dublin.

The Trial Monitoring Plan incorporated two site initiation visits, one at St James’s Hospital and one at St Vincent’s University Hospital. The site initiation visit was planned to occur prior to ‘greenlighting’ the trial for screening and recruitment at both sites to ensure compliance with the following:

1. All necessary approvals e.g., ethics, hospital approval are in place

2. All planned trial procedures are GCP compliant and compliant with the approved protocol

3. All trial staff are trained appropriately, and training documentation is stored appropriately

4. Adequate and appropriate resources are in place for the trial

5. Insurance cover is in place for the trial

6. The investigator site file is set up and managed appropriately

7. All trial documentation is ready for trial commencement including the trial protocol, case report forms, deviation log, incident report forms, participant identification log, screening documentation, data management plan, and randomisation plan.

8. Trial Management, Governance and Safety TOR is in place.

The Trial Monitoring Plan includes an interim monitoring visit at St James’s Hospital, the intervention site, to ensure continued quality control during the intervention, continued adherence to GCP during the intervention, and continued adherence to the approved protocol during the intervention. Accordingly, the Trial Monitoring Plan schedules the interim monitoring visit early in the trial, and as soon as possible after the first cohort of patients are enrolled onto the trial. The interim monitoring visit includes review of the following documents for existence and accuracy: eligibility screening and enrolment logs, informed consent forms (existence, version control, signatures and dates), the consent process, source data, randomisation records, investigator site file maintenance, protocol amendments and approvals, protocol deviation log, incident log and incident reporting, staff training records, equipment calibration and service records, and confidential patient identification log. Source data verification of data within the case report form and other data collection systems is conducted on the first participant, and up to 20% of the cohort enrolled at the time of the interim monitoring visit. This involves reviewing the paper case report forms against the source data (e.g., study visit notes), raising data queries and flagging missing or inaccurate data captured. The interim monitoring visit concludes with a meeting between the monitor and the study team to discuss findings. Additional interim monitoring visits can be completed if required.

Following completion of the ReStOre II trial, an in-person close out visit will be conducted by the monitoring team in the SJH-CRF on request. The close-out visit will verify the existence of original informed consent forms for enrolled subjects, collection of all Case Report Forms, and collection and completion of safety data. Any outstanding issues from prior monitoring visits will be resolved and the monitoring team will remind the site study team of their responsibilities for record retention and inspect the investigator site file for completeness prior to study closure.

Results

Implementation of the Clinical Monitoring Strategy

The Clinical Monitoring Strategy for the ReStOre II Trial has achieved the following deliverables to date (June 2023):

Trial Governance Deliverables

1. Establishment of the Trial Management, Governance and Safety Monitoring TOR (29 th August 2019).2. Implementation of the Trial Management, Governance and Safety Monitoring TOR. Between award commencement (May 2019) and June 2023 the following number of management group meetings have occurred: Trial management Group (n=18), Trial Steering Committee (n=5), IDMC (n=3).

Trial Monitoring Plan Deliverables

1. Finalisation of the Trial Monitoring Plan (18 th January 2022).

2. The site initiation visit was conducted in St James’s Hospital on the 18 th January 2022, with greenlight received on the 3 rd February 2022.

3. The site initiation visit was conducted in St Vincent’s University Hospital on the 11 th January 2023, with greenlight received on the 20 th January 2023.

4. The interim monitoring visit was conducted on the 7 th February 2023 at St James’s Hospital and included the following: review of investigator site file, source document/ case report form review for n=7 participants, all informed consent forms were reviewed for completeness and accuracy, a review of enrolment status, review of protocol violations, incident forms, and intervention accountability. A number of minor action points were identified and addressed by the site team in writing on the 23 rd February 2023 and the monitoring team in the SJH-CRF confirmed on the 24 th February 2023 that they were satisfied with the site teams’ response to all action points.

5. Version 2 of the Trial Monitoring Plan by the SJH-CRF QRA Department on the 26 th June 2023.

The following deliverables are ongoing:

1. The Trial Management Group will continue to meet quarterly, and the Trial Steering Committee and IDMC biannually for the remainder of the trial (due for completion Q4 2024).

2. Following completion of the trial a close out visit will be conducted by the SJH-CRF monitoring team on request.

Discussion

This research note outlines the development and initial implementation of the Clinical Monitoring Strategy for the ReStOre II Trial (a non-regulated trial) in collaboration with a SJH-CRF. We will produce a final report describing complete implementation this strategy upon completion of the trial (Q4 2024). Our experience of developing and implementing this strategy has greatly enhanced the quality and rigor of this non-regulated trial.

Although there is no legal requirement for these studies to be monitored, monitoring is an extremely important part of the research cycle particularly within a teaching hospital setting. Monitoring provides a quality framework and ensures that the principles of the Declaration of Helsinki and relevant ICH GCP principles are followed. Monitoring helps assure patient safety and data integrity for non-IMP/Medical Device trials. It also trains researchers to consider quality at each step of the research process and trains them to work on studies where ICH-GCP is mandatory (IMP/Medical Device trials). All research staff involved in the ReStOre II trial complete ICH GCP training as a mandatory training requirement. Through working with the SJH-CRF and QRA team, the research team understands the importance of Good Documentation, version control, and updating of the Participant Information Leaflet (PIL) and Informed Consent Form (ICF) documentation for ethics applications. Increasing the awareness and utilisation of monitoring strategies within non-regulated trials may be useful for improving the quality and validity of non-regulated trials going forward.

Acknowledgments

The authors would like to acknowledge the assistance and support of the Wellcome Trust/Health Research Board Clinical Research Facility at St. James’ Hospital, Dublin, Trinity College Dublin (study Sponsor) and our infrastructural partners the Health Research Board Trials Methodology Research Network (HRB-TMRN). The authors would greatly like to thank the commitment of the current and previous members of the Trial Management Groups. We also thank the members of our Independent Data Monitoring Committee (IDMC) Dr Julie Broderick, Dr Jennifer Ryan, Dr Jessie Elliot, Mr Shane Feeney, Ms Ruben Keane and Ms Fergal Seeballuck (past member).

Data availability

Underlying data

All data underlying the results are available as part of the article and no additional source data are required.

Extended data

Open Science Framework: Development and initial Implementation of a Clinical Monitoring Strategy in a Non-regulated Trial: a research note from the ReStOre II Trial

https://doi.org/10.17605/OSF.IO/NMX3G 25

This project contains the following extended data:

ReStOre II Study Risk Assessment Form signed.pdf

Data are available under the terms of the Creative Commons Zero "No rights reserved" data waiver (CC0 1.0 Public domain dedication).

10.21956/hrbopenres.15166.r37350
Reviewer response for version 2
Love Sharon B. 1Referee https://orcid.org/0000-0002-6695-5390

1 MRC Clinical Trials Unit at UCL, Institute of Clinical Trials and Methodology, University College London, London, England, UK
14 9 2024 Copyright: © 2024 Love SB
2024
https://creativecommons.org/licenses/by/4.0/ This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Version 2recommendationapprove
Thank you to the authors for updating the manuscript. I have no further comments.

Is the work clearly and accurately presented and does it cite the current literature?

Partly

If applicable, is the statistical analysis and its interpretation appropriate?

Not applicable

Are all the source data underlying the results available to ensure full reproducibility?

No source data required

Is the study design appropriate and is the work technically sound?

Yes

Are the conclusions drawn adequately supported by the results?

Yes

Are sufficient details of methods and analysis provided to allow replication by others?

Yes

Reviewer Expertise:

I am a researcher in clinical trial monitoring I lead the UKCRC Monitoring group and the TMRP Monitoring group. I have been the trial statistician on many trials

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

10.21956/hrbopenres.15166.r37349
Reviewer response for version 2
Jackson Mandy 1Referee https://orcid.org/0009-0002-4390-4707

Varghese Pooja 1Co-referee https://orcid.org/0000-0002-9540-7828

1 RCSI Sponsorship Office, Royal College of Surgeons in Ireland, Dublin, Ireland
14 9 2024 Copyright: © 2024 Jackson M and Varghese P
2024
https://creativecommons.org/licenses/by/4.0/ This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Version 2recommendationapprove
happy that previous comments have been addressed

Is the work clearly and accurately presented and does it cite the current literature?

Partly

If applicable, is the statistical analysis and its interpretation appropriate?

Not applicable

Are all the source data underlying the results available to ensure full reproducibility?

Yes

Is the study design appropriate and is the work technically sound?

Yes

Are the conclusions drawn adequately supported by the results?

Yes

Are sufficient details of methods and analysis provided to allow replication by others?

Partly

Reviewer Expertise:

Reviewer able to assess entirety of the article. Background in clinical monitoring, quality, pharmacovigilance, regulatory affairs and nursing.

We confirm that we have read this submission and believe that we have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

10.21956/hrbopenres.15166.r41350
Reviewer response for version 2
Ozair Ahmad 12Referee https://orcid.org/0000-0001-6570-4541

1 Baptist Health South Florida,, Miami Cancer Institute,, Miami,, Florida, USA
2 School of Public Health, Johns Hopkins Bloomberg, Baltimore, Maryland, USA
1 8 2024 Copyright: © 2024 Ozair A
2024
https://creativecommons.org/licenses/by/4.0/ This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Version 2recommendationapprove-with-reservations
This review pertains to the revised version (version 2) of a paper describing the monitoring of a non-regulated trial called ReStOre II conducted in Ireland, where non-regulated refers to trials that are not required to comply with ICH GCP and EU Regulations, in accordance with the IMP regulatory framework (analogous to the FDA IND/IDE process). The monitoring described in this research note is related to the risk aspects of the trial.

Given below are my comments in no particular order.

1. First, given the limited word count of 2500 words, authors should heavily cut short the introduction and focus on describing methods and results. More use of EXTENDED DATA should be done.

2. I share the concerns by Prof. Sharon Love that were provided for version 1. The focus of the manuscript is the clinical monitoring strategy, which I don't believe is described in sufficient detail that external individuals (within and outside Ireland) can sufficiently learn from.

3. Prof. Love previously recommended the use of subheadings for structuring the methods and results. While the authors have carried out minor revisions to their manuscript in response, they need to formally use subheadings to reduce word count while adding greater detail.

4. " committee would consist monthly " - the wrong usage needs correction.

5. Where abbreviation is used once only, please remove the abbreviation to avoid unnecessary jargon.

6. Please consider providing the last available protocol, patient information leaflet, and consent form in the extended data. If not possible, at least the protocol should be provided. 

7. Some sentences are unsupported by the evidence presented. Their 'conclusion' like style needs to be overhauled or toned down.

e.g. "Our experience of developing and implementing this strategy has greatly enhanced the quality and rigor of this non-regulated trial."

I don't think this N-of-1, uncontrolled intervention provides sufficient proof that this strategy alone "greatly enhanced the quality and rigor". 

8. Vague, unnecessary, or redundant statements should be removed to make space for a greater description of the methods and results.

e.g."Although there is no legal requirement for these studies to be monitored, monitoring is an extremely important part of the research cycle, particularly within a teaching hospital setting. "

This is already described in the introduction. 

9. If the authors have space, consider adding some discussion of FDA guidance and regulations in this regard.

Is the work clearly and accurately presented and does it cite the current literature?

Partly

If applicable, is the statistical analysis and its interpretation appropriate?

Not applicable

Are all the source data underlying the results available to ensure full reproducibility?

No source data required

Is the study design appropriate and is the work technically sound?

Yes

Are the conclusions drawn adequately supported by the results?

No

Are sufficient details of methods and analysis provided to allow replication by others?

No

Reviewer Expertise:

I am able to assess the entirety of the article. I have a formal medical background with experience and education in clinical trials, clinical research, oncology, and biostatistics.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

10.21956/hrbopenres.15055.r36711
Reviewer response for version 1
Love Sharon B. 1Referee https://orcid.org/0000-0002-6695-5390

1 MRC Clinical Trials Unit at UCL, Institute of Clinical Trials and Methodology, University College London, London, England, UK
1 11 2023 Copyright: © 2023 Love SB
2023
https://creativecommons.org/licenses/by/4.0/ This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Version 1recommendationapprove-with-reservations
This is a paper describing the setup of the monitoring for a non-regulated trial called ReStOre II. It includes the set up of three committees as well as on-site monitoring. The monitoring is related to the risk aspects of the trial.

As the authors say, it is useful to know how others have set up their monitoring and the basis of the decisions that were made. There is still not much literature about the practicalities of monitoring and even less in non-regulated trials. This paper is to be applauded.

My main concern is with the methods section of the paper. The methods section to me jumps around. It would be far more useful and encourage readers to continue reading if it was in a logical order. I would like to know that a trial monitoring plan was envisaged, grant funding was obtained and how it was implemented. Whereas in this paper, this text

The Trial Monitoring Plan for the ReStOre II trial consists of the following monitoring strategies:

1. Site initiation visit

2. Interim monitoring visit

3. Close out visit

4. Trial management group meetings (Trial Management Group and Trial Steering Committee)

5. IDMC

Occurs in the middle of the methods. It would be more useful at the beginning. This is in a different order and using different terminology than when it was introduced in the methods. Whichever order is chosen, it is easier to read if the order remains the same throughout the document and the methods section is written in that same order.  Maybe part of the results paragraph is about how the clinical monitoring strategy changed from grant applications to how you did it in practice.

Minor points

Ref 4 – please could you give an url

Prof Hussey – replace by author initials, “one of the authors (JH)

Establishment of the Trial Management, Governance and Safety Monitoring Terms of Reference – the paragraph on IDMC does not detail that the committee members are independent of the trial – I think its worth repeating although the word independent is in the title.

Results – PIL – needs defining

Results – ICF – needs defining

Acknowledgements – I guess you also meant to thank members of the IDMC

Is the work clearly and accurately presented and does it cite the current literature?

Partly

If applicable, is the statistical analysis and its interpretation appropriate?

Not applicable

Are all the source data underlying the results available to ensure full reproducibility?

No source data required

Is the study design appropriate and is the work technically sound?

Yes

Are the conclusions drawn adequately supported by the results?

Yes

Are sufficient details of methods and analysis provided to allow replication by others?

Yes

Reviewer Expertise:

I am a researcher in clinical trial monitoring I lead the UKCRC Monitoring group and the TMRP Monitoring group. I have been the trial statistician on many trials

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

Guinan Emer Department of Surgery, Trinity College, Dublin, Dublin, Ireland

1 12 2023 This is a paper describing the setup of the monitoring for a non-regulated trial called ReStOre II. It includes the set up of three committees as well as on-site monitoring. The monitoring is related to the risk aspects of the trial.

As the authors say, it is useful to know how others have set up their monitoring and the basis of the decisions that were made. There is still not much literature about the practicalities of monitoring and even less in non-regulated trials. This paper is to be applauded.

Thank you for your complimentary feedback on the value of this paper.

My main concern is with the methods section of the paper. The methods section to me jumps around. It would be far more useful and encourage readers to continue reading if it was in a logical order. I would like to know that a trial monitoring plan was envisaged, grant funding was obtained and how it was implemented. Whereas in this paper, this text

The Trial Monitoring Plan for the ReStOre II trial consists of the following monitoring strategies:

1. Site initiation visit

2. Interim monitoring visit

3. Close out visit

4. Trial management group meetings (Trial Management Group and Trial Steering Committee)

5. IDMC

Occurs in the middle of the methods. It would be more useful at the beginning. This is in a different order and using different terminology than when it was introduced in the methods. Whichever order is chosen, it is easier to read if the order remains the same throughout the document and the methods section is written in that same order.  Maybe part of the results paragraph is about how the clinical monitoring strategy changed from grant applications to how you did it in practice.

Thank you for this helpful feedback. We have tried to apply your feedback and restructure the methods to improve readability and flow. We hope that these edits improve the paper.

Minor points

Ref 4 – please could you give an url

A URL has been provided for reference 4 which has also been updated to the latest version of these guidance.

Prof Hussey – replace by author initials, “one of the authors (JH)

Prof Hussey has been replaced with ‘One of the authors (JH)’

Establishment of the Trial Management, Governance and Safety Monitoring Terms of Reference – the paragraph on IDMC does not detail that the committee members are independent of the trial – I think its worth repeating although the word independent is in the title.

Thank you for this point – the word independent has been added as appropriate to that paragraph.

Results – PIL – needs defining

Results – ICF – needs defining

Definitions of PIL and ICF have both been included.

Acknowledgements – I guess you also meant to thank members of the IDMC

Apologies for omitting to list the members of the IDMC. Their names have now been included in the acknowledgements section.

A number of other minor edits have been made to the manuscript to keep within word count limits.

10.21956/hrbopenres.15055.r36117
Reviewer response for version 1
Jackson Mandy 1Referee https://orcid.org/0009-0002-4390-4707

Varghese Pooja 1Co-referee https://orcid.org/0000-0002-9540-7828

1 RCSI Sponsorship Office, Royal College of Surgeons in Ireland, Dublin, Ireland
29 9 2023 Copyright: © 2023 Jackson M and Varghese P
2023
https://creativecommons.org/licenses/by/4.0/ This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Version 1recommendationapprove-with-reservations
This article was very interesting and is a very under researched area. Most academic Sponsors do not have the resources to conduct monitoring in unregulated studies so it was interesting to read about the impact such monitoring had on the Restore II trial.

For the above question: "Are sufficient details of methods and analysis provided to allow replication by others?"-this was answered as partly because at this point, the reviewer is of the opinion that further research in the area is needed (as implied in the discussion).

Please see below some constructive feedback divided into the sections as they appear in the article:

INTRODUCTION Reference is made to the International Conference on harmonisation. This has been changed to the International Council on harmonisation and GCP guideline that was current at the time of the article was Revision 2. I believe reference #4 therefore should be updated.

In the introduction, the following sentence does not read correctly:

“In Ireland, regulated clinical trials falls under the remit of the Health Product Regulation Authority (HPRA), and typically involve an Investigational Medicinal Product (IMP) or Medical Device, regulated under the IMP regulatory framework (SI 190/2004),”

SI 190 2004 is specifically governing clinical trials involving IMP and not medical devices so please consider revising this sentence

In the introduction, second paragraph reference to ICH GCP (#3) does not correspond with the reference provided in the reference section. Should this reference be #4?

In Introduction para 2, “Trial monitoring can involve independent data monitoring committees (IDMC), independent statistical analysis, centralised monitoring, site monitoring, and endpoint adjudication”. Sentence not referenced so reviewer unable to verify if this sentence is factual.

Introduction para 3, “The practicalities and efficacy of implementing monitoring strategies in non-regulated clinical trials is poorly documented. It stands to reason that the quality of trials involving complex interventions (e.g., behavioural, psychological, nursing, nutritional, exercise and rehabilitation), where ICH-GCP is not a legal requirement, may benefit from standardised monitoring procedures”. Sentence not referenced so reviewer unable to verify if this sentence is factual.

METHODS If the close out visit will verify the existence of original informed consent forms for enrolled subjects, this may be difficult to achieve remotely (as proposed). Would the team consider an in person close out visit in order to achieve the objectives outlined in the last paragraph of the methods section?

In the Finalising the Trial Monitoring Plan section in the methods section it reads “Any outstanding issues from prior monitoring visits will be resolved and the monitoring team will remind the site study team of their responsibilities for record retention and inspect the investigator site file for completeness prior to study closure” As per GCP record retention is the responsibility of the sponsor and the Investigator and not solely the site staff. This responsibility can be delegated by the investigator to site staff so please consider adding wording to clarify this in the sentence.

DISCUSSION “The aim of this research report is to outline the development and initial implementation of the ReStOre II Clinical Monitoring Strategy” the discussion implies there is more to the aim than just to outline the development and initial implementation of the strategy but that the authors were also investigating the impact of the monitoring strategy on the research team’s awareness of good clinical practice and the quality & rigor of the data. Should the aims be changed?

Is the work clearly and accurately presented and does it cite the current literature?

Partly

If applicable, is the statistical analysis and its interpretation appropriate?

Not applicable

Are all the source data underlying the results available to ensure full reproducibility?

Yes

Is the study design appropriate and is the work technically sound?

Yes

Are the conclusions drawn adequately supported by the results?

Yes

Are sufficient details of methods and analysis provided to allow replication by others?

Partly

Reviewer Expertise:

Reviewer able to assess entirety of the article. Background in clinical monitoring, quality, pharmacovigilance, regulatory affairs and nursing.

We confirm that we have read this submission and believe that we have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however we have significant reservations, as outlined above.

Guinan Emer Department of Surgery, Trinity College, Dublin, Dublin, Ireland

1 12 2023 This article was very interesting and is a very under researched area. Most academic Sponsors do not have the resources to conduct monitoring in unregulated studies so it was interesting to read about the impact such monitoring had on the Restore II trial.

For the above question: "Are sufficient details of methods and analysis provided to allow replication by others?"-this was answered as partly because at this point, the reviewer is of the opinion that further research in the area is needed (as implied in the discussion).

Please see below some constructive feedback divided into the sections as they appear in the article:

Thank you for your positive summary of the value of this article. Your constructive feedback is greatly appreciated.

INTRODUCTION

Reference is made to the International Conference on harmonisation. This has been changed to the International Council on harmonisation and GCP guideline that was current at the time of the article was Revision 2. I believe reference #4 therefore should be updated.

Thank you for highlighting this updated reference. Reference #4 has been updated to International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use (ICH). ICH Harmonised Guideline: Integrated Addendum to ICH E6(R1): Guideline for Good Clinical Practice E6(R2). December 1, 2016. Available from: https://www.ema.europa.eu/documents/scientific-guideline/ich-guideline-good-clinical-practice-e6r2-step-5_en.pdf

In the introduction, the following sentence does not read correctly:

“In Ireland, regulated clinical trials falls under the remit of the Health Product Regulation Authority (HPRA), and typically involve an Investigational Medicinal Product (IMP) or Medical Device, regulated under the IMP regulatory framework (SI 190/2004),”

SI 190 2004 is specifically governing clinical trials involving IMP and not medical devices so please consider revising this sentence

 Thank you for highlighting this point – the words ‘or medical device’ has been removed from this sentence.

In the introduction, second paragraph reference to ICH GCP (#3) does not correspond with the reference provided in the reference section. Should this reference be #4?

Thank you for highlighting this error – reference #3 in the second paragraph has been updated to reference #4.

In Introduction para 2, “Trial monitoring can involve independent data monitoring committees (IDMC), independent statistical analysis, centralised monitoring, site monitoring, and endpoint adjudication”. Sentence not referenced so reviewer unable to verify if this sentence is factual.

 References #2 and #3 have been added as references to support this statement.

Introduction para 3, “The practicalities and efficacy of implementing monitoring strategies in non-regulated clinical trials is poorly documented. It stands to reason that the quality of trials involving complex interventions (e.g., behavioural, psychological, nursing, nutritional, exercise and rehabilitation), where ICH-GCP is not a legal requirement, may benefit from standardised monitoring procedures”. Sentence not referenced so reviewer unable to verify if this sentence is factual.

We include this sentence as a summary of the evidence presented in the introduction and state it as our own conclusion which gives rise to the need for this paper. We do not feel that it requires a specific reference.

METHODS

If the close out visit will verify the existence of original informed consent forms for enrolled subjects, this may be difficult to achieve remotely (as proposed). Would the team consider an in person close out visit in order to achieve the objectives outlined in the last paragraph of the methods section?

Thank you for this suggestion and following consultation with our team we have agreed that we can include an in-person close out visit in our plan.  

In the Finalising the Trial Monitoring Plan section in the methods section it reads “Any outstanding issues from prior monitoring visits will be resolved and the monitoring team will remind the site study team of their responsibilities for record retention and inspect the investigator site file for completeness prior to study closure” As per GCP record retention is the responsibility of the sponsor and the Investigator and not solely the site staff. This responsibility can be delegated by the investigator to site staff so please consider adding wording to clarify this in the sentence.

Thank you for raising this point. This is an academic trial and therefore it is Investigator lead study and some sponsor tasks are delegated to the sponsor. The TMF is combined with the ISF and the responsibility of maintaining the ISF/TMF falls under the investigator.

DISCUSSION

“The aim of this research report is to outline the development and initial implementation of the ReStOre II Clinical Monitoring Strategy” the discussion implies there is more to the aim than just to outline the development and initial implementation of the strategy but that the authors were also investigating the impact of the monitoring strategy on the research team’s awareness of good clinical practice and the quality & rigor of the data. Should the aims be changed?

This is an interesting point and in hindsight it would have been useful to have added in measures of impact but no, at present, we have no plans to measure the impact of the monitoring strategy on the research team. We state in the discussion that all members of the research team complete mandatory training in ICH-GCP but we do not measure any outcomes from that training.

Competing interests: No competing interests were disclosed.

Competing interests: No competing interests were disclosed.

Competing interests: No competing interests were disclosed.

Competing interests: No competing interests were disclosed.

Competing interests: No competing interests

Competing interests: No competing interests were disclosed.

Competing interests: No competing interests
==== Refs
1 Filippatos GS de Graeff P Bax JJ : Independent academic Data Monitoring Committees for clinical trials in cardiovascular and cardiometabolic diseases. Eur J Heart Fail. 2017;19 (4 ):449–456. 10.1002/ejhf.761 28271595
2 Fleming TR DeMets DL Roe MT : Data monitoring committees: Promoting best practices to address emerging challenges. Clin Trials. 2017;14 (2 ):115–123. 10.1177/1740774516688915 28359194
3 Macefield RC Beswick AD Blazeby JM : A systematic review of on-site monitoring methods for health-care randomised controlled trials. Clin Trials. 2013;10 (1 ):104–124. 10.1177/1740774512467405 23345308
4 International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use (ICH): ICH Harmonised Guideline: Integrated Addendum to ICH E6(R1): Guideline for Good Clinical Practice E6(R2).2016. Reference Source
5 https://www.hrb.ie/fileadmin/1._Non-plugin_related_files/RSF_files/Policies_and_principles/Grant_Policies/HRB_Policy_on_Clinical_Trials_and_Interventions_Governance.pdf
6 Fougerou-Leurent C Laviolle B Tual C : Impact of a targeted monitoring on data-quality and data-management workload of randomized controlled trials: A prospective comparative study. Br J Clin Pharmacol. 2019;85 (12 ):2784–2792. 10.1111/bcp.14108 31471967
7 Love SB Yorke-Edwards V Lensen S : Monitoring in practice - How are UK academic clinical trials monitored? A survey. Trials. 2020;21 (1 ): 59. 10.1186/s13063-019-3976-1 31918743
8 Brosteanu O Schwarz G Houben P : Risk-adapted monitoring is not inferior to extensive on-site monitoring: Results of the ADAMON cluster-randomised study. Clin Trials. 2017;14 (6 ):584–596. 10.1177/1740774517724165 28786330
9 Gates A Caldwell P Curtis S : Reporting of data monitoring committees and adverse events in paediatric trials: a descriptive analysis. BMJ Paediatr Open. 2019;3 (1 ): e000426. 10.1136/bmjpo-2018-000426 31206076
10 Simon GE Shortreed SM Rossom RC : Principles and procedures for data and safety monitoring in pragmatic clinical trials. Trials. 2019;20 (1 ): 690. 10.1186/s13063-019-3869-3 31815644
11 Yao B Zhu L Jiang Q : Safety monitoring in clinical trials. Pharmaceutics. 2013;5 (1 ):94–106. 10.3390/pharmaceutics5010094 24300399
12 Schöffski P : Importance and role of independent data monitoring committees (IDMCs) in oncology clinical trials. BMJ Open. 2021;11 (10 ): e047294. 10.1136/bmjopen-2020-047294 34697110
13 Administration FaD: Establishment and Operation of Clinical Trial Data Monitoring Committees.2006. Reference Source
14 Bennett AE O'Neill L Connolly D : Perspectives of Esophageal Cancer Survivors on Diagnosis, Treatment, and Recovery. Cancers (Basel). 2020;13 (1 ):100. 10.3390/cancers13010100 33396253
15 Guinan EM Doyle SL Bennett AE : Sarcopenia during neoadjuvant therapy for oesophageal cancer: characterising the impact on muscle strength and physical performance. Support Care Cancer. 2018;26 (5 ):1569–1576. 10.1007/s00520-017-3993-0 29197960
16 O'Neill L Moran J Guinan EM : Physical decline and its implications in the management of oesophageal and gastric cancer: a systematic review. J Cancer Surviv. 2018;12 (4 ):601–618. 10.1007/s11764-018-0696-6 29796931
17 Gannon JA Guinan EM Doyle SL : Reduced fitness and physical functioning are long-term sequalae after curative treatment for esophageal cancer: a matched control study. Dis Esophagus. 2017;30 (8 ):1–7. 10.1093/dote/dox018 28575241
18 Guinan EM Bennett AE Doyle SL : Measuring the impact of oesophagectomy on physical functioning and physical activity participation: a prospective study. BMC cancer. 2019;19 (1 ): 682. 10.1186/s12885-019-5888-6 31299920
19 Guinan EM Doyle S O'Neill L : Rehabilitation strategies following oesophageal cancer: The ReStOre trial—A feasibility study. Journal of Clinical Oncology. 2016;34 (3_suppl ):176–176. 10.1200/jco.2016.34.3_suppl.176
20 O'Neill L Guinan E Doyle S : Rehabilitation strategies following esophageal cancer(the ReStOre trial); a feasibility study. Diseases of the Esophagus. 2017;30 (5 ):1–8. 10.1093/dote/dow012
21 O'Neill LM Guinan E Doyle SL : The RESTORE Randomized Controlled Trial: Impact of a Multidisciplinary Rehabilitative Program on Cardiorespiratory Fitness in Esophagogastric cancer Survivorship. Ann Surg. 2018;268 (5 ):747–755. 10.1097/sla.0000000000002895 30004915
22 Kennedy SA Annett SL Dunne MR : Effect of the Rehabilitation Program, ReStOre, on Serum Biomarkers in a Randomized Control Trial of Esophagogastric Cancer Survivors. Front Oncol. 2021;11 : 669078. 10.3389/fonc.2021.669078 34604026
23 O’Neill L Guinan E Doyle S : Rehabilitation strategies following oesophagogastric and Hepatopancreaticobiliary cancer (ReStOre II): a protocol for a randomized controlled trial. BMC cancer. 2020;20 (1 ): 415. 10.1186/s12885-020-06889-z 32404096
24 Organisation SCT: Guidelines for Good Operation Practice V2.2011. Reference Source
25 Guinan E O’Neill L Hussey J : Development and initial Implementation of a Clinical Monitoring Strategy in a Non-regulated Trial: a research note from the ReStOre II Trial.2023. 10.17605/OSF.IO/NMX3G
