
==== Front
BMJ Open
BMJ Open
bmjopen
bmjopen
BMJ Open
2044-6055
BMJ Publishing Group BMA House, Tavistock Square, London, WC1H 9JR

39231549
10.1136/bmjopen-2024-090084
bmjopen-2024-090084
Protocol
Genetics and Genomics
1697
1506
Genetics Navigator: protocol for a mixed methods randomized controlled trial evaluating a digital platform to deliver genomic services in Canadian pediatric and adult populations
http://orcid.org/0000-0002-1068-3154
D'Amours Guylaine 1guylaine.damours@mail.mcgill.ca

Clausen Marc 1marc.clausen@unityhealth.to

Luca Stephanie 2stephanie.luca@sickkids.ca

Reble Emma 1Emma.Reble@unityhealth.to

Kodida Rita 1Rita.Kodida@unityhealth.to

http://orcid.org/0000-0001-8124-3247
Assamad Daniel 2daniel.abdassamad@sickkids.ca

Bernier Francois 3francois.bernier@albertahealthservices.ca

http://orcid.org/0000-0003-0468-6663
Chad Lauren 45lauren.chad@sickkids.ca

Costain Gregory 46gregory.costain@sickkids.ca

Dhalla Irfan 78irfan.dhalla@unityhealth.to

Faghfoury Hanna 910Hanna.Faghfoury@uhn.ca

Friedman Jan M 11jmf@bcchr.ca

Hewson Stacy 46stacy.hewson@sickkids.ca

Jamieson Trevor 12jamiesont@smh.ca

Silver Josh 69Josh.Silver@uhn.ca

Shuman Cheryl 6cheryl.shuman@utoronto.ca

Osmond Matthew 13MaOsmond@cheo.on.ca

Carroll June C 1415June.Carroll@sinaihealth.ca

Jobling Rebekah 4rebekah.jobling@sickkids.ca

Laberge Anne-Marie 1617anne-marie.laberge.med@ssss.gouv.qc.ca

Aronson Melyssa 618Melyssa.Aronson@sinaihealth.ca

Liston Eriskay 46eriskay.liston@sickkids.ca

http://orcid.org/0000-0003-3685-5679
Lerner-Ellis Jordan 1920jordan.lerner-ellis@sinaihealth.ca

Marshall Christian 2021crm@sickkids.ca

Brudno Michael 22brudno@cs.toronto.edu

https://twitter.com/qthipie
http://orcid.org/0000-0002-0540-4181
Pham Quynh 823q.pham@uhn.ca

http://orcid.org/0000-0002-1139-3423
Rudzicz Frank 2425frank@dal.ca

Cohn Ronald 426ronald.cohn@sickkids.ca

Mamdani Muhammad 27Muhammad.Mamdani@unityhealth.to

Smith Maureen 28maureen_smith@rogers.com

Shastri-Estrada Serena 2930s.shastri@mail.utoronto.ca

http://orcid.org/0000-0002-8723-5915
Seto Emily 823emily.seto@utoronto.ca

http://orcid.org/0000-0002-7586-3893
Thorpe Kevin 31kevin.thorpe@utoronto.ca

Ungar Wendy 28wendy.ungar@sickkids.ca

http://orcid.org/0000-0003-3269-8004
Hayeems Robin Z 28robin.hayeems@sickkids.ca

http://orcid.org/0000-0002-9516-4539
Bombard Yvonne 18yvonne.bombard@utoronto.ca

1 Genomics Health Services Research Program, Li Ka Shing Knowledge Institute, St. Michael’s Hospital, Toronto, Ontario, Canada
2 Program in Child Health Evaluative Sciences, The Hospital for Sick Children, Toronto, Ontario, Canada
3 Department of Medical Genetics, Alberta Children's Hospital, Calgary, Alberta, Canada
4 Division of Clinical and Metabolic Genetics, The Hospital for Sick Children, Toronto, Ontario, Canada
5 Department of Pediatrics, University of Toronto, Toronto, Ontario, Canada
6 Department of Molecular Genetics, University of Toronto, Toronto, Ontario, Canada
7 Care Experience Institute, Unity Health Toronto, Toronto, Ontario, Canada
8 Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, Ontario, Canada
9 Fred A. Litwin Family Centre in Genetic Medicine, University Health Network, Toronto, Ontario, Canada
10 Sinai Health, Toronto, Ontario, Canada
11 Department of Medical Genetics, The University of British Columbia, Vancouver, British Columbia, Canada
12 Department of Medicine, University of Toronto, Toronto, Ontario, Canada
13 Children's Hospital of Eastern Ontario Research Institute, University of Ottawa, Ottawa, Ontario, Canada
14 Department of Family Medicine, Sinai Health, Toronto, Ontario, Canada
15 Department of Family and Community Medicine, University of Toronto, Toronto, Ontario, Canada
16 Division of Medical Genetics, CHU Sainte-Justine, Montreal, Québec, Canada
17 Department of Pediatrics, Université de Montréal, Montreal, Québec, Canada
18 Zane Cohen Centre for Digestive Diseases, Sinai Health, Toronto, Ontario, Canada
19 Pathology and Laboratory Medicine, Mount Sinai Hospital, Toronto, Ontario, Canada
20 Department of Laboratory Medicine and Pathobiology, University of Toronto, Toronto, Ontario, Canada
21 Genome Diagnostics, Department of Pediatric Laboratory Medicine, The Hospital for Sick Children, Toronto, Ontario, Canada
22 HPC4Health Consortium, Toronto, Ontario, Canada
23 Centre for Digital Therapeutics, Toronto General Hospital Research Institute, University Health Network, Toronto, Ontario, Canada
24 Faculty of Computer Science, Dalhousie University, Halifax, Nova Scotia, Canada
25 Vector Institute for Artificial Intelligence, Toronto, Ontario, Canada
26 Program in Genetics & Genome Biology, The Hospital for Sick Children, Toronto, Ontario, Canada
27 Department of Data Science and Advanced Analytics, Li Ka Shing Knowledge Institute, St. Michael’s Hospital, Toronto, Ontario, Canada
28 Patient Partner, Canadian Organization for Rare Disorders, Toronto, Ontario, Canada
29 Department of Occupational Science and Occupational Therapy, Temerty Faculty of Medicine, University of Toronto, Toronto, Ontario, Canada
30 Genetics Navigator Advisory Board, Toronto, Ontario, Canada
31 Dalla Lana School of Public Health, University of Toronto, Toronto, Ontario, Canada
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

YB and MC are cofounders of Genetics Adviser.

DrYvonneBombard; yvonne.bombard@utoronto.ca
2024
03 9 2024
14 9 e09008416 6 2024
21 8 2024
Copyright © Author(s) (or their employer(s)) 2024. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.
2024
https://creativecommons.org/licenses/by-nc/4.0/ This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

Abstract

Introduction

Genetic testing is used across medical disciplines leading to unprecedented demand for genetic services. This has resulted in excessive waitlists and unsustainable pressure on the standard model of genetic healthcare. Alternative models are needed; e-health tools represent scalable and evidence-based solution. We aim to evaluate the effectiveness of the Genetics Navigator, an interactive patient-centred digital platform that supports the collection of medical and family history, provision of pregenetic and postgenetic counselling and return of genetic testing results across paediatric and adult settings.

Methods and analysis

We will evaluate the effectiveness of the Genetics Navigator combined with usual care by a genetics clinician (physician or counsellor) to usual care alone in a randomised controlled trial. One hundred and thirty participants (adults patients or parents of paediatric patients) eligible for genetic testing through standard of care will be recruited across Ontario genetics clinics. Participants randomised into the intervention arm will use the Genetics Navigator for pretest and post-test genetic counselling and results disclosure in conjunction with their clinician. Participants randomised into the control arm will receive usual care, that is, clinician-delivered pretest and post-test genetic counselling, and results disclosure. The primary outcome is participant distress 2 weeks after test results disclosure. Secondary outcomes include knowledge, decisional conflict, anxiety, empowerment, quality of life, satisfaction, acceptability, digital health literacy and health resource use. Quantitative data will be analysed using statistical hypothesis tests and regression models. A subset of participants will be interviewed to explore user experience; data will be analysed using interpretive description. A cost-effectiveness analysis will examine the incremental cost of the Navigator compared with usual care per unit reduction in distress or unit improvement in quality of life from public payer and societal perspectives.

Ethics and dissemination

This study was approved by Clinical Trials Ontario. Results will be shared through stakeholder workshops, national and international conferences and peer-reviewed journals.

Trial registration number

NCT06455384.

delivery of health care, integrated
eHealth
genetics
health services
randomized controlled trial
methods
http://dx.doi.org/10.13039/501100000037 Institute of Health Services and Policy Research Team Grant / PHT 178433
==== Body
pmcSTRENGTHS AND LIMITATIONS OF THIS STUDY

Our study uses a randomised controlled trial design and includes a comprehensive range of outcome measures, which will provide a rigorous and thorough assessment of an e-health platform that integrates all the components of genetics services delivery.

This study will include a cost-effectiveness analysis, which will provide quantitative evidence about the economic impact from the perspective of the healthcare system and society.

The qualitative component in our study will generate evidence to foster deeper understanding of the needs and experiences of patients requiring genetics services.

All participants in our study are English speaking and need to have access to a computer and the internet, which could limit the generalisability of our results.

Introduction

Genetic testing, including targeted panels, exome and genome sequencing (GS), is increasingly being used in mainstream care across medical disciplines as it improves diagnosis and patient management. With substantial improvements in diagnostic performance, there has been unprecedented demand for genetic testing for a broad range of clinical indications.13 The volume of testing and complexity of genomic analysis places unsustainable pressure on the standard model of care for delivering genetics services, which is heavily dependent on multiple interconnected specialists including medical geneticists, genetic counsellors, clinical laboratory directors, bioinformaticians and genome analysts based in tertiary care centres.2 4 This pressure is intensified by already critical workforce shortages in genetics, shortages that will become more severe in coming years as genetic knowledge and clinical applications continue to expand.2 3 These shortages have resulted in excessive waitlists, prompting the use of alternative models of care including remote genetic counselling (telephone, e-mail and videoconferencing),58 electronic consultations between specialists and patients or primary care,911 and the integration of genetics education strategies into primary care.1215 While these approaches have contributed to mitigating current service delivery challenges, comprehensive adaptive patient-facing e-health tools can provide more scalable and patient-centred solutions.

The patient preference literature supports the use of e-health tools in a range of clinical settings.1619 A systematic review of patient portals found that they can improve psychological outcomes such as decision-making and self-efficacy.20 In interviews, patients with cancer report that waiting for a verbal confirmation of results from a provider was more distressing than accessing the results online through a portal as it enabled a sense of control.2123 Portals also provide patients with timely access to their health information, increasing efficiency of service delivery.24 25 Patients have also found e-health tools to be acceptable, easy to use, trustworthy and facilitative of personalised care in various clinical settings.2633 Many e-health tools exist in clinical genetics, and these have also been well-received by patients, particularly with respect to enabling self-paced, private and informative counselling.1618 However, the e-health tools developed to date, including chatbots,34 support single components of genetic service delivery, such as pretest or risk education,1617 24 25 3537 decision-making to pursue testing and receive various result types,1822 23 3840 or results disclosure.4144 These innovations are limited in scope and scale, as none of them integrates all of the components of the service delivery pathway for both paediatric and adult populations.

We developed the Genetics Navigator, a patient-centred e-health navigation platform for genetic service delivery in paediatric and adult populations. This platform extends the Genetics Adviser45 to include collection of medical and family history, and a chatbot to provide responses to frequently asked questions and store additional patient questions (the chatbot’s development and evaluation will be described in a separate manuscript).46

Aims and hypotheses

Aims

We aim to evaluate the effectiveness, cost-effectiveness and user experience of the Genetics Navigator compared with usual care (clinician-delivered care) with patients undergoing genetic testing.

Theoretical framework and hypotheses

Consistent with the Theory of Planned Behavior (TPB),47 we hypothesise that using the Genetics Navigator will improve emotional functioning (decrease distress, anxiety/depression and decisional conflict), knowledge and satisfaction with decision-making compared with usual care. In addition, we hypothesise that the Genetics Navigator will be cost effective compared with usual care and associated with less health resource use.

Methods

Study design

This is a mixed-method, non-blinded randomised controlled superiority trial where we will evaluate the effectiveness and the cost-effectiveness of the Genetics Navigator and usual care (intervention) in reducing patient distress compared with usual care alone (control). Within this trial, patients will receive genetic test results related to a range of clinical indications as part of their standard of care testing. All participants will complete baseline measures and randomisation, after which they will either use the Genetics Navigator platform prior to or after their medical geneticist/genetic counsellor appointment (intervention) or receive usual care and meet with their medical geneticist/genetic counsellor as is standard clinic practice (control). Outcomes will be measured at baseline (T0), pretest (T1–T2) and post-test (T3–T4) (figure 1). Consistent with the TPB,47 the primary outcome is patient distress after receiving genetic test results, and secondary outcomes include knowledge, decisional conflict, anxiety/depression, empowerment, quality of life, satisfaction, acceptability, digital health literacy and health resource use (table 1). We will also conduct a qualitative study with a subgroup of participants, who will be interviewed about their experience using the Genetics Navigator. The study was approved by Clinical Trials Ontario and this manuscript follows the 2013 and 2022 Standard Protocol Items: Recommendations for Interventional Trials guidance for protocols on clinical trials (see online supplemental file 1).48 49

Table 1 Summary of study measures across time points* for each participant arm†

Measure	Pre-test	Post-test	
Baseline	+~2 weeks	+ 1 month	+ 6 months	+ 9 months	
T0	T1I	T2I/T2C	T2IA/T2CA	T3I/T3IA	T3C	T4I/T4IA	T4C	
Demographics	•								
Brief Health Literacy Screening Tool (BRIEF)	•								
Digital Literacy Scale (DHLS)	•								
University of North Carolina Genomic Knowledge Scale (UNC-GKS)	•	•	•		•	•	•	•	
Hospital Anxiety and Depression Scale (HADS)	•	•	•	•	•	•	•	•	
SF-36: Quality of life	•	•	•		•	•	•	•	
Decisional Conflict (SURE)			•	•					
Genomics Outcome Scale (GOS)		•	•		•	•	•	•	
Preparation for Decision Making Scale			•						
Satisfaction with Decision Making			•						
Acceptability eScale		•	•		•		•		
Duration of platform use (minutes)		•			•		•		
Frequency of platform access		•			•		•		
Health resource use (RUQ)			•‡				•§	•§	
Multidimensional Impact of Cancer Risk Assessment (MICRA–Adapted)					•¶	•¶	•	•	
* Time points abbreviations. T0: baseline; T1: 2 weeks before first appointment with clinician; T2: immediately after pre-test meeting with clinician; T3: 2 weeks after post-test meeting with clinician; T4: 3 months after T3 measures.

† Participant arm abbreviations. A: alternative pathway; C: control group; I: intervention group (see online supplemental table 1online supplemental table 1 for specific measures in each arm).

‡ In the past 3 months.

§ Since receiving results.

¶ Primary outcome (test-specific distress subscale).

Figure 1 Study flow: timing of follow-up measures for each participant group. Follow-up measures will be collected from participants at baseline (T0, yellow) and four subsequent timepoints (T1, T2, T3 and T4) in the intervention arm (I, light blue, pink and orange) and three subsequent timepoints (T2, T3 and T4) in the control arm (C, dark blue and purple). Participants who do not proceed with genetic testing will complete a subset of measures (T2IA and T2CA). All participants in the intervention arm will complete the same measures. Primary outcome will be collected 2 weeks after meeting with the clinician for post-test counselling (T3) in both arms (T3I, T3IA and T3C). *A: alternative pathway; C: control arm; I: intervention arm; T0: baseline; T1: 2 weeks before first appointment with clinician; T2: immediately after pretest meeting with clinician; T3: 2 weeks after post-test meeting with clinician; T4: 3 months after T3 measures.

Population

We will recruit adult genetics patients at Sinai Health, and parents/legal guardians of paediatric patients at the Hospital for Sick Children (SickKids) who have been referred to clinical genetics and for whom genetic testing may be offered as part of their standard care. These specialised clinics (area of specialisation listed in the inclusion criteria below) were selected for feasibility purposes, since they evaluate patients with a more predictable clinical pathway who have a high likelihood of being eligible for genetic testing.

Eligibility criteria

Inclusion

Adult patients (18 years of age or older) who are referred to the clinical genetics service at Sinai Health for the following primary indication: at risk for a cancer predisposition syndrome.

Parents/legal guardians (18 years of age or older) of paediatric patients who are referred to the clinical genetics service at SickKids for one or more of the following primary indications: epilepsy, neurodevelopmental or cardiac conditions.

Exclusion

Known not to be eligible for clinical genetic testing coverage in Ontario.

Requires urgent clinical genetic testing or prenatal genetic testing.

Not fluent in English (speaking and reading).

Intervention

Participants in the intervention arm will use the Genetics Navigator to support the delivery of genetic services, including intake, education, pretest and post-test counselling, and results delivery with physician-generated management recommendations. All participants will use the Genetics Navigator prior to their pretest visit with their genetics clinician. They will also use the Genetics Navigator as part of results delivery by their genetics clinician, either before, or after their post-test genetic counselling visit. The Navigator will be supported by a mix of in-person/phone/video-conferencing consults with genetics professionals. The timing of use of the Genetics Navigator will be determined by the participant’s clinician. This is further described under data collection.

Control

Control arm participants will complete their scheduled appointments (pre-test and post-test) with their genetics clinician. During these appointments, the clinician will follow standard of care procedures, as further described under data collection.

Outcomes

Primary outcomes

The primary outcome is Test-Specific Distress 2 weeks after receiving results (post-test genetic counselling visit), as measured by an adapted version of the Multidimensional Impact of Cancer Risk Assessment (MICRA).50 The MICRA is a validated 25-item questionnaire with excellent psychometric properties that measures distress, uncertainty and positive experiences following genetic testing.50 Questions will be adapted for non-cancer-based genetic test results. There are 6 items measuring distress, scored from 0 to 5, and total score for this domain ranges from 0 (no distress) to 30 (very high distress).

Secondary outcomes

The following secondary outcomes will be measured for patient and parent participants: Test-Specific Uncertainty and Positive Experiences (measured by the MICRA),50 Knowledge (measured by the University of North Carolina Genomic Knowledge Scale),51 Decisional Conflict (measured by the SURE),52 Anxiety and Depression (measured by the Hospital Anxiety and Depression Scale,53 Empowerment (measured by the Genomics Outcome Scale),54 Health Literacy (measured by the Brief Health Literacy Screening Tool),55 Quality of Life (measured by the 36-item Short Form Survey (SF-36)),56 Satisfaction (measured by the Satisfaction with Decision Making Scale and the Preparation for Decision Making Scale),57 58 Acceptability (measured by the acceptability eScale),59 mental health resources utilisation (measured by the Resource Use Questionnaire developed by Ungar et al) and Digital Health Literacy (measured by the Digital Health Literacy Scale).60 Duration of visits with the genetics clinician (physician and/or genetic counsellor) and time spent on clinical tasks by genetics assistants (if applicable) will be tracked in 15-min increments by the participating clinician for a representative subset of participants in the intervention and the control groups. Medical information will be collected via chart review. Frequency of platform use, duration of platform session and readiness (for genetic testing and for results disclosure) will be gathered from the platform.

Demographics variables to be collected include age, sex, gender,61 62 education, self-reported ethnicity, employment status, marital status, number of children, geographical location of residence (via first segment of the postal code to identify Forward Sortation Area) and reason for referral to genetics.

Sample size

The primary outcome is patient/parent distress; Cella et al reported a SD ranging 2–7 on the MICRA distress subscale.50 Using a minimally clinically important difference (MCID) of 0.5 SDs results in absolute differences ranging between 1 and 3.5 depending on SD.50 Assuming this, we estimate that 65 participants/arm will be required to have 80% power at a two-sided significance level of 0.05 to detect the MCID using a standard two-sample t-test. The MICRA distress subscale has a range of 0–30; based on Cella et al,50 we expect the control group to have means between 1 and 6, which leaves sufficient room for improvement on the scale. Based on previous studies attrition rates, we anticipate having to possibly oversample to account for attrition and to account for participants who are not offered or opt not to receive genetic testing, up to a total sample size of 170 participants (85 per group).63

Recruitment

Adult patients or parents/legal guardians of paediatric patients will be recruited from participating genetics clinics at Sinai Health and the SickKids, starting July 2024. Prospectively, adult patients or parents/legal guardians of paediatric patients referred to clinical genetics for testing will be identified by the medical geneticist to whom they have been referred. Eligible participants will be briefly informed about the study by the respective clinic’s coordinator or genetics assistant during their phone consult to book their standard-of-care genetics appointment. Names and contact information of prospective participants who agree to be contacted by the study team for more information will be provided by the genetic clinic to the study team via secure file transfer or over the phone. Potential participants will also have the option of contacting the study coordinator directly and the genetics clinic will offer to provide the study invitation package, as well as the study coordinator’s phone and email contact information. The study team will then contact interested participants via phone and explain the study further and explore potential participants’ level of interest.

Any patients who are not interested in participating in the study will be asked by the genetics clinic or the study team if they are willing to provide a reason for declining. This information will be recorded on the recruitment log. After the study is completed, data on individuals who declined to participate in the study will be aggregated so that this subpopulation can be accurately described. Recruitment will last for up to 1 year, and data collection is expected to end in July 2026.

Randomisation

Participants will be consecutively randomised and allocated using a computer-generated randomisation scheme in a 1:1 ratio with random permuted blocks of varying sizes, stratified by adult patients versus parents of paediatric patients. A list of allocations will be generated using the randomisation function within REDCap.

Data collection

Once the participant has consented to the study, they will be assigned a study ID. The study coordinator will administer the baseline survey (T0) and then obtain allocation from REDCap. The distribution of study measures is schematised in figure 1, and specific measures collected at each timepoint are summarised in table 1. Outcomes will be measured at baseline (T0), in the pretest period (T1–T2), and in the post-test period (T3–T4). There will be four outcome measure timepoints after baseline: 2 weeks before (T1I) and immediately after the pretest visit with the genetics clinician (T2I/T2C), 2 weeks after (T3I/T3IA/T3C) and 3 months after (T4I/T4IA/T4C) the post-test visit with the genetics clinician.

Health resource use will be assessed immediately after the pretest visit with the genetics clinician (T2I/T2C) and 3 months after the post-test visit (T4I/T4IA/T4C). The health resource use questionnaire focuses on the use of mental health and support services during and immediately following the study period. Finally, the duration of visits with the genetics clinician and the time spent on clinical tasks by genetics assistants (if applicable) will be tracked by the participating clinician for a subset of participants selected across all clinicians by the research team in both groups. For the intervention group, the duration of participants’ platform use and frequency of platform access will be assessed immediately after using the Genetics Navigator (T1I), and 2 weeks after the post-test visit with the clinician (T3I/T3IA), as well as 3 months after the post-test visit (T4I/T4IA) via platform data collection and google analytics.

Intervention

At the end of the baseline session (T0), intervention arm participants will be given the link to the online Genetics Navigator with instructions to log into the Genetics Navigator within 2 weeks before their scheduled pretest appointment with their genetics clinician. Two weeks prior to the participant’s appointment with their clinician, the study coordinator will send a reminder email with the link and instructions. The use of the Genetics Navigator will take approximately 30 min. After completing this pretest portion, the Genetics Navigator will prompt the participant to complete the follow-up measures (T1I).

The Genetics Navigator platform will notify the study coordinator that the participant has completed the pretest portion of the Genetics Navigator via an automated email. Prior to the clinician and participant appointment, the clinician will review the participant’s medical and family history, and readiness for genetic testing, which will be discussed with the participant during the appointment. The clinician will also address other clarifications or questions they may have. After the appointment, the clinician will order the genetic testing, if indicated and desired by the participant.

If the participant does not wish to pursue testing or the clinician determines that testing is not appropriate for the participant, they will be asked to complete a subset of follow-up measures (T2IA). After the participant has completed the clinician visit, the clinician will notify the study coordinator, who will email the participant a REDCap link instructing them to complete a set of follow-up measures (T2I/T2IA).

Once the participant’s genetic test results are ready, they will be contacted by the clinician to schedule an appointment to discuss their results following their standard clinical practice. The clinical team will upload the results and care plan to the platform and determine if they can be delivered to the patient before the appointment based on provider’s criteria. The recruiting clinic will also notify the study team of the participant’s appointment and the nature of their results.

Intervention participants—results returned online before the appointment with clinician

Participants will be able to view their results prior to meeting with the clinician. Participants will be notified 2 weeks prior to their scheduled clinician appointment to log in to the Genetics Navigator to review their results and care plan.

The participant will then meet with their clinician to review their results and complete a final set of post-test follow-up measures (T3I and T4I).

Intervention participants—results returned during appointment with clinician

During the results appointment, the clinician will explain the participant’s results and any recommended referrals or management plans.

After the appointment, participants will use the Genetics Navigator platform to review their results and care plan.

The participant will complete a final set of post-test follow-up measures (T3IA and T4IA).

All participant accounts in the Genetics Navigator will remain functional until the end of the study and will automatically be disabled once their participation is over.

Control

Control arm participants will complete their scheduled pretest appointment with their genetics clinician, during which standard of care procedures will be followed. If genetic testing is indicated and the participant chooses to proceed with testing, the clinician will order testing after their meeting. If the participant does not choose to go through with genetic testing or if their clinician determines that testing is not appropriate for the participant, they will be asked to complete a subset of follow-up measures (T2CA).

Following the clinician visit, the clinician will notify the study coordinator that the participant has completed their meeting. The study coordinator will then send the participant a link to REDCap survey via email with instructions to complete the pretest follow-up measures (T2C/T2CA).

If testing is performed, once the participant’s results are ready, they will be contacted by the clinician to schedule an appointment to receive and discuss their results following their standard clinical practice.

Control participants

During the results appointment, the clinician will explain the participant’s results and any recommended referrals or management plans (standard of care).

The clinician will then notify the study coordinator that the participant has completed their clinician meeting.

After the appointment, the study coordinator will send the participant a link (via email) to REDCap with instructions to complete the post-test follow-up measures (T3C and T4C).

Medical chart data

For both intervention and control participants, medical information will be obtained from the medical chart notes from pretest and post-test visits. The study team will access medical chart information for all participants (with their consent), or child of the participants, from the recruiting clinic. The following information will be extracted: clinical indication for testing, medical history, genetic testing history and dates of clinical visits, genetic testing, test result and result disclosure. Wait times for clinic appointments will not be captured and are not expected to differ between randomised groups as patients will have their appointments scheduled the same way for both arms (scheduled according to standard clinical procedures at their clinic). The interval between testing and results disclosure will be calculated for each participant and mean intervals are expected to be similar between the control and intervention arms. If they differ, this variable will be included as a potential confounder for the analysis of the primary outcome.

Data analysis

Primary and secondary outcomes

Adult patient and parent/legal guardian data will be analysed together as the primary analysis, and separately as a subanalysis. Each analysis will follow the intention-to-treat approach. As this trial is positioned as a superiority trial, the null and alternative hypotheses are posited for the primary outcome: the Genetics Navigator is not better than or the same as the standard of care (H0), or the Genetics Navigator is better than the standard of care (H1). The mean scores for test-specific distress (distress subscale on the MICRA) will be compared using a t-test and the mean difference with 95% CI will be reported. The t-test is robust to non-normality; however, if diagnostics reveal significant concerns with respect to the distribution, a non-parametric test will be used to compare the locations of the distributions and bootstrapping will be used to derive 95% CIs on the mean difference. MICRA subscales (uncertainty and positive experience) and secondary outcomes of knowledge, decisional conflict, anxiety/depression, empowerment, quality of life, satisfaction, acceptability, digital health literacy and consult time will also be analysed using a t-test or regression model adjusting for baseline where appropriate with treatment effects expressed a mean (adjusted) differences with 95% CIs. These additional analyses are for hypothesis-generating purposes; therefore, no multiplicity corrections will be applied.

The primary time point of comparison will be T3I/T3IA (pooled) for the intervention group (Genetics Navigator+clinician) versus T3C for the control group (clinician only) (figure 1).

Exploratory outcomes

Secondary exploratory analyses will examine the impact of the platform at baseline and follow-up timepoints (ie, T1, T2 and T4) on all outcomes using mixed-effects regression models. Mean total consult times and frequency of genetics encounters between those assigned to the platform and those assigned to usual care will be analysed descriptively and compared using a t-test if appropriate. Proportion of cases from specific geographical regions assigned to the platform and usual care will be compared using a χ2 test.

Cost-effectiveness analysis

An incremental cost-effectiveness analysis (CEA) that compares the GN platform costs to usual care per unit improvement in MICRA score from the perspectives of the healthcare system and society will be undertaken alongside the randomised controlled trial (RCT). Adult patients and parents will be analysed separately. The CEA will be undertaken using data from cases who have completed the resource-use questionnaire and the MICRA. No imputation will be undertaken. The economic evaluation will use recommended methods.64

Costing

Direct healthcare resource use for diagnosis, genetic counselling and education pre-GS and post-GS will be measured over the study period for both study groups in adults and in parents. Use of the platform (navigation time and costs), mental health services, genetic testing, genetic counselling, genetics education, laboratory tests and any other services related to diagnosis and mental health will be collected prospectively using a customised team-developed resource use questionnaire. Data collection will occur in tandem with other RCT assessments. Prices for interventions, services and other direct costs will be obtained from study partners, provincial programme administrators, agencies, provincial fee schedules and published sources. Total costs per patient and per parent will be determined by multiplying the volume of resource use for each item over the study period by a corresponding unit price. For indirect costs, time losses from paid and unpaid labour of patients and parents will be collected in the resource questionnaire and multiplied by a standard wage. The mean cost per patient and parent will then be calculated for each group and aggregated to major cost categories. In the case of skewed total costs, non-parametric bootstrapping or log transformation will be applied.

Outcomes

The primary measure of effectiveness used in the CEA will be distress as measured with the MICRA. A secondary analysis will examine the incremental cost per unit improvement in SF-36 total score and mental health score.

Analysis

Person-level regressions will be conducted to determine mean costs and mean effectiveness per person for each group and results will be represented in an incremental cost-effectiveness ratio—the ratio of the difference between groups in mean cost per person to the difference in mean effectiveness. Subgroups based on age group, clinical diagnosis, GS result type, sex or other important clinical or demographic factors may be analysed. Sensitivity analysis will be undertaken to test the robustness of the results to variations in underlying assumptions regarding costly resources, such as the cost of the platform and costs of genetic testing. Results will be presented from the perspectives of the healthcare system and society.

Qualitative study

The qualitative study will explore participants’ experience using semistructured interviews. We will use an interpretive description approach65 to analyse the data. This methodology is particularly appropriate to examine and understand clinical phenomena with the goal to inform practical applications in clinical settings.66 67

Sampling

All individuals participating in the study will be asked to take part in the qualitative component of the study during the consent process. After the study is completed, a subset of participants in the intervention group will be selected to participate in the qualitative portion of the study (up to 20 adult patients and 20 parents/legal guardians in each arm). Participants will be selected by purposeful sampling, in order to attain a maximally varied sample of participants across a range of clinical indications, test results and demographic characteristics (age, sex, gender and ethnicity). Participation in the qualitative component is optional: participants will be able to decline to take part in the qualitative component and still participate the main trial.

Data collection

The study coordinator will schedule an interview with selected participants who agreed to participate in the qualitative component of the study. This interview will take place by videoconferencing or over the phone after the participant has completed all the other study activities. These interviews will take approximately 1 hour to complete and will be audio recorded and transcribed. The interview guide, developed based on a literature review and expert input, will explore participants’ perceived value and usefulness of the Navigator (see online supplemental file 2).

Data analysis

The qualitative analyses will draw on interpretive description. We will use coding and constant comparison to identify common and divergent themes to characterise the entire dataset. Two researchers will code transcripts independently; consensus on codes will be reached through discussion. Data analysis will consider participants’ sociodemographic factors that may influence perceptions of the Navigator. In keeping with qualitative methodology, data analysis will occur in conjunction with data collection. Ongoing analysis will inform the development of progressive iterations of the interview guides. Validation methods may include triangulation and member checking.68 We will use a mixed-method matrix to integrate quantitative and qualitative data to better understand participants’ experience.

Data management

All data collection activities will be coordinated from St. Michaels Hospital (SMH). Only participant recruitment and usual care clinical visits (see below) will take place at the participating hospitals (Sinai and SickKids). Contact forms from all recruiting sites will be emailed to the study team at SMH via secure file transfer. Data from medical charts will be abstracted on site and entered by the study team into a REDCap database, located at Applied Health Research Centre at SMH. Answers to follow-up questions for both groups will also be entered into REDCap. Patient name, date of birth (to confirm identity), contact information (email), personal and family health history, and results will be stored on the Genetics Navigator platform. Participants will also be able to upload photos of themselves to the platform that can be helpful for their genetics clinician to make clinical decisions, such as which genetic test to order; this will be optional and it will be made clear to participants that they are not required to upload photos to the platform. The Genetics Navigator will be stored on a server hosted by MedStack, a secure PHIPA-compliant platform for digital healthcare. A service provider agreement will be set up in order to ensure data on the server are adequately protected. Participants will be able to set up their own password when they log into the Genetics Navigator for the first time. Only study staff will have access to the secure REDCap database and the study data on the MedStack server within Canada, in compliance with privacy reviews at the institutions involved in the study. Once the study is over, the data will be removed from the REDCap and MedStack servers and stored on a secure restricted access server at SMH.

Patient and public involvement

The intervention was codeveloped with end-users and patient partners (manuscript in preparation). Patient partners were also involved in the development of the study design and will be involved throughout its conduct. In collaboration with our patient partners, we will disseminate study progress and updates through our study website and emails to participants and hold workshop with stakeholders at the end of our study to disseminate and exchange knowledge gained as described in our dissemination strategy below.

Ethics and dissemination

Ethical approval

This study has been approved by Clinical Trials Ontario (REB#4033).

Ethics

Informed consent will take place over the phone or Zoom Healthcare and will be conducted by the study coordinator. All participants will receive a copy of the consent for their own records (see online supplemental file 3). The study coordinator will review the information contained in the consent form and answer any questions regarding the study. The consent will ask for permission for access to medical records at recruiting clinics and to recontact participants for future studies. If applicable, consent or assent to access medical records will be solicited from the paediatric patients (children of participants), depending on their age and capacity. All information collected during this study, including personal or genetic information, will be kept confidential. Major protocol modifications will be communicated to institutional boards and the trial registry without delay.

Dissemination

This trial will evaluate the effectiveness of a comprehensive patient platform—the Genetics Navigator—to improve genetic service delivery and patient experience across paediatric and adult populations. We will convene a stakeholder workshop with key opinion leaders, champions of societies, laboratories and broader groups of end-users to discuss optimal approaches to integrating the platform into clinic and laboratory workflows. We will provide platform training, deliberate opportunities and strategies for its implementation in a range of clinical settings and develop a strategy to integrate and evaluate the platform within clinical and laboratory systems. We will disseminate our results to academic, patient, provider and technology development audiences, through peer-reviewed journal articles, and presentations at national and international academic meetings.

supplementary material

10.1136/bmjopen-2024-090084 online supplemental file 1

10.1136/bmjopen-2024-090084 online supplemental file 2

10.1136/bmjopen-2024-090084 online supplemental file 3

10.1136/bmjopen-2024-090084 online supplemental file 4

Review Process File
03 09 2024

Funding: This study is supported by a CIHR Team Grant (PHT 178433). Yvonne Bombard is supported by the Canada Research Chair in Genomics Health Services and Policy. Wendy J. Ungar is supported by the Canada Research Chair in Economic Evaluation and Technology Assessment in Child Health. Anne-Marie Laberge is supported by a FRQS Senior Clinician-Scientist Salary Support Grant.

Prepublication history and additional supplemental material for this paper are available online. To view these files, please visit the journal online (https://doi.org/10.1136/bmjopen-2024-090084).

Provenance and peer review: Not commissioned; peer reviewed for ethical and funding approval prior to submission.

Patient consent for publication: Not applicable.

Patient and public involvement: Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the Methods section for further details.
==== Refs
References

1 GeneDx 2000 Available https://www.genedx.com Accessed 1-Mar-2019
2 Hoskovec JM Bennett RL Carey ME et al Projecting the supply and demand for certified genetic counselors: a workforce study J Genet Couns 2018 27 16 20 10.1007/s10897-017-0158-8 29052810
3 Cooksey JA Forte G Benkendorf J et al The state of the medical geneticist workforce: findings of the 2003 survey of American Board of Medical Genetics certified geneticists Genet Med 2005 7 439 43 10.1097/01.gim.0000172416.35285.9f 16024977
4 Concert Genetics Landscape of genetic testing market growth, reimbursement trends, challenges and opportunities Am J Hum Genet 2018
5 Otten E Birnie E Ranchor AV et al Online genetic counseling from the providers’ perspective: counselors’ evaluations and a time and cost analysis Eur J Hum Genet 2016 24 1255 61 10.1038/ejhg.2015.283 26785833
6 Zierhut HA MacFarlane IM Ahmed Z et al Genetic counselors’ experiences and interest in telegenetics and remote counseling J Genet Couns 2018 27 329 38 10.1007/s10897-017-0200-x 29362948
7 Rayes N Bowen DJ Coffin T et al MAGENTA (Making Genetic testing accessible): a prospective randomized controlled trial comparing online genetic education and telephone genetic counseling for hereditary cancer genetic testing BMC Cancer 2019 19 648 10.1186/s12885-019-5868-x 31266460
8 Elliott AM Mhanni AA Marles SL et al Trends in telehealth versus on-site clinical genetics appointments in Manitoba: a comparative study J Genet Couns 2012 21 337 44 10.1007/s10897-011-9406-5 21997346
9 Kibar Y Frimberger D Kropp BP et al Accuracy of perinatal diagnosis of 45,X/46,XY mosaicism and electronic consultation of affected parents J Pediatr Urol 2009 5 274 8 10.1016/j.jpurol.2008.12.002 19329364
10 Bhola PT Liddy C Afkham A et al A pilot eConsultation service in Eastern Ontario: bridging clinical genetics and primary care Eur J Hum Genet 2019 27 1026 32 10.1038/s41431-019-0342-4 30778171
11 Carroll JC Liddy C Afkham A et al Use of eConsult to enhance genetics service delivery in primary care: A multimethod study Genet Med 2022 24 2034 41 10.1016/j.gim.2022.07.003 35947109
12 Carroll JC Makuwaza T Manca DP et al Primary care providers’ experiences with and perceptions of personalized genomic medicine Can Fam Physician 2016 62 e626 35 27737998
13 Harding B Webber C Ruhland L et al Primary care providers’ lived experiences of genetics in practice J Community Genet 2019 10 85 93 10.1007/s12687-018-0364-6 29700759
14 Harding B Webber C Rühland L et al Bridging the gap in genetics: a progressive model for primary to specialist care BMC Med Educ 2019 19 195 10.1186/s12909-019-1622-y 31185964
15 Carroll JC Grad R Allanson JE et al The gene messenger impact project: An innovative genetics continuing education strategy for primary care providers J Contin Educ Health Prof 2016 36 178 85 10.1097/CEH.0000000000000079 27583994
16 Green MJ Biesecker BB McInerney AM et al An interactive computer program can effectively educate patients about genetic testing for breast cancer susceptibility Am J Med Genet 2001 103 16 23 10.1002/ajmg.1500 11562929
17 Green MJ McInerney AM Biesecker BB et al Education about genetic testing for breast cancer susceptibility: Patient preferences for a computer program or genetic counselor Am J Med Genet 2001 103 24 31 10.1002/ajmg.1501 11562930
18 Green MJ Peterson SK Baker MW et al Effect of a computer-based decision aid on knowledge, perceptions, and intentions about genetic testing for breast cancer susceptibility: a randomized controlled trial JAMA 2004 292 442 52 10.1001/jama.292.4.442 15280342
19 Biesecker BB Lewis KL Umstead KL et al Web platform vs in-person genetic counselor for return of carrier results from exome sequencing: A randomized clinical trial JAMA Intern Med 2018 178 338 46 10.1001/jamainternmed.2017.8049 29356820
20 Stinson J Wilson R Gill N et al A systematic review of internet-based self-management interventions for youth with health conditions J Pediatr Psychol 2009 34 495 510 10.1093/jpepsy/jsn115 19029142
21 Royal College of Physicians Consultant physicians working with patients 2013
22 Schwartz MD Valdimarsdottir HB DeMarco TA et al Randomized trial of a decision aid for BRCA1/BRCA2 mutation carriers: impact on measures of decision making and satisfaction Health Psychol 2009 28 11 9 10.1037/a0013147 19210013
23 Bombard Y Clausen M Shickh S et al Effectiveness of the genomics ADvISER decision aid for the selection of secondary findings from genomic sequencing: a randomized clinical trial Genet Med 2020 22 727 35 10.1038/s41436-019-0702-z 31822848
24 Hanprasertpong T Rattanaprueksachart R Janwadee S et al Comparison of the effectiveness of different counseling methods before second trimester genetic amniocentesis in Thailand Prenat Diagn 2013 33 1189 93 10.1002/pd.4222 23963662
25 Sanderson SC Suckiel SA Zweig M et al Development and preliminary evaluation of an online educational video about whole-genome sequencing for research participants, patients, and the general public Genet Med 2016 18 501 12 10.1038/gim.2015.118 26334178
26 Birnie KA Kulandaivelu Y Jibb L et al Usability testing of an interactive virtual reality distraction intervention to reduce procedural pain in children and adolescents with cancer J Pediatr Oncol Nurs 2018 35 406 16 10.1177/1043454218782138 29950139
27 Jibb LA Cafazzo JA Nathan PC et al Development of a mHealth real-time pain self-management app for adolescents with cancer: An iterative usability testing study J Pediatr Oncol Nurs 2017 34 283 94 10.1177/1043454217697022 28376666
28 Stiles-Shields C Garcia B Villota K et al Exploring an existing weight management app for use with adolescents and young adults with spina bifida: usability study JMIR Pediatr Parent 2019 2 e15153 10.2196/15153 31603432
29 Sandhu H Wilson K Reed N et al A mobile phone app for the self-management of pediatric concussion: Development and usability testing JMIR Hum Factors 2019 6 e12135 10.2196/12135 31152527
30 Hanghøj S Boisen KA Hjerming M et al Usability of a mobile phone app aimed at adolescents and young adults during and after cancer treatment: qualitative study (preprint) JMIR Cancer Preprint 10.2196/preprints.15008
31 Korus M Cruchley E Stinson JN et al Usability testing of the internet program:“teens taking charge: managing my transplant online Am J Hum Genet 2015 19 107 17
32 Breakey VR Warias AV Ignas DM et al The value of usability testing for Internet-based adolescent self-management interventions: “Managing Hemophilia Online.” BMC Med Inform Decis Mak 2013 13 10.1186/1472-6947-13-113
33 Stinson J McGrath P Hodnett E et al Usability testing of an online self-management program for adolescents with juvenile idiopathic arthritis J Med Internet Res 2010 12 e30 10.2196/jmir.1349 20675293
34 Schmidlen T Schwartz M DiLoreto K et al Patient assessment of chatbots for the scalable delivery of genetic counseling J Genet Couns 2019 28 1166 77 10.1002/jgc4.1169 31549758
35 Wang C Gonzalez R Milliron KJ et al Genetic counseling for BRCA1 / 2 : A randomized controlled trial of two strategies to facilitate the education and counseling process Am J Med Genet 2005 134A 66 73 10.1002/ajmg.a.30577 15690408
36 Yee LM Wolf M Mullen R et al A randomized trial of a prenatal genetic testing interactive computerized information aid Prenat Diagn 2014 34 552 7 10.1002/pd.4347 24578289
37 Castellani C Perobelli S Bianchi V et al An interactive computer program can effectively educate potential users of cystic fibrosis carrier tests Am J Med Genet 2011 155 778 85 10.1002/ajmg.a.33870
38 Green MJ Peterson SK Baker MW et al Use of an educational computer program before genetic counseling for breast cancer susceptibility: effects on duration and content of counseling sessions Genet Med 2005 7 221 9 10.1097/01.gim.0000159905.13125.86 15834239
39 Hooker GW Leventhal K-G DeMarco T et al Longitudinal changes in patient distress following interactive decision aid use among BRCA1/2 carriers: a randomized trial Med Decis Making 2011 31 412 21 10.1177/0272989X10381283 20876346
40 Birch P Adam S Bansback N et al DECIDE: a decision support tool to facilitate parents’ choices regarding genome-wide sequencing J Genet Couns 2016 25 1298 308 10.1007/s10897-016-9971-8 27211035
41 Biesecker BB Lewis KL Biesecker LG Web-based platform vs genetic counselors in educating patients about carrier results from exome sequencing-reply JAMA Intern Med 2018 178 999 10.1001/jamainternmed.2018.2236
42 Haga SB Barry WT Mills R et al Impact of delivery models on understanding genomic risk for type 2 diabetes Public Health Genomics 2014 17 95 104 10.1159/000358413 24577154
43 Williams JL Rahm AK Zallen DT et al Impact of a patient-facing enhanced genomic results report to improve understanding, engagement, and communication J Genet Couns 2018 27 358 69 10.1007/s10897-017-0176-6 29204811
44 Tabor HK Jamal SM Yu J-H et al My46: a Web-based tool for self-guided management of genomic test results in research and clinical settings Genet Med 2017 19 467 75 10.1038/gim.2016.133 27632689
45 Shickh S Hirjikaka D Clausen M et al Genetics Adviser: a protocol for a mixed-methods randomised controlled trial evaluating a digital platform for genetics service delivery BMJ Open 2022 12 e060899 10.1136/bmjopen-2022-060899
46 Luca S Clausen M Shaw A et al Finding the sweet spot: a qualitative study exploring patients’ acceptability of chatbots in genetic service delivery Hum Genet 2023 142 321 30 10.1007/s00439-022-02512-2 36629921
47 Ajzen I The theory of planned behavior Organ Behav Hum Decis Process 1991 50 179 211 10.1016/0749-5978(91)90020-T
48 Chan A-W Tetzlaff JM Altman DG et al SPIRIT 2013 statement: defining standard protocol items for clinical trials Ann Intern Med 2013 158 200 7 10.7326/0003-4819-158-3-201302050-00583 23295957
49 Butcher NJ Monsour A Mew EJ et al Guidelines for reporting outcomes in trial protocols: The SPIRIT-outcomes 2022 extension JAMA 2022 328 2345 56 10.1001/jama.2022.21243 36512367
50 Cella D Hughes C Peterman A et al A brief assessment of concerns associated with genetic testing for cancer: the Multidimensional Impact of Cancer Risk Assessment (MICRA) questionnaire Health Psychol 2002 21 564 72 12433008
51 Langer MM Roche MI Brewer NT et al Development and validation of a genomic knowledge scale to advance informed decision making research in genomic sequencing MDM Policy Pract 2017 2 2381468317692582 10.1177/2381468317692582 29928697
52 Légaré F Kearing S Clay K et al Are you SURE?: Assessing patient decisional conflict with a 4-item screening test Can Fam Physician 2010 56 e308 14 20705870
53 Snaith RP The hospital anxiety and depression scale Health Qual Life Outcomes 2003 1 29 10.1186/1477-7525-1-29 12914662
54 Grant PE Pampaka M Payne K et al Developing a short-form of the genetic counselling outcome scale: The genomics outcome scale Eur J Med Genet 2019 62 324 34 10.1016/j.ejmg.2018.11.015 30496830
55 Haun J Luther S Dodd V et al Measurement variation across health literacy assessments: implications for assessment selection in research and practice J Health Commun 2012 17 Suppl 3 141 59 10.1080/10810730.2012.712615
56 McHorney CA Ware JE Raczek AE The MOS 36-Item Short-Form Health Survey (SF-36): II. Psychometric and clinical tests of validity in measuring physical and mental health constructs Med Care 1993 31 247 63 10.1097/00005650-199303000-00006 8450681
57 Holmes-Rovner M Kroll J Schmitt N et al Patient satisfaction with health care decisions: the satisfaction with decision scale Med Decis Making 1996 16 58 64 10.1177/0272989X9601600114 8717600
58 Bennett C Graham ID Kristjansson E et al Validation of a preparation for decision making scale Patient Educ Couns 2010 78 130 3 10.1016/j.pec.2009.05.012 19560303
59 Tariman JD Berry DL Halpenny B et al Validation and testing of the acceptability E-scale for Web-based patient-reported outcomes in cancer care Appl Nurs Res 2011 24 53 8 10.1016/j.apnr.2009.04.003 20974066
60 Nelson LA Pennings JS Sommer EC et al A 3-Iitem measure of digital health care literacy: Development and validation study JMIR Form Res 2022 6 e36043 10.2196/36043 35486413
61 Pelletier R Ditto B Pilote L A composite measure of gender and its association with risk factors in patients with premature acute coronary syndrome Psychosom Med 2015 77 517 26 10.1097/PSY.0000000000000186 25984818
62 Oswald PA An examination of the current usefulness of the bem sex-role inventory Psychol Rep 2004 94 1331 6 10.2466/pr0.94.3c.1331-1336 15362413
63 Shickh S Clausen M Mighton C et al Health outcomes, utility and costs of returning incidental results from genomic sequencing in a Canadian cancer population: protocol for a mixed-methods randomised controlled trial BMJ Open 2019 9 e031092 10.1136/bmjopen-2019-031092
64 CADTH Guidelines for the economic evaluation of health technologies: Canada 2017
65 Thorne S Kirkham SR MacDonald-Emes J Interpretive description: A noncategorical qualitative alternative for developing nursing knowledge Res Nurs Health 1997 20 169 77 10.1002/(SICI)1098-240X(199704)20:2<169::AID-NUR9>3.0.CO;2-I 9100747
66 Thorne S Kirkham SR O’Flynn-Magee K The analytic challenge in interpretive description Int J Qual Methods 2004 3 1 11 10.1177/160940690400300101
67 Hunt MR Strengths and challenges in the use of interpretive description: reflections arising from a study of the moral experience of health professionals in humanitarian work Qual Health Res 2009 19 1284 92 10.1177/1049732309344612 19690208
68 Charmaz KC Qualitative interviewing and grounded theory analysis. Inside interviewing: New lenses, new concerns Thousand Oaks, CA Sage Publications Inc 2003 311 30
