
==== Front
HGG Adv
HGG Adv
Human Genetics and Genomics Advances
2666-2477
Elsevier

S2666-2477(24)00086-1
10.1016/j.xhgg.2024.100346
100346
Report
Facilitating return of actionable genetic research results from a biobank repository: Participant uptake and utilization of digital interventions
Phung Lillian 1
Wood Elisabeth 1
Egleston Brian 2
Hoffman-Andrews Lily 3
Ofidis Demetrios 1
Howe Sarah 1
Mim Rajia 1
Griffin Hannah 1
Fetzer Dominique 1
Owens Anjali 3
Domchek Susan 1
Pyeritz Reed 4
Katona Bryson 5
Kallish Staci 4
Sirugo Giorgio 4
Weaver JoEllen 4
Nathanson Katherine L. 4
Rader Daniel J. 4
Bradbury Angela R. angela.bradbury@pennmedicine.upenn.edu
167∗
1 The University of Pennsylvania, Abramson Cancer Center and Division of Hematology-Oncology, Philadelphia, PA, USA
2 Fox Chase Cancer Center, Temple University, Philadelphia, PA, USA
3 The University of Pennsylvania, Division of Cardiovascular Medicine, Philadelphia, PA, USA
4 The University of Pennsylvania, Division of Translational Medicine and Human Genetics, Philadelphia, PA, USA
5 The University of Pennsylvania, Division of Gastroenterology, Philadelphia, PA, USA
6 The University of Pennsylvania, Department of Medical Ethics and Health Policy, Philadelphia, PA, USA
∗ Corresponding author angela.bradbury@pennmedicine.upenn.edu
7 Lead contact

24 8 2024
10 10 2024
24 8 2024
5 4 10034624 1 2024
20 8 2024
© 2024 The Author(s)
2024
https://creativecommons.org/licenses/by-nc-nd/4.0/ This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).
Summary

Research participants report interest in receiving genetic research results. How best to return results remains unclear. In this randomized pilot study, we sought to assess the feasibility of returning actionable research results through a two-step process including a patient-centered digital intervention as compared with a genetic counselor (GC) in the Penn Medicine biobank. In Step 1, participants with an actionable result and procedural controls (no actionable result) were invited to digital pre-disclosure education and provided options for opting out of results. In Step 2, those with actionable results who had not opted out were randomized to receive results via a digital disclosure intervention or with a GC. Five participants (2%) opted out of results after Step 1. After both steps, 52 of 113 (46.0%) eligible cases received results, 5 (4.4%) actively declined results, 34 (30.1%) passively declined, and 22 (19.5%) could not be reached. Receiving results was associated with younger age (p < 0.001), completing pre-disclosure education (p < 0.001), and being in the GC arm (p = 0.06). Being older, female, and of Black race were associated with being unable to reach. Older age and Black race were associated with passively declining. Forty-seven percent of those who received results did not have personal or family history to suggest the mutation, and 55.1% completed clinical confirmation testing. The use of digital tools may be acceptable to participants and could reduce costs of returning results. Low uptake, disparities in uptake, and barriers to confirmation testing will be important to address to realize the benefit of returning actionable research results.

Research participants report interest in receiving genetic research results, but how best to return results remains unclear. In this randomized pilot study, almost half of participants received results. Disparities in uptake and barriers to confirmation testing will be important to address to realize the benefit of returning actionable research results.

Keywords

Genetic Research Results
Biobank Repository
Actionable Genetic Findings
Participant Uptake
Digital Health Interventions
Return of Research Results
Genomic Data Utilization
Genetic Counseling
Precision Medicine
Patient Engagement in Research
==== Body
pmcIntroduction

Whole exome sequencing data from research participants’ DNA in biobanks raises questions about the obligation to share individual research results.1,2,3,4,5,6,7 Based on principles of autonomy and respect for persons,8,9,10,11,12,13,14,15,16 there is consensus that participants should be informed about the return of genetic research results, including the type of results, the process, and the option to decline.2,4,17,18,19,20,21,22,23,24 However, standard best practices for implementation of these recommendations in large biobank and cohort studies are unknown. Most programs send participants a letter outlining a plan to return results and the option to “opt out,”25,26,27 which may not suffice for well-informed decision making, particularly when details at consent were limited and the American College of Medical Genetics and Genomics (ACMG) actionable results list covers a range of cancers and cardiovascular and other medical conditions.25,28,29,30,31 Pre-test counseling by genetic counselors (GCs) is a standard that could be applied to ensure informed decision making. However, this approach is costly and impractical for research programs, underscoring the need for alternative cost-effective strategies.32

Similarly, how best to communicate genetic research results to research participants remains unclear. Unlike patients with clinical genetic testing, research participants may not be prepared for actionable results, particularly if the results are unrelated to the reason they enrolled in research, or if they enrolled as healthy unselected participants (e.g., large biobanks). While the communication through GCs may be ideal to ensure understanding, minimize psychosocial distress, and optimize medical follow-up and cascade testing,25 GC workforce shortages and costs are barriers.25,33 Many programs have used GCs to share genetic research results,4,25,27,34,35,36 while others have provided results to local health care providers37,38 who may lack the necessary expertise or resources, resulting in suboptimal patient outcomes.39 Interactive patient-centered digital interventions and/or videos developed by genetic providers may provide a cost-effective alternative.33,40,41,42 Further, these interventions could be paired with genetic counseling for participants with remaining questions, while still reducing overall provider time. Potential disadvantages include poor understanding, increased test-related short-term and/or long-term affective responses or suboptimal post-test behaviors (e.g., medical management and cascade testing). Limited data exist on patient outcomes, clinical outcomes, and costs associated with various approaches. Empirical data are needed to inform best approaches for pre-disclosure education and returning results to realize benefits for participants.

Although research participants have reported high interest in receiving research results,9,37,43,44,45,46,47,48,49,50 uptake has been lower when results have been offered (48%–86%), particularly in biobank studies where return of results was not emphasized during enrollment and uptake has generally been around 50%.26,28,51,52,53,54,55,56,57 There are limited patient-reported outcomes in research settings,51,52,53,58 although some genomic implementation studies report no psychological harms with return of genetic findings.59,60 In the RESPECT2 study, whereby women with a personal or family history of breast cancer contributed a biospecimen for research, women could choose pre-disclosure education with a GC or a patient-centered digital intervention. All participants received results with a GC. Favorable cognitive (e.g., knowledge) and affective (e.g., distress and uncertainty) responses were found, although the number of actionable results was relatively small in both studies.61,62

This randomized pilot study assessed the feasibility of offering digital pre-disclosure education and returning actionable research results through a patient-centered multi-modality eHealth (e.g., digital) intervention compared with a GC in a biobank. In this real-word biobank, return of results was not a primary focus, unlike genomic implementation studies or precision health projects where participants expect results from their genetic sequencing. Secondary aims included evaluating factors associated with uptake of results and confirmation testing, as well as early data on patient-reported outcomes and potential benefits, risks, and limitations of returning actionable genetic research results.

Subjects, material, and methods

Participants were English-speaking adults who enrolled in the Penn Medicine Biobank (PMBB), which links genomic data to electronic health record (EHR) phenotype data at the University of Pennsylvania.63 We reviewed whole exome sequencing (WES) for actionable variants (according to ACMG SF v.3.0).64 Variants included those with consensus of pathogenicity or likely pathogenic in ClinVar and protein truncating variants (PTVs) (nonsense, frameshift), excluding the last exon and the last 50 base pairs (bp) of the penultimate exon, in genes where PTVs are a disease mechanism. We excluded participants who were deceased or had clinical testing identifying the actionable variant. From the remaining participants with actionable variants, we selected 130 “cases” and matched them on age, sex, and race/ethnicity to 130 “procedural controls” with no actionable variants. Procedural controls allowed participants to learn about the return of results program and opt out before notification of an actionable result. This step was felt to be particularly important for this study as the return of results was not a primary focus at the time of consent to the PMBB, and the types of results being returned and procedures for return were not included at the time of original consent. Institutional review board approval was obtained for return of result procedures and collection of survey data.

Digital pre-disclosure education and return of results intervention

Informed by prior research, we developed a mobile-ready, patient-centered multi-modality eHealth (e.g., digital) pre-disclosure education (digital education) and return of results intervention (digital-ROR) as alternatives to genetic counseling to reduce participant burdens and steps to receiving actionable genetic research results. The study team adapted the digital education tool for this study from the stakeholder-informed RESPECT multi-modality pre-disclosure web education (e.g., digital) intervention and modified it based on user and usability testing with biobank participants.62 Digital-ROR was specifically developed for this study. Both digital interventions cover the same content as genetic counseling and were developed with patient feedback to meet the needs of a diverse clinical population. Informed by the tiered-binned model, the interventions include “indispensable” Tier 1 information presented to all participants, and “optional” Tier 2 information for varying needs.65 The digital-education pre-disclosure intervention consisted of six modules and nine optional videos (Table 1). The digital-ROR intervention had four modules and three optional videos (Table 2).Table 1 Digital pre-disclosure education intervention

Module	Tier 1 Content (# screens)a	Tier 2 Content (# screens)	Tier 2 Videosb[minutes]	
WELCOME/LANDING PAGE	NA	NA	NA	
1: Introduction	• What is the Penn Biobank Return of Results Program? (2)

• What to expect (overview of content and option to opt out) (1)

• Option to speak with a person (1)

			
2: Genetics Overview	• Genes and inheritance (7)

• Genes and health risks (1)

		• Basic genetics overview [1:19]

	
3: Types of results and health implications	• What types of genes and conditions could be found (1)

• How can genes impact health? (1)

• How might learning my research results help me? (1)

	• More information on cancer genes (1)

• More information on cardiovascular genes (1)

	• How cancer genes affect health [1:46]

• How cardiovascular genes affect health [1:08]

	
4: Research vs. clinical testing	• Need for confirmation testing (1)

• Research vs. clinical testing (1)

• Medical follow-up (1)

• How costs of confirmation testing will be covered (1)

	• More information on medical care options for cancer genes (1)

• More information on medical care options for cardiovascular genes (1)

	• Research vs. clinical testing [1:52]

• Medical care for cancer genes [0:51]

• Medical care for cardiovascular genes [0:56]

	
5: Risks, benefits and limitations of receiving results	• Potential benefits of receiving results (1)

• Potential limitations of receiving results (1)

• Potential risks of receiving results (1)

		• Benefits [0:16], limitations [0:46] and risks [0:36] of receiving results

	
6: Next steps	• Overview: What’s next? (1)

• What happens after I receive my results? (2)

• HIPAA/privacy (3)

• How would you like to proceed?c (1)

• Survey request (1)

			
Optional content		Glossary of terms		
The intervention is informed by the tiered-binned model for genetic education and informed consent and was reviewed with cancer genetics providers and experts in health disparities and health communication. Readability testing was completed for all content and modifications made to achieve a readability score of ninth grade or lower. The linear digital intervention includes six modules and optional videos. Participants can view modules for as long, and as many times as desired. They can also go back to previously viewed topics. Content was user tested with 17 participants (82.4% female and 35.3% non-White race).

Completing Tier 1 content takes 8–14 min; completing Tier 1 and 2 content (excluding videos) takes 11–17 min.

a All pages offer “Ask for Help” option.

b Videos include a genetic counselor explaining specific topics. Content in some videos is intentionally redundant to Tier 1 and Tier 2 content and designed to provide an alternative method for reviewing content for participants with different learning preferences.

c Participants can choose to (1) opt out of receiving results, (2) speak with a staff member/GC to make a decision, (3) acknowledge they have enough information and understand they will be contacted if something is found in their sample.

Table 2 Return of results digital intervention

Module	Tier 1 Content (# screens)a	Tier 2 Content (# screens)	
1: Welcome page/Instructions	• Overview and option to speak with a genetic counselor (GC) (1)

• How to use this website (1)

• Option to review educationb (2)

• Option to review specific education items (1)

	• Basic geneticsc

• How genes impact healthc

• Research vs. clinical testingc

• Genetic mutations and medical carec

• Benefits [0:16], limitations [0:46] and risks [0:36]c

	
2: Test result	• Ready to receive your results? (option for more time or to speak with a GC) (1)

• Genetic test result (1)

	• Copy of test report

	
3: Explanation of result	• Implications if result is confirmed (2–3 screens based on gened)

• What does this mean for my relatives? (1)

	• Risks and guidelines

	
4: Next steps	• Need for confirmation testing and medical follow-up if confirmed (1)

• FAQ for confirmation testing (1)

• Are you ready for confirmation testing? (yes, send me a kit or no, I would like to speak with a person)

	• PDF of clinical test result (confirmation testing) available for download once result is available

	
Completing Tier 1 content takes 1–5 min; completing Tier 1 and 2 content (excluding videos) takes 5–12 min.

a All pages offer “Ask for Help” option.

b If participants have not previously viewed the digital pre-disclosure education intervention they have to actively decline education on this screen. If they decline, they are provided a single screen with key points (minimum information).

c Refers to duplicate screens from education website.

d Option to select Risks and Guidelines for females vs. males for some genes.

Notification of the option to decline ROR and pre-disclosure education

The Opt-out Notification Letter (Step 1 letter) was sent to selected cases and procedural controls (Figure S1), providing access to pre-disclosure digital education through an individual user ID and passcode. Letter 1 was also accompanied by a study information sheet and an FAQ. After viewing letter 1 or the digital education, all participants were asked to complete a baseline survey (T0) and a consent statement for survey responses.

Randomization

After at least 4 weeks, participants with actionable results (e.g., cases) and had not opted out received a second letter (Step 2 letter) informing them of an important health finding and next steps for receiving their result (Figure S2). These participants were randomized to receive results by remote phone/video conference with a GC (Arm 1) or the private ROR digital intervention with the option to speak with a GC (Arm 2). Randomization via a permuted block design stratified by cancer vs. cardiovascular genes and gender. Procedural controls did not receive the Step 2 letter, but any who opted out of results had their decision recorded if they were later found to have an actionable result with additional sequencing or analysis. Participants who did not respond or log onto digital-ROR after being sent the Step 2 letter were contacted by phone, and if unreachable after three calls, a summary letter was sent, which included ways for them to contact the team if their interest changes.

Disclosure of genetic research results

Participants who had not completed pre-disclosure education were asked to do so or actively decline before GC disclosure sessions or proceeding to results on digital-ROR. GCs utilized counseling checklists for all counseling sessions. Digital-ROR participants had the option to speak with a GC to receive their results or any time before or after result disclosure.

Confirmation testing in a Clinical Laboratory Improvement Amendments-certified lab and clinical follow-up

Participants receiving actionable research results were recommended for confirmatory testing via mailed saliva kits or clinical visit phlebotomy. All testing was covered by research funds to eliminate real or perceived cost barriers. Results were provided per the original randomization (i.e., remote phone/video conference or digital-ROR intervention), followed by a referral to clinical genetics programs.

Patient-reported outcomes

Quantitative instruments assessed psychosocial and behavioral outcomes at baseline (T0) and 2–7 days post-disclosure (T1).

Knowledge of genetic disease (T0-T2) was evaluated using an adapted version of the Clin Seq. knowledge scale utilized in the RESPECT study and adapted to include cardiovascular conditions (Cronbach’s α = 0.63).61,66

Psychological adjustment included assessments of general anxiety (T0-T2) and depression (T0, T2) using the NIH PROMIS measures (Cronbach’s α = 0.88 and 0.86).

Disease-specific distress (T0-T2) was measured using the Revised Impact of Events Scale (Cronbach’s α = 0.88).67

Multidimensional responses to genetic testing, including negative and positive responses and uncertainty, were assessed using the Multidimensional Impact of Cancer Risk Assessment Questionnaire (Cronbach’s α = 0.85 for negative responses, 0.77 for uncertainty, 0.81 for positive responses).68

Decisional regret (T1, T2) was assessed using the validated Decision Regret Scale (Cronbach’s α = 0.86).69,70

Sociodemographic data, health literacy,71 and comfort with technology (selected items from the Health Information National Trends Survey, HINTS)72 were assessed at T0.

Statistical considerations and analyses

Feasibility of a future larger trial was assessed by evaluating completion rates of study steps, including (1) uptake of pre-disclosure education, (2) opting out, (3) receipt of results in each arm, (4) confirmation testing, (5) clinical follow-up, and (6) survey completion. With 50 participants per arm, we would have 93.9% power to detect a discouraging completion rate of 55% (null hypothesis) for any of these outcomes vs. an encouraging completion rate of 80% (alternative hypothesis). The hypotheses were based on previous findings.73 The power calculation assumes 1% type I error (one-sided) and the use of a binomial exact test. With 50 participants, we would hence reject the discouraging null hypothesis if 36 or more people complete tasks. We set the type I error rate to 1% to control for multiple hypothesis testing (two completion tasks examined independently within two randomization arms). The primary analysis used an intention-to-treat paradigm.

We used descriptive statistics to characterize the data and tested unadjusted associations using t-tests and Fisher’s exact tests. Multivariable associations were tested using full-model logistic regressions of use of the digital interventions or GC, completing pre-disclosure education, and receiving results. Linear regressions compared patient-reported outcomes (PROs). Missing data in PROs were accounted for by using the multiple imputation by chained equations methods of Raghunathan and colleagues.74 Statistical significance was set at p < 0.05. STATA (College Station, TX) and SAS (Cary, NC) were used for analyses.

Results

Participant characteristics

A total of 499 PMBB participants with actionable ACMG variants (330 cancer cases and 169 cardiovascular cases) were identified with research WES data (Figure 1). Exclusions included 105 participants (21.0%) who had clinical testing for the actionable variant(17.2% cancer and 3.8% cardiovascular excluded), 95 (19.0%) recorded deceased individuals (9.2% cancer, 9.8% cardiovascular) according to the medical record or other public sources, and 21 (4.2%) were deceased with prior clinical testing. One individual was excluded due to cognitive impairment. Of the remaining 277participants with actionable results, 130 participants with actionable variants (“cases”) and 130 matched “procedural controls” without actionable variants were selected. Participants were on average 62.5 years old (SD 14.3), 58.5% male, 37.7% Black, and 3.5% Hispanic (Table 3).Figure 1 Study CONSORT

# Cases include BRCA2 (43), BRCA1 (13), TTR (24), LDLR (12), KCNQ1 (6), TNNI (5), PMS2 (5), DSP (4), MYH7 (4), FBN (3), MYBPC (3), LMNA (2), SDHC (2), MSH6 (1), PKP2 (1), RET (1), and SCN (1). ˆ Two web disclosures were completed by GC at participant request (8.7%). ∗14 (GC arm) and 10 (digital arm) were eligible for clinical follow-up. Four were excluded as they either did not have results confirmed (n = 3) or already had a clinical diagnosis and was connected with the clinical team (n = 1). In the GC arm, 3 declined clinic follow-up and 1 was not local. In the digital arm, 2 declined clinic follow-up and 2 were not local.

Table 3 Participant characteristics (N = 260)

	Cases	Procedural controls	Total	
n = 130	n = 130	N = 260	
Age	
	
Mean (SD)	62.52 (14.36)	62.54 (14.29)	62.53 (14.30)	
Min, Max	22.0, 90.0	25.0, 91.0	22.0, 91.0	
n (% missing)	130 (0.0)	130 (0.0)	260 (0.0)	
	
Sex	
	
F	54 (41.54)	54 (41.54)	108 (41.54)	
M	76 (58.46)	76 (58.46)	152 (58.46)	
	
Race	
	
White	78 (60.00)	76 (58.46)	154 (59.23)	
Black	48 (36.92)	50 (38.46)	98 (37.69)	
Asian	3 (2.31)	3 (2.31)	6 (2.31)	
Other	1 (0.77)	1 (0.77)	2 (0.77)	
	
Ethnicity	
	
NHL	126 (96.92)	125 (96.15)	251 (96.54)	
HL	4 (3.08)	5 (3.85)	9 (3.46)	
	
Marital status	
	
Married	72 (55.38)	79 (60.77)	151 (58.08)	
Divorced	18 (13.85)	8 (6.15)	26 (10.00)	
Separated	1 (0.77)	1 (0.77)	2 (0.77)	
Single	35 (26.92)	36 (27.69)	71 (27.31)	
Widowed	4 (3.08)	6 (4.62)	10 (3.85)	
	
First attempt contact method	
	
EMAIL	14 (10.77)	14 (10.77)	28 (10.77)	
MPM	83 (63.85)	84 (64.62)	167 (64.23)	
USPS	33 (25.38)	32 (24.62)	65 (25.00)	
	
Final contact method used	
	
EMAIL	54 (41.54)	53 (40.77)	107 (41.15)	
MPM	37 (28.46)	40 (30.77)	77 (29.62)	
Returned mail	1 (0.77)	3 (2.31)	4 (1.54)	
USPS	38 (29.23)	34 (26.15)	72 (27.69)	
HL, Hispanic or Latino; NHL, Non-Hispanic or Latino.

Values are n (%) unless otherwise specified. Cases include BRCA2 (43), BRCA1 (13), TTR (24), LDLR (12), KCNQ (6), TNNI (5), PMS2 (5), DSP (4), MYH (4), DSP (4), FBN (3), MYBPC (3), LMNA (2), SDHC (2), MSH6 (1), PKP (1), RET (1), SCN (1). There are no significant differences between cases and controls.

Step 1: Pre-disclosure education and option to opt out of return of results

A total of 260 participants (130 case-control pairs) received the Step 1 letter (notification of the return of results program and option to opt out) (Figure 1). Among those without returned mail, 39 of 255 (15.3%) logged onto the eHealth pre-disclosure education intervention (digital education) and four (1.6%) asked to speak with a GC. Receipt of the Step 1 letter via MyPennMedicine (MPM), Penn Medicine's EHR tool, was a predictor of logging onto digital education (MPM OR 5.0 vs. US mail, p = 0.011), while Black race was associated with not logging in (OR 2.9 for not logging in vs. White, p = 0.033). Five participants (2%), including three cases, opted out after the Step 1 letter, having viewed digital education and provided their decision via the site.

Step 2: Return of actionable results by randomized arms

The remaining 125 cases with actionable variants were randomized to GC or digital-ROR and contacted via Step 2 letter to receive their research results (Figure 1). There were no significant differences in participant characteristics by randomized arm (Table S1). Among eligible randomized participants (e.g., believed to be living and without returned mail, 52 GC arm, 61 digital-ROR arm), 19 of 133 (16.8%) had completed pre-disclosure digital education before receiving the Step 2 letter. Overall, 26 of 52 (50.0%) in the GC-ROR arm and 21 of 61 (34.4%) in the digital-ROR arm (p = 0.18) completed pre-disclosure education (41.6% overall, Table 4). Sixteen cases (34.0% of the 47 who completed education) requested to speak with a GC for pre-disclosure education, five instead of completing digital education and 11 in addition to digital education. Completing pre-disclosure digital education was associated with younger age (OR 1.06/year younger, p = 0.005), whereas Black participants were less likely to speak with a GC for pre-disclosure education (OR 0.15, p = 0.033).Table 4 Primary feasibility outcomes and uptake of results among those who received notification of an actionable result (Step 2 letter)

Outcome	GC-ROR arm (n = 52)	Digital-ROR arm (n = 61)	
Uptake of pre-disclosure educationa	26 (50.0)	21 (34.4)	
Frequency of opting out (active decliners)	2 (3.8)	3 (4.9)	
Frequency of passive decliners	15 (28.8)	19 (31.1)	
Receipt of resultsa	29 (55.8)	23 (37.7)	
Uptake of confirmation testinga,b	15 (55.6)	13 (59.1)	
Clinical follow-upa,c	10 (71.4)	6 (60)	
Completion of surveys	T0: 39/52 (75)d; T1: 30/52 (57.7)d	
Values are n (%). p value between arms was >0.05 for all outcomes, except p = 0.06 for receipt of results.

a p values for all were >0.05 concluding that our rates are not significantly above hypothesized 55% for any outcome in either arm (uptake education, ROR, uptake confirmation testing and clinical follow-up).

b Denominator includes participants who received results and need confirmation testing, 27 in GC arm and 22 in digital arm.

c Denominator reflects those eligible for clinical follow-up, 14 in the GC arm, 10 in the digital arm.

d T0 completion among those who received result was 61.5% in wave 1, but 88.5% in wave 2. T1 among those who received results was 46.2% in wave 1, but 69.2% in wave 2.

Among the 113 eligible participants believed to have received the Step 2 letter, 52 (46.0%) received their research results (Figure 1); this rate was higher in the GC arm (55.8% vs. 37.7% in digital arm, Table 4). After Step 2 letters, 14 of 52 (26.9% of eligible in the GC arm) participants scheduled their disclosure session and 14 of 61 (22.9% of eligible in the digital-ROR arm) accessed the digital-ROR intervention or requested a GC appointment (n = 1). Most participants did not respond to the letter and required a follow-up call. Of these, 22 (11.5% in GC arm, 26.2% in digital-ROR arm) could not be reached. Among those we could reach, 25 (39.6%) ultimately received their research results. Nine percent (2 of 23) in the digital-ROR arm requested to speak with a GC to receive their results.

Five of 113 (4.4%) eligible participants actively declined to receive results. Thirty-four (30.1%) were passive decliners (spoke with research staff, and did not decline but did not proceed), which did not differ between arms (p = 0.84).

Participant factors associated with receipt of results

Receiving results was associated with younger age (56.1 vs. 65.9 years old, p < 0.001), completing pre-disclosure education (77% v. 10%, p < 0.001) and randomization to the GC arm (p = 0.06). In a multinomial logistic regression of the status of receiving results, with receiving results as the reference outcome, being older age (OR = 1.13 per year, p = 0.003), female (OR = 7.57, p = 0.012), Black (OR = 8.48, p = 0.018), and married (OR = 9.53 for married relative to divorced, separated, or widowed, p = 0.031) were associated with being unable to reach (e.g., never made contact). Older age (OR = 1.09 per year, p = 0.002), Black race (OR = 6.5, p = 0.005) and being married (OR 7.7, p = 0.034) were associated with passive decline.

Clinical confirmation testing and clinical follow-up

Twenty-eight of 49 participants (57.1%) completed clinical confirmation testing, with no significant differerence between arms (p = 1.00, Table 4; Figure 2). Three participants already had clinical testing (n = 2) or diagnosis (n = 1). Five participants of 49 (10.2%) actively declined confirmation testing (due to lack of interest or concern) and 15 of 49 (30.6%) passively declined (agreed to confirmation but did not return testing kit). Twenty-five of 28 (89.2%) clinical confirmation results were concordant with research results. Discordant results included two variants in KCNQ1, which had lower-quality sequencing, and one in PKP2.Figure 2 Outcomes among 113 participants contacted with actionable research results∗

∗Excluding those with returned mail or we learned were deceased.

Of 24 with confirmed results and recommended for follow-up, 16 participants (66.7%) attended the recommended clinical follow-up visit, with no difference between arms (p = 0.67, Table 4; Figure 2). Three had moved away from the Penn Medicine service area, and five passively or actively declined. Those not local were recommended to contact their usual source of care provider, and to call us with any barriers or questions.

Excluding those not confirmed, 14 of 52 (28.6%) met clinical criteria for testing before return of results. Twelve (24.5%) had a suggestive personal or family history but details in the EHR were insufficient to know if they would have met criteria. Twenty-three participants (46.9%) had no documented history suggesting testing candidacy.

Patient-reported outcomes

The comparison of PROs among randomized participants were considered exploratory given small numbers. As shown in Table 5, there were no significant differences in affective or cognitive outcomes between arms, although participants in the digital-ROR arm had a greater decline in knowledge as compared with participants in the GC-ROR arm.Table 5 Patient-reported outcomes by randomized arm

Outcome (scale range)	GC-ROR mean (SD)
n = 25	Digital-ROR mean (SD)
n= 17	Total mean (SD)
n = 42	p value	
Depression T0 (range 4–20)	6.29 (2.98)	6.77 (2.93)	6.48 (2.94)	0.64	
Depression T1-T0	−0.15 (3.67)	−0.03 (3.96)	−0.1 (3.76)	0.94	
Anxiety T0 (range 4–20)	6.92 (2.89)	7.65 (4.09)	7.23 (3.40)	0.55	
Anxiety T1-T0	−0.02 (3.61)	0.36 (5.54)	0.13 (4.47)	0.84	
Disease-specific distress T0 (range 0–40)	7.58 (9.44)	6.02 (8.59)	6.95 (9.04)	0.62	
Disease-specific distress T1-T0	−1.31 (9.12)	1.11 (11.94)	−0.33 (10.33)	0.54	
Knowledge T0 (range 0–16)	10.17 (3.51)	10.17 (3.99)	10.17 (3.67)	1.00	
Knowledge T1-T0	0.98 (5.06)	−2.98 (5.64)	−0.63 (5.62)	0.06	
MICRA negative response T1 (range 0–30)	7.38 (8.73)	8.64 (9.35)	7.89 (8.97)	0.74	
MICRA uncertainty T1 (range 0–20)	12.05 (11.38)	14.54 (12.48)	13.05 (11.82)	0.59	
MICRA positive response T1 (range 0–45)	10.03 (7.20)	11.04 (8.19)	10.44 (7.60)	0.75	
MICRA total T1 (range 0–95)	29.45 (19.55)	34.32 (19.44)	31.38 (19.57)	0.56	
Decisional regret T1 (range 5–25)	9.45 (5.52)	10.13 (5.69)	9.72 (5.58)	0.78	
Higher scores indicate greater depression, general anxiety, disease-specific distress, knowledge, negative and positive responses to genetic information, uncertainty with genetic information and decisional regret.

MICRA, Multidimensional Impact of Cancer Risk Assessment.

Discussion

This study evaluated the feasibility of a two-step model and a patient-centered digital alternative for the return of actionable genetic research results (ROR) within a real-world biobank of medical patients where ROR was not a primary focus (e.g., in contrast to genomic implementation studies or precision health projects where participants enroll expecting results). Although designed as a feasibility study for a future larger randomized trial, several key observations were identified. Despite lower uptake of results than our predefined metric for success (55%), 46% of actionable results were successfully returned, and 47% of participants who received results would not have been candidates for clinical testing according to personal and family history documented in the EHR. Additionally, digital pre-disclosure education and return of results was acceptable for many participants, although some still preferred speaking with a GC. Notably, some participants actively or passively declined receiving their results, and 42% actively or passively declined confirmation testing.

While uptake of actionable results was lower than expected, almost half of participants received actionable results that could impact their health. Importantly, almost half would not otherwise have been identified as genetic carriers based on EHR data, which is consistent with two other programs.51,75 For these individual patients and their families, the return of actionable results has the potential to significantly improve their health. Thus, low uptake rates should not diminish the potential value of receiving results for those who elected to receive their results. Uptake of results in this study was lower than published hypothetical interest76,77 or in genomic implementation studies75 (where patients are expecting return of results), but it is similar to levels reported in our prior studies in women from breast cancer families62,78 and two other large-scale biobanks.51,79 These data support continued investigation of barriers to returning results and optimal procedures to support returning results in biobanks. Given the association between uptake of pre-disclosure and receiving results, we are considering strategies to increase uptake of education in our subsequent study. Across all settings, longitudinal follow-up and assessment of screening and risk-reducing behaviors and communication to at-risk relatives will help confirm the medical benefits of returning actionable research results.

Importantly, our two-step process allowed research participants to consider if they wanted actionable results before being notified of an identified actionable result. The necessity of providing individuals the “right of refusal” has been debated for medically actionable results,80 and is only important if there are individuals who demonstrate an informed choice to not receive results. Even in this small sample, only 5% of research participants actively declined results, both before and after knowing that they have a result. These data suggest that the right of refusal should be maintained in biobanks where return of results is not anticipated at enrollment in research. Further, 30% of participants passively declined results, some of whom may be making an informed choice to refuse results. Passive decliners have been reported by others,51 warranting further understanding of this group and whether these individuals are making informed choices or whether there are barriers contributing to inequity in return of results.

One challenge with returning genetic results is the cost in human effort. From verifying sequencing results, reviewing the EHR, generating and sending letters for follow-up, to returning of results with a provider and the follow-up calls to ensure clinical confirmation testing and referrals, many human resources are required.51 Digital tools, if successful, could not only reduce costs for research programs, but also reduce participant burden.42,81 Consistent with the RESPECT2 study, many participants chose digital pre-disclosure education.62 In this pilot study, many participants accepted the patient-centered digital intervention for receiving genetic research results, suggesting a viable lower-cost alternative. Of note, some participants still requested to speak with a genetic provider either for pre-disclosure education (up to 34%) or for receipt of results (8.7%), suggesting a value to maintaining this option and a critical consideration for the field. Although lower uptake of results in participants randomized to digital-ROR was observed, a larger number of individuals were unable to be reached in this arm. Establishing that offering digital results is no worse than provider-mediated disclosure will be critical to understanding the overall risks and benefits of digital alternatives.

Similar to our prior studies and other published reports, many participants were unreachable or were passive decliners. Barriers to contact and uptake of results may include socioeconomic determinants such as digital inequity, lower health literacy, and individual characteristics such as prioritization of current health issues, disability status, time since sample collection, and cultural values.82 Similar to findings from the RESPECT studies, we found that older individuals and Black participants were more likely to be unreachable or to passively decline.62,78 These associations suggest the possibility of exacerbating existing health disparities, which will be critical to consider in future research.

Despite extensive education regarding the importance of confirmation testing, the distinction between research and clinical testing, multiple reminders, and extensive support and coverage for testing costs, only 55% of eligible participants completed confirmation testing. Higher adherence in the RESPECT studies may be related to patient characteristics and contextual factors.61,62 Factors such as the COVID-19 pandemic, patients’ comorbidities, and their more advanced age may have contributed to lower adherence rates in this study. Understanding barriers to confirmation testing is critical for realizing the benefits of returning actionable research results and warrants further inquiry.

We found no significant increases in distress or declines in knowledge in PROs between randomized arms. However, these were secondary analyses with small sample sizes. Other recent studies suggest similar outcomes for digital and provider-mediated disclosure of genetic test results, although they involve clinical populations seeking genetic testing, including lower-risk carrier testing results or include negative results.83,84 Outcomes may differ in a medical population not actively seeking genetic testing and having vital status or confirmation of receipt of study letters among those we were not able to reach could be valuable. Data from other biobanks suggest that some patients may experience some distress with return of results, although more precise estimates, longitudinal follow-up, and rigorous PROs are needed.79,85 Our ongoing randomized trial is powered to assess non-inferiority for uptake of results and PROs by randomized arm for actionable results with significant clinical implications and risks.

In conclusion, patient-centered digital tools may be acceptable to participants and warrant further study to reduce costs and burdens of returning actionable research results. Low uptake, disparities in contact and uptake by race/ethnicity, and barriers to confirmation testing need evaluation in future studies to realize the benefits of returning actionable results.

Data and code availability

The datasets used and/or analyzed during this study are available from the corresponding author on reasonable request.

Acknowledgments

Primary financial support for this work was provided by NIH R01 CA190871 (Bradbury). Support from the Basser Center for BRCA was provided to A.R.B., S.D., and K.L.N. 10.13039/100001006 The Breast Cancer Research Foundation provided support for S.D. and K.L.N.

Declaration of interests

A.R.B. receives partial research funding from AstraZeneca and Merck for other studies.

Supplemental information

Document S1. Figures S1, S2, and Table S1

Document S2. Article plus supplemental information

Supplemental information can be found online at https://doi.org/10.1016/j.xhgg.2024.100346.
==== Refs
References

1 Evans J.P. Rothschild B.B. Return of results: not that complicated? Genetics in medicine Genet. Med. 14 2012 358 360 10.1038/gim.2012.8 22481183
2 Wolf S.M. Return of individual research results and incidental findings: facing the challenges of translational science Annu. Rev. Genomics Hum. Genet. 14 2013 557 577 10.1146/annurev-genom-091212-153506 23875796
3 Bledsoe M.J. Clayton E.W. McGuire A.L. Grizzle W.E. O'Rourke P.P. Zeps N. Return of research results from genomic biobanks: a call for data Genet. Med. 15 2013 159 160 10.1038/gim.2012.163 23386185
4 Jarvik G.P. Amendola L.M. Berg J.S. Brothers K. Clayton E.W. Chung W. Evans B.J. Evans J.P. Fullerton S.M. Gallego C.J. Return of genomic results to research participants: the floor, the ceiling, and the choices in between Am. J. Hum. Genet. 94 2014 818 826 10.1016/j.ajhg.2014.04.009 24814192
5 Knoppers B.M. Zawati M.H. Sénécal K. Return of genetic testing results in the era of whole-genome sequencing Nat. Rev. Genet. 16 2015 553 559 10.1038/nrg3960 26239711
6 Bledsoe M.J. Grizzle W.E. Clark B.J. Zeps N. Practical implementation issues and challenges for biobanks in the return of individual research results. Genetics in medicine Genet. Med. 14 2012 478 483 10.1038/gim.2011.67 22323073
7 West K.M. Blacksher E. Cavanaugh K.L. Fullerton S.M. Umeukeje E.M. Young B.A. Burke W. At the Research-Clinical Interface: Returning Individual Genetic Results to Research Participants Clin. J. Am. Soc. Nephrol. 15 2020 1181 1189 10.2215/CJN.09670819 32041801
8 Manolio T.A. Taking our obligations to research participants seriously: disclosing individual results of genetic research Am. J. Bioeth. 6 2006 32 W12 10.1080/15265160600935993 author reply W10-W12
9 Fernandez C.V. Santor D. Weijer C. Strahlendorf C. Moghrabi A. Pentz R. Gao J. Kodish E. The return of research results to participants: pilot questionnaire of adolescents and parents of children with cancer Pediatr. Blood Cancer 48 2007 441 446 10.1002/pbc.20766 16425279
10 Fernandez C.V. Weijer C. Obligations in offering to disclose genetic research results Am. J. Bioeth. 6 2006 44 W12 10.1080/15265160600938575 author reply W10-W12
11 Ravitsky V. Wilfond B.S. Disclosing individual genetic results to research participants Am. J. Bioeth. 6 2006 8 17 10.1080/15265160600934772
12 Sharp R.R. Foster M.W. Clinical utility and full disclosure of genetic results to research participants Am. J. Bioeth. 6 2006 42 W12 10.1080/15265160600938443 author reply W10-W12 17085408
13 Beskow L.M. Burke W. Offering individual genetic research results: context matters. Research Support, N.I.H., Extramural Sci. Transl. Med. 2 2010 38cm20 10.1126/scitranslmed.3000952
14 Richardson H.S. Belsky L. The ancillary-care responsibilities of medical researchers. An ethical framework for thinking about the clinical care that researchers owe their subjects Hastings Cent. Rep. 34 2004 25 33 15098404
15 Wolf S.M. Annas G.J. Elias S. Point-counterpoint. Patient autonomy and incidental findings in clinical genomics Science 340 2013 1049 1050 10.1126/science.1239119 23686341
16 Townsend A. Adam S. Birch P.H. Friedman J.M. Paternalism and the ACMG recommendations on genomic incidental findings: patients seen but not heard. Comment Letter. Genetics in medicine Genet. Med. 15 2013 751 752 10.1038/gim.2013.105 24008256
17 Burke W. Antommaria A.H.M. Bennett R. Botkin J. Clayton E.W. Henderson G.E. Holm I.A. Jarvik G.P. Khoury M.J. Knoppers B.M. Recommendations for returning genomic incidental findings? We need to talk! Research Support, N.I.H., Extramural Research Support, Non-U.S. Gov't Review Genet. Med. 15 2013 854 859 10.1038/gim.2013.113 23907645
18 Anticipate and Communicate Ethical Management of Incidental and Secondary Findings in the Clinical, Research and Direct-to-Consumer Contexts. Report of the Presidential Commission for the Study of Bioethical Issues 2013 https://www.genome.gov/Pages/PolicyEthics/HealthIssues/Anticipate_Communicate.pdf
19 Griffin C.A. Axilbund J.E. Codori A.M. Deise G. May B. Pendergrass C. Tillery M. Trimbath J.D. Giardiello F.M. Patient preferences regarding recontact by cancer genetics clinicians Fam. Cancer 6 2007 265 273 10.1007/s10689-007-9117-0 17308889
20 Clift K.E. Halverson C.M.E. Fiksdal A.S. Kumbamu A. Sharp R.R. McCormick J.B. Patients' views on incidental findings from clinical exome sequencing Appl. Transl. Genom. 4 2015 38 43 10.1016/j.atg.2015.02.005 26937348
21 Shahmirzadi L. Chao E.C. Palmaer E. Parra M.C. Tang S. Gonzalez K.D.F. Patient decisions for disclosure of secondary findings among the first 200 individuals undergoing clinical diagnostic exome sequencing Genet. Med. 16 2014 395 399 10.1038/gim.2013.153 24113345
22 Dressler L.G. Smolek S. Ponsaran R. Markey J.M. Starks H. Gerson N. Lewis S. Press N. Juengst E. Wiesner G.L. GRRIP Consortium IRB perspectives on the return of individual results from genomic research Genet. Med. 14 2012 215 222 10.1038/gim.2011.10 22241094
23 Sciences NAo Returning Individual Research Results to Participants: Guidance for a New Research Paradigm http://nationalacademies.org/hmd/reports/2018/returning-individual-research-results-to-participants.aspx 2018
24 Fossey R. Kochan D. Winkler E. Pacyna J.E. Olson J. Thibodeau S. Connolly J.J. Harr M. Behr M.A. Prows C.A. Ethical Considerations Related to Return of Results from Genomic Medicine Projects: The eMERGE Network (Phase III) Experience J. Pers. Med. 8 2018 2 10.3390/jpm8010002 29301385
25 Forrest L.E. Young M.A. Clinically Significant Germline Mutations in Cancer-Causing Genes Identified Through Research Studies Should Be Offered to Research Participants by Genetic Counselors J. Clin. Oncol. 34 2016 898 901 10.1200/JCO.2015.60.9388 26786918
26 Keogh L.A. Southey M.C. Maskiell J. Young M.A. Gaff C.L. Kirk J. Tucker K.M. Rosenthal D. McCredie M.R.E. Giles G.G. Hopper J.L. Uptake of offer to receive genetic information about BRCA1 and BRCA2 mutations in an Australian population-based study Cancer Epidemiol. Biomarkers Prev. 13 2004 2258 2263 15598789
27 Graves K.D. Sinicrope P.S. Esplen M.J. Peterson S.K. Patten C.A. Lowery J. Sinicrope F.A. Nigon S.K. Borgen J. Gorin S.S. Communication of genetic test results to family and health-care providers following disclosure of research results. Genetics in medicine Genet. Med. 16 2014 294 301 10.1038/gim.2013.137 24091800
28 Crook A. Plunkett L. Forrest L.E. Hallowell N. Wake S. Alsop K. Gleeson M. Bowtell D. Mitchell G. Australian Ovarian Cancer Study GroupYoung M.A. Connecting patients, researchers and clinical genetics services: the experiences of participants in the Australian Ovarian Cancer Study (AOCS) Eur. J. Hum. Genet. 23 2015 152 158 10.1038/ejhg.2014.86 24824132
29 Hallowell N. Alsop K. Gleeson M. Crook A. Plunkett L. Bowtell D. Mitchell G. Australian Ovarian Cancer Study GroupYoung M.A. The responses of research participants and their next of kin to receiving feedback of genetic test results following participation in the Australian Ovarian Cancer Study Genet. Med. 15 2013 458 465 10.1038/gim.2012.154 23448721
30 McBride K.A. Hallowell N. Tattersall M.H.N. Kirk J. Ballinger M.L. Thomas D.M. Mitchell G. Young M.A. Timing and context: important considerations in the return of genetic results to research participants J. Community Genet. 7 2016 11 20 10.1007/s12687-015-0231-7 26004130
31 Hylind R. Smith M. Rasmussen-Torvik L. Aufox S. Great expectations: patient perspectives and anticipated utility of non-diagnostic genomic-sequencing results J. Community Genet. 9 2018 19 26 10.1007/s12687-017-0314-8 28656483
32 Sutton E.J. Kullo I.J. Sharp R.R. Making pretest genomic counseling optional: lessons from the RAVE study Genet. Med. 20 2018 1157 1158 10.1038/gim.2017.240 29388941
33 Roberts M.C. Wood E.M. Gaieski J.B. Bradbury A.R. Possible barriers for genetic counselors returning actionable genetic research results across state lines. Genetics in medicine Genet. Med. 19 2017 1202 1204 10.1038/gim.2017.34 28425980
34 Christensen K.D. Roberts J.S. Shalowitz D.I. Everett J.N. Kim S.Y.H. Raskin L. Gruber S.B. Disclosing individual CDKN2A research results to melanoma survivors: interest, impact, and demands on researchers Cancer Epidemiol. Biomarkers Prev. 20 2011 522 529 10.1158/1055-9965.EPI-10-1045 21307304
35 Roberts J.S. Shalowitz D.I. Christensen K.D. Everett J.N. Kim S.Y.H. Raskin L. Gruber S.B. Returning individual research results: development of a cancer genetics education and risk communication protocol J. Empir. Res. Hum. Res. Ethics. 5 2010 17 30 10.1525/jer.2010.5.3.17 20831418
36 Jouni H. Haddad R.A. Marroush T.S. Brown S.A. Kruisselbrink T.M. Austin E.E. Shameer K. Behnken E.M. Chaudhry R. Montori V.M. Kullo I.J. Shared decision-making following disclosure of coronary heart disease genetic risk: results from a randomized clinical trial J. Investig. Med. 65 2017 681 688 10.1136/jim-2016-000318
37 Shaibi G.Q. Kullo I.J. Singh D.P. Sharp R.R. De Filippis E. Cuellar I. Hernandez V. Levey S. Radecki Breitkopf C. Olson J.E. Developing a Process for Returning Medically Actionable Genomic Variants to Latino Patients in a Federally Qualified Health Center Public Health Genomics 21 2018 77 84 10.1159/000494488 30522109
38 Kullo I.J. Olson J. Fan X. Jose M. Safarova M. Radecki Breitkopf C. Winkler E. Kochan D.C. Snipes S. Pacyna J.E. The Return of Actionable Variants Empirical (RAVE) Study, a Mayo Clinic Genomic Medicine Implementation Study: Design and Initial Results Mayo Clin. Proc. 93 2018 1600 1610 10.1016/j.mayocp.2018.06.026 30392543
39 Plon S.E. Cooper H.P. Parks B. Dhar S.U. Kelly P.A. Weinberg A.D. Staggs S. Wang T. Hilsenbeck S. Genetic testing and cancer risk management recommendations by physicians for at-risk relatives Genet. Med. 13 2011 148 154 10.1097/GIM.0b013e318207f564 21224735
40 Yu J.H. Jamal S.M. Tabor H.K. Bamshad M.J. Self-guided management of exome and whole-genome sequencing results: changing the results return model Genet. Med. 15 2013 684 690 10.1038/gim.2013.35 23619276
41 Tabor H.K. Jamal S.M. Yu J.H. Crouch J.M. Shankar A.G. Dent K.M. Anderson N. Miller D.A. Futral B.T. Bamshad M.J. My46: a Web-based tool for self-guided management of genomic test results in research and clinical settings Genet. Med. 19 2017 467 475 10.1038/gim.2016.133 27632689
42 Bombard Y. Hayeems R.Z. How digital tools can advance quality and equity in genomic medicine Nat. Rev. Genet. 21 2020 505 506 10.1038/s41576-020-0260-x 32601319
43 Edwards K.L. Goodman D. Johnson C.O. Wenzel L. Condit C. Bowen D. Controversies among Cancer Registry Participants, Genomic Researchers, and Institutional Review Boards about Returning Participants' Genomic Results Public Health Genomics 21 2018 18 26 10.1159/000490235 30227419
44 Facio F.M. Eidem H. Fisher T. Brooks S. Linn A. Kaphingst K.A. Biesecker L.G. Biesecker B.B. Intentions to receive individual results from whole-genome sequencing among participants in the ClinSeq study. Research Support, N.I.H., Intramural Eur. J. Hum. Genet. 21 2013 261 265 10.1038/ejhg.2012.179 22892536
45 Halverson C.M. Jones S.H. Novak L. Simpson C. Velez Edwards D.R. Zhao S.K. Clayton E.W. What Results Should Be Returned from Opportunistic Screening in Translational Research? J. Pers. Med. 10 2020 13 10.3390/jpm10010013
46 Karlson E.W. Boutin N.T. Hoffnagle A.G. Allen N.L. Building the Partners HealthCare Biobank at Partners Personalized Medicine: Informed Consent, Return of Research Results, Recruitment Lessons and Operational Considerations J. Pers. Med. 6 2016 2 10.3390/jpm6010002
47 Hoell C. Wynn J. Rasmussen L.V. Marsolo K. Aufox S.A. Chung W.K. Connolly J.J. Freimuth R.R. Kochan D. Hakonarson H. Participant choices for return of genomic results in the eMERGE Network. Genetics in medicine Genet. Med. 22 2020 1821 1829 10.1038/s41436-020-0905-3 32669677
48 Joffe S. Sellers D.E. Ekunwe L. Antoine-Lavigne D. McGraw S. Levy D. Splansky G.L. Preferences for Return of Genetic Results Among Participants in the Jackson Heart Study and Framingham Heart Study Circ. Genom. Precis. Med. 12 2019 e002632 10.1161/CIRCGEN.119.002632
49 Carey D.J. Fetterolf S.N. Davis F.D. Faucett W.A. Kirchner H.L. Mirshahi U. Murray M.F. Smelser D.T. Gerhard G.S. Ledbetter D.H. The Geisinger MyCode community health initiative: an electronic health record-linked biobank for precision medicine research. Genetics in medicine Genet. Med. 18 2016 906 913 10.1038/gim.2015.187 26866580
50 Bollinger J.M. Scott J. Dvoskin R. Kaufman D. Public preferences regarding the return of individual genetic research results: findings from a qualitative focus group study. Genetics in medicine Genet. Med. 14 2012 451 457 10.1038/gim.2011.66 22402755
51 Blout Zawatsky C.L. Shah N. Machini K. Perez E. Christensen K.D. Zouk H. Steeves M. Koch C. Uveges M. Shea J. Returning actionable genomic results in a research biobank: Analytic validity, clinical implementation, and resource utilization Am. J. Hum. Genet. 108 2021 2224 2237 10.1016/j.ajhg.2021.10.005 34752750
52 Beil A. Hornsby W. Uhlmann W.R. Aatre R. Arscott P. Wolford B. Eagle K.A. Yang B. McNamara J. Willer C. Roberts J.S. Disclosure of clinically actionable genetic variants to thoracic aortic dissection biobank participants BMC Med. Genomics 14 2021 66 10.1186/s12920-021-00902-5 33648514
53 Leitsalu L. Palover M. Sikka T.T. Reigo A. Kals M. Pärn K. Nikopensius T. Esko T. Metspalu A. Padrik P. Tõnisson N. Genotype-first approach to the detection of hereditary breast and ovarian cancer risk, and effects of risk disclosure to biobank participants Eur. J. Hum. Genet. 29 2021 471 481 10.1038/s41431-020-00760-2 33230308
54 Johnson G. Lawrenz F. Thao M. An empirical examination of the management of return of individual research results and incidental findings in genomic biobanks. Research Support, N.I.H., Extramural. Genetics in medicine Genet. Med. 14 2012 444 450 10.1038/gim.2012.20 22361899
55 Fiallos K. Applegate C. Mathews D.J. Bollinger J. Bergner A.L. James C.A. Choices for return of primary and secondary genomic research results of 790 members of families with Mendelian disease Eur. J. Hum. Genet. 25 2017 530 537 10.1038/ejhg.2017.21 28272539
56 Keogh L.A. Fisher D. Sheinfeld Gorin S. Schully S.D. Lowery J.T. Ahnen D.J. Maskiell J.A. Lindor N.M. Hopper J.L. Burnett T. How do researchers manage genetic results in practice? The experience of the multinational Colon Cancer Family Registry J. Community Genet. 5 2014 99 108 10.1007/s12687-013-0148-y 23703702
57 Wynn J. Martinez J. Bulafka J. Duong J. Zhang Y. Chiuzan C. Preti J. Cremona M.L. Jobanputra V. Fyer A.J. Impact of Receiving Secondary Results from Genomic Research: A 12-Month Longitudinal Study J. Genet. Couns. 27 2018 709 722 10.1007/s10897-017-0172-x 29168042
58 Graves K.D. Leventhal K.G. Nusbaum R. Salehizadeh Y. Hooker G.W. Peshkin B.N. Butrick M. Tuong W. Mathew J. Goerlitz D. Behavioral and psychosocial responses to genomic testing for colorectal cancer risk Genomics 102 2013 123 130 10.1016/j.ygeno.2013.04.002 23583311
59 Robinson J.O. Wynn J. Biesecker B. Biesecker L.G. Bernhardt B. Brothers K.B. Chung W.K. Christensen K.D. Green R.C. McGuire A.L. Psychological outcomes related to exome and genome sequencing result disclosure: a meta-analysis of seven Clinical Sequencing Exploratory Research (CSER) Consortium studies. Genetics in medicine Genet. Med. 21 2019 2781 2790 10.1038/s41436-019-0565-3 31189963
60 Hart M.R. Biesecker B.B. Blout C.L. Christensen K.D. Amendola L.M. Bergstrom K.L. Biswas S. Bowling K.M. Brothers K.B. Conlin L.K. Secondary findings from clinical genomic sequencing: prevalence, patient perspectives, family history assessment, and health-care costs from a multisite study. Genetics in medicine Genet. Med. 21 2019 1100 1110 10.1038/s41436-018-0308-x 30287922
61 Bradbury A.R. Patrick-Miller L. Egleston B.L. Maxwell K.N. DiGiovanni L. Brower J. Fetzer D. Bennett Gaieski J. Brandt A. McKenna D. Returning Individual Genetic Research Results to Research Participants: Uptake and Outcomes Among Patients With Breast Cancer JCO Precis. Oncol. 2 2018 1 24 10.1200/po.17.00250
62 Kilbride M. Egleston B.L. Chung W.K. Olopade O. Maxwell K.N. Shah P. Churpek J.E. Fleisher L. Terry M.B. Fetzer D. Uptake of Genetic Research Results and Patient-Reported Outcomes With Return of Results Incorporating Web-Based Predisclosure Education J. Clin. Oncol. 41 2023 4905 4915 10.1200/JCO.22.00516 37611220
63 Verma A. Damrauer S.M. Naseer N. Weaver J. Kripke C.M. Guare L. Sirugo G. Kember R.L. Drivas T.G. Dudek S.M. The Penn Medicine BioBank: Towards a Genomics-Enabled Learning Healthcare System to Accelerate Precision Medicine in a Diverse Population J. Pers. Med. 12 2022 1974 10.3390/jpm12121974
64 Miller D.T. Lee K. Chung W.K. Gordon A.S. Herman G.E. Klein T.E. Stewart D.R. Amendola L.M. Adelman K. Bale S.J. ACMG SF v3.0 list for reporting of secondary findings in clinical exome and genome sequencing: a policy statement of the American College of Medical Genetics and Genomics (ACMG). Genetics in medicine Genet. Med. 23 2021 1381 1390 10.1038/s41436-021-01172-3 34012068
65 Bradbury A.R. Patrick-Miller L. Long J. Powers J. Stopfer J. Forman A. Rybak C. Mattie K. Brandt A. Chambers R. Development of a tiered and binned genetic counseling model for informed consent in the era of multiplex testing for cancer susceptibility. Genetics in medicine Genet. Med. 17 2015 485 492 10.1038/gim.2014.134 25297947
66 Kaphingst K.A. Facio F.M. Cheng M.R. Brooks S. Eidem H. Linn A. Biesecker B.B. Biesecker L.G. Effects of informed consent for individual genome sequencing on relevant knowledge Clin. Genet. 82 2012 408 415 10.1111/j.1399-0004.2012.01909.x 22694298
67 Horowitz M. Wilner N. Alvarez W. Impact of Event Scale: a measure of subjective stress Psychosom. Med. 41 1979 209 218 10.1097/00006842-197905000-00004 472086
68 Cella D. Hughes C. Peterman A. Chang C.H. Peshkin B.N. Schwartz M.D. Wenzel L. Lemke A. Marcus A.C. Lerman C. A brief assessment of concerns associated with genetic testing for cancer: the Multidimensional Impact of Cancer Risk Assessment (MICRA) questionnaire Health Psychol. 21 2002 564 572 12433008
69 Brehaut J.C. O'Connor A.M. Wood T.J. Hack T.F. Siminoff L. Gordon E. Feldman-Stewart D. Validation of a decision regret scale Med. Decis. Making 23 2003 281 292 10.1177/0272989X03256005 12926578
70 Gray S.W. Martins Y. Feuerman L.Z. Bernhardt B.A. Biesecker B.B. Christensen K.D. Joffe S. Rini C. Veenstra D. McGuire A.L. CSER Consortium Outcomes and Measures Working Group Social and behavioral research in genomic sequencing: approaches from the Clinical Sequencing Exploratory Research Consortium Outcomes and Measures Working Group. Genetics in medicine Genet. Med. 16 2014 727 735 10.1038/gim.2014.26 24625446
71 Chew L.D. Griffin J.M. Partin M.R. Noorbaloochi S. Grill J.P. Snyder A. Bradley K.A. Nugent S.M. Baines A.D. Vanryn M. Validation of screening questions for limited health literacy in a large VA outpatient population. Multicenter Study Research Support, U.S. Gov't, Non-P.H.S. Validation Studies J. Gen. Intern. Med. 23 2008 561 566 10.1007/s11606-008-0520-5 18335281
72 Institute NC HINTS 5, Cycle 2 https://hints.cancer.gov/ 2018
73 Bradbury A. Patrick-Miller L. Harris D. Stevens E. Egleston B. Smith K. Mueller R. Brandt A. Stopfer J. Rauch S. Utilizing Remote Real-Time Videoconferencing to Expand Access to Cancer Genetic Services in Community Practices: A Multicenter Feasibility Study J. Med. Internet Res. 18 2016 e23 10.2196/jmir.4564
74 Raghunathan T.E. Lepkowski J.M. Van Hoewyk J. Solenberger P. A multivariate technique for multiplying imputing missing values using a sequence of regression models Surv. Methodol. 27 2001 85 95
75 Buchanan A.H. Lester Kirchner H. Schwartz M.L.B. Kelly M.A. Schmidlen T. Jones L.K. Hallquist M.L.G. Rocha H. Betts M. Schwiter R. Clinical outcomes of a genomic screening program for actionable genetic conditions Genet. Med. 22 2020 1874 1882 10.1038/s41436-020-0876-4 32601386
76 Best M.C. Butow P. Savard J. Jacobs C. Bartley N. Davies G. Napier C.E. Ballinger M.L. Thomas D.M. Biesecker B. Preferences for return of germline genome sequencing results for cancer patients and their genetic relatives in a research setting Eur. J. Hum. Genet. 30 2022 930 937 10.1038/s41431-022-01069-y 35277654
77 Faucett W.A. Davis F.D. How Geisinger made the case for an institutional duty to return genomic results to biobank participants Appl. Transl. Genom. 8 2016 33 35 10.1016/j.atg.2016.01.003 27047758
78 Bradbury A.R. Patrick-Miller L. Egleston B.L. Maxwell K.N. DiGiovanni L. Brower J. Fetzer D. Bennett Gaieski J. Brandt A. McKenna D. Returning Individual Genetic Research Results to Research Participants: Uptake and Outcomes Among Patients With Breast Cancer JCO Precis. Oncol. 2 2018 1 24 10.1200/PO.17.00250
79 Stefansdottir V. Thorolfsdottir E. Hognason H.B. Patch C. van El C. Hentze S. Cordier C. Mendes Á. Jonsson J.J. Web-based return of BRCA2 research results: one-year genetic counselling experience in Iceland Eur. J. Hum. Genet. 28 2020 1656 1661 10.1038/s41431-020-0665-1 32523053
80 Gold N.B. Green R.C. Reevaluating the "right not to know" in genomics research Genet. Med. 24 2022 289 292 10.1016/j.gim.2021.10.003 34906468
81 Bombard Y. Ginsburg G.S. Sturm A.C. Zhou A.Y. Lemke A.A. Digital health-enabled genomics: Opportunities and challenges Am. J. Hum. Genet. 109 2022 1190 1198 10.1016/j.ajhg.2022.05.001 35803232
82 Sabatello M. Zhang Y. Chen Y. Appelbaum P.S. In Different Voices: The Views of People with Disabilities about Return of Results from Precision Medicine Research Public Health Genomics 23 2020 42 53 10.1159/000506599 32294660
83 Swisher E.M. Rayes N. Bowen D. Peterson C.B. Norquist B.M. Coffin T. Gavin K. Polinsky D. Crase J. Bakkum-Gamez J.N. Remotely Delivered Cancer Genetic Testing in the Making Genetic Testing Accessible (MAGENTA) Trial: A Randomized Clinical Trial JAMA Oncol. 9 2023 1547 1555 10.1001/jamaoncol.2023.3748 37707822
84 Biesecker B.B. Lewis K.L. Umstead K.L. Johnston J.J. Turbitt E. Fishler K.P. Patton J.H. Miller I.M. Heidlebaugh A.R. Biesecker L.G. Web Platform vs In-Person Genetic Counselor for Return of Carrier Results From Exome Sequencing: A Randomized Clinical Trial JAMA Intern. Med. 178 2018 338 346 10.1001/jamainternmed.2017.8049 29356820
85 Baker A. Tolwinski K. Atondo J. Davis F.D. Goehringer J. Jones L.K. Pisieczko C.J. Sturm A.C. Williams J.L. Williams M.S. Understanding the Patient Experience of Receiving Clinically Actionable Genetic Results from the MyCode Community Health Initiative, a Population-Based Genomic Screening Initiative J. Pers. Med. 12 2022 1511 10.3390/jpm12091511
