
==== Front
JAMA Netw Open
JAMA Netw Open
JAMA Network Open
2574-3805
American Medical Association

39287947
10.1001/jamanetworkopen.2024.32760
zoi240987
Research
Original Investigation
Online Only
Health Informatics
Electronic Health Record Nudges and Health Care Quality and Outcomes in Primary Care
A Systematic Review
EHR Nudges and Health Care Quality and Outcomes in Primary Care
EHR Nudges and Health Care Quality and Outcomes in Primary Care
Nguyen Oliver T. MSHI 1 2
Kunta Avaneesh R. BS 3
Katoju SriVarsha BS 4
Gheytasvand Sara MD 5
Masoumi Niloofar PharmD 6
Tavasolian Ronia MPH 7
Alishahi Tabriz Amir MD PhD MPH 1 8 9
Hong Young-Rock PhD MPH 10
Hanna Karim MD 11
Perkins Randa MD MBA 12
Parekh Arpan BA 13
Turner Kea PhD MPH MA 1 8 9
1 Department of Health Outcomes and Behavior, H. Lee Moffitt Cancer Center & Research Institute, Tampa, Florida
2 Department of Industrial and Systems Engineering, University of Wisconsin at Madison, Madison
3 College of Medicine, University of Central Florida, Orlando
4 Department of Community Health and Family Medicine, University of Florida, Gainesville
5 Tabriz University of Medical Sciences, Tabriz, Iran
6 College of Pharmacy, Tehran University of Medical Sciences, Tehran, Iran
7 Department of Clinical Science and Nutrition, University of Chester, England
8 Department of Oncologic Science, University of South Florida, Tampa
9 Department of Gastrointestinal Oncology, H. Lee Moffitt Cancer Center & Research Institute, Tampa, Florida
10 Department of Health Services Research, Management, and Policy, University of Florida, Gainesville
11 Department of Family Medicine, University of South Florida, Tampa
12 Department of Internal Medicine, H. Lee Moffitt Cancer Center & Research Institute, Tampa, Florida
13 College of Medicine, University of Miami, Miami, Florida
Article Information

Accepted for Publication: July 4. 2024.

Published: September 17, 2024. doi:10.1001/jamanetworkopen.2024.32760

Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2024 Nguyen OT et al. JAMA Network Open.

Corresponding Author: Oliver T. Nguyen, MSHI, Department of Industrial and Systems Engineering, University of Wisconsin at Madison, 1513 University Ave, Madison, WI 53706 (fnguyen2@wisc.edu).
Author Contributions: Mr Nguyen and Dr Turner had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: Nguyen, Kunta, Hanna, Perkins, Turner.

Acquisition, analysis, or interpretation of data: Nguyen, Kunta, Katoju, Gheytasvand, Masoumi, Tavasolian, Alishahi Tabriz, Hong, Parekh, Turner.

Drafting of the manuscript: Nguyen, Gheytasvand, Hanna.

Critical review of the manuscript for important intellectual content: Kunta, Katoju, Masoumi, Tavasolian, Alishahi Tabriz, Hong, Hanna, Perkins, Parekh, Turner.

Statistical analysis: Kunta, Masoumi.

Administrative, technical, or material support: Gheytasvand, Perkins, Parekh, Turner.

Supervision: Nguyen, Tavasolian, Alishahi Tabriz, Hanna, Turner.

Conflict of Interest Disclosures: None reported.

Data Sharing Statement: See Supplement 2.

Additional Contributions: We would like to thank Mary Kate, medical librarian, Haver, for her feedback on our search strategy.

17 9 2024
9 2024
17 9 2024
7 9 e24327608 1 2024
4 7 2024
Copyright 2024 Nguyen OT et al. JAMA Network Open.
https://creativecommons.org/licenses/by/4.0/ This is an open access article distributed under the terms of the CC-BY License.
jamanetwopen-e2432760.pdf

Key Points

Question

What is the association between electronic health record (EHR) nudges and health care quality and outcomes in primary care?

Findings

In this systematic review including 54 randomized clinical trials, EHR nudges were associated with improvements in some areas of health care quality, such as descriptive and patient-centeredness measures. However, the evidence on other health care quality areas and clinical outcomes was less consistent.

Meaning

These results suggest that EHR nudges may help improve clinicians’ documentation behaviors through higher completeness and accuracy of office notes.

This systematic review investigates whether there is an association between electronic health record (EHR) nudges and health care outcomes in primary care settings and describes implementation facilitators and barriers.

Importance

Nudges have been increasingly studied as a tool for facilitating behavior change and may represent a novel way to modify the electronic health record (EHR) to encourage evidence-based care.

Objective

To evaluate the association between EHR nudges and health care outcomes in primary care settings and describe implementation facilitators and barriers.

Evidence Review

On June 9, 2023, an electronic search was performed in PubMed, Embase, PsycINFO, CINAHL, and Web of Science for all articles about clinician-facing EHR nudges. After reviewing titles, abstracts, and full texts, the present review was restricted to articles that used a randomized clinical trial (RCT) design, focused on primary care settings, and evaluated the association between EHR nudges and health care quality and patient outcome measures. Two reviewers abstracted the following elements: country, targeted clinician types, medical conditions studied, length of evaluation period, study design, sample size, intervention conditions, nudge mechanisms, implementation facilitators and barriers encountered, and major findings. The findings were qualitatively reported by type of health care quality and patient outcome and type of primary care condition targeted. The Risk of Bias 2.0 tool was adapted to evaluate the studies based on RCT design (cluster, parallel, crossover). Studies were scored from 0 to 5 points, with higher scores indicating lower risk of bias.

Findings

Fifty-four studies met the inclusion criteria. Overall, most studies (79.6%) were assessed to have a moderate risk of bias. Most or all descriptive (eg, documentation patterns) (30 of 38) or patient-centeredness measures (4 of 4) had positive associations with EHR nudges. As for other measures of health care quality and patient outcomes, few had positive associations between EHR nudges and patient safety (4 of 12), effectiveness (19 of 48), efficiency (0 of 4), patient-reported outcomes (0 of 3), patient adherence (1 of 2), or clinical outcome measures (1 of 7).

Conclusions and Relevance

This systematic review found low- and moderate-quality evidence that suggested that EHR nudges were associated with improved descriptive measures (eg, documentation patterns). Meanwhile, it was unclear whether EHR nudges were associated with improvements in other areas of health care quality, such as effectiveness and patient safety outcomes. Future research is needed using longer evaluation periods, a broader range of primary care conditions, and in deimplementation contexts.
==== Body
pmcIntroduction

Primary care clinicians are pivotal in population health by lowering health care costs and reducing mortality globally.1,2,3 Despite its importance, primary care delivery varies substantially across settings. Across the many conditions treated in primary care, evidence-to-practice gaps exist.4,5,6 To address this, research has examined how to change clinician behavior to encourage the use of evidence-based care. Nudges, or interventions that subtly guide individuals toward specific behaviors while preserving freedom of choice, represent one approach of interest.7

Nudges have been increasingly studied for their potential use in facilitating behavior change among individuals.8,9,10,11 Nudges do not require extensive resources to implement and target automatic cognitive processing, which may be used in situations when time is limited (eg, primary care setting). Nudges may potentially affect primary care clinicians’ behaviors.10,12 For instance, studies have found that a variety of nudges (eg, public commitments, social norm feedback) can potentially improve antibiotic prescribing for viral infections.10 Consequently, these encouraging results have led some health systems to examine how to improve the reach and sustainability of nudge interventions.

Since adoption of electronic health record (EHR) systems is high,13 the EHR represents an additional medium to disseminate nudge interventions.14 The EHR also offers the opportunity to present the nudge to clinicians in their workflows and allows health systems to monitor the effectiveness of implemented nudges across different medical specialties and clinician types. Clinicians may also benefit from EHR nudges through the reduction of cognitive load, especially those that aim to provide reminders, offer preselected options at decision points for review, or provide additional contextual information (eg, relative costs of medications) at the point of decision-making. In turn, this may help, in part, facilitate desired clinician behaviors. In light of a growing number of randomized clinical trials (RCTs) evaluating EHR nudges in primary care, there is a growing need to summarize these findings and determine if the benefits of EHR nudges apply uniformly to the different types of conditions that primary care clinicians typically manage. To our knowledge, EHR nudges have only been reviewed in the inpatient setting,11 which may differ from those used in primary care settings due to differences in EHR workflows, patient complexity, and alert fatigue.

To address this research gap, we conducted a systematic review to summarize how clinician-facing EHR nudges are associated with health care quality (eg, documentation patterns, ordering behaviors) and patient outcomes in primary care settings. We also summarized implementation facilitators and barriers of EHR nudge interventions. Findings from this line of research may reveal future research areas and influence policies aimed at improving evidence-based care delivery in primary care.

Methods

We reported this systematic review according to the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) reporting guideline.15 This review was registered with PROSPERO (CRD42022309599).

Data Sources and Search

We considered clinician-facing EHR nudges to include subtle modifications integrated into an EHR’s choice architecture, specifically designed to influence medical decision-making without limiting clinicians’ autonomy. These nudges are intended to guide clinicians toward preferred actions or decisions (eg, generic formulations over brand-name drugs) while allowing them to disregard the suggestions if they are not suitable or practical for a given patient (eg, patient allergies, lack of insurance coverage). Clinician-facing EHR nudges could include order sets or default dosages that represent the health system’s preferred treatment pathways for a given clinical situation. They may include alerts and/or reminders designed to more easily complete a desired action (eg, adding immunization orders, adding diagnosis to EHR problem list) and redesigning checklists or dropdown menus in documentation templates to display the health system’s preferred options first. Accordingly, in consultation with a health sciences librarian, we developed a search strategy (eTable 1 in Supplement 1) that included search terms for clinicians, EHRs, and nudges.

On June 9, 2023, we ran the search strategy on MEDLINE, Embase, PsycINFO, CINAHL, and Web of Science to identify relevant peer-reviewed literature. Our inclusion criteria included articles written in English, conducted in primary care settings (ie, family medicine, general internal medicine, general pediatrics), used a RCT design, used nudges developed within the EHR environment, and evaluated how the nudge was associated with health care quality and patient outcome measures. We excluded articles that were nonempirical (ie, commentaries, other reviews) or did not explicitly discuss what clinicians could do if they disagreed with a recommendation. Lastly, we hand-searched the reference lists of included studies to identify other potentially relevant articles.

Study Selection

Two reviewers were randomly assigned to independently screen the titles and abstracts of included studies. For full-text reviews, 2 reviewers were also randomly assigned to independently assess each article for eligibility. In both review stages, conflicts were resolved by a third reviewer. All reviews were conducted using the web-based software, Covidence.

Data Extraction and Quality Assessment

For each included study, 2 reviewers abstracted the following elements: country, targeted clinician types, medical conditions studied, length of evaluation period, study design, sample size, intervention characteristics, nudge mechanisms, implementation facilitators and barriers encountered, and major findings. Since a nudging intervention could involve several mechanisms, we mapped identified nudges to the nudging ladder model used in a prior systematic review.16 We adapted the Risk of Bias 2.0 tools to evaluate the studies based on randomized clinical trial design (cluster, parallel, crossover). Our team scored studies from 0 to 5 points, where those that scored 0 to 1 point were considered high risk of bias, 2 to 3 points were considered moderate risk of bias, and 4 to 5 points were considered low risk of bias.

Statistical Analysis

We were unable to conduct a meta-analysis because outcome definitions across studies differed immensely, multiple studies had missing information on precision estimates, and there was high prevalence of studies of moderate or high risk of bias.17,18 Thus, we used the narrative review approach to report results by outcome type and clinical concept.17 For health care quality, measure types were derived from a quality framework and included patient safety, effectiveness, patient-centeredness, timeliness, efficiency, and descriptive.19 For patient outcomes, measure types were formed after identifying themes in the findings and included patient-reported outcomes, patient adherence, and clinical outcomes. For clinical concept, we adapted a previously published taxonomy of the most common primary care conditions to classify disease types.20 For each reported finding, we included point estimates and P values or 95% CIs. We considered a 2-sided threshold of P < .05 as statistically significant. We also examined potential differences in findings if studies with high risk of bias were excluded from the analysis.

Results

Study Characteristics

Overall, we found 18 381 articles and included 54 studies in this systematic review. The Figure illustrates the outcomes of our review process. Most studies were located in the US (79.6% [43 of 54]) and had a cluster RCT study design (63.0% [34 of 54]). Over time, there was a general trend in an increasing number of studies that use a RCT design to evaluate the EHR nudge. Approximately 68.5% of studies (37 of 54) implemented the EHR nudge alongside another intervention (eg, providing education, providing reference or support materials). Of the 34 studies that reported which EHR vendor their nudge was implemented in, 15 (44.1%) used Epic. Most studies evaluated process measures with few studies attempting to assess clinical outcome measures. Furthermore, when examining primary care conditions targeted, most measures pertained to chronic care management (eg, diabetes, hypertension) and preventative care (eg, breast cancer screening). Although physicians were the most studied clinician type, 15 studies also examined other types of clinicians, including nurse practitioners, physician assistants, registered nurses, and medical assistants. Most studies (79.6% [43 of 54]) used EHR nudges to facilitate the uptake of a targeted behavior while the remaining studies (20.4% [11 of 54]) used EHR nudges to facilitate a decrease in a targeted behavior. The number of studies that reported an association with improved outcomes was higher among studies facilitating an increase in a targeted behavior compared with those facilitating a decrease in a targeted behavior (54.7% vs. 9.1%). Lastly, only 7 studies formally assessed implementation factors. All of these studies identified implementation barriers, but none reported implementation facilitators (eTable 2 in Supplement 1). Table 1 displays a summary of the proportion of studies that showed positive, negative, and null associations of EHR nudges and each outcome type. Detailed results on how EHR nudges were associated with health care quality and outcome measures are shown in Table 2 and Table 3, respectively.

Figure. Study Flow Diagram

EHR indicates electronic health record.

Table 1. Direction of Associations Between Electronic Health Record Nudges and Health Care Quality and Patient Outcomes

Outcome type	Total No. of measures	No. (%)	
No. of measures with associations	No. of measures	
With improved outcomes	With worsened outcomes	With mixed associations	With null associations	
Patient safety	12	4 (33.3)	0	2 (16.7)	6 (50.0)	
Effectiveness	48	19 (39.6)	0	13 (27.1)	16 (25.0)	
Patient-centeredness	3	3 (100.0)	0	0	0	
Timeliness	0	NA	NA	NA	NA	
Efficiency	4	0	0	0	4 (100.0)	
Descriptive	38	30 (78.9)	0	1 (2.6)	7 (18.4)	
Patient-reported outcomes	3	0	0	1 (33.3)	2 (66.7)	
Patient adherence	2	1 (50.0)	0	0	1 (50.0)	
Clinical outcomes	7	1 (14.3)	0	0	6 (85.7)	
Abbreviation: NA, not applicable.

Table 2. Overall Findings of Association Between Electronic Health Record Nudges and Health Care Quality Measures

Measure type and concept	Citation(s)	Measures	Overall findingsa	
Patient safety	
Appropriate prescribing	Campbell et al,21 2021	Anticholinergic discontinuation	7.8% vs 8.2%; P = .65	
Appropriate prescribing	Kraemer et al,22 2022	Opioid prescriptions at index visit	OR = 0.74 (95% CI, 0.46-1.18)	
Appropriate prescribing	Kraemer et al,22 2022	Continued use of opioid prescriptions	OR = 1.08 (95% CI, 0.94-1.24)	
Appropriate prescribing	Kraemer et al,22 2022	Concurrent opioid and benzodiazepine use	OR = 1.10 (95% CI, 0.94-1.29)	
Appropriate prescribing	Tamblyn et al,23 2012	Psychotropic prescription	Immediate-acting benzodiazepines: mean difference = −0.008 (95% CI, −0.05 to 0.03); long-acting benzodiazepines: mean difference = −0.006 (95% CI, −0.001 to −0.000); antidepressants: mean difference = −0.011 (95% CI, −0.03 to 0.01);
immediate potency opiates: mean difference = 0.001 (95% CI, −0.00 to 0.01);
low potency opiates: mean difference = −0.004 (95% CI, −0.01 to 0.00);
anticonvulsants: mean difference = 0.006 (95% CI, −0.00 to 0.01);
antipsychotics: mean difference = 0.005 (95% CI, −0.00 to 0.01)
No.of psychotropic medications prescribed: mean difference = −0.02 (95% CI, −0.09 to 0.05)	
Appropriate prescribing	Gill et al,24,2011; Abdel-Kader et al,25 2011	Discontinued NSAIDs	Gill et al: OR = 1.18 (95% CI, 0.99-1.40); Abdel-Kader et al: OR = 1.43 (95% CI, 0.32-6.33)	
Appropriate prescribing	Tamblyn et al,23 2012	Drug-related injuries	Mean difference: −0.17 (95% CI, −0.32 to −0.02)	
Appropriate prescribing	Flottorp et al,26 2002; Høye et al,27 2013; Meeker et al,28 2016; Gulliford et al,29 2019	Ordering antibiotics for viral infections	Flottorp et al: −3.0% difference, P = .003 (pharyngitis); −0.4% difference, P = .64 (urinary tract infections); Høye et al: OR = 0.72 (95% CI, 0.60-0.86) (upper respiratory infections); Meeker et al: DiD: −5% (95% CI, −7.8 to 0.1) (suggested alternatives mechanism); DiD: −7% (95% CI, −9.1 to −2.9) (accountable justification mechanism); Gulliford et al: IRR = 0.88 (95% CI, 0.78-0.99 (upper respiratory infections)	
Appropriate prescribing	Gill et al,24 2011	Provided guideline-concordant care	OR = 1.19 (95% CI, 1.01-1.42)	
Appropriate prescribing	Tamblyn et al,30 2003	Inappropriate prescribing of new medications	RR = 0.82 (95% CI, 0.69-0.98)	
Appropriate prescribing	Tamblyn et al,30 2003	Inappropriate discontinuation of medications	RR = 1.06 (95% CI, 0.89-1.26)	
Appropriate prescribing	Fortuna et al,31 2009	Prescribing of heavily marketed hypnotics	RR = 0.74 (95% CI, 0.57-0.96)	
Effectiveness	
Diabetes	Sequist et al,32 2005	Eye exam screening	HR = 1.38 (95% CI, 0.81-2.32)	
Diabetes	Sequist et al,32 2005	Receiving recommended diabetes care	OR = 1.30 (95% CI, 1.01-1.67)	
Diabetes, hypertension	Sequist et al,32 2005; Abdel-Kader et al,25 2011; Sequist et al,33 2018; Tamblyn et al,34 2018	Antihypertensive prescriptions	Sequist et al (2005): HR = 1.42 (95% CI, 0.94-2.14) (ACE inhibitors for patients with diabetes); HR = (95% CI, 0.72 - 1.63); (β-blockers for CAD); Abdel-Kader et al: OR = 0.84 (95% CI, 0.50-1.41) (ACE or ARB for CKD); Sequist et al (2018): 76% vs 79%, P = .17 (ACE or ARB for high-risk CKD); 64% vs 65%, P = .57 (ACE or ARB for low-risk CKD); Tamblyn et al: RR = 1.65 (95% CI, 1.17-2.33) (diuretics for newly diagnosed hypertension); RR = 0.61 (95% CI, 0.43-0.86) (other antihypertensives for newly diagnosed hypertension); RR = 1.10 (95% CI, 0.73-1.66) (prescribing 1 antihypertensive for newly diagnosed hypertension); RR = 0.91 (95% CI, 0.60-1.37) (prescribing 2 or more antihypertensives for newly diagnosed hypertension); RR = 1.09 (95% CI, 0.79-1.52) (diuretic for established hypertension); RR = 0.91 (95% CI, 0.66-1.27) (other antihypertensives for established hypertension); RR = 1.13 (95% CI, 0.90-1.42) (1 antihypertensive prescribed for established hypertension); RR = 0.88 (95% CI, 0.70-1.11) (2 or more antihypertensives prescribed for established hypertension)	
Diabetes, other (CAD, hyperlipidemia)	Sequist et al,32 2005; Gill et al,35 2009; Adusumalli et al,36 2023	Statin prescriptions	Sequist et al: HR = 1.10 (95% CI, 0.65-0.85 (only patients with diabetes); Sequist et al: HR = 1.51 (95% CI, 1.05-2.17) (only patients with CAD); Gill et al: OR = 0.05, P > .05
Adusumalli et al: 5.5% difference (95% CI, 3.2%-8.1%)	
Immunizations	Frank et al,37 2004	Tetanus immunizations	RR = 1.89 (95% CI, 1.59-2.25)	
Immunizations	Frank et al,37 2004; Loo et al,38 2011	Pneumococcal immunizations	Frank et al: RR = 1.70 (95% CI, 1.10-2.62); Loo et al: OR = 2.01 (95% CI, 1.30-3.11)	
Immunizations	Frank et al,37 2004	Measles, mumps, and rubella immunizations	RR = 1.25 (95% CI, 0.82-1.93)	
Immunizations	Frank et al,37 2004; Fiks et al,39 2009; Loo et al,38 2011; Szilagyi et al,40 2015	Influenza immunizations	Frank et al: RR = 0.96 (95% CI, 0.78-1.18); Fiks et al: OR = 1.22 (95% CI, 0.94-1.61); Loo et al: OR = 1.53 (95% CI, 1.23-1.91); Szilagyi et al: OR = 0.93 (95% CI, 0.69-1.25); OR = 0.89 (95% CI, 0.69-1.16) (reported data across 2 sites separately)	
Immunizations	Fiks et al,41 2013; Szilagyi et al,40 2015	HPV immunizations	Fiks et al: HR = 1.5 (95% CI, 1.2 - 2.0 (dose 1); HR = 1.0 (95% CI, 0.8-1.1) (dose 2); HR = 1.1 (95% CI, 0.9-1.3) (dose 3); Szilagyi et al: OR = 0.92 (95% CI, 0.60-1.40; OR = 0.96 (95% CI, 0.59-1.56) (dose 1 at 2 clinics); OR = 1.01 (95% CI, 0.57-1.77); OR = 1.06 (95% CI, 0.68-1.66) (dose 2 at 2 clinics); OR = 0.93 (95% CI, 0.69-1.25); OR = 1.13 (95% CI, 0.68-1.88) (dose 3 at 2 clinics)	
Immunizations	Stockwell et al,42 2015; Szilagyi et al,40 2015; Stephens et al,43 2021	Pediatric immunizations	Stockwell et al: RR = 0.90 (95% CI, 0.83-0.98) (up-to-date immunizations); Szilagyi et al: OR = 1.44 (95% CI, 0.82-2.56); OR = 1.16 (95% CI, 0.68-1.99) (Tdap from 2 clinics); OR = 1.15 (95% CI, 0.64-2.05); OR = 1.08 (95% CI, 0.82-1.41) (MCV4 at 3 clinics); Stephens et al: 3.7% difference (95% CI, 1.8%-5.6%) (among young children); 3.2% difference (95% CI, 0.6%-6.9%) (among adolescents); 0.8% difference, 95% CI, −0.3% to 1.8%) (under-immunizations)	
Immunizations	Fiks et al,39 2009	Having up-to-date influenza immunization	3.4% difference (95% CI, −1.4 to 9.1)	
Other (ADHD)	Co et al,44 2010	Assessing ADHD care	OR = 2.2 (95% CI, 1.2-4.0)	
Other (asthma)	Bell et al,45 2010	Prescribing asthma controller	Urban practices: 6% difference, P = .006; suburban practices: 14% difference, P = .03	
Other (atrial fibrillation)	Karlsson et al,46 2018	Anticoagulant prescription	70.3% vs 70.0%, P = .01	
Other (atrial fibrillation)	McKie et al,47 2020	Guideline-concordant care for atrial fibrillation	OR = 0.94 (95% CI, 0.15 - 5.94)	
Other (CAD)	Sequist et al,32 2005; Sequist et al,48 2012	Aspirin prescriptions	Sequist et al (2005): HR = 2.36 (95% CI, 1.37-4.07) (CAD); Sequist et al (2012): 20% vs 18%, P = .43	
Other (CAD)	Sequist et al,32 2005	Receiving recommended coronary artery disease care	OR = 1.25 (95% CI, 1.01-1.55)	
Other (chest pain)	Sequist et al,48 2012	Echocardiograms	51% vs 48%, P = .33	
Other (domestic violence screening)	Feder et al,49 2011	Referral to domestic violence agency	IRR = 22.1 (95% CI, 11.5-42.4)	
Other (GERD, drug side effects)	Player et al,50 2010; Gill et al,24 2011	Prescribing gastroprotective medications	Player et al: OR = 1.11 (95% CI, 0.86-1.43) (newly diagnosed GERD); OR = 1.37 (95% CI, 1.12-1.68) (established patients with GERD); Gill et al: OR = 1.33 (95% CI, 1.01-1.74 (among patients receiving NSAIDs)	
Other (heart failure)	McKie et al,47 2020	Guideline-concordant care for heart failure	OR = 7.6 (95% CI, 1.2-47.5)	
Other (hyperlipidemia)	McKie et al,47 2020	Guideline-concordant care for hyperlipidemia	OR = 1.1 (95% CI, 0.6-1.8)	
Other (kidney disease)	Abdel-Kader et al,25 2011; Sequist et al,33 2018	Proteinuria assessment	Abdel-Kader et al: OR = 1.73 (95% CI, 0.77-3.87); Sequist et al: 71% vs 70%, P = .35 (patients with high risk of CKD); 45% vs 21%, P < .001 (patients with low risk of CKD)	
Other (kidney disease)	Sequist et al,33 2018	eGFR test	Patients with high risk of CKD: 89% vs 89%, P = .90; patients with low-risk of CKD: 82% vs 80%, P = .20	
Other (kidney disease)	Sequist et al,33 2018	Hemoglobin test	Patients with high risk of CKD: 73% vs 73%, P = .63; patients with low risk of CKD: 61% vs 61%, P = .87	
Other (kidney disease)	Sequist et al,33 2018	Phosphorous lab test	Patients with high risk of CKD: 49% vs 38%, P < .001; patients with low risk of CKD: 23% vs 13%, P < .001	
Other (kidney disease)	Sequist et al,33 2018	25-OH vitamin D lab	Patients with high risk of CKD: 53% vs 45%, P = .002; patients with low risk of CKD: 31% vs 24%, P = .004	
Other (kidney disease)	Sequist et al,33 2018	Calcium lab	Patients with high risk of CKD: 75% vs 69%, P = .01; patients with low risk of CKD: 59% vs 54%, P = .11	
Other (kidney disease)	Sequist et al,33 2018	Parathyroid hormone lab	Patients with high risk of CKD: 49% vs 39%, P < .001; patients with low risk of CKD: 24% vs 14%, P < .001	
Other (kidney disease)	Sequist et al,33 2018	Annual nephrology visit	Patients with high risk of CKD: 45% vs 34%, P < .001
Patients with low risk of CKD: 17% vs 11%, P = .001	
Other (lab monitoring)	Palen et al,51 2006; Feldstein et al,52 2006	Monitoring laboratory values	Palen et al: 56.6% vs 57.1%, P = .31; Feldstein et al: HR = 2.5 (95% CI, 1.8-3.5), P < .001	
Other (osteoporosis)	Loo et al,38 2011	Bone density scan	OR = 1.43 (95% CI, 0.94-2.17)	
Other (substance use)	Linder et al,66 2009	Smoking cessation prescriptions	2.0% vs 2.0%, P = .40	
Other (substance use)	Linder et al,66 2009	Smoking counseling referrals	4.5% vs 0.4%, P < .001	
Other (substance use)	Lee et al,54 2023	Alcohol treatment initiation	7.8% vs 6.2%, P = .04	
Other (substance use)	Lee et al,54 2023	Positive alcohol screen	18.0% vs 5.0%, P < .001	
Other (substance use)	Lee et al,54 2023	Assessment DSM-5 Alcohol Symptom Checklist	80.9% vs 4.1%, P < .001	
Other (weight issues)	Schriefer et al,55 2009	Weight loss prescriptions	0.5% vs 0.2%, P = .86	
Other (weight issues)	Schriefer et al,55 2009	Weight loss program referrals	1.1% vs 1.3%, P = 1.00	
Other (weight issues)	Schriefer et al,55 2009	Bariatric surgery referrals	0.8% vs 0.6%, P = 1.00	
Other (weight issues)	Schriefer et al,55 2009	Combination therapy for patients with obesity	11.9% vs 6.6%, P = .01	
Routine health maintenance (cancer screening)	Frank et al,37 2004	Cervical smear test	RR = 1.09, 95% CI 0.91-1.29	
Routine health maintenance (cancer screening)	Hsu et al,56 2013	HPV screening	40.9% vs 1.1%, P < .001	
Routine health maintenance (hepatitis screening)	Hsu et al,56 2013; Chak et al,57 2018; Chak et al,58 2020	Hepatitis B screening	Hsu et al: 34.1% vs 0.0%, P < .001; Chak et al (2018): OR = 3.13 (95% CI, 2.18-4.48); Chak et al (2020): OR = 3.23 (95% CI, 2.24-4.67)	
Routine health maintenance (hepatitis screening)	Federman et al,59 2017	Hepatitis C screening	OR = 8.99 (95% CI, 7.57-10.70)	
Routine health maintenance (wellness screening)	Frank et al,37 2004; Sequist et al,32 2005; van Wyk et al,60 2008; Gill et al,35 2009; O’Connor et al,61 2011; Sequist et al,33 2018	Lipids screening	Frank et al: RR = 0.89 (95% CI, 0.73-1.09); Sequist et al (2005): HR = 1.41 (95% CI, 1.15-1.72) (among patients with diabetes); HR = 0.99 (95% CI, 0.75-1.29) (among patients with CAD); van Wyk et al: RR = 1.76 (95% CI, 1.41-2.20) (dyslipidemia screening); RR = 1.40 (95% CI, 1.15-1.70) (dyslipidemia treatment); Gill et al: OR = 15.00, P < .05 (for patients of high risk); OR = 1.47, P > .05 (for patients of moderate risk); OR = 0.97, P > .05 (for patients of low risk); O’Connor et al: 3.3% difference, P = .14 (among patients with diabetes); Sequist et al (2018): 82% vs 83%, P = .24 (for patients of high risk of CKD); 72% vs 70%, P = .19 (for patients of low risk of CKD)	
Routine health maintenance (wellness screening)	Frank et al,37 2004; Sequist et al,32 2005; O’Connor et al,61 2011; Zera et al,62 2015; Weiner et al,63 2020	Diabetes screening	Frank et al: RR = 0.98 (95% CI, 0.65-1.48); Sequist et al: HR = 1.14 (95% CI, 0.89-1.46) (among patients with diabetes); O’Connor et al: 4.1% difference, P = .045 (among patients with diabetes); Zera et al: OR = 1.04 (95% CI, 0.79-1.38); Weiner et al: 0.72 vs 0.74, P = .07 (A1C tests); 1.55 vs 1.63, P = .49 (No.of glucose tests); 1.20 vs 1.22, P = .63 (No. of creatinine tests)	
Routine health maintenance (wellness screening)	McDowell et al,64 1989; Frank et al,37 2004; O’Connor et al,61 2011; Kharbanda et al,65 2018	Measuring blood pressure	McDowell et al: 30.7% vs 21.1%, P < .001; Frank et al: HR = 1.02 (95% CI, 0.90-1.16); O’Connor et al: 0.8% difference, P = .28 (patients with diabetes); Kharbanda et al: 14.3% vs 10.6%, P = .07 (within 30 d of index visit); 26.0% vs 23.4%, P = .46 (within 90 d of index visit)	
Patient-centeredness	
Other (weight issues)	Schriefer et al,55 2009	Diet counseling for patients with obesity	14.0% vs 7.3%, P = .002	
Other (weight issues)	Schriefer et al,55 2009	Exercise counseling for patients with obesity	12.1% vs 7.1%, P = .02	
Hypertension	Kressin et al,67 2016	Blood pressure counseling	β = 1.10, P = .01	
Timeliness	
NA	None	NA	NA	
Efficiency	
Upper respiratory infections, urinary tract infections	Flottorp et al,26 2002	Ordering labs for viral infections	Urinary tract infections: −5.1% difference, P = .05; pharyngitis: −0.5% difference, P = .64	
Other (lab monitoring)	Lo et al,69 2009	Appropriate ordering of labs	OR = 1.05 (95% CI, 0.75-1.46)	
Other (chest pain)	Sequist et al,48 2012	Cardiac stress testing	10% vs 9%, P = .40	
Other (musculoskeletal pain)	Zafar et al,68 2019	Lumbar spine MRI	Same-day orders: OR = 0.92 (95% CI, 0.68-1.25); orders within 30 d of visit: OR = 0.94 (95% CI, 0.70-1.25)	
Descriptive	
Routine health maintenance (wellness screening)	Frank et al,37 2004	Documenting allergies	RR = 1.81 (95% CI, 1.63-2.02)	
Routine health maintenance (wellness screening)	Frank et al,37 2004	Documenting weight	RR = 1.28 (95% CI, 1.13-1.44)	
Routine health maintenance (wellness screening)	Frank et al,37 2004	Documenting smoking status	RR = 1.12 (95% CI, 0.90-1.39)	
Other (weight issues)	Schriefer et al,55 2009; Tang et al,70 2012	Diagnosis of obesity	Schriefer et al: 16.6% vs 10.7%, P = .02; Tang et al: OR = 4.1 (95% CI, 1.3-12.7)	
Other (ADHD)	Co et al,44 2010	Documenting ADHD symptoms	100% vs 61.3%, P < .001	
Other (ADHD)	Co et al,44 2010	Documenting ADHD treatment effectiveness	96.6% vs 54.8%, P < .001	
Other (ADHD)	Co et al,44 2010	Documenting ADHD adverse events	96.6 vs 40.3%, P < .001	
Other (GERD)	Player et al,50 2010	Diagnosis of GERD	OR = 1.33 (95% CI, 1.13-1.56)	
Other (asthma)	Bell et al,45 2010	Filing up-to-date asthma care plan	Urban practices: 1% difference, P > .05; suburban practices: 25% difference, P = .03	
Other (asthma)	Bell et al,45 2010	Documenting spirometry	Urban practices: 3% difference, P = .04; suburban practices: 13% difference, P = .003	
Other (kidney disease)	Abdel-Kader et al,25 2011	CKD documentation	OR = 1.23 (95% CI, 0.60-2.51)	
Other (domestic violence screening)	Feder et al,49 2011	Documenting domestic violence	IRR = 3.1 (95% CI, 2.2-4.3)	
Other (weight issues)	Tang et al,70 2012	Documenting results of weight counseling	OR = 2.1 (95% CI, 1.1-4.1)	
Other (ADHD)	Wright et al,71 2012	ADHD problem list	OR = 2.23 (P < .0001)	
Other (asthma)	Wright et al,71 2012	Asthma/COPD problem list	OR = 2.98 (P < .0001)	
Other (cancer)	Wright et al,71 2012	Breast cancer problem list	OR = 1.78 (P < .001)	
Other (CAD)	Wright et al,71 2012	CAD problem list	OR = 4.66 (P < .0001)	
Other (coagulopathy)	Wright et al,71 2012	Congenital coagulopathy problem list	OR = 2.06, P = .04, finding not significant due to a Bonferroni correction	
Other (CHF)	Wright et al,71 2012	CHF problem list	OR = 7.56, P < .0001	
Diabetes	Wright et al,71 2012	Diabetes mellitus problem list	OR = 1.97, P < .0001	
Other (glaucoma)	Wright et al,71 2012	Glaucoma problem list	OR = 3.78, P < .0001	
Hypertension	Wright et al,71 2012	Hypertension problem list	OR = 4.12, P < .0001	
Other (thyroid issues)	Wright et al,71 2012	Hyperthyroidism problem list	OR = 1.30, P = .29	
Other (thyroid issues)	Wright et al,71 2012	Hypothyroidism problem list	OR = 3.99, P < .0001	
Other (autoimmune issues)	Wright et al,71 2012	Myasthenia gravis problem list	OR = 2.10, P = .11	
Other (osteoporosis)	Wright et al,71 2012	Osteoporosis/Osteopenia problem list	OR = 3.40, P < .0001	
Other (autoimmune issues)	Wright et al,71 2012	Rheumatoid arthritis problem list	OR = 3.97, P < .0001	
Other (kidney issues)	Wright et al,71 2012	Renal failure/insufficiency problem list	OR = 8.22, P < .0001	
Other (sickle cell disease)	Wright et al,71 2012	Sickle cell disease problem list	OR = 1.66, P = .29	
Other (stroke)	Wright et al,71 2012	Stroke problem list	OR = 2.35, P = .0002	
Other (social determinants of health screening)	Weiner et al,73 2022	Contextual factors integrated into care plan	OR = 2.67 (95% CI, 1.32-5.41)	
Other (gun storage screening)	Sigel et al,72 2023	Documenting gun storage counseling	51.2% vs 20.0%, P = .04	
Other (substance use)	Lee et al,54 2023	Alcohol intervention documented	57% vs 11%, P < .001	
Routine health maintenance (wellness screening)	Lee et al,54 2023	Alcohol screening documented	83.2% vs 20.8%, P < .001	
Other (substance use)	Lee et al,54 2023	Alcohol diagnoses	33.8% vs 28.8%, P = .003	
Other (designating a proxy)	Loo et al,38 2011	Designation of health care proxy	OR = 1.55 (95% CI, 1.00-2.41)	
Other (social determinants of health screening)	Weiner et al,63 2022	Improving or resolving “red flags”	OR = 0.96 (95% CI, 0.57-1.63)	
Other (social determinants of health screening)	Weiner et al,63 2022	Probing “red flags”	OR = 2.12 (95% CI, 1.14-3.93)	
Abbreviations: ACE, angiotensin-converting enzyme inhibitors; ADHD, attention-deficit/hyperactivity disorder; ARB, angiotensin receptor blockers; CAD, coronary artery disease; CHF, coronary heart failure; CKD, chronic kidney disease; COPD, chronic obstructive pulmonary disease; DiD, difference-in-differences; eGFR, estimated glomerular filtration rate; EHR, electronic health records system; GERD, gastroesophageal reflux disease; HPV, human papillomavirus; HR, hazard ratio; IRR, incidence risk ratio; MRI, magnetic resonance imaging; NSAIDs, nonsteroidal anti-inflammatory drugs; OR, odds ratio.

a All findings are reported with the EHR nudge first before the usual care. For studies reporting ratios, the comparator is usual care.

Table 3. Overall Findings of Association Between EHR Nudges and Patient Outcomes in Primary Care

Measure type and concept	Citation(s)	Measures	Overall findingsa	
Patient-reported outcomes	
Hypertension	Kressin et al,67 2016	Antihypertensive adherence via self-reports	β = −0.34, P = .39	
Other (pain management)	Dhingra et al,74 2021	Pain intensity	Worst pain intensity scores: β = −0.50 (95% CI, −0.91 to −0.08); average pain intensity scores: β = −0.12 (95% CI, −0.57 to 0.33)	
Other (pain management)	Dhingra et al,74 2021	Pain interference	β = −1.33 (95% CI, −3.05 to 0.39)	
Patient adherence	
Routine health maintenance (cancer screening)	Guiriguet et al,75 2016	Colorectal cancer screening completed by patients	OR = 1.11 (95% CI, 1.02-1.22)	
Hypertension	Tamblyn et al,34 2018	Antihypertensive adherence via EHR data	Newly diagnosed uncomplicated hypertension: risk difference = 1.72 (95% CI, −6.91 to 10.36); established diagnoses: risk difference = 1.36 (95% CI, −1.55 to 4.28)	
Clinical outcomes	
Other (hyperlipidemia)	Gill et al,35 2009	At-goal lipid values	High-risk patients: OR = 1.17, P > .05; moderate-risk patients: OR = 0.29, P > .05; low-risk patients: OR = 1.74, P > .05	
Hypertension	Abdel-Kader et al,25 2011	Blood pressure <130/80	OR = 2.07 (95% CI, 0.83-5.17)	
Hypertension	Kressin et al,67 2016	Diastolic blood pressure	β = −0.70, P = .06	
Hypertension	Kressin et al,67 2016	Systolic blood pressure	β = −0.62, P = .05	
Other (atrial fibrillation)	Karlsson et al,46 2018	Having stroke, transient ischemic attack, or systemic thromboembolism	49 vs 47, P = .64	
Other (atrial fibrillation)	Karlsson et al,46 2018	Having bleeding	12 vs 16, P = .04	
Diabetes	Weiner et al,63 2020	Hypoglycemia	5% vs 5%, P = .99	
Abbreviations: EHR, electronic health records system; OR, odds ratio.

a All findings are reported with the EHR nudge first before the usual care. For studies reporting ratios, the comparator is usual care.

Study Quality Assessment Results

Overall, most studies (79.6%) were assessed to have a moderate risk of bias. For all cluster and parallel RCTs, there was limited measurement of clinicians’ adherence rate to the nudge intervention (ie, acknowledging the presence of the nudge by explicitly accepting or declining recommendations) in the final analysis. For crossover RCTs, there were concerns of short washout periods before administering the next phase. Across all RCTs, it was largely unclear whether outcome assessors were masked on study phases during the analysis and whether analytic methods used were predetermined prior to data collection. Some studies also experienced unexpected challenges in implementing or evaluating their EHR nudge as initially planned, such as hardware and software issues, concurrent quality improvement efforts, and changes in insurance coverage. Overall trends in study findings persisted even after removal of studies with high risk of bias (eTable 3 in Supplement 1).

Health Care Quality

Patient Safety

Eleven studies together assessed 12 unique patient safety measures with all targeting appropriate medication prescribing patterns.21,22,23,24,25,26,27,28,29,30,31 Medications examined included antibiotics, opiates, anticholinergics, hypnotics, benzodiazepines, and nonsteroidal anti-inflammatory drugs (NSAIDs). Overall, studies reported mixed associations. EHR nudges were associated with improvements for 4 of the 12 measures. Another 2 measures were associated with improvements but under specific contexts (ie, disease type, nudge characteristics). For example, EHR nudges were associated with improvements in antibiotic prescribing for pharyngitis (−3.0% difference; P = .003) but not for urinary tract infections (−0.4% difference; P = .64). Nudges that included accountable justification mechanisms were associated with more appropriate prescribing compared with nudges that suggested alternatives.

Effectiveness

Thirty-seven studies together assessed 48 unique effectiveness measures.24,25,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66 Clinical concepts included immunizations, wellness screening, cancer screening, and chronic care management. Overall, we observed mixed associations. EHR nudges were associated with improvements for 19 of the measures, such as increased rates of tetanus and pneumococcal immunizations; higher screening rates for hepatitis B and alcohol use; and providing guideline-concordant care for patients with heart failure or coronary artery disease. An additional 13 measures also were associated with improvements under specific contexts (eg, medical condition, patient risk level, time since diagnosis).

Patient-Centeredness

Two studies together assessed 3 unique patient-centeredness measures.55,67 Clinical concepts were limited to weight management and hypertension. Overall, EHR nudges were associated with improvements in counseling rates for exercise, diet, and blood pressure.

Timeliness

No study that we found assessed the association between EHR nudges and timeliness of care. Therefore, we were unable to assess for timeliness of care.

Efficiency

Four studies together assessed 4 unique efficiency measures in the context of reducing inappropriate or contraindicated laboratory tests and imaging.26,48,68,69 Overall, EHR nudges were not associated with improvements in any study.

Descriptive

Thirteen studies together assessed 38 unique descriptive measures.25,37,38,44,45,49,50,54,55,70,71,72,73 Overall, most studies found that EHR nudges were associated with improvements of various documentation and care planning patterns with fewer studies reporting null associations.

Patient Outcomes

Patient-Reported Outcomes

Two studies together assessed 3 unique patient-reported outcome measures. These included measures for pain symptoms and self-reported medication adherence.67,74 EHR nudges were associated with improvements in pain intensity and null associations with medication adherence and pain interference.

Patient Adherence

Two studies together assessed 2 unique patient adherence measures using objective data sources (eg, EHR data).34,75 One study found that EHR nudges were associated with improved rates of completed colorectal screening kits by patients, whereas the other study found null association for antihypertensive adherence based on EHR data.

Clinical Outcomes

Five studies together assessed 7 unique clinical outcomes, including blood pressure, blood glucose, and hyperlipidemia levels as well as risk levels for bleeding among patients with atrial fibrillation and stroke.25,35,46,63,67 Overall, EHR nudges had null associations for most measures. However, EHR nudges were associated with improvements in the risk of bleeding among patients with atrial fibrillation.

Implementation Facilitators and Barriers

Seven studies assessed clinicians’ viewpoints on implementation facilitators and barriers. Clinicians identified barriers, such as lack of knowledge or disagreement with guidelines and perceived errors in the programming of the nudge in the EHR (eTable 2 in Supplement 1).

Discussion

This systematic review aimed to summarize available RCTs evaluating how EHR nudges were associated with health care quality and outcomes in primary care settings. Overall, our findings suggest that EHR nudges may improve specific dimensions of health care quality (ie, descriptive, patient-centeredness). There were less consistent findings about how EHR nudges were associated with other health care quality measures (eg, effectiveness) and patient outcomes. To our knowledge, no study has attempted to assess how EHR nudges were associated with timeliness of care.

We found that EHR nudges may be associated with modest improvements in descriptive measures of health care quality. This suggests that EHR nudges may facilitate improvements in clinicians’ documentation patterns, such as completeness or accuracy. Interestingly, this pattern was less consistent for studies targeting clinicians’ ordering behaviors. This difference may be, in part, due to differences in nudge mechanisms and uptake of the nudge. For instance, templates were more common among studies that aimed to improve documentation patterns compared with ordering behaviors. Although most studies in our review did not measure clinician adherence to the nudge, other studies have found that primary care clinicians have increasingly adopted templates to support their documentation (29% in 2007-2008 vs more than 90% in 2018-2020).53,76 This suggests that higher clinician acceptance of a nudge or its fit in the workflow may be important factors that influence the effectiveness of the nudge. Other factors may include implementation contexts and approaches. Few studies in our review attempted to assess implementation facilitators and barriers of their nudges. Thus, further qualitative research on clinicians’ attitudes and perceptions on current EHR nudges are needed to inform efforts on how to optimize the delivery of EHR nudges in primary care contexts.

In comparison to descriptive and patient-centeredness measures, other types of health care quality measures (ie, patient safety, effectiveness, efficiency) showed relatively less consistent improvement. Although there were several studies that suggested associations between EHR nudges and improved ordering behaviors, there were also just as many studies reporting null associations. One reason for these different findings may be that the effect varies across clinical conditions. For instance, most studies examining EHR nudges related to substance use revealed a pattern of improved outcomes. Another reason for these differences in findings may stem from the presence of additional factors, such as patient preferences and insurance coverage, that may influence whether orders are placed. One dimension of health care quality that has been relatively understudied is timeliness in primary care, revealing an area in need of additional research.

Our review found that most of the studies assessing patient outcomes assessed them within a 1-year timeframe, suggesting that additional studies measuring sustainability of EHR nudges are needed. Consequently, our findings should be interpreted as short-term outcomes. Like other systematic reviews of nudge interventions in and outside of the EHR,10,11 a majority of the studies in our review only examined short-term outcomes. Thus, the long-term outcomes of EHR nudges remain unclear. Furthermore, most of the available studies examined metabolic diseases (hyperlipidemia, hypertension, diabetes). Meanwhile, to our knowledge, no RCT has been conducted that targets other diseases that are treated and managed in primary care, such as depression, anxiety, chronic obstructive pulmonary disease, and asthma. Consequently, additional research is needed on more diverse types of patient outcomes and with longer evaluation periods.

Our systematic review revealed that there were only 11 studies (20.4%) that used the EHR nudge to facilitate a decrease in a targeted behavior, such as reducing antibiotic prescribing for viral infections, imaging orders for low back pain, and reducing overprescribing of hypnotics and psychotropic drugs. Together, these accounted for 11 unique measures. Only 1 of the 11 measures (9.1%) reported that EHR nudges were associated with a decrease in the targeted behavior (a decrease in inappropriate prescribing across a variety of conditions). In contrast, EHR nudges were associated with improvements for 52 of the 95 measures (54.7%) where the EHR nudge was designed and implemented to facilitate an increase in a targeted behavior. Due to the low number of available studies involving EHR nudges to deimplement specific practices, it may be premature to conclude that EHR nudges may be better suited for increasing behavior frequency. Nonetheless, it remains unclear what the reasons are for this potential difference between nudges designed to decrease a behavior compared with those designed to increase a behavior. There exists an ongoing need for additional studies examining EHR nudges with the intent to decrease a behavior as well as qualitative research on how EHR nudges could help overcome inertia of reducing, restricting, removing, or replacing low-value clinical practices. These findings may provide additional insights as to whether EHR nudges should be designed differently to deimplement a particular practice or whether different concurrent interventions (eg, audit and feedback, insurance payment structures) need to be in place.

Limitations

This review has limitations. First, most studies targeted physicians or nurses. Furthermore, of the few studies that examined nonphysicians, combined samples of physicians and another clinician type were used. It remains unclear if different members of the care team respond differently to EHR nudges. Second, few studies were conducted in safety-net settings, such as federally qualified health centers, that may require different approaches to implementing EHR nudges due to lower available resources and less advanced EHR systems.77 Third, EHR nudges may be partly influenced by user interface design elements by EHR vendors. The most commonly used EHR was Epic, limiting generalizability to other EHR vendors. Fourth, most individual measures were assessed by only 1 study, precluding definitive conclusions on how EHR nudges are associated with performance in specific areas (eg, documenting allergies, blood pressure). Additionally, our quality appraisal revealed that none of the included studies controlled for adherence to evaluate EHR nudges. The observed outcomes of an intervention may be partly influenced by how often it is used, which makes the interpretation of findings challenging. For instance, it is unclear whether null results were due to low efficacy of the EHR nudge vs implementation issues of the nudge (eg, lack of awareness among clinicians of the nudge’s existence).

Based on these limitations, we discuss additional areas for future research. First, studies are needed that explicitly target other key members of the care team, such as social workers, pharmacists, and dietitians. Different care team members often have differing EHR workflows, which may influence how they respond to EHR nudges. Second, although outside of the scope of this review, patients can also be targeted by nudge interventions. Further studies are needed that assess how patients respond to nudges as well as studies that compare nudge effectiveness between patients and clinicians. Third, additional research is needed to assess potential facilitators and barriers that may be encountered when implementing EHR nudges in care settings. In a similar vein, there is a need for comparative evaluations of different EHR interfaces used to deliver the nudge to better understand what specific design elements facilitate the salience and effectiveness of the nudge. Additionally, because the number of studies that examined process measures outnumbered those that examined patient outcome measures, there persists a need for studies that assess how EHR nudges may influence downstream patient outcomes.

Conclusions

EHR-based nudge interventions are a promising strategy for improving evidence-based care delivery in primary care, a setting that is critical for supporting population health. Findings from this systematic review suggest that nudges may change specific dimensions of health care quality, such as descriptive measures (ie, documentation patterns). Their associations with other dimensions and clinical outcomes have been less consistent. To strengthen the evidence for nudge interventions, additional implementation studies are needed to understand the context of when nudge interventions work.

Supplement 1. eTable 1. Search Strategy

eTable 2. Study Characteristics

eTable 3. Study Quality Assessment Results

Supplement 2. Data Sharing Statement
==== Refs
References

1 Franks P, Fiscella K. Primary care physicians and specialists as personal physicians. health care expenditures and mortality experience. J Fam Pract. 1998;47 (2 ):105-109.9722797
2 Basu S, Berkowitz SA, Phillips RL, Bitton A, Landon BE, Phillips RS. Association of primary care physician supply with population mortality in the United States, 2005-2015. JAMA Intern Med. 2019;179 (4 ):506-514. doi:10.1001/jamainternmed.2018.7624 30776056
3 Baker R, Freeman GK, Haggerty JL, Bankart MJ, Nockels KH. Primary medical care continuity and patient mortality: a systematic review. Br J Gen Pract. 2020;70 (698 ):e600-e611. doi:10.3399/bjgp20X712289 32784220
4 Akincigil A, Matthews EB. National rates and patterns of depression screening in primary care: results from 2012 and 2013. Psychiatr Serv. 2017;68 (7 ):660-666. doi:10.1176/appi.ps.201600096 28196461
5 Pedersen RA, Petursson H, Hetlevik I. Stroke follow-up in primary care: a prospective cohort study on guideline adherence. BMC Fam Pract. 2018;19 (1 ):179. doi:10.1186/s12875-018-0872-9 30486788
6 Niven DJ, Mrklas KJ, Holodinsky JK, . Towards understanding the de-adoption of low-value clinical practices: a scoping review. BMC Med. 2015;13 :255. doi:10.1186/s12916-015-0488-z 26444862
7 Thaler RH, Sunstein CR. Nudge: Improving Decisions about Health, Wealth, and Happiness. Yale University Press; 2008.
8 Ledderer L, Kjær M, Madsen EK, Busch J, Fage-Butler A. Nudging in public health lifestyle interventions: a systematic literature review and metasynthesis. Health Educ Behav. 2020;47 (5 ):749-764. doi:10.1177/1090198120931788 32517522
9 Bucher T, Collins C, Rollo ME, . Nudging consumers towards healthier choices: a systematic review of positional influences on food choice. Br J Nutr. 2016;115 (12 ):2252-2263. doi:10.1017/S0007114516001653 27185414
10 Raban MZ, Gonzalez G, Nguyen AD, . Nudge interventions to reduce unnecessary antibiotic prescribing in primary care: a systematic review. BMJ Open. 2023;13 (1 ):e062688. doi:10.1136/bmjopen-2022-062688 36657758
11 Raban MZ, Gates PJ, Gamboa S, Gonzalez G, Westbrook JI. Effectiveness of non-interruptive nudge interventions in electronic health records to improve the delivery of care in hospitals: a systematic review. J Am Med Inform Assoc. 2023;30 (7 ):1313-1322. doi:10.1093/jamia/ocad083 37187160
12 Van Den Bulck S, Spitaels D, Vaes B, Goderis G, Hermens R, Vankrunkelsven P. The effect of electronic audits and feedback in primary care and factors that contribute to their effectiveness: a systematic review. Int J Qual Health Care. 2020;32 (10 ):708-720. doi:10.1093/intqhc/mzaa128 33057648
13 Henry J, Pylypchuk Y, Searcy T, Patel V. Adoption of electronic health record systems among U.S. non-federal acute care hospitals: 2008-2015. Accessed February 18, 2019. https://dashboard.healthit.gov/evaluations/data-briefs/non-federal-acute-care-hospital-ehr-adoption-2008-2015.php
14 Wolf A, Sant’Anna A, Vilhelmsson A. Using nudges to promote clinical decision making of healthcare professionals: a scoping review. Prev Med. 2022;164 :107320. doi:10.1016/j.ypmed.2022.107320 36283484
15 Moher D, Liberati A, Tetzlaff J, Altman DG; PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ. 2009;339 :b2535. doi:10.1136/bmj.b2535 19622551
16 Last BS, Buttenheim AM, Timon CE, Mitra N, Beidas RS. Systematic review of clinician-directed nudges in healthcare contexts. BMJ Open. 2021;11 (7 ):e048801. doi:10.1136/bmjopen-2021-048801 34253672
17 Shamseer L, Shea B, Hutton B, Moher D. Reporting and appraisal of systematic reviews. In: Egger M, Higgins JPT, Smith GD, eds. Systematic Reviews in Health Research: Meta-Analysis in Context. John Wiley & Sons; 2022:109-128. doi:10.1002/9781119099369.ch7
18 McKenzie JE, Brennan SE. Synthesizing and presenting findings using other methods. Published 2023. Accessed May 10, 2024. https://training.cochrane.org/handbook/current/chapter-12#section-12-1
19 Agency for Healthcare Research and Quality. Examples of physician quality measures for consumers. Accessed December 20, 2023. https://www.ahrq.gov/talkingquality/measures/setting/physician/examples.html
20 Finley CR, Chan DS, Garrison S, . What are the most common conditions in primary care? systematic review. Can Fam Physician. 2018;64 (11 ):832-840.30429181
21 Campbell NL, Holden RJ, Tang Q, . Multicomponent behavioral intervention to reduce exposure to anticholinergics in primary care older adults. J Am Geriatr Soc. 2021;69 (6 ):1490-1499. doi:10.1111/jgs.17121 33772749
22 Kraemer KL, Althouse AD, Salay M, . Effect of different interventions to help primary care clinicians avoid unsafe opioid prescribing in opioid-naive patients with acute noncancer pain: a cluster randomized clinical trial. JAMA Health Forum. 2022;3 (7 ):e222263. doi:10.1001/jamahealthforum.2022.2263 35983579
23 Tamblyn R, Eguale T, Buckeridge DL, . The effectiveness of a new generation of computerized drug alerts in reducing the risk of injury from drug side effects: a cluster randomized trial. J Am Med Inform Assoc. 2012;19 (4 ):635-643. doi:10.1136/amiajnl-2011-000609 22246963
24 Gill JM, Mainous AG III, Koopman RJ, . Impact of EHR-based clinical decision support on adherence to guidelines for patients on NSAIDs: a randomized controlled trial. Ann Fam Med. 2011;9 (1 ):22-30. doi:10.1370/afm.1172 21242557
25 Abdel-Kader K, Fischer GS, Li J, Moore CG, Hess R, Unruh ML. Automated clinical reminders for primary care providers in the care of CKD: a small cluster-randomized controlled trial. Am J Kidney Dis. 2011;58 (6 ):894-902. doi:10.1053/j.ajkd.2011.08.028 21982456
26 Flottorp S, Oxman AD, Håvelsrud K, Treweek S, Herrin J. Cluster randomised controlled trial of tailored interventions to improve the management of urinary tract infections in women and sore throat. BMJ. 2002;325 (7360 ):367. doi:10.1136/bmj.325.7360.367 12183309
27 Høye S, Gjelstad S, Lindbæk M. Effects on antibiotic dispensing rates of interventions to promote delayed prescribing for respiratory tract infections in primary care. Br J Gen Pract. 2013;63 (616 ):e777-e786. doi:10.3399/bjgp13X674468 24267861
28 Meeker D, Linder JA, Fox CR, . Effect of behavioral interventions on inappropriate antibiotic prescribing among primary care practices: a randomized clinical trial. JAMA. 2016;315 (6 ):562-570. doi:10.1001/jama.2016.0275 26864410
29 Gulliford MC, Prevost AT, Charlton J, . Effectiveness and safety of electronically delivered prescribing feedback and decision support on antibiotic use for respiratory illness in primary care: REDUCE cluster randomised trial. BMJ. 2019;364 :l236. doi:10.1136/bmj.l236 30755451
30 Tamblyn R, Huang A, Perreault R, . The medical office of the 21st century (MOXXI): effectiveness of computerized decision-making support in reducing inappropriate prescribing in primary care. CMAJ. 2003;169 (6 ):549-556.12975221
31 Fortuna RJ, Zhang F, Ross-Degnan D, . Reducing the prescribing of heavily marketed medications: a randomized controlled trial. J Gen Intern Med. 2009;24 (8 ):897-903. doi:10.1007/s11606-009-1013-x 19475459
32 Sequist TD, Gandhi TK, Karson AS, . A randomized trial of electronic clinical reminders to improve quality of care for diabetes and coronary artery disease. J Am Med Inform Assoc. 2005;12 (4 ):431-437. doi:10.1197/jamia.M1788 15802479
33 Sequist TD, Holliday AM, Orav EJ, Bates DW, Denker BM. Physician and patient tools to improve chronic kidney disease care. Am J Manag Care. 2018;24 (4 ):e107-e114.29668213
34 Tamblyn R, Winslade N, Qian CJ, Moraga T, Huang A. What is in your wallet? a cluster randomized trial of the effects of showing comparative patient out-of-pocket costs on primary care prescribing for uncomplicated hypertension. Implement Sci. 2018;13 (1 ):7. doi:10.1186/s13012-017-0701-x 29321043
35 Gill JM, Chen YX, Glutting JJ, Diamond JJ, Lieberman MI. Impact of decision support in electronic medical records on lipid management in primary care. Popul Health Manag. 2009;12 (5 ):221-226. doi:10.1089/pop.2009.0003 19848563
36 Adusumalli S, Westover JE, Jacoby DS, . Effect of passive choice and active choice interventions in the electronic health record to cardiologists on statin prescribing: a cluster randomized clinical trial. JAMA Cardiol. 2021;6 (1 ):40-48.33031534
37 Frank O, Litt J, Beilby J. Opportunistic electronic reminders. improving performance of preventive care in general practice. Aust Fam Physician. 2004;33 (1-2 ):87-90.14988972
38 Loo TS, Davis RB, Lipsitz LA, . Electronic medical record reminders and panel management to improve primary care of elderly patients. Arch Intern Med. 2011;171 (17 ):1552-1558. doi:10.1001/archinternmed.2011.394 21949163
39 Fiks AG, Hunter KF, Localio AR, . Impact of electronic health record-based alerts on influenza vaccination for children with asthma. Pediatrics. 2009;124 (1 ):159-169. doi:10.1542/peds.2008-2823 19564296
40 Szilagyi PG, Serwint JR, Humiston SG, . Effect of provider prompts on adolescent immunization rates: a randomized trial. Acad Pediatr. 2015;15 (2 ):149-157. doi:10.1016/j.acap.2014.10.006 25748976
41 Fiks AG, Grundmeier RW, Mayne S, . Effectiveness of decision support for families, clinicians, or both on HPV vaccine receipt. Pediatrics. 2013;131 (6 ):1114-1124. doi:10.1542/peds.2012-3122 23650297
42 Stockwell MS, Catallozzi M, Camargo S, . Registry-linked electronic influenza vaccine provider reminders: a cluster-crossover trial. Pediatrics. 2015;135 (1 ):e75-e82. doi:10.1542/peds.2014-2616 25548331
43 Stephens AB, Wynn CS, Hofstetter AM, . Effect of electronic health record reminders for routine immunizations and immunizations needed for chronic medical conditions. Appl Clin Inform. 2021;12 (5 ):1101-1109. doi:10.1055/s-0041-1739516 34911126
44 Co JPT, Johnson SA, Poon EG, . Electronic health record decision support and quality of care for children with ADHD. Pediatrics. 2010;126 (2 ):239-246. doi:10.1542/peds.2009-0710 20643719
45 Bell LM, Grundmeier R, Localio R, . Electronic health record-based decision support to improve asthma care: a cluster-randomized trial. Pediatrics. 2010;125 (4 ):e770-e777. doi:10.1542/peds.2009-1385 20231191
46 Karlsson LO, Nilsson S, Bång M, Nilsson L, Charitakis E, Janzon M. A clinical decision support tool for improving adherence to guidelines on anticoagulant therapy in patients with atrial fibrillation at risk of stroke: A cluster-randomized trial in a Swedish primary care setting (the CDS-AF study). PLoS Med. 2018;15 (3 ):e1002528. doi:10.1371/journal.pmed.1002528 29534063
47 McKie PM, Kor DJ, Cook DA, . Computerized advisory decision support for cardiovascular diseases in primary care: a cluster randomized trial. Am J Med. 2020;133 (6 ):750-756.e2. doi:10.1016/j.amjmed.2019.10.039 31862329
48 Sequist TD, Morong SM, Marston A. Electronic risk alerts to improve primary care management of chest pain: a randomized, controlled trial. J Gen Intern Med. 2012;27 :438-444. doi:10.1007/s11606-011-1911-6 21993999
49 Feder G, Davies RA, Baird K, . Identification and Referral to Improve Safety (IRIS) of women experiencing domestic violence with a primary care training and support programme: a cluster randomised controlled trial. Lancet. 2011;378 (9805 ):1788-1795. doi:10.1016/S0140-6736(11)61179-3 22000683
50 Player MS, Gill JM, Mainous AG III, . An electronic medical record-based intervention to improve quality of care for gastro-esophageal reflux disease (GERD) and atypical presentations of GERD. Qual Prim Care. 2010;18 (4 ):223-229.20836938
51 Palen TE, Raebel M, Lyons E, Magid DM. Evaluation of laboratory monitoring alerts within a computerized physician order entry system for medication orders. Am J Manag Care. 2006;12 (7 ):389-395.16834525
52 Feldstein A, Elmer PJ, Smith DH, . Electronic medical record reminder improves osteoporosis management after a fracture: a randomized, controlled trial. J Am Geriatr Soc. 2006;54 (3 ):450-457. doi:10.1111/j.1532-5415.2005.00618.x 16551312
53 Linder JA, Schnipper JL, Middleton B. Method of electronic health record documentation and quality of primary care. J Am Med Inform Assoc. 2012;19 (6 ):1019-1024. doi:10.1136/amiajnl-2011-000788 22610494
54 Lee AK, Bobb JF, Richards JE, . Integrating alcohol-related prevention and treatment into primary care: a cluster randomized implementation trial. JAMA Intern Med. 2023;183 (4 ):319-328. doi:10.1001/jamainternmed.2022.7083 36848119
55 Schriefer SP, Landis SE, Turbow DJ, Patch SC. Effect of a computerized body mass index prompt on diagnosis and treatment of adult obesity. Fam Med. 2009;41 (7 ):502-507.19582636
56 Hsu L, Bowlus CL, Stewart SL, . Electronic messages increase hepatitis B screening in at-risk Asian American patients: a randomized, controlled trial. Dig Dis Sci. 2013;58 (3 ):807-814. doi:10.1007/s10620-012-2396-9 23073671
57 Chak E, Taefi A, Li CS, . Electronic medical alerts increase screening for chronic hepatitis B: a randomized, double-blind, controlled trial. Cancer Epidemiol Biomarkers Prev. 2018;27 (11 ):1352-1357. doi:10.1158/1055-9965.EPI-18-0448 30089680
58 Chak E, Li CS, Chen MS Jr, MacDonald S, Bowlus C. Electronic health record alerts enhance mass screening for chronic hepatitis B. Sci Rep. 2020;10 (1 ):19153. doi:10.1038/s41598-020-75842-8 33154429
59 Federman AD, Kil N, Kannry J, . An electronic health record-based intervention to promote hepatitis C virus testing among adults born between 1945 and 1965: a cluster-randomized trial. Med Care. 2017;55 (6 ):590-597. doi:10.1097/MLR.0000000000000715 28288075
60 van Wyk JT, van Wijk MAM, Sturkenboom MCJM, Mosseveld M, Moorman PW, van der Lei J. Electronic alerts versus on-demand decision support to improve dyslipidemia treatment: a cluster randomized controlled trial. Circulation. 2008;117 (3 ):371-378. doi:10.1161/CIRCULATIONAHA.107.697201 18172036
61 O’Connor PJ, Sperl-Hillen JM, Rush WA, . Impact of electronic health record clinical decision support on diabetes care: a randomized trial. Ann Fam Med. 2011;9 (1 ):12-21. doi:10.1370/afm.1196 21242556
62 Zera CA, Bates DW, Stuebe AM, Ecker JL, Seely EW. Diabetes screening reminder for women with prior gestational diabetes: a randomized controlled trial. Obstet Gynecol. 2015;126 (1 ):109-114. doi:10.1097/AOG.0000000000000883 26241263
63 Weiner M, Cummins J, Raji A, . A randomized study on the usefulness of an electronic outpatient hypoglycemia risk calculator for clinicians of patients with diabetes in a safety-net institution. Curr Med Res Opin. 2020;36 (4 ):583-593. doi:10.1080/03007995.2020.1717451 31951747
64 McDowell I, Newell C, Rosser W. A randomized trial of computerized reminders for blood pressure screening in primary care. Med Care. 1989;27 (3 ):297-305. doi:10.1097/00005650-198903000-00008 2494397
65 Kharbanda EO, Asche SE, Sinaiko A, . Evaluation of an electronic clinical decision support tool for incident elevated BP in adolescents. Acad Pediatr. 2018;18 (1 ):43-50. doi:10.1016/j.acap.2017.07.004 28723587
66 Linder JA, Rigotti NA, Schneider LI, Kelley JHK, Brawarsky P, Haas JS. An electronic health record-based intervention to improve tobacco treatment in primary care: a cluster-randomized controlled trial. Arch Intern Med. 2009;169 (8 ):781-787. doi:10.1001/archinternmed.2009.53 19398690
67 Kressin NR, Long JA, Glickman ME, . A brief, multifaceted, generic intervention to improve blood pressure control and reduce disparities had little effect. Ethn Dis. 2016;26 (1 ):27-36. doi:10.18865/ed.26.1.27 26843793
68 Zafar HM, Ip IK, Mills AM, Raja AS, Langlotz CP, Khorasani R. Effect of clinical decision support-generated report cards versus real-time alerts on primary care provider guideline adherence for low back pain outpatient lumbar spine MRI orders. AJR Am J Roentgenol. 2019;212 (2 ):386-394. doi:10.2214/AJR.18.19780 30476451
69 Lo HG, Matheny ME, Seger DL, Bates DW, Gandhi TK. Impact of non-interruptive medication laboratory monitoring alerts in ambulatory care. J Am Med Inform Assoc. 2009;16 (1 ):66-71. doi:10.1197/jamia.M2687 18952945
70 Tang JW, Kushner RF, Cameron KA, Hicks B, Cooper AJ, Baker DW. Electronic tools to assist with identification and counseling for overweight patients: a randomized controlled trial. J Gen Intern Med. 2012;27 (8 ):933-939. doi:10.1007/s11606-012-2022-8 22402982
71 Wright A, Pang J, Feblowitz JC, . Improving completeness of electronic problem lists through clinical decision support: a randomized, controlled trial. J Am Med Inform Assoc. 2012;19 (4 ):555-561. doi:10.1136/amiajnl-2011-000521 22215056
72 Sigel EJ, Arredondo Mattson S, Runyan CW. Utilizing the electronic medical record to enhance health care provider delivery of messages about the safe storage of firearms. Clin Pediatr (Phila). 2023;62 (8 ):894-900. doi:10.1177/00099228221146998 36694403
73 Weiner SJ, Schwartz A, Weaver F, . Effect of electronic health record clinical decision support on contextualization of care: a randomized clinical trial. JAMA Netw Open. 2022;5 (10 ):e2238231. doi:10.1001/jamanetworkopen.2022.38231 36279133
74 Dhingra L, Schiller R, Teets R, . Pain management in primary care: a randomized controlled trial of a computerized decision support tool. Am J Med. 2021;134 (12 ):1546-1554. doi:10.1016/j.amjmed.2021.07.014 34411523
75 Guiriguet C, Muñoz-Ortiz L, Burón A, . Alerts in electronic medical records to promote a colorectal cancer screening programme: a cluster randomised controlled trial in primary care. Br J Gen Pract. 2016;66 (648 ):e483-e490. doi:10.3399/bjgp16X685657 27266861
76 Rule A, Hribar MR. Frequent but fragmented: use of note templates to document outpatient visits at an academic health center. J Am Med Inform Assoc. 2021;29 (1 ):137-141. doi:10.1093/jamia/ocab230 34664655
77 van Zyl C, Badenhorst M, Hanekom S, Heine M. Unravelling ‘low-resource settings’: a systematic scoping review with qualitative content analysis. BMJ Glob Health. 2021;6 (6 ):e005190. doi:10.1136/bmjgh-2021-005190 34083239
