
==== Front
J Telemed Telecare
J Telemed Telecare
JTT
spjtt
Journal of Telemedicine and Telecare
1357-633X
1758-1109
SAGE Publications Sage UK: London, England

36567431
10.1177/1357633X221139892
10.1177_1357633X221139892
RESEARCH/Original Articles
Telehealth-guided provider-to-provider communication to improve rural health: A systematic review
https://orcid.org/0000-0002-9100-8678
Totten Annette M
Womack Dana M
Griffin Jessica C
McDonagh Marian S
Davis-O’Reilly Cynthia
Blazina Ian
Grusing Sara
Elder Nancy
Oregon Health & Science University, Portland, OR, USA
Annette M Totten, Pacific Northwest Evidence-based Practice Center, Oregon Health & Science University, 3181 S.W. Sam Jackson Park Road, Mail code: BICC, Portland, OR 97239, USA. Email: totten@ohsu.edu
25 12 2022
9 2024
30 8 12091229
© The Author(s) 2022
2022
SAGE Publications
https://creativecommons.org/licenses/by-nc/4.0/ This article is distributed under the terms of the Creative Commons Attribution-NonCommercial 4.0 License (https://creativecommons.org/licenses/by-nc/4.0/) which permits non-commercial use, reproduction and distribution of the work without further permission provided the original work is attributed as specified on the SAGE and Open Access page (https://us.sagepub.com/en-us/nam/open-access-at-sage).
Introduction

Telehealth may address healthcare disparities for rural populations. This systematic review assesses the use, effectiveness, and implementation of telehealth-supported provider-to-provider collaboration to improve rural healthcare.

Methods

We searched Ovid MEDLINE®, CINAHL®, EMBASE, and Cochrane CENTRAL from 1 January 2010 to 12 October 2021 for trials and observational studies of rural provider-to-provider telehealth. Abstracts and full text were dual-reviewed. We assessed the risk of bias for individual studies and strength of evidence for studies with similar outcomes.

Results

Seven studies of rural uptake of provider-to-provider telehealth documented increases over time but variability across geographic regions. In 97 effectiveness studies, outcomes were similar with rural provider-to-provider telehealth versus without for inpatient consultations, neonatal care, outpatient depression and diabetes, and emergency care. Better or similar results were reported for changes in rural clinician behavior, knowledge, confidence, and self-efficacy. Evidence was insufficient for other clinical uses and outcomes. Sixty-seven (67) evaluation and qualitative studies identified barriers and facilitators to implementing rural provider-to-provider telehealth. Success was linked to well-functioning technology, sufficient resources, and adequate payment. Barriers included lack of understanding of rural context and resources. Methodologic weaknesses of studies included less rigorous study designs and small samples.

Discussion

Rural provider-to-provider telehealth produces similar or better results versus care without telehealth. Barriers to rural provider-to-provider telehealth implementation are common to practice change but include some specific to rural adaptation and adoption. Evidence gaps are partially due to studies that do not address differences in the groups compared or do not include sufficient sample sizes.

Rural health
remote consultations
telehealth
telemedicine
Extension for Community Healthcare Outcomes (ECHO)
systematic review
video education
Agency for Healthcare Research and Quality https://doi.org/10.13039/100000133 Contract No. 75Q80120D00006/Task Order 75Q80120F32 typesetterts19
==== Body
pmcNumerous studies have documented health disparities for people living in rural areas of the United States.1–4 Rural-urban differences in access and quality of care contribute to higher mortality4–7 and morbidity from conditions including substance/opioid misuse,8,9 chronic illnesses,10–13 and human immunodeficiency virus/human papillomavirus and other infectious diseases.14,15 Causes of rural-urban differences are varied and related to complex macro and micro sociologic-demographic forces 16 and economic trends;17,18 nevertheless, the need for solutions is urgent. Both the coronavirus disease 2019 (COVID-19) pandemic and acknowledgement of structural racism have focused attention on broad inequities in healthcare in the United States, disproportionate burden on rural populations, and the resulting harms to underserved populations.

Telehealth utilizes technology to provide health care across time and/or distance 19 , encompassing a wide range of interventions, modes, and clinical functions. Telehealth is frequently proposed to address limited access and health disparities20,21 and there is a sizable body of research, including systematic reviews and reviews of reviews,22–29 suggesting telehealth can be effective. However, implementation and spread have been slow,30–32 with steady, but small increases in telehealth use before the COVID-19 pandemic. 33 Rapid increases in the use of telehealth to limit the risk of exposure and transmission during the COVID-19 pandemic may continue to fuel wider adoption and more long-term use.34,35 Growth in telehealth has focused attention on the need to address the digital divide, promote widespread access to telehealth, 36 and assure telehealth promotes equitable healthcare. 37

Rural provider-to-provider telehealth (RPPT) is a subset of telehealth interventions focused on supporting health care providers that treat rural patients and populations through consultations and collaborative patient care as well as mentoring and education.38–40 We conducted this systematic review to identify and synthesize the available research on whether telehealth-supported provider-to-provider communication and collaboration can contribute to improving the health and well-being of rural residents and communities. This review was conducted to inform a National Institutes of Health (NIH) Office of Disease Prevention (ODP) Pathways to Prevention (P2P) workshop, Improving Rural Health through Telehealth-Guided Provider-to-Provider Communication, held on October 12–14, 2021). The panel report and recommendations from this workshop are also published in this journal. 41

Methods

We conducted this review using methods outlined in the Agency for Healthcare Research and Quality (AHRQ) Methods Guide for Effectiveness and Comparative Effectiveness Reviews (hereafter “AHRQ Methods Guide”). 42 Our protocol is available on the AHRQ website (https://effectivehealthcare.ahrq.gov/products/rural-telehealth/protocol) and registered with PROSPERO (registration no. CRD42021233545). Detailed methods and additional information in the full report are available at https://effectivehealthcare.ahrq.gov/products/rural-telehealth/research.

Key questions

This review was designed to answer the following four questions:

Key Question 1: What is the uptake of different types of provider-to-provider telehealth in rural areas?

Key Question 2: What is the effectiveness of provider-to-provider telehealth for rural patients?

Key Question 3: What strategies are effective and what are the barriers and facilitators to implementation and sustainability of provider-to-provider telehealth in rural areas?

Key Question 4: What are the methodological weaknesses of the included studies of provider-to-provider telehealth for rural patients and what improvements in study design (e.g. focus on relevant comparisons and outcomes) might increase the impact of future research?

Data sources and searches

We developed strategies with an expert librarian, had these peer-reviewed by a second librarian, and searched Ovid MEDLINE®, CINAHL®, EMBASE, and Cochrane CENTRAL for studies published between 1 January 2010 and 13 October 2021 (Search strategies included in online Supplement 1). Studies on telehealth have been published before January 2010. However, this start date was selected in consultation with the key informants and NIH ODP stakeholders and corresponded to their request to focus the review on studies using current technology that would be most relevant to future policy and decision-making.

Study selection

Two team members reviewed abstracts and full-text articles to identify studies that met our inclusion criteria (Supplemental Table 1), and any disagreements were resolved through discussion and consensus. To be included, studies had to be of rural patients or populations, or report results separately for rural patients or populations. Trials and observational studies that provided data on use, effectiveness, and facilitators or barriers to implementation of RPPT for inpatient, outpatient, or emergency care and studies of telehealth used for training health care providers who care for rural populations were included. Descriptive articles and studies that reported only data after telehealth implementation (no comparison group) were excluded. Studies had to be conducted in the United States or countries identified as having very high or high human development levels by the United Nations Human Development Report. 43

We defined RPPT as any form of interactive support using telecommunications technology provided to health care professionals who care for rural patients and populations. This included: video, audio, or digital remote consultations across space (e.g. video) or time (e.g. store and forward) as well as remote mentoring or education including rounds or case reviews. We excluded (a) telehealth for patient encounters, (b) remote patient monitoring if the data were transmitted only from a patient to a single provider and were not used as part of provider-to-provider consultations, and (c) communications that were limited to referring a patient to another provider for care and with no other interaction.

To assess use of RPPT we included studies with indicators or measures of uptake such as rates of use as outcomes. For effectiveness we included studies reporting clinical outcomes (e.g. mortality and morbidity) and intermediate outcomes (e.g. treatment, satisfaction, health care services utilization, and economic outcomes) We excluded studies if the outcomes were projections or the results of simulations.

Data extraction and risk of bias assessment

One person extracted data on the study design, setting, patient population, telehealth intervention, participating providers, and outcomes from the included studies. If a study was reported in multiple publications, the study was included once in our counts, but information about the study and data on outcomes were abstracted from all available publications. When extracting data from effectiveness studies, we also extracted methodological weaknesses listed by the study authors. Extracted data were verified for completeness and accuracy by a second person. Two team members independently assessed the risk of bias for individual studies using criteria appropriate for the study design and consistent with the approach in the AHRQ Methods Guide. 42 Studies were rated as low, moderate, or high risk of bias.

Data synthesis and analysis

We summarized surveys and analyses that provided counts or estimates to assess use of RPPT. To address the effectiveness of RPPT, we grouped studies by health care setting, then clinical indication, and summarized findings based on the direction and magnitude of effect. We also assessed the strength of evidence (SOE) for outcomes prioritized by the ODP working group, following the approach described in the AHRQ Methods Guide. 42 Our assessment of implementation summarized the few identified studies that compared approaches, then summarized barriers and facilitators cited in included studies. We classified barriers and facilitators using an existing framework, the Consolidated Framework for Implementation Research (CFIR). 44 We then summarized how frequently barriers and facilitators were reported across all identified studies. Our synthesis of the methodologic weaknesses of the effectiveness studies consisted of summarizing the weaknesses identified by the study authors as well as those identified by the review team.

Results

Our search produced 6329 citations, after triage of abstracts we reviewed 1024 full-text articles and included 166 studies reported in 179 publications (Figure 1). Seven studies met the inclusion criterial for Key Question 1 and provided data on use of provider-to-provider telehealth is rural areas (Key Question 1), 97 studies in 106 publications provided data on effectiveness of RPPT(Key Question 2) and were also used to identify methodological weaknesses in the body of evidence (Key Question 4), and 67 studies in 71 publications were used to summarize barriers and facilitators to implementation of RPPT (Key Question 3). Detailed descriptions of the included studies, as well as the data extracted, are included in the appendix to the full report at https://effectivehealthcare.ahrq.gov/products/rural-telehealth/research.

Figure 1. Literature flow diagram. Key Question 1: What is the uptake of different types of provider-to-provider telehealth in rural areas? Key Question 2: What is the effectiveness of provider-to-provider telehealth for rural patients? How does provider-to-provider telehealth affect outcomes for patients and populations? How does provider-to-provider telehealth affect outcomes for healthcare providers? How does provider-to-provider telehealth affect outcomes for private and public payers? Key Question 3: What strategies are effective and what are the barriers and facilitators to implementation and sustainability of provider-to-provider telehealth in rural areas? Key Question 4: What are the methodological weaknesses of the included studies of provider-to-provider telehealth for rural patients and what improvements in study design (e.g. focus on relevant comparisons and outcomes) might increase the impact of future research

Use of provider-to-provider telehealth for rural populations

We did not identify comprehensive data on the use of RPPT in the United States. However, we identified seven studies that reported on national or regional use of provider-to-provider telehealth for emergency or inpatient care and included data specific to rural areas.45–51 These studies were published since 2015 but included data collected prior to the COVID-19 pandemic.

Most studies reported lower use in rural than in urban areas, but studies that examined changes over time reported increases in both rural and urban hospitals. A study conducted in 2018 found that urban hospitals were twice as likely as rural hospitals to use telehealth for intensive care unit (ICU) and stroke care. 46 A study of telehealth use for heart attack and stroke assessment reported that rural hospital use increased from 6% in 2012 to 16% in 2017, though this was still lower than urban hospitals. 45 One study that reported higher use in rural than in urban areas focused on rates of telestroke use (8.6 vs. 2.3 per 1000), using Medicare fee-for-service claims to compare rates at the patient, rather than hospital, level. 51 Two studies of telehealth use for any reason by emergency departments (EDs) in New England in 2014 50 and nationwide in 2016 49 reported use by 49% and 54% of rural EDs, though this differed by region, with less use in the southern United States. Additionally, studies of telepsychiatry use in rural EDs in 2016 47 and by rural mental health facilities from 2010 to 2107 48 reported less than 30% of those surveyed used telehealth.

Effectiveness of provider-to-provider telehealth for rural patients

We assessed effectiveness and SOE after grouping studies by setting and clinical topic. Table 1 presents outcomes for patients, providers, and payers. Findings are summarized in the text below by setting and in terms of SOE by outcome in Supplemental Table 2.

Table 1. Effectiveness in inpatient, outpatient and emergency care.

Clinical topic (number of studies)	Patient outcomes (reference) a	Provider outcomes a	Payer outcomes a	
Inpatient care	
Multiple conditions (4)	∼ Mortality in hospital 52,53
∼ Transfers 52
∼ Length of stay 52,53
∼ Readmission 53	∼ Drug prescribing 54
+ Communication ratings 52	+ Hospital revenue 55
+ Professional billing revenue 55	
Infectious disease (3)	+ Mortality 56
∼ Transfers 56
– Length of stay 56 b
∼ Length of stay 57
∼ 30-day readmission 56	+ Provider satisfaction 56
+ Improved antimicrobial use or infection rate 57,58
∼ Antibiotic use 57
+ Appropriate prescribing and adherence to guidelines 57	None reported	
Stroke (1)	∼ Length of stay 59	None reported	+ Cost 59	
Spinal fracture (1)	+ Length of stay 60	+ Knowledge, skills, confidence 60	None reported	
Mental and behavioral disorders (1)	+ Transfers 61	None reported	None reported	
Neonates (5)	+ Transfers 62,63
∼ Length of stay 64,65
∼ Enteral feeding 64,65
∼ Ventilation/oxygen 64,65
∼ Proportion of deliveries at community hospitals 66
+ Statewide infant mortality to 1 year and telehealth hospital death before discharge 62
∼ Morbidity 62	None reported	None reported	
ICU (2)	∼ Mortality in high dependency unit 67
∼ Mortality in hospital 67
∼ Mortality total 67
+ Mortality 90-day 68
+ Transfers 67	None reported	None reported	
Outpatient Care	
Depression (3)	M Response (varies by time point [6, 12, 18 months]) 69–75
M Remission (varies by time point [6, 12, 18 months]) 69–75
∼ Depression-free days 69–75
∼ Depression symptom score (BDI) 3 and 6 months 69–75
M Quality of Life (varies by time point [6, 12, 18 months] and scale [Quality of Well-being Scale and SF-36 Mental]) 69–75
∼ Treatment adherence 3 months,+ 6 months 69–75
+ Medication adherence 6 and 12 months 69–75	None reported	∼ Utilization: overall,+ outpatient only69,70,73,74
∼ Total costs 70,73
– Adjusted total cost 72
+ Incremental cost/ depression-free days 72
– Depression-related primary and mental health costs 70,73	
PTSD (1)	+ Prescribed any medication for PTSD 76
∼ Adherence to medication regimen >80% 76
∼ Any psychiatric encounter at 12 months 76
+ Mean number of CPT sessions at 12 months 76
M Quality of Life (varies by scale [Quality of Well-being Scale and SF-36 Mental and Physical scales]) 76	None reported	– Total outpatient costs 76,77	
ADHD (1)	∼ Teacher ratings (VADRS) 78
∼ Caregiver ratings (VADRS-role performance and CIS-P scale) 78	None reported	None reported	
Diabetes (4)	+ A1c 79–82 and self-monitoring of blood glucose 82
+ Diabetes Self-Management Education score 79,82
∼ Total cholesterol 79,82 and systolic blood pressure 81	None reported	None reported	
Diabetic neuropathy (1)	+ Screening rate 83	+ Screening referral or reminder 83	None reported	
Hepatitis C (1)	∼ Sustained virologic response 84
+ Completion of therapy 84
∼ Mean weeks of therapy 84
+ Mean face-to-face visits 84
+ Anemia and withdrawal due to adverse events 84	None reported	None reported	
Rheumatology (2)	+ Distance to visit 85
∼ Quality of life, disease-related function and visit satisfaction 86	None reported	+ Cost of visit 85	
Oncology (2)	∼ Receipt of radiotherapy 87	None reported	+ Total costs 88	
Echocardiography (1)	+ Total process time 89
+ Time to specialist consultation 89	None reported	None reported	
Hemodialysis (1)	+ Amount of urea removed 90
∼ Other clinical measures 90	None reported	None reported	
Fracture clinic (1)	None reported	None reported	+ Cost compared to transfer or sending a specialist 91	
Dementia care (1)	∼ Yearly change in MMSE: all patients 92
+ Yearly change in MMSE: patients with baseline MMSE 15–30 92	None reported	None reported	
Ulcer care (1)	+ Time from referral to appointment 93
+ Leg ulcer healing time 93	None reported	∼ Cost per patient 93	
Pharmacy (2)	∼ A1c and systolic BP: patients with diabetes or HTN 94
+ A1c and systolic BP: patients with diabetes and HTN 94	+ Guideline adherence 95	None reported	
Remote consultation (3)	+ Hospital visits 96
+ Patient function
+ Medication adherence 97
+ Health-related quality of life 97	∼ Overall referrals or to eConsult specialties 98	None reported	
Dermatology (2)	∼ Clinical course 99	None reported	M Cost per participant 100	
Endoscopy (1)	None reported	None reported	∼ Costs 101	
Blood pressure (1)	+ Blood pressure 102
+ Stroke recurrence 102	None reported	None reported	
Palliative care (1)	∼ Patient symptoms and function
+ Nurse or general practitioner visits, hospital admissions	None reported	None reported	
Ultrasound for pregnancy (1)	+ Increase in comprehensive ultrasound for all and high-risk pregnancies 103	None reported	None reported	
Emergency care	
Stroke
(10)
TeleED
Video-conference	∼ In-hospital, 30-day and 90-day mortality 51,104–107
+ 30-day all-cause mortality, super rural patients 51
∼ Symptom onset to tPA time 51,104–111
+ tPA within 3 h of symptom onset 51,104–106,108–111
∼ 90-day modified Rankin Scale 104,106
∼ Post-tPA intracranial hemorrhage 104,106,107
M tPA use 51,104–106,108–111
∼ Length of stay, overall and rural patients 51,104–106,108–111
+ Length of stay, super rural patients 51
+ Appropriate transfer to high-volume center or higher level of care 109
+ Discharge to home or rehab 107,112	+ Correctness of decision-making/accurate triage 104,112	– Total medical expenditures per event 51	
Heart attack/STEMI
(2)
TeleECG	+ On time and time to treatment 113,114	None reported	None reported	
Heart attack/STEMI
(1)
Text triage	+ Time to hospital arrival 115
M Arrival-to-balloon time 115
∼ False-positive STEMI 115	None reported	None reported	
Chest pain/ MI
(2)
TeleED videoconference	M Time from ED arrival to ECG 108,116
∼ Time from ED arrival to fibrinolytic 108,116
+ Likelihood of receiving fibrinolytic when eligible 116	None reported	None reported	
Multiple
(1)
TeleED robot	+ Transfer 117
+ Length of stay (robot/transfer vs. no robot/no transfer) 117	None reported	None reported	
Multiple
(4)
TeleED video-conference	∼ Patient deaths in ER 118
M Transfer to another facility 118,119
+ Admit to provider time 120
– ED length of stay, non-transferred patients 120
+ ED length of stay, transferred patients 120
– Likelihood of discharge from ED 118
– Rural hospital admission 118
– Discharged AMA from ED 118	None reported	∼ Total ED patient volume 118
+ Lower ED costs and operating expenses 121	
Critical care
(3)
TeleED video-conference	+ Transfer avoided 122
+ Transfer to lower level of care 122
+ Parent/guardian satisfaction 123
+ Accuracy of clinical picture prior to arrival 122	+ Physician-related ED medication errors 124
+ Quality of care scores 123
+ Referring physician satisfaction 123	None reported	
Trauma
(1)
TeleED video-conference	∼ Mortality
+ Chest tube, intubation 125
+ ED length of stay 125
M Increased diagnostic imaging 125
+ Time to arrival at final hospital 125
∼ Transfer 125	None reported	None reported	
Hand trauma
(1)
TeleED video-conference	∼ Length of stay at first hospital 126
∼ Type of transfer, air or ground 126
+ Admitted from ED 126	None reported	∼ Transport cost	
Sepsis/septic shock
(1)
TeleED video-conference	None reported	+ Adherence to sepsis treatment bundle 127	None reported	
Behavioral health; suicidal ideation or attempt
(2)
TeleED video-conference	+ ED wait time 128
– ED length of stay 128 b
+ Hospital admission 129
+ Involuntary hold placement 129
∼ 30-day readmission 129	None reported	None reported	
Sexual abuse
(1)
TeleED video-conference	None reported	+ Quality, completeness, accuracy of abuse examination forms 130	None reported	
AMA: against medical advice; BP: blood pressure; CIS-P: Columbia impairment scale-parent version; ECG: electrocardiogram; ED: emergency department; HTN: hypertension; ICER: incremental cost-effectiveness ratio; ICU: intensive care unit; MMSE: mini-mental state examination; PTSD: post-traumatic stress disorder; SF-36: 36 item short form survey; STEMI: ST-elevated myocardial infarction; tPA: tissue plasminogen activator; VADRS: Vanderbilt Attention-Deficit/Hyperactivity Disorder Rating Scale.

a Symbol meaning: +: Improved outcome with telehealth; ∼: Similar outcome with telehealth; –: Worse outcome with telehealth; M: Outcomes were not consistent across studies.

b Length of stay was longer with telehealth, and longer stays are usually considered a negative outcome. However mortality decreased which may indicate that more care was appropriate. The authors note that the consultations tended to occur later in the hospital stay in this study and a study in which the consultation happened sooner would provide clearer evidence about the impact on length of stay.

Effectiveness for inpatient care

Seventeen studies, reported in 18 articles, evaluated RPPT for inpatient care (Table 1). The majority (n = 12) were studies that compared outcomes before and after the implementation of telehealth,52,54–62,66,67 two (reported in three articles) were prospective cohort studies,63,64,131 and three were retrospective cohort studies.53,65,68 Eleven were conducted in the United States,52,53,55,56,58,62–66,68 five in Australia,54,57,60,61,67 and one in Scotland. 59 Risk of bias was rated as high for six studies52,57,58,60,62,66 and medium for 11 studies.53–56,59,61,63–65,67,68

Table 2. Summary of evidence of effectiveness for provider education and mentoring.

Modality	Clinical topic (number of studies)	Patient outcomes a	Provider outcomes a	
ECHO video-conference	Antibiotic therapy (1)	∼ In-hospital mortality and length of stay 132	+ Antibiotic prescribing 132	
COVID-19 in long-term care (1)	None reported	+ Self-efficacy and satisfaction 133	
Dementia (1)	None reported	+ Comfort with assessment and treatment 134	
Diabetes (2)	+ A1c 135	+ Self-efficacy in patient coaching/education; identification of psychosocial treatment barriers 136	
Liver disease (3)	∼ Sustained viral response 137
∼ Serious adverse events 137
∼ Access to direct acting antiviral treatment in rural areas 138	+ Hepatitis C virus awareness, knowledge, abilities, and intention to screen at-risk patients 139	
Mental health (5)	None reported	+ Opioid use disorder diagnosis/prescribing 140
+ Reduction in patients prescribed ≥3 psychotropic medications 141
+ Provider knowledge and self-efficacy 142
+ General development and Autism-specific screening 143
+ Pediatric behavioral health knowledge and patient management 141
+ Satisfaction with sessions 141,142,144	
Non-ECHO video-conference	Childhood obesity (1)	∼ Child nutrition and physical activity 145	+ Documentation and counseling 145
∼ Family centered care 145	
Dermatology (1)	None reported	+ Knowledge of and ability to provide dermatology procedures 146
∼ Ability to provide liquid nitrogen 146	
Mental health (1)	None reported	+ Confidence in managing behavioral and psychological symptoms of dementia 147
+ Satisfaction with educational program 147	
Pediatric burns (1)	None reported	+ Knowledge of burn prevention, first aid, airway/inhalation injury, and circulation and fluid resuscitation 148
M Chemical and electrical burns, burn wound, pain and itch management 148	
Perioperative care (1)	None reported	∼ Perioperative training scores 149
∼ Rating program as a success 149	
Online education course	Mental health (3)	None reported	+ Role adequacy, legitimacy, and support 150
M Role motivation, work satisfaction, and task-specific self-esteem 150
+ Education completion 151
+ Knowledge, skills, confidence, and utilization of CBT 151
+ Computer and internet-related skills 152
+ Knowledge about mental health service, confidence in responding to mental health problems 152	
Low vision screening (1)	None reported	+ Knowledge of low vision screening and treatment 138	
Short messaging service	Multiple conditions (1)	None reported	∼ Medical knowledge 153
+ Satisfaction with intervention 153	
A1c: glycated hemoglobin; CBT: cognitive-behavioral therapy; ECHO: Extension for Community Healthcare Outcomes.44,154

a Symbol meaning: +: Improved outcome with telehealth; ∼: Similar outcome with telehealth; –: Worse outcome with telehealth; M: Outcomes were not consistent across studies.

Summarizing these studies, we found evidence that telehealth consultations in rural hospitals resulted in no difference in length of hospital stay (six studies; low SOE) or transfers (three studies; low SOE) compared with usual care, which was in-person or phone consultations. Additionally, telehealth-supported care for neonates at rural hospitals resulted in no difference in clinical outcomes when compared to transfer and care at a hospital with a Level 4 neonatal intensive care unit (two studies; Low SOE). Also, mortality rates were not different when patients were treated in rural hospitals with remote ICUs rather than transported to more distant hospitals (two studies; low SOE).

Effectiveness for outpatient care

Thirty-two studies (in 35 publications) evaluated the use of RPPT interventions to support outpatient care.69–74,76–82,84–103,155 Sixteen studies (reported in 17 publications) were randomized controlled trials (RCTs),69–73,76–79,81,86,89,95,98–100,102 six were prospective cohort studies,74,83,85,92,96,155 six were retrospective cohort studies,80,84,87,88,91,101 four were pre-post study designs (same group measured before and after implementation),82,90,94,97 and two were before-after studies (different groups/systems measured before and after implementation).93,103 Eleven of the studies were conducted in the United States;69–71,76,78,82,84,85,94,95,99,100,103 the remainder were conducted in Canada,86,90,98 Australia,83,88,91,155 Korea,79,92,97 China, 102 Denmark, 80 New Zealand, 87 Spain, 96 United Kingdom,93,101 Chile, 74 Taiwan, 81 and Sweden. 89 Risk of bias was rated as high for eight studies,80,82,89,91,93,97,101,103 medium for 21 studies,71,74,76,78,79,81,83–88,90,92,94–96,99,100,155,156 and low for two studies.98,102

RPPT for outpatient care varied widely across interventions and outcomes (Table 1). Outpatient telehealth consultations with specialists in various clinical specialties (e.g. diabetes, depression, outpatient medication management) resulted in improvements in clinical outcomes compared with care without specialist involvement. For patients with diabetes, RPPT resulted in improvements in A1c and self-management in patients with diabetes but had no effect on blood pressure or cholesterol levels (4 studies; Low SOE). In patients with depression, RPPT was associated with some improvement in treatment response, medication adherence, and satisfaction (three studies; Low SOE), and higher utilization and corresponding costs for outpatient consultations due to increased access, resulting in overall benefit based on cost-effectiveness analyses (two studies; Low SOE). Studies found that outpatient telepharmacy consultations improved guideline adherence and patient outcomes but only for patients with both diabetes and hypertension (two studies; Low SOE).

Effectiveness for emergency care

We included 28 studies of RPPT for emergency care, either by emergency medical services (EMS) or EDs.51,104–130 Two of these studies were RCTs,104,112 10 were prospective cohort studies,106,108,110,111,113–117,127 eight were retrospective cohort studies,51,121–125,129,130 and eight were studies that compared outcomes before and after telehealth initiation.105,107,109,118–120,126,128 Eighteen of these studies were conducted in the United States,51,104,108,111,116,118–130 two were conducted in Italy,113,114 two in Australia,105,107 and one each in Canada, 117 Spain, 110 Finland, 106 Turkey, 115 and Japan. 109 Risk of bias was rated as high for one study, 108 medium for 26 studies,51,105–107,109–130 and low for one RCT. 104

Telehealth consultations for emergency assessment and initial care of stroke, heart attack, or chest pain at rural hospitals resulted in similar rates of mortality (five studies; Low SOE) and similar time to treatment when patients were treated locally as opposed to transferred, suggesting telehealth did not cause delays (eight studies; Low SOE). Telehealth consultations by specialists for critical care and trauma patients in rural EDs had a generally positive impact on transfers, with results reporting fewer unnecessary transfers, more appropriate transfers, or similar rates compared with care without RPPT (five studies; Low SOE).

Effectiveness for education and mentoring

Twenty-three studies evaluated RPPT for education and mentoring (Table 2),132–153,157 including three RCTs144,151,153 and 20 observational studies. Observational study designs included pre-post,132–136,138,139,141–143,146,148,150,152 prospective cohort,137,149 and retrospective cohort studies.140,156 All studies were conducted across multiple clinical sites or health care organizations. Sixteen studies were performed in the United States,132,134–141,143–146,149,150,157 four in Australia,147,148,151,152 one in Canada, 142 and one in Vietnam. 153 One study was rated low risk of bias, 142 11 were rated medium risk of bias,132,133,137,138,140,144,145,149,151,153,157 and 11 were rated high risk of bias.134–136,139,141,143,146–148,150,152

Extension for Community Healthcare Outcomes (ECHO) programs is a model that uses video for remote instruction and case-based learning and was designed to promote best practices in rural healthcare158,159 and has been use to expand access to specialty care. 160 ECHO programs were associated with better or equivalent patient outcomes (two studies; Low SOE), including reduction in A1c in patients of trainees after ECHO compared with before participation (one study), and similar Hepatitis C viral response and serious adverse events rates at the “spoke” site with ECHO participation to those at an academic medical center (one study). ECHO and non-ECHO video training programs (a) resulted in desired changes in provider behavior (e.g. increased appropriate prescribing practices, screening, and patient counseling) (eight studies; Low SOE) and (b) were associated with increased provider confidence, efficacy, and scores on knowledge tests (ten studies; Low SOE).

Implementation of provider-to-provider telehealth for rural patients

Sixty-seven studies in 71 publications addressed the implementation of RPPT.49,59,118,161–227 Most of these studies were program evaluations that combined data from several sources, such as site visits, observations, surveys, and interviews. A smaller number were qualitative research studies that analyzed interviews, focus groups, or documents and then categorized or cataloged specific barriers and facilitators to initial implementation, ongoing operations, longer-term sustainment, or spread of the use of telehealth.

Information on the barriers and facilitators was first recorded as described by the study authors. Then, because studies used different terms, we mapped these using CFIR constructs (mapping tables are provided in the full report) to standardize our description and allow us to summarize and compare these across studies. Identified barriers and facilitators mapped to 19 of the 39 constructs that make up CFIR.

Figure 2 illustrates how the 219 facilitators and 192 barriers we identified map to the CFIR constructs. Each construct and the number of times it occurred in our data is included in the label, and the width of the line represents the relative frequency. Short definitions of each construct are included as notes to the figure; comprehensive descriptions are available in CFIR articles and tools.44,154 The figure illustrates the point that most constructs can either facilitate or impede implementation, depending both on context and their presence or absence. The two most frequently cited constructs in our analysis were Available Resources and Access to Knowledge & Information. Available Resources is a broad concept including the wide range of investments an organization dedicates to implementing or sustaining an innovation or program. This was cited 60 times, and almost equally split between mention as a facilitator and as a barrier. Access to Knowledge & Information was cited 57 times, most frequently, though not exclusively, as a facilitator. We repeated the analysis by health care setting (available in the full report). Barriers and facilitators were similar across different settings. Assessed and summarized using CFIR constructs, the included implementation studies and program evaluations provided information about what is needed to translate knowledge about effectiveness of RPPT into actual practice.

Figure 2. Facilitators and barriers to implementation*.

*Definitions: Access to Knowledge & Information: Access to digestible information and knowledge about the innovation and how to incorporate it into work tasks; Formally Appointed Internal Implementation Leaders: Individuals from within the organization who have been formally appointed with responsibility for implementing an innovation as coordinator, project manager, team leader, or other similar role; Planning: Degree to which a scheme or method of behavior and tasks for implementing an innovation are developed in advance, and the quality of those schemes or methods; Leadership Engagement: Commitment, involvement, and accountability of leaders and managers with the implementation of the innovation; Engaging: Attracting and involving appropriate individuals in the implementation and use of the innovation through a combined strategy of social marketing, education, role modeling, training, and other similar activities; Available Resources: Level of resources organizational dedicated for implementation and on-going operations including physical space and time; Networks & Communications: Nature and quality of webs of social networks, and the nature and quality of formal and informal communications within an organization; Reflecting & Evaluating: Quantitative and qualitative feedback about the progress and quality of implementation accompanied with regular personal and team debriefing about progress and experience; Cost: Costs of the innovation and costs associated with implementing the innovation including investment, supply, and opportunity costs; External Policy & Incentives: External strategies to spread innovations including policy and regulations (governmental or other central entity), external mandates, recommendations and guidelines, pay-for-performance, collaboratives, and public or benchmark reporting; Relative Priority: Individuals’ shared perception of the importance of the implementation within the organization; Implementation Climate: Absorptive capacity for change, shared receptivity of involved individuals to an innovation, and the extent to which use of that innovation will be rewarded, supported, and expected within their organization; Readiness for Implementation: Tangible and immediate indicators of organizational commitment to its decision to implement an innovation; Adaptability: Degree to which an innovation can be adapted, tailored, refined, or reinvented to meet local needs; Needs & Resources of Those Served by the Organization: Extent to which the needs of those served by the organization (e.g. patients), as well as barriers and facilitators to meet those needs, are accurately known and prioritized by the organization; Compatibility: Degree of tangible fit between meaning and values attached to the innovation by involved individuals, how those align with individuals’ own norms, values, and perceived risks and needs, and how the innovation fits with existing workflows and systems; Knowledge & Beliefs about the Innovation: Individuals’ attitudes toward and value placed on the innovation, as well as familiarity with facts, truths, and principles related to the innovation; Complexity: Perceived difficulty of the innovation, reflected by duration, scope, radicalness, disruptiveness, centrality, and intricacy and number of steps required to implement; Executing: Carrying out or accomplishing the implementation according to plan.
CFIR: Consolidated Framework for Implementation Research.44,154

Many facilitators and barriers were not specific to rural settings. The studies highlight that telehealth needs to alleviate burden on providers, the technology needs to work, staff resources and reimbursement need to be allocated to provide both start-up infrastructure and ongoing support, and training of both support staff and clinicians is needed. Engagement from a range of stakeholders from patients to health system leadership and governments was cited in studies as essential. Some issues were raised in identified studies that are specific to rural programs. In the United States, insufficient internet in rural areas remains a persistent barrier. Lack of understanding of the rural environment by urban-based consultants and educators risks reducing the utility of teleconsultations and remote training programs. Remote consultants for the care of rural patients can be used for either frequent events or serve as a resource for rare events in rural healthcare, but needed technology and procedures are different in these two cases and technology and operations needed to be tailored to frequency of use. Most RPPT programs are local or in a single health system, yet sustainment requires long-term commitment and resources, on a scale that may not be feasible for smaller rural organizations.

Methodologic challenges in studying provider-to-provider telehealth for rural patients

We abstracted limitations cited by the authors and combined these with our risk of bias and applicability assessments to identify and categorize the methodological weaknesses of the available evidence. Studies did not routinely control for confounders related to patients, providers, facilities, and differences in telehealth implementation across study sites, and have been hampered by data limitations, such as missing data or inaccuracies in data collected for reasons other than research. In our assessment, it is often difficult to attribute impact to telehealth because less rigorous study designs, rather than RCTs or prospective, well-designed observational studies, were used in more than two-thirds of the included studies (Table 3).

Table 3. Study designs and number of sites by setting for effectiveness studies.

	Inpatient	Outpatient	EMS/ED	Education/Mentoring	Total	
Study design	
RCT	0	15	2	3	20	
Prospective cohort	2	6	10	2	20	
Retrospective cohort	3	6	8	2	19	
Pre/Post a	0	4	0	14	18	
Before/after b	13	3	8	2	26	
Number of sites	
Multisite	13	27	22	23	85	
Single center	5	7	6	0	18	
Total	18	34	28	23	103	
ED: emergency department; EMS: emergency medical services; RCT: randomized controlled trial.

a Same patients or providers evaluated pre and post-intervention.

b Do not have the same patients or providers in both time points.

Discussion

The research on RPPT for collaboration in the delivery of rural health care provided an evidence base that addressed questions about use, effectiveness, implementation, and methodological weaknesses, but the evidence was uneven and could not support definitive, universally applicable conclusions.

To answer Key Question 1, we identified seven studies that reported national or regional trends of increasing telehealth use, with differences across specialties and geographic locations. Increasing use was evident even before the onset of the COVID-19 pandemic. For effectiveness (Key Question 2), we identified and synthesized studies that assessed the impact of RPPT and found evidence that outcomes were better or similar with RPPT for applications in inpatient, outpatient, and emergency care. We also summarized studies demonstrating that telehealth for rural provider education and mentoring, including ECHO programs, may improve patient outcomes, change provider behavior, and increase provider knowledge and confidence in treating specific conditions. To address Key Question 3, we categorized barriers and facilitators, finding most were common to practice change initiatives, though some were specific to the rural context. We identified important methodological weaknesses (Key Question 4) in the RPPT effectiveness studies, including less rigorous study designs and small sample sizes.

Our synthesis of the available evidence was qualitative because the modes, functions, and outcomes studied and how they are measured were heterogeneous. We grouped studies of similar topics that assessed similar outcomes allowing these conclusions. However, there were several instances where only one study was identified, or two or more studies reported conflicting results leading to an assessment of “unclear effect” or “insufficient evidence.” Classification of barriers and facilitators to implementation in 67 studies confirmed that common barriers for change in practice, including inadequate provider time, technology, and other resources, are limiting the spread of RPPT. However, there are also specific barriers, such as incomplete understanding of rural context and lack of long-term commitments to maintaining the infrastructure and staffing needed. Our assessment of the methodological weaknesses of the effectiveness studies found these frequently employed less rigorous designs, had small sample sizes, and often did not minimize possible bias through design or analytic approach.

Our review had limitations associated with our methods. We only included studies published in English about research conducted in developed countries. Searching for studies of RPPT was challenging as telehealth is a broad term and studies do not consistently include “rural” in titles or abstracts. We were unable to conduct quantitative synthesis (i.e. meta-analyses) and our qualitative synthesis combined studies with similar, but not identical, outcomes.

There were also limitations due to the nature of the evidence base. Research on telehealth in general is often not based on a clear model of how telehealth is expected to affect outcomes and whether telehealth needs to produce outcomes that are better than standard care or if equivalence with in-person alternatives is sufficient. It is often unclear if the goal of telehealth is to provide care that is as good as care provided without telehealth or if the investment in telehealth requires outcomes to be better. While telehealth should increase patient and provider satisfaction, there is no agreement on how to prioritize clinical outcomes, resource use, costs, and potential harms.

Our assessment of the methodological weaknesses of the included studies suggests directions for future research. Additional research is needed to measure outcomes at multiple time points or over longer periods of time, as short-term outcomes may differ from longer-term outcomes (e.g. provider retention of knowledge acquired through remote education and mentoring). Additional trials would strengthen the evidence base, as would observational studies that include contemporaneous comparison groups and multiple sites. The former would allow the impact of telehealth to be separated from historical change or the potentially unique characteristics of specific sites or providers/consultants. More complete descriptions are needed of both telehealth interventions and comparators. Clear statements of the intended impact of telehealth would help inform assessments of fit and help clarify whether telehealth was designed to replace in-person services or add additional services.

The existing evidence base was insufficient to allow us to unequivocally endorse all potential uses of RPPT as tools for improving health care for rural patients. Nevertheless, the studies we identified and summarized do not report harm or negative consequences. More importantly, they suggest it is likely that the application of telehealth can improve patient outcomes such as access to and quality of care, provider outcomes such as knowledge and self-efficacy, and payer outcomes such as reduced costs or maintenance of payments to rural providers.

Supplemental Material

sj-docx-1-jtt-10.1177_1357633X221139892 - Supplemental material for Telehealth-guided provider-to-provider communication to improve rural health: A systematic review

Supplemental material, sj-docx-1-jtt-10.1177_1357633X221139892 for Telehealth-guided provider-to-provider communication to improve rural health: A systematic review by Annette M Totten, Dana M Womack, Jessica C Griffin, Marian S McDonagh, Cynthia Davis-O'Reilly, Ian Blazina, Sara Grusing, and Nancy Elder in Journal of Telemedicine and Telecare

Acknowledgements

AHRQ staff, an NIH ODP working group, an NIH-convened content area expert group, and a technical expert panel helped refine the project scope. The draft report was presented at an NIH ODP P2P workshop. Experts in the field, AHRQ and NIH partners, and the public reviewed earlier drafts of the full report. The investigators are solely responsible for the contents of this manuscript.

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding: This project was funded by the National Institutes of Health (NIH) through an interagency agreement with the Agency for Healthcare Research and Quality (AHRQ), U.S. Department of Health and Human Services (HHS) under Contract No. 75Q80120D00006/Task Order 75Q80120F32001. The findings and conclusions in this document are those of the authors, who are responsible for its contents; the findings and conclusions do not necessarily represent the views of AHRQ or NIH. Therefore, no statement in this report should be construed as an official position of AHRQ, NIH, or HHS.

ORCID iD: Annette M Totten https://orcid.org/0000-0002-9100-8678

Supplemental material: Supplemental material for this article is available online.
==== Refs
References

1 Rural health. Health Aff (Millwood) 2019; 38 : 1964–1965. DOI: 10.1377/hlthaff.2019.01365.31794298
2 Galambos CM . Health care disparities among rural populations: A neglected frontier. Health Soc Work 2005; 30 : 179–181.16190293
3 Garcia MC Faul M Massetti G , et al. Reducing potentially excess deaths from the five leading causes of death in the rural United States. MMWR Surveill Summ 2017; 66 : 1–7.
4 Singh GK Siahpush M . Widening rural-urban disparities in all-cause mortality and mortality from major causes of death in the USA, 1969-2009. J Urban Health 2014; 91 : 272–292.24366854
5 Garcia MC Rossen LM Bastian B , et al. Potentially excess deaths from the five leading causes of death in metropolitan and nonmetropolitan counties - United States, 2010-2017. MMWR Surveill Summ 2019; 68 : 1–11.
6 Hall JE Moonesinghe R Bouye K , et al. Racial/ethnic disparities in mortality: contributions and variations by rurality in the United States, 2012-2015. Int J Environ Res Public Health 2019; 16 : 436.30717345
7 Villapiano N Iwashyna TJ Davis MM . Worsening rural-urban gap in hospital mortality. J Am Board Fam Med 2017; 30 : 816–823.29180557
8 Keyes KM Cerdá M Brady JE , et al. Understanding the rural-urban differences in nonmedical prescription opioid use and abuse in the United States. Am J Public Health 2014; 104 : e52–e59.
9 Monnat SM Rigg KK . Examining rural/urban differences in prescription opioid misuse among US adolescents. J Rural Health 2016; 32 : 204–218.26344571
10 Boring MA Hootman JM Liu Y , et al. Prevalence of arthritis and arthritis-attributable activity limitation by urban-rural county classification - United States, 2015. MMWR Morb Mortal Wkly Rep 2017; 66 : 527–532.28542117
11 Croft JB Wheaton AG Liu Y , et al. Urban-rural county and state differences in chronic obstructive pulmonary disease - United States, 2015. MMWR Morb Mortal Wkly Rep 2018; 67 : 205–211.29470455
12 Krishna S Gillespie KN McBride TM . Diabetes burden and access to preventive care in the rural United States. J Rural Health 2010; 26 : 3–11.20105262
13 Primm K Ferdinand AO Callaghan T , et al. Congestive heart failure-related hospital deaths across the urban-rural continuum in the United States. Prev Med Rep 2019; 16 : 101007.31799105
14 Nelson JA Kinder A Johnson AS , et al. Differences in selected HIV care continuum outcomes among people residing in rural, urban, and metropolitan areas-28 US jurisdictions. J Rural Health 2018; 34 : 63–70.27620836
15 Zahnd WE Rodriguez C Jenkins WD . Rural-urban differences in human papillomavirus-associated cancer trends and rates. J Rural Health 2019; 35 : 208–215.29808500
16 Probst JC Moore CG Glover SH , et al. Person and place: the compounding effects of race/ethnicity and rurality on health. Am J Public Health 2004; 94 : 1695–1703.15451735
17 Austin B Glaeser E Summers L . Saving the heartland: place-based policies in 21st century America. In: BPEA conference draft, Spring 2018.
18 Porter E . The hard truths of trying to “save” the rural economy, https://www.nytimes.com/interactive/2018/12/14/opinion/rural-america-trump-decline.html (2018, accessed 9 September 2020).
19 Tuckson RV Edmunds M Hodgkins ML . Telehealth. N Engl J Med 2017; 377 : 1585–1592.29045204
20 Castro D Miller B Nager A . Unlocking the potential of physician-to-patient telehealth services, https://itif.org/publications/2014/05/12/unlocking-potential-physician-patient-telehealth-services (2014, accessed 5 September 2020).
21 Lustig T . The role of telehealth in an evolving health care environment - workshop summary. Washington DC: The National Academies Press. 2012.
22 Bashshur RL Shannon GW Smith BR , et al. The empirical foundations of telemedicine interventions for chronic disease management. Telemed e-Health 2014; 20 : 769–800.
23 Ekeland AG Bowes A Flottorp S . Effectiveness of telemedicine: a systematic review of reviews. Int J Med Inf 2010; 79 : 736–771.
24 Hersh WR Wallace JA Patterson PK , et al . Telemedicine for the medicare population-evidence reports/technology assessments, No. 24. Rockville, MD. July 2001.
25 Hersh WR Hickman D Severance S , et al . Telemedicine for the Medicare population: update - evidence reports/technology assessments, No. 131. Rockville, MD. February 2006.
26 Hersh WR Wallace JA Patterson PK , et al. Telemedicine for the medicare population: pediatric, obstetric, and clinician-indirect home interventions. Evid Rep Technol Assess (Summ) 2001; 24 Suppl: 1–32.
27 Totten AM Womack DM Eden KB , et al. Telehealth: mapping the evidence for patient outcomes from systematic reviews - technical brief No. 26. Rockville, MD: Pacific Northwest Evidence-based Practice Center, 2016.
28 Snoswell CL Chelberg G De Guzman KR , et al. The clinical effectiveness of telehealth: a systematic review of meta-analyses from 2010 to 2019. J Telemed Telecare 2021: 1357633x211022907. DOI: 10.1177/1357633x211022907.
29 Snoswell CL Stringer H Taylor ML , et al. An overview of the effect of telehealth on mortality: a systematic review of meta-analyses. J Telemed Telecare 2021: 1357633x211023700. DOI: 10.1177/1357633x211023700.
30 Scott Kruse C Karem P Shifflett K , et al. Evaluating barriers to adopting telemedicine worldwide: a systematic review. J Telemed Telecare 2018; 24 : 4–12.29320966
31 Adler-Milstein J Kvedar J Bates DW . Telehealth among US hospitals: several factors, including state reimbursement and licensure policies, influence adoption. Health Aff (Millwood) 2014; 33 : 207–215.24493762
32 Broderick A Lindeman D . Scaling telehealth programs: lessons from early adopters. New York, NY: Commonwealth Fund, http://www.commonwealthfund.org/Publications/Case-Studies/2013/Jan/Telehealth-Synthesis.aspx (2013, accessed 8 July 2016).
33 Rae M Cox C Claxton G . Coverage and utilization of telemedicine services by enrollees in large employer plans, https://www.healthsystemtracker.org/brief/coverage-and-utilization-of-telemedicine-services-by-enrollees-in-large-employer-plans/ (2020, accessed 5 September 2020).
34 Smith AC Thomas E Snoswell CL , et al. Telehealth for global emergencies: implications for coronavirus disease 2019 (COVID-19). J Telemed Telecare 2020; 26 : 309–313.32196391
35 Thomas EE Haydon HM Mehrotra A , et al. Building on the momentum: sustaining telehealth beyond COVID-19. J Telemed Telecare 2022; 28 : 301–308.32985380
36 Gallegos-Rejas VM Thomas EE Kelly JT , et al. A multi-stakeholder approach is needed to reduce the digital divide and encourage equitable access to telehealth. J Telemed Telecare 2022: 1357633x221107995. DOI: 10.1177/1357633x221107995.
37 Shah DA Sall D Peng W , et al. Exploring the role of telehealth in providing equitable healthcare to the vulnerable patient population during COVID-19. J Telemed Telecare 2022: 1357633x221113711. DOI: 10.1177/1357633x221113711.
38 Cameron MP Ray R Sabesan S . Physicians’ perceptions of clinical supervision and educational support via videoconference: a systematic review. J Telemed Telecare 2014; 20 : 272–281.24906649
39 McBain RK Sousa JL Rose AJ , et al. Impact of project ECHO models of medical tele-education: a systematic review. J Gen Intern Med 2019; 34 : 2842–2857.31485970
40 Totten AM Hansen RN Wagner J , et al . Telehealth for acute and chronic care consultations. Rockville, MD: Agency for Healthcare Research and Quality (US), 2019.
41 Mary W, Joanne MC, Sara M, et al. National institutes of health pathways to prevention workshop: Improving rural health through telehealth-guided provider-to-provider communication. J Telemedicine Telecare 2022. DOI: 10.1177/1357633X221139630.
42 Berkman ND Lohr KN Ansari MT , et al . Grading the strength of a body of evidence when assessing health care interventions: an EPC update. J Clin Epidemiol 2015; 68 : 1312–1324.25721570
43 United Nations Development Programme. Human development report 2020: the next frontier: human development and the Anthropocene. New York, NY: United Nations Development Programme, 2020.
44 Damschroder LJ Aron DC Keith RE , et al. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci 2009; 4 : 50.19664226
45 Alishahi Tabriz A Turner K Williams D , et al. Association of financial factors and telemedicine adoption for heart attack and stroke care among rural and urban hospitals: a longitudinal study. Telemed J E Health 2021; 24 : 24.
46 Chen J Amaize A Barath D . Evaluating telehealth adoption and related barriers among hospitals located in rural and urban areas. J Rural Health 2021; 37 : 801–811.33180363
47 Freeman RE Boggs KM Zachrison KS , et al. National study of telepsychiatry use in U.S. Emergency departments. Psychiatr Serv 2020; 71 : 540–546.32019430
48 Spivak S Spivak A Cullen B , et al. Telepsychiatry use in U.S. Mental health facilities, 2010-2017. Psychiatr Serv 2020; 71 : 121–127.31615370
49 Zachrison KS Boggs KM Hayden EM , et al. Understanding barriers to telemedicine implementation in rural emergency departments. Ann Emerg Med 2020; 75 : 392–399.31474481
50 Zachrison KS Hayden EM Schwamm LH , et al. Characterizing New England emergency departments by telemedicine use. West J Emerg Med 2017; 18 : 1055–1060.29085537
51 Zhang D Wang G Zhu W , et al. Expansion of telestroke services improves quality of care provided in super rural areas. Health Aff (Millwood) 2018; 37 : 2005–2013.30633675
52 Kuperman EF Linson EL Klefstad K , et al. The virtual hospitalist: a single-site implementation bringing hospitalist coverage to critical access hospitals. J Hosp Med 2018; 13 : 759–763.30255859
53 Boltz M Cuellar NG Cole C , et al. Comparing an on-site nurse practitioner with telemedicine physician support hospitalist programme with a traditional physician hospitalist programme. J Telemed Telecare 2019; 25 : 213–220.29498301
54 Poulson LK Nissen L Coombes I . Pharmaceutical review using telemedicine--a before and after feasibility study. J Telemed Telecare 2010; 16 : 95–99.20139138
55 Dharmar M Sadorra CK Leigh P , et al. The financial impact of a pediatric telemedicine program: a children’s hospital’s perspective. Telemed e-Health 2013; 19 : 502–508.
56 Tande AJ Berbari EF Ramar P , et al. Association of a remotely offered infectious diseases eConsult service with improved clinical outcomes. Open Forum Infect Dis 2020; 7 : 1–4.
57 Avent ML Walker D Yarwood T , et al. Implementation of a novel antimicrobial stewardship strategy for rural facilities utilising telehealth. Int J Antimicrob Agents 2021; 57 : 106346.33882332
58 Yam P Fales D Jemison J , et al. Implementation of an antimicrobial stewardship program in a rural hospital. Am J Health-Syst Pharm 2012; 69 : 1142–1148.22722593
59 Barber M Frieslick J Maclean A , et al. The western isles stroke telerehabilitation (specialist medical consultation) service–implementation and evaluation. Eur Res Telemed 2015; 4 : 19–24.
60 Gallagher R Giles M Morison J , et al. Telehealth-based model of care redesign to facilitate local fitting and management of patients with a spinal fracture requiring a thoracic lumbar sacral orthosis in rural hospitals in New South Wales. Aust J Rural Health 2018; 26 : 181–187.29573012
61 Buckley D Weisser S . Videoconferencing could reduce the number of mental health patients transferred from outlying facilities to a regional mental health unit. Aust N Z J Public Health 2012; 36 : 478–482.23025371
62 Kim EW Teague-Ross TJ Greenfield WW , et al. Telemedicine collaboration improves perinatal regionalization and lowers statewide infant mortality. J Perinatol 2013; 33 : 725–730.23579490
63 Haynes SC Dharmar M Hill BC , et al. The impact of telemedicine on transfer rates of newborns at rural community hospitals. Acad Pediatr 2020; 20 : 636–641.32081766
64 Makkar A McCoy M Hallford G , et al. Evaluation of neonatal services provided in a level II NICU utilizing hybrid telemedicine: a prospective study. Telemed e-Health 2020; 26 : 176–183.
65 Makkar A McCoy M Hallford G , et al. A hybrid form of telemedicine: a unique way to extend intensive care service to neonates in medically underserved areas. Telemed e-Health 2018; 24 : 717–721.
66 Hall RW Hall-Barrow J Garcia-Rill E . Neonatal regionalization through telemedicine using a community-based research and education core facility. Ethn Dis 2010; 20 : S1–136–140.
67 Panlaqui OM Broadfield E Champion R , et al. Outcomes of telemedicine intervention in a regional intensive care unit: A before and after study. Anaesth Intensive Care 2017; 45 : 605–610.28911290
68 Kahn JM Le TQ Barnato AE , et al. ICU Telemedicine and critical care mortality: a national effectiveness study. Med Care 2016; 54 : 319–325.26765148
69 Davis TD Deen T Bryant-Bedell K , et al. Does minority racial-ethnic status moderate outcomes of collaborative care for depression? Psychiatr Serv 2011; 62 : 1282–1288.22211206
70 Fortney JC Maciejewski ML Tripathi SP , et al. A budget impact analysis of telemedicine-based collaborative care for depression. Med Care 2011; 49 : 872–880.21623240
71 Fortney JC Pyne JM Mouden SB , et al. Practice-based versus telemedicine-based collaborative care for depression in rural federally qualified health centers: a pragmatic randomized comparative effectiveness trial. Focus 2017; 15 : 361–372.32015700
72 Pyne JM Fortney JC Mouden S , et al. Cost-effectiveness of on-site versus off-site collaborative care for depression in rural FQHCs. Psychiatr Serv 2015; 66 : 491–499.25686811
73 Pyne JM Fortney JC Tripathi SP , et al. Cost-effectiveness analysis of a rural telemedicine collaborative care intervention for depression. Arch Gen Psychiatry 2010; 67 : 812–821.20679589
74 Rojas G Guajardo V Martinez P , et al. A remote collaborative care program for patients with depression living in rural areas: open-label trial. J Med Internet Res 2018; 20 : e158.
75 Fortney JC Pyne JM Edlund MJ , et al . A randomized trial of telemedicine-based collaborative care for depression. J Gen Intern Med 2007; 22 : 1086–1093.17492326
76 Fortney JC Pyne JM Kimbrell TA , et al. Telemedicine-based collaborative care for posttraumatic stress disorder: a randomized clinical trial. JAMA Psychiatry 2015; 72 : 58–67.25409287
77 Painter JT Fortney JC Austen MA , et al. Cost-effectiveness of telemedicine-based collaborative care for posttraumatic stress disorder. Psychiatr Serv 2017; 68 : 1157–1163.28669290
78 Myers K Vander Stoep A Zhou C , et al. Effectiveness of a telehealth service delivery model for treating attention-deficit/hyperactivity disorder: a community-based randomized controlled trial. J Am Acad Child Adolesc Psychiatry 2015; 54 : 263–274.25791143
79 Cho JH Kwon HS Kim HS , et al. Effects on diabetes management of a health-care provider mediated, remote coaching system via a PDA-type glucometer and the internet. J Telemed Telecare 2011; 17 : 365–370.21933896
80 Levin K Madsen JR Petersen I , et al. Telemedicine diabetes consultations are cost-effective, and effects on essential diabetes treatment parameters are similar to conventional treatment: 7-year results from the Svendborg Telemedicine Diabetes Project. J Diabetes Sci Technol 2013; 7 : 587–595.23759390
81 Liou JK Soon MS Chen CH , et al. Shared care combined with telecare improves glycemic control of diabetic patients in a rural underserved community. Telemed e-Health 2014; 20 : 175–178.
82 McLendon SF Wood FG Stanley N . Enhancing diabetes care through care coordination, telemedicine, and education: evaluation of a rural pilot program. Public Health Nurs 2019; 36 : 310–320.30868661
83 Crossland L Askew D Ware R , et al. Diabetic retinopathy screening and monitoring of early stage disease in Australian general practice: tackling preventable blindness within a chronic care model. J Diabetes Res 2016; 2016 : 8405395.26798655
84 Rossaro L Torruellas C Dhaliwal S , et al. Clinical outcomes of hepatitis C treated with pegylated interferon and ribavirin via telemedicine consultation in Northern California. Dig Dis Sci 2013; 58 : 3620–3625.24154637
85 Wood PR Caplan L . Outcomes, satisfaction, and costs of a rheumatology telemedicine program: a longitudinal evaluation. J Clin Rheumatol 2019; 25 : 41–44.30461466
86 Taylor-Gjevre R Nair B Bath B , et al. Addressing rural and remote access disparities for patients with inflammatory arthritis through video-conferencing and innovative inter-professional care models. Musculoskelet 2018; 16 : 90–95.
87 Stevens G Loh J Kolbe J , et al. Comparison of recommendations for radiotherapy from two contemporaneous thoracic multidisciplinary meeting formats: co-located and video conference. Intern Med J 2012; 42 : 1213–1218.22530597
88 Thaker DA Monypenny R Olver I , et al. Cost savings from a telemedicine model of care in northern Queensland, Australia. Med J Aust 2013; 199 : 414–417.24033216
89 Boman K Olofsson M Berggren P , et al. Robot-assisted remote echocardiographic examination and teleconsultation: a randomized comparison of time to diagnosis with standard of care referral approach. JACC Cardiovasc Imaging 2014; 7 : 799–803.25124011
90 Sicotte C Moqadem K Vasilevsky M , et al. Use of telemedicine for haemodialysis in very remote areas: the Canadian first nations. J Telemed Telecare 2011; 17 : 146–149.21303935
91 McGill AF North JB . Teleconference fracture clinics: a trial for rural hospitals. ANZ J Surg 2012; 82 : 2–3.22507484
92 Kim H Jhoo JH Jang JW . The effect of telemedicine on cognitive decline in patients with dementia. J Telemed Telecare 2017; 23 : 149–154.26541171
93 Summerhayes C McGee JA Cooper RJ , et al. Introducing leg ulcer telemedicine into rural general practice. Wounds UK 2012; 8 : 28–36.
94 Anderson EJ Axon DR Taylor AM , et al. Impact evaluation of a four-year academic-community partnership in provision of medication management and tertiary prevention services for rural patients with diabetes and/or hypertension. Prev Med Rep 2020; 17 : 101038.31956473
95 Carter BL Levy B Gryzlak B , et al. Cluster-randomized trial to evaluate a centralized clinical pharmacy service in private family medicine offices. Circ Cardiovasc Qual Outcomes 2018; 11 : e004188.
96 De Batlle J Massip M Vargiu E , et al. Implementing mobile health-enabled integrated care for complex chronic patients: intervention effectiveness and cost-effectiveness study. JMIR Mhealth Uhealth 2021; 9 : e22135.
97 Kwak MY Hwang EJ Lee TH . Effects of the physician-primary-healthcare nurse telemedicine model (P-NTM) on medication adherence and health-related quality of life (HRQoL) of patients with chronic disease at remote rural areas. Int J Environ Res Public Health 2021; 18 : 2502.33802513
98 Liddy C Moroz I Keely E , et al. Understanding the impact of a multispecialty electronic consultation service on family physician referral rates to specialists: a randomized controlled trial using health administrative data. Trials 2019; 20 : 348.31182123
99 Whited JD Warshaw EM Kapur K , et al. Clinical course outcomes for store and forward teledermatology versus conventional consultation: a randomized trial. J Telemed Telecare 2013; 19 : 197–204.23666440
100 Datta SK Warshaw EM Edison KE , et al. Cost and utility analysis of a store-and-forward teledermatology referral system: a randomized clinical trial. JAMA Dermatol 2015; 151 : 1323–1329.26375589
101 Van der Pol M McKenzie L . Costs and benefits of tele-endoscopy clinics in a remote location. J Telemed Telecare 2010; 16 : 89–94.20139140
102 Yan LL Gong E Gu W , et al. Effectiveness of a primary care-based integrated mobile health intervention for stroke management in rural China (SINEMA): a cluster-randomized controlled trial. PLoS Med 2021; 18 : e1003582.
103 Long MC Angtuaco T Lowery C . Ultrasound in telemedicine: its impact in high-risk obstetric health care delivery. Ultrasound Q 2014; 30 : 167–172.25148484
104 Demaerschalk BM Raman R Ernstrom K , et al. Efficacy of telemedicine for stroke: pooled analysis of the stroke team remote evaluation using a digital observation camera (STRokE DOC) and STRokE DOC Arizona telestroke trials. Telemed e-Health 2012; 18 : 230–237.
105 Nagao KJ Koschel A Haines HM , et al. Rural Victorian telestroke project. Intern Med J 2012; 42 : 1088–1095.21981424
106 Sairanen T Soinila S Nikkanen M , et al. Two years of Finnish Telestroke: thrombolysis at spokes equal to that at the hub. Neurology 2011; 76 : 1145–1152.21368283
107 Bladin CF Moloczij N Ermel S , et al. Victorian Stroke Telemedicine Project: implementation of a new model of translational stroke care for Australia. Intern Med J 2015; 45 : 951–956.26011155
108 Mohr NM Young T Harland KK , et al. Telemedicine is associated with faster diagnostic imaging in stroke patients: a cohort study. Telemed e-Health 2019; 25 : 93–100.
109 Nagayoshi Y Oshima S Ogawa H . Clinical impact of telemedicine network system at rural hospitals without on-site cardiac surgery backup. Telemed e-Health 2016; 22 : 960–964.
110 Pedragosa A Alvarez-Sabin J Rubiera M , et al. Impact of telemedicine on acute management of stroke patients undergoing endovascular procedures. Cerebrovasc Dis 2012; 34 : 436–442.23207552
111 Swanson MB Miller AC Ward MM , et al. Emergency department telemedicine consults decrease time to interpret computed tomography of the head in a multi-network cohort. J Telemed Telecare 2021; 27 : 343–352.31684801
112 Helwig SA Ragoschke-Schumm A Schwindling L , et al. Prehospital stroke management optimized by use of clinical scoring vs mobile stroke unit for triage of patients with stroke: a randomized clinical trial. JAMA Neurol 2019; 76 : 1484–1492.31479116
113 Brunetti ND Dell’Anno A Martone A , et al. Prehospital ECG transmission results in shorter door-to-wire time for STEMI patients in a remote mountainous region. Am J Emerg Med 2020; 38 : 252–257.31079977
114 Brunetti ND Di Pietro G Aquilino A , et al. Pre-hospital electrocardiogram triage with tele-cardiology support is associated with shorter time-to-balloon and higher rates of timely reperfusion even in rural areas: data from the Bari- Barletta/Andria/Trani public emergency medical service 118 registry on primary angioplasty in ST-elevation myocardial infarction. Europ Heart J Acute Cardiovasc Care 2014; 3 : 204–213.
115 Astarcioglu MA Sen T Kilit C , et al. Time-to-reperfusion in STEMI undergoing interhospital transfer using smartphone and WhatsApp messenger. Am J Emerg Med 2015; 33 : 1382–1384.26299691
116 Miller AC Ward MM Ullrich F , et al. Emergency department telemedicine consults are associated with faster time-to-electrocardiogram and time-to-fibrinolysis for myocardial infarction patients. Telemed e-Health 2020; 26 : 1440–1448.
117 Holt T Sari N Hansen G , et al. Remote presence robotic technology reduces need for pediatric interfacility transportation from an isolated northern community. Telemed e-Health 2018; 24 : 927–933.
118 Sterling SA Seals SR Jones AE , et al. The impact of the TelEmergency program on rural emergency care: an implementation study. J Telemed Telecare 2017; 23 : 588–594.27470505
119 Natafgi N Mohr NM Wittrock A , et al. The association between telemedicine and emergency department (ED) disposition: a stepped wedge design of an ED-based telemedicine program in critical access hospitals. J Rural Health 2020; 36 : 360–370.31013552
120 Mohr NM Young T Harland KK , et al. Emergency department telemedicine shortens rural time-to-provider and emergency department transfer times. Telemed e-Health 2018; 24 : 582–593.
121 Williams D Simpson AN King K , et al. Do hospitals providing telehealth in emergency departments have lower emergency department costs? Telemed e-Health 2020; 27 : 1011–1020.
122 Harvey JB Yeager BE Cramer C , et al. The impact of telemedicine on pediatric critical care triage. Pediatr Crit Care Med 2017; 18 : e555–e560.
123 Dharmar M Romano PS Kuppermann N , et al. Impact of critical care telemedicine consultations on children in rural emergency departments. Crit Care Med 2013; 41 : 2388–2395.23921273
124 Dharmar M Kuppermann N Romano PS , et al . Telemedicine consultations and medication errors in rural emergency departments. Pediatrics 2013; 132 : 1090–1097.24276844
125 Mohr NM Vakkalanka JP Harland KK , et al. Telemedicine use decreases rural emergency department length of stay for transferred North Dakota trauma patients. Telemed e-Health 2018; 24 : 194–202.
126 Tripod M Tait M Bracey J , et al. The use of telemedicine decreases unnecessary hand trauma transfers. Hand 2020; 15 : 422–427.30417694
127 Mohr NM Campbell KD Swanson MB , et al. Provider-o-provider telemedicine improves adherence to sepsis bundle care in community emergency departments. J Telemed Telecare 2021; 27 : 518–526.31903840
128 Fairchild RM Ferng-Kuo SF Laws S , et al. Telehealth decreases rural emergency department wait times for behavioral health patients in a group of critical access hospitals. Telemed e-Health 2019; 25 : 1154–1164.
129 Vakkalanka JP Harland KK Wittrock A , et al. Telemedicine is associated with rapid transfer and fewer involuntary holds among patients presenting with suicidal ideation in rural hospitals: a propensity matched cohort study. J Epidemiol Community Health 2019; 73 : 1033–1039.31492762
130 Miyamoto S Dharmar M Boyle C , et al. Impact of telemedicine on the quality of forensic sexual abuse examinations in rural communities. Child Abuse Negl 2014; 38 : 1533–1539.24841062
131 Haynes SC Hoffman KR Patel S , et al. The use of telemedicine for stabilization of neonates transferred from rural community hospitals. Telemed e-Health 2021; 27 : 1393–1398.
132 Wilson BM Banks RE Crnich CJ , et al. Changes in antibiotic use following implementation of a telehealth stewardship pilot program. Infect Control Hosp Epidemiol 2019; 40 : 810–814.31172905
133 Lingum NR Sokoloff LG Meyer RM , et al. Building long-term care staff capacity during COVID-19 through just-in-time learning: evaluation of a modified ECHO model. J Am Med Dir Assoc 2021; 22 : 238–244.e1.33238143
134 Lindauer A Wild K Natonson A , et al. Dementia 360 ECHO: using technology to facilitate diagnosis and treatment. Gerontol Geriatr Educ 2022; 43 : 202–208.33078687
135 Watts SA Roush L Julius M , et al. Improved glycemic control in veterans with poorly controlled diabetes mellitus using a specialty care access network-extension for community healthcare outcomes model at primary care clinics. J Telemed Telecare 2016; 22 : 221–224.26253745
136 Bouchonville MF Hager BW Kirk JB , et al. Endo ECHO improves primary care provider and community health worker self-efficacy in complex diabetes management in medically underserved communities. Endocr Pract 2018; 24 : 40–46.29368967
137 Arora S Thornton K Murata G , et al. Outcomes of treatment for hepatitis C virus infection by primary care providers. N Engl J Med 2011; 364 : 2199–2207.21631316
138 Nipp CM Vogtle LK Warren M . Clinical application of low vision rehabilitation strategies after completion of a computer-based training module. Occup Ther Health Care 2014; 28 : 296–305.24971896
139 Momin B Mera J Essex W , et al. Implementation of liver cancer education among health care providers and community coalitions in the Cherokee nation. Prev Chronic Dis 2019; 16 : 1–7.
140 Gadomski A Anderson J Chung YK , et al. Full agonist opioid prescribing by primary care clinicians after buprenorphine training. Subst Abus 2022; 43 : 69–75.32150525
141 Hostutler CA Valleru J Maciejewski HM , et al. Improving pediatrician’s behavioral health competencies through the project ECHO teleconsultation model. Clin Pediatr (Phila) 2020; 59 : 1049–1057.32506939
142 Sockalingam S Arena A Serhal E , et al. Building provincial mental health capacity in primary care: an evaluation of a project ECHO mental health program. Acad Psychiatry 2018; 42 : 451–457.28593537
143 Bellesheim KR Kizzee RL Curran A , et al. ECHO Autism: integrating maintenance of certification with extension for community healthcare outcomes improves developmental screening. J Dev Behav Pediatr 2020; 41 : 420–427.32735419
144 Zittleman L Curcija K Sutter C , et al. Building capacity for medication assisted treatment in rural primary care practices: the IT MATTTRs practice team training. J Prim Care Community Health 2020; 11 : 2150132720953723.32909491
145 Shaikh U Nettiksimmons J Joseph JG , et al. Collaborative practice improvement for childhood obesity in rural clinics: the healthy eating active living telehealth community of practice (HEALTH COP). Am J Med Qual 2014; 29 : 467–475.24170936
146 McFarland LV Raugi GJ Taylor LL , et al. Implementation of an education and skills programme in a teledermatology project for rural veterans. J Telemed Telecare 2012; 18 : 66–71.22198956
147 Doyle C Jackson D Loi S , et al. Videoconferencing and telementoring about dementia care: evaluation of a pilot model for sharing scarce old age psychiatry resources. Int Psychogeriatr 2016; 28 : 1567–1574.27189501
148 McWilliams T Hendricks J Twigg D , et al. Burns education for non-burn specialist clinicians in Western Australia. Burns 2015; 41 : 301–307.25239848
149 Seibert PS Reddy T Whitmore T , et al. The use of telemedicine to train perioperative nurses in rural settings. J Telemed Telecare 2013; 19 : 311–314.24163294
150 Puskar KR Heeyoung L Mitchell AM , et al. Interprofessional collaborative education for substance use screening: rural areas and challenges. Online J Rural Nurs Health Care 2016; 16 : 76–96.
151 Bennett-Levy J Hawkins R Perry H , et al. Online cognitive behavioural therapy training for therapists: outcomes, acceptability, and impact of support. Aust Psychol 2012; 47 : 174–182.
152 Robinson T Hills D Kelly B . The evaluation of an online orientation to rural mental health practice in Australia. J Psychiatr Ment Health Nurs 2011; 18 : 629–636.21848598
153 Gill CJ Le Ngoc B Halim N , et al. The mCME project: a randomized controlled trial of an SMS-based continuing medical education intervention for improving medical knowledge among Vietnamese community based physicians’ assistants. PLoS One 2016; 11 : e0166293.
154 CFIR Research Team-Center for Clinical Management Research. Consolidated Framework for Implementation Research. (2021, accessed 12 July 2021).
155 Jiang B Bills M Poon P . Integrated telehealth-assisted home-based specialist palliative care in rural Australia: a feasibility study. J Telemed Telecare 2020: 1357633X20966466. DOI: 10.1177/1357633X20966466.
156 Fortney JC Pyne JM Mouden SB , et al. Practice-based versus telemedicine-based collaborative care for depression in rural federally qualified health centers: a pragmatic randomized comparative effectiveness trial. Am J Psychiatry 2013; 170 : 414–425.23429924
157 Tran L Feldman R Riley T III , et al . Association of the extension for community healthcare outcomes project with use of direct-acting antiviral treatment among US adults with hepatitis C. JAMA Netw Open 2021; 4 : e2115523.
158 Arora S Geppert CM Kalishman S , et al. Academic health center management of chronic diseases through knowledge networks: project ECHO. Acad Med 2007; 82 : 154–160.17264693
159 Arora S Thornton K Jenkusky SM , et al. Project ECHO: Linking university specialists with rural and prison-based clinicians to improve care for people with chronic hepatitis C in New Mexico. Public Health Rep 2007; 122 : 74–77.17542458
160 Arora S Kalishman S Dion D , et al. Partnering urban academic medical centers and rural primary care clinicians to provide complex chronic disease care. Health Aff (Millwood) 2011; 30 : 1176–1184.21596757
161 Katzman JG Gygi K Swift R , et al. How hands-on pain skills intensive trainings complement ECHO pain and opioid management programs: a program evaluation with the Indian Health Service. Pain Med 2020; 21 : 1769–1778.32556294
162 Zhu X Merchant KAS Mohr NM , et al. Real-time learning through telemedicine enhances professional training in rural emergency departments. Telemed e-Health 2021; 27 : 441–447.
163 Weigel P Bhagianadh D Merchant KA , et al. Tele-emergency behavioural health in rural and underserved areas. J Telemed Telecare 2021; 27 : 453–462.31726903
164 Weigel PA Merchant KA Wittrock A , et al. Paediatric tele-emergency care: a study of two delivery models. J Telemed Telecare 2021; 27 : 23–31.30966860
165 Weaver MS Neumann ML Navaneethan H , et al. Human touch via touchscreen: rural nurses’ experiential perspectives on telehealth use in pediatric hospice care. J Pain Symptom Manage 2020; 60 : 1027–1033.32525081
166 Shea CM Gertner AK Green SL . Barriers and perceived usefulness of an ECHO intervention for office-based buprenorphine treatment for opioid use disorder in North Carolina: a qualitative study. Subst Abus 2021; 42 : 54–64.31809679
167 Peracca SB Jackson GL Lamkin RP , et al. Implementing teledermatology for rural veterans: an evaluation using the RE-AIM framework. Telemed e-Health 2021; 27 : 218–226.
168 Mahmoud H Vogt EL Dahdouh R , et al. Using continuous quality improvement to design and implement a telepsychiatry program in rural Illinois. Psychiatr Serv 2020; 71 : 860–863.32290807
169 Brunet N Moore DT Lendvai Wischik D , et al. Increasing buprenorphine access for veterans with opioid use disorder in rural clinics using telemedicine. Subst Abus 2022; 43 : 39–46.32078492
170 Simpson AN Harvey JB DiLembo SM , et al. Population health indicators associated with a statewide telestroke program. Telemed e-Health 2020; 26 : 1126–1133.
171 Tynan A Deeth L McKenzie D , et al. Integrated approach to oral health in aged care facilities using oral health practitioners and teledentistry in rural Queensland. Aust J Rural Health 2018; 26 : 290–294.
172 Adcock AK Choi J Alvi M , et al. Expanding acute stroke care in rural America: a model for statewide success. Telemed e-Health 2020; 26 : 865–871.
173 Lesher AP Fakhry SM DuBose-Morris R , et al. Development and evolution of a statewide outpatient consultation service: leveraging telemedicine to improve access to specialty care. Popul Health Manag 2020; 23 : 20–28.31161963
174 Thies KM Anderson D Beals-Reid C . Project ECHO chronic pain: a qualitative analysis of recommendations by expert faculty. Pain Med 2019; 20 : 1450–1452.30840075
175 Jewer J Parsons MH Dunne C , et al. Evaluation of a mobile telesimulation unit to train rural and remote practitioners on high-acuity low-occurrence procedures: pilot randomized controlled trial. J Med Internet Res 2019; 21 : e14587.
176 White AH Crowther SA Lee SH . Supporting rural midwifery practice using a mobile health (mHealth) intervention: a qualitative descriptive study. Rural Remote Health 2019; 19 : 5294.31446762
177 Cronin T . Implementing a stroke program using telemedicine. J Emerg Nurs 2013; 39 : 613–618.22748668
178 Marsh-Feiley G Eadie L Wilson P . Paramedic and physician perspectives on the potential use of remotely supported prehospital ultrasound. Rural Remote Health 2018; 18 : 4574.30207737
179 Sabesan S Senko C Schmidt A , et al. Enhancing chemotherapy capabilities in rural hospitals: implementation of a telechemotherapy model (QReCS) in North Queensland, Australia. J Oncol Pract 2018; 14 : e429–e437.
180 Tynan A Deeth L McKenzie D . An integrated oral health program for rural residential aged care facilities: a mixed methods comparative study. BMC Health Serv Res 2018; 18 : 515.29970073
181 Hensel JM Yang R Rai M , et al. Optimizing electronic consultation between primary care providers and psychiatrists: mixed-methods study. J Med Internet Res 2018; 20 : e124.
182 Trondsen MV Tjora A Broom A , et al. The symbolic affordances of a video-mediated gaze in emergency psychiatry. Soc Sci Med 2018; 197 : 87–94.29222999
183 Ness TE Annese MF Martinez-Paz N , et al. Using an innovative telehealth model to support community providers who deliver perinatal HIV care. AIDS Educ Prev 2017; 29 : 516–526.29283274
184 Bagot KL Cadilhac DA Kim J , et al. Transitioning from a single-site pilot project to a state-wide regional telehealth service: the experience from the Victorian Stroke Telemedicine programme. J Telemed Telecare 2017; 23 : 850–855.29081268
185 Newell MC Strauss CE Freier T , et al. Design and initial results of the Minneapolis Heart Institute TeleHeart program. Circ Cardiovasc Qual Outcomes 2017; 10 : 1–4.
186 Narva AS Romancito G Faber T , et al. Managing CKD by telemedicine: the Zuni telenephrology clinic. Adv Chronic Kidney Dis 2017; 24 : 6–11.28224944
187 Williams KM Kirsh S Aron D , et al. Evaluation of the Veterans Health Administration’s specialty care transformational initiatives to promote patient-centered delivery of specialty care: a mixed-methods approach. Telemed e-Health 2017; 23 : 577–589.
188 Al Kasab S Adams RJ Debenham E , et al. Medical university of South Carolina telestroke: a telemedicine facilitated network for stroke treatment in South Carolina-a progress report. Telemed e-Health 2017; 23 : 674–677.
189 Ray KN Felmet KA Hamilton MF , et al. Clinician attitudes toward adoption of pediatric emergency telemedicine in rural hospitals. Pediatr Emerg Care 2017; 33 : 250–257.26785087
190 Evans K Lerch S Boyce TW , et al. An innovative approach to enhancing access to medical screening for miners using a mobile clinic with telemedicine capability. J Health Care Poor Underserved 2016; 27 : 62–72.27818414
191 Hofmeyer J Leider JP Satorius J , et al. Implementation of telemedicine consultation to assess unplanned transfers in rural long-term care facilities, 2012-2015: a pilot study. J Am Med Dir Assoc 2016; 17 : 1006–1010.27477614
192 Ward MM Ullrich F MacKinney AC , et al. Tele-emergency utilization: In what clinical situations is tele-emergency activated? J Telemed Telecare 2016; 22 : 25–31.26026189
193 Jhaveri D Larkins S Kelly J , et al. Remote chemotherapy supervision model for rural cancer care: perspectives of health professionals. Eur J Cancer Care (Engl) 2016; 25 : 93–98.25871852
194 Pimentel CB Gately M Barczi SR , et al. GRECC Connect: geriatrics telehealth to empower health care providers and improve management of older veterans in rural communities. Fed Pract 2019; 36 : 464–470.31768097
195 Habashi P Bouchard S Nguyen GC . Transforming access to specialist care for inflammatory bowel disease: the PACE telemedicine program. J Can Assoc Gastroenterol 2019; 2 : 186–194.31616860
196 Alschuler KN Stobbe GA Hertz DP , et al. Impact of multiple sclerosis project ECHO (extension for community healthcare outcomes) on provider confidence and clinical practice. Int J MS Care 2019; 21 : 143–150.31474806
197 Pindyck T Kalishman S Flatow-Trujillo L , et al. Treating hepatitis C in American Indians/Alaskan Natives: a survey of project ECHO(R) (extension for community healthcare outcomes) utilization by Indian Health Service providers. Sage Open Med 2015; 3 : 2050312115612805.26770809
198 Singh R Mathiassen L Switzer JA , et al. Assimilation of web-based urgent stroke evaluation: a qualitative study of two networks. JMIR Med Inform 2014; 2 : e6.
199 MacKinney AC Ward MM Ullrich F , et al. The business case for tele-emergency. Telemed e-Health 2015; 21 : 1005–1011.
200 Hilt RJ Barclay RP Bush J , et al. A statewide child telepsychiatry consult system yields desired health system changes and savings. Telemed e-Health 2015; 21 : 533–537.
201 Cadilhac DA Vu M Bladin C . Experience with scaling up the Victorian Stroke Telemedicine programme. J Telemed Telecare 2014; 20 : 413–418.25400003
202 Trondsen MV Bolle SR Stensland GO , et al. Video-confidence: a qualitative exploration of videoconferencing for psychiatric emergencies. BMC Health Serv Res 2014; 14 : 544.25359404
203 Ramnath VR Khazeni N . Centralized monitoring and virtual consultant models of tele-ICU care: a side-by-side review. Telemed e-Health 2014; 20 : 962–971.
204 Kulcsar M Gilchrist S George MG . Improving stroke outcomes in rural areas through telestroke programs: an examination of barriers, facilitators, and state policies. Telemed e-Health 2014; 20 : 3–10.
205 Khalil H Cullen M Chambers H , et al. Implementation of a successful electronic wound documentation system in rural Victoria, Australia: a subject of collaboration and community engagement. Int Wound J 2014; 11 : 314–318.23418740
206 Johansson AM Lindberg I Soderberg S . The views of health-care personnel about video consultation prior to implementation in primary health care in rural areas. Prim Health Care Res Dev 2014; 15 : 170–179.23402617
207 Conn DK Madan R Lam J , et al. Program evaluation of a telepsychiatry service for older adults connecting a university-affiliated geriatric center to a rural psychogeriatric outreach service in Northwest Ontario, Canada. Int Psychogeriatr 2013; 25 : 1795–1800.23870297
208 Haozous E Doorenbos AZ Demiris G , et al. Role of telehealth/videoconferencing in managing cancer pain in rural American Indian communities. Psycho-Oncol 2012; 21 : 219–223.
209 Moffatt JJ Eley DS . Barriers to the up-take of telemedicine in Australia--a view from providers. Rural Remote Health 2011; 11 : 1581.21385004
210 Lucas JAM Day K Honey MLL . Clinician’s perceptions of telehealth for emergency care on the west coast of New Zealand: findings of a descriptive study. Emergency Nurse New Zealand 2016: 6–10.
211 Nqala MO Rout CC Aldous CM . Remote clinical support by telephone for rural district hospital medical officers in the Eastern Cape. S Afr Fam Pract 2015; 57 : 286–290.
212 Olenik K Lehr B . Counteracting brain drain of health professionals from rural areas via teleconsultation: analysis of the barriers and success factors of teleconsultation. J Public Health 2013; 21 : 357–364.
213 Ritter LA Robinette TR Cofano J . Evaluation of a statewide telemedicine program. Calif J Health Promot 2010; 8 : 1–9.25750596
214 Pandit T Ray RA Sabesan S . Managing emergencies in rural North Queensland: the feasibility of teletraining. Int J Telemed Appl 2018; 2018 : 1–7.
215 Rogove HJ McArthur D Demaerschalk BM , et al. Barriers to telemedicine: survey of current users in acute care units. Telemed e-Health 2012; 18 : 48–53.
216 Wood T Freeman S Banner D , et al. Exploring user perspectives of factors associated with use of teletrauma in rural areas. Australas Emerg Care 2022; 25 : 106–114.33972192
217 Banbury A Smith AC Mehrotra A , et al. A comparison study between metropolitan and rural hospital-based telehealth activity to inform adoption and expansion. J Telemed Telecare 2021: 1357633X21998201. DOI: 10.1177/1357633X21998201.
218 Morrissette S Pearlman RL Kovar M , et al. Attitudes and perceived barriers toward store-and-forward teledermatology among primary care providers of the rural Mississippi. Arch Dermatol Res 2022; 314 : 37–40.33630147
219 Howland M Tennant M Bowen DJ , et al. Psychiatrist and psychologist experiences with telehealth and remote collaborative care in primary care: a qualitative study. J Rural Health 2021; 37 : 780–787.33022079
220 Haque SN DeStefano S Banger A , et al. Factors influencing telehealth implementation and use in frontier critical access hospitals: qualitative study. JMIR Form Res 2021; 5 : e24118.
221 Parks J Hunter A Taylor A , et al. Design, development and implementation of the virtual, coordination, access, referral and escalation service in western New South Wales. Aust J Rural Health 2021; 29 : 794–800.34586702
222 Luscombe GM Hawthorn J Wu A , et al. ‘Empowering clinicians in smaller sites’: a qualitative study of clinician’s experiences with a rural virtual paediatric feeding clinic. Aust J Rural Health 2021; 29 : 742–752.34490941
223 Lim M Liberali SAC Calache H , et al. Specialist networks influence clinician willingness to treat individuals with special needs. JDR Clin Trans Res 2022; 7 : 267–276.34148391
224 Powell KR Alexander GL . Consequences of rapid telehealth expansion in nursing homes: promise and pitfalls. Appl Clin Inform 2021; 12 : 933–943.34614517
225 Mundt AP Irarrazaval M Martinez P , et al. Telepsychiatry consultation for primary care treatment of children and adolescents receiving child protective services in Chile: mixed methods feasibility study. JMIR Public Health Surveill 2021; 7 : e25836.
226 Becevic M Smith E Golzy M , et al. Melanoma extension for community healthcare outcomes: a feasibility study of melanoma screening implementation in primary care settings. Cureus 2021; 13 : e15322.
227 May S Jonas K Fehler GV , et al. Challenges in current nursing home care in rural Germany and how they can be reduced by telehealth - an exploratory qualitative pre-post study. BMC Health Serv Res 2021; 21 : 925.34488746
