
==== Front
JAMIA Open
JAMIA Open
jamiaoa
JAMIA Open
2574-2531
Oxford University Press

39100988
10.1093/jamiaopen/ooae068
ooae068
Review
AcademicSubjects/SCI01530
AcademicSubjects/MED00010
AcademicSubjects/SCI01060
Barriers and facilitators to the uptake of electronic collection and use of patient-reported measures in routine care of older adults: a systematic review with qualitative evidence synthesis
Hettiarachchi Senarath Gayanika M Department of Human-Centred Computing, Faculty of Information Technology, Monash University, Clayton, VIC 3800, Australia

Delir Haghighi Pari Department of Human-Centred Computing, Faculty of Information Technology, Monash University, Clayton, VIC 3800, Australia

Bai Lu Department of Human-Centred Computing, Faculty of Information Technology, Monash University, Clayton, VIC 3800, Australia

https://orcid.org/0000-0002-6805-078X
Shannon Michelle M Academic Unit, Frankston Hospital, Peninsula Health, Frankston, VIC 3199, Australia

Andrew Nadine E Academic Unit, Frankston Hospital, Peninsula Health, Frankston, VIC 3199, Australia
Department of Medicine, Peninsula Clinical School, Central Clinical School, Monash University, Frankston, VIC 3199, Australia
National Centre for Healthy Ageing, Frankston, VIC 3199, Australia

Srikanth Velandai Academic Unit, Frankston Hospital, Peninsula Health, Frankston, VIC 3199, Australia
Department of Medicine, Peninsula Clinical School, Central Clinical School, Monash University, Frankston, VIC 3199, Australia
National Centre for Healthy Ageing, Frankston, VIC 3199, Australia

Snowdon David A Academic Unit, Frankston Hospital, Peninsula Health, Frankston, VIC 3199, Australia
Department of Medicine, Peninsula Clinical School, Central Clinical School, Monash University, Frankston, VIC 3199, Australia
National Centre for Healthy Ageing, Frankston, VIC 3199, Australia

O’Connor Denise A School of Public Health and Preventive Medicine, Monash University, Melbourne, VIC 3004, Australia

Corresponding author: Gayanika M. Hettiarachchi Senarath, Department of Human-Centred Computing, Faculty of Information Technology, 25 Exhibition Walk (Building 63), Monash University, Clayton, Victoria 3800, Australia (gayanika.hettiarachchisenarath@monash.edu)
10 2024
02 8 2024
02 8 2024
7 3 ooae06809 1 2024
20 5 2024
01 7 2024
15 7 2024
© The Author(s) 2024. Published by Oxford University Press on behalf of the American Medical Informatics Association.
2024
https://creativecommons.org/licenses/by/4.0/ This is an Open Access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted reuse, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Objective

The aims of this systematic review were to (1) synthesize the available qualitative evidence on the barriers and facilitators influencing implementation of the electronic collection and use of patient-reported measures (PRMs) in older adults’ care from various stakeholder perspectives and (2) map these factors to the digital technology implementation framework Non-adoption, Abandonment, challenges to the Scale-up, Spread, Sustainability (NASSS) and behavior change framework Capability, Opportunity, Motivation, Behaviour (COM-B).

Materials and Methods

A search of MEDLINE, CINAHL Plus, and Web of Science databases from 1 January 2001 to 27 October 2021 was conducted and included English language qualitative studies exploring stakeholder perspectives on the electronic collection and use of PRMs in older adults’ care. Two authors independently screened studies, conducted data extraction, quality appraisal using the Critical Appraisal Skills Programme (CASP), data coding, assessed confidence in review findings using Grading of Recommendations Assessment, Development, and Evaluation Confidence in the Evidence from Reviews of Qualitative Research (GRADE CERQual), and mapped the findings to NASSS and COM-B. An inductive approach was used to synthesize findings describing the stakeholder perspectives of barriers and facilitators.

Results

Twenty-two studies were included from the 3368 records identified. Studies explored older adult, caregiver, healthcare professional, and administrative staff perspectives. Twenty nine of 34 review findings (85%) were graded as having high or moderate confidence. Key factors salient to older adults related to clinical conditions and socio-cultural factors, digital literacy, access to digital technology, and user interface. Factors salient to healthcare professionals related to resource availability to collect and use PRMs, and value of PRMs collection and use.

Conclusion

Future efforts to implement electronic collection and use of PRMs in older adults’ care should consider addressing the barriers, facilitators, and key theoretical domains identified in this review. Older adults are more likely to adopt electronic completion of PRMs when barriers associated with digital technology access, digital literacy, and user interface are addressed. Future research should explore the perspectives of other stakeholders, including those of organizational leaders, digital technology developers and implementation specialists, in various healthcare settings and explore factors influencing implementation of PREMs.

PROSPERO registration number

CRD42022295894

electronic patient reported outcome measure
electronic patient reported experience measure
barriers
facilitators
implementation science
==== Body
pmcIntroduction

Patient Reported Outcome Measures (PROMs) and Patient Reported Experience Measures (PREMs), collectively referred to as Patient Reported Measures (PRMs), capture patient perceptions of health (eg, symptoms, functional status) and experiences (eg, accessibility of services, patient-healthcare professional interaction) with healthcare services. PROMs are validated tools in the form of questionnaires used to report the treatment outcomes perceived by patients (also known as Patient Reported Outcomes (PROs)).1 Consideration of PRMs is integral to achieving value-based care for older adults given these individuals frequently experience multimorbidity and are more likely to have complex care needs.2

PRMs are used to monitor population health, assess the effects of clinical interventions in randomized trials, and more recently to inform the provision of routine clinical care.3 In routine care, they seek to complement clinical and health provider reported outcomes by providing data about patients’ perceptions of their own health, and their needs, preferences and values.2 Several systematic reviews have found that use of PRMs in routine clinical care leads to improved patient-provider communication and greater patient satisfaction, and may improve monitoring of treatment response and detection of unrecognized problems.4–6 In oncology settings, the use of PROMs has improved management of patient symptoms and reduced hospitalizations.7 A potential benefit of using PREMs in clinical practice is the quality improvement of care processes.8 Studies have reported the use of PRM tools in the routine care of older adults in community, palliative, outpatient, and emergency care settings.9–11

Despite the demonstrated benefits of using PRMs in clinical practice, their sustained implementation in routine care, including for older adults, remains suboptimal.12,13 Electronic collection and use of PRMs have been advocated as an alternative to traditional, paper-based PRM methods to increase uptake, and reduce administrative burden and costs in routine older adults’ care.14,15 Benefits of the electronic mode include streamlining of data collection processes, improving integration with existing information systems, and enhancing accessibility for healthcare providers.14,16,17 However, electronic completion of PRMs pose unique challenges in the context of older adults’ care, such as aging related visual, cognitive, and functional limitations.10 While a review of the factors influencing uptake of electronic administration of PRMs across health care more broadly is available,18 a systematic review of the barriers and facilitators to the uptake of electronic administration of PRMs specifically in the context of older adults’ care is needed to identify the unique issues faced in this context and key targets for change when designing implementation efforts. The synthesis of evidence from primary qualitative studies is ideally suited for establishing a greater understanding of the issues influencing implementation, through a rich interpretation of experiences.19

Therefore, the aims of this systematic review were to: (1) synthesize the available qualitative evidence on the barriers and facilitators influencing the uptake of electronic collection and use of PRMs in older adults’ care from the perspectives of older adults and/or their caregivers and healthcare service staff and (2) to map these factors to contemporary implementation science frameworks to inform future implementation efforts.

Methods

Protocol development and registration

A systematic review protocol20 was developed in accordance with the Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols (PRISMA-P) 2015 statement21 and the methods described by the Cochrane Qualitative and Implementation Methods Groups and the Cochrane Handbook.22 The manuscript is reported in accordance with PRISMA reporting guidelines23,24 and the Enhancing Transparency in Reporting the Synthesis of Qualitative Research statement (ENTREQ).25

Eligibility criteria

The study inclusion and exclusion criteria (Supplementary Appendix S1) were developed using the Perspective, Setting, Phenomenon of interest, Environment, Comparison, Time/Timing and Findings (PerSPecTIF) question framework for evaluating evidence relevant to complex interventions.26 Peer-reviewed, English language full-text qualitative or mixed-method studies were included if:

qualitative data collection and analysis methods were used;

participants included older adults aged 65 years and above, or stakeholders (eg, caregivers, healthcare service staff such as healthcare professionals and administrative staff) involved in the care of older adults;

the setting was any healthcare setting and any environment except clinical trials;

reported outcomes included experiences, attitudes, preferences, beliefs and perceptions of electronic collection and/or use of PRMs in older adults’ care from any of the stakeholders specified above; and

the stage of digital change included planning, development, implementation or use.

Search methods

MEDLINE (via OVID), Cumulative Index to Nursing and Allied Health Literature (CINAHL) Plus (via EBSCO Host), and Web of Science databases were searched from 1 January 2001 to 27 October 2021. The search strategy (Supplementary Appendix S2) included three main concepts: “patient-reported measures,” “electronic surveys or questionnaires,” and “qualitative research.” Additional potentially relevant studies were identified by searching the reference lists of the included studies.

Selection of studies

All citations and potentially relevant full-text articles were independently screened by two authors (GHS or PDH). Any disagreements were resolved through discussion with a third review author (DOC). The search and screening results were summarized in a PRISMA flow diagram.24

Quality assessment

The quality of the included studies was independently assessed by two of three authors (GHS, MMS, LB) using the Critical Appraisal Skills Programme (CASP) checklist.27 Any disagreements were resolved through discussion with a third review author (DAS or DOC). All eligible studies were included irrespective of quality.

Data extraction

Two of three authors (GHS, MMS, LB) independently extracted qualitative data from included studies (ie, primary results and secondary analysis) using a standardized data collection form. This was imported into an excel spreadsheet and Nvivo 12 software28 for coding.

An inductive coding approach29 was used to code the extracted data “line-by-line” and derive codes as each study was reviewed (first step). Descriptive findings were then developed from these codes (second step). All studies were independently coded by two of three authors (GHS, MMS, LB). The findings, consisting of barriers and facilitators, were developed by the primary author and reviewed and refined by the wider review team. Findings were categorized as those unique to the electronic collection and use of PRMs, and those related to PRMs implementation in general.

The barriers and facilitators were then independently mapped to two implementation science theoretical frameworks by two authors (GHS and LB). Non-adoption, Abandonment, challenges to the Scale-up, Spread and Sustainability (NASSS), a healthcare digital technology change implementation framework30 and Capability, Opportunity, Motivation, Behaviour (COM-B), a behavior change framework31 were selected and used in combination in this review. NASSS was considered relevant for understanding multi-level factors (ie, organizational and wider context) influencing digital innovations in healthcare and COM-B was considered relevant for understanding factors influencing individual-level adopter behavior. Findings were first mapped to NASSS domains, and this mapping was then aligned with relevant COM-B domains. The mapping of the barriers and facilitators to the theoretical domains was discussed and refined until consensus was achieved amongst the review team.

Confidence assessment

Two of three authors (GHS, MMS, LB) independently assessed the confidence in the review findings using the Grading of Recommendations Assessment, Development, and Evaluation Confidence in the Evidence from Reviews of Qualitative Research (GRADE CERQual) tool.32 The following four components were considered in assessing the confidence in each review finding:

The extent to which there are methodological limitations of studies contributing to a review finding based on CASP assessments.

The extent to which data from studies supporting a review finding are relevant to the review question (ie, population, phenomenon of interest).

The extent to which the review finding is coherent (ie, well-supported) with data from studies.

The determination that data from studies supporting review finding is adequate (ie, richness and quantity of data).

Each component was individually rated as no or very minor concerns, minor concerns, moderate concerns, or serious concerns. A judgement on the overall confidence of a review finding was made either as high (ie, highly likely that the review finding reasonably represents the phenomenon of interest); moderate (ie, likely that the review finding reasonably represents the phenomenon of interest); low (ie, possible that the review finding reasonably represents the phenomenon of interest); or very low (ie, unclear if the review finding reasonably represents the phenomenon of interest). The confidence assessment started as high by default and was downgraded depending on the severity of concerns and number of domains showing concerns. Discrepancies were resolved through discussion among the review team.

Results

Results of search and study selection

As shown in the PRISMA flowchart (Figure 1), we identified 3368 titles and abstracts after removing duplicates from the electronic database searches, of which 107 full-text records were screened for inclusion. We excluded 85 studies due to: lack of relevance to electronic collection and use of PRMs (n = 24), no reference to older adults (n = 16), qualitative methods not utilized (n = 41), was a protocol (n = 3), or conference proceeding (n = 1). Twenty-two studies met our inclusion criteria and were included in this review.

Figure 1. PRISMA Chart. Created by the authors in accordance with PRISMA guidelines. Abbreviations: CINAHL = Cumulative Index to Nursing and Allied Health Literature; PRISMA = Preferred Reporting Items of Systematic Reviews and Meta-Analysis; PRM = patient-reported measures.

Description of included studies

The characteristics of the included studies are described in Table 1 (Supplementary Appendix S3 provides additional information). Studies were conducted in the United States (n = 9),10,36–38,40,43,47,51 United Kingdom (n = 1),33 Netherlands (n = 2),34,50 Canada (n = 5),35,39,41,45,52 Denmark (n = 4),42,44,46,48 and Austria (n = 1).49 The studies sampled participants from the outpatient setting (n = 19),10,33–36,38,40–50,52,53 inpatient setting (n = 1),39 and a combination of inpatient and outpatient settings (n = 1).51 The healthcare setting could not be determined for one study.37 Studies explored patients’ perspectives (n = 9),33,37,40,42,45,47,48,49,52 healthcare professionals’ perspectives (n = 6),34,38,39,41,50,53 patients’ and healthcare professionals’ perspectives (n = 4),43,44,46,51 patients’, healthcare professionals’ and administrative staff perspectives (n = 1),35 healthcare professionals’ and administrative staff perspectives (n = 1)36 and patients’, caregivers’, and administrative staff perspectives (n = 1).10 The studies included 374 patients, 183 healthcare professionals, 3 caregivers, and 5 administrative staff. Fifteen studies included qualitative interviews,10,35–37,41,43,45–53 three used interviews and focus groups,39,42,44 and single studies used either focus groups,40 surveys with open-ended questions,34 interviews and observations,33 or interviews and surveys with open-ended questions.38

Table 1. Characteristics of included studies (N = 22)

Reference and country	Healthcare setting	Study Type	Patients’ characteristics	Other stakeholders’ characteristics	Data collection method(s)	Data analysis method	Stage of digital change focused	Methodology limitations a	
Long et al10

USA

	Outpatient—orthopaedic surgery	Qualitative	• Patients (n = 12)

• Age range: 20-81 years (not limited to patients); Median age: 44.5 years

	• Caregivers (n = 3)

• Administrative staff (n = 2)

	Semi-structured interviews	Framework analysis	Use (outpatient visit related)	None	
Aiyegbusi et al33

UK

	Outpatient—nephrology	Mixed-methods	• Patients (n = 8)

• Age range: 36-87 years; Mean age: 64.3 years

	–	Interviews and observations	Content analysis	Development (usability assessment)	Major

(Research design, data collection, data analysis)

	
Amini et al34

Netherlands

	Outpatient—multidisciplinary care	Mixed-methods	–	• Healthcare professionals (n = 61):

• Physicians (n = 31)

• Nursing (n = 15)

• Allied health (n = 7)

• Physician researchers (n = 2)

• Physician assistant (n = 2)

• Other (n = 4)

	Survey containing open-ended questions	Content analysis	Planning, Development and Implementation	Major

(Recruitment strategy, researcher-participant relationship, data analysis)

	
Kaur et al35

Canada

	Outpatient—cosmetic surgery	Qualitative	• Patients (n = 11)

• Age range: 18-83 years; Mean age: 43 years

	• Healthcare professionals: Surgeons (n = 3)

• Administrative staff (n = not reported)

	Semi-structured interviews	Thematic analysis using inductive approach	Development	Major

(Recruitment strategy, data collection, researcher-participant relationship, data analysis)

	
Spaulding et al36

USA

	Outpatient—psychiatry	Qualitative	• Age range: 19-100 years; Mean age: 53.3 years

	• Healthcare professionals (n = 6):

• Physicians (n = 4)

• Nursing (n = 2)

 • Administrative staff: Front desk staff (n = 3)

	Semi-structured interviews	Thematic analysis using deductive approach, guided by RE-AIM framework	Implementation	Minor

(Researcher-participant relationship)

	
Delgado-Herrera et al37

USA

	Setting unclear—gastroenterology	Qualitative	• Patients (n = 25)

• Age range: 26.1-79.2 years; Mean age: 52.7 years

	–	Open-ended interviews	Content analysis	Development (usability assessment)	Minor

(Recruitment strategy, researcher-participant relationship)

	
Mou et al38

USA

	Outpatient—multidisciplinary care	Mixed-methods	–	• Healthcare professionals: Primary Care Providers (n = 28)

	Semi-structured interviews

Survey containing open-ended questions

	Thematic analysis using a deductive approach

Content analysis

	Implementation	Minor

(Recruitment strategy, researcher-participant relationship)

	
Krawczyk et al39

Canada

	Inpatient (hospital-based acute care)—palliative care	Qualitative	–	• Healthcare professionals (n = 25):

• Physicians (n = 2)

• Nursing (n = 19)

• Allied Health (n = 4)

	Focus groups and interviews	Thematic analysis using inductive approach, guided by micro-meso-macro framework	Development (design), Implementation and Use	Minor

(Recruitment strategy, researcher-participant relationship)

	
Navarro-Millán et al40

USA

	Outpatient—rheumatology	Qualitative	• Patients (n = 31)

• Age range: 25-84 years; Mean age: 51 years

	–	Focus groups	Thematic analysis using deductive approach, guided by Andersen-Newman framework	Planning	Minor

(Researcher-participant relationship)

	
Schick-Makaroff and Molzahn41

Canada

	Outpatient—nephrology	Mixed-methods	• Age range: 34-88 years; Mean age: 67 years

	• Healthcare professionals:

Nurses (n = 11)

	Semi-structured interviews	Interpretive description using inductive approach	Use (outpatient visit related)	None	
Brochmann et al42

Denmark

	Outpatient—oncology (hematology)	Mixed-methods	• Patients (n = 17): Above 60 years (n = 8)

• Age range: 39-80 years; Mean age of patients 60 years and above: 72 years

	–	Focus groups and interviews	Content analysis using deductive approach	Development (feasibility)	Major

(Research design, researcher-participant relationship, data analysis)

	
Snyder et al43

USA

	Outpatient—oncology	Mixed-methods	• Patients (n = 47)

• Age range: 28-81 years; Median age: 58 years

	• Healthcare professionals: Clinicians (n = 11)

	Open-ended interviews	Content analysis	Development (pilot-testing)	Major

(Researcher-participant relationship, data analysis, statement of findings)

	
Tolstrup et al44

Denmark

	Outpatient—oncology	Mixed-methods	• Patients (n = 14)

• Age range: 41-79 years; Median age: 67 years

	• Healthcare professionals (n = 5):

• Physicians (n = 3)

• Nurses (n = 2)

	Semi-structured interviews and Focus group	Content analysis using deductive approach	Implementation	None	
Yamada et al45

Canada

	Outpatient—respirology	Qualitative	• Patients (n = 12)

• Age range: 25–60 years and above

	–	Semi-structured interviews	Directed content analysis guided by TDF	Use (outpatient visit related)	None	
Baeksted et al46

Denmark

	Outpatient—oncology	Mixed-methods	• Patients (n = 4)

• Age range: 56-79 years; Median age: 69 years

	• Healthcare professionals: Oncologists (n = 5)

	Semi-structured interviews	Thematic analysis using deductive approach	Development (feasibility)	Minor

(Data analysis)

	
Samuel et al47

USA

	Outpatient—oncology	Mixed-methods	• Patients (n = 40)

• Mean age: Black—62.1; White—66.9

	–	Semi-structured interviews	Thematic analysis using inductive approach	Use (outpatient visit related)	Minor

(Researcher-participant relationship)

	
Nielsen et al48

Denmark

	Outpatient—gastroenterology	Qualitative	• Patients (n = 16): 65 years and above (n = 2)

• Age range: 29-73 years; Mean age: 49.3 years

	–	Semi-structured interviews	Thematic analysis using abductive approach guided by the ReadyHy framework	Use (outpatient visit related)	None	
Lehmann et al49

Austria

	Outpatient—oncology	Mixed-methods	• Patients (n = 102)

• Age range: 39-83 years; Mean age: 59.9 years

	–	Semi-structured interviews	Content analysis	Use (outpatient visit related)	Minor

(Data analysis, statement of findings)

	
Duman-Lubberding et al50

Netherlands

	Outpatient—Oncology	Mixed-methods	• Mean age: 63.5 years

	• Healthcare professionals: Surgeons (n = 6)

	Semi-structured interviews	Thematic analysis using inductive approach	Use (outpatient visit related)	Major

(Research design, recruitment strategy, data collection, researcher-participant relationship, data analysis)

	
Grossman et al51

USA

	Inpatient and outpatient—cardiology	Mixed-methods	Stages 1 and 3:

• Patients (n = 25)

• Age range: 30-78 years; Median age: Stage 1-53 years; Stage 2-57 years

	Stage 1:

• Healthcare professionals (n = 11):

• Physicians (n = 8)

• Nurse practitioners (n = 3)

	Semi-structured interviews	Thematic analysis	Development (usability)	None	
Moradian et al52

Canada

	Outpatient—oncology	Mixed-methods	• Patients (n = 10)

• Age range: 18-78 years; Mean age: 68 years

	–	Semi-structured interviews	Thematic analysis	Development (usability)	Minor

(Researcher-participant relationship)

	
Sandhu et al53

USA

	Outpatient—oncology (multiple clinical domains)	Qualitative	–	• Healthcare professionals: Oncologists (n = 16)	Semi-structured interviews	Thematic analysis using inductive approach	Implementation	None	
Table created by the authors.

a Study quality assessment: “None”: all “yes” answers on CASP assessment; “minor”: ≤2 “no” answers on CASP assessment; “major”: >2 “no” answers on CASP assessment.

Abbreviations: RE-AIM = Reach, Effectiveness, Adoption, Implementation, Maintenance; TDF = Theoretical Domains Framework.

Methodological limitations of studies

The results of the CASP assessment (Supplementary Appendix S4) indicated that all studies had a clear statement of aims, used appropriate qualitative methodology and considered ethical issues. Most studies had appropriate research designs (n = 19), adequately justified data collection (n = 19), and provided a clear statement of findings (n = 19). Methodological limitations included inadequate explanation and/or consideration of the recruitment strategy (n = 7), inadequate explanation and/or consideration of the relationship between participants and researchers (n = 12), and insufficient rigor in data analysis (n = 7). Sixteen studies had minor or no methodological limitations and six studies were judged to have major methodological limitations.

Synthesis of findings

Thirty-four findings consisting of barriers and facilitators from the perspectives of different stakeholders were derived (Supplementary Appendix S5 provides detail on these findings). Supportive quotes for all findings are presented in Supplementary Appendix S6. The review findings were grouped into 11 thematic categories: (1) older adult’s characteristics; (2) digital technology; (3) support from social circle; (4) knowledge and skills; (5) motivation and incentives for capture and use of PRMs; (6) emotional experience; (7) older adults’ autonomy; (8) patient—healthcare professional communication; (9) workflow; (10) organizational factors; and (11) PRMs questionnaire selection and design. Findings were graded as high confidence (n = 3), moderate confidence (n = 26), and low confidence (n = 5). Findings that were graded as high confidence included: (1) user interface for healthcare professionals; (2) regular exposure enhancing health knowledge; and (3) questionnaire length and complexity of questions. The confidence of a review finding was typically downgraded due to methodological limitations, data adequacy limitations (ie, few studies contributing to the finding) and relevance (ie, low number of older adults in contributing studies). The confidence assessments are reported in the CERQual evidence profiles, in Supplementary Appendix S7.

Table 2 illustrates the barriers and facilitators from different stakeholder perspectives mapped to the theoretical domains of NASSS and COM-B frameworks. Barriers and facilitators unique to the electronic collection and use of PRMs are outlined in Table 3. Findings relevant to both electronic and general implementation of PRMs, such as support from social circle to complete PRMs, have been elaborated in supplemental appendix 5. A brief summary of the findings, organized by the NASSS framework with reference to relevant COM-B domains, is described below.

Table 2. Barriers and facilitators to uptake of electronic collection and use of PRMs mapped to the theoretical domains of NASSS and COM-B frameworks.

NASSS	COM-B	Older adults	Healthcare Professionals	Administrative staff (front desk)	Caregivers	
Patient conditiona: clinical condition, comorbidities, sociocultural and socio-economic aspects that determine individual’s ability to use digital technology	Capability

(Memory, attention and decision processes)

	Clinical conditions and socio-cultural factors—F1 (B)	Clinical conditions and socio-cultural factors—F1 (B)	Clinical conditions and socio-cultural factors—F1 (B)	Clinical conditions and socio-cultural factors—F1 (B)	
Capability

(Knowledge; Skills)

	Digital knowledge and skills—F10 (B)(F)

Health knowledge and literacy in general among older adults—F11 (B)

	Digital knowledge and skills—F10 (B)

Health knowledge and literacy in general among older adults—F11 (B)

	Digital knowledge and skills—F10 (B)

Health knowledge and literacy in general among older adults—F11 (B)

	Health knowledge and literacy in general among older adults—F11 (B)	
Opportunity

(Environmental context and resources)

	Access to digital technology—F2 (B)(F)	Access to digital technology—F2 (B)(F)	–	–	
Technology: digital technology equipment, features and functionality, knowledge generated knowledge and support needed, and supply model

	Opportunity

(Environmental context and resources)

	User interface for older adults—F3, F4 (B)(F)

Electronic device for PRMs completion—F6 (B)(F)

Technical challenges—F7 (B)

	User interface for older adults—F3, F4 (B)(F)

User interface for healthcare professionals—F5 (B)(F)

Technical challenges—F7 (B)

	User interface for older adults—F3 (F)	User interface for older adults—F4 (B)	
Motivation

(Beliefs about consequences)

	Privacy and security of personal data—F8 (B)(F)	Privacy and security of personal data—F8 (B)	–	–	
Value proposition: whether or not the digital technology is worth developing and implementing for older adults, healthcare professionals, healthcare providers and suppliers	Motivation

(Beliefs about consequences)

	PRMs utilization—F15, F16, F17, F18 (B)(F)

Location of PRMs collection—F20 (B)(F)

Timing and frequency of PRMs completion—F21 (B)(F)

Access to additional resources based on PRMs responses—F22 (B)(F)

Patient—healthcare professional communication—F25 (B)(F)

	PRMs utilization—F15, F16, F17, F18 (B)(F)

PRMs data completeness—F17, F18 (B)(F)

Enhanced clinical documentation—F17 (F)

Access to PRMs responses—F17, F18 (B)(F)

Discrepancies in patient’s assessment of health—F18 (B)

Location of PRMs collection—F20 (B)(F)

Timing and frequency of PRMs completion—F21 (B)(F)

Access to additional resources based on PRMs responses—F22 (B)(F)

Patient—healthcare professional communication—F25 (B)(F)

Automated administration of PRMs—F26 (F)

	–	–	
Adopter system: changes in role, identity, and acceptance of digital technology by different stakeholders

	Capability

(Knowledge; Skills)

	Rationale for PRMs collection and use—F14 (B)	PRMs interpretation knowledge and skills—F12 (B)

Rationale for PRMs collection and use—F14 (B)

	–	–	
Capability

(Memory, attention and decision processes)

	Questionnaire length and complexity of questions—F32 (B)(F)

Questionnaire relevance to patient’s health—F33 (B)(F)

PRMs response capture option—F34 (B)(F)

	Questionnaire length and complexity of questions—F32 (B)(F)

Questionnaire relevance to patient’s health—F33 (B)(F)

PRMs response capture option—F34 (B)(F)

	–	–	
Motivation

(Beliefs about consequences)

	Regular exposure enhancing older adults’ health knowledge—F13 (F)

Efficiencies, time constraints and changes to work routines—F26 (F)

Additional work with PRMs completion—F27 (B)

	Regular exposure enhancing older adults’ health knowledge—F13 (F)

Efficiencies, time constraints and changes to work routines—F26 (B) (F)

Additional work with PRMs completion—F27 (B)

	Acceptance of intervention by older adults—F19 (F)

Efficiencies, time constraints and changes to work routines—F26 (F)

	–	
Motivation

(Emotions)

	Emotional experiences—F23 (B)(F)	Emotional experiences—F23 (B)(F)	–	–	
Motivation

(Intentions)

	Older adults’ owning responsibility to improve own health—F15 (F)	–	–	–	
Motivation

(Optimism)

	–	Healthcare professional’s age and length of service—F18 (B)	–	–	
Opportunity

(Social influences)

	Support from social circle—F9 (F)

Trust on healthcare service—F15 (F)

Older adults’ autonomy—F24 (B)

	Support from social circle—F9 (F)

Older adults’ autonomy—F24 (B)

	Support from social circle—F9 (F)	Support from social circle—F9 (F)	
Organization: readiness, capacity, and work needed to implement digital technology in organization

	Opportunity

(Environmental context and resources)

	Change management—F30 (F)

Resources—F31:

- Education (F)

-Technical support (F)

	Team culture and collaboration—F28 (B)(F)

Leadership and champions—F29 (F)

Change management—F30 (B)(F)

Resources—F31:

- Education (B)(F)

- IT infrastructure (B)(F)

- Support staff (B)(F)

- Funding (B)(F)

	Change management—F30 (F)

Resources—F31:

- Education (F)

- Technical support (F)

	–	
Wider context: policy context, regulatory or professional body influence, public perceptions and inter-organizational networking

	Opportunity

(Environmental context and resources)

	–	Regulatory directives—F17, F18 (B)(F)

Resources—F31:

- Funding (F)

Government funding to cover electronic system costs

	–	–	
Barrier—(B); Facilitator—(F). F1 to F34—Findings describing stakeholder perspectives found in Supplemental Appendix S5. Theoretical Domains Framework (TDF) domains relevant to COM-B are also listed.

a Factors reported in the patient condition domain also apply to the adopter system domain.

Table 3. Barriers and facilitators unique to the digital aspects of electronic collection and use of PRMs with illustrative quotes and interpretations.

Barrier (B)/Facilitator (F)	Example quotes/interpretations	
Clinical conditions and socio-cultural factors—F1 (B) Pt	Dexterity impairment:

Patients had varying levels of dexterity issues controlling the mouse. Some had difficulty controlling the cursor—Nothing complicated … it’s controlling the mouse33  Pt

Visual impairment:

One patient reported his poor eyesight made using the iPad difficult51Pt

	
Access to digital technology—F2 (B)(F) Pt HCP	A lot of my patients don’t use MyChart, they don’t have email, older lung cancer patients that live out in the country, they just don’t have that technology available. 53  HCP

Participants that owned either a portable (laptop or tablet) or desktop computer or smartphone, or a combination of the 3, were comfortable accessing the internet and checking electronic mail (e-mail)35  Pt

	
User interface for older adults—F3, F4 (B)(F) Pt C Adm HCP	it’s frustrating when you have to go in and out, in and out, you think you’re finished and then you get an email saying you’re not finished 10  C

I do not like seeing how many more pages I have. It is just like oh, I have five more pages to go 35  Pt

For elderly patients or people that are visually impaired and [PROs] can be stressful for them to click through all of the screens because it’s a lot of questions 38 HCP

Participants noted that the psychological aspect of not having to view all the questions (as with the paper format) can be less overwhelming for patients. In the ePSRM tool, questions are presented one at a time36  HCP Adm

	
User interface for healthcare professionals—F5 (B)(F) HCP	The frustration of even finding the PROs is pretty high. [Once I find the data] I can't even tell which [PRO question] is which because they're just all like shoved down there into one big blob of text. I wish there were a way to just visually tell like a like a good dashboard in your car if it’s a red light that means there’s something wrong38  HCP	
Electronic device of PRMs completion—F6 (B)(F) Pt	I’m pretty competent on the internet and maybe on an iPad like on a larger device it might have been more free flowing 45  Pt	
Technical challenges—F7 (B) Pt HCP	There was some sort of technical difficulty at… their end of it. Because it just went blank on me… where the phone call is made, and the beginning of the survey is just—it was like there was something wrong technically at the other end, because the phone just went dead 47 Pt

[Patients] get frustrated with [the PRO tablets] if they log themselves out. They have to enter the encounter ID again… [which] they don’t remember 38  HCP

	
Privacy and security of personal data—F8 (B)(F) Pt HCP	I would want to know who had access to it (PROM data) and what it was being used for. Like I would want to know those things before I decided whether I was going to complete it. And I think like I would only want the surgeon and key staff in his office to have access to it, and I think it should only be used for improving your surgical care, like your results… 35 Pt

The surgeons were also concerned regarding the ownership and privacy of the data when utilizing the TickiT platform34  HCP

	
Support from social circle—F9 (F) Pt HCP	Once my older patients get some help from the front desk and realize that it's going to be a tap they, they seem to do well 36  HCP

I’m all about apps and stuff like that. But for people who aren’t, have someone in the office to show them, walk them through it step by step and make sure that they’re okay with it before they leave 40 Pt

	
Digital knowledge and skills—F10 (B)(F) Pt Adm HCP	For older adults:

When I first got the tablet in my hand, a little anxiety came over me because I’m not much into computers […] when you’re doing something you’re not familiar with, I get overwhelmed a little 10 Pt

Staff reported that elderly patients more often had difficulty completing PROMs due to low computer and/or technology literacy10  Adm

.. they just don’t have that technology available. And even if they do, like a smartphone or a computer, they don’t feel comfortable. Just filling out a little survey with a mouse is hard 53  HCP

For healthcare professionals:

I came to the department after the project had started. I had some questions about how to use the system 46 HCP

	
Trust on healthcare service—F15 (F) Pt	Most participants reported that their primary motivation for using the digital PRO system was that the hospital had asked them to. Participants reported a high degree of trust in, and a good relationship with, the clinic—You got to trust it. I think I do that, and it is only because I hope… this department has always been good…48  Pt	
Healthcare professionals’ motivation:

- Enhanced clinical documentation—F17 (F) HCP

- Access to PRMs results—F17, F18 (B)(F) HCP

- Age and length of service—F18 (B) HCP

	Enhanced documentation:

They found that the new ePSRM creates uniformity in how notes are generated36  HCP

Access to PRMs results:

I tried to look but when I looked in EPR only her older scores were there. I didn’t have time to then go to try to look on the website before meeting with her 43  HCP

Age and length of service:

My guess is most younger physicians are probably going to be comfortable, but people who are towards the, end of their career probably won't be very open to making a whole lot of changes 36  HCP

	
Acceptance of digital system by older adults—F19 (F) Adm	Really just encouraging them that if the patient is satisfied with this program then, patient satisfaction equals our satisfaction I believe 36  Adm	
Location of PRMs completion—F20 (B)(F) Pt	Utilizing the computer at the clinic was susceptible to infection or hygiene risks and unprofessional behavior by other patients [..]35  Pt

It was probably more convenient at home to do it on your time, plus you could stop, come back, finish if somethin’ came up or somethin’ you had to do 47  Pt

	
Emotional experiences—F23 (F) HCP	The nurses believed that it may have been easier for the patients to initially complete the ePROs rather than talk about the items, especially for challenging topics such as depression, anxiety, sex, or pain—maybe they're embarrassed a little bit or it's easier to put it on a screen maybe than talk about it?41  HCP	
Patient—healthcare professional communication—F25:

- Interactive conversations with electronic system (F) HCP Pt

- Feels impersonal (B) Pt

	Interactive conversations with electronic system:

[..] the electronic PRM format lends itself to more interactive conversational sessions with patients. For example, physicians are able to view and review patients' responses before meeting with patients that enhances their interaction, spend less time on interview, and spend more time for comprehensive discussions on treatment plans with patients. It also improved feedback to patients. – ..I used to draw pictures for the patient that there's like graphs and diagrams; I don't have to do that anymore36  HCP

Most participants identified that using digital PROs informed and enhanced their face-to-face consultations—I guess it is to get the information, to be prepared and to get a holistic view—also backwards to see, if there is anything to see, when you get enough questionnaires filled out48  Pt

Feels impersonal:

Some participants had concerns about being lost in the system after being assigned to the digital PRO system, therefore perceiving the digital PROs as a barrier to interaction with their healthcare providers—It took a while before I got one [a PRO questionnaire]. It is almost… I think it is a year after we discussed it, that I got one. Why, I don’t know. But then again, they had not promised that it would be fast. But it took a long time. I did come to think I was forgotten48  Pt

	
Efficiencies, time constraints and changes to work routines for healthcare professionals—F26 (B)(F) HCP Adm	It is a barrier that you have to log in to another system [AmbuFlex] if it does not substitute other tasks 46  HCP

We type our notes and it used to take me for a complicated patient sometimes up to 40 minutes or so to write that lengthy report. Now it takes me about 18 minutes … all of that time that is saved is then used to actually talk to the patient 36  HCP

the providers are able to do what they need to do quicker and get the patient back, versus waiting on them to fill out the paper, going over it, and then calling the patient back 36  Adm

	
Time consuming for older adults to complete PRMs electronically—F27 (B) HCP	Using OncoQuest takes too much time according to their patients50  HCP	
Change management—F30:

- Flexibility in developing electronic system (F) HCP

- Operational workflows (B)(F) HCP Adm

	Flexibility in developing electronic system:

The other thing was the inherent flexibility … we could go through various iterations very quickly and that allowed us to get to a point where we were not afraid of experimenting 36 HCP

Operational workflows:

No, I didn’t log onto the website. It is too much trouble. I forgot that I could find the results in EPR 43 HCP

One continued maintenance aspect expressed by multiple participants focused on continued training. - Consistent training on it so that we [are] always kept up to speed on what we have to do. Training new staff if they come in would maintain consistency across the board I believe36  Adm

	
Resources to manage collection and use of PRMs—F31:

- Education to use electronic system (F) Pt Adm

- Technical support (B)(F) HCP Adm Pt

- IT infrastructure (B)(F) HCP

- Funding (B)(F) HCP

	Education and technical support:

Patients that expressed difficulty with technology indicated that having formal instruction or someone to assist or engage them in the electronic communication could empower them to consider this avenue40  Pt

Technical support:

Insufficient staff for support and insufficient staff for coordination were reported as barriers34HCP

There were only minor technical challenges, and the patients were very compliant and contacted the department in case of any technical problems44  Pt

We have great help if we don't know what to do or how to do it. Someone's there to help us, so we know where to, who to reach out to 36  Adm

IT infrastructure:

The availability of OncoQuest outcomes during consultation is hampered due to difficulties to trace OncoQuest results at their computer screen50  HCP

Funding:

Financial support extended to cover the cost of the e-PROM platform by the Ministry of Health was proposed as a facilitator35  HCP

	
Abbreviations: EHR = electronic health record; EPR = electronic patient record; PRM = patient-reported measures.

Stakeholder perspectives indicated as Pt—Older adults; HCP—Healthcare professional; Adm—Administrative staff and C—Caregivers.

Patient condition

The lack of access to digital technology, digital knowledge and skill gaps, and dexterity and visual impairment to using electronic devices were key Opportunity and Capability barriers specific to the electronic completion of PRMs among older adults. More broadly, visual and cognitive impairment (eg, memory loss), language difficulties, low literacy-associated reading difficulty and health knowledge gaps hindered older adults to complete PRMs in general.

Technology

Barriers and facilitators in this domain were specific to the electronic collection and use of PRMs. All factors except one (privacy and security of personal data) were related to Opportunity. Optimal design of the user interface was key to having older adults and healthcare professionals engage with digital technology to complete and use PRMs. Large font and screen size, and less clicks and scrolls were some of the features older adults preferred in the user interface. Unnecessary pop-up alerts, cumbersome drop-down menus and certain display graphics (eg, multiple colors, smiley faces) were some features disliked by older adults. Some features that both older adults and healthcare professionals considered important were clear labeling of options and responses, appropriate display graphics, intuitive interface allowing for easy progression of either the questionnaire or viewing of results and an optimal number of clicks to navigate the electronic system. Technical barriers, such as slow response times and loss of internet connectivity, were experienced by both older adults and healthcare professionals. A larger electronic device like a tablet or computer was preferred by most older adults over a smartphone. Privacy and security concerns with electronic completion of PRMs among older adults and healthcare professionals related to Motivation.

Value proposition

Older adults having the ability to remotely complete PRMs using the electronic system, typically from home was cited as a facilitator. However, some older adults feared that electronic completion of PRMs may replace face-to-face interactions with their healthcare professional. Healthcare professionals having easy and immediate access to PRMs data in electronic health records, and graphical visualization of patient problems was a key facilitator. Improved clinic efficiency through automated administration of PRMs and enhanced electronic health record documentation through the electronic capture of PRMs was a notable benefit. More broadly, patient—healthcare professional communication, appropriate timing and frequency for PRMs completion were facilitators for both older adults and healthcare professionals. Additionally, completion of PRMs was dependent on whether or not PRMs were utilized and informed care planning, provision of additional resources to address problem areas (eg, education or social services) and advanced quality of care and research. All factors in the value proposition domain were related to Motivation.

Adopter system

The knowledge and skills to use electronic systems for completion and analysis of PRMs was identified as both a barrier and facilitator among older adults and a barrier among healthcare professionals. Efficiencies and time constraints associated with using electronic systems to capture and analyze PRMs were identified by healthcare professionals and administrative staff as a barrier. Change resistance to engage with digital technology among healthcare professionals who were older and had longer service emerged as an additional barrier. Older adults receiving support from their social circle to use electronic systems to complete PRMs was a facilitator. More broadly, healthcare professionals lacking knowledge and skills to interpret PRMs, discrepancies in health assessment between older adults and healthcare professionals (e.g., symptoms not judged as severe as reported by older adults), and changes to existing work routines were identified as barriers. Social influences on older adults (ie, support to complete PRMs and support leading to suppression of autonomy), trust in healthcare provider, and emotional experiences (eg, distressed when answering questions about health, personal or sensitive topics) were other key factors identified in this review. Additionally, understanding the rationale for electronic completion and use of PRMs, with clear messaging of the benefits was salient for older adults and healthcare professionals. Questionnaire relevance, optimal design of questionnaire (eg, number of questions and simple language), and response capture options allowing for numerical ratings and free text fields were facilitators cited by older adults and healthcare professionals. Factors in the adopter system domain were related to Capability, Motivation, and Opportunity.

Organization

Resources to implement digital technology for PRMs administration highlighted the need for education on use of electronic system among users, technical support to troubleshoot technical errors, digital technology infrastructure enabling seamless data integration (eg, with electronic health records) and adequate funding to acquire and maintain the digital technology. Change management that supported clear workflows in the electronic system (eg, finding PRMs results in the electronic system and sending of PRMs information to healthcare professionals) and processes that allowed for multiple iterations of the electronic system facilitated the adoption of electronic collection and use of PRMs among healthcare professionals. More broadly, access to resources such as education to guide PRMs collection and use, adequate support (eg, support staff and reminders to support PRMs administration) and adequate staff to act on PRMs responses (eg, increasing workload with PRMs administration for healthcare professionals) were identified. Appropriate change management (eg, clear operational workflows for acting PRMs results, support and training), involvement of leadership and champions, and positive team culture were identified as facilitators to PRMs collection and use. All factors were linked to Opportunity.

Wider context

Access to government funding to acquire digital technology for PRMs collection was reported as a facilitator, and was unique to the electronic collection and use of PRMs in older adults’ care. More broadly, regulatory directives mandating PRMs completion was considered a barrier when clear messaging of purpose was absent. However, being mandated to collect PRMs under a regulatory requirement was considered a motivator by healthcare professionals to encourage their engagement in PRMs collection. Both factors were linked to Opportunity.

Discussion

Summary of the main findings

We synthesized 22 studies reporting the stakeholder perspectives of 374 older adults, 183 healthcare professionals, 3 caregivers, and 5 administrative staff on the barriers and facilitators influencing uptake of electronic collection and use of PRMs in older adults’ care. Thirty-four findings describing barriers and facilitators were derived and grouped into 11 thematic categories that included: (1) older adult’s characteristics; (2) digital technology; (3) support from social circle; (4) knowledge and skills; (5) motivation and incentives for capture and use of PRMs; (6) emotional experience; (7) older adults’ autonomy; (8) patient—healthcare professional communication; (9) workflow; (10) organizational factors; and (11) questionnaire selection and design. Most findings were graded as high (n = 3) or moderate (n = 26) confidence indicating end users such as patients, healthcare professionals, healthcare service administrators, and researchers can be generally confident in our findings. Furthermore, we mapped the findings to the theoretical domains of NASSS and COM-B frameworks to identify possible determinants of change which can inform the design of implementation interventions.

Comparison with other reviews

A number of findings synthesized in this review are novel and not previously reported in reviews of factors influencing PRMs implementation more generally54,55 or electronic delivery of PRMs in various healthcare settings.18 These include older adults’ positive emotional experiences when using a digital interface to respond to questions on personal and sensitive topics, older adults’ and healthcare professionals’ privacy and security concerns with electronic completion of PRMs, easy to use electronic devices for PRMs completion among older adults, benefits of electronic systems for automated administration of PRMs and documentation in electronic patient records, and gaps in older adults’ health knowledge. Bi-directional factors (ie, serving as barriers or facilitators) include the location, timing and frequency of PRMs collection as well as the length, complexity and capture options for PRMs.

Our review highlights that aging-related and socio-economic challenges influence older adults’ ability to engage with digital technology and electronically complete PRMs. Our findings suggest that suboptimal design of digital technology for PRMs completion by older adults is likely to reflect a failure to adequately consider the specific characteristics and needs of older adults. Additionally, older adults’ access to digital technology (ie, electronic devices for questionnaire completion and internet access) is likely to influence their ability to complete the questionnaire independently and remotely. Older adults having access to digital technology appears to be associated with greater familiarity and skills to navigate through the digital devices to complete the questionnaire. This means that older adults are inherently likely to be adopters so long as issues of access, digital literacy and user interface are addressed.

Some of our findings are consistent with previous reviews. Namely, providing support for patients to complete PRMs, user-friendly digital technology, provision of services to address issues identified in PRMs responses, improving patient—healthcare professional communication, and change management that supports clear operational workflows and trialability of systems prior to implementation served as facilitators.54,55 Barriers identified in this review, consistent with prior reviews, include lack of appropriate IT infrastructure, lack of PRMs questionnaire relevance to patient health and clinical assessment, gaps in patients’ digital and language literacy to complete PRMs, and cognitive or physical impairments hindering PRMs completion.18,54,55

Strengths and limitations

The strengths of this study include a comprehensive search strategy, grading of review findings and independent synthesis of findings by two authors. The use of the NASSS, a digital technology implementation framework, and COM-B, a behavior change framework, in this review produced a comprehensive set of explanations that could be important determinants of change in implementing electronic collection and use of PRMs in older adults’ care.

As a limitation of this study, no grey literature was searched which may have prevented additional relevant issues being considered. However, the comprehensive search strategy enabled the identification of a good proportion of studies to include in this review for an in-depth analysis of relevant factors. The search date for this review is a limitation, however, potentially eligible studies report no perspectives of stakeholders other than those already in this review. Most studies included in this review reported experiences of older adults and healthcare professionals. Other stakeholder perspectives including that of the caregivers and administrative staff appear to be under-reported. This suggests the need for primary qualitative studies to consider wider stakeholder views such as those of administrative staff and caregivers, in addition to those such as organization leaders, implementation specialists and digital technology developers. The generalizability of the findings in contexts other than those reported in this review may be limited to countries (eg, United States, United Kingdom, Canada, and Netherlands), healthcare settings (eg, oncology) and type of PRM (ie, PROMs). This calls for further research in various healthcare settings, countries and PREMs implementations. Additionally, no factors were mapped to the NASSS domain embedding and adaptation over time, indicating a need a for future research on the digital technology adaptability in changing context.

Conclusion

We synthesized 22 studies that identified barriers and facilitators to the implementation of electronic collection and use of PRMs in older adults’ care from the perspective of older adults, caregivers, healthcare professionals and administrative staff. Through the mapping of these findings to NASSS and COM-B frameworks, a comprehensive set of explanations are provided to inform the design of future implementation efforts. It is important to address barriers associated with digital technology access, digital literacy and user interface, to increase the likelihood of older adults’ adopting digital technology to complete PRMs electronically. This review highlighted a lack of evidence on, and need for future research to explore, the perspectives of other stakeholders, such as organizational leaders, digital technology developers and implementation specialists; a knowledge gap on the adaptability of electronic collection and use of PRMs in changing healthcare contexts; and the need for more research exploring the factors influencing implementation of PREMs.

Supplementary Material

ooae068_Supplementary_Data

Author contributions

G.H.S., P.D.H., N.A., V.S., D.A.S., and D.O.C. contributed to the conception and design of the study. Search terms and databases were selected by G.H.S., P.D.H., N.A., D.A.S., and D.O.C. Search was conducted and results uploaded to Covidence by G.H.S. Study selection was carried out by G.H.S. and P.D.H. Quality assessments, data extraction, and confidence assessments were carried out by G.H.S., L.B., and M.M.S. G.H.S., P.D.H., L.B., M.M.S., D.A.S., and D.O.C. contributed to the interpretation of data. G.H.S. drafted the manuscript, and all authors made contributions to the review, revision and approval of the manuscript version to be submitted.

Supplementary material

Supplementary material is available at JAMIA Open online.

Funding

This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.

Conflicts of interests

The authors have no competing interests to declare.

Data availability

The data underlying this article are available in the article and in its online supplementary material.
==== Refs
References

1 Kingsley C , PatelS.  Patient-reported outcome measures and patient-reported experience measures. BJA Education. 2017;17 (4 ):137-144. 10.1093/bjaed/mkw060
2 Schick-Makaroff K , Karimi-DehkordiM, CuthbertsonL, et al  Using patient- and family-reported outcome and experience measures across transitions of care for frail older adults living at home: a meta-narrative synthesis. Gerontologist. 2020;61 (3 ):e23-e38. 10.1093/geront/gnz162
3 Porter I , Gonçalves-BradleyD, Ricci-CabelloI, et al  Framework and guidance for implementing patient-reported outcomes in clinical practice: evidence, challenges and opportunities. J Comp Eff Res. 2016;5 (5 ):507-519. 10.2217/cer-2015-0014 27427277
4 Boyce MB , BrowneJP.  Does providing feedback on patient-reported outcomes to healthcare professionals result in better outcomes for patients? A systematic review. Qual Life Res. 2013;22 (9 ):2265-2278. 10.1007/s11136-013-0390-0 23504544
5 Chen J , OuL, HollisSJ.  A systematic review of the impact of routine collection of patient reported outcome measures on patients, providers and health organisations in an oncologic setting. BMC Health Services Research. 2013;13 (1 ):211. 10.1186/1472-6963-13-211 23758898
6 Valderas JM , KotzevaA, EspallarguesM, et al  The impact of measuring patient-reported outcomes in clinical practice: a systematic review of the literature. Qual Life Res. 2008;17 (2 ):179-193. 10.1007/s11136-007-9295-0 18175207
7 Basch E , DealAM, KrisMG, et al  Symptom monitoring with patient-reported outcomes during routine cancer treatment: a randomized controlled trial. J Clin Oncol. 2016;34 (6 ):557-565. 10.1200/JCO.2015.63.0830 26644527
8 Sundaram CS , CampbellR, JuA, KingMT, RutherfordC.  Patient and healthcare provider perceptions on using patient-reported experience measures (PREMs) in routine clinical care: a systematic review of qualitative studies. J Patient Rep Outcomes. 2022;6 (1 ):122. 10.1186/s41687-022-00524-0 36459251
9 Howard FD , GreenR, HarrisJ, RossJ, NicholsonC.  Understanding the extent to which PROMs and PREMs used with older people with severe frailty capture their multidimensional needs: a scoping review. Palliat Med. 2024;38 (2 ):184-199. 10.1177/02692163231223089 38268061
10 Long C , BeresL, WuA, GiladiA.  Patient-level barriers and facilitators to completion of patient-reported outcomes measures. Qual Life Res. 2021;31 (6 ):1711-1718. 10.1007/s11136-021-02999-8 34533759
11 van Oppen JD , AlshibaniA, CoatsTJ, et al  A systematic review and recommendations for prom instruments for older people with frailty in emergency care. J Patient Rep Outcomes. 2022;6 (1 ):30. 10.1186/s41687-022-00438-x 35362836
12 Doolin JW , HalpinM, BerryJL, HshiehT, ZerilloJA.  Why focus on patient-reported outcome measures in older colorectal cancer patients?  Eur J Surg Oncol. 2020;46 (3 ):394-401. 10.1016/j.ejso.2019.07.028 31377032
13 Rikkert O , van der WeesPJ, SchoonY, WestertGP.  Using patient reported outcomes measures to promote integrated care. Int J Integr Care. 2018;18 (2 ):8-7. 10.5334/ijic.3961
14 Meirte J , HellemansN, AnthonissenM, et al  Benefits and disadvantages of electronic patient-reported outcome measures: systematic review. JMIR Perioper Med. 2020;3 (1 ):e15588. 10.2196/15588 33393920
15 Yu J , GoldbergT, LaoN, FeldmanB, GohY.  Electronic forms for patient reported outcome measures (PROMs) are an effective, time-efficient, and cost-minimizing alternative to paper forms. Pediatr Rheumatol Online J. 2021;19 (1 ):67. 10.1186/s12969-021-00551-z 33941208
16 De Rosis S , CerasuoloD, NutiS.  Using patient-reported measures to drive change in healthcare: the experience of the digital, continuous and systematic PREMs observatory in Italy. BMC Health Serv Res. 2020;20 (1 ):315. 10.1186/s12913-020-05099-4 32299440
17 Hernar I , GraueM, RichardsD, et al  Electronic capturing of patient-reported outcome measures on a touchscreen computer in clinical diabetes practice (the DiaPROM trial): a feasibility study. Pilot Feasibility Stud. 2019;5 :29. 10.1186/s40814-019-0419-4 30820340
18 Glenwright BG , SimmichJ, CottrellM, et al  Facilitators and barriers to implementing electronic patient-reported outcome and experience measures in a health care setting: a systematic review. J Patient Rep Outcomes. 2023;7 (1 ):13. 10.1186/s41687-023-00554-2 36786914
19 Flemming K , BoothA, GarsideR, TunçalpÖ, NoyesJ.  Qualitative evidence synthesis for complex interventions and guideline development: clarification of the purpose, designs and relevant methods. BMJ Glob Health. 2019;4(Suppl 1) :e000882. 10.1136/bmjgh-2018-000882
20 Hettiarachchi Senarath G , HaghighiPD, O’ConnorD, SnowdonD, AndrewN, SrikanthV.  Barriers and facilitators to the implementation of electronic collection of patient-reported measures in routine care of older adults: protocol for a qualitative evidence synthesis. OSF Preprints. 2022:1-12. 10.31219/osf.io/3w8xz
21 Moher D , ShamseerL, ClarkeM, PRISMA-P Group, et al  Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev. 2015;4 (1 ):1. 10.1186/2046-4053-4-1 25554246
22 Noyes J , BoothA, CargoM, et al Chapter 21: qualitative evidence. 2021. Accessed September 22, 2021. https://training.cochrane.org/handbook/current/chapter-21
23 Beller EM , GlasziouPP, AltmanDG, PRISMA for Abstracts Group, et al  PRISMA for abstracts: reporting systematic reviews in journal and conference abstracts. PLOS Med. 2013;10 (4 ):e1001419. 10.1371/journal.pmed.1001419 23585737
24 Moher D , LiberatiA, TetzlaffJ, AltmanDG, The PRISMA Group  Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6 (7 ):e1000097. 10.1371/journal.pmed.1000097 19621072
25 Tong A , FlemmingK, McInnesE, OliverS, CraigJ.  Enhancing transparency in reporting the synthesis of qualitative research: ENTREQ. BMC Med Res Methodol. 2012;12 :181. 10.1186/1471-2288-12-181 23185978
26 Booth A , NoyesJ, FlemmingK, MooreG, TunçalpÖ, ShakibazadehE.  Formulating questions to explore complex interventions within qualitative evidence synthesis. BMJ Glob Health. 2019;4(Suppl 1) :e001107. 10.1136/bmjgh-2018-001107
27 Critical Appraisal Skills Programme (CASP). CASP Checklist: 10 questions to help you make sense of a Qualitative research. 2020. Accessed: November 30, 2020. https://casp-uk.net/casp-tools-checklists/.
28 Lumivero. NVivo (Version 12). 2021. Accessed December 15, 2021. https://lumivero.com/products/nvivo/
29 Thomas J , HardenA.  Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Med Res Methodol. 2008;8 :45. 10.1186/1471-2288-8-45 18616818
30 Greenhalgh T , WhertonJ, PapoutsiC, et al  Beyond adoption: a new framework for theorizing and evaluating nonadoption, abandonment, and challenges to the scale-up, spread, and sustainability of health and care technologies. J Med Internet Res. 2017;19 (11 ):e367. 10.2196/jmir.8775 29092808
31 Michie S , van StralenMM, WestR.  The behaviour change wheel: a new method for characterising and designing behaviour change interventions. Implement Sci. 2011;6 :42. 10.1186/1748-5908-6-42 21513547
32 Lewin S , BoothA, GlentonC, et al  Applying GRADE-CERQual to qualitative evidence synthesis findings: Introduction to the series. Implement Sci. 2018;13(Suppl 1) :2. 10.1186/s13012-017-0688-3 29384079
33 Aiyegbusi O , KyteD, CockwellP, et al  Development and usability testing of an electronic patient-reported outcome measure (ePROM) system for patients with advanced chronic kidney disease. Comput Biol Med. 2018;101 :120-127. 10.1016/j.compbiomed.2018.08.012 30130638
34 Amini M , OemrawsinghA, VerweijLM, et al  Facilitators and barriers for implementing patient-reported outcome measures in clinical care: an academic center’s initial experience. Health Policy. 2021;125 (9 ):1247-1255. 10.1016/j.healthpol.2021.07.001 34311981
35 Kaur M , PusicA, GibbonsC, KlassenAF.  Implementing electronic patient-reported outcome measures in outpatient cosmetic surgery clinics: an exploratory qualitative study. Aesthet Surg J. 2019;39 (6 ):687-695. 10.1093/asj/sjy280 30335134
36 Spaulding A , NordanL, BlanchfieldL, et al  Qualitative study of implementation of patient self-reported measures in a consultation-liaison psychiatry practice. J Eval Clin Pract. 2019;25 (3 ):482-490. 10.1111/jep.13160 31037796
37 Delgado-Herrera L , BanderasB, OjoO, KothariR, ZeiherB.  Diarrhea-predominant irritable bowel syndrome: creation of an electronic version of a patient-reported outcome instrument by conversion from a pen-and-paper version and evaluation of their equivalence. Patient Relat Outcome Meas. 2017;8 :83-95. 10.2147/PROM.S126605 28790875
38 Mou D , HornDM, HengM, et al  Primary care physician’s (PCP) perceived value of patient-reported outcomes (PROs) in clinical practice: a mixed methods study. BMJ Qual Saf. 2021;31 (3 ):221-229. 10.1136/bmjqs-2020-012206
39 Krawczyk M , SawatzkyR, Reimer-KirkhamS, et al  Micro-meso-macro practice tensions in using patient-reported outcome and experience measures in hospital palliative care. Qual Health Res. 2019;29 (4 ):510-521. 10.1177/1049732318761366 29542400
40 Navarro-Millan I , ZinskiA, ShurbajiS, et al  Perspectives of rheumatoid arthritis patients on electronic communication and patient-reported outcome data collection: a qualitative study. Arthritis Care Res (Hoboken). 2019;71 (1 ):80-87. 10.1002/acr.23580 29669191
41 Schick-Makaroff K , MolzahnAE.  Evaluation of real-time use of electronic patient-reported outcome data by nurses with patients in home dialysis clinics. BMC Health Serv Res. 2017;17 (1 ):439. 10.1186/s12913-017-2377-y 28651528
42 Brochmann N , ZwislerA-D, KjerholtM, FlachsEM, HasselbalchHC, AndersenCL.  A new internet-based tool for reporting and analysing patient-reported outcomes and the feasibility of repeated data collection from patients with myeloproliferative neoplasms. Qual Life Res. 2015;25 (4 ):835-846. 10.1007/s11136-015-1125-1 26358064
43 Snyder CF , BlackfordAL, WolffAC, CarducciMA, HermanJM, WuAW, PatientViewpoint Scientific Advisory Board. Feasibility and value of PatientViewpoint: a web system for patient-reported outcomes assessment in clinical practice. Psychooncology. 2013;22 (4 ):895-901. 10.1002/pon.3087 22544513
44 Tolstrup LK , PappotH, BastholtL, ZwislerA-D, DieperinkKB.  Patient-reported outcomes during immunotherapy for metastatic melanoma: mixed methods study of patients’ and clinicians’ experiences. J Med Internet Res. 2020;22 (4 ):e14896. 10.2196/14896 32271150
45 Yamada J , KouriA, SimardS-N, SegoviaSA, GuptaS.  Barriers and enablers to using a patient-facing electronic questionnaire: a qualitative theoretical domains framework analysis. J Med Intern Res. 2020;22 (10 ):e19474. 10.2196/19474
46 Baeksted C , PappotH, NissenA, et al  Feasibility and acceptability of electronic symptom surveillance with clinician feedback using the patient-reported outcomes version of common terminology criteria for adverse events (PRO-CTCAE) in Danish prostate cancer patients. J Patient Rep Outcomes. 2017;1 (1 ):1. 10.1186/s41687-017-0005-6 29757324
47 Samuel CA , SmithAB, ElkinsW, et al  Racial differences in user experiences and perceived value of electronic symptom monitoring in a cohort of black and white bladder and prostate cancer patients. Qual Life Res. 2021;30 (11 ):3213-3227. 10.1007/s11136-020-02442-4 32112276
48 Nielsen AS , AppelCW, LarsenBF, KayserL, HannaL.  Patient perspectives on digital patient reported outcomes in routine care of inflammatory bowel disease. J Patient Rep Outcomes. 2021;5 (1 ):92. 10.1186/s41687-021-00366-2 34533682
49 Lehmann J , BuhlP, GiesingerJM, et al  Using the computer-based health evaluation system (CHES) to support self-management of symptoms and functional health: evaluation of hematological patient use of a web-based patient portal. J Med Internet Res. 2021;23 (6 ):e26022. 10.2196/26022 34100765
50 Duman-Lubberding S , van Uden-KraanCF, JansenF, et al  Durable usage of patient-reported outcome measures in clinical practice to monitor health-related quality of life in head and neck cancer patients. Support Care Cancer. 2017;25 (12 ):3775-3783. 10.1007/s00520-017-3808-3 28702685
51 Grossman LV , FeinerSK, MitchellEG, Masterson CreberRM.  Leveraging patient-reported outcomes using data visualization. Appl Clin Inform. 2018;9 (3 ):565-575. 10.1055/s-0038-1667041 30068012
52 Moradian S , KrzyzanowskaMK, MaguireR, et al  Usability evaluation of a mobile phone-based system for remote monitoring and management of chemotherapy-related side effects in Canadian cancer patients: mixed-methods study. JMIR Cancer. 2018;4 (2 ):e10932. 10.2196/10932 30578238
53 Sandhu S , KingZ, WongM, et al  Implementation of electronic Patient-Reported outcomes in routine cancer care at an academic center: Identifying opportunities and challenges. JCO Oncol Pract. 2020;16 (11 ):e1255-e1263.10.1200/OP.20.00357 32926662
54 Briggs MS , RethmanKK, CrookesJ, et al  Implementing patient-reported outcome measures in outpatient rehabilitation settings: a systematic review of facilitators and barriers using the consolidated framework for implementation research. Arch Phys Med Rehabil. 2020;101 (10 ):1796-1812. 10.1016/j.apmr.2020.04.007 32416149
55 Foster A , CrootL, BrazierJ, HarrisJ, O’CathainA.  The facilitators and barriers to implementing patient reported outcome measures in organisations delivering health related services: a systematic review of reviews. J Patient Rep Outcomes. 2018;2 :46. 10.1186/s41687-018-0072-3 30363333
