
==== Front
BMJ Health Care Inform
BMJ Health Care Inform
bmjhci
bmjhci
BMJ Health & Care Informatics
2632-1009
BMJ Publishing Group BMA House, Tavistock Square, London, WC1H 9JR

39306332
10.1136/bmjhci-2024-101048
bmjhci-2024-101048
Original Research
1506
Emotional and psychological safety in the context of digital transformation in healthcare: a mixed-method strategic foresight study
http://orcid.org/0000-0002-2221-4531
Kuske Silke 1kuske@fliedner-fachhochschule.de

Vondeberg Carmen 1vondeberg@fliedner-fachhochschule.de

Minartz Peter 1peter_minartz@yahoo.de

Vöcking Mara 1voecking@fliedner-fachhochschule.de

Obert Laura 1obert@fliedner-fachhochschule.de

Hemming Bernhard 1hemming@fliedner-fachhochschule.de

Bleck Christian 2christian.bleck@hs-duesseldorf.de

Znotka Matti 3mattiznotka@web.de

Ose Claudia 1ose@fliedner-fachhochschule.de

Heistermann Peter 1heistermann@fliedner-fachhochschule.de

Schmitz-Kießler Jutta 4jutta.schmitz-kiessler@hsbi.de

Karrenbrock Anne 5anne.karrenbrock@th-koeln.de

Cürlis Diana 6diana.cuerlis@fh-muenster.de

1 Fliedner Fachhochschule Düsseldorf, Düsseldorf, Germany
2 Hochschule Düsseldorf, Düsseldorf, Germany
3 Katholisches Klinikum Koblenz Montabaur gGmbH, Koblenz, Germany
4 University Duisburg Essen, Essen, Germany
5 Technische Hochschule Köln, Köln, Germany
6 FH Münster, Münster, Germany
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

Additional supplemental material is published online only. To view, please visit the journal online (https://doi.org/10.1136/bmjhci-2024-101048).

None declared.

Ph.D.SilkeKuske; kuske@fliedner-fachhochschule.de
2024
20 9 2024
31 1 e10104811 2 2024
26 8 2024
Copyright © Author(s) (or their employer(s)) 2024. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.
2024
https://creativecommons.org/licenses/by-nc/4.0/ This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

Abstract

Background

Perceived safety has received attention in the digital transformation of healthcare. However, the impact of perceived safety on the future of digital transformation has not been fully elucidated.

Aim

To investigate perceived safety in the context of the digital transformation of healthcare while considering relevant needs, influencing factors and impacts, including crisis events, to provide recommendations for action based on a participatory, multiperspective, strategic 5-year foresight viewpoint.

Methods

A strategic foresight study is conducted via a participatory mixed-methods design to understand the present related factors that are likely to be relevant to future developments in the digital transformation of healthcare.

Results

We observed that feeling safe plays a complex role in the digital transformation of healthcare. How perceived safety is considered has and will continue to impact the individual, organisational and system levels. Regarding a potential crisis event, controversial consequences have been observed. At its core, digital (health) literacy related to equity of access and human support is one of the crucial aspects in the context of perceived safety related to the successful implementation of digital technologies in healthcare.

Conclusions

The scenarios showed that a continuation of the current situation over the next 5 years may result in partly desirable and partly undesirable outcomes. Concrete key factors should be used in practice to support both education and healthcare quality development and research. The essence of the scenarios should serve as a starting point for research agenda setting and political decision-making in the future. However, additional research is needed to quantify the correlations among the relevant factors.

Information Technology
Health Services Research
Safety Management
Patient Involvement
Patient-Centered Care
http://dx.doi.org/10.13039/501100002347 Bundesministerium für Bildung und Forschung 16INS108 The funder did not influence the study design, data collection, data analysis, interpretation of results or report writing. The authors are responsible for the content of this publication.
==== Body
pmcWHAT IS ALREADY KNOWN ON THIS TOPIC

Perceived safety has received increasing attention in the context of quality and safety research and the digital transformation of healthcare but remains poorly understood.

WHAT THIS STUDY ADDS

Emotional and psychological safety plays a critical role in both present and future healthcare in a complex and controversial way, including in the case of a crisis.

HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY

This study sheds new light on key factors and future projections that can facilitate the enhancement of perceived safety in digital transformation in the contexts of education, politics and research.

Background

Digital transformation (DTR) refers to an improvement process that involves facilitating changes based on the use of data, information and digital technologies (DTs).1 DTR can be viewed in context at the individual, organisational and societal levels,2 and it focuses on digitalisation and digitisation,3 which refer to the use of DTs to manage data and processes2 and to convert analogue data into digital data, respectively.3 DTs, for example, systems that focus on diagnostics, consulting, therapy4 and care,5 aim to increase healthcare benefits.6 In this context, perceived safety is of increasing interest because it is related to the healthcare recipient’s (HCRs) needs and trust, including the implementation and acceptance of DTs in healthcare.6 Emotional safety refers to a state of being free from threats of physical or emotional harm and is often conceptualised as part of a continuum of feeling more or less safe owing to the influence of internal and external factors.6 Recent studies have shown that emotional safety is closely related with (physical) safety, but that restrictive perceived safety can have negative consequences on basic care and patient safety.7 8 From a recent psychological perspective, the fulfilment of human needs, especially the need for social belonging, is necessary for feeling safe. Additionally, safety judgements are associated with social aspects.9 Similarly, psychological safety ‘as an interpersonal phenomenon, is most clearly understood to characterise small social systems like work groups in which individuals interact with each other’.10 In this context, studies have shown that the psychological safety of healthcare providers (HCPs) is related to several factors at the individual, team and organisational levels, such as teamwork, with consequences for patient safety.11 Research on this topic related to DTs in healthcare is scarce, with only a few studies to date having focused on this phenomenon.6 Generally, overarching societal and ethical perspectives are needed to reflect the benefits and challenges of DTs, also related to (perceived) safety12; however, knowledge concerning future scenarios in the context of feeling safe and DTR is lacking. Therefore, we aimed to investigate perceived safety in the context of the DTR of healthcare while considering the relevant needs, influencing factors and impacts, including crisis events, with the goal of developing recommendations for action on the basis of a participatory, multiperspective, strategic 5-year foresight approach.

Method

In this strategic foresight study, we employed a participatory, mixed-method (qualitative and quantitative), parallel and sequential research design.13 Strategic foresight focuses on understanding the present with the goal of systematically obtaining ideas regarding the factors that are likely to be relevant to future changes, thus shaping present-day decisions.13 The participatory approach14 entails continuous structured participant involvement and member checking,15 including reflection, adaptation to study results, and decision-making performed in collaboration with the research team. The scientific standards of research involving mixed methods and strategic foresight are considered.16 This study is registered with the Open Science Framework at the Centre for Open Science: https://doi.org/10.17605/OSF.IO/UTSQN.

Sample design and recruitment

A multiperspective mixed sample of HCRs (eg, patients and residents), HCPs (eg, physicians) and (caring) relatives was recruited for this study, which provided a wide range of ages, school degrees and levels of education. A criterion-based sample of DT experts from the field of strategic foresight and safety research, as well as experts with research expertise and/or (structural) responsibilities in ethical, health-related, legal, political and economic settings, was also invited. All participants were invited to take part in all phases repeatedly, and a sample size of 30 participants in each of the scenario rounds was calculated. The broad recruiting approach used in this research (from 20 February 2023 to 20 June 2023) involved public relations strategies (social media, institutional homepages and press releases) that included contacting the public, social services, healthcare institutions and scientific networks.

Data collection and analysis

The strategic foresight process included five core phases16: (1) scenario field identification; (2) key factor (KF) identification; (3) KF projection; (4) scenario development and (5) scenario finalisation and interpretation/transfer (see figure 1). Several data collection methods were used, namely, guide-based workshops, interviews and an online survey. The data collection process was based on a combined quantitative systematic-formalised and qualitative creative-narrative approach.16 Sociodemographic data and expertise were initially collected. In each phase, previous results were presented, reflected on, discussed and approved or disapproved.

Figure 1 The strategic foresight approach (process and methods) based on Kosow and Gaßner.16

Scenario field identification

The first phase (22 June 2022–30 January 2023), which involved a description of the situation, focused on synthesised data (domains, key dimensions and categories) pertaining to the context of perceived safety and DTs, including needs, influencing factors and related outcomes. A scoping review,6 5 participatory workshops and 10 design ethnographic use cases17 were used. The ‘Classification of Digital Health Interventions V.1.0’18 was applied for selecting a broad range of DTs, for example, artificial intelligence, robotics, electronic health records, DTs and information systems used in smart homes and (smart) hospitals, health apps, virtual reality, telemedicine, digital simulation training in emergency care, mobile ECG cases and a closed-loop system. The data were subjected to content analysis19 and then synthesised and triangulated20 for the subsequent phase.

KF identification

The second phase (from 20 February 2023 to 8 March 2023 (reminder 28 February 2023), involved the systematic selection of KFs via a voluntary cross-sectional online survey that considered scientific standards.21 A structured questionnaire based on n=105 multifactorial dimensions and subdivided into n=16 overarching interrelated domains of the scenario field identification results was used (see online supplemental appendix 1a). Two core questions based on a 5-year perspective of DTR focused on factors that are very likely to be related to feeling safe and factors that are particularly relevant to feeling safe in future. These factors were rated as ‘yes’, ‘no’ or ‘not ratable’. A pretest was also conducted (n=2) that focused on the consistency of the document, the wording of the questions, the level of difficulty and the time required to complete the online survey. The quantitative data were examined for missing data (without replacement) and plausibility, that is, data sets indicating that a factor was unlikely to be related to perceived safety were excluded from the relevance analysis. The final eligible KFs were selected on the basis of the following inclusion and exclusion criteria, considering qualitative and quantitative data equally. At a minimum, one factor was required to be included per domain, the most frequent factors were included, no factors were excluded beforehand on the basis of probability and a feasible number of KFs (n=10–25)16 was required.

Additionally, an impact analysis was performed by the research team using an impact matrix with the goal of determining the extent of the reciprocal relationships of the selected KFs via a Likert scale: ‘0=no influence; 1=weak relationship; 2=medium relationship; 3=strong relationship’16; and 99=not rateable. For this purpose, the KFs were rated in two directions regarding the pairs of KFs: (1) the extent to which one factor can influence the other factor, that is, the calculated active sum (AS) and (2) the extent to which one factor can be influenced by the other factor, that is, the calculated passive sum (PS).16 The AS and PS were calculated by summing all the entered values for each KF of each row of the impact matrix such that each KF resulted in an AS and PS.16 The KFs were determined by their type, namely, active—impulsive (high AS and low PS); balanced (neither high nor low AS or PS); or passive—reactive (high PS and low AS); they were then calculated by the quotients of the AS and PS of each KF. Based on these results, the KFs were examined for further exclusion.

KF projection, scenario development/finalisation: interpretation/transfer

The aim of the final three phases was to develop a concrete best-case scenario (BS), worst-case scenario (WS) and trend scenario (TS) while also considering crisis events. The BS, defined as the desirable scenario, reflects the participants’ needs; the WS, defined as the least desirable scenario, reflects unpleasant traits; and the TS defined as the scenario that represents a continuation of the status quo.

The scenario data collection involved two guide-based scenario rounds that were held in situ on 14 March 2023 (phases 3 and 4) and 20 June 2023 (phase 5). A qualitative creative-narrative workshop approach16 was applied and divided into two steps on the first day of the workshop, according to the scenario guide. First, the identified KFs were approved, analysed and then projected in positive, negative and trend directions based on a 5-year perspective. These projections were required to be probable, realistic, distinct, precise and complete.16 The projections developed were documented in a structured form during the moderated subgroup sessions and considered a mix of experiences. A final analysis was subsequently conducted. Second, probable and realistic raw scenarios and drafts of scenario cores were developed on the basis of these projections. Therefore, before the second group session was held on the same day, the KFs and their projections were bundled by the project team to develop raw scenarios. These bundled KFs and the corresponding projections were discussed, validated and concretised by the participants. Additionally, the first drafts of the defined scenario cores were developed, and the discussions were audiotaped, transcribed and analysed using deductive content analysis featuring inductive subcategory development.16 Based on participant feedback, the research team subsequently accepted the projections and confirmed their discriminability.

Additionally, a consistency analysis16 22 of all scenarios was performed via Parmenides EIDOS software23 (Parmenides AG, Poecking, Germany) for scenario calculation. We calculated the most consistent bundles of the raw scenarios with the KFs and corresponding projections that exhibited the best fit. The number of KF bundles is a result of the product of the number of projection alternatives per KF.16 In our case, 3 projections, given 19 KFs were calculated by 319 and resulted in 1 162 261 467 possible KF bundles. The Likert scale developed by Hinkeldein (2009)24 was used by two researchers to rate the consistency in the software consistency matrix as follows: −3=mutually exclusive; −2=contradictory; −1=bad fit; 0=unrelated; 1=good fit; 2=supportive of each other and 3=belong together. The consistency of a scenario is expressed in terms of the average consistency,16 which is calculated by summing one consistency (C) value (ranging from −3 to 3 on the Likert scale) from each pair of KF projections (per KF column of the consistency matrix) and dividing that by the sum of the investigated values per column, in our case, n=171 per calculated raw scenario. For example, in column 1, the C values of K1a and K2a were summed with the C values in column 2 of K1a and K3a and then summed with the C values of K2a and K3a, and so on. The final sum was then divided by the sum of the n=171 investigated values. The most consistent BS and WS were chosen for final inclusion. The TS, which exhibited lower consistency on its own terms, was an exeption. Minorly consistent bundles were excluded following the suggestions of Tourki et al.24

A contribution analysis was performed to determine whether the scenario components were consistent with those pertaining to one selected scenario (internal consistency).16 In this context, the C values, which are associated with each row of the KF projections pertaining to one selected scenario, were each multiplied by a proportion of 2/19 KFs and then summed. For example, 2/19×(C value of K1a and K2a)+2/19×(C value of K1a and K3a)+2/19×(C value of K1a and K4a), etc. These sums were subsequently normed by calculating the quotient in relation to the maximum or minimum internal C value associated with the projections of a selected scenario.

In the final scenario round (phase 5), the final scenarios and their cores were approved plenarily. The corresponding consequences at the individual, organisational and system levels of each scenario were subsequently determined while also considering participant expertise. Recommendations for action at the educational, research and political levels were developed according to the scenarios and on the basis of three randomly formed subgroups. Realistic crises that impacted the scenarios were reconciled, and the consequences of each scenario as well as the recommendations for action were developed by planners who considered education, research and politics.

Results

In summary, 19 KFs and the corresponding 57 projections formed the basis of the final BS, WS and TS (scenario cores) when considering the selected crisis event, that is, war, and in the context of perceived safety and DTR in healthcare. This process was based on a 5-year perspective. The TS represented a realistic future direction, whereas the BS represented additional future needs and objectives. Finally, the WS focused more on the future consequences of a restricted consideration of perceived safety in the context of DTR. In total, 92 participants participated in multiple phases, and 130 participants participated in all phases (see table 1).

Table 1 Participant characteristics regarding the scenario phases

Total: N=92 participants (130 participations in all phases including multiple participation)Phase 1: N=50 participants (51 participations including multiple participation)phase 2–5: N=49 participants (79 participations including multiple participation)	Total(without multiple participations)	Phase 1Scenario fieldidentification (WS/DEA)	Phase 2key factor identification (Survey)	Phase 3/4Key factor projection/scenario development	Phase 5Scenario finalisation and interpretation/transfer	
N=92	N=50*	N=35*	N=19*†	N=25*	
Project collaborators included	6 (6.52)	1	4	4	5	
Gender (%)	Female	65 (70.65)	37	25	11	13	
Male	25 (27.17)	12	9	8	12	
Various	2 (2.17)	1	1	–	–	
Age	In years RA±SD	11–8640.18±18.78	11–8640.04±21.98	20–6639.17±14.46	21–6742.89±14.60	21–7145.04±13.94	
Missing value (no data available)	1 (1.09)	–	–	1	1	
Migration background (%)	Immigrational	7 (7.61)	3	4	2	2	
Final school degree (%)	No degree	2 (2.17)	2	–	–	–	
Lower-level degree	10 (10.87)	10	–	–	–	
Average-level degree	7 (7.61)	3	4	1	1	
Higher-level degree	72 (78.26)	34	31	18	24	
Other (eg, primary school diploma)	1 (1.09)	1	–	–	–	
Education† (%)(multiple response)	No educational qualification	2 (2.17)	–	2	–	–	
Vocational (school or academy)	27 (29.35)	15	9	4	5	
Technical/master (school or academy)	2 (2.17)	1	1	–	–	
University or college degree	57 (61.96)	26	25	15	21	
Missing value (no data available)	5 (5.43)	5	–	–	–	
Other education (%)	For example, unspecified, gardener, carpenter, midwife, dental nurse, diploma, licence to practice, state exam, doctorate, professorship	24 (26.09)	5	13	8	14	
Target group roles (%)(multiple response)	Healthcare recipients	38 (41.30)	27	11	3	4	
Of this group, people living with dementia	2 (2.17)	2	–	–	–	
Healthcare providers	31 (33.70)	18	14	7	6	
Informal caregivers (family members)	3 (3.26)	2	1	–	1	
(Technology) experts	52 (56.52)	3	35	19	25	
Other, not further defined	17 (18.48)	–	6	5	11	
Disease (%)(n=50)	Chronic disease	10 (20.00)	10	NA	NA	NA	
Acute disease	1 (2.00)	1	NA	NA	NA	
Acute and chronic disease	5 (10.00)	5	NA	NA	NA	
Neither	34 (68.00)	34	NA	NA	NA	
Technical affinity (%)(n=50)	Highly tech-savvy	38 (76.00)	38	NA	NA	NA	
Moderately tech-savvy	2 (4.00)	2	NA	NA	NA	
Low tech-savvy	10 (20.00)	10	NA	NA	NA	
Expertise(n=49)(R in years)(multiple response)	Safety and security	3 (5–10)	NA	3 (5–10)	1 (10)	3 (5–10)	
Ethics	5 (6–18)	NA	1 (11)	–	5 (6–18)	
Law	2 (5)	NA	1 (5)	1 (5)	2 (5)	
Economics	5 (5–10)	NA	3 (5–10)	2 (5–10)	4 (5–10)	
Social	11 (3–30)	NA	7 (3–11)	6 (5–30)	6 (6–24)	
Health	16 (0.75–40)	NA	10 (0.75–40)	5 (3–40)	8 (3–40)	
Indicator development	1 (10)	NA	1 (10)	1 (10)	1 (10)	
Other expertise(n=49)(R in years)(multiple response)	Community responsibility	4 (5–10)	NA	3 (5–10)	4 (5–10)	4 (5–10)	
Research	5 (5–16)	NA	3 (5–16)	2 (7–10)	5 (5–16)	
Management, for example, (medical) quality management, business administration, patient logistics, project management, project manager	7 (5–50)	NA	6 (5–50)	2 (9–15)	3 (5–9)	
Management (head/chair), for example, teaching and studies commission, institute of health services research and clinical epidemiology, corporate development	2 (5–10)	NA	2 (5–10)	2 (5–10)	2 (5–10)	
Education (eg, teaching, (nursing) pedagogy)	13 (1–36)	NA	5 (5–36)	5 (5–36)	13 (1–36)	
Construction industry	1 (20)	NA	1 (20)	1 (20)	–	
Informal caregiving	1 (3)	NA	–	–	1 (3)	
Communication/marketing	2 (6)	NA	1 (6)	–	1 (6)	
Social design	1 (3)	NA	1 (3)	–	–	
Biology/pharmacy	2 (0.75)	NA	2 (0.75)	–	–	
Healthcare provision	4 (3–8)	NA	4 (3–8)	–	–	
Specialised doctor status (psychiatry, surgery)	2 (31–40)	NA	2 (31–40)	2 (31–40)	2 (31–40)	
Scholarship holder	1 (5)	NA	1 (5)	1 (5)	–	
Missing value (no data available)	13	NA	9	3	6	
* Participation of individuals who have also participated in other data collection methods.

† Five participants gave their written feedback.

AaverageDEAdesign ethnographic approachNAnot applicableRrangeWSworkshop

The scenario field

The scenario field was characterised by 16 overarching interrelated domains (see figure 2) based on 105 multifactorial dimensions of the influencing factors (see online supplemental appendix 1a) and 216 main categories that focused on several context levels, that is, the individual and the DT itself, the community/organisation and the system. Apart from the domain ‘equity’, which was only related to emotional safety, all domains included dimensions of both emotional and psychological safety. We observed that the scenario field was enriched by data collected via our various methods. Compared with our scoping review,6 almost two-thirds of the new main categories came from the workshops and use cases. The data indicated that perceived safety was related to positive and negative outcomes, among other outcomes, regarding DT implementation. A lower level of perceived safety could lead to avoidance of DT use, for example, when DTs do not meet the participant’s needs, when competences are limited, when negative feelings result in the continued use of analogue alternatives or when DTs are used differently, partially or only under certain conditions. In contrast, a higher level of perceived safety was associated with, for example, DT acceptance, adoption, adherence and positive feelings of well-being, trust and self-confidence. However, increased DT use can also promote refusal behaviours related to the unknown (see online supplemental appendix 1b).

Figure 2 Domains of emotional and psychological safety resulted from our data sources on system level; community/organisational level; individual level and digital technology level. Black sphere=covered by most frequently by 12 dimensions (3 data sources); grey sphere=covered by 3–11 dimensions (3 data sources); light grey sphere=covered by 6 and 3 dimensions (2 data sources); *Only emotional safety.

The domains contained several dimensions (range: n=3–12). Considering these results, the domain ‘knowledge and competence’ may be considered essential as it was emphasised most often (n=12) and contained most of the main categories. Here, perceived safety was related to, for example, digital (health) literacy and its promotion, empowerment, technical understanding and capabilities pertaining to DT use, and readiness. The domain ‘control’ may be also considered essential as it was emphasised very often (n=11) and from several perspectives, that is, being controlled by DT, having control due to DT and gaining control over DT. ‘Experiences and attitudes’ (n=10) were related to, for example, previous experiences, openness and the individual’s own habits/rituals. The other domains, for example, ‘perceived benefits in healthcare’ (n=8) and ‘perceived disadvantages in healthcare’ (n=7), were associated with positive and negative feelings of safety, respectively. In contrast, positively addressing ‘enhanced treatment options by DT’ could also result in negative impacts in cases featuring implementation concerns due to a lack of concepts regarding digital care. Although other domains were also associated with 7 or a lower number of dimensions, they were not of minor importance because they appeared in all data sources or in two data sources as in the case of ‘communication’ (n=6) and ‘professionality’ (n=3).

Identified KFs and projections

21 KFs were identified by the participants as very likely (probability, range=52.94%–91.42%) and particularly relevant (relevance, range=68.75%–96.97%) in the contexts of perceived safety and DT. The lowest probability was associated with the factor ‘research gaps regarding DT (…)’, whereas the highest probability was associated with two factors, that is, ‘understandable language/communication between humans and DT’ and ‘reliability of the DT (…)’. This factor was also rated as the highest in terms of relevance. The lowest value was associated with ‘professionality due to DT (…)’. The number of KFs could not be reduced by impact analysis (see online supplemental appendix 2a,b).

Final scenarios and scenario cores

On the basis of participant feedback, a qualitative synthesis of four factors resulted in a final set of 19 KFs, each of which was associated with the three (positive, negative and trend) projections. Specifically, (n=57) represents the (raw) BS, WS and TS (see table 2).

Table 2 19 final key factors and their projections according to the development of best, worst and trend scenarios

Key factor	DV	Five-year perspective on perceived safety in digital transformation	
Domain 1: equity	
 Accessibility of healthcare by the DT (no. 1)	B	Various (target group-specific) DTs are customised to the needs at hand so that they can facilitate accessibility and are barrier-free (including establishing a basis for DT use, eg, in terms of language).	
W	The situation will become more segregated (ie, more marginalising for certain target groups) than it is now.	
T	An increase in DT accessibility is a given, but no comprehensive, digital accessibility is established/barriers may persist and emerge in the context of DT.	
Domain 2: needs orientation	
 Demand-oriented, flexible availability and usability of DT in healthcare (no. 2)	B	The needs of the various interest groups are considered more frequently in terms of the needs-oriented, flexible availability and usability of DT. The trade-off between needs and requirements is transparent.	
W	The needs of the various interest groups are not sufficiently considered by the needs-oriented, flexible availability and usability of DT. The trade-off between needs and requirements is not sufficiently transparent.	
T	The needs of various interest groups are considered only in certain areas (improvement/stagnation) by the needs-oriented, flexible availability and usability of DT. The trade-off between needs and requirements is partially transparent.	
Domain 3: knowledge	
 Knowledge concerning the opportunities and limitations of DT in healthcare (no. 3)	B	The opportunities and limitations associated with DT are evident and depend on the complexity of DT. In addition, knowledge concerning DT is drawn from other sources of information (not exclusively from the DT itself) and becomes part of people’s everyday knowledge. Perceived safety is present due to knowledge transfer. The generational digital gap is closing (generational gap; people who are not digital natives).	
W	The generational digital gap continues to widen. In addition, users are becoming ‘knowledge saturated’ due to excessive information about DT. This information is deliberately ignored, which is associated with expected negative consequences.	
T	Differentiation between two groups (digital natives (++) and nonnatives (0)) depending on the technical complexity of DT. The ‘generational digital gap’ (digital skills) is increasing.	
Domain 4: technical functionality	
 Reliability of DT functionality, operability, freedom from interference in healthcare (no. 4./5)*	B	The reliability of DT improves, taking ethical aspects into account (provider-independent and platform-independent reliability).	
W	The reliability of DT deteriorates and consideration of ethical aspects is low (provider-dependent and platform-dependent unreliability).	
T	The reliability of DT improves in line with current progress in certain areas, which is associated with increasing, although occasional, consideration of ethical aspects (provider-dependent and platform-dependent reliability).	
Domain 5: professionality	
 Professionality due to DT in healthcare (no. 6)	B	Professionalism increases based on an adequate user interface (human‒machine interaction) and the serious design of the DT itself.	
W	Professionalism deteriorates due to an inadequate and overly complex user interface (human‒machine interaction) of the DT.	
T	Professionalism improves and deteriorates depending on the area in question due to very slow development and an overly complex user interface (human‒machine interaction) of the DT.	
Domain 6: communication	
 Understandable and visualisable communication/ interaction with people due to DT in healthcare (no. 7/8)*	B	Human-centred and individualised (understandable) communication/interaction with humans due to DT contributes in particular to clarity and enlightenment (visualisation capability must be available).	
W	Target group-oriented communication/interaction (understandability) on the part of DT, in which context humans play no role. The capabilities of visualisation lead to a lack of transparency (a black box; the difference between humans and DT as counterparts is no longer transparent).	
T	Opportunities for (visualised and intelligible) communication/interaction with humans due to DT are being expanded and increasing the level of perceived safety, even if not all target groups are considered; furthermore, intuitive communication between humas and DT has not been achieved or made all-encompassing.	
Domain 7: data security and protection	
 Data security related to protection against cyber-attacks in healthcare (no. 9)	B	High data security standards have been implemented and are being further developed with vigour and sufficient resources, leading to perceived safety.	
W	External (eg, economic and political) incentives for data theft are increasing. The data security situation is deteriorating.	
T	Data insecurity/data threat (leading to a decrease in perceived safety) and awareness of data security (eg, legislation is being adapted) are increasing.	
 Data protection (privacy and anonymity) and transparency of data management of DT in healthcare (no. 10)	B	A balance between data protection regulation and the provision of healthcare is established. Adequate data protection is necessary for objective medical emergencies (eg, the physician providing treatment receives patient data access in the event of unconsciousness).	
W	An imbalance between data protection regulation and health-related services is evident. Excessive data protection regulations stifle the ability of institutions and stakeholders to act. Insufficient data protection regulations are evident. Data protection depends on the life situation.	
T	A consistent level of patient autonomy regarding data protection is evident. A fragile and vulnerable balance between data protection regulation and health-related services is established.	
Domain 8: support	
 Available personnel to support the implementation of DT in healthcare (no. 11)	B	Personnel resources are available in a customised manner or can be made available if required and are competent.	
W	Human resources are becoming scarcer as jobs for DT are cut.	
T	Human resources to support the implementation of DT remain scarce (with a negative effect on perceived safety).	
Domain 9: control	
 Controllability of the functionality of DT in healthcare (no. 12)	B	High standards with regard to the controllability of the functionality of DT that are user-friendly.	
W	Insufficient evidence-based standards for monitoring DT functionality (hardly any and tendency way of (state) bodies to act (gatekeepers)).	
T	Depending on the device and its self-diagnostic function, digital assistance, correct functionality cannot be assessed by ‘everyone’ (competence and demand for controllability of DT are limited) (possibly with a negative influence on perceived safety).	
Domain 10: quality requirements	
 Perceptible high quality of DT in healthcare (national quality seal) (no. 13)	B	Criteria for the authorisation procedure are improving (high-quality materials, haptics, applicability); recommendations by independent healthcare providers are provided.	
W	Healthcare providers are receiving a reputation of being for sale due to economic interests and influence. DTs no longer meet existing standards. High quality is simulated by the use of high-quality materials.	
T	Existing authorisation procedures (standards) are continued and further developed.	
Domain 11: confidence	
 Trust as a prerequisite for the use of DT in healthcare (no. 14)	B	Conditions conducive to trust in DTs are established.	
W	Absolute mistrust/fear of using DTs.	
T	Trust depends on personal prerequisites (status as a digital native) and on habits in digital interaction.	
 Self-confidence in addressing DT in healthcare (no. 15)	B	Every user gains self-confidence in their ability to address DT, such as by using or practising with it and through a good, intelligible introduction provided by the DT itself/or by another person.	
W	Users who lack confidence with regard to using DTs are excluded.	
T	No widespread self-confidence in addressing DTs and the differences in self-confidence among different users remain (possibly with a negative influence on perceived safety).	
Domain 12: design and handling	
 Easy, efficient handling of DT in healthcare (no. 16)	B	Easy, efficient handling of DT is viewed as a given, despite increasingly complex DTs.	
W†	Easy, efficient handling of DT is limited by poor underlying analogue processes and structures that are copied into DT without reflection.	
T†	Easy, efficient handling of DT is limited by poor underlying analogue processes and structures that are copied into DT without reflection (possibly with a negative influence on perceived safety).	
 Suitability of DT for everyday use in healthcare (no. 17)	B	The suitability of DT for everyday use is increased by taking into account the real-life needs of the particular target groups.	
W	The suitability of DT for everyday use is not enhanced by taking the everyday needs of the particular target groups less into account.	
T	The suitability of DT for everyday use is inadequately enhanced by insufficient consideration of the everyday needs of the particular target groups.	
Domain 13: considerations	
 Consideration of healthcare receiver health status and resources when using DT in healthcare (no. 18)	B	Optimal user-centredness and targeted consideration of the healthcare receivers’ state of health and resources (in light of their individual needs) when using DTs.	
W	Less consideration of the healthcare receivers (in light of their individual needs) when using DTs.	
T	Increasing consideration of the healthcare receivers’ state of health and resources (in light of their individual needs) when using DTs.	
Domain 14: attitude	
 Openness of healthcare receivers with respect to the use of DT in healthcare (no. 19)	B	Openness towards DT has no influence on the quality of healthcare, for example, through the freedom of choice with regard to whether and how DT is used (general, positive openness arises from transparency and perceived validity by the media and in external communication). No disadvantages/stigmatisation of conventional continued use of DTs (or lack thereof) are evident.	
W	Closing one’s mind to DT/rejecting DT leads to poorer healthcare.	
T	Openness towards DT continues depending on the personal user experience (even in areas that are not directly related to healthcare DTs) and the prerequisites (access options) of the users.	
Domain 15: perceived benefits in healthcare	
 Autonomy (maintenance) and participation due to DT in healthcare (no. 20)	B	DT facilitates a significantly higher degree of autonomy and participation.	
W	DT facilitates autonomy and participation only in certain areas.	
T	Continuous expansion of autonomy and participation	
Domain 16: perceived disadvantages in healthcare	
 Research gaps regarding to DT in healthcare (no. 21)	B	Standards for the approval of DTs are increasing, as no research gaps are tolerated before the DTs are brought to the market/applied.	
W	Excessive research gaps are tolerated. DT use is enforced. Research gaps cannot be problematised (or politicised) by users.	
T	The pressure to bridge research gaps increases. Research gap exists but do not appear to have an excessively high impact on healthcare receiver’ perceived safety regarding DT use; however, they may have a strong impact on healthcare providers. Research entails a promise of impact. Certain groups pay attention to the evidence while other groups do not.	
* (‘Reliable functionality and freedom from interference during DT use’/‘reliability of the DT (functionality, correctness of data acquisition and diagnostics)’ and ‘opportunity of visual interaction and communication due DT’/‘understandable language/communication between humans and DT’) were summarised based on participant feedback.

† Equal projection development.

BbestDTdigital technologyDVdevelopmentTtrendWworst

Although we observed a mix of BS, WS and TS projections, considering the consistency of the raw scenarios, the following exhibited a good fit: BS (C=1.80), WS (C=1.60) and TS (1.00). During the scenario process, the TS projections that appeared in the WS or BS could result in corresponding developments. The WS, which included trend projections, showed slightly better consistency than did the alternatives (1.63 vs 1.60). Thus, the more consistent scenario was chosen. The TS projections included were as follows: ‘Human resources to support the implementation of DT remain scarce (with a negative effect on perceived safety)’ and ‘easy, efficient handling of DT is limited by poor underlying analogue processes and structures that are copied into DT without reflection (…)’. The qualitative data also indicated that the latter projection was equal to the WS/TS projection. The included trend projections in the BS were not chosen due to their lower consistency. Finally, the contribution values provided an overview of the core elements of each scenario (see online supplemental appendix 2b).

These scenarios and the final scenarios were approved by the participants in light of the qualitatively developed scenario cores. For all scenarios, especially those pertaining to the BS, the projection of ‘demand-oriented, flexible availability and usability of DT’ (value of 1.0) was critical. With respect to the WS, three further projections (each 0.9) offered the greatest contributions: ‘knowledge concerning the opportunities and limitations of DT (…)’; ‘consideration of (HCRs’) health status and resources when using DT (…)’; and ‘professionalism due to DT (…)’. With respect to the TS, ‘self-confidence in addressing DT (…)’ was associated with the highest contribution value (0.9). Although the quantitative data focused on slightly different aspects than did the qualitative data, the participants completed the following three scenario cores, captured the KFs and, to some degree, the consequences: (BS) ‘the user in focus (…) reliability, user-friendliness, equity of access, suitability of DT for everyday use’; (WS) ‘lasting damage to the individual, the organisation and society’ and (TS) ‘act of establishing a balance among various demands and realities pertaining to digitalisation in healthcare - self-confidence/trust (as key) - so that the gap between people who are digitally ‘left behind’ and those who are digitally competent’ can be considered. At the core, the TS emphasises ‘tensions between requirements for change and reality’ (see figure 3 and online supplemental appendix 3a, b).

Figure 3 Trend scenario.

Consequences of the scenarios

Ethical, psychosocial, health-related, legal, political and economic consequences were determined, and controversies related to digital (health) literacy or affinity were identified. For example, in the case of the BS, many KFs could be improved over 5 years from an ethical perspective, with the exception that not every target group could be reached. This situation might be characterised by ambivalence towards self-determination options in the context of digitalised healthcare (technology open vs technology denying) by questioning the self-determination options of technology-denying people. Hence, in the case of the WS and TS, social divisions may occur due to the inadequate and unequal distribution of digital healthcare services. Political radicalisation and a lack of solidarity resulting from a low-needs orientation are expected, but an increasing aversion to (digital) healthcare is also expected due to inhibitory attitudes among the population. In the case of the TS, negative consequences were observed regarding perceptions of inequalities in healthcare depending on digital competence given that being digitally competent could be related to loss perceptions resulting from inequalities due to equalisation with respect to digital offers, whereas being less digitally competent may result in a feeling of need and being left alone.

Considering the relevant legal aspects for both the BS and TS, new requirements for legislation regarding DTR are necessary to consider the context of perceived safety. In the case of the TS, the delegitimisation of the system (increasing the (legal) probability of legal action), for example, due to unequal opportunities, could be expected. With respect to politics in the case of the TS, perceptions of responsibility are emphasised, whereas with respect to education in the case of the WS, a loss of competencies of care is expected. With respect to the research, additional private research findings are expected to be forthcoming. From an economic perspective, in the case of the BS/TS, increased efficiency regarding the provision of healthcare via DTs is expected, but in the case of the TS, this expectation holds only among HCPs and digitally competent people. In contrast, digital and analogue healthcare inefficiencies are expected in the case of the TS, especially among people with a lower level of digital competence. Regarding the WS, setbacks in DT development and usage as well as negative effects regarding patient safety are expected.

In general, disadvantages in healthcare are assumed to exacerbate or preserve the inequality of opportunities and the inequities regarding healthcare access. In contrast, a newly informed culture of digitalisation that is enlightened by politics may be implemented in society. Alternatively, however, a balanced relationship between humans and DT in healthcare that considers the importance of people’s concerns, replacement and dehumanisation may emerge or increased healthcare quality may result.

Scenarios, scenario cores, corresponding consequences: considering crisis events

The crisis war was chosen after two election rounds (pro a12/b10), followed by climate change (pro a11/b5) and cyber-attacks (pro a11/b1). All scenarios were characterised by a point in time at which the crisis would occur and be associated with controversies. If a war were to occur instantly, the insufficient perceived safety associated with DTR would have consequences concerning the resilience of the healthcare system. In other words, the scenarios lose relevance (see online supplemental appendix 3c). Such a crisis could be characterised by a general shift in priorities and the threat of digital warfare with respect to health data. However, it was also expected that a war could lead to innovations in healthcare through transfer actions.

In cases in which the different scenarios were reached over 5 years, the TS was determined to be the most vulnerable because of the high degree of uncertainty resulting from persistent and exacerbated disadvantages in healthcare, for example, a reduction in equal opportunities and access due to the absence of backup systems (analogue/digital) or the absence of advantages resulting from the other scenarios. Knowledge, however, can be drawn from other parties. For example, the BS and WS were simultaneously viewed positively allowing the digital communication could be performed (BS) and analogue’s resilient backup of healthcare structures remains available (WS). Although perceived safety decreases in the case of war, acceptance of DTs could increase.

Participants’ overarching recommendations for action

The overarching recommendations for action provided by participants were related to education, politics and research and did not differ even when a crisis was considered. In summary, promoting digital (health) literacy throughout society (global), establishing a low threshold for access to care, reducing inequalities and offering consults and teaching in digitalisation were emphasised. Furthermore, guaranteeing technical standards, enhancing the credibility and legitimacy of healthcare policy, and facilitating the efficient implementation of DTs were viewed as necessary. Finally, open and transparent scientific communication, which can also ensure high-quality and multidimensional research, was recommended.

Discussion

To the best of the authors’ knowledge, this research represents the first 5 year foresight study on perceived safety in DTR and shows that perceived safety plays a critical role in both present and future healthcare in a complex and controversial way, including in the case of a crisis. Depending on how perceived safety is considered, this factor continues to impact DTR at the individual, organisational and system levels.

Many factors were determined to be related to perceived safety and DT, and the identified needs and influencing factors impacted the healthcare system, including successful implementation, in several ways. Consistent with the psychological view of perceived safety,9 it was further observed that social aspects are relevant for perceived safety and that, according to recent publications,8 patient safety is interrelated. However, similar to models of perceived safety in inpatient care,25 it was noted that perceived safety is a complex phenomenon. When considering DTs in this context, we were able to show that multiple dimensions are essential for emotional and psychological safety and that these dimensions extend beyond previous studies6 that have been restricted by a lower number of selected DTs, such as those associated with robotics and the target groups on which they have focused, that is, older people and their relatives. Nevertheless, depending on the context of the selected DTs, stakeholders and settings, the results herein are somewhat comparable to the extant research. That said, we were able to obtain a broader and more detailed picture because we used several methodical approaches that considered a wide range of DTs and target groups. Central to our research, we determined that, on the basis of all the knowledge acquired from the scenarios and related discussion points, our findings are in line with existing reflections on DT implementation26 and interventions aimed at facilitating improvement, such as strengthening digital (health) literacy through education in the context of DTR while also taking patient safety into account.1 2 Other published scenarios regarding digital health also indicate that health and data literacy is essential.27 However, the degree to which perceived safety is considered is low, in contrast to our study that showed that knowledge and competence are essentially related to perceived safety.

Furthermore, reflections on the implementation of DT and considering crises such as COVID-19 indicate that thoughts of DT focus on the context of the needs of HCRs, the roles of HCPs, the benefits of DT and the issues pertaining to implementation.26 Most interestingly, this discussion is in line with our results regarding the transfer of analogue-related and DT-related healthcare, and it highlights the need for the fundamental reorganisation of processes and structures.

In our study, the prioritisation of factors that are crucial for the contemporary world differed from the KFs that were most relevant in the scenarios; they also differed slightly when examining the quantitative and qualitative data. These differences could be related to the processes of negotiation that characterise the use of data collected via mixed methods, including member verification, which captured both the present and the future. These methodological challenges have been identified as representing one of the core challenges of mixed-method research.15 28 However, the fact that various degrees of data abstraction16 were used, ranging from concrete data of use cases to abstract projected KFs, may explain this situation. Moreover, these challenges can be viewed as the keys to DTR. From the perspective of complexity theory, the whole system should be considered while simultaneously considering its parts. Nonetheless, more attention should be given to the system process than to its state. This process focuses on (implementation) learning29 30 and identifying interacting factors rather than making a decision for one determinant (interrelationships).30 Thus, this process enables us to obtain a differentiated picture of the various factors that are relevant in this context and the reasons underlying that relevance.30 Taking these approaches into account, the data collected for this study have several uses. The dimensions that characterise the present situation include concrete examples of events occurring in real-world healthcare settings, which may enhance the intelligibility of practice. The overarching scenario cores, for example, digital (health) literacy as expressed in terms of ‘(…) the gap between people who are digitally ‘left behind’ and those who are digitally competent’, allow us to obtain an overview of future educational, political, and research-related trends pertaining to DTR and decision-making.

Limitations

We employed a comprehensive, multiperspective, strategic 5-year foresight approach. In general, the scenario approach is perceived as acceptable considering current discussions regarding consistency and traceability. Consistency is perceived as a ‘substitute for empirical validation’31 in light of the inherent problem of anticipated objective accessibility and the corresponding complex and adaptive characteristics.31 However, our research followed the usual standards of foresight research.16 Although participatory research14 offered a high level of added value to our study, we concluded that participants’ engagement in a study over 2 years is very time-consuming and demanding for the participants given the usual demands of daily life.

Conclusion

This foresight study, which was conducted from a 5-year perspective, represents the first study to offer a broad and multifaceted picture of perceived safety in the context of DTR. The findings indicate that perceived safety in a DTR environment is influenced by several factors and has substantial impacts on DT implementation and healthcare. The scenarios developed as part of this research suggest that continuing the current state over the next 5 years may result, to some degree, in controversial outcomes. Therefore, concrete results could be used in practice to support education and healthcare quality development and research. These scenarios could further serve as additional starting points for research agenda setting and political decision-making for future healthcare. However, future research is needed to quantify the factors’ correlations with perceived safety and the corresponding impacts on DTR.

supplementary material

10.1136/bmjhci-2024-101048 online supplemental file 1

Acknowledgements

We are thankful to Ewa Dönitz, who provided expertise regarding consistency analysis.

Data availability statement

Data are available on reasonable request. All data relevant to the study are included in the article or uploaded as online supplemental information.

Funding: The SteTiG project titled 'Emotional safety as a condition for success in the digital transformation in healthcare' is funded by the Federal Ministry of Education and Research (BMBF) (funding no. 16INS108).

Provenance and peer review: Not commissioned; externally peer reviewed.

Patient consent for publication: Not applicable.

Ethics approval: This study involves human participants and ethical advice was obtained from the Ethical Committee of Ärztekammer Nordrhein: 2022107and Ethical Committee of Fliedner Fachhochschule Düsseldorf: 04/06-2022. Participants gave informed consent to participate in the study before taking part.
==== Refs
References

1 Stoumpos AI Kitsios F Talias MA Digital Transformation in Healthcare: Technology Acceptance and Its Applications Int J Environ Res Public Health 2023 20 3407 10.3390/ijerph20043407 36834105
2 Kraus S Schiavone F Pluzhnikova A et al Digital transformation in healthcare: analyzing the current state-of-research J Bus Res 2021 123 557 67 10.1016/j.jbusres.2020.10.030
3 Iyamu I Xu AXT Gómez-Ramírez O et al Defining Digital Public Health and the Role of Digitization, Digitalization, and Digital Transformation: Scoping Review JMIR Public Health Surveill 2021 7 e30399 10.2196/30399 34842555
4 Senbekov M Saliev T Bukeyeva Z et al The Recent Progress and Applications of Digital Technologies in Healthcare: A Review Int J Telemed Appl 2020 2020 8830200 10.1155/2020/8830200 33343657
5 Åkerlind C Martin L Gustafsson C eHomecare and safety: The experiences of older patients and their relatives Geriatr Nurs 2018 39 178 85 10.1016/j.gerinurse.2017.08.004 28988833
6 Minartz P Aumann CM Vondeberg C et al Feeling safe in the context of digitalization in healthcare: a scoping review Syst Rev 2024 13 62 10.1186/s13643-024-02465-9 38331923
7 Veale D Robins E Thomson AB et al No safety without emotional safety Lancet Psychiatry 2023 10 65 70 10.1016/S2215-0366(22)00373-X 36442491
8 Lyndon A Davis D-A Sharma AE et al Emotional safety is patient safety BMJ Qual Saf 2023 32 369 72 10.1136/bmjqs-2022-015573
9 Eller E Frey D Psychological perspectives on perceived safety: social factors of feeling safe Perceived safety 2019 43 60
10 Edmondson AC Bransby DP Psychological safety comes of age: observed themes in an established literature Annu Rev Organ Psychol Organ Behav 2023 10 55 78 10.1146/annurev-orgpsych-120920-055217
11 Grailey KE Murray E Reader T et al The presence and potential impact of psychological safety in the healthcare setting: an evidence synthesis BMC Health Serv Res 2021 21 773 10.1186/s12913-021-06740-6 34353319
12 Nakrem S Solbjør M Pettersen IN et al Care relationships at stake? Home healthcare professionals’ experiences with digital medicine dispensers - a qualitative study BMC Health Serv Res 2018 18 26 10.1186/s12913-018-2835-1 29334953
13 Iden J Methlie LB Christensen GE The nature of strategic foresight research: a systematic literature review Technol Forecast Soc Change 2017 116 87 97 10.1016/j.techfore.2016.11.002
14 Vaughn LM Jacquez F Participatory research methods – choice points in the research process J Particip Res Methods 2020 1 10.35844/001c.13244
15 Birt L Scott S Cavers D et al Member Checking: A Tool to Enhance Trustworthiness or Merely a Nod to Validation? Qual Health Res 2016 26 1802 11 10.1177/1049732316654870 27340178
16 Kosow H Gaßner R Methods of future and scenario analysis: overview, assessment, and selection criteria 2008
17 Vöcking M Karrenbrock A Beckmann A et al Emotional and Psychological Safety in Healthcare Digitalization: A Design Ethnographic Study Int J Public Health 2024 69 10.3389/ijph.2024.1607575
18 World Health Organization Classification of digital health interventions v1. 0: a shared language to describe the uses of digital technology for health 2018
19 Krippendorff K Content Analysis: An Introduction to Its Methodology Thousand Oaks, California German Development Institute (DIE) 2019
20 Sandelowski M Voils CI Leeman J et al Mapping the Mixed Methods-Mixed Research Synthesis Terrain J Mix Methods Res 2012 6 317 31 10.1177/1558689811427913 23066379
21 Eysenbach G Improving the quality of Web surveys: the Checklist for Reporting Results of Internet E-Surveys (CHERRIES) J Med Internet Res 2004 6 e34 10.2196/jmir.6.3.e34 15471760
22 Tietje O Identification of a small reliable and efficient set of consistent scenarios Eur J Oper Res 2005 162 418 32 10.1016/j.ejor.2003.08.054
23 Bishop P Hines A Collins T The current state of scenario development: an overview of techniques Foresight (Colch) 2007 9 5 25 10.1108/14636680710727516
24 Tourki Y Keisler J Linkov I Scenario analysis: a review of methods and applications for engineering and environmental systems Environ Syst Decis 2013 33 3 20 10.1007/s10669-013-9437-6
25 Yan L Liu L Wang F et al Barriers and facilitators to feeling safe for inpatients: a model based on a qualitative meta-synthesis Front Public Health 2024 12 1308258 10.3389/fpubh.2024.1308258 38481849
26 Kaye R Rosen-Zvi M Ron R Digitally-Enabled Remote Care for Cancer Patients: Here to Stay Semin Oncol Nurs 2020 36 151091 10.1016/j.soncn.2020.151091 33223410
27 Volpert A Hery D Kraushaar J et al Digitale Gesundheitsversorgung 2033: trends Szenarien und Thesen Fraunhofer-Gesellschaft eV, München 2023 2023
28 Dawadi S Shrestha S Giri RA Mixed-methods research: a discussion on its types, challenges, and criticisms J P S E 2021 2 25 36 10.46809/jpse.v2i2.20
29 Long KM McDermott F Meadows GN Being pragmatic about healthcare complexity: our experiences applying complexity theory and pragmatism to health services research BMC Med 2018 16 94 10.1186/s12916-018-1087-6 29921277
30 Greenhalgh T Papoutsi C Studying complexity in health services research: desperately seeking an overdue paradigm shift BMC Med 2018 16 95 10.1186/s12916-018-1089-4 29921272
31 Kosow H New outlooks in traceability and consistency of integrated scenarios Eur J Futures Res 2015 3 10.1007/s40309-015-0077-6
