
==== Front
BMJ Open
BMJ Open
bmjopen
bmjopen
BMJ Open
2044-6055
BMJ Publishing Group BMA House, Tavistock Square, London, WC1H 9JR

39260855
10.1136/bmjopen-2024-084398
bmjopen-2024-084398
Original Research
Health Informatics
1702
1506
Barriers and facilitators to implementing imaging-based diagnostic artificial intelligence-assisted decision-making software in hospitals in China: a qualitative study using the updated Consolidated Framework for Implementation Research
http://orcid.org/0000-0002-2349-8775
Liao Xiwen 12ceceliao94@outlook.com

Yao Chen 12yaochen.pucri@foxmail.com

http://orcid.org/0000-0002-4991-0158
Jin Feifei 34feifeijin92@163.com

Zhang Jun 5jun.zhang19@merck.com

Liu Larry 67liu.larry@merck.com

1 Peking University First Hospital, Beijing, China
2 Clinical Research Institute, Institute of Advanced Clinical Medicine, Peking University, Beijing, China
3 Trauma Medicine Center, Peking University People's Hospital, Beijing, China
4 Key Laboratory of Trauma treatment and Neural Regeneration, Peking University, Ministry of Education, Beijing, China
5 MSD R&D (China) Co., Ltd, Beijing, China
6 Merck & Co Inc, Rahway, New Jersey, USA
7 Weill Cornell Medical College, New York City, New York, USA
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

JZ is an employee of MSD R&D (China). LL is affiliated with Merck Sharp & Dohme, a subsidiary of Merck & Co., Rahway, New Jersey, USA, which funded the study and monitored the conduct of the study.

ChenYao; yaochen.pucri@foxmail.com
2024
10 9 2024
14 9 e08439817 1 2024
26 8 2024
Copyright © Author(s) (or their employer(s)) 2024. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.
2024
https://creativecommons.org/licenses/by-nc/4.0/ This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

Abstract

Objectives

To identify the barriers and facilitators to the successful implementation of imaging-based diagnostic artificial intelligence (AI)-assisted decision-making software in China, using the updated Consolidated Framework for Implementation Research (CFIR) as a theoretical basis to develop strategies that promote effective implementation.

Design

This qualitative study involved semistructured interviews with key stakeholders from both clinical settings and industry. Interview guide development, coding, analysis and reporting of findings were thoroughly informed by the updated CFIR.

Setting

Four healthcare institutions in Beijing and Shanghai and two vendors of AI-assisted decision-making software for lung nodules detection and diabetic retinopathy screening were selected based on purposive sampling.

Participants

A total of 23 healthcare practitioners, 6 hospital informatics specialists, 4 hospital administrators and 7 vendors of the selected AI-assisted decision-making software were included in the study.

Results

Within the 5 CFIR domains, 10 constructs were identified as barriers, 8 as facilitators and 3 as both barriers and facilitators. Major barriers included unsatisfactory clinical performance (Innovation); lack of collaborative network between primary and tertiary hospitals, lack of information security measures and certification (outer setting); suboptimal data quality, misalignment between software functions and goals of healthcare institutions (inner setting); unmet clinical needs (individuals). Key facilitators were strong empirical evidence of effectiveness, improved clinical efficiency (innovation); national guidelines related to AI, deployment of AI software in peer hospitals (outer setting); integration of AI software into existing hospital systems (inner setting) and involvement of clinicians (implementation process).

Conclusions

The study findings contributed to the ongoing exploration of AI integration in healthcare from the perspective of China, emphasising the need for a comprehensive approach considering both innovation-specific factors and the broader organisational and contextual dynamics. As China and other developing countries continue to advance in adopting AI technologies, the derived insights could further inform healthcare practitioners, industry stakeholders and policy-makers, guiding policies and practices that promote the successful implementation of imaging-based diagnostic AI-assisted decision-making software in healthcare for optimal patient care.

Clinical Decision-Making
Implementation Science
China
Information technology
MSD R&D (China) Co., Ltd. N/A
==== Body
pmcSTRENGTHS AND LIMITATIONS OF THIS STUDY

Used the updated Consolidated Framework for Implementation Research to systematically identify barriers and facilitators.

Conducted semistructured interviews with a wide range of key stakeholders, both from clinical settings and industry.

Potential generalisability limitations due to purposive sampling of artificial intelligence software and the cluster of healthcare institutions and study participants in big cities.

The inclusion of perspectives from patients should be addressed in future research.

Introduction

Clinical decision-making (CDM) is a challenging and complex process.1 Effective and informed CDM requires a delicate balance between the best available evidence, environmental and organisational factors, knowledge of the patient and comprehensive professional capabilities, such as clinical skills and experiences.2 3 However, the ability of healthcare professionals to make such decision is often restricted by the dynamic and uncertain nature of clinical practices.3 In response, decision-making tools have been developed to enhance and streamline CDM for optimal healthcare outcomes. Traditional decision-making tools heavily depend on computerised clinical knowledge bases, supporting CDM by matching individual patient data with the knowledge base to provide patient-specific assessments or recommendations.4 5 However, relying solely on knowledge-based tools has become insufficient to fulfil the growing need for accessible, efficient and personalised healthcare services, due to its inherent limitations such as time-consuming processes, disruptions in routine clinical workflow and challenges in constructing complex queries.6 7

Non-knowledge-based decision support tools, on the other hand, harness artificial intelligence (AI) algorithms to analyse large and complex datasets and learn continuously for more accurate and individualised recommendations.5 8 AI technology has been rapidly advancing since 2000, unleashing substantial potential to revolutionise the conventional CDM process and driving a fundamental shift in healthcare paradigm.9 The development and extensive growth of clinical real-world data (RWD) have made integrating AI technology into the healthcare sector a priority for both the healthcare industry and regulatory agencies.

The US Food and Drug Administration (FDA) has well recognised the use of AI techniques combined with clinical RWD for both drug and medical device development. In drug development, the FDA has reported a marked increase in the number of drug and biological application submissions with AI components across different stages of life cycle.10 AI algorithms have been actively integrated into biomarker discovery, eligible population identification and prescreening, clinical drug repurposing, and adverse event (AE) detection, ranging from drug discovery and premarket clinical studies to postmarket safety surveillance.11 Particularly, the number of studies on AE detection with the use of natural language processing increased from 1 between 2005 and 2018 to 32 between 2017 and 2020.11

In addition to the pharmaceutical industry, AI medical devices, whether intended for decision-making or other purposes, have experienced rapid development between 2000 and 2018.12 During this period, the growing sophistication of imaging medical supplies, such as CT scanners, has led to an exponential increase in the volume of high-dimensional imaging data. This surge has gradually shifted the focus of AI medical devices towards imaging-based diagnostic decision-making software, making lung nodule detection and diabetic retinopathy screening popular areas of research.13 These AI systems, classified as software as medical devices, are designed to support diagnostic decision-making using clinical RWD, particularly imaging data generated by medical devices, leveraging AI technology to perform functions independently of hardware medical devices.1416 Notably, the FDA’s approval of the first imaging-based AI-assisted medical device for detecting diabetes-related eye diseases in 2018 marked major progress towards the implementation of imaging-based diagnostic AI-assisted decision-making software.17 In China, a major milestone was achieved with the approval of the first AI-related software for coronary artery fractional flow reserve by the National Medical Products Administration (NMPA) in 2020,18 highlighting the continuous development and integration of AI technologies in healthcare practices. Following this breakthrough, AI-assisted decision-making software has experienced rising popularity in China. The number of regulatory approvals for AI software expanded from 1 in 2020 to 62 in 2022.12 Key functions included disease identification, lesion segmentation, and risk prediction, and risk classification, covering therapeutic areas from cardiovascular diseases to various types of cancers.12 19

Evidence from randomised controlled trials has demonstrated the safety and effectiveness of AI-assisted decision-making software across various therapeutic areas. For disease detection, these tools have facilitated the early identification of patients with low ejection fraction, improved detection rates for actionable lung nodules and increased the identification of easily missed polyps.2023 Furthermore, AI software has significantly reduced diagnostic times compared with senior consultants in diagnosing childhood cataracts, improving clinical efficacy.24 In disease management, AI-assisted decision-making software has decreased treatment delays for cardiovascular diseases and lowered in-hospital mortality for severe sepsis,25 26 contributing to improved healthcare quality. Additionally, studies have indicated that AI tools are effective across diverse populations, significantly enhancing follow-up rates for diabetic retinopathy in paediatric populations and improving referral adherence in adult patients within low-resource settings.27 28 However, a priori scoping review found a disparity between substantial research investment and limited real-world clinical implementation.19 Further, employing stratified cluster sampling from six provinces across China, a study revealed that only 23.75% of the surveyed hospitals had implemented AI-assisted decision-making software.29 Accordingly, existing literature has emphasised the deficiency in implementation science expertise for understanding AI implementation efforts in clinical settings.30 To bridge the gap, the current study aimed to explore the barriers and facilitators of implementing existing imaging-based diagnostic AI-assisted decision-making software in China through qualitative interviews, using the updated Consolidated Framework for Implementation Research (CFIR). The key strength of using the updated CFIR lied in its adaptability to capture both the breadth and depth of qualitative data, as well as its applicability to explore the implementation of technology across various healthcare settings, further enhancing the rigour and comprehensiveness.31 32 In addition, the study provided tailored implementation strategies to address key barriers, exploiting the full potential of imaging-based diagnostic AI-assisted decision-making software for the improved quality of care in China.

Methods

Innovation selection

A recent scoping review identified imaging-based diagnostic decision-making software using medical imaging data as the predominant AI-assisted decision-making software in China, especially those designed for the lung nodules detection and diabetic retinopathy screening.19 To manage the increasing submissions in these areas, the Center for Medical Device Evaluation of NMPA issued two corresponding guidelines in 2022, delineating specific regulatory requirements.33 34 Therefore, the current study purposively chose to investigate AI-assisted decision-making software for lung nodules and diabetic retinopathy screening, serving as two representative imaging-based diagnostic AI applications. Two lists were obtained from the NMPA database, from which one software for lung nodules and another for diabetic retinopathy were ultimately selected (online supplemental tables 1 and 2). The vendors were selected through convenience sampling and their voluntary consent to participate. Characteristics of the selected AI-assisted decision-making software are shown in table 1.

Table 1 Characteristics of the selected AI-assisted decision-making software

	EyeWisdom	Dr. Wise AI-assisted diagnosis system for lung CT	
Vendor	Vistel Technology	Deepwise Technology	
Therapeutic area	Diabetic retinopathy	Lung nodules	
NMPA approval year	2021	2020	
Classification	Class III	Class III	
Intended use	Designed to analyse non-mydriatic colour fundus images of both eyes in adult diabetic patients, the system provides healthcare practitioners with diagnostic recommendations on whether moderate non-proliferative diabetic retinopathy (or higher) is detected.	Designed for the display, processing, measurement, and analysis of chest CT images, the system can automatically identify lung nodules of 4 mm and larger and analyse their radiological characteristics.	
NMPANational Medical Products Administration

Study setting and participants

With the aim of reflecting diversified perspectives from a wide range of stakeholders, participants from both clinical settings and industry were included. Specifically, clinical stakeholders consisted of three different roles, including healthcare practitioners, hospital informatics specialists and hospital administrators. Industry stakeholders were vendors of the selected AI-assisted decision-making software, which was further divided into three subroles, including data scientists, database experts and algorithm engineers. While the perspectives of patients are valuable, the current study aimed to gather in-depth insights regarding practical and systemic challenges from stakeholders who are directly involved in the implementation, deployment and development of the selected AI-assisted decision-making software. The included stakeholders possessed either the operational or technical knowledge necessary to identify specific barriers and facilitators related to the implementation of the selected AI software in clinical settings. Thus, patients were not included as a stakeholder group in this study.

The selection of study participants involved a two-stage process, wherein initial screening occurred at the institutional level, followed by the individual-level selection. For clinical stakeholders, two lists of hospitals that implemented the selected AI-assisted decision-making software were acquired from the corresponding software vendor. Employing stratified purposive sampling, one tertiary hospital and one primary or secondary hospital were selected from each of the lists, respectively. All selected healthcare institutions were located in big cities, including the Cancer Hospital of the Chinese Academy of Medical Sciences, Beijing Hospital, Beijing Shuili Hospital and the community healthcare centre of Qingpu, Shanghai. The snowball sampling technique was subsequently used to select study participants by asking the Scientific Research Division to identify relevant clinical department and related healthcare practitioners, health informatics specialists and hospital administrators.35 Additionally, healthcare practitioners were stratified by their professional titles, including junior, intermediate and senior. A similar two-stage process was applied to select stakeholders related to AI-assisted software vendors. CY was responsible to contact the hospitals and AI-assisted software vendors for study participation and interview arrangements.

Eligibility criteria for participant selection were as follows:

Inclusion criteria

Participants from clinical settings should either have user experience with the selected AI-assisted decision-making software or have experience in deploying or managing such software at the hospital level.

Participants from the industry should have working experience in the development of AI-assisted CDM software.

Participants should be formal staff at the stakeholder’s institution.

Participants should be at least 18 years old.

Participants should be able to sign the informed consent form voluntarily.

Exclusion criteria

Participants were excluded from the study if:

Participants could not sign the informed consent form.

Participants could not provide at least 15 min for the interview.

No sensitive information was collected during the interviews. To maintain confidentiality, all qualitative data were anonymised, and each participant who signed the informed consent voluntarily was assigned a unique identification number. Deidentified transcriptions and audio recordings were stored securely on a protected research drive with access restricted to the research team. The research data will be destroyed after 5 years of the study’s conclusion.

Theoretical framework: the CFIR

The CFIR, a well-established conceptual framework in implementation science, was originally developed in 2009 to systematically assess complex and multilevel implementation contexts for the identification of determinants impacting the successful implementation of an innovation.36 The original CFIR is an exhaustive and standardised meta-theoretical framework synthesised from 19 pre-existing implementation theories, models and frameworks, which was modified in 2022 in response to user feedback and critiques.37 The updated CFIR consists of 48 constructs and 19 subconstructs across 5 domains including innovation, outer setting, inner setting, individual and implementation process.37 It provided a fundamental structure for the exploratory evaluation of the barriers and facilitators to implementing imaging-based diagnostic AI-assisted decision-making software in China. Studies employing the CFIR in healthcare settings extensively explored various technological areas, including the implementation of electronic health record systems,38 telemedicine39 and various innovative tools, such as the frailty identification tool and decision-support systems in emergency care settings.40 41 The wide adoption of the CFIR across diverse healthcare contexts emphasised its value in capturing the complex dynamics involved in implementing technology innovations. The thorough and flexible application of the updated CFIR in data collection, analysis and reporting within the current study aimed to increase study efficiency, produce generalisable research findings to inform AI implementation practice and build a scientifically sound evidence base for tailoring implementation strategies to address key barriers.

Data collection procedures

Semistructured in-person interviews were conducted. Study-related data were collected subsequent to obtaining informed consent from the participants. Guided by the updated CFIR, different interview guides were developed for four distinct stakeholder roles, including healthcare practitioners, hospital informatics specialists, hospital administrators and vendors of AI-assisted decision-making software (online supplemental appendix 1). The interview guides were designed specifically to elicit participants’ perspectives, experiences and insights in implementing or delivering AI-assisted decision-making software (CY, XL and FJ). Prior to initiating data collection, the interview guides were pilot tested with four non-study participants to ensure clarity and reliability. Necessary modifications were made based on the feedback. The interviews were conducted by two interviewers with extensive training and experience in qualitative interviews (XL and FJ). Interview time, location, stakeholder role and basic demographic information were collected. Interviews continued until constructs of the updated CFIR were adequately represented in the data, indicating data saturation.42

Data analysis

The interviews were audio recorded, transcribed verbatim in Chinese and coded independently by two coders (XL and FJ). Deductive content analysis was primarily used for data analysis. As a systematic and objective qualitative approach, content analysis is used to describe and quantify phenomena by deriving replicable and reliable inferences from qualitative data within relevant context.43 For deductive content analysis, data were coded based on existing theories or framework defined a priori. However, the current study allowed new themes that did not fit into any of the pre-existing CFIR constructs to emerge through inductive analysis of the data.

Steps of deductive data analysis were as follows44:

Selecting the unit of analysis

Each interview was selected as unit of analysis, wherein conversational turns that contributed to the understanding of research questions were identified as meaning units. A turn consisted of an uninterrupted segment, which could be a single word or a few sentences. Following independent transcription of the audio recordings by two coders (XL and FJ), CY reviewed and compared the transcriptions, finalising the transcript to be analysed. To be immersed in the data, two coders (XL and FJ) engaged in a thorough reading of the transcripts and made relevant annotations.

Developing structured codebook

Before coding, a standardised, publicly available codebook template based on the original CFIR was employed and adapted to the study context collectively (XL, FJ and CY) (online supplemental appendix 2).45 Adaptations were multifaceted, which included aligning the original CFIR domains and constructs with those of the updated CFIR, tailoring language specific to the implementation of imaging-based diagnostic AI-assisted decision-making software in China, refining operational definitions and developing eligibility criteria for each construct.

Data coding

Two coders (XL and FJ), who were trained rigorously in using the codebook, performed coding independently. To ensure reliability, 10% of the transcripts were randomly selected for pilot testing. Two coders independently applied the codebook to generate preliminary codes for each meaning unit and subsequently categorised them within the updated CFIR framework. On completion, a group discussion with CY or JZ was warranted, which involved a comprehensive review and comparison of coding discrepancies to ensure consistency in the interpretation and categorisation of units. Disagreements were resolved through consensus, and any necessary adjustments to the operational definitions and eligibility criteria were promptly and appropriately made.

The main coding process was then structured into several iterative rounds to ensure coding consistency. In each round, individual coders were responsible to code four distinct transcripts individually and a fifth transcript collaboratively, addressing any inconsistencies through comprehensive discussion until a consensus was reached. ATLAS.ti (V.23.1.1) was used to identify, label and categorise themes and patterns within the qualitative data.46 Additionally, it facilitated data management, ensuring the storage, systematic organisation and retrieval of interview transcripts.

Reporting the data by category

Identified categories across the five domains of the updated CFIR were reported descriptively with direct quotes from participants.

Trustworthiness

The study employed several methodological strategies to ensure rigour and reliability. Multiple data sources and perspectives were incorporated to achieve triangulation, including distinct stakeholders directly involved in the implementation, deployment and development of the selected AI-assisted decision-making software. Throughout the data coding process, peer debriefing was employed. Two coders independently analysed the transcripts and collaboratively discussed the interpretations and coding decisions to reach a consensus. Moreover, an audit trail was conducted to ensure transparency, with thorough documentation of study processes such as the prespecified research protocol, deidentified transcriptions, informed consent forms, interview codebooks, typed notes, audio recordings and analyses of qualitative data. External audits were further performed to validate credibility. Experts independent of the study reviewed the study protocol, interview guides and findings, providing objective suggestions.

Patient and public involvement

There was no patient or public involvement in this study. Participants were only invited to participate in qualitative interviews.

Results

Characteristics of study setting and participants

Interviews were conducted between May and August 2023. Table 2 provides an overview of the characteristics of the selected healthcare institutions. A total of 43 participants were invited for study enrolment, and 40 (93.0%) agreed to participate, including 23 healthcare practitioners, 6 hospital informatics specialists, 4 hospital administrators and 7 vendors of the selected AI-assisted decision-making software (table 3). Non-participants included two senior healthcare practitioners and one vendor of AI-assisted decision-making software. Most participants held at least a master’s degree, and 57.5% of them were male.

Table 2 Characteristics of the selected healthcare institutions

	Cancer hospital of the Chinese Academy of Medical Sciences	Beijing Hospital	Beijing Shuili Hospital	Community healthcare centre of Qingpu district	
Location	Beijing	Beijing	Beijing	Shanghai	
Hospital type	Specialised	General	General	Clinic	
Tier	Tertiary	Tertiary	Secondary	Primary	
AI-assisted decision-making software used	Lung nodules screening	Diabetic retinopathy screening	Lung nodules screening	Diabetic retinopathy screening	
AIartificial intelligence

Table 3 Demographic characteristics of study participants

Variables	N (%)	
Stakeholder group		
 Hospital informatics specialists	6 (15)	
 Hospital administrators	4 (10)	
 Vendors of AI-assisted decision-making software	7 (17.5)	
 Healthcare practitioners		
  Junior	5 (12.5)	
  Intermediate	10 (25.0)	
  Senior	8 (20.0)	
Level of education		
 Bachelor’s degree	4 (10)	
 Master’s degree	17 (42.5)	
 Doctorate degree	19 (47.5)	
Sex		
 Male	23 (57.5)	
 Female	17 (42.5)	
Total	40 (100)	
AIartificial intelligence

Barriers and facilitators to implementing imaging-based diagnostic AI-assisted decision-making software

Among the 48 CFIR constructs and 19 subconstructs, 21 of them across 5 domains were found to be relevant in the context of implementing imaging-based diagnostic AI-assisted decision-making software in China (figure 1). Specifically, 10 were identified as barriers, 8 as facilitators and 3 as both barriers and facilitators (tables4 5).

Figure 1 Identified CFIR constructs and their impact on the implementation of imaging-based diagnostic AI-assisted decision-making software in China. ‘−’ indicated barriers; ‘+’ indicated facilitators. AI, artificial intelligence; CFIR, Consolidated Framework for Implementation Research.

Table 4 Barriers to implementing imaging-based diagnostic AI-assisted decision-making software using the updated CFIR

CFIR domain	Barrier	Short description	
I. Innovation			
 Innovation relative advantage	Unsatisfactory clinical performance	The clinical performance of the AI software needed improvement due to the complex clinical conditions in real-world settings.	
 Innovation adaptability	Lack of adaptability for generated report	The AI-generated clinical reports were inflexible and lengthy, failing to meet the hospital’s clinical documentation requirements.	
 Innovation cost	Financial burden of AI software	The cost of AI software led to financial burden on the hospital.	
II. Outer Setting			
 Partnerships and connections	Lack of a collaborative network between primary/secondary and tertiary hospitals	A referral and follow-up mechanism for patients with positive or indeterminate results from primary or secondary hospitals to tertiary hospitals was not established.	
 Policies and laws	Lack of information security measures and certification	The AI software lacked the required information protection qualifications and confidentiality measures.	
III. Inner setting			
 Relational connections	Lack of collaboration between specialised and non-specialised clinical departments	Non-specialist users had difficulties signing AI-generated reports without a collaborative network between specialised and non-specialised departments.	
 Compatibility	Suboptimal data quality	Lower real-world data quality compared with training data might negatively impact the performance of AI software.	
 Mission alignment	Misalignment between software functions and goals of healthcare institutions	Existing AI software functions did not align with the goals of comprehensive tertiary hospitals focused on specialised care and managing complex conditions.	
 Available sources: materials and equipment	Lack of necessary medical supplies	The AI software could not be implemented without essential medical supplies to generate clinical source data.	
 Access to knowledge and information	Lack of adequate training	Inadequate training on AI software made clinicians unfamiliar with its full functionalities and led to usage difficulties.	
IV. Individuals			
 Need	Unmet clinical needs	The existing AI algorithm only met general, non-personalised needs, failing to fulfil clinicians’ diverse clinical requirements.	
 Capability	Incompetence in understanding AI reasoning mechanism	Clinicians' lack of understanding of AI algorithms reduced trust and led to reluctance to use.	
V. Implementation process			
 Reflecting and evaluating: innovation	Lack of feedback incorporation	Clinicians’ suggestions were not incorporated into future updates during the implementation process.	
AIartificial intelligenceCFIRConsolidated Framework for Implementation Research

Table 5 Facilitators to implementing imaging-based diagnostic AI-assisted decision-making software using the updated CFIR

CFIR domain	Facilitator	Short description	
I. Innovation			
 Innovation relative advantage	Improved clinical efficiency	The AI software’s quick clinical judgement improved efficiency.	
 Innovation evidence base	Strong empirical evidence of effectiveness	Strong evidence showing clinical effectiveness equal to or better than human performance promoted AI software implementation.	
 Innovation trialability	AI software trialability	Pilot testing the AI software and comparing it with the standard practice fostered its implementation.	
 Innovation complexity	Easiness of use	The AI software was easy to use.	
II. Outer setting			
 Policies and laws	National guidelines related to AI	National guidelines promoting AI in healthcare fostered its implementation.	
 External pressure: market pressure	Deployment of AI software in peer hospitals	AI software utilisation in peer hospitals created peer pressure, encouraging its implementation.	
III. Inner setting			
 Compatibility	Integration of AI software into existing hospital systems	Integrating AI software into PACS ensured interoperability and reduced interruptions in routine clinical workflow.	
 Communications	Regular communication channels within department	Engaging in regular meetings to discuss AI software use within the department kept clinicians well informed.	
IV. Individuals			
 Mid-level leaders	Engagement of department head	The active engagement and promotion activities of the AI software led by the department head promoted its adoption.	
 High-level leaders	Engagement of hospital administrator	The active engagement and promotion activities of the AI software led by the hospital administrator fostered its adoption.	
V. Implementation process			
 Engaging: innovation recipients	Involvement of clinicians	During the process of implementation, clinicals were actively engaged and user feedback was collected.	
AIartificial intelligenceCFIRConsolidated Framework for Implementation ResearchPACSpicture archiving and communication system

Innovation

Innovation evidence base (+): strong empirical evidence of effectiveness

The innovation evidence base was suggested to be a key determinant facilitating implementation. Participants in this study reported evidence supporting that the clinical performance of AI software was comparable to or even surpassed that of human beings, further leading to decreased diagnostic time, reduced risk of medical errors and enhanced patient outcomes. As the AI software supported healthcare practitioners in making critical judgements regarding patient care, the robust clinical findings of efficacy and accuracy were instrumental in fostering trust and acceptance among participants towards implementation.

Before we started using the AI software in our department, I checked out articles published in some highly respected peer-reviewed journals. They reported that clinical performance of AI was comparable to human performance. This encourages me to start using the software.—intermediate clinician

Innovation relative advantage (±)

Improved clinical efficiency (+)

One of the crucial benefits gained by using AI-assisted decision-making software was the improved clinical efficacy. Some key functions of AI-assisted software included the detection of anomalies and lesions at risk, automated volumetric measurements and classification of disease severity. Study participants noted that the average interpretation time for a human reader was markedly longer than that of AI-assisted software alone or concurrent reading with the software, regardless of the level of clinical experience and complexity of diseases.

The AI software makes decisions very quickly, as compared to my decision-making time. It greatly supports my clinical judgement and improves routine clinical efficacy.—junior clinician

Unsatisfactory clinical performance (−)

However, the real-world clinical performance of the AI-assisted decision-making software remained suboptimal. Despite a strong evidence base, compromised accuracies, high false-positive rates, overestimation of lesion size and misclassification of lesion types were commonly reported by study participants. The participating healthcare practitioners highlighted the need to improve clinical performance of the AI-assisted software, particularly under complex real-world clinical conditions.

In my daily practice, I consider my clinical judgment as the gold standard. The AI software’s performance, especially in distinguishing between part-solid and solid lung nodules, doesn’t meet my expectations. The performance of the software should be improved for better usability.—intermediate clinician

Innovation adaptability (–): lack of adaptability for generated report

The generation of a diagnostic report was recognised as a pivotal function of AI-assisted decision-making software, providing diagnostic recommendations based on the analysis of patient data. The direct and automatic integration of findings into the diagnostic report was a time-saving aspect for healthcare practitioners in terms of medical documentation. However, study participants perceived that diagnostic reports generated by the AI software lacked customisation options necessary to align with the standard documentation practices of healthcare practitioners. This limitation, along with the software’s insufficient flexibility to fully comply with the hospital’s documentation standards, hindered its seamless incorporation into clinical workflow.

Personally, I don’t use the reports generated by the software. The automatically generated repots don’t align with my documentation style or the hospital’s requirement, and it doesn’t allow me to change any elements within the report. I prefer to write the reports by myself.—senior clinician

Innovation trialability (+): AI software trialability

The ability to test or experiment with AI-assisted decision-making software before full implementation was determined as a pivotal factor facilitating successful implementation. Trialability allowed participating healthcare practitioners to assess the AI software on a smaller scale, supporting their familiarity with the new innovation. More importantly, a trial period enabled evaluation of the software’s compatibility with existing workflows, identification of potential implementation barriers, and assessment of the clinical performance and reliability in real-world settings.

As we prepared for the official implementation, our department pilot tested the AI software for several weeks. This allowed me to personally experience the software, making comparisons with our standard clinical practices.—intermediate clinician

Innovation complexity (+): easiness of use

The perceived easiness of use promoted successful integration of AI-assisted decision-making software into healthcare institutions. According to the participants, the AI software featured user-friendly interfaces designed in a straightforward manner for easy navigation. In addition, the AI software generated automated, clear and comprehensible output to support medical decisions, contributing to a smooth learning curve that was conducive to quick adoption and acceptance of study participants.

The good thing is that AI software is straightforward and easy to use. I learned how to use it with minimal hassle because it provided clear and understandable output with just a few mouse clicks—intermediate clinician

Innovation cost (−): financial burden of AI software

Currently, the cost of AI-assisted decision-making software is not covered by any insurance plans. Healthcare institutions sometimes face financial constraints when acquiring the software and managing ongoing maintenance costs. Participated hospital administrators, especially those from primary and secondary hospitals, expressed the need to reallocate budgetary resources from other areas, such as staff resources and infrastructure, to accommodate the high cost associated with AI software. The perceived lack of cost-effectiveness discouraged further investment.

The insurance plans don’t cover the cost of AI software now, and patients are not paying for it either. Cost-effectiveness is one of our top priorities, and we won’t spend a lot money on the software.—hospital administrator

Outer setting

Partnerships and connections (−): lack of a collaborative network between primary/secondary and tertiary hospitals

AI-assisted decision-making software was valuable in early disease detection and intervention. However, study participants from primary care reported that the absence of partnerships and communication channels with tertiary hospitals created challenges for patients diagnosed with diseases. These challenges included delays in receiving informed referrals to tertiary hospitals, potentially resulting in late medical intervention and discontinuity in care. Further, the lack of established connections impeded the sharing and exchange of patient data between hospitals. Tertiary hospitals received incomplete or insufficient patient profiles from primary hospitals, contributing to an inadequate understanding of the patient’s condition and history. In such scenarios, patients might undergo redundant diagnostic tests at different facilities, leading to both patient inconvenience and increased healthcare costs.

For the efficient and effective utilisation of AI-assisted decision-making software in healthcare settings and optimal patient care, participants highlighted the importance of establishing a mechanism to refer and follow up with patients who have positive or indeterminate disease findings from primary hospitals to tertiary hospitals.

Patients diagnosed at our hospital with positive or indeterminate results usually need to be referred to a tertiary or specialized hospital for further treatment. However, ensuring patient compliance is a challenge. Partnering with those hospitals and establishing some referral and follow-up mechanisms will be beneficial.—intermediate clinician

Policies and laws (±)

National guidelines related to AI (+)

With the rapid advancements in AI technology, China released a series of national policies and guidelines to rigorously promote the interdisciplinary integration of AI into healthcare sector.4749 In response, clinical institutions took necessary steps forward, proactively incorporating AI-assisted decision-making software into conventional healthcare practices. To date, well-established regulatory frameworks clearly outlined and regulated the development, approval and classification of AI-assisted decision-making software as a medical device. Compliance with these regulations increased the confidence of study participants in the implementation of AI-assisted software.

We decided to bring this software in our hospital because our country is promoting the widespread adoption of AI, and it’s also the trend across different economic sectors nationwide. There are several national guidelines supporting its development and use in healthcare system, which increased our confidence in implementing the software.—hospital administrator

Lack of information security measures and certification (−)

Conversely, to ensure data security and protect patient privacy, legislation such as the cybersecurity law mandated a multilevel protection scheme. In accordance, the ‘Information Security Technology—Baseline for Classified Protection of Cybersecurity’ defined four levels of security requirements, which provided baseline guidance for securing platforms and systems handling personal information. Information systems in healthcare institutions must comply with level 3 standards, the highest for non-banking institutions, given the sensitivity of patient electronic data. Consequently, participating vendors of AI software seeking collaboration with hospitals were required to have robust information security measures and level 3 security certification as prerequisites to fulfil safety obligations. The absence of such measures and certification, not uncommon among innovative technology companies, posed barriers to successful implementation.

In order to ensure the confidentiality of patient electronic data and comply with cybersecurity protection requirements, our hospital can’t implement AI decision-making software without robust data security measures or Level 3 security certification.—hospital informatics specialist

External pressure–market pressure (+): deployment of AI software in peer hospitals

Study participants noted that the implementation of AI-assisted decision-making software in peer hospitals, such as those within the same academic affiliation, fostered a competitive atmosphere and exerted a form of peer pressure, facilitating its widespread implementation. This was especially evident in the context of China’s medical informatisation development, where hospitals without AI software implementation felt compelled to stay competitive with their peers to gain a strategic advantage.

We’ve learned that some peer hospitals have already been using such software for quite some time. We are late adopters.—hospital informatics specialist

Inner setting

Relational connections (−): lack of collaboration between specialised and non-specialised clinical departments

When patient care involved multiple clinical departments, ambiguity arose regarding the authorisation of reports generated by AI-assisted decision-making software. These reports were intended to complement the clinical decisions made by human clinicians who ultimately held the responsibility. However, non-specialists faced challenges in endorsing automated reports due to differences in clinical expertise, varying criteria for report validation, and concerns regarding liability. On the other hand, specialists often regarded AI-generated reports as less reliable than their own specialised assessments, potentially leading to reluctance in signing the reports.

Study participants emphasised the importance of establishing an interdepartmental collaborative network between specialty and non-specialty clinical departments and providing clear definitions of the roles and responsibilities within these departments to address this barrier.

As doctors without specialized expertise in ophthalmology, my colleague and I may not be authorized to sign the clinical report produced by the AI software. A collaborative network or mechanism with the department of ophthalmology will be helpful.—intermediate clinician

Communications (+): regular communication channels within department

Participants suggested that establishing regular communication channels, like weekly meetings, ensured that all members of the clinical department stayed informed about AI-assisted decision-making software. It also provided a platform for educational opportunities, such as workshops, to keep healthcare practitioners well informed and up to date. Open communication effectively addressed concerns and questions related to the implementation of AI software, fostering confidence and competence among healthcare practitioners.

I’m glad that I have the opportunity to discuss personal experience with my colleague during our weekly meetings, where case studies are shared and insights are exchanged. The open dialogue enhances our knowledge and improves my proficiency and confidence.—junior clinician

Compatibility (±)

Suboptimal data quality (−)

The effective performance of AI-assisted decision-making software relied on the availability of high-quality and accurate source data. In the process of software development, machine learning and deep learning algorithms used to analyse and interpret imaging data were trained with dataset that underwent meticulous cleaning and curation, ensuring the removal of poor-quality data containing imaging noise and artefacts before analysis. However, in real-world clinical settings, various factors, like equipment limitations, patient motion and varying proficiency of technicians, potentially introduced imperfections in imaging data. As a result, participants pointed out that AI-assisted software was not highly compatible with and adequately trained on data collected during routine clinical practice.

The real-world imaging quality is often less than optimal, which can lead to inaccuracy or failure of AI diagnosis. We can’t always ask the patient to redo the examination for better quality data, in consideration of their time and healthcare cost.—senior clinician

Integration of AI software into existing hospital systems (+)

In contrast, the integration of AI-assisted software into established hospital systems, such as the picture archiving and communication system (PACS), streamlined clinical workflows and facilitated the effective implementation. The compatibility with PACS enabled interoperability between AI-assisted software and healthcare information systems, providing healthcare practitioners with a familiar working environment and mitigating interruptions in workflow.

Our AI software integrates with the PACS. Clinicians don’t have to learn a new standalone system; instead, they can access AI-generated insights directly within their existing PACS environment, minimizing any disruptions to their workflow.—vendor of AI-assisted software

Mission alignment (−): misalignment between software functions and goals of healthcare institution

A misalignment between functions of AI-assisted decision-making software and the core hospital missions, especially for comprehensive tertiary hospitals, was revealed by study participants. Currently, diagnostic AI-assisted software predominantly supports the diagnosis of general and non-complicated diseases, which divergess from the main strategic objectives of tertiary hospitals dedicated to managing complex medical conditions and delivering high-level care through specialised expertise. Alternatively, AI software appeared more suitable for primary hospitals, where it could be used for general disease diagnosis and population-level screening. Tertiary hospitals prioritised other initiatives perceived to be more critical to their mission. This prioritisation further contributed to the reluctance among healthcare practitioners to embrace AI-assisted software, as they identified the introduction of AI software as a distraction from the hospital’s core mission.

At times, it’s difficult for us to establish collaborations with high-level tertiary hospitals. These hospitals often have highly experienced clinicians, focusing on the improvement of care quality for complex diseases and rare conditions. They have the perception that our AI software may not perform well in their setting. Instead, they suggest that our software may be better suited for primary hospitals where initial diagnoses take place.—vendor of AI-assisted decision-making software

Available resources–materials and equipment (–): lack of necessary medical supplies

The availability of essential medical supplies was integral to the successful implementation of AI-assisted decision-making software that relied on medical imaging data as the primary data source for accurate assessment. In primary and secondary hospitals, where resources were relatively scarce, the limited access to equipment, like CT scanners, hindered the implementation of AI-assisted software.

The implementation of AI decision-making software is not possible at hospitals without necessary medical supplies like CT scanners. —hospital informatics specialist

Access to knowledge and information (–): lack of adequate training

For advanced technologies like AI-assisted CDM software, participated healthcare practitioners sometimes lacked the necessary knowledge and information required for effective use. Inadequate training possibly contributed to a reluctance to adopt the technology, due to unfamiliarity with the software’s complete functionalities and challenges in its practical application.

I believe that I haven’t received thorough training on using the AI software. In fact, I’ve explored it on my own, and I’m not completely aware of all its functions.—intermediate clinician

Individuals

High-level and mid-level leaders (+)

Engagement of hospital administrator (+)

Participants in the study indicated that effective implementation of AI-assisted decision-making software was facilitated by the hospital’s active leadership engagement and promotional initiatives from the hospital to the department level. Hospital administrators took proactive steps to align the AI-assisted software with the institution’s long-term strategic goals through the initiation and oversight of pilot programmes. The endorsement and active support at the hospital level greatly fostered a collaborative environment among the clinical department, information technology department and vendor of AI-assisted decision-support software, positioning the AI-assisted software as an integral component of the hospital.

Strong support from the top level, especially from our hospital administrators, really makes a difference in introducing AI software and running it smoothly. They ensure its fit with the hospital through a pilot program and rigorously and effectively promote multi-stakeholder communication and collaboration.—hospital informatics specialist

Engagement of department head (+)

At the departmental level, leaders, such as department heads, who supported AI-assisted software, actively championed its implementation. They cultivated an atmosphere of support and knowledge-sharing within the department through the organisation of workshops and seminars, stressing the prospective clinical benefits of implementation. Beyond intradepartmental communication, they facilitated efficient interdepartmental communication with the information technology department to ensure seamless integration of AI-assisted decision-making software into the real-world clinical setting.

Our department head actively supports the implementation of AI software by integrating discussions about relevant knowledge and experiences into our weekly meetings, shedding light on the potential clinical benefits. In fact, they play a very important role in facilitating the integration of AI software into the existing PACS, making the entire implementation process much more efficient and effective.—junior clinician

Need (–): unmet clinical needs

Study participants revealed that AI-assisted decision-making software failed to meet the diverse clinical needs of healthcare practitioners. Currently, the underlying AI algorithm was predominantly designed and trained to address general and non-personalised clinical needs. Clinicians perceived AI-assisted software as insufficient in cases that were complex and multifaceted, requiring a comprehensive approach and an in-depth understanding of the patient’s medical history. Incorporating customisation options, enhancing adaptability in AI algorithms and demonstrating a commitment to ongoing improvement were essential to ensure that AI-assisted decision-making software aligned with the disparate needs of healthcare practitioners across various specialties and clinical settings.

The AI software we have is good for the basics, but we definitely expect more. Currently, its functions are too simplified, and it struggles in tricky and complex situations where you need a deep dive into the patient’s history.—senior clinician

Capability (–): incompetence in understanding AI reasoning mechanism

Participated healthcare practitioners faced challenges in implementing AI-assisted decision-making software in clinical practice due to a limited capability in understanding AI algorithms. The deficiency in necessary knowledge and expertise led to difficulties in comprehending the rationale behind the AI’s recommendations and decisions. This lack of clarity contributed to a lack of trust and reluctance towards the implementation of AI-assisted software. Participants suggested that addressing case-specific reasoning and providing global transparency, such as the algorithm’s functionality, strengths and limitations, would be helpful in opening the ‘black box’ of AI technology.

I sometimes find it hard to trust and embrace the software’s recommendations. I struggle with the complexity of the underlying rationale, since the software provides recommendations based on these algorithms. It’s not clear to me what’s inside the black box, like how it works, what its weakness and strengths are, etc. Clarifications on those factors would be helpful.—senior clinician

Implementation process

Engaging–innovation recipient (+): involvement of clinicians

It was reported that active engagement substantially facilitated the implementation of AI-assisted decision-making software, particularly through the active involvement of key stakeholders during the process of implementation. In specific, the pilot testing phase was conducted to collect valuable insights and suggestions provided by users, determining limitations and identifying areas for improvement that closely aligned with their clinical needs and workflow. The healthcare practitioners, on the other hand, were empowered by actively shaping the software’s functionality and streamlining the process of implementation.

During the pilot testing phase, we collaborated with the entire department to answer any questions and gather suggestions. This active engagement of the clinicians was helpful not only for us to continuously improve the software, but also for the clinicians to feel involved and make an impact; it’s a win-win situation.—vendor of AI-assisted decision-making software

Reflecting and evaluating (–): lack of feedback incorporation

Reflecting and evaluating were central components of the continuous feedback-driven improvements that promoted the seamless integration of AI-assisted decision-making software into clinical settings. However, study participants noted that their suggestions and qualitative feedback, shared during the pilot testing phase, were not adequately reflected and implemented for process enhancement. Furthermore, there was a notable absence of quantitative assessment of the clinical performance of the AI-assisted software following its implementation. The absence of informative reflection on provided feedback and a structured evaluation process contributed to unaddressed challenges and the frustration of participating healthcare practitioners who felt that their inputs were not sufficiently valued.

My colleague and I provided feedback and suggestions about this AI software during the pilot testing phase, but we see no corresponding actions taken by the vendors, which is disappointing.—intermediate clinician

I don’t think there is any systematic evaluation mechanism related to the clinical performance of AI software at our hospital. It is, however, important to periodically and systematically evaluate the performance of the software to make it more accurate and usable.—hospital informatics specialist

Comparison of stakeholder perspectives

It should be noted that the perceptions regarding the selected AI-assisted decision-making software varied considerably among different stakeholder roles. Recognising these unique perspectives is essential to the development of effective implementation strategies that address the varied concerns and priorities of each stakeholder group.

Clinicians, as the primary users, juxtaposed the potential benefits and limitations of the software. Junior clinicians, who have limited clinical experiences, generally held positive attitudes towards the implementation, highlighting the software’s ability to support clinical judgement and enhance routine clinical efficacy. While recognising the value of AI implementation, intermediate clinicians, who used the software more insightfully, gained practical perspectives and emphasised the need for strong interdepartmental collaboration, adequate training and referral mechanisms to tertiary or specialised hospitals for patients with positive or indeterminate disease findings. Senior clinicians provided the most critical feedback, expecting higher standard and improved performance in clinical effectiveness, reliability and transparency, particularly in complex clinical scenarios. On the other hand, hospital administrators focused on financial implications, like cost-effectiveness and budgetary constraints, and informatics specialists highlighted the importance of robust information security measures. Moreover, the selected vendors underscored the necessity of aligning the AI functions with the mission of the healthcare institution to ensure successful implementation.

Discussion

To the best of our knowledge, this study was the first qualitative assessment that leveraged a well-established implementation framework to systematically guide the identification of barriers and facilitators of AI-assisted decision-making software in China’s healthcare system. The implementation of AI-assisted decision-making software in clinical practice is characterised by the inherent complexity and dynamic nature of both AI technology and healthcare environment. Previous literature attempted to synthesise and understand relevant determinants, with minimal application of theories or frameworks in implementation science, particularly in developing countries.30 50 The use of the updated CFIR played a fundamental role in understanding the context of implementation and establishing a strategic roadmap, consistently and efficiently producing collective and generalisable knowledge for the development of context-specific implementation strategies tailored to China’s healthcare system.51 The dynamic and continuous interaction among the five domains of the updated CFIR collectively shaped the outcome and effectiveness of imaging-based diagnostic AI-assisted software implementation.36 52 The current study validated several barriers identified in prior research across diverse clinical settings, including suboptimal clinical performance,5355 compromised RWD quality,5658 insufficient training,54 59 60 deficit in transparency and trust,6062 financial constraint,54 57 insufficiency of necessary equipment,59 60 and limited interdepartmental communication.54 More importantly, study findings contributed novel insights to the continuous exploration of the implementation of imaging-based diagnostic AI-assisted decision-making software from the unique perspective of China’s healthcare system, establishing a theoretical foundation to guide the development of practical recommendations and implementation strategies for future improvement efforts.

Given the different perspectives of various stakeholder roles, the prioritisation of barriers, as well as the feasibility and cost-effectiveness of recommendations, the following three barriers and their corresponding suggestions were discussed in further detail (figure 2).

Figure 2 Barriers and suggestions for implementing imaging-based diagnostic AI-assisted decision-making software in China. AI, artificial intelligence.

Barrier: misalignment between software functions and goals of healthcare institutions.

Suggestion: shift the focus of imaging-based diagnostic AI-assisted decision-making software implementation towards primary and secondary healthcare settings, where the AI software’s strengths in diagnosing generalised and non-complex conditions can be leveraged effectively.

The AI-assisted decision-making software has been disproportionately implemented in tertiary hospitals in China.29 However, a notable misalignment between the functionality of imaging-based diagnostic AI-assisted decision-making software and the strategic goals of tertiary hospitals was found in the current study. Specifically, the implementation of AI-assisted decision-making software demonstrated its effectiveness in diagnosing generalised and non-complex medical conditions. Tertiary hospitals, in contrast, mainly served as hubs that provide specialised and advanced healthcare services, particularly for complex medical conditions. Despite the great potential of AI technologies to revolutionise healthcare, it has become evident that the complexity of conditions, frequently encountered by high-level healthcare institutions, has not been adequately addressed by existing AI competency.55 60 Given the pivotal role that tertiary hospitals play in China’s healthcare system, it is necessary for imaging-based diagnostic AI-assisted decision-making software to further advance to meet the multifaceted clinical needs of tertiary hospitals in the near future. On the other hand, to effectively promote the implementation of existing imaging-based diagnostic AI-assisted software, a shift in the focus of implementation towards the primary or secondary level of healthcare, such as primary hospitals, physical examination centres or secondary hospitals, would offer a more cohesive fit. This shift would create a more suitable context to effectively implement imaging-based diagnostic AI-assisted decision-making software, leveraging its strengths while accommodating the unique challenges faced by healthcare institutes at the primary or secondary level. Primary healthcare in China typically addresses a broader spectrum of clinical needs and medical cases. However, it is often not the initial point of medical contact due to the suboptimal quality of care.63 With substantial disparities between primary and tertiary care, residents in China perceived primary healthcare as of poor quality, as reflected in a low doctor-to-patient ratio in tertiary care.64 65 Various contributing factors were reported, including insufficient knowledge among healthcare professionals, a gap between knowledge and practice, disproportionate distribution of health workforce and inadequate continuity of care across the entire healthcare system.63 66 Implementing imaging-based diagnostic AI-assisted decision-making software at the primary level, aligning its functionality with the overarching goals of primary care, holds promise in addressing these challenges and bridging gaps, thereby potentially diverting patients with common medical needs towards primary healthcare facilities. AI technologies have the potential to facilitate a diagnosis at least equivalent to that of an intermediate-level clinician, complement clinical expertise and optimise medical resource allocation, enhancing early disease detection and ultimately promoting the quality of patient care across the healthcare hierarchy in China.6769

Barrier: lack of a collaborative network between primary/secondary and tertiary hospitals.

Suggestion: establish an integrated healthcare ecosystem driven by a hub-and-spoke model to promote the sharing of clinical data and improve patient referrals, ensuring seamless coordination between primary/secondary and tertiary healthcare institutions.

As mentioned above, one of the key challenges in China’s healthcare system lied in the fragmentation of healthcare delivery. The integration of imaging-based diagnostic AI-assisted decision-making software into primary care stressed the absence of a comprehensive collaborative network connecting primary and tertiary healthcare institutions. This deficiency exacerbated inefficiencies in patient referrals, with positive or indeterminate AI-assisted diagnoses at primary hospitals not being effectively referred to tertiary hospitals. As a result, patients could experience delays in receiving specialised treatment. Furthermore, the scattered and isolated electronic medical systems in China posed substantial challenges to joint healthcare initiatives.63 Sharing and transfer of clinical data related to disease diagnosis were hindered due to the heterogeneity of systems, potentially leading to unnecessary and duplicated medical examinations. To address this issue, establishing an integrated healthcare ecosystem driven by the hub-and-spoke model would be a promising solution to promote the sharing of clinical data and medical knowledge, as well as facilitate best medical practices. This model, when applied in healthcare system, enhances peripheral services by connecting them with resource-replete centres.70 In this context, basic medical needs are met through spokes, like primary healthcare institutions while medical resources and investments are centralised at hubs, such as tertiary healthcare institutions. The utilisation of imaging-based diagnostic AI-assisted decision-making software in a hub-and-spoke network of stroke care showed improved clinical efficiency, including decreased time to notification and transfer time between spokes to hubs, leading to a shorter length of stay.71 72 Currently, a similar network tailored to the healthcare system in China has yet to be implemented, and its potential clinical benefits remain unclear. To fully leverage the capabilities of imaging-based diagnostic AI-assisted decision-making software, it is essential to seamlessly refer patients with positive or indeterminate diagnoses at spoke sites to the hub sites for specialised care, minimising delays in early treatment.

Barrier: lack of information security measures and certification.

Suggestion: establish an independent information platform with robust data security measures to ensure the protection of clinical data privacy and facilitate the integration and data exchange across primary/secondary and tertiary healthcare institutions.

The cybersecurity and data protection regulations in China are undergoing rapid changes, positioning it as one of the most stringent globally. The ‘Information security technology—baseline for classified protection of cybersecurity (GB/T 22239-2019)’, jointly issued by the State Market Regulatory Administration and Standardization Administration of China, came into effect in December 2019.73 The standard defined level 1 to level 4 security requirements and specified the baseline guidance for information security technology to protect information platforms and systems responsible for collecting, storing, transmitting and processing personal information.73 As China’s healthcare informatisation continues to advance, the security of hospital information infrastructures is becoming increasingly critical, given that any disruptions could have substantial consequences for individuals and society as a whole. To address this concern, the National Health Commission released the ‘Guidelines for Information Security Level Protection in the Healthcare Industry’, stipulating that core systems in healthcare institutions should adhere to level 3 information security protection standards, the highest level for non-banking institutions.74 Level 3 security protection mainly covers 5 aspects of security technical requirements and 5 aspects of security management requirements. The multidimensional assessment involves 73 categories, with nearly 300 specific requirements, covering aspects such as information protection, security auditing, communication confidentiality, etc.73 While AI technology software may not be explicitly covered by this specific requirement, it is often a mandatory administrative step for healthcare institutions, particularly tertiary hospitals, to request a level 3 security certification to ensure the protection of clinical data privacy during the integration and data exchange of AI software. As an integrated approach addressing the unique challenges faced by China’s healthcare system, an independent information platform with robust data security measures or level 3 security qualification to facilitate the implementation of imaging-based diagnostic AI-assisted decision-making software should be established. This platform acts as a vital link connecting the AI software, primary and tertiary healthcare institutions. It is designed to collect basic demographic information and medical history while transmitting deidentified imaging data to AI-assisted software for an initial diagnosis in primary care settings. In cases where patients receive positive or indeterminate reports, referral to the collaborated tertiary hospital within the hub-and-spoke network is warranted. Relevant clinical information, including collected demographic data, medical history, clinical reports and referral forms, is seamlessly transferred to enhance overall efficiency. The successful establishment of this platform requires multistakeholder engagement for proficient and collective design and management, addressing interinstitutional data sharing, security and governance challenges stemming from legal, technical and operational requirements.70 75

Implications for policy-makers and healthcare practitioners

Globally, there are marked differences in the implementation of AI in healthcare systems. These variations are associated with factors including the type of AI software, healthcare infrastructure, existing policies and technological advancements. Despite ongoing criticism and multiple implementation challenges, AI-assisted decision-making software and health information technologies have demonstrated substantial potential for enhancing diagnostic procedures in primary care, especially with strong regulatory support, in both resource-rich and under-resourced settings.7679 Primary care is an ideal setting for AI tools to improve clinical efficiency and reduce medical errors due to its role in managing a large number of patients and making decisions under uncertainty.76

In Germany, the Federal Ministry of Health has been proactive in supporting AI integration in healthcare. The ‘Smart Physician Portal for Patients with Unclear Disease’ project provided ongoing support to general practitioners (GPs) using an AI-based tool to diagnose uncertain cases.80 This user-centred decision support tool was design specifically to address GP’s essential clinical needs through interviews and workshops, ensuring a seamless fit into the routine workflow while allowing for more efficient patient diagnosis. Similarly, the National Health Service in the UK has actively incorporated AI technologies into primary care through the use of Babylon’s Triage and Diagnostic system streamlining the diagnostic process.81 Furthermore, the European Union’s 2021 Proposal sought to establish a global standard for safe, reliable and ethical AI by creating a comprehensive legal framework designed to enhance trust and encourage broad implementation, ensuring that the AI systems are both technically and ethically sound.82 83 Therefore, regulatory support to increase trust, robust technical infrastructure, strong ethical standards and user-centred design are crucial for the extensive integration of AI in healthcare, ultimately improving patient outcomes and clinical efficiency.

In contrast, China faces particular difficulties due to its large and heterogeneous healthcare system, regional disparities in healthcare infrastructure and rapidly evolving regulatory environment. The successful integration of AI is hindered by the disparity in healthcare resources and the critical need for interoperability among various hospital information systems, especially in primary care settings. In particular, primary healthcare facilities in rural areas, constrained by financial and resource limitations, often lack access to the advanced AI technologies that are more steadily available in tertiary hospitals in big cities.84 More importantly, while electronic medical record (EMR) systems are widely adopted in primary hospitals, their average level and functionality are typically lower as than those in tertiary hospitals.85 According to the National Health Commission of the People’s Republic of China, as of 2022, the average level of EMR systems in the tertiary public hospitals was 4.0, indicating a medium stage of EMR development that enabled basic clinical decision support.86 However, to fully facilitate intelligent clinical decision support, EMR systems need to reach at least level 5, posing an even greater challenge to the systematic integration of AI.87

Financial incentives and policy support for EMR infrastructure facilitating the use of AI in primary healthcare settings could drive broader implementation and improve care quality. Guidelines for the thorough assessment of AI-assisted decision-making software for cost-effectiveness, efficacy and safety are also urgently needed.88 To improve care coordination across different levels of healthcare institutions, policies should also support collaborative networks and data-sharing platforms. To increase healthcare practitioners’ familiarity with and confidence in AI-assisted decision-making software, major implementation barriers must be addressed and overall trust in AI technologies must be increased through thorough training and continued regulatory support.

Strengths and limitations

The current study has several strengths and limitations. To ensure scientific rigour and validity, the updated CFIR was thoroughly employed to guide the design of interview guides through the reporting of results. Although primarily descriptive, the study extended beyond identifying barriers and facilitators by providing practical suggestions tailored to China’s healthcare system. In order to capture a broad spectrum of perspectives, a wide range of key stakeholders, ranging from healthcare practitioners to industry vendors, were involved, allowing for a qualitative exploration of various roles and the provision of comprehensive insights. However, the inclusion of perspectives from patients should be warranted in future research, particularly through doctor–patient shared decision-making. Given the extensive impact of AI technology on the professional autonomy of healthcare practitioners, existing literature suggested a negative perception among patients towards physicians using AI-assisted software.89 90 While the current study specifically focused on two representative AI applications, namely imaging-based diagnostic AI-assisted decision-making software for lung nodules and diabetic retinopathy, study findings were further generalised to the general diagnostic AI-assisted decision-making software using medical imaging as source data. Despite employing the updated CFIR as a systematic approach to understanding barriers and facilitators in the implementation process for enhanced generalisability, it is important to acknowledge potential variations across different types of diagnostic AI software. The current study might not fully capture certain software-specific differences and contextual factors associated with implementation. Moreover, purposive sampling was adopted, and all selected healthcare institutions were located in well-resourced areas, potentially leading to limited generalisability of findings beyond the selected healthcare institutions and software. The results should be interpreted considering this context, emphasising the strong need for cross-comparisons of the findings and the validation of recommendations in other settings, particularly in rural areas.

Conclusion

The rapid advancement of AI techniques is fuelling a global shift in the conventional medical decision-making paradigm. By using the updated CFIR, the current study contributed to a comprehensive understanding of the barriers and facilitators in implementing imaging-based diagnostic AI-assisted decision-making software in China’s evolving healthcare landscape. The findings served as a solid theoretical foundation, providing a possible roadmap for future efforts aimed at optimising the effective implementation of imaging-based diagnostic AI-assisted decision-making software. The tangible suggestions could further inform healthcare practitioners, industry stakeholders and policy-makers in both China and other developing countries, facilitating the unleashing of the full potential of imaging-based diagnostic AI-assisted decision-making software for optimal patient care.

supplementary material

10.1136/bmjopen-2024-084398 online supplemental file 1

10.1136/bmjopen-2024-084398 online supplemental file 2

10.1136/bmjopen-2024-084398 online supplemental file 3

Acknowledgements

The authors thank all individuals who took the time to participate in the interviews and those who provided constructive suggestions on the manuscript.

Data availability statement

Data are available on reasonable request.

Review Process File
10 09 2024

Funding: This work was supported by Merck Sharp & Dohme, a subsidiary of Merck & Co., Rahway, New Jersey, USA. The sponsor participated in the design and development of the study, as well as the revision and editing of this manuscript.

Prepublication history and additional supplemental material for this paper are available online. To view these files, please visit the journal online (https://doi.org/10.1136/bmjopen-2024-084398).

Provenance and peer review: Not commissioned; externally peer reviewed.

Patient consent for publication: Not applicable.

Ethics approval: This study involves human participants and was approved by the Institutional Review Board of Peking University (IRB00001052-22138). Participants gave informed consent to participate in the study before taking part.

Data availability free text: Study protocol and interview transcripts are available on request by contacting the corresponding author.

Patient and public involvement: Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
==== Refs
References

1 Berman S Clinical decision making Bajaj L Hambidge SJ Kerby G et al Berman’s pediatric decision making 5th edn Saint Louis Mosby 2011 1 6
2 Bijani M Abedi S Karimi S et al Major challenges and barriers in clinical decision-making as perceived by emergency medical services personnel: a qualitative content analysis BMC Emerg Med 2021 21 11 10.1186/s12873-021-00408-4 33468045
3 Smith M Higgs J Ellis E Factors influencing clinical decision making. Clinical reasoning in the health professions 3rd ed. / 8 (section 2) Sydney Butterworth-Heinemann 2008 89 100
4 Sim I Gorman P Greenes RA et al Clinical decision support systems for the practice of evidence-based medicine J Am Med Inform Assoc 2001 8 527 34 10.1136/jamia.2001.0080527 11687560
5 Winters-Miner LA Bolding PS Hilbe JM et al Chapter 3 - biomedical informatics Winters-Miner LA Bolding PS Hilbe JM et al Practical predictive analytics and decisioning systems for medicine Academic Press 2015 42 59
6 Porat T Delaney B Kostopoulou O The impact of a diagnostic decision support system on the consultation: perceptions of GPs and patients BMC Med Inform Decis Mak 2017 17 79 10.1186/s12911-017-0477-6 28576145
7 Meunier PY Raynaud C Guimaraes E et al Barriers and Facilitators to the Use of Clinical Decision Support Systems in Primary Care: A Mixed-Methods Systematic Review Ann Fam Med 2023 21 57 69 10.1370/afm.2908 36690490
8 Sutton RT Pincock D Baumgart DC et al An overview of clinical decision support systems: benefits, risks, and strategies for success NPJ Digit Med 2020 3 17 10.1038/s41746-020-0221-y 32047862
9 Chen H Li L Chen Y Explore success factors that impact artificial intelligence adoption on telecom industry in China J Manag Anal 2021 8 36 68 10.1080/23270012.2020.1852895
10 Administration USFaD Artificial intelligence and machine learning (AI/ML) for drug development 2023 Available https://www.fda.gov/science-research/science-and-research-special-topics/artificial-intelligence-and-machine-learning-aiml-drug-development
11 Chen Z Liu X Hogan W et al Applications of artificial intelligence in drug development using real-world data Drug Discov Today 2021 26 1256 64 10.1016/j.drudis.2020.12.013 33358699
12 Center for Medical Device Evaluation N “AI+Healthcare” – a new driving force for the growth of the medical device industry 2023 Available https://www.cmde.org.cn//xwdt/zxyw/20230407085114162.html
13 Technology CAoIaC White paper on the development of the artificial intelligence medical device industry 2023
14 Center for Medical Device Evaluation N Announcement on the key points for the evaluation of medical device software assisted in deep learning and decision making (no.7 of 2019) 2019 Available https://www.cmde.org.cn//xwdt/shpgzgg/gztg/20190703141714991.html
15 Center for Medical Device Evaluation N Notice of the center for medical device evaluation of NMPA on issuing the guiding principles for the registration and review of artificial intelligence medical devices 2022 Available https://www.cmde.org.cn//xwdt/shpgzgg/gztg/20220309090800158.html
16 Forum IMDR Software as a medical device (SaMD): key definitions 2013 Available https://www.imdrf.org/sites/default/files/docs/imdrf/final/technical/imdrf-tech-131209-samd-key-definitions-140901.pdf
17 Administration USFaD FDA permits marketing of artificial intelligence-based device to detect certain diabetes-related eye problems 2018 Available https://www.fda.gov/news-events/press-announcements/fda-permits-marketing-artificial-intelligence-based-device-detect-certain-diabetes-related-eye
18 Administration NMP The software product for calculating coronary flow reserve scores has been approved for market launch 2020 Available https://www.nmpa.gov.cn/yaowen/ypjgyw/ylqxyw/20200115153901893.html
19 Liao X Yao C Zhang J et al Recent advancement in integrating artificial intelligence and information technology with real-world data for clinical decision-making in China: A scoping review J Evid Based Med 2023 16 534 46 10.1111/jebm.12549 37772921
20 Xu L He X Zhou J et al Artificial intelligence-assisted colonoscopy: A prospective, multicenter, randomized controlled trial of polyp detection Cancer Med 2021 10 7184 93 10.1002/cam4.4261 34477306
21 Nam JG Hwang EJ Kim J et al AI Improves Nodule Detection on Chest Radiographs in a Health Screening Population: A Randomized Controlled Trial Radiology 2023 307 e221894 10.1148/radiol.221894 36749213
22 Wang P Liu X Berzin TM et al Effect of a deep-learning computer-aided detection system on adenoma detection during colonoscopy (CADe-DB trial): a double-blind randomised study Lancet Gastroenterol Hepatol 2020 5 343 51 10.1016/S2468-1253(19)30411-X 31981517
23 Yao X Rushlow DR Inselman JW et al Artificial intelligence-enabled electrocardiograms for identification of patients with low ejection fraction: a pragmatic, randomized clinical trial Nat Med 2021 27 815 9 10.1038/s41591-021-01335-4 33958795
24 Lin H Li R Liu Z et al Diagnostic Efficacy and Therapeutic Decision-making Capacity of an Artificial Intelligence Platform for Childhood Cataracts in Eye Clinics: A Multicentre Randomized Controlled Trial EClinMed 2019 9 52 9 10.1016/j.eclinm.2019.03.001
25 Shimabukuro DW Barton CW Feldman MD et al Effect of a machine learning-based severe sepsis prediction algorithm on patient survival and hospital length of stay: a randomised clinical trial BMJ Open Respir Res 2017 4 e000234 10.1136/bmjresp-2017-000234
26 Lin C Liu W-T Chang C-H et al Artificial intelligence–powered rapid identification of ST-elevation myocardial infarction via electrocardiogram (ARISE) — a pragmatic randomized controlled trial NEJM AI 2024 1 AIoa2400190 10.1056/AIoa2400190
27 Wolf RM Channa R Liu TYA et al Autonomous artificial intelligence increases screening and follow-up for diabetic retinopathy in youth: the ACCESS randomized control trial Nat Commun 2024 15 421 10.1038/s41467-023-44676-z 38212308
28 Mathenge W Whitestone N Nkurikiye J et al Impact of Artificial Intelligence Assessment of Diabetic Retinopathy on Referral Service Uptake in a Low-Resource Setting: The RAIDERS Randomized Trial Ophthalmol Sci 2022 2 100168 10.1016/j.xops.2022.100168 36531575
29 Ji M Chen X Genchev GZ et al Status of AI-Enabled Clinical Decision Support Systems Implementations in China Methods Inf Med 2021 60 123 32 10.1055/s-0041-1736461 34695871
30 Chomutare T Tejedor M Svenning TO et al Artificial Intelligence Implementation in Healthcare: A Theory-Based Scoping Review of Barriers and Facilitators Int J Environ Res Public Health 2022 19 16359 10.3390/ijerph192316359 36498432
31 Ross J Stevenson F Lau R et al Factors that influence the implementation of e-health: a systematic review of systematic reviews (an update) Implement Sci 2016 11 146 10.1186/s13012-016-0510-7 27782832
32 Richardson JE Abramson EL Pfoh ER et al Bridging informatics and implementation science: evaluating a framework to assess electronic health record implementations in community settings AMIA Annu Symp Proc 2012 2012 770 8 23304351
33 Center for Medical Device Evaluation N Notice of the center for medical device evaluation of NMPA on the issuance of guiding principles for the registration and review of CT image assisted detection software for pulmonary nodules (no.21 of 2022) 2022 Available https://www.cmde.org.cn//xwdt/shpgzgg/gztg/20220526102356105.html
34 Center for Medical Device Evaluation N Notice of the center for medical device evaluation of NMPA on issuing the guidelines for registration and examination of fundus image assisted diagnosis software for diabetes retinopathy (no.23 in 2022) 2022 Available https://www.cmde.org.cn//xwdt/shpgzgg/gztg/20220602103048151.html
35 Parker C Scott S Geddes A Snowball sampling Atkinson P Delamont S Cernat A et al SAGE research methods foundations London SAGE Publications Ltd 2019
36 Damschroder LJ Aron DC Keith RE et al Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science Impl Sci 2009 4 50 10.1186/1748-5908-4-50
37 Damschroder LJ Reardon CM Widerquist MAO et al The updated Consolidated Framework for Implementation Research based on user feedback Impl Sci 2022 17 75 10.1186/s13012-022-01245-0
38 Acharya S Werts N Toward the Design of an Engagement Tool for Effective Electronic Health Record Adoption Perspect Health Inf Manag 2019 16 1g
39 Van Citters AD Dieni O Scalia P et al Barriers and facilitators to implementing telehealth services during the COVID-19 pandemic: A qualitative analysis of interviews with cystic fibrosis care team members J Cyst Fibros 2021 20 Suppl 3 23 8 10.1016/j.jcf.2021.09.004 34930537
40 Fujimori R Liu K Soeno S et al Acceptance, Barriers, and Facilitators to Implementing Artificial Intelligence-Based Decision Support Systems in Emergency Departments: Quantitative and Qualitative Evaluation JMIR Form Res 2022 6 e36501 10.2196/36501 35699995
41 Warner G Lawson B Sampalli T et al Applying the consolidated framework for implementation research to identify barriers affecting implementation of an online frailty tool into primary health care: a qualitative study BMC Health Serv Res 2018 18 395 10.1186/s12913-018-3163-1 29855306
42 Saunders B Sim J Kingstone T et al Saturation in qualitative research: exploring its conceptualization and operationalization Qual Quant 2018 52 1893 907 10.1007/s11135-017-0574-8 29937585
43 Krippendorff K Content analysis: an introduction to its methodology Thousand Oaks, California 2019 Available https://methods.sagepub.com/book/content-analysis-4e
44 Elo S Kyngäs H The qualitative content analysis process J Adv Nurs 2008 62 107 15 10.1111/j.1365-2648.2007.04569.x 18352969
45 Research TCFfI Tools and templates 2023 Available https://cfirguide.org/tools/tools-and-templates/
46 GmbH AtSSD ATLAS.Ti Mac.V 23.2.1 ATLAS.ti Scientific Software Development GmbH 2023
47 Council GOotS Development plan on new generation of artificial intelligence 2017 Available http://www.gov.cn/zhengce/content/2017-07/20/content_5211996.htm
48 Council GOotS Notice of the state council on the issuance of the “13th five-year plan” for national scientific and technological innovation 2016 Available http://www.gov.cn/zhengce/content/2016-08/08/content_5098072.htm
49 Council GOotS Opinions of the general office of the state council on promoting the development of ’Internet plus medical and health 2018 Available http://www.gov.cn/zhengce/content/2018-04/28/content_5286645.htm
50 Wang L Zhang Z Wang D et al Human-centered design and evaluation of AI-empowered clinical decision support systems: a systematic review Front Comput Sci 2023 5 10.3389/fcomp.2023.1187299
51 Damschroder LJ Hagedorn HJ A guiding framework and approach for implementation research in substance use disorders treatment Psychol Addict Behav 2011 25 194 205 10.1037/a0022284 21443291
52 Kirk MA Kelley C Yankey N et al A systematic review of the use of the Consolidated Framework for Implementation Research Implement Sci 2016 11 72 10.1186/s13012-016-0437-z 27189233
53 Schwartz JM George M Rossetti SC et al Factors Influencing Clinician Trust in Predictive Clinical Decision Support Systems for In-Hospital Deterioration: Qualitative Descriptive Study JMIR Hum Factors 2022 9 e33960 10.2196/33960 35550304
54 Strohm L Hehakaya C Ranschaert ER et al Implementation of artificial intelligence (AI) applications in radiology: hindering and facilitating factors Eur Radiol 2020 30 5525 32 10.1007/s00330-020-06946-y 32458173
55 Romero-Brufau S Wyatt KD Boyum P et al A lesson in implementation: A pre-post study of providers’ experience with artificial intelligence-based clinical decision support Int J Med Inform 2020 137 104072 10.1016/j.ijmedinf.2019.104072 32200295
56 Singh RP Hom GL Abramoff MD et al Current Challenges and Barriers to Real-World Artificial Intelligence Adoption for the Healthcare System, Provider, and the Patient Transl Vis Sci Technol 2020 9 45 10.1167/tvst.9.2.45
57 Schuh C de Bruin JS Seeling W Clinical decision support systems at the Vienna General Hospital using Arden Syntax: Design, implementation, and integration Artif Intell Med 2018 92 24 33 10.1016/j.artmed.2015.11.002 26706047
58 Beede E Baylor E Hersch F et al A human-centered evaluation of a deep learning system deployed in clinics for the detection of diabetic retinopathy Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems Honolulu HI USA: Association for Computing Machinery 2020 1 12 10.1145/3313831.3376718
59 Borges do Nascimento IJ Abdulazeem H Vasanthan LT et al Barriers and facilitators to utilizing digital health technologies by healthcare professionals NPJ Digit Med 2023 6 161 10.1038/s41746-023-00899-4 37723240
60 Wang D Wang L Zhang Z et al “Brilliant AI doctor” in rural clinics: challenges in AI-powered clinical decision support system deployment Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems Yokohama Japan: Association for Computing Machinery 2021 Article 697
61 Petitgand C Motulsky A Denis JL et al Investigating the Barriers to Physician Adoption of an Artificial Intelligence- Based Decision Support System in Emergency Care: An Interpretative Qualitative Study Stud Health Technol Inform 2020 270 1001 5 10.3233/SHTI200312 32570532
62 Tanguay-Sela M Benrimoh D Popescu C et al Evaluating the perceived utility of an artificial intelligence-powered clinical decision support system for depression treatment using a simulation center Psychiatry Res 2022 308 114336 10.1016/j.psychres.2021.114336 34953204
63 Wang A Zhu B Huang J et al Quality of primary healthcare in China: challenges and strategies Hong Kong Med J 2023 29 372 4 10.12809/hkmj235149 37794614
64 Yuan S Wang F Li X et al Facilitators and barriers to implement the family doctor contracting services in China: findings from a qualitative study BMJ Open 2019 9 e032444 10.1136/bmjopen-2019-032444
65 Zou Y Zhang X Hao Y et al General practitioners versus other physicians in the quality of primary care: a cross-sectional study in Guangdong Province, China BMC Fam Pract 2015 16 134 10.1186/s12875-015-0349-z 26452648
66 Li X Krumholz HM Yip W et al Quality of primary health care in China: challenges and recommendations Lancet 2020 395 1802 12 10.1016/S0140-6736(20)30122-7 32505251
67 Jian W Li Q Zhang L The Performance of Primary Healthcare in China: The Need for a Systematic Design for Improvement Comment on “Quality and Performance Measurement in Primary Diabetes Care: A Qualitative Study in Urban China.” Int J Health Policy Manag 2023 12 7889 10.34172/ijhpm.2023.7889 37579387
68 Gu C Wang Y Jiang Y et al Application of artificial intelligence system for screening multiple fundus diseases in Chinese primary healthcare settings: a real-world, multicentre and cross-sectional study of 4795 cases Br J Ophthalmol 2024 108 424 31 10.1136/bjo-2022-322940 36878715
69 Liu CY Tang CX Zhang XL et al Deep learning powered coronary CT angiography for detecting obstructive coronary artery disease: The effect of reader experience, calcification and image quality Eur J Radiol 2021 142 109835 10.1016/j.ejrad.2021.109835 34237493
70 Elrod JK Fortenberry JL The hub-and-spoke organization design: an avenue for serving patients well BMC Health Serv Res 2017 17 457 10.1186/s12913-017-2341-x 28722550
71 Hassan AE Ringheanu VM Rabah RR et al Early experience utilizing artificial intelligence shows significant reduction in transfer times and length of stay in a hub and spoke model Interv Neuroradiol 2020 26 615 22 10.1177/1591019920953055 32847449
72 Elijovich L Dornbos Iii D Nickele C et al Automated emergent large vessel occlusion detection by artificial intelligence improves stroke workflow in a hub and spoke stroke system of care J Neurointerv Surg 2022 14 704 8 10.1136/neurintsurg-2021-017714 34417344
73 Ma L Zhu G Lu L Baseline for classified protection of cybersecurity (GB/T 22239-2019) standard interpretation Netinfo Sec 2019 19 77 84
74 Center NHCSI Notice of the ministry of health on issuing the guiding opinions on the protection of information security levels in the health industry 2011 Available http://www.nhc.gov.cn/mohwsbwstjxxzx/s8551/201112/53600.shtml
75 Gold R Middendorf M Heintzman J et al Challenges involved in establishing a web-based clinical decision support tool in community health centers Healthc (Amst) 2020 8 100488 10.1016/j.hjdsi.2020.100488 33132174
76 Harada T Miyagami T Kunitomo K et al Clinical Decision Support Systems for Diagnosis in Primary Care: A Scoping Review Int J Environ Res Public Health 2021 18 8435 10.3390/ijerph18168435 34444182
77 Wahl B Cossy-Gantner A Germann S et al Artificial intelligence (AI) and global health: how can AI contribute to health in resource-poor settings? BMJ Glob Health 2018 3 e000798 10.1136/bmjgh-2018-000798
78 Bozyel S Şimşek E Koçyiğit Burunkaya D et al Artificial Intelligence-Based Clinical Decision Support Systems in Cardiovascular Diseases Anatol J Cardiol 2024 28 74 86 10.14744/AnatolJCardiol.2023.3685 38168009
79 Gomez-Cabello CA Borna S Pressman S et al Artificial-Intelligence-Based Clinical Decision Support Systems in Primary Care: A Scoping Review of Current Clinical Implementations Eur J Investig Health Psychol Educ 2024 14 685 98 10.3390/ejihpe14030045
80 Schütze D Holtz S Neff MC et al Requirements analysis for an AI-based clinical decision support system for general practitioners: a user-centered design process BMC Med Inform Decis Mak 2023 23 144 10.1186/s12911-023-02245-w 37525175
81 Baker A Perov Y Middleton K et al A Comparison of Artificial Intelligence and Human Doctors for the Purpose of Triage and Diagnosis Front Artif Intell 2020 3 543405 10.3389/frai.2020.543405 33733203
82 Jones C Thornton J Wyatt JC Artificial intelligence and clinical decision support: clinicians’ perspectives on trust, trustworthiness, and liability Med Law Rev 2023 31 501 20 10.1093/medlaw/fwad013 37218368
83 EUR-Lex Proposal for a regulation of the European Parliament and of the council laying down harmonised rules on artificial intelligence (Artificial Intelligence Act) and amending certain union legislative acts 2021
84 Li R Yang Y Wu S et al Using artificial intelligence to improve medical services in China Ann Transl Med 2020 8 711 10.21037/atm.2019.11.108 32617331
85 Xia Z Gao W Wei X et al Perceived Value of Electronic Medical Records in Community Health Services: A National Cross-Sectional Survey of Primary Care Workers in Mainland China Int J Environ Res Public Health 2020 17 8510 10.3390/ijerph17228510 33212868
86 China NHCotPsRo Notification on the issuance of the national monitoring and analysis of performance evaluation of tertiary public hospitals for the year 2022 2022 Available http://www.nhc.gov.cn/yzygj/s3594r/202402/516d9853b9204e31867972c1a0e0be36.shtml
87 Liang J Li Y Zhang Z et al Adoption of Electronic Health Records (EHRs) in China During the Past 10 Years: Consecutive Survey Data Analysis and Comparison of Sino-American Challenges and Experiences J Med Internet Res 2021 23 e24813 10.2196/24813 33599615
88 Fraser H Coiera E Wong D Safety of patient-facing digital symptom checkers Lancet 2018 392 2263 4 10.1016/S0140-6736(18)32819-8 30413281
89 Lorenzini G Arbelaez Ossa L Shaw DM et al Artificial intelligence and the doctor-patient relationship expanding the paradigm of shared decision making Bioethics 2023 37 424 9 10.1111/bioe.13158 36964989
90 Shaffer VA Probst CA Merkle EC et al Why do patients derogate physicians who use a computer-based diagnostic support system? Med Decis Making 2013 33 108 18 10.1177/0272989X12453501 22820049
