
==== Front
BMJ Open
BMJ Open
bmjopen
bmjopen
BMJ Open
2044-6055
BMJ Publishing Group BMA House, Tavistock Square, London, WC1H 9JR

39182935
10.1136/bmjopen-2023-082547
bmjopen-2023-082547
Original Research
Public Health
1724
1506
Validation of a quantitative instrument measuring critical success factors and acceptance of Casemix system implementation in the total hospital information system in Malaysia
Mustafa Noor Khairiyah 12p115190@siswa.ukm.edu.my

http://orcid.org/0000-0002-4741-5970
Ibrahim Roszita 1roszita@ppukm.ukm.edu.my

Awang Zainudin 3zainudinawang@unisza.edu.my

Aizuddin Azimatun Noor 14azimatunnoor@ppukm.ukm.edu.my

Syed Junid Syed Mohamed Aljunid 5syedmohamed@imu.edu.my

1 Department of Public Health Medicine, Universiti Kebangsaan Malaysia Fakulti Perubatan, Cheras, Federal Territory of Kuala Lumpur, Malaysia
2 Ministry of Health Malaysia, Putrajaya, Malaysia
3 Faculty of Business Management, Universiti Sultan Zainal Abidin, Kuala Terengganu, Malaysia
4 International Casemix Centre (ITCC), Hospital Universiti Kebangsaan Malaysia, Cheras, Kuala Lumpur, Malaysia
5 Department of Public Health and Community Medicine, International Medical University, Kuala Lumpur, Federal Territory of Kuala Lumpur, Malaysia
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

None declared.

Dr; roszita@ppukm.ukm.edu.my
2024
24 8 2024
14 8 e08254728 11 2023
05 7 2024
Copyright © Author(s) (or their employer(s)) 2024. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.
2024
https://creativecommons.org/licenses/by-nc/4.0/ This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

Abstract

Objectives

This study aims to address the significant knowledge gap in the literature on the implementation of Casemix system in total hospital information systems (THIS). The research focuses on validating a quantitative instrument to assess medical doctors’ acceptance of the Casemix system in Ministry of Health (MOH) Malaysia facilities using THIS.

Designs

A sequential explanatory mixed-methods study was conducted, starting with a cross-sectional quantitative phase using a self-administered online questionnaire that adapted previous instruments to the current setting based on Human, Organisation, Technology-Fit and Technology Acceptance Model frameworks, followed by a qualitative phase using in-depth interviews. However, this article explicitly emphasises the quantitative phase.

Setting

The study was conducted in five MOH hospitals with THIS technology from five zones.

Participants

Prior to the quantitative field study, rigorous procedures including content, criterion and face validation, translation, pilot testing and exploratory factor analysis (EFA) were undertaken, resulting in a refined questionnaire consisting of 41 items. Confirmatory factor analysis (CFA) was then performed on data collected from 343 respondents selected via stratified random sampling to validate the measurement model.

Results

The study found satisfactory Kaiser-Meyer-Olkin model levels, significant Bartlett’s test of sphericity, satisfactory factor loadings (>0.6) and high internal reliability for each item. One item was eliminated during EFA, and organisational characteristics construct was refined into two components. The study confirms unidimensionality, construct validity, convergent validity, discriminant validity and composite reliability through CFA. After the instrument’s validity, reliability and normality have been established, the questionnaire is validated and deemed operational.

Conclusion

By elucidating critical success factor and acceptance of Casemix, this research informs strategies for enhancing its implementation within the THIS environment. Moving forward, the validated instrument will serve as a valuable tool in future research endeavours aimed at evaluating the adoption of the Casemix system within THIS, addressing a notable gap in current literature.

quality in health care
public health
health economics
==== Body
pmcSTRENGTHS AND LIMITATIONS OF THIS STUDY

The rigorous validation process of the questionnaire, including initial validation, translation, pre-testing and exploratory factor analysis using pilot test data, followed by confirmatory factor analysis using field data, enhances the reliability and validity of the instrument used for data collection.

The use of statistical techniques such as the Kaiser-Meyer-Olkin (KMO) measure, Bartlett’s test of sphericity, factor loadings, Cronbach’s alpha and various validity tests (unidimensionality, construct validity, convergent validity, discriminant validity) ensures the robustness of the analysis.

While the large sample size enhances generalisability to some extent, the study was conducted in only five selected hospitals in Malaysia; thus, the findings may not be representative of all hospitals in the country or other healthcare systems.

This study does not include other professional roles, such as paramedics, medical record officers, information technology officers and finance officers because the knowledge and involvement of these roles in the Casemix system are not comparable to that of medical doctors.

The findings of the study may be specific to the healthcare context in Malaysia and may not be directly applicable to other countries or healthcare systems with different sociocultural, organisational or technological characteristics.

Introduction

The global healthcare landscape is witnessing profound evolution driven by an array of challenges, including the rise of non-communicable diseases, the resurgence of communicable diseases, demographic shifts and escalating healthcare costs.1 Governments and healthcare authorities worldwide are under mounting pressure to navigate these complexities while optimising operational efficiency and ensuring equitable access to quality healthcare services.1 Within this context, Malaysia has emerged as a proactive player, spearheading innovative strategies to streamline healthcare delivery and bolster system performance. The Ministry of Health (MOH) Malaysia’s proactive stance is exemplified by its robust efforts to standardise and enhance the quality of healthcare services through the implementation of clinical standards and pathways based on international best practices.2 Notably, initiatives such as the hospital information system (HIS) and the Casemix system have been instrumental in revolutionising healthcare management practices and fostering a culture of continuous improvement.36

Background of Hospital Information System (HIS)

The HIS stands as a cornerstone of technological innovation in healthcare management, offering a comprehensive platform for efficient data collection, storage and processing.7 HIS responsibilities include managing shared information, enhancing medical record quality, overseeing healthcare quality and error reduction, promoting institutional transparency, analysing healthcare economics and reducing examination and treatment durations.813 In Malaysia, the adoption of HIS, categorised into total hospital information system (THIS), intermediate hospital information system (IHIS) and basic hospital information system, has paved the way for seamless integration of patient data, administrative tasks, and financial transactions and appointment management into a single system within a hospital.1419 The pioneering implementation of a fully integrated paperless system as a THIS facility at Hospital Selayang underscores Malaysia’s commitment to embracing cutting-edge technology to enhance healthcare delivery.2022 Today, 19 out of 149 Malaysian hospitals have IT facilities.23 24 Despite challenges during implementation, the overall advantage of using a comprehensive system is priceless.222529

Background of Casemix system

The Casemix system is a global system that categorises patient information and treatments based on their types and associated costs, aiming to identify patients with similar resource needs and treatment expenses.30 31 It is widely used globally such as in the USA, Western Europe, Australia, Eastern Europe and Asia, playing a crucial role in hospital financing.32 33 Originating from Australia, it optimises resource utilisation, improves cost transparency and enhances healthcare service efficiency.34 35 However, its adoption in developing nations like Malaysia faces challenges due to technological constraints and resource limitations.23 36 37 The Malaysian diagnosis-related group (MalaysianDRG) Casemix system categorises patients based on healthcare costs, improving efficiency and resource allocation.3840 This system enhances provider payment measurement, healthcare service quality, equity and efficiency, and assists policymakers in allocating cash for hospitals.24 41 The information from the MalaysianDRG is integrated into the executive information system, providing access to system outputs such as DRG, severity of illness, average cost per disease and Casemix Index.3840

Integration of Casemix within HIS

The integration of Casemix within HIS frameworks represents a paradigm shift in healthcare management, offering a unified platform for data-driven decision-making, performance monitoring and quality improvement initiatives.42 In the USA, there is a need to evaluate existing HIS against advanced hardware and software.42 As hospitals face public opposition due to rising medical expenses, governments are under pressure to manage healthcare costs more effectively.42 Casemix-based reimbursement policies aim to compensate medical expenses based on Casemix rather than the number of services provided.42 By consolidating clinical, administrative and financial data within a single system, Casemix-based systems are multifaceted and require organisational restructuring and educational initiatives for successful implementation.33 Strategies such as providing feedback to clinicians and integrating decentralised databases into HIS are crucial for ensuring data credibility and accuracy.33 Transitioning from traditional medical record management to health information management requires careful planning and adjustments due to the lack of automation in the current HIS.33

Theoretical and conceptual framework

Multiple frameworks are commonly used to evaluate technology systems’ acceptance and success attributes. There are noteworthy frameworks, such as the technology acceptance model (TAM), the DeLone and McLean Information Systems Success Model (ISSM), the HOT-Fit Evaluation Framework and the Unified Theory of Acceptance and Use of Technology (UTAUT). The TAM is a widely used framework for assessing the acceptability and success of technology systems, particularly in HIS.4347 It suggests that user perceptions of ease of use, usefulness and intention to use significantly impact system usage.4347 The DeLone and McLean ISSM evaluates the effectiveness of information systems by examining relationships between system quality, information quality, user happiness, individual impact, organisational impact and overall system success.48 49 The HOT-Fit Evaluation Framework, evolved from the ISSM, evaluates the congruence of persons, organisations and technology within an information system, considering technological variables, organisational factors and human factors.12 50 The UTAUT enhances the TAM by incorporating additional elements such as social impact, enabling situations, and behavioural intentions.43 51 52

By integrating these frameworks within the context of Casemix implementation within THIS, the investigators aim to assess critical success factors and address barriers to adoption and acceptance, facilitating seamless integration and maximising the potential of healthcare modernisation efforts. Hence, the investigators opted to integrate HOT-Fit and TAM frameworks as this study’s conceptual framework to achieve the research’s specific objectives, scope and contextual considerations (see figure 1). HOT-Fit offers a comprehensive framework for examining the alignment between human, organisational and technological factors, while TAM provides a focused lens on individual-level technology acceptance dynamics.124447 Based on the current study’s conceptual framework, the HOT-Fit framework focuses on technological constructs like system, information and service quality, while the TAM framework covers human dimensions like perceived ease of use, usefulness, intention to use and acceptance. The integration of these frameworks is crucial for achieving the study’s specific and general objectives. Thus, these two frameworks are suitable and deemed appropriate for this study. On the other hand, UTAUT does not appear suitable for the current investigation due to the broad scope and complexity of existing TAM with additional external variables and ISSM was also not selected due to its simplicity.43 51 52

Figure 1 Conceptual framework.

This current study aims to evaluate the critical success factors (CSFs) and doctors’ acceptance of Casemix implementation within the THIS environment to understand the issues MOH Malaysia facilities experience better, fill a research gap on Casemix implementation and help shape plans for modernising healthcare. A comprehensive tool, such as a questionnaire, was created to meet the study objectives. This paper aims to examine a multidimensional instrument that was created to meet the study objectives. Consequently, the exploratory factor analysis (EFA) is instrumental in uncovering underlying factors within observed variables to ensure precision and robustness, while confirmatory factor analysis (CFA) was needed to verify the measurement model’s linkages and confirm that the theoretical model was valid, reliable and suitable for data collection, thereby yielding valuable insights.5356 Given its merits, the current study used CFA to evaluate the measurement model’s validity. After validation processes, structural equation modelling (SEM) was employed to analyse how exogenous, mediating and endogenous constructs interrelate and determine parameters into a structural model to analyse direct, mediating and moderating effects on the study’s goals and hypotheses. While the technology evaluation frameworks offer crucial insights, it is essential to note that Casemix is designed to organise patient data and treatment costs rather than analyse the acceptability and success of technology systems. Moreover, meeting the study objectives for evaluating Casemix adoption in THIS can be done without a separate instrument for each system. It can assist healthcare organisations and policymakers in understanding CSFs facilitating the implementation and acceptance of the Casemix system, and guiding the development of targeted strategies for seamless implementation, enhancing patient care, work efficiency and resource allocation. Therefore, a reliable and valid quantitative instrument is required to achieve these goals.

Methodology

Study design and ethical approval

Study design

This study employed a sequential explanatory mixed-methods design. Nevertheless, the researchers in the present article solely highlight the exploration and development of items, as well as the reliability and validation of the quantitative study. The data collection for the quantitative pilot study was from 1–14 February 2023, the quantitative phase was from 1 April to 31 June 2023, the qualitative pilot study was on 15 September 2023 and the qualitative field study was from 17 October 2023 to 4 January 2024. This paper highlights on the development of instruments for quantitative phase procedures and findings of the validation of quantitative study only. The quantitative phase used a cross-sectional study design to gather data throughout a specified duration.53 57 58

Ethical approval

This study has obtained ethical approval from:

The Medical Research Ethics Committee of the Faculty of Medicine, Universiti Kebangsaan Malaysia (JEP-2022–777), see (online supplemental file 1), and

The Medical Research Ethics Committee of the Ministry of Health Malaysia (NMRR ID-22–02621-DKX), see (online supplemental file 2).

Study instrument

This study used a self-administered questionnaire to collect data on the CSF and acceptance of Casemix in THIS environment. The instrument was developed in Malay and English for a better understanding of the respondents due to the geographical areas of the study where Malay is the national language of Malaysia. The questionnaire comprised 60 items divided into three sections, each with a limited number of constructs. Section 1a consists of 8 questions that collected demographic information such as age, gender, educational background and work experience in the MOH Malaysia and current hospital. Section 1b assessed the comprehension/knowledge level of the Casemix system using 10 items. Meanwhile, Section 2 represented the perceived Critical Success Factors of Casemix implementation in the THIS context, consisting of 37 items within six constructs: system quality (SY)—4 items, information quality (IQ)—5 items, service quality (SQ)—5 items, organisational factors (O)—9 items, perceived ease of use (PEOU)—5 items, perceived usefulness (PU)—4 items and intention to use (ITU)—5 items. Section 3 encompasses the outcome of the study which is the user acceptance (UA) construct, which contains 5 items.

The study incorporates and modifies existing scholarly works rooted in the Human Organisation Technology (HOT-Fit) and TAM frameworks for sections 2 and 3. The two sections, each evaluated using a 10-point interval Likert scale. The 10-point interval scale offers respondents a greater range of response possibilities that align with their precise evaluation of a question.55 56 59 60 A score of 1 represents ‘strongly disagree’, while a score of 10 represents ‘strongly agree’. The constructs and components of the instrument were derived from previous research.1243 44 48 50 6164 These items represented eight constructs: SY, IQ, SQ, ORG, PEOU, PU, ITU and User Acceptance.

The constructs described in sections 2 and 3 underwent initial validation, reliability testing and EFA, using pilot data. CFA was also performed using field data. Details regarding the development validation and reliability procedures of the instrument are provided in subsequent sections. Hence, to facilitate transparency and reproducibility, a blank copy of the measurement instrument developed and validated in this study has been included as a supplementary file (see online supplemental file 3: Blank Copy of Quantitative Instrument).

Independent variables

A few constructs have been examined in this study as mentioned in Subsection 1.6, the conceptual framework comprising technology, organisation and human dimensions.

Technological factors

Constructs such as system quality (SY), information quality (IQ) and service quality (SY) constitute the technological factors. Addressing system quality issues is imperative for fostering user acceptance and realising system benefits.43 Reliable and accurate systems with dependable functionality enhance user acceptance, while a user-friendly interface and seamless performance enhance user experience. Integration with existing systems promotes acceptability and interoperability.43 44 Conversely, information quality, encompassing data security and privacy, is crucial in safeguarding patient data, bolstering user confidence and fostering system adoption.65 Service quality encompasses the support and assistance provided during and after system implementation, with practical training, responsive helpdesk support, and ongoing maintenance contributing to user satisfaction and system success.51 66 67 Hence, these three constructs encompassing technological dimensions were adapted from the HOT-Fit framework.12 50

Organizational characteristics

Organisational dimensions, such as an organisational structure and environment, can limit or facilitate the acceptance or implementation of technical advancements.68 The elements of organisational dimension were the most generally surveyed attributes in IT adoption in organisations.69 Previous research has identified relative benefit, centralisation, formalisation, top management support and perceived cost as essential organisational elements influencing any organisation’s decision to embrace current information systems technologies. Management barriers are defined as a lack of efficient planning, a lack of trained people, and limits linked to training courses, according to Abdulrahman and Subramanian.70 The management, technological, ethical-legal and financial barriers were all integrated into the organisational factor category in this study. Previous research has found that technology adoption rates are related to preparedness and impediments to readiness.71 Along with several other studies, senior leaders play a critical role in using information systems at the organisational level.72 Direct involvement of senior executives in IS operations demonstrates the importance of IS and ensures their support and involvement in the overall performance of IS efforts in the organisation.73 Organisational environment and structure can influence user acceptance of information technology, underscoring the importance of organisational improvement initiatives to enhance user acceptance.7477 Hence, this primary construct encompassing organisational dimensions was adapted from the HOT-Fit framework.12 50

Human factors

The TAM is a framework that consists of five fundamental elements: PEOU, PU, ITU, actual system use and external Variables.7881 PEOU is a subjective evaluation of a technology’s ease of use, influenced by usability, training and user assistance.7881 PU quantifies the level of usefulness attributed to technology, influenced by factors such as usefulness and compatibility with user needs and responsibilities.7881 Intention to Use (ITU), External factors, such as organisational regulations, access and availability, can also influence the interactions within the model.7881 External variables, such as individual variances, cultural influences and supportive environments, can either amplify or reduce the impact of perceived ease of use and usefulness on behavioural intention and actual use.7881 The TAM has been a crucial paradigm for understanding technology acceptance and has significantly impacted research in information systems and technology adoption. The HOT-Fit Evaluation technique, which focuses on system use and user satisfaction, is suitable for this study.12 50 These two constructs are interconnected to PEOU and PU, delineated by the TAM framework.7881 For successful implementation of an information system, medical doctors perceive it as easy to use (PEOU) through adequate training, user-friendly interfaces and intuitive system design.7881 Healthcare providers should also perceive the system as useful (PU) to ensure successful implementation, highlighting its benefits such as improved efficiency, quality of care and cost control.7881

Dependent variable

The only dependent variable in this study is acceptance which is adapted from the TAM.44 45 82 The study presents a pragmatic taxonomy of eight different implementation outcomes, including acceptability/acceptance, adoption, appropriateness, feasibility, fidelity, implementation cost, penetration and sustainability.64 Acceptability is a crucial aspect of implementation, referring to the acceptance of a specific intervention, practice, technology or service within a specific care setting.64 It can be measured from the perspective of various stakeholders, such as administrators, payers, providers and consumers.64 Ratings of acceptability are assumed to be dynamic and may differ during pre-implementation and throughout various stages of implementation. In similar literature, Proctor et al delineated examples of measuring provider and patient acceptability/acceptance including case managers’ acceptance of evidence-based procedures in a child welfare system and patients’ acceptance of alcohol screening in an emergency department.64 The terms acceptability and acceptance are interchangeably used to describe implementation outcomes. Therefore, in this study, the researchers would like to explore the acceptance of the Casemix system in the MOH’s THIS facilities.

Patients and public involvement

Participants in this study were medical doctors and this study did not involve any patients or the public. Hence, there was no patient or public involvement in this study.

Initial validation processes

The initial validation procedures were conducted to establish the content, criteria and face validity/pre-test of the instrument for the field study.

Content validity

Content validity is significant when developing new measurement tools because it links abstract ideas with tangible and measurable indicators.83 This involves two main steps: identifying the all-inclusive domain of the relevant content and developing items that correspond to this domain.83 The Content Validity Index (CVI) is often used to measure this validity.8486 Recent studies have demonstrated the content validity of assessment tools using the CVI.8790 The best method for calculating the CVI, suggesting that the number of experts reviewing an instrument should range from 2 to 20.848691 92 Typically, the number of experts varies from 2 to 20 individuals.93 For the current study, two experts from the Hospital Financing (Casemix Subunit) at MOH Malaysia were selected. This is coherent with the number of experts that are recommended by a few literature in online supplemental file 4A.84 There are two types of CVI: I-CVI for individual items and S-CVI for overall scales.848691 92 S-CVI can be calculated by averaging the I-CVI scores (S-CVI/Ave) or by the proportion of items rated as relevant by all experts (S-CVI/UA).848691 92 Before calculating CVI, relevance ratings are converted to binary scores. The relevance rating was re-coded as 1 (scale of 3 or 4) or 0 (scale of 1 or 2), as indicated in online supplemental file 4B. Online supplemental file 4C reveals two experts’ item-scale relevance evaluations to exhibit CVI Index calculation. In this study, the experts validated the questionnaire contents, achieving perfect scores of 3 or 4 for all items, resulting in S-CVI/Ave and S-SCVI/UA scores of 1.00. In conclusion, a thorough methodological approach to content validation, based on current data and best practices, is essential to confirm the overall validity of an evaluation.

Criterion validity

Criterion validity denotes to the degree of correlation between a measure and other established measures for the same construct.62 88 89 94 95 An academic statistics expert and an expert in questionnaire development and validation procedures reviewed criterion validity. This can be reviewed in online supplemental file 5. Subsequently, a certified translator translated the instrument from English to Malay back-to-back precisely.

Face validity

A face validity assessment was undertaken to evaluate the questionnaire’s consistency of responses, clarity, comprehensibility, ambiguity and overall comments. Before commencing the pilot study and fieldwork, the researchers acknowledged and resolved the concerns that were previously mentioned.62 90 96 Following the validation process, 11 respondents were purposefully selected for face validity also known as pre-testing to accomplish the prerequisite for face validation. Furthermore, they must meet exclusion criteria like those stipulated for participants in the field study. Subsequently, these respondents were excluded from participation in the quantitative field study. The study population will be described further in Subsection 2.6.2. The objective of this pre-test or face-validation process was to assess the consistency of responses, and clarity, ambiguity and overall design of the questionnaire.97 This will be done through the evaluation from the online Google Form of the Questionnaire. Before conducting the pilot study and fieldwork, the researchers took into consideration the concerns that had been raised.97 The face validity result has been uploaded as online supplemental file 6.

Quantitative pilot test and EFA

The pilot study was conducted at a Federal Territory hospital in Malaysia, Hospital W. The pilot study population also possess similar characteristics to the participants/samples involved in the subsequent quantitative field study. Additionally, these respondents were excluded from participation in the quantitative field study. This study used a minimum of 100 samples to ensure valid results for the EFA.97 98 Hence, since the current pilot study is using EFA, the minimal sample size of 100 is therefore supported by a few studies and books experienced in research and validation procedures.545697 99 Therefore, to account for a projected drop-out rate of 20%, the minimum sample size for this preliminary pilot study was determined to be 125 medical doctors.100 The research was conducted without participant or public involvement in the design, conduct, reporting or dissemination strategies. The data collection method was also like the field study. It was employed using an online Google Form Questionnaire. Participants were asked to scan a Google Form link or QR code to access information sheets, consent forms and online questionnaires. Each participant was notified that their information would be kept private their anonymity would be retained solely for the study, and they could withdraw at any time.

The pilot study will use EFA to measure data from a collection of hidden concepts. EFA is a method that generates more accurate results when each shared component is represented by many measured variables, either exogenous or endogenous constructs.545697 98 101 The collected data will be used to identify and quantify the dimensionality of items that assess the construct.535659 60 104 EFA is essential to determine whether items in a construct produce distinct dimensions from those found in previous studies.535659 60 104 Factors’ dimensionality may change as they are transported from other domains to a new research topic, and fluctuations in the population’s cultural heritage, socioeconomic status and passage of time might affect dimensionality. The EFA methodology uses principal component analysis (PCA) to decrease the amount of data, but it fails to discern between common and unique changes efficiently.97 98 PCA is indicated when there is no known theoretical framework or model, and it is used to create the first solutions in EFA. Four requirements of PCA included (1) components with eigenvalues more than one, (2) factor loadings greater than 0.60 for practical relevance, (3) no item cross-loadings greater than 0.50 and (4) each factor has at least three items to be retained.97 98 The data’s eligibility for factor analysis was determined using the Kaiser-Meyer-Olkin Measure of Sampling Adequacy (KMO) of >0.6 and Bartlett’s test of sphericity.5556 105107 The effectiveness of Bartlett’s test for factor analysis hinges on the significant result, with a value near p<0.001 (p<0.05) indicating acceptability.5356107 The scree plot also determined the best number of constructs to keep.5356

Quantitative field study

Study location

The present study gathered data from five hospitals situated in various Malaysian zones—South, North, West, East and East Malaysia—that are outfitted with the Total Hospital Information System (THIS) and Casemix system. The study used cluster sampling to select study sites in Malaysia, dividing the country into five distinct clusters. Five hospitals that had successfully implemented Casemix for at least 3 years were chosen to represent different regions of Malaysia. Hospital N was selected for the northern region, Hospital E for the eastern region, Hospital S for the southern region and Hospital W for the central/western region. Hospital EM was chosen for East Malaysia. Cluster sampling is suitable when the research encompasses a vast geographical expanse.

Target population for the study

The target population for this study was medical doctors by profession working in hospitals under MOH in 2023. The study collectively obtained a sampling frame of 3580 medical doctors by profession, encompassing hospital directors, deputy directors (medical division), consultants/specialists, medical officers and house officers from the five selected hospitals. These doctors should fulfil the inclusion and exclusion criteria of this study as follows:

Inclusion criteria

Permanent/ contract of service medical doctors who were posted to current participating hospital.

Has working experience in the current participating hospital for at least 3 months.

Agree to participate in the study.

Exclusion criteria

Attachment medical doctors.

Refuse to participate in the study.

The study population of face validation/pre-test and pilot test has characteristics similar to those of the study population in the field study. The pre-test and pilot-test samples will also be excluded from samples in the field study. Participants were given surveys to complete at their own pace, without fear or pressure.

Sample size and sampling method

The target population was selected using proportionate stratified random sampling, dividing the total population into homogeneous groups.16108110 Proportionate stratified random sampling is a probability sampling method that includes separating the entire population into similar groups (strata) to conduct the sampling process.

The authors are concerned about the sample size needed for CFA validation of the measurement model. However, current studies do not have a consensus on the appropriate sample size. For small indicators, a minimum sample size of 100–150 respondents is often needed,111113 whereas, precise analysis for CFA may require 250–500 respondents.114 115 Some authors suggested the following suggestions for the sample size requirement: (a) a sample size to parameter ratio of 5 or 10, (b) ten cases per observation/indicator and (c) 100 cases/observations per group for multigroup modelling.116118 In conclusion, the researchers opted to employ five times the number of indicators in the questionnaire because the number of indicators for latent variables is large.116 119 The final questionnaires contain 59 items, requiring a total sample size of 295. However, there is an additional 20% anticipated dropout rate. The sample size was estimated using the formula n=n/1-d (n=total samples, n=minimum required samples and d=drop out rates), yielding a minimum sample size of 369.100 This is also corroborated by other research, which states that because the conceptual framework in this study consists of eight constructs, each with at least four items, the required sample size is 300, with an additional 20% expected drop-out rate, the calculated sample size was 375.56 97 100 102 As a result, the researchers opted to distribute questionnaires to the 375 participants using proportionate stratified random sampling depending on their professional roles as suggested.56 97 102 116

Data collection methods

The data collection method for the quantitative field study is similar to the techniques used in the quantitative pilot study. This data collection method was elaborated in Subsection 2.4.4. However, the link for the participant information sheet (PIS) and informed consent forms was included on the first page of the questionnaire which is https://bit.ly/3F8IF2e. The participant’s information sheet and informed consent forms are attached as onlinesupplemental files 7 8, respectively. Similarly to the quantitative pilot study, respondents may do so freely without losing their data if they withdraw from the survey midway. Participants were assured that their information would be kept confidential and that their anonymity would be strictly protected during the field study. Participants who wish to participate must first consent and complete all survey questions. They were also instructed to contact the lead investigator with any questions. The participants have up to 2 weeks to complete and submit the online questionnaire. All survey information was linked to a research identification number. For example, study identifications 001 to 375 on the subject data sheets will be used instead of the subject’s name. The appropriate senior management and Casemix System Coordinators (CSCs), the department’s Casemix Coordinator and Heads of Department will be contacted 3 days before the data gathering session concludes. All measures were taken to safeguard participants’ privacy and anonymity.

Data analysis using Confirmatory Factor Analysis (CFA)

Once the EFA technique has been completed, these constructs and emerging components of the revised conceptual framework were used in the field study. Hair et al and Awang et al described two distinct models in the field study: the measurement model used in the CFA technique and the structural model used to estimate paths using the SEM.545697 99 This study paradigm has the features of a confirmatory form of research, with a focus on behavioural components. This type of SEM is known as covariance based-SEM (CB-SEM) and exhibits theory testing or theory-driven research that integrates existing theories to replicate an established theory into a new domain, confirming a pre-specified relationship.545697 99

The SPSS Analysis of Moment Structures (AMOS) V.24.0 software was used in CFA to evaluate the unidimensionality, validity and reliability of the measurement model.53 54 56 The instrument’s normality is also achieved using CFA.53 54 56 There are two ways to validate measurement models: pooled and individual CFA.5456120 121 Pooled-confirmatory factor analysis’ (Pooled-CFA) higher degree of freedom enables model identification even when some constructs have fewer than four components.5456120 121 The missing data will be omitted/discarded from the analysis. To ensure unidimensionality, the permissible loading factor for each latent construct is calculated, and items that cannot fit into the measurement model due to low factor loading are excluded.5355 56 97 122125 The cut-off value for acceptable factor loading varies depending on the research goal. However, this study used a threshold value of 0.5 to minimise item deletion.53 55 56 97 121 122 126 Convergent validity is assessed by calculating the average variance explained (AVE) for each construct.53 55 56 97 111 122 Meanwhile, composite reliability (CR) assesses how often a construct’s underlying variables are used in structural equation modeling.53 55 56 97 122 A latent construct’s CR must be 0.6 to achieve composite reliability.53 55 56 97 122

Several fitness indicators were reported among scholars. Some recommendations are to report fit indices as absolute fit (chi-squared goodness-of-fit (Χ2) and standardised root mean square residual, or SRMR), parsimony-corrected fit (root mean square error of approximation, or RMSEA), Comparative Fit Index (CFI) and comparative fit (Tucker-Lewis Fit Index (TLI)).545699 123 124 126 They advised using at least one index from the three fitness categories: absolute fit, incremental fit and parsimonious fit.5456123 124 126 A model fit was indicated using a set of cut-off values: RMSEA values from 0.05 to 1.00, CFI >0.90 and Chisq/df<5.00, which would imply a reasonable fit.5356126 129

Results

Findings for the pilot test through exploratory factor analysis

Out of the required minimum sample size of 125, a total of 106 participants took part in the quantitative pilot study, resulting in an 84.8% response rate. According to Hair et al and Awang et al, in order to conduct an EFA, at least 100 samples are needed.545697 However, considering a potential drop-out rate of 20%, the minimum required sample size for this pilot study is 125. Researchers performed an EFA to find the primary dimensions from a wide set of latent constructs represented by 42 items before conducting the CFA. EFA uses PCA as the extraction method to reduce data and create a hypothesis or model without pre-existing preconceptions about the variables’ quantity or nature.545697 132 The EFA deemed indicators above 0.60 significant, and indicators loading into the same component were combined to match the measurement model.97 The measurement model (for CFA) and structural model (for path estimation) of SEM will use EFA results.545697 99 EFA was used to evaluate and appraise the items measuring the construct, while CFA was used to validate the measurement.12 43 44 50 61 EFA and CFA used pilot and field study data, respectively. EFA is a method used to select factors for retention or removal, using PCA and varimax rotation. It is a popular orthogonal factor rotation approach that clarifies factor analysis.53 55 56 97 122 The extraction technique reduces the organisational factors (O) from nine to eight items, with one item, ‘Organisational competency to provide the resources for the implementation of the Casemix system in THIS setting,’ not reaching the factor loading of 0.6, hence it was 55 97 see table 1.

Table 1 Factor loading of EFA with PCA and varimax rotation

	Rotated component matrix*component	
1	2	3	4	5	6	7	8	9	
PEOU1			0.908							
PEOU2			0.916							
PEOU3			0.889							
PEOU4			0.919							
PEOU5			0.895							
PU1								0.872		
PU2								0.888		
PU3								0.914		
PU4								0.872		
O1				0.590						
O2							0.873			
O3							0.746			
O4							0.874			
O5							0.861			
O6				0.827						
O7				0.864						
O8				0.808						
O9				0.882						
SY1					0.826					
SY2					0.821					
SY3					0.836					
SY4					0.853					
IQ1									0.614	
IQ2									0.625	
IQ3									0.654	
IQ4									0.689	
IQ5									0.619	
SQ1		0.754								
SQ2		0.796								
SQ3		0.788								
SQ4		0.848								
SQ5		0.830								
ITU1	0.825									
ITU2	0.792									
ITU3	0.770									
ITU4	0.925									
ITU5	0.890									
A1						0.676				
A2						0.694				
A3						0.625				
A4						0.878				
A5						0.833				
Extraction Mmethod: principal component analysis.

Rotation Mmethod: Vvarimax with Kaiser Normalizationnormalisation.

* Rotation converged in 8eight iterations.

To prepare for the next stage, the researcher reorganises the objects into their respective components and begins data collection in the field study. The EFA results also reveal that the two components of the organisational characteristics (O) construct were later named organisational structure (STR) and organisational environment (ENV).53 55 56 97 122 The instrument was used for 41 items in the field study and analysed with Cronbach’s alpha, ensuring its internal reliability for the field study,535697 133 see table 2 below.

Table 2 The number of items for each construct before and after EFA and Cronbach’s alpha

No. of construct	Name of construct	Name of component	Item codes	Number of items before EFA	Number of items dropped	Number of items retained after EFA	Cronbach’salpha (>0.7)	
1	System quality		SY1-SY5	4	–	4	0.968	
2	Information quality		IQ1-IQ4	5	–	5	0.950	
3	Service quality		SQ1-SQ5	5	–	5	0.902	
4	Organisational factors		O1-O9	9	1	8	0.933	
		Structure	O2-O5				0.958	
		Environment	O6-O9				0.919	
5	Perceived ease of use		PEOU1-PEOU5	5	–	5	0.969	
6	Perceived usefulness		PU1-PU4	4	–	4	0.914	
7	Intention to use		ITU1-ITU5	5	–	5	0.949	
8	User acceptance		A1-A5	5	–	5	0.952	
Total			42	1	41		

Consolidating correlated variables was EFA’s primary goal. EFA established eight constructs from the pilot study data and according to the researcher’s conceptual framework (See figure 1).5355 The overall results of KMO and Bartlett’s sphericity test for all constructs, see table 3. The KMO value was 0.859, which is larger than 0.6. The result of Bartlett’s test of sphericity shows that p value <0.001 yielded statistically significant findings, which is p value <0.05.53 55 56 97 122 Therefore, it is appropriate to proceed with further study.

Table 3 Results of the KMO and Bartlett’s test of sphericity

Kaiser-Meyer-Olkin measure of sampling adequacy	0.859	
Bartlett’s test of sphericity	Approx. chi-square	4886.279	
df	861	
Sig.	0.000 (p<0.001)	

The amount of variance accounted for, referred to as total variance explained (TVE),535697 see table 1 (online supplemental file 9). Each component had an eigenvalue larger than 1 and the TVE was 84.07%, exceeding 60%.535697 The researcher should contemplate incorporating more items to assess the structures as it indicates that the existing items are inadequate for accurately assessing the constructs if the TVE is less than 60%. However, this does not occur in the present study.

The EFA approach also includes the scree plot. The researcher can ascertain the number of components by observing the distinct slopes in the scree plot.535697 The scree plot exhibits nine distinct slopes, as shown in figure 1 (online supplemental file 9). Hence, the EFA identifies a total of nine components.

Cronbach’s alpha would calculate measuring each item’s internal reliability. Internal reliability assesses how well the selected items measure the same construct.535697 133 All constructs topped 0.7 Cronbach’s Alpha. Hence, this instrument is reliable for use in field study.

Findings for the field study through the confirmatory factor analysis

The ultimate measurement tool for field study comprises 41 elements from the EFA procedure. To adequately address the intricacy of the quantitative instrument for the field study, the researchers determined that a minimum of 300 samples was necessary to implement CFA.97 An additional 20% drop-out rate resulted in a minimum sample size of 375 individuals for the field study. Hence, out of this sample, only 343 participants answered, indicating a response rate of 91.5%.100 No missing data was reported.

CFA validates factor loading and assessment in this study. The researcher tests a theory or model using CFA. Unlike EFA, CFA is a form of structural equation modelling that makes assumptions and expectations about the number of factors and which factor theories or models best suit prior theory.535697 EFA relied mainly on outer loading; however, factor loadings and fitness indices are now considered. Researchers must confirm that both folds meet standards. CFA also lets academics test financial literacy indicators and measurement models. Thus, a proper measuring model helps researchers interpret their data.

Validity, unidimensionality and reliability were necessary for all latent construct assessment models.53 55 56 97 122 The latent construct measurement model needed convergent, construct and discriminant validity.53 55 56 97 122 AVE assesses convergent validity, while measurement model fitness indicators determine construct validity.5456 On the other hand, composite reliability (CR) was used to calculate instrument reliability since it was better than Cronbach’s alpha.5456133

Figure 2 shows that Pooled-CFA validated all latent constructs in the measurement model simultaneously. These constructs were aggregated using double-headed arrows to execute a Pooled-CFA. Pooled-CFA’s increased degree of freedom allows model identification even when some constructs have fewer than four components.5456 Pooled-CFA was employed in this investigation since only one construct has two components.

Figure 2 Result from Pooled-CFA procedure.

Uni-dimensionality

Unidimensionality is a set of variables that can be explained by one construct.79 Unidimensionality is achieved when all construct-specific measuring items have acceptable factor loading.5456 Remove CFA components with low factor loadings from the measurement model until fit indices are met.535697 134 Table 4 summarises the build items with factor loadings >0.6.5456

Table 4 Factor loading of all items, composite reliability (CR) and average variant extracted (AVE) and normality testing

Construct	Component	Items	Factor loading	CR	AVE	Skewness	
			(>0.6)	(>0.6)	(>0.5)	(−1.5 to 1.5)	
System quality		SY1	0.876	0.936	0.787	−0.252	
SY2	0.880	−0.162	
SY3	0.917	−0.226	
SY4	0.873	−0.243	
Information quality		IQ1	0.923	0.954	0.806	−0.294	
IQ2	0.914	−0.055	
IQ3	0.895	−0.138	
IQ4	0.879	−0.018	
IQ5	0.877	−0.127	
Service quality		SQ1	0.829	0.925	0.710	−0.213	
SQ2	0.852	−0.233	
SQ3	0.815	−0.266	
SQ4	0.863	−0.175	
SQ5	0.852	−0.246	
Organisational characteristics	Structure		0.883	0.923	0.857		
Environment	0.967		
Structure	STR1	0.898	0.922	0.747	−0.390	
STR2	0.912	−0.227	
STR3	0.801	−0.284	
STR4	0.842	−0.109	
Environment	ENV1	0.875	0.903	0.699	−0.184	
ENV2	0.876	−0.315	
ENV3	0.773	−0.088	
ENV4	0.816	−0.107	
Perceived ease of use		PEOU1	0.901	0.951	0.796	−0.267	
PEOU2	0.896	−0.314	
PEOU3	0.896	−0.360	
PEOU4	0.889	−0.313	
PEOU5	0.879	−0.284	
Perceived usefulness		PU1	0.845	0.920	0.742	−0.336	
PU2	0.893	−0.335	
PU3	0.879	−0.516	
PU4	0.828	−0.495	
Intention to use		ITU1	0.899	0.951	0.794	−0.107	
ITU2	0.907	−0.209	
ITU3	0.879	−0.325	
ITU4	0.884	−0.306	
ITU5	0.885	−0.331	
User Acceptance		A1	0.913	0.932	0.733	−0.532	
A2	0.894	−0.542	
A3	0.903	−0.441	
A4	0.788	−0.395	
A5	0.773	−0.413	

Validity

Convergent validity

Convergent validity is a group of indicators that measures a construct.545697 135 It assesses the strength of correlations between items that are hypothesised to measure the same latent construct.56 97 The average variance extracted (AVE) statistic can be used to verify the convergent validity of a construct. If the concept’s AVE is more than 0.5, it possesses convergent validity.53 56 97 136 Table 4 shows that the AVE for all structures was more than 0.5. Organisational characteristics/factors (ORG) AVE shows the highest AVE, which was 0.857, and environment component, the lowest AVE, which is 0.699. The model is, therefore, convergently valid.

Construct validity

When all model fitness indices met the criteria, construct validity was attained.55 56 97 Construct validity was established using absolute, incremental and parsimonious fit indices.55 56 97 Some researchers recommend using one fitness index from each model fit category.55 56 97 This study employed RMSEA, CFI and normed chi-square (x2)/df as its main indicators. According to table 5, this instrument met all three fitness indices: (1) the RMSEA value was below the threshold of 0.08 (0.054), confirming the absolute fit index; (2) the instrument achieved the incremental fit index category by obtaining a CFI value above 0.90; and (3) the parsimonious fit index, measured using Chisq/df, yielded a value of 2.014, which is below the accepted value of 3.0.55 56 97 This study proved the instrument’s construct validity.

Table 5 Fitness index summary

Fitness category	Name of the fitness index	Full name	Level of acceptance	Index value	Comment	Literature	
Absolute fit	Chi-square	Discrepancy χ2	p>0.05	<0.001	Not applicable (sample size >200)	97 163	
RMSEA	Root mean square of error approximation	<0.1 (The best <0.08)	0.054	Achieved	53 56 97	
GFI	Goodness of Fit Index	>0.85 (The best >0.9)(0.1–1.0)*	0.819	Achieved	53 56 97163*	
Incremental fit	AGFI	Adjusted goodness of fit	>0.85 (The best >0.9)(0.1–1.0)*	0.792	Achieved	53 56 97164*	
TLI	Tucker-Lewis Index	>0.85 (The best>0.9)	0.943	Achieved	53 56 97	
CFI	Comparative Fit Index	>0.85 (The best >0.9)	0.948	Achieved	53 56 97	
NFI	Normed Fit Index	>0.85 (The best >0.9)(0.1–1.0)*	0.902	Achieved	53 56 97165 *	
Parsimonious fit	Chi-square/df	Chi-square/degree of freedom	<5.0 (The best <3.0)	2.014	Achieved	53 56 97	
level of acceptance is referred to literature with designator*

Discriminant validity

The survey’s discriminant validity was tested to ensure no redundant constructs were found in the model. The model is discriminant when the square root of the average variance extracted (AVE) for each construct is greater than its correlation value with other constructs.55 56 136 Table 6 summarises the discriminant validity index, which showed that all constructs met the threshold.55 56 136 The diagonal values (bold font) in this table were greater than all other values in their row and column, suggesting discriminant validity for all constructs.55 56 136

Table 6 Discriminant Validity Index

	SY	IQ	SQ	ORG	PEOU	PU	ITU	UA	
System quality (SY)	0.887								
Information quality (IQ)	0.777	0.898							
Service quality (SQ)	0.725	0.780	0.842						
Organisational characteristics (ORG)	0.818	0.760	0.696	0.926					
Perceived ease of use (PEOU)	0.183	0.298	0.271	0.246	0.892				
Perceived usefulness (PU)	0.488	0.592	0.511	0.493	0.274	0.861			
Intention to use (ITU)	0.680	0.780	0.633	0.697	0.276	0.599	0.891		
User acceptance (UA)	0.660	0.770	0.641	0.711	0.219	0.563	0.848	0.856	

Composite reliability

Estimating model reliability uses composite reliability (CR).55 56 97 CR between 0.6 and 0.7 is acceptable.55 56 97 Table 4 above shows that the instrument’s composite reliability exceeded 0.6 for all structures. The environment component had the lowest CR (0.903), while the information quality construct had the highest (0.954). Therefore, this instrument’s composite reliability is accomplished.

Normality assessment

Each item evaluating the construct’s distributional normality was assessed. All skewness values must be within the usual range.56 97 Skewness between −1.5 and 1.5 is considered acceptable. All model components’ skewness values are between −1.5 and 1.5, indicating their normal distribution.56 97 The instrument’s data distribution met the normality condition, as shown in table 4.

Discussion

This study focused on redeveloping and validating an instrument to gauge medical doctors’ intent to use and accept the Casemix system within the Total Hospital Information System (THIS) context. The EFA and CFA indicated that the instrument was well-designed and validated for assessing medical practitioners’ acceptance of the Casemix system in THIS setting.55 56 97 The acceptance of the Casemix system among medical physicians in hospital information systems was found to be influenced by various factors including system and service quality, perceived ease of use, usefulness, relevance to clinical practice, training and good organisational support, impact on efficiency and productivity, and confidence in information quality involving data accuracy and security. Healthcare organisations must address these components to gain physician acceptance.43 44 137 They can optimise Casemix system use, improving patient care and results.137

Principal findings

Findings of Exploratory Factor Analysis (EFA)

The pilot test data was analysed using EFA, which helps researchers understand complex datasets and discover observed variable correlations.55 56 97 EFA reduces variable dimensions by identifying common patterns, shaping fundamental factors that influence observable variables and grouping related variables.122 126 138 It simplifies model design by computing factor loadings, which indicate the intensity and direction of factor-observable variable interactions. EFA also finds underlying components in a dataset, while CFA analyses and confirms an EFA-proposed factor structure.55 56 97

All structures underwent KMO and Bartlett’s sphericity tests, with all structures having KMO values over 0.6.5556 105107 The scree plot, part of EFA, was used to count components and found nine constructs on 42 items.5556 105107 The study found that one construct should now have two parts, mainly due to demographic changes, particularly socioeconomic status and education. Component 1 explained 14.115% of construct variance, while component 9 explained 6.610%. All constructs had 84.07% total variance Explained (TVE), exceeding the minimum threshold of 60%.55 56 60 112 129

The EFA discovered nine components, including O1-O9 for organisational factors.43 45 50 139 41 of 42 items had factor loadings above 0.6, requiring item O1 to be eliminated.53 55 56 97 122 Only organisational factors (O) had nine items reduced to eight following extraction. The remaining seven constructs had only one component and no additional components, resembling HOT-Fit and TAM framework organisational constructs.

The study stresses tool dependability and internal consistency, using markers such as Cronbach’s alpha (α), person reliability, person measure and valid responses.133 140 A Cronbach’s alpha coefficient of 0.7 or above is acceptable in social science and other studies.53 138 141 142 Internal reliability is measured by how well-selected items measure the same idea.535697 98 133 143 The researcher reordered questionnaire items for the field investigation, and CFA authenticated and confirmed all eight constructs on field data, which is elaborated further in the next Subsection 4.1.2.

Findings of Confirmatory Factor Analysis (CFA)

Once the pilot data was assessed and the EFA was commenced, the final questionnaire will be used in the quantitative field study. Eventually, another procedure will be conducted to validate the questionnaire, also known as CFA, based on the field study data. The CFA will validate the instrument’s convergent, construct and discriminant validity. Unidimensionality, composite reliability and normality evaluations are also needed to reveal whether the instrument’s items are valid.535697 Therefore, the findings of this study demonstrate that the quantitative instrument has been validated and proven reliable for assessing medical practitioners’ intention to use and accept the Casemix system within the context of THIS. Using EFA and CFA is imperative for ensuring the instrument’s validity, reliability and trustworthiness.535697

By using EFA, the organisational factors (O) emerged into two components. The organisational factors (O) construct was renamed as organisational characteristics (ORG) in the measurement model, and the newly emerged components were named organisational structure (STR) and organisational environment (ENV). Measurement models refer to the implicit or explicit models that relate the latent variable to its indicators.55 56 97 The organisational characteristics (ORG) construct is assessed as a second-order construct due to the emerged components. When dealing with a complex framework, researchers can choose to do the CFA individually for each second-order construct, and then followed by Pooled-CFA, through item parcelling or straight away employ Pooled-CFA.55 56 The use of Pooled-CFA is beneficial because of its improved efficiency, effectiveness and ability to address identification difficulties.55 56 However, although there are many constructs in this study, this measurement model only includes one second-order construct, which is the (ORG) construct with two emerged components. The other seven constructs are made up exclusively of first-order constructs, each consisting of a maximum of five items. Therefore, a direct Pooled-CFA was employed.55 56

This study uses CFA to validate factor loading and assessment in a theory or model.535697 CFA is a form of structural equation modelling that makes assumptions and expectations about the number of factors and which factor theories or models best suit prior theory.535697 According to Baharum et al in their few studies, they measured success factors in newly graduated nurses’ adaptation and validation procedures.129 144 145 Likewise, for example, CFA also allows academics to test financial literacy indicators and measurement models, ensuring that a proper measuring model helps researchers interpret their data as elaborated in a few studies.146148

Validity, unidimensionality and reliability were necessary for all latent construct assessment models.53 55 56 97 122 The latent construct measurement model needed convergent, construct and discriminant validity.53 55 56 97 122 Convergent validity is assessed using the average variance extracted (AVE) statistic, while construct validity is determined by measurement model fitness indicators.5456 Composite reliability (CR) was used to calculate instrument reliability since it was better than Cronbach’s alpha.5456133

Unidimensionality is a set of variables that can be explained by one construct.79 Unidimensionality is achieved when all construct-specific measuring items have acceptable factor loading.5456 Convergent validity is a group of indicators that are considered to measure a construct.545697 135 Convergent validity is achieved when the concept’s AVE is more than 0.5, and the highest AVE for all structures was 0.857.53 56 97 136 Normality assessment was conducted on each item evaluating the construct’s distributional normality, with skewness values within the usual range (–1.5 to 1.5).56 97 The instrument’s data distribution met the normality condition.

Construct validity is attained when all model fitness indices meet the criteria, using absolute, incremental and parsimonious fit indices.55 56 97 The instrument met all three fitness indices, confirming the absolute fit index with RMSEA=0.054 (aim<0.1), achieving the incremental fit index category by obtaining a CFI value above 0.90 and yielding a parsimonious fit index of 2.014 (aim<5.0).55 56 97

Discriminant validity was tested to ensure no redundant constructs were found in the model.55 56 136 The model obtained discriminant validity since each construct’s square root of average variance extracted (AVE) is bigger than its correlation value with other constructs.55 56 136 The summary discriminant validity index showed all constructs met discriminant validity.

The instrument’s composite reliability exceeded 0.6 for all structures, with the environment component having the lowest CR (0.903) and the information quality construct having the highest (0.954).55 56 136 Calculating model reliability with composite reliability (CR).55 56 97 Acceptable CR is 0.6–0.7.55 56 97 As shown in table 1, the instrument’s composite reliability exceeded 0.6 for all constructs. The environment component (ENV) had the lowest CR (0.903), while information quality had the highest (0.954). Thus, this instrument’s composite reliability is achieved.

Therefore, all necessary procedures to determine validity, reliability and normalcy were conducted, and no items were excluded. As a result, the total number of items remained at 41. Construct, convergent, discriminant validities and composite reliability have all been attained. All things satisfied the criteria of normality.

Strengths and weaknesses of the study

There are various ways in which this study could benefit the medical community and policymakers.149 150 The research assesses important success elements that affect physicians’ adoption of the Casemix system in hospitals that have a THIS. Policymakers and hospital administrators may find it easier to pinpoint the critical elements influencing the Casemix system’s effective deployment with the aid of the study’s findings.151 To successfully implement clinical pathway/case management programmes, policymakers may find the study to help understand the significance of ongoing clinician support and acceptance, top management leadership and support, and a committed team of case managers, nurses and paramedical professionals.151 152 Policymakers can potentially use the findings to impact admissions decisions, thereby increasing clinical practice openness.152154

Strengths and limitations exist in this research. One of the strengths of the study was that it employed a sequential explanatory mixed-method approach to investigate the CSFs and acceptance of the Casemix system among medical practitioners in THIS.58 155 156 The findings revealed that there might be unnoticed CSFs in the quantitative phase, suggesting the need for a qualitative method to identify more CSFs, perceptions and challenges/barriers. Quantitative data support hypothesised associations, but qualitative data provide in-depth data to supplement quantitative conclusions.157 The mixed-method approach is expected to improve research design and yield more valid results.

Additionally, another strength of this study is that it uses a strict methodological approach to instrument development and validation. It uses both EFA with pilot test data and CFA using field data, which makes the instrument used for data collection more reliable and valid. Many statistical tests were used to make sure the instrument worked well and the analysis was accurate. These included the KMO measure, Bartlett’s test of sphericity, systematic deletion of items based on factor loadings, Cronbach’s alpha and different validity tests such as unidimensionality, construct validity, convergent validity and discriminant validity.5556 105107

Although the study had a large sample size, it was only conducted in five selected hospitals in Malaysia. Therefore, the findings may not accurately represent all THIS hospitals in the country or other healthcare systems. Other professional positions, including paramedics, medical record officers, information technology officers and finance officers, are not included in this study since their involvement and level of understanding in the Casemix system are not similar to that of medical practitioners, despite being relatively involved in the Casemix system. Hence, this may limit the generalisability of the findings could be a potential weakness of the study. The study’s findings are likely to be distinctive/unique to the healthcare setting in Malaysia and may or may not be directly transferable to other nations or healthcare systems that have distinct sociocultural, organisational or technological characteristics. While this study’s findings are rooted in Malaysia’s healthcare setting, where the Casemix system and THIS are prevalent, their applicability to other countries or healthcare systems with different sociocultural, organisational or technological characteristics should be carefully considered. Despite this, there are potential avenues through which the insights gained from this research could benefit other nations or healthcare systems. For example, the principles of efficiency and effectiveness in healthcare management highlighted in this study could be adapted and implemented in various settings. Additionally, the lessons learnt from the challenges faced in Malaysia’s healthcare system could serve as valuable guidance for other countries looking to improve their systems.

Strengths and weaknesses concerning other studies

Compared with previous studies, this research contributes to the field by providing a validated instrument tailored to assess the acceptance of the Casemix system within the THIS environment. Prior literature has examined various aspects of Casemix implementation in Malaysia as well as in other countries. However, no one has investigated Casemix in THIS or even in HIS. Thus, this study offers a comprehensive evaluation tool that addresses critical success factors influencing medical doctors’ acceptance, filling a significant research gap. Given the absence of prior research in this area, the newly created quantitative tool would be advantageous in achieving the study objectives and serve as a point of reference for future investigations.

However, previous literature by Beth Reid describes the importance of developing Casemix-based hospital information system management.33 The Casemix-based hospital information system is a comprehensive approach to healthcare management that involves estimating costs per diagnosis-related group (DRG), building a Casemix-based system and addressing organisational design and education issues for successful implementation.33 It is crucial to provide Casemix reports to hospital staff and clinicians to identify errors in data. Improving the quality of data is essential for both hospitals and universities. To ensure the credibility of the HIS, it must tap into decentralised databases to ensure common input data for each patient’s diseases and procedures.33 Sharing data is beneficial for clinicians as it allows them to avoid investing time and effort in ensuring database accuracy to discover that the data used for Casemix activities, such as funding, is obtained from the medical record.40 This approach is essential for ensuring the accuracy and efficiency of healthcare management.33

Additionally, a study by Saizan showed that THIS hospital showed the lowest Casemix performance in terms of accuracy of the main diagnosis, the completeness of other diagnoses, and the coding of main and other diagnoses.16 This article outlines two themes with three subthemes, each theme based on why the performance is the lowest. These two themes are the poor commitment of clinicians and obstacles in the work process. Furthermore, another study revealed that one THIS hospital in Malaysia had the lowest Casemix performance in terms of main diagnosis accuracy, other diagnosis completeness, and main diagnosis and other diagnostic coding accuracy.16 This article presents two overarching themes, each consisting of three subthemes based on the qualitative, in-depth interview findings. These themes are centred around the underlying reasons behind the lowest Casemix performance. The two main themes identified are the lack of dedication among professionals and the challenges encountered in the workflow.

Meaning of the study: possible explanations and implications

The validated and reliable instrument developed in this study holds implications for clinicians, policymakers and healthcare organisations aiming to optimise Casemix system implementation within HIS. Identifying critical factors influencing acceptance, such as system, information and service quality, is imperative to meet study objectives. Organisational characteristics such as environment and structure, as well as human factors such as perceived ease of use and perceived usefulness, the findings offer actionable insights for enhancing system adoption, utilisation and success. Policymakers and hospital administrators can use these findings to streamline Casemix deployment strategies, improving patient care outcomes and operational efficiency within the THIS.

First, while the specific details of the findings may not directly translate to other contexts, the underlying principles and methodologies employed in this study can serve as a valuable template for researchers in different settings. By adapting and contextualising the research methods and instruments used in this study, researchers in other countries can conduct similar investigations tailored to their healthcare environments.158 159

Second, the identification and evaluation of critical success factors for implementing healthcare information systems, such as the Casemix system, are universal challenges healthcare organisations face worldwide.33 158 160 Because of this, the conceptual framework and analytical methods created in this study can help us understand what makes people accept and use these kinds of systems in different situations. Researchers and policymakers in other countries can leverage these insights to inform their strategies for implementing and optimising healthcare information systems.

Additionally, while the contexts and details of the Casemix system and THIS may vary across different countries, the broader goals of improving resource allocation, clinical decision-making and quality of care are shared objectives across healthcare systems globally. Therefore, the findings of this study, particularly regarding the factors influencing system acceptance and success, have the potential to resonate with stakeholders in other countries who are working towards similar goals.151 161 162

Overall, while recognising the contextual specificity of the study’s findings, there is potential for the insights generated to contribute to the broader body of knowledge on healthcare information systems and inform practices in other countries or healthcare settings with distinct characteristics. Through collaboration and adaptation, the lessons learnt from this research can be extrapolated and applied to diverse healthcare contexts, ultimately contributing to advancing healthcare delivery worldwide.33 158 160 By sharing best practices and lessons learnt, healthcare systems around the world can benefit from the findings of this study and improve their information systems. This collaborative approach can lead to more efficient and effective healthcare delivery on a global scale.

Unanswered questions and future research

The current study proposes employing this instrument in future research, broadening the target population to include more professional occupations and increasing the sample size for more robust results. The novelty of this research lies in its comprehensive analysis of the direct and indirect effects of these parameters on user acceptance of implementing Casemix within THIS environment. SEM was employed to investigate the proposed model. Apart from that, mediating effects have been examined in this study involving a few critical constructs, such as PEOU, PU and ITU, using similar analysis methods. Additionally, more information on moderating characteristics, including age, gender, professional positions, degree of education, years of experience in MOH Malaysia and current THIS hospital and Casemix system knowledge, could improve the instrument. These moderating effects were examined using SEM as well.

The innovation of this study is that it examines the CSFs that influence the acceptance of the Casemix system in the THIS environment, specifically in MOH hospitals in Malaysia. The immediate findings have clear significance for healthcare organisations and policymakers in Malaysia, and even globally. However, the more significant implications for readers in other countries are also relevant. First and foremost, recognising CSF in implementing the Casemix system provides valuable information that can be applied to healthcare systems, especially those equipped with THIS facility universally. Gaining insight into these aspects can provide valuable strategic decision-making guidance in other nations seeking to implement or improve similar systems within their healthcare infrastructure.

Furthermore, the study uses a methodological approach that involves the use of a mixed-methods approach. The quantitative phase, elaborated on in this article, employs a reliable quantitative instrument that validates exploratory and confirmatory factor analyses and reliability testing. Moreover, semi-structured, in-depth interviews were conducted with the Deputy Directors representing the top management and the CSCs of 5 participating hospitals. Hence, these mixed-methods studies provide a strong foundation for evaluating the adoption of the Casemix system within healthcare information systems. Readers from different countries might use and modify these approaches to conduct comparable investigations in their specific circumstances, enhancing the comprehension of healthcare informatics worldwide.

Moreover, the study highlights the significance of interdisciplinary collaboration among healthcare practitioners, technology specialists and policymakers in facilitating the practical application of the Casemix system as one of the clinical and costing modules essential in healthcare settings, especially in facilities equipped with HIS. This interdisciplinary approach to tackling issues in healthcare informatics is generally applicable and can be implemented in various countries and healthcare systems.

To summarise, this study’s immediate findings may address the CSF of the Casemix system implementation within THIS of the healthcare system in Malaysia. However, its broader significance lies in providing valuable insights, methodological frameworks and interdisciplinary approaches that can be applied globally to adopt the Casemix system within the realm of the HIS in other countries, and it is not only applicable locally in the Malaysian setting.

Conclusion

In summary, this research has comprehensively evaluated the fundamental principles outlined in the conceptual framework. Various methodological approaches, including content validity, criterion validity, translation, pre-testing for face validity, pilot testing using EFA and field study employing CFA, have been employed to assess the validity of the items.12 43 44 50 61 The EFA analysis computed KMO, Bartlett’s test for sphericity and Cronbach’s alpha values, all meeting the criteria for sample adequacy, sphericity and internal reliability.535697 Additionally, the CFA analysis tested for unidimensionality, construct validity, convergent validity, discriminant validity, composite reliability and normality, further confirming the validity and reliability of the instrument used to evaluate critical success factors and the acceptance of the Casemix system within the THIS context.535697

Consequently, this validated instrument holds promise for future quantitative analyses, including covariance-based structural equation modeling (CB-SEM) or variance-based structural equation modeling (VB-SEM). In this study, CB-SEM, in conjunction with SPSS-AMOS V.24.0, was used to explore the direct, indirect, mediating and moderating effects among the constructs outlined in the conceptual framework. The findings from these quantitative analyses will be presented in forthcoming articles, providing further insights into the Casemix system’s applicability within the current healthcare landscape. Moreover, the instrument’s demonstrated statistical reliability and validity position is a valuable tool for future research endeavours concerning the Casemix system in the THIS context, addressing an existing research gap. With the establishment of the instrument’s normality, validity and reliability, it can now be considered operational and validated for use in subsequent studies. This research holds the potential to enhance our understanding of the critical success factors and acceptance of the Casemix system, thereby facilitating its improved implementation within the THIS setting. Moving forward, the instrument will be instrumental in conducting further research initiatives to assess the adoption and effectiveness of the Casemix system in THIS environment, addressing a current scarcity of literature.

supplementary material

10.1136/bmjopen-2023-082547 online supplemental file 1

10.1136/bmjopen-2023-082547 online supplemental file 2

10.1136/bmjopen-2023-082547 online supplemental file 3

10.1136/bmjopen-2023-082547 online supplemental file 4

10.1136/bmjopen-2023-082547 online supplemental file 5

10.1136/bmjopen-2023-082547 online supplemental file 6

10.1136/bmjopen-2023-082547 online supplemental file 7

10.1136/bmjopen-2023-082547 online supplemental file 8

10.1136/bmjopen-2023-082547 online supplemental file 9

Acknowledgements

In recognition of their involvement and contributions to this study, the authors would like to express their gratitude to the respondents. In addition, the authors would like to express their gratitude to all content and criterion validators of this study: Dr. Fawzi Zaidan and Dr. Nuratfina from the Hospital Financing (Casemix) Unit of the Ministry of Health Malaysia, and Prof. Dr. Zainudin Awang from Universiti Sultan Zainal Abidin. Their remarks and recommendations made a significant contribution to the advancement of this instrument.We express our appreciation to the Casemix System Coordinators, as well as the Hospital and the Deputy Directors from Hospitals W, E, S, N, and EM, for their great collaboration in distributing the questionnaire link and for actively engaging in this study.

Additionally, for their suggestions on improving this paper, the authors would like to express their gratitude to the reviewers. Finally, we also want to express our appreciation to Associate Professor Ts. Dr. Mohd Sharizal for proofreading this article.

Data availability statement

No data are available.

Review Process File
24 8 2024

Funding: The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

Prepublication history and additional supplemental material for this paper are available online. To view these files, please visit the journal online (https://doi.org/10.1136/bmjopen-2023-082547).

Provenance and peer review: Not commissioned; externally peer reviewed.

Patient consent for publication: Consent obtained directly from patient(s).

Ethics approval: This study was approved by both the Medical Research Ethics Committee from the Ministry of Health and the Medical Research Ethics Committee from the Faculty of Medicine, Universiti Kebangsaan Malaysia with the reference numbers: NMRR ID-22-02621-DKX and JEP-2022-777 respectively. Informed consent was obtained from all participants through the Google form with a statement that all data would be confidential. All methods were carried out under the ethical standards of the institutional research committee and conducted according to the Declaration of Helsinki. All methods were performed based on the relevant guidelines and regulations. This study was not funded by any grants. The authors declare there were no conflicts of interest concerning this article.

Patient and public involvement: Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
==== Refs
References

1 Braithwaite J Westbrook J Coiera E et al A systems science perspective on the capacity for change in public hospitals Isr J Health Policy Res 2017 6 16 10.1186/s13584-017-0143-6 28352457
2 Medical Development Division MOH The MOH casemix system which is called the MalaysianDRG is now in its 6th year 2016 Available https://www.facebook.com/medicaldevelopment/posts/the-moh-casemix-system-which-is-called-the-malaysiandrg-is-now-in-its-6th-year-o/641387939355450/
3 van Boekholt TA Duits AJ Busari JO Health care transformation in a resource-limited environment: exploring the determinants of a good climate for change J Multidiscip Healthc 2019 12 173 82 10.2147/JMDH.S194180 30881009
4 Putera KAS Jihan Noris N Lean healthcare implementation in malaysian specialist hospitals: challenges and performance evaluation Jummec 2022 25
5 Ministry of Health Malaysia Telemedicine flagship application: malaysia’ S telemedicine blueprint: leading healthcare into the information age Telemed Bluepr 1997 1 47
6 Grantham S Redesigning primary health care teams for population health and quality improvement Agency Healthc Res Qual 2017 17 1 12
7 Borzekowski DLG Considering children and health literacy: A theoretical approach Pediatrics 2009 124 S282 8 10.1542/peds.2009-1162D 19861482
8 Vaganova E Ishchuk T Zemtsov A et al Health information systems: background and trends of development worldwide and in russia 10th International Conference on Health Informatics Porto, Portugal 2017 424 8 10.5220/0006244504240428
9 Masrom M Rahimly A Universiti Teknologi Malaysia et al Overview of data security issues in hospital information systems PAJAIS 2015 51 66 10.17705/1pais.07404
10 Heeks R Health information systems: failure, success and improvisation Int J Med Inform 2006 75 125 37 10.1016/j.ijmedinf.2005.07.024 16112893
11 Tachinardi U Gutierrez MA Moura L et al Integrating hospital information systems The challenges and advantages of (Re-) starting now
12 Yusof MM Paul RJ Stergioulas LK Towards a framework for health information systems Proc Annu Hawaii Int Conf Syst Sci 2006 5 1 10 10.1109/HICSS.2006.491
13 Ismail NI Abdullah NH Implementation and acceptance of hospital information system 1st FPTP postgrad semin 2013 2013 1 5
14 Merican I bin Yon R Health care reform and changes: the malaysian experience Asia Pac J Public Health 2002 14 17 22 10.1177/101053950201400105 12597513
15 Abdul Hamid NB ICT planning, implementation and procurement in development projects 2016
16 Saizan S Jaudin R Nor MZM et al The importance of clinical documentation in the malaysiandrg casemix system: a sequential explanatory mixed-method study of ministry of health hospitals in malaysia Malays J Med Heal Sci 2021 17 50 6
17 Ismail NI Abdullah NH Shamsudin A et al Implementation differences of hospital information system (HIS) in malaysian public hospitals IJSSH 2013 115 20 10.7763/IJSSH.2013.V3.208
18 Alipour J Mehdipour Y Karimi A Factors affecting acceptance of hospital information systems in public hospitals of zahedan university of medical sciences: A cross-sectional study J Med Life 2019 12 403 10 10.25122/jml-2019-0064 32025259
19 Desi Hertin R Ismael Al-Sanjary O Performance of hospital information system in malaysianpublic hospital: a review IJET 2018 7 24 10.14419/ijet.v7i4.11.20682
20 Sulaiman H Wickramasinghe N Assimilating healthcare information systems in a malaysian hospital CAIS 2014 34 1291 318 10.17705/1CAIS.03477
21 Lee HW Ramayah T Zakaria N External factors in hospital information system (HIS) adoption model: a case on malaysia J Med Syst 2012 36 2129 40 10.1007/s10916-011-9675-4 21384267
22 Ismail NI Abdullah NH Shamsuddin A Adoption of hospital information system (HIS) in malaysian public hospitals Proc Soc Behav Sci 2015 172 336 43 10.1016/j.sbspro.2015.01.373
23 Mustaffa HR Aljunid SM Ahmed Z et al Web based application for casemix grouper (online casemix grouper) BMC Public Health 2014 14 10.1186/1471-2458-14-S1-P15
24 Sulong S Nur AM Sapar A et al Casemix groups and severity levels of patients managed in HUKM using IR-DRG classification system Malays J Public Heal 2005 5 5
25 Ismail A Taufik Jamil A Fareed A et al The implementation of hospital information system (HIS) in tertiary hospitals in malaysia: a qualitative study Malays J Public Health Med 2010 10
26 Ahmadi H Nilashi M Ibrahim O et al Prioritizing critical factors to successful adoption of total hospital information system J Soft Comput Decis Support Syst 2015 2 6 16 Available http://www.jscdss.com
27 Mohd H Mastura S Mohamad S Acceptance model of electronic medical record J Adv Inf Manag Stud 2005 2 75 92 Available http://repo.uum.edu.my/2246/
28 Hassan R Implementation of total hospital information system (THIS) in malaysian public hospitals: challenges and future prospects Int J Bus Soc Res (IJBSR) 2012 2 33 41 Available 10.18533/ijbsr.v2i2.189
29 Ismail NI Abdullah NH Hospital information system (HIS) implementation in malaysian public hospitals Inf 2016 19 2833 8
30 Zafirah SA Nur AM Puteh SEW et al Potential loss of revenue due to errors in clinical coding during the implementation of the malaysia diagnosis related group (MY-DRG®) casemix system in a teaching hospital in malaysia BMC Health Serv Res 2018 18 38 10.1186/s12913-018-2843-1 29370785
31 Sukma P Kusumadewi S Linda R International conference on recent trends in physics 2016 (icrtp2016) J Phys: Conf Ser 2016 755 011001 10.1088/1742-6596/755/1/011001
32 Roger France FH Case mix use in 25 countries: A migration success but international comparisons failure Int J Med Inform 2003 70 215 9 10.1016/S1386-5056(03)00044-3 12909172
33 Reid B Casemix systems and their applications Stud Heal Technol Inf 2013 193 316 31 Available https://pubmed.ncbi.nlm.nih.gov/24018526/
34 Heslop L Diers D Gardner B et al Using clinical data for nursing research and management in health services Contemp Nurse 2004 17 8 18 10.5172/conu.17.1-2.8 17929732
35 Heslop L Activity-based funding for safety and quality: A policy discussion of issues and directions for nursing-focused health services outcomes research Int J Nurs Pract 2019 25 e12775 10.1111/ijn.12775 31414554
36 Kasra K Nur AM Aljunid SM The impact of casemix system on quality of patient care in a class b hospital in west sumatera province, indonesia BMC Health Serv Res 2012 12 10.1186/1472-6963-12-S1-O9
37 Hamzah Al-Junid SM Aljunid SM Ahmed Z et al Development of UNU-casemix grouper standard integration tool kit for hospital information system (HIS) in selected hospitals in malaysia and indonesia BMC Public Health 2014 14 10.1186/1471-2458-14-S1-P17
38 Medical Development Division MOH Strategic framework medical programme of the 2021–2025 Ministry of Health Malaysia 2020 Available https://www.moh.gov.my/moh/resources/Pelan_Strategik_KKM.pdf
39 Medical Development Division MOH MalaysianDRG Findings 2017 - 2018: National Base Rate, Demographic and Quality Indicator - Key Findings 2020 Available https://www.moh.gov.my/moh/resources/Penerbitan/Casemix/Garis Panduan/Casemix_%0AInfographic-2017_2018_.pdf
40 Medical Development Division MOH Casemix malaysiandrg way forward 2021 Available https://www.coursehero.com/file/162500649/2-CSMOT-way-forwardpdf/
41 Ali Jadoo SA Aljunid SM Nur AM et al Development of MY-DRG casemix pharmacy service weights in UKM medical centre in malaysia DARU J Pharm Sci 2015 23 1 8 10.1186/s40199-014-0075-4
42 Fedorowicz J Hospital information systems: are we ready for case mix applications? Health Care Manage Rev 1983 8 33 41 10.1097/00004010-198300840-00005 6417053
43 Venkatesh V Morris MG Davis GB et al User acceptance of information technology: toward A unified view MIS Q 2003 27 425 10.2307/30036540
44 Venkatesh V Davis FD A theoretical extension of the technology acceptance model: four longitudinal field studies Manage Sci 2000 46 186 204 10.1287/mnsc.46.2.186.11926
45 Venkatesh V Bala H Technology acceptance model 3 and a research agenda on interventions Decis Sci 2008 39 273 315 10.1111/j.1540-5915.2008.00192.x
46 Po-An Hsieh JJ Wang W Explaining employees’ extended use of complex information systems Eur J Inf Syst 2007 16 216 27 10.1057/palgrave.ejis.3000663
47 Nadri H Rahimi B Lotfnezhad Afshar H et al Factors affecting acceptance of hospital information systems based on extended technology acceptance model: A case study in three paraclinical departments Appl Clin Inform 2018 9 238 47 10.1055/s-0038-1641595 29618139
48 DeLone WH McLean ER The delone and mclean model of information systems success: A ten-year update J Manag Inf Syst 2003 19 9 30 10.1080/07421222.2003.11045748
49 DeLone WH McLean ER Measuring e-commerce success: applying the delone & mclean information systems success model Int J Electron Comm 2004 9 31 47 10.1080/10864415.2004.11044317
50 Yusof MM Papazafeiropoulou A Paul RJ et al Investigating evaluation frameworks for health information systems Int J Med Inform 2008 77 377 85 10.1016/j.ijmedinf.2007.08.004 17904898
51 Wong WT Norman Huang NT The effects of E-learning system service quality and users’ acceptance on organizational learning the effects of E-learning system service quality and users Accept Organ Learn Int J Bus Info 2011 Available https://www.researchgate.net/publication/268304721
52 Khechine H Lakhal S Pascot D et al UTAUT model for blended learning: the role of gender and age in the intention to use webinars IJELL 2014 10 033 52 10.28945/1994
53 Awang Z Research methodology and data analysis 2nd edn UiTM Press 2012 334
54 Awang Z A handbook on SEM-analyzing the SEM structural model A handbook on SEM 2015 71 86
55 Awang Z Lim SH Zainudin NFS Pendekatan mudah SEM 2018 165
56 Awang Z Afthanorhan A Lim SH et al SEM made simple 2.0: A gentle approach of structural equation modelling Kuala Nerus: Universiti Sultan Zainal Abidin 2023 1 178
57 Sekaran U Research methods for business: a skill building approach 4th edn John Wiley & Sons, Inc 2003 466 Available http://www.wiley.com/college
58 Creswell JW Clark VLP Research design: qualitative, quantitative and mixed methods approaches 3rd edn Thousand Oaks, California Sage publications 2009 1 403
59 Hoque A Awang Z Siddiqui BA Upshot of generation ‘Z’entrepreneurs’ E-lifestyle on bangladeshi SME performance in the digital era Int J Entrep Small Mediu Enterp 2018 5 97 118
60 Hoque A Siddiqui BA Awang Z et al Exploratory factor analysis of entrepreneurial orientation in the context of bangladeshi small and medium enterprises (SMES) Eur J Manag Mark Stud 2018 3 81 94 Available 10.5281/zenodo.1292331
61 Erlirianto LM Ali AHN Herdiyanti A The implementation of the human, organization, and technology–fit (hot–fit) framework to evaluate the electronic medical record (emr) system in a hospital Procedia Comput Sci 2015 72 580 7 10.1016/j.procs.2015.12.166
62 Parasuraman A Zeithaml VA Berry LL SERVQUAL: A multiple-item scale for measuring consumer perceptions of service quality J Retail 1988 64 12 40
63 Proctor EK Landsverk J Aarons G et al Implementation research in mental health services: an emerging science with conceptual, methodological, and training challenges Adm Policy Ment Health 2009 36 24 34 10.1007/s10488-008-0197-4 19104929
64 Proctor E Silmere H Raghavan R et al Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda Adm Policy Ment Health 2011 38 65 76 10.1007/s10488-010-0319-7 20957426
65 Strong DM Lee YW Wang RY Data quality in context Commun ACM 1997 40 103 10 10.1145/253769.253804
66 Davoudizadeh R Hosseini Seno SA Department of Management, Ferdowsi University of Mashhad, Mashhad, Iran et al Analyzing advantages and benefits of information technologies in organizations sjis 2020 2 7 19 10.29252/sjis.2.1.7
67 Johnson EC Karlay JS Impact of service quality on customer satisfaction. Case study: Liberia revenue authority University of Gavle 2018
68 Cao Q Jones DR Sheng H Contained nomadic information environments: technology, organization, and environment influences on adoption of hospital RFID patient tracking Inf & Manag 2014 51 225 39 10.1016/j.im.2013.11.007
69 Hameed MA Counsell S Swift S A meta-analysis of relationships between organizational characteristics and IT innovation adoption in organizations Inf & Manag 2012 49 218 32 10.1016/j.im.2012.05.002
70 Abdulrahman MD Subramanian N Barriers in implementing reverse logistics in chinese manufacturing sectors: an empirical analysis. In: 23rd annual POMS conference 2012
71 Zayyad MA Toycan M Factors affecting sustainable adoption of e-health technology in developing countries: an exploratory survey of nigerian hospitals from the perspective of healthcare professionals PeerJ 2018 6 e4436 10.7717/peerj.4436 29507830
72 Lian J-W Yen DC Wang Y-T An exploratory study to understand the critical factors affecting the decision to adopt cloud computing in taiwan hospital Int J Inf Manage 2014 34 28 36 10.1016/j.ijinfomgt.2013.09.004
73 Armstrong CP Sambamurthy V Information technology assimilation in firms: the influence of senior leadership and IT infrastructures Inf Syst Res 1999 10 304 27 10.1287/isre.10.4.304
74 Meri A Hasan MK Dauwed M et al Organizational and behavioral attributes’ roles in adopting cloud services: an empirical study in the healthcare industry PLoS ONE 2023 18 e0290654 10.1371/journal.pone.0290654 37624836
75 Committee on Diagnostic Error in Health Care Board on Health Care Services Institute of Medicine Improving Diagnosis in Health Care Washington, DC 2015 1 472 Available http://www.nap.edu/catalog/21794
76 Leso BH Cortimiglia MN Ghezzi A The contribution of organizational culture, structure, and leadership factors in the digital transformation of smes: a mixed-methods approach Cogn Technol Work 2023 25 151 79 10.1007/s10111-022-00714-2 36118918
77 Kelly S Kaye SA Oviedo-Trespalacios O What factors contribute to the acceptance of artificial intelligence? A systematic review Tele Inform 2023 77 101925 10.1016/j.tele.2022.101925
78 Davis FD Bagozzi RP Warshaw PR User acceptance of computer technology: A comparison of two theoretical models Manage Sci 1989 35 982 1003 10.1287/mnsc.35.8.982
79 Davis FD Perceived usefulness, perceived ease of use, and user acceptance of information technology MIS Q 1989 13 319 10.2307/249008
80 Davies FD Venkatesh V Measuring user acceptance of emerging information technologies: an assessment of possible method biases Twenty-Eighth Annual Hawaii International Conference on System Sciences Wailea, HI, USA 1995 729 36 10.1109/HICSS.1995.375675 Available 10.1109/HICSS.1995.375675
81 Davis FD Information technology introduction 2013 13 319 40
82 Davis FD Venkatesh V A critical assessment of potential measurement biases in the technology acceptance model: three experiments Int J Hum Comput Stud 1996 45 19 45 10.1006/ijhc.1996.0040
83 Carmines E Zeller R Reliability and validity assessment 2455 Teller Road,Thousand Oaks, CA SAGE Publications Inc 1979 Available https://methods.sagepub.com/book/reliability-and-validity-assessment
84 Davis LL Instrument review: getting the most from a panel of experts Appl Nurs Res 1992 5 194 7 10.1016/S0897-1897(05)80008-4
85 Polit DF Beck CT The content validity index: are you sure you know what’s being reported? critique and recommendations Res Nurs Health 2006 29 489 97 10.1002/nur.20147 16977646
86 Polit DF Beck CT Owen SV Is the CVI an acceptable indicator of content validity? appraisal and recommendations Res Nurs Health 2007 30 459 67 10.1002/nur.20199 17654487
87 Hadie SNH Hassan A Ismail ZIM et al Anatomy education environment measurement inventory: A valid tool to measure the anatomy learning environment Anat Sci Educ 2017 10 423 32 10.1002/ase.1683 28135037
88 Lau ASY Yusoff MSB Lee YY et al Development and validation of A chinese translated questionnaire: Asingle simultaneous tool for assessing gastrointestinal and upper respiratory tract related illnesses in pre-school children J Taibah Univ Med Sci 2018 13 135 41 10.1016/j.jtumed.2017.11.003 31435316
89 Ozair MM Baharuddin KA Mohamed SA et al Development and validation of the knowledge and clinical reasoning of acute asthma management in emergency department (K-CRAMED) EIMJ 2017 9 1 17 10.21315/eimj2017.9.2.1
90 Mohamad Marzuki MF Yaacob NA Yaacob NM Translation, cross-cultural adaptation, and validation of the malay version of the system usability scale questionnaire for the assessment of mobile apps JMIR Hum Factors 2018 5 e10308 10.2196/10308 29759955
91 Lynn MR Determination and quantification of content validity Nurs Res 1986 35 382 5 10.1097/00006199-198611000-00017 3640358
92 Yusoff MSB Department of Medical Education, School of Medical Sciences, Universiti Sains Malaysia, Malaysia ABC of content validation and content validity index calculation EIMJ 2019 11 49 54 10.21315/eimj2019.11.2.6
93 Armstrong TS Cohen MZ Eriksen L et al Content validity of self-report measurement instruments: an illustration from the development of the brain tumor module of the M.D. anderson symptom inventory Oncol Nurs Forum 2005 32 669 76 10.1188/05.ONF.669-676 15897941
94 Yoo S Lim K Jung SY et al Examining the adoption and implementation of behavioral electronic health records by healthcare professionals based on the clinical adoption framework BMC Med Inform Decis Mak 2022 22 1 9 10.1186/s12911-022-01959-7 34983500
95 Venkatesh V Thong JYL Xu X Consumer acceptance and use of information technology: extending the unified theory of acceptance and use of technology MIS Q 2012 36 157 10.2307/41410412
96 Petter S DeLone W McLean E Measuring information systems success: models, dimensions, measures, and interrelationships Eur J Inf Syst 2008 17 236 63 10.1057/ejis.2008.15
97 Hair JF Black WC Babin BJ et al Multivariate data analysis 2019 95 120 Available 10.13106/jafeb.2021.vol8.no2.0943
98 Hair JF Black WC Babin BJ et al Multivariate data analysis 7th edn 2010 1 767
99 Hair JF Howard MC Nitzl C Assessing measurement model quality in PLS-SEM using confirmatory composite analysis J Bus Res 2020 109 101 10 10.1016/j.jbusres.2019.11.069
100 Sakpal TV Sample size estimation in clinical trial Perspect Clin Res 2010 1 67 9 21829786
101 MacCallum RC Widaman KF Zhang S et al Sample size in factor analysis Psychol Methods 1999 4 84 99 10.1037//1082-989X.4.1.84
102 Hair JF Black WC Babin BJ et al Multivariate data analysis Prentice hall Upper Saddle River, NJ 1998
103 Alkhawaja MI Sobihah M Awang Z Exploring and developing an instrument for measuring system quality construct in the context of E-learning IJARBSS 2020 10 10.6007/IJARBSS/v10-i11/7953
104 Yahaya TAB Idris K Suandi T et al Adapting instruments and modifying statements: the confirmation method for the inventory and model for information sharing behavior using social media 105267/jmsl 2018 8 271 82 10.5267/j.msl.2018.4.021
105 Kaiser HF An index of factorial simplicity Psychometrika 1974 39 31 6 10.1007/BF02291575
106 Pallant J SPSS survival manual 6th edn Open University Press 2016 359
107 Bartlett MS TESTS of significance in factor analysis Br J Stat Psychol 1950 3 77 85 10.1111/j.2044-8317.1950.tb00285.x
108 Hopfe M Stucki G Marshall R et al Capturing patients’ needs in casemix: A systematic literature review on the value of adding functioning information in reimbursement systems BMC Health Serv Res 2016 16 40 10.1186/s12913-016-1277-x 26847062
109 Ali Jadoo SA Sulku SN Aljunid SM et al Validity and reliability analysis of knowledge of, attitude toward and practice of a case-mix questionnaire among turkish healthcare providers JHEOR 2014 2 96 107 10.36469/9891 37664081
110 Bujang MA Ghani PA Soelar SA et al Sample size guideline for exploratory factor analysis when using small sample: taking into considerations of different measurement scales 2012 International Conference on Statistics in Science, Business and Engineering (ICSSBE 2012) 2012 Langkawi 10.1109/ICSSBE.2012.6396605
111 Ding L Velicer WF Harlow LL Effects of estimation methods, number of indicators per factor, and improper solutions on structural equation modeling fit indices Struct Equ Modeling 1995 2 119 43 10.1080/10705519509540000
112 Fitriana N Hutagalung FD Awang Z et al Happiness at work: A cross-cultural validation of happiness at work scale PLoS ONE 2022 17 e0261617 10.1371/journal.pone.0261617 34986180
113 Tinsley HEA Tinsley DJ Uses of factor analysis in counseling psychology research J Couns Psychol 1987 34 414 24 10.1037//0022-0167.34.4.414
114 Boomsma A Hoogland JJ The robustness of LISREL modeling revisited 2001
115 Hoogland JJ Boomsma A Robustness studies in covariance structure modeling Sociol Methods Res 1998 26 329 67 10.1177/0049124198026003003
116 Bentler PM Chou CP Practical issues in structural modeling Sociol Methods Res 1987 16 78 117 10.1177/0049124187016001004
117 Nunnally JC Bernstein IH The assessment of reliability Psychom Theory 1994 3 248 92
118 Wang J Wang X Structural equation modeling Shewhart WA Wilks SS Applications using Mplus New Jersey John Wiley & Sons Available https://onlinelibrary.wiley.com/doi/book/10.1002/9781118356258
119 Nunnally JC Bernstein IH Psychometric theory 3rd edn New York, NY McGraw-Hill 1994
120 Afthanorhan A Awang Z Fazella S Perception of tourism impact and support tourism development in terengganu, malaysia Soc Sci (Basel) 2017 6 106 10.3390/socsci6030106
121 Awang Z Afthanorhan A Mohamad M et al An evaluation of measurement model for medical tourism research: the confirmatory factor analysis approach IJTP 2015 6 29 10.1504/IJTP.2015.075141
122 Shkeer AS Awang Z EXPLORING the items for measuring the marketing information system construct: an exploratory factor analysis IRMM 2019 9 87 97 10.32479/irmm.8622
123 Dani RM Mansor N Awang Z et al A confirmatory factor analysis of the fraud pentagon instruments for measurement of fraud in the context of asset misappropriation in malaysia 35 2022 9 70 9 10.18488/35.v9i2.3063
124 Mohamad M Afthanorhan A Awang Z et al Comparison between CB-SEM and PLS-SEM: testing and confirming the maqasid syariah quality of life measurement model TJSSR 2019 5 608 14 10.32861/jssr.53.608.614
125 Peterson J McGillis Hall L O’Brien-Pallas L et al Job satisfaction and intentions to leave of new nurses J Res Nurs 2011 16 536 48 10.1177/1744987111422423
126 Noor NM Aziz AA Mostapa MR et al Validation of the malay version of the inventory of functional status after childbirth questionnaire Biomed Res Int 2015 2015 972728 10.1155/2015/972728 25667932
127 Rahlin NA Awang Z Afthanorhan A et al The art of covariance based analysis in behaviour-based safety performance study using confirmatory factor analysis: evidence from SMES Int J Innov Creat Chang 2019 7 351 70
128 Ehido A Awang Z Abdul Halim B et al DEVELOPING items for measuring quality of work life among malaysian academics: an exploratory factor analysis procedure HSSR 2020 8 1295 309 10.18510/hssr.2020.83132
129 Baharum H Ismail A Awang Z et al The study adapted instruments based on confirmatory factor analysis (CFA) to validate measurement models of latent constructs Int J Environ Res Public Health 2023 20 10.3390/ijerph20042860
130 Awang Z Hanim Ahmad J Mohamed Zin N Modelling job satisfaction and work commitment among lecturers: a case of uitm kelantan 2010 241 55
131 Awang Z Wan Afthanorhan WMA Asri MAM Parametric and non parametric approach in structural equation modeling (SEM): the application of bootstrapping MAS 2015 9 10.5539/mas.v9n9p58
132 Bentler & Kano Multivariate behavioral on the equivalence of factors and components Mult Behav Res 1990 25 67 74 10.1207/s15327906mbr2501_8
133 Cronbach LJ Coefficient alpha and the internal structure of tests Psychometrika 1951 16 297 334 10.1007/BF02310555
134 Asnawi AA Awang Z Afthanorhan A et al The influence of hospital image and service quality on patients’ satisfaction and loyalty 105267/jmsl 2019 9 911 20 10.5267/j.msl.2019.2.011
135 Kline RB Principles and practice of structural equation modeling 4th edn New York The Guilford Press 2016 14 20 Available https://www.researchgate.net/publication/361910413
136 Fornell C Larcker DF Evaluating structural equation models with unobservable variables and measurement error J Market Res 1981 18 39 50 10.1177/002224378101800104
137 Bajwa NK Singh H De KK Critical success factors in electronic health records (EHR) implementation Int J Healthc Inf Syst Inform 2017 12 1 17 10.4018/IJHISI.2017040101
138 Mohamad MM Sulaiman NL Sern LC et al Measuring the validity and reliability of research instruments Proc Soc Behav Sci 2015 204 164 71 10.1016/j.sbspro.2015.08.129
139 Yusof MM Kuljis J Papazafeiropoulou A et al An evaluation framework for health information systems: human, organization and technology-fit factors (HOT-fit) Int J Med Inform 2008 77 386 98 10.1016/j.ijmedinf.2007.08.011 17964851
140 Abdul Aziz A Jusoh MS Amlus MH et al Construct validity: a rasch measurement model approaches J Appl Sci Agric 2014 9 7 12 Available https://www.researchgate.net/publication/266676182
141 Nur Izzati R Kebolehpercayaan dan kesahan dalam kajian pensyarah nur izzati binti rozman 2018 1 28
142 Kamarul Azmi J Kesahan dan kebolehpercayaan dalam kajian kualitatif J Pendidik Maktab Perguru Islam 2012 61 82
143 Azma Rahlin N Awang Z Zulkifli Abdul Rahim M et al THE impact of employee safety climate on safety behavior in small & medium enterprises: an empirical study HSSR 2020 8 163 77 10.18510/hssr.2020.8318
144 Baharum H Ismail A Awang Z et al Validating an instrument for measuring newly graduated nurses’ adaptation Int J Environ Res Public Health 2023 20 2860 10.3390/ijerph20042860 36833559
145 Baharum H Ismail A McKenna L et al Success factors in adaptation of newly graduated nurses: a scoping review BMC Nurs 2023 22 125 10.1186/s12912-023-01300-1 37069647
146 Afthanorhan A Mamun AA Zainol NR et al Framing the retirement planning behavior model towards sustainable wellbeing among youth: the moderating effect of public profiles Sustainability 2020 12 8879 10.3390/su12218879
147 Baistaman J Awang Z Afthanorhan A et al DEVELOPING and validating the measurement model for financial literacy construct using confirmatory factor analysis HSSR 2020 8 413 22 10.18510/hssr.2020.8247
148 Sabri MF Anthony M Law SH et al n.d. Impact of financial behaviour on financial well-being: evidence among young adults in malaysia J Financ Serv Mark 10.1057/s41264-023-00234-8
149 Hovenga EJS Lowe C Staffing resource allocation, budgets and management Measuring capacity to care using nursing data Elsevier 2020 181 235 Available 10.1016/b978-0-12-816977-3.00007-1
150 Hovenga EJS Lowe C Nursing and midwifery work measurement methods and use Measuring capacity to care using nursing data Elsevier 2020 81 122
151 Choo J Critical success factors in implementing clinical pathways/case management Ann Acad Med Singap 2001 30 17 21 Available https://pubmed.ncbi.nlm.nih.gov/11721273/ 11721273
152 Gleditsch KS “This research has important policy implications….” Peace Econ, Peace Sci Public Policy 2023 29 1 17 10.1515/peps-2023-0002
153 Horrigan JB How americans get in touch with government internet users benefit from the efficiency of e-government, but multiple channels agencies and solve problems findings Spec 2004
154 Erismann S Pesantes MA Beran D et al How to bring research evidence into policy? synthesizing strategies of five research projects in low-and middle-income countries Health Res Policy Syst 2021 19 29 10.1186/s12961-020-00646-1 33676518
155 Creswell JW Poth CN Qualitative inquiry and research design: choosing among five approaches Sage Publications 2016
156 Creswell JW Creswell JD Research design qualitative, quantitative, and mixed methods approaches 5th edn Los Angeles, CA SAGE Publications 2018 1 418
157 Ridenour CS Newman I Mixed methods research: exploring the interactive continuum Fire risk management 2nd edn Carbondale Southern Ilionois: University Press 2009 24 7
158 Evans D Coad J Cottrell K et al Public involvement in research: assessing impact through a realist evaluation Health Serv Deliv Res 2014 2 1 128 10.3310/hsdr02360
159 Rosly RM Khalid F Evaluation of the “e-daftar” system using the technology acceptance model (tam) CE 2018 09 675 86 10.4236/ce.2018.95049
160 Reid B Henry P Children W et al Casemix-based hospital information systems Aust Med Rec J 1991 21 128 32 10.1177/183335839102100404
161 Baba NM Baharudin AS Determinants of users’ intention to use iot: a conceptual framework Adv Intell Syst Comput 2020 1073 980 90 10.1007/978-3-030-33582-3_92
162 Adams DA Nelson R Todd PA et al Perceived usefulness, ease of use, and usage of information technology: A replication increasing systems usage perceived usefulness, ease of use, and usage of information technology: A replication Source MIS Q 1992 16 227 47 10.2307/249577
163 Jöreskog KG Sörbom D LISREL 8: user’s reference guide 2nd edn Chicago Scientific Software International 1996
164 Tanaka JS Huba GJ A fit index for covariance structure models under arbitrary GLS estimation Brit J Math & Statis 1985 38 197 201 10.1111/j.2044-8317.1985.tb00834.x
165 McDonald RP Bollen KA Structural equations with latent variables J Am Stat Assoc 1990 85 1175 10.2307/2289630
