
==== Front
Inquiry
Inquiry
INQ
spinq
Inquiry: A Journal of Medical Care Organization, Provision and Financing
0046-9580
1945-7243
SAGE Publications Sage CA: Los Angeles, CA

39264055
10.1177/00469580241275324
10.1177_00469580241275324
Original Research
Characteristics of Medical Quality in Tertiary Traditional Chinese Medicine Hospitals by TOPSIS and RSR Methods
Ma Lijiao B.S 1
https://orcid.org/0000-0002-5179-7983
Li Yuanyuan MMed 2
Xue Jianlan B.S 1
Xu Lin MMsc 3
Li Xianwen MMed 2
https://orcid.org/0009-0009-8711-4695
Chang Xuhong MDs 1
1 School of Public Health, Lanzhou University, Chengguan District, Lanzhou, China
2 Medical Department,Gansu Provincial Hospital of Traditional Chinese Medicine, Qilihe District, Lanzhou, China
3 Finance Department, Gansu Provincial Hospital of Traditional Chinese Medicine, Qilihe District, Lanzhou, China
Yuanyuan Li, Medical department, Gansu Provincial Hospital of Traditional Chinese Medicine, 518 Guazhou Road, Qilihe District, Lanzhou City, Gansu Province, China. Email: li-yoyo@163.com
Xuhong Chang, School of Public Health, Lanzhou University, Chengguan District, Lanzhou 730030, China. Email: changxh@lzu.edu.cn
12 9 2024
Jan-Dec 2024
61 0046958024127532411 3 2024
11 7 2024
30 7 2024
© The Author(s) 2024
2024
SAGE Publications
https://creativecommons.org/licenses/by-nc/4.0/ This article is distributed under the terms of the Creative Commons Attribution-NonCommercial 4.0 License (https://creativecommons.org/licenses/by-nc/4.0/) which permits non-commercial use, reproduction and distribution of the work without further permission provided the original work is attributed as specified on the SAGE and Open Access pages (https://us.sagepub.com/en-us/nam/open-access-at-sage).
Performance evaluation is important for improving medical quality and services. But, there is a lack of research for medical quality in traditional Chinese medicine (TCM) hospitals. This study examines the medical quality and various indicators of tertiary public traditional Chinese medicine hospitals in Gansu Province, to establish a foundation for improving the medical and management standards of these hospitals. This study collected performance assessment data from 10 tertiary TCM hospitals in Gansu Province from 2019 to 2022. Thirteen indicators with TCM characteristics were selected and categorized into 3 aspects: control of medical costs, internal operational dimensions, and comprehensive management. The level of medical quality in different hospitals and in different years were determined using the TOPSIS method for ranking and the RSR method for grading. Firstly, in terms of TCM characteristic indicators, hospital H had the highest control of medical costs and comprehensive management among different hospitals, with 45.87% and 24.20% respectively. The highest values for control of medical costs and comprehensive management were observed in 2020, with 40.65% and 18.69% respectively among different years. When evaluating the medical quality of different hospitals using the TOPSIS method, it was found that hospital H had the highest ranking from 2020 to 2022, with Ci values of 0.725, 0.778, and 0.667 respectively. Additionally, the RSR method indicated that hospital H had a high level of grading from 2020 to 2022, with Pi values of 0.687, 0.690, and 0.723 respectively. These findings suggest that the medical quality of hospital H is at a high and stable level of development. Based on the TOPSIS method to evaluate the performance appraisal results and ranking of different hospitals from 2019 to 2022. The results showed that the highest ranking was hospital B(Ci = 0.913) in 2019. The highest ranking was hospital C(Ci = 0.809)in 2020. The highest ranking was hospital D(Ci = 0.689) in 2021. The highest ranking was hospital J(Ci = 0.865) in 2022. The RSR method indicated that high grading level were hospitals B(Pi = 0.899),F(Pi = 0.795) in 2019. The highest grading level was hospital C(Pi = 0.809) in 2020. The highest grading level were hospitals A(Pi = 0.868), D(Pi = 0.813), E(Pi = 0.689), G(Pi = 0.873), J(Pi = 0.813), K(Pi = 0.842) in 2022. Based on the above results indicate that there is a large variation in the medical quality profile of different hospitals from 2019 to 2022. By comparing the results of TOPSIS and RSR method from 2019 to 2022, we found that the hospitals with identical ranking were D and J, and the hospitals with ≤2 difference in ranking was A,B,C,E in 2019, the hospitals with >2 ranking was A, F in 2020, the hospitals with >2 ranking were C, G in 2021, and the hospitals with identical ranking results were B,D,E,G,J in 2022. Comparing the ranking results of TOPSIS and RSR methods, showed that the hospitals with identical rankings were B, F from 2019 to 2022. The difference in ranking results ≤2 were A, C, D, E, G, H, J, K, indicating that high consistency between TOPSIS and RSR methods and credible results. The findings reveal significant fluctuations in medical quality across different years, while the overall level of medical quality remains relatively stable among the various hospitals. It is recommended that TCM hospitals focus on improving management efficiency, optimizing hospital operations, enhancing the utilization of medical resources, and fostering the efficient development of hospitals.

Traditional Chinese Medical Hospitals
TOPSIS method
RSR method
medical quality
performance appraisal
Gansu Province Chinese medical Research Program GZKB-2022-40 cover-dateJanuary-December 2024
typesetterts1
==== Body
pmc What do we already know about this topic?

Previous studies have shown that evaluating medical quality is an important part of new medical reform and is necessary to solve the problem of limited and unbalanced medical resources.

How does your research contribute to the field?

This study aimed to evaluate the degree of excellence of medical quality and other indicators in tertiary public TCM hospitals in Gansu Province, evaluating the relative efficiency of hospital operations, and determine the influencing factors of hospital efficiency by the TOPSIS and the RSR method. Provide a basis for comprehensively improving the medical level and management level of tertiary public TCM hospitals in Gansu Province.

What are your research’s implications toward theory, practice, or policy?

The tertiary public TCM hospitals in Gansu Province have different developmental shortcomings in terms of medical quality, and need to take appropriate measures to continuously improve the quality of their medical services. The government should strive to narrow the gap in the efficiency of medical resources utilization in different regions. Hospital managers should improve their ability to promote the effective utilization of medical resources. This will result in improved medical technology ability and management ability.

Introduction

Medical quality played a crucial role in assessing the performance of tertiary hospitals, and enhancing the management levels was imperative for the continuous improvement of medical quality and services for the people. 1 Performance appraisal not only aligned with development goals, but also continuously improved the performance allocation scheme, leading to enhanced efficiency and motivation. 2 Therefore, performance appraisal has a significant value in improving medical quality and services.

Combined Chinese medicine and clinical practice could promote the ability of research for formulating strategic goals and fulfilling the public welfare duties.3,4 The evaluation of Chinese medicine clinical served as a basis for formulating national health policies and promoting medical fairness in China.5,6 The United States integrated Chinese medicine with clinical practice to provide restorative treatment for patients with long-term COVID infections. 7 However, there is a lack of research on the performance appraisal of Traditional Chinese Medicine (TCM) hospitals in China.

Technique for Order Preference by Similarity to Ideal Solution (TOPSIS) methodology determined the best option by performance scores and ranked.8,9 TCM hospital performance appraisal is usually scored by a percentage system, exists 1-sided and cannot reflect the fairness of the assessment. 10 Human and technological factors were the most important factors for medical tourism adoption in Malaysia by Fuzzy TOPSIS. 11 Base on the shortcomings of the traditional TOPSIS method, this paper introduces the principle of RSR, aiming at achieving the sorting of the relative closeness sought above, so that it can be more objective, comprehensive and real analysis and assessment. Rank-sum ratio (RSR) solved the problem of grading the evaluation object by combining the concepts and methods of parametric analysis. 12 RSR method are a relatively new way to evaluate TCM hospitals performance appraisals, which are effective methods frequently utilized in multi-objective decision analysis, 13 the policy and measures implement in the healthcare reform promoted child healthcare in China by RSR. 14 Currently, TOPSIS and RSR method were a relatively new way to evaluate TCM hospital performance appraisals, which were effective methods frequently utilized in multi-objective decision analysis. 15 Health education oriented strategic model was the best model for managing schistosomiasis transmission by TOPSIS and RSR method. 16 The effectiveness of community-based public health services was evaluated using the TOPSIS and RSR methods in Inner Mongolia Autonomous Region. 17 The TOPSIS method and the RSR method were used to analyze the health resource allocation in the Chengdu-Chongqing from 2017 to 2021. 18 To comprehensively evaluate have jointly promoted the development of child health care in China in the past 20 years by TOPSIS and RSR method, providing a scientific theoretical basis for future policy-making to promote child health care provide theoretical support for future decision-making on the policies of child health care in China. 14 However, fewer studies have been conducted to evaluate the quality of Chinese hospitals using these 2 methods.

In this study, 13 performance assessment indicators in TCM hospitals from Gansu Province were chosen based on the Internal Performance Assessment and Evaluation System of tertiary Hospitals. Then, based on the Pilot Project: Establishing Performance Indicator Systems for Public Chinese Medicine Hospitals that Combine Western and Traditional Medicines developed by China, we categorized these indicators into 3 groups: Control of Medical Costs, Internal Operational Dimensions, and Comprehensive Management, a comparison of data from these 3 aspects reveals trends in medical quality changes and provides a dynamic view of hospital management. The strengths and weaknesses of medical quality were evaluated by TOPSIS and RSR method for identifying the areas for improvement, addressing shortcomings, and enhancing the medical and management levels. The results will provide data support for Chinese hospitals to improve medical quality.

Materials and Methods

Data Sources

We collected performance appraisal data from 10 tertiary TCM hospitals in Gansu Province from 2019 to 2022, with the names of the hospitals denoted by A, B, C, D, E, F, G, H, J, K. The data came from 6 hospitals (A, B, C, D, E, J) in 2019 and from 10 hospitals in 2020, 2021 and 2022.

Indicators Determination

The evaluation indicators adopted herein are based on the Pilot Project: Establishing Performance Indicator Systems for Public Chinese Medicine Hospitals that Combine Western and Traditional Medicines developed by China. To evaluate the performance appraisal of 10 tertiary TCM hospitals in Gansu Province from 2019 to 202l, we selected 13 indicators (X1-X13) including Control of medical costs (X1-X4), Internal Operational Dimensions (X5-X9) and Comprehensive management (X10-X13). X1 stand for the proportion of outpatient Chinese medicine prescriptions. X2 stand for the proportion of outpatient prescriptions for bulk and sachet Chinese divine tablets. X3 stand for the utilization rate of outpatient Chinese medicine tablets. X4 stand for the utilization rate of discharged patients’ Chinese medicine tablets. X5 stand for the proportion of outpatients using non-pharmacological therapies of traditional Chinese medicine (X6) stand for the proportion of discharged patients using non-pharmacological therapies of traditional Chinese medicine. X7 stand for the proportion of discharged patients who were mainly treated by traditional Chinese medicine. X8 stand for the proportion of medical service income (excluding income from medicines, consumables, examinations, and tests) to medical income. X9 stand for the proportion of income from traditional Chinese medicines to medicine income. X10 stand for the proportion of income from TCM to medicine income. X11 stand for the proportion of income from traditional Chinese medicine preparations to medicine income in medical institutions, X12 stand for the proportion of income from outpatient TCM medical service items to outpatient medical income. X13 stand for the proportion of income from inpatient TCM to hospitalization income.

Methods

TOPSIS

The TOPSIS method is a technique for order preference according to similarity to the ideal solution. Based on the normalized raw data matrix, this method forms a space for both positive and negative ideal solutions of priority solutions. The distance between a point and the positive and ideal solutions is obtained. This distance helps identify the relative closeness between the solution to be evaluated and the positive ideal solution and provides a basis for evaluating the advantages and disadvantages of this solution.19,20 In order to ensure that the evaluation indicators have a positive trend, we transformed the 13 indicators of 10 hospitals with the same trend, normalized the indicators to find out the optimal and least optimal matrix vectors, calculated the distance of the indicators of each Chinese hospital from the positive and negative ideals (D+ and D−), and ranked the Ci value indicating the close to the optimal solution.

Technique for Order Preference by Similarity to an Ideal Solution (TOPSIS). 21 Basic steps:

(1) The same trending method was used for all the indicators.

(2) The original data matrix with the same trend was normalized.

(1) Zij=xij∑i=1nxij2

In the equation, xij represents the value of the i-th evaluation object of the j-th indicator. The normalized matrix Z can be presented as

(2) Z=[Z11Z12⋯Z1mZ21Z22⋯Z2m⋯⋯⋯⋯Zn1Zn2⋯Znm]

(3) According to the normalized matrix Z, the positive ideal (optimal vector) and negative ideal solutions (worst-case vector) were calculated as follows:

Positive ideal solution: Z+=(zi1+,zi2+,⋯,zim+).

Negative ideal solution: Z−=(zi1−,zi2−,⋯,zim−).

(4) The weighted Euclidean distance between D+ i (positive ideal solution) and D− i (negative ideal solution) for each indicator values of every laboratory were calculated as follows:

(3) Di+=∑j=1m[ωj(Zij−Zij+)]2

(4) Di−=∑j≡1m[ωj(Zij⋅Zij)]2

(5) The relative closeness between the positive ideal and negative ideal solutions as well as the indicator values were calculated as follows:

(5) Ci=Di−Di++Di−

The value of Ci was set between 0 and 1. Ci value close to 1 indicates a higher likelihood of positive ideal solution (optimal level), whereas a Ci value close to 0 indicates a higher likelihood of negative ideal solution (worst level).

RSR

The RSR method is a comprehensive evaluation and analysis method proposed by the Chinese statistician, Professor Fengtiao Tian in 1988. RSR method is performed by ranking the indicators to obtain the RSR value of the dimensionless statistic, which grades the evaluated objects depending on the RSR value. In this study, all indicators were high superiority indicators. Combining the principle of optimal grading and the table of reasonable grading numbers, 17 the grading level is divided into upper, middle and lower levels, which are expressed as 3, 2, and 1.

Rank Sum Ratio (RSR) Basic steps:

(1) Data matrix

In order to calculate the computed Pi values, we ranked the raw data as a matrix of m×n, (where m denotes the hospitals and n is the 13 evaluation indicators).

(2) Rank

Then applies the parametric statistical analysis to determine a set of quantitative methods. and ranked them from smallest to largest for the high superiority indicators and from largest to smallest for the low superiority indicators.

(3) Calculation of RSR

In a matrix with n rows and m columns, the RSR was calculated as follows:

(6) RSRi=1m⋅n∑j=1mRij

(4) Determination of RSR distribution

The distribution of RSR refers to the specific downward cumulative frequency of RSR values expressed in the probability unit (Pi), the calculation of the Pi value is carried out and sorted into grades, and the larger the value, the higher the level.

Combination of TOPSIS and RSR methods

The TOPSIS method can be applied to the relative closeness value Ci, which ranges from 0 to 1, and the RSR value distributed in the same rank and ratio method can be analyzed using the RSR method. Here are the basic steps involved:

(1) Determination of RSR distribution for Ci. The RSR distribution of Ci refers to the speciﬁc downward cumulative frequency of Ci expressed in the Probit. Speciﬁc steps include compiling the frequency distribution table of the Ci values; listing the frequency f of different groups and calculating the cumulative frequency Σf of each group; determining the rank R, average rank R, and R/n values for each group of Ci values.

(2) Calculation of the regression equation.

Using Y as the independent variable and Ci value as the dependent variable, the regression equation becomes Ci = a+b×Probit

(3) Hypothesis testing of the regression equation.

Determine whether the Ci value is normally distributed and whether the regression equation is relevant.

(4) Rank by level.

The evaluation objects are ranked by level based on the reasonable RSR rank method and the corresponding estimated values obtained using the regression equation.

Statistical Analysis

The data was collected and organized using Excel. The 13 evaluation indicators were statistically described in the form of component ratios. The data was then synthesized and ranked using the TOPSIS and RSR methods of SPSS 22.0 statistical analysis software.

Results

Description of Indicators With Chinese Medicine Characteristics

In different hospitals, we found that the highest control of medical costs was H hospital (45.87%), and the lowest was D hospital (22.10%). The highest for the Internal Operational Dimensions indicator was G hospital (91.16%), and the lowest was D hospital (11.23%). The highest for the Comprehensive management Indicator was H hospital (24.20%), and the lowest was J hospital (1.33%; Table 1).

Table 1. Describing Indicators of Chinese Medicine in Different Hospitals.

Hospitals	Control of medical costs	Internal operational dimensions	Comprehensive management	
N	n (%)	N	n (%)	N	n (%)	
A	10 350 894	4 578 127 (44.23)	892 211	423 484.00 (47.76)	705 824	116 860.0 (16.56)	
B	6 545 184	2 568 816 (39.25)	2 303 349	358 506.12 (15.56)	339 193	22 453.62 (6.62)	
C	6 434 201	2 727 509 (42.39)	2 025 606	324 658.93 (16.03)	191 558	23 154.78 (12.09)	
D	3 519 457	777 827 (22.10)	1 982 496	222 565.15 (11.23)	193 224	23 085.07 (11.95)	
E	5 442 235	2 062 639 (37.90)	2 212 038	304 696.26 (13.77)	173 347	15 947.94 (9.20)	
F	1 652 044	473 397 (28.66)	570 287	91 321.90 (16.01)	52 299	8528.69 (16.31)	
G	1 307 644	453 655 (34.69)	553 010	504 103.41 (91.16)	57 581	12 101.92 (21.02)	
H	1 220 367	559 754 (45.87)	641 117	74 507.36 (11.62)	40 012	9764.08 (24.20)	
J	4 454 245	1 415 096 (31.77)	1 964 240	323 209.65 (16.45)	218 710	2901.36 (1.33)	
K	1 196 964	489 730 (40.91)	577 147	53 528.31 (9.27)	33 044	6502.12 (19.68)	
Note. N stands for target value, n stands for actual value.

In different years, we found that the highest Control of medical costs was 2020 (40.65%), and the lowest was 2022 (36.30%). The highest for the Internal Operational Dimensions indicator was 2019 (87.50%), and the lowest was 2022 (9.22%). The highest for the Comprehensive management indicator was 2020 (18.59%), and the lows was 2019 (10.87%; Table 2).

Table 2. Describing Indicators of Chinese Medicine From 2019 to 2022.

Years	Control of medical costs	Internal operational dimensions	Comprehensive management	
N	n (%)	N	n (%)	N	n (%)	
2019	8 204 708	3 233 419 (39.41)	3 150 670.53	2 756 782.95 (87.50)	472 351.23	51 321.83 (10.87)	
2020	10 261 521	4 171 006 (40.65)	3 799 203.41	905 362.28 (23.83)	423 449.64	78 721.90 (18.59)	
2021	11 510 302	4 326 491 (37.59)	3 958 057.55	857 927.46 (21.68)	533 331.91	81 123.33 (15.21)	
2022	12 146 704	4 409 730 (36.30)	5 278 304.79	486 646.57 (9.22)	482 306.53	75 182.34 (15.59)	
Note. N stands for target value, n stands for actual value.

Evaluation of the Medical Quality in Different Hospitals

Ranking for medical quality by the TOPSIS method

Based on the concept of sorting after TOPSIS normalization process, we analyzed the ranking of 13 indicators with TCM characteristics in different hospitals from 2019 to 2022. We found that the highest ranked hospital in 2019 was C(Ci = 0.630), and the lowest ranked hospital was D(Ci = 0.265). The highest ranked hospital in from 2020 to 2022 was H(Ci = 0.725,0.778,0.667). The lowest ranked was F(Ci = 0.334) in 2020 and the lowest ranked was D(Ci = 0.287,0.216) in from 2021 to 2022. The indicated that higher ranking was H, and lower ranking was D from 2021 to 2022 (Table 3, Figure 1A).

Table 3. Ranking of Ci Values of Different Hospitals by TOPSIS Method.

Hospitals	2019	2020	2021	2022	
D+	D−	Ci	Rank	D+	D−	Ci	Rank	D+	D−	Ci	Rank	D+	D−	Ci	Rank	
A	47.701	61.292	0.562	4	74.913	105.537	0.585	4	48.564	79.146	0.620	2	54.757	68.066	0.554	3	
B	42.380	61.884	0.594	2	83.152	91.229	0.523	7	62.371	57.519	0.480	5	75.932	40.542	0.348	8	
C	41.366	70.374	0.630	1	80.803	100.248	0.554	5	63.509	68.109	0.517	3	69.037	58.296	0.458	5	
D	76.902	27.717	0.265	6	104.223	70.903	0.405	8	88.556	35.574	0.287	10	91.273	25.166	0.216	10	
E	42.114	55.885	0.570	3	81.160	97.638	0.546	6	63.975	58.309	0.477	6	68.319	55.394	0.448	6	
F	-	-	-	-	114.842	57.713	0.334	10	68.994	64.985	0.485	4	81.320	47.262	0.368	7	
G	-	-	-	-	66.383	98.603	0.598	3	68.179	54.427	0.444	7	55.053	66.223	0.546	4	
H	-	-	-	-	44.783	118.109	0.725	1	27.784	97.573	0.778	1	45.126	90.377	0.667	1	
J	74.055	41.355	0.358	5	113.562	74.884	0.397	9	94.242	41.004	0.303	9	90.819	35.821	0.283	9	
K	-	-	-	-	50.020	116.864	0.676	2	72.025	55.349	0.435	8	58.453	72.887	0.555	2	
Note. D+: positive ideal solution distance; D−: negative ideal solution distance.

Grading for medical quality by the RSR method

Based on the relevant principles of RSR grading method, we analyzed the 4-year performance data analysis of 10 tertiary public TCM hospitals in Gansu Province. The results showed that, the highest grading level was B(Pi = 0.739) in 2019, the highest grading level was A(Pi = 0.772) and H(Pi = 0.687), the lowest was J(Pi = 0.364) in 2020. The top of grading level were A(Pi = 0.777) and H(Pi = 0.690) in 2021. The top of grading levels were A(Pi = 0.832) and H(Pi = 0.723) in 2022. The findings indicated that the gradings of hospitals A and H showed stable trend from 2019 to 2022 (Table 4, Figure 1B).

Table 4. Grading of Pi Values of Different Hospitals by RSR Method.

Hospitals	2019	2020	2021	2022	
Value	Rank	Pi	Grade	Value	Rank	Pi	Grade	Value	Rank	Pi	Grade	Value	Rank	Pi	Grade	
A	0.679	3	0.599	2	0.70	1	0.772	3	0.785	1	0.777	3	0.800	1	0.832	3	
B	0.692	1	0.739	3	0.50	8	0.459	2	0.546	6	0.526	2	0.440	8	0.435	2	
C	0.679	2	0.657	2	0.577	5	0.557	2	0.485	7	0.493	2	0.480	7	0.478	2	
D	0.436	6	0.448	2	0.354	9	0.419	2	0.431	9	0.417	2	0.290	10	0.314	1	
E	0.538	4	0.552	2	0.569	6	0.525	2	0.569	4	0.575	2	0.550	6	0.519	2	
F	-	-	-	-	0.546	7	0.493	2	0.569	4	0.575	2	0.570	5	0.559	2	
G	-	-	-	-	0.615	4	0.591	2	0.577	3	0.634	2	0.630	4	0.602	2	
H	-	-	-	-	0.7	2	0.687	3	0.723	2	0.690	3	0.710	2	0.723	3	
J	0.474	5	0.506	2	0.3	10	0.364	1	0.362	10	0.316	1	0.360	9	0.384	2	
K	-	-	-	-	0.638	3	0.631	2	0.454	8	0.458	2	0.670	3	0.653	2	

Figure 1. Comparison of differences in TOPSIS and RSR Methods Across Hospitals and Years. (A) Ranking of different hospitals by TOPSIS method, (B) Grading of different hospitals by RSR method, (C) Ranking of hospitals by using TOPSIS method from 2019 to 2022, (D) Grading of hospitals by using RSR method from 2019 to 2022.

By comparing the results of TOPSIS and RSR method from 2019 to 2022, we found that the hospitals with identical ranking was D and J, and the hospitals with ≤2 difference in ranking was A,B,C,E in 2019, the hospitals with >2 ranking was A,F in 2020, the hospitals with >2 ranking were C,G in 2021, and the hospitals with identical ranking results were B,D,E,G,J in 2022 (Table 5).

Table 5. Comparing the Results of TOPSIS and RSR Method From 2019 to 2022.

Sorting results	2019	2020	2021	2022	
Identical	2	2	2	5	
Sorting gap ≤ 2	4	6	6	5	
Sorting gap>2	0	2	2	0	
Total	6	10	10	10	

Evaluation of the Medical Quality From 2019 to 2022

Ranking for medical quality by the TOPSIS method

Based on the TOPSIS method to evaluate the performance appraisal results and ranking of different hospitals from 2019 to 2022. The results showed that the highest ranking was hospital B(Ci = 0.913) and the lowest was hospital E(Ci = 0.312) in 2019. The highest ranking was hospital C(Ci = 0.809) and the lowest was hospital J(Ci= 0.168) in 2020. The highest ranking was hospital D(Ci = 0.689) and the lowest was hospital K(Ci = 0.338) in 2021. The highest ranking was hospital J(Ci = 0.865) and the lowest was hospital B(Ci = 0.155) in 2022 (Table 6, Figure 1C).

Table 6. Comprehensive Evaluation Results and Ranking of Hospitals by Using TOPSIS Method From 2019 to 2022.

Years	A	B	C	D	E	F	G	H	J	K	
Ci	Rank KKkKk	Ci	Rank
NKnk	Ci	Rank	Ci	Rank	Ci	Rank	Ci	Rank	Ci	Rank	Ci	Rank	Ci	Rank	Ci	Rank	
2019	0.548	3	0.913	1	0.591	2	0.473	3	0.312	4	0.716	1	0.777	1	0.654	2	0.635	2	0.477	3	
2020	0.737	1	0.627	2	0.809	1	0.442	4	0.547	1	0.644	2	0.548	2	0.667	1	0.168	4	0.536	1	
2021	0.657	2	0.540	3	0.365	3	0.689	1	0.414	3	0.161	4	0.360	4	0.339	4	0.507	3	0.338	4	
2022	0.314	4	0.155	4	0.346	4	0.627	2	0.438	2	0.285	3	0.537	3	0.354	3	0.865	1	0.504	2	

Grading for medical quality by the RSR method

The performance appraisal indicators of different hospitals were evaluated from 2019 to 2022 grading levels. The indicated that high grading level were hospitals B(0.899), F(0.795) in 2019. The highest grading level was hospital C(0.809) in 2020. The highest grading level were hospitals A(Pi = 0.868), D(Pi = 0.813),E(Pi = 0.689),G(Pi = 0.873),J(Pi = 0.813),K(Pi = 0.842) in 2022 (Table 7, Figure 1D).

Table 7. Comprehensive Evaluation Results and Grading of A,B,C,D by Using RSR Method From 2019 to 2022.

Years	A	B	C	D	E	F	G	H	J	K	
Pi	Grade	Pi	Grade	Pi	Grade	Pi	Grade	Pi	Grade	Pi	Grade	Pi	Grade	Pi	Grade	Pi	Grade	Pi	Grade	
2019	0.402	2	0.899	3	0.546	2	0.562	2	0.566	2	0.795	3	0.688	2	0.578	2	0.562	2	0.553	2	
2020	0.544	2	0.694	2	0.711	3	0.452	2	0.604	2	0.668	2	0.542	2	0.496	2	0.452	2	0.680	2	
2021	0.686	2	0.534	2	0.647	2	0.673	2	0.641	2	0.469	2	0.397	2	0.661	2	0.673	2	0.425	2	
2022	0.868	3	0.373	2	0.596	2	0.813	3	0.689	3	0.568	2	0.873	3	0.766	3	0.813	3	0.842	3	

Comparing the consistency of results between TOPSIS and RSR methods

Comparison of TOPSIS and RSR ranking in different hospitals from 2019 to 2022 showed that the hospitals with identical rankings were B, F. The difference in ranking results ≤2 were A, C, D, E, G, H, J, K (Table 8). The findings indicated that the rankings of hospitals B and F showed decreasing trend in the ranking, while the ranking of A and E showed an overall increasing trend, and the D, H, and J showed decreasing trends in the rankings from 2019 to 2022.

Table 8. Comparison of the Results of TOPSIS and RSR Method of Ranking in Different Hospitals From 2019 to 2022.

Sorting results	Number	Items	
Identical	2	B, F	
Sorting gap ≤ 2	8	A, C, D, E, G, H, J, K	
Sorting gap>2	0		
Total	10		

Discussion

The evaluation indicators are influenced by factors such as the number of visits between hospitals and the number of patients, which can be easily affected by the social environment and resident population. 22 According to the TOPSIS method, Hospital F is ranked the lowest due to its low number of patients. This can be attributed to the small resident population in the city where Hospital F is located, which mainly consists of young adults who have their own affiliated medical institutions at their workplaces. The evaluation was also influenced by the services and positioning of the hospitals themselves. 23 The RSR grading level reveals that hospitals A and H are ranked at the top, while hospital J was ranked at the bottom. One possible reason was that Hospital J was a combined Chinese and Western medicine hospital, and patients who seek Western medical services at the hospital may give lower evaluation scores to the Chinese medicine aspects. The development of the Chinese herbal medicine industry, supported by government and policies, also has an impact on the evaluation indicators. 24 Hospital H has the highest combined indicators related to integrated management and cost control, which may be attributed to the development of the Chinese herbal medicine cultivation industry in the city where Hospital H is located. On the other hand, hospital D has the lowest level of internal operation and cost control indicators. This could be due to the slow economic development of the city and state where hospital D is located, resulting in a lack of medical resources and a lag in talent cultivation. 25

The combined use of the RSR and TOSIS methods can objectively categorize hospital assessments, resulting in additional hospital construction and reform, as well as an improvement in overall hospital quality. The study’s analysis results show that the performance appraisal ranking results from the TOPSIS and RSR methods are highly consistent. Only 2 of the 10 Chinese hospitals in 2020 and 2021 had a difference in performance appraisal rankings >2. In 2019, there were 2 hospitals with identical rankings in 6 Chinese hospitals, and 4 hospitals with a ranking >2. In 2022, only 2 of the 10 Chinese hospitals that used the TOSIS and RSR methods had a ranking >2. Similarly, of 10 Chinese hospitals using these methods, only 2 had a ranking ≤2. In addition, the TOPSIS method and RSR method sorting results of 10 Chinese medicine hospitals’ performance appraisal for 4 consecutive years showed that there were 2 hospitals with identical sorting results and 8 hospitals with a sorting result gap of ≤2. The findings of this study confirm that the combination of the 2 evaluation methods is more practical, and the results obtained are more objective, making it useful as a reference for hospital managers and providing a scientific theoretical basis for further improving the quality of hospital work.

In recent years, the growing of demand for medical treatment and influence of Chinese medicine led to a continuous workload increasing in TCM hospitals under the context of modernized healthcare development.26,27 The healthcare strength of the hospital’s location significantly influences the hospital’s performance appraisal. 28 Hospitals H demonstrate a stronger integrated medical strength, leading to higher performance assessment results. This attributed to the rapid development of the Chinese herbal medicine planting industry in the city where H Institute is situated. The city has a long history, receives governmental attention and policy support, and considers the Chinese herbal medicine industry as a crucial initiative to adjust the structure of agricultural production. Consequently, it has become one of the fastest-growing and advantageous industries in the local national economy, with immense potential for further development. 25 The indicators of hospital D are lower compared to all other hospitals. This could be attributed to the slower economic development of the city and state where hospital D is located. Additionally, there may be issues such as a lack of medical resources, insufficient talent pool, and inadequate talent development, all of which can contribute to unsatisfactory performance appraisal results for the hospitals.29,30 The TOPSIS method and the RSR method were used to evaluate the health resource allocation in the Chengdu-Chongqing from 2017 to 2021, the health resource allocation in Chongqing was more polarized and the health resource allocation in Sichuan Province was more balanced, but the advantaged regions were not prominent enough. 18 These findings align with the results of our study. Therefore, it is important for our tertiary TCM hospitals in Gansu Province to maintain their operational efficiency while addressing the shortcomings in medical quality development. This will help promote overall improvement in the medical quality of each hospital.

The study findings reveal significant fluctuations in performance assessment results, 31 which align with our own research. Notably, Hospitals B and F experienced a decline in rankings from 2019 to 2022, while Hospitals A and E demonstrated overall improvements in their rankings over the years. These changes were attributed to multiple factors. Firstly, the widespread implementation of home quarantine measures during the 3-year epidemic led to a substantial decrease in hospital visits. 32 Additionally, considerable medical resources were allocated toward the detection, isolation, and treatment of patients infected with the coronavirus. 33 Moreover, the utilization of Chinese medicine in the prevention, treatment, and rehabilitation of coronavirus pneumonia may have contributed to better performance evaluation results for certain hospitals. 34 Therefore, this could have resulted in higher healthcare expenditure indicators and general management indicators in Chinese hospitals. This implies that hospitals within the same region with significant disparities in medical quality, as well as those with notable differences in medical quality rankings from previous years, should consider the indicators of the national tertiary hospital performance assessment. 25 They should strive to understand their own shortcomings and actively seek policy support to enhance various management systems to continually improve the quality of medical care.

This study focuses on the tertiary in Gansu Province as the research sample. However, it does not address the variations between different levels and types of medical institutions within each tertiary, nor the differences between departments within the same medical institution. While using the region as a research sample provides some insights into the variations in medical quality among hospitals, it does not thoroughly analyze the factors influencing efficiency. Further research is needed to delve into this aspect in detail. Although we used the entropy weight-TOPSIS method and RSR method to conduct a comprehensive evaluation of the performance appraisal of Gansu Provincial Hospital of Traditional Chinese Medicine in 2019 to 2022, there are still some limitations in this study. First, the evaluation indexes were selected with reference to the evaluation indexes commonly used in similar studies. Second, Chinese medicine hospitals in Gansu Province can conduct a horizontal comparison study with other regions, but this study does not. Third, this study mainly analyzes from the perspective of performance evaluation efficiency and does not consider the issue of health resource allocation.

Conclusions

This article evaluates the medical quality capacity of 10 tertiary public Traditional Chinese Medicine hospitals in Gansu Province from 2019 to 2022 by the TOPSIS and RSR methods. The findings reveal significant fluctuations in medical quality across different years, while the overall level of medical quality remains relatively stable among the various hospitals. The TOPSIS and RSR methods were used to evaluate the medical quality of different hospitals with consistent and scientifically credible results. These include improving the hospitals’ attractiveness to patients, enhancing the existing medical service measures, and diversifying the range of medical services offered to enhance the overall medical quality of Chinese medicine hospitals.

The authors thank TCM monitoring and direct reporting network and all hospitals which supplied the datum.

Author Contributions: All authors were involved in the conceptualization and design of the study. Lijiao Ma and Jianlan Xue contributed to the research methodology. Lin Xu and Xianwen Li collected, processed, and normalized the data. Lijiao Ma and Xuhong Chang wrote the original draft of the manuscript. Xuhong Chang, Yuanyuan Li reviewed and edited the manuscript.

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding: The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research has been funded by Gansu Administration of Traditional Chinese Medicine Research Program /grant number GZKB-2022-40, Monitoring and Statistical Research Center, National Administration of Traditional Chinese Medicine Policy Research program/grant number YGZXKT2024366.

ORCID iDs: Yuanyuan Li https://orcid.org/0000-0002-5179-7983

Xuhong Chang https://orcid.org/0009-0009-8711-4695
==== Refs
References

1 Amraie F Mahfoozpour S Vahdat S Hesam S. Performance appraisal models in the county health centers of Iran: a systematic review. J Biol Res Accad Soc Ital Med Biol Sper. 2022;95 (2 ):10536. doi:10.4081/jbr.2022.10536
2 Longaray AA Marube N Ensslin L Dutra A Ensslin S. Performance evaluation of public healthcare supply chain management: a critical literature review. Int J Healthc Technol Manag. 2023;20 (2 ):144-170. doi:10.1504/ijhtm.2023.131518
3 Zhuang W He T Jia BB , et al . Interaction between Chinese medicine and digoxin: clinical and research update. Front Pharmacol. 2023;14 :1663-9812. doi:10.3389/fphar.2023.1040778
4 Theodosiou T Wu S Zhang J , et al . Developing TCM clinical practice guidelines: a comparison between traditional Chinese medicine and western medicine. Integr Med Res. 2023;12 (2 ):100952. doi:10.1016/j.imr.2023.100952 37187680
5 Xie Y-M Wang L-X. Combined application of Chinese and western medicine, precise positioning for clinical efficacy. China J Chin Mater Med.2016;41 (24 ):4479-4482. doi:10.4268/cjcmm20162401
6 Guo X Lin Y He F , et al . Identification of active compounds of traditional chinese medicine derived from maxing shigan decoction for COVID-19 treatment: a meta-analysis and in silico study. Expert Rev Anti-Infect Ther. 2023;21 (8 ):871-889. doi:10.1080/14787210.2023.2238899 37481738
7 Estores IM Ackerman P. Integrative medicine in long COVID. Phys Med Rehabil Clin N Am. 2023;34 (3 ):677-688. doi:10.1016/j.pmr.2023.03.006 37419539
8 Mahmood A Abbas M. Influence model and doubly extended TOPSIS with TOPSIS based matrix of interpersonal influences. J Intell Fuzzy Syst. 2020;39 (5 ):7537-7546. doi:10.3233/jifs-200833
9 Lu M . Evaluation of medical and health service ability of TCM hospitals in Yangtze River delta region based on entropy weight-TOPSIS method. Adv Appl Math. 2022;11 (01 ):84-92. doi:10.12677/aam.2022.111013
10 Vărzaru AA. An empirical framework for assessing the balanced scorecard impact on sustainable development in healthcare performance measurement. Int J Environ Res Public Health. 2022;19 (22 ):15155. doi:10.3390/ijerph192215155
11 Lo HW Liou JJH . Comments on “Factors influencing medical tourism adoption in Malaysia: a DEMATEL-Fuzzy TOPSIS approach.” Comput Ind Eng. 2021;157 :157107333. doi:10.1016/j.cie.2021.107333
12 Yin H Chen X Zhou Y Xu J Huang D. Contribution quality evaluation of table tennis match by using TOPSIS-RSR method - an empirical study. BMC Sports Sci Med Rehabil. 2023;15 (1 ):132-132. doi:10.1186/s13102-023-00738-9 37833812
13 Wu X Huang Z Shen S . Comprehensive evaluation of medical service ability of TCM hospitals in 30 provinces. Autonomous Regions and Municipalities of China in 2017 Based on Entropy Weight TOPSIS Method and RSR Method. ACM; 2019 :108-112.
14 Wu M Liu Q Wang Z. A comparative evaluation of child health care in China using multicriteria decision analysis methods. BMC Health Serv Res. 2023;23 (1 ):1217. doi:10.1186/s12913-023-10204-4 37932716
15 Wu X Huang Z Shen S . Comprehensive Evaluation of Medical Service Capability in TCM Hospitals by Using the Fuzzy Combined Method of TOPSIS and Rank Sum Ratio. IEEE; 2019 :244-248.
16 Feng J Guo Z Qian P , et al . Field validation of different intervention modes for the potential transmission risk of schistosomiasis during post-transmission interruption phase. PLoS Negl Trop Dis. 2023;17 (11 ):e0011739. doi:10.1371/journal.pntd.0011739
17 Dai X Jiang Y Li Y , et al . Evaluation of community basic public health service effect in a city in Inner Mongolia Autonomous Region—based on entropy weight TOPSIS method and RSR fuzzy set. Arch Public Health. 2023;81 (1 ):149. doi:10.1186/s13690-023-01151-x 37592329
18 Zhou M. Regional differences in health resource allocation: a longitudinal study in the Chengdu-Chongqing economic circle, China. BMJ Open. 2024;14 (3 ):e082721. doi:10.1136/bmjopen-2023-082721
19 Wang D Wan K Song X. Coal miners' livelihood vulnerability to economic shock: multi-criteria assessment and policy implications. Energy Policy. 2018;114 :301-314. doi:10.1016/j.enpol.2017.12.017
20 Chen X Ding H Wang J , et al . A Comprehensive Evaluation Method on Operation Quality of Power Backbone Optical Communication Network Based On Improved TOPSIS. 2018 IEEE 3rd International Conference on Cloud Computing and Internet of Things (CCIOT), Dalian, China, October 20-21, 2018, IEEE. pp. 506–511.
21 Yoon K. Systems Selection by Multiple Attribute Decision Making. Kansas State University. 1980.
22 Angelini M Teglia F Casolari G Astolfi L Boffetta P. Decrease of visits and hospital admissions for cancer patients during the COVID-19 pandemic. a systematic review and meta-analysis. J Public Health. 2023;32 (6 ):909-915. doi:10.1007/s10389-023-01857-w
23 Ranchon F Chanoine S Lambert-Lacroix S , et al . Development of artificial intelligence powered apps and tools for clinical pharmacy services: a systematic review. Int J Med Inform. 2023;172 :172104983. doi:10.1016/j.ijmedinf.2022.104983
24 Zhang CC Wang S Wang YF , et al . Application of tissue culture technology of medicinal plants in sustainable development of Chinese medicinal resources. China J Chin Mater Med. 2023;48 (5 ):1186-1193. doi:10.19540/j.cnki.cjcmm.20221017.104
25 Ahumada-Canale A Jeet V Bilgrami A , et al . Barriers and facilitators to implementing priority setting and resource allocation tools in hospital decisions: a systematic review. Soc Sci Med. 2023;322 :115790. doi:10.1016/j.socscimed.2023.115790 36913838
26 Yi EH Baek J. Analysis on the determinants of medical demand in China: focusing on the different effects of supplier-induced demand. J Manag Econ. 2020;42 (1 ):61-80. doi:10.22828/meri.2020.42.1.004
27 Wang L Suo S Li J , et al . An investigation into traditional Chinese medicine hospitals in China: development trend and medical service innovation. Int J Health Policy Manag. 2016;6 (1 ):19-25. doi:10.15171/ijhpm.2016.72
28 Sheng P Yang T Zhang T. The unmet medical demand among China's urban residents. Int J Environ Res Public Health. 2021;18 (21 ):11708. doi:10.3390/ijerph182111708
29 Al-Hussaniy HA. Medical scientific research challenges in Iraq. med Pharm J. 2023;2 (1 ):1-3. doi:10.55940/medphar202228
30 DeClercq C Gharipour M Mohagheghi S , et al . A novel approach to locating community clinics to promote health care accessibility and reduce health disparities in Baltimore, Maryland. Inq J Heal Care Organ Provis Financ. 2023;60 :00469580221135953. doi:10.1177/00469580221135953
31 Li ZG Wei H. A comprehensive evaluation of China's TCM medical service system: an empirical research by integrated factor analysis and TOPSIS. Front Public Health.2020;8 :532420. doi:10.3389/fpubh.2020.532420
32 Ayittey FK Ayittey MK Chiwero NB Kamasah JS Dzuvor C. Economic impacts of Wuhan 2019-nCoV on China and the world. J Med Virol. 2020;92 (5 ):473-475. doi:10.1002/jmv.25706 32048740
33 Liu X Jia C Cheng J , et al . Measuring productivity and relative efficiency of public tertiary traditional Chinese Medicine Hospitals in Hubei, China. Inq J Heal Care Organ Provis Financ. 2023;60 :00469580231190576. doi:10.1177/00469580231190576
34 Wang X Wang J Pang F. Analysis on the selection of herbs in TCM COVID-19 treatment protocols between Malaysia and China. Int J Gen Med. 2023;16 :3655-3663. doi:10.2147/IJGM.S426168 37637714
