
==== Front
J Educ Health Promot
J Educ Health Promot
JEHP
J Edu Health Promot
Journal of Education and Health Promotion
2277-9531
2319-6440
Wolters Kluwer - Medknow India

JEHP-13-95
10.4103/jehp.jehp_1007_22
Original Article
Psychometric properties of the COVID-19 safety measures questionnaire in the employees of the radiation therapy center
Saber Korosh
Madadizadeh Farzan 1
Abdollahi-Dehkordi Sepideh 2
Azmoonfar Rasool 3
Hamzian Nima 2
Shabani Masoud 4
Department of Medical Physics, School of Medicine, Isfahan University of Medical Sciences, Isfahan, Iran
1 Biostatistics and Epidemiology Department, School of Public Health, Shahid Sadoughi University of Medical Sciences, Yazd, Iran
2 Medical Physics Department, School of Medicine, Shahid Sadoughi University of Medical Sciences, Yazd, Iran
3 Department of Radiology, School of Allied Medical Science, Hamadan University of Medical Sciences, Hamadan, Iran
4 Department of Radiation Oncology, School of Medicine, Shahid Sadoughi University of Medical Sciences, Yazd, Iran
Address for correspondence: Dr. Nima Hamzian, Medical Physics Department, School of Medicine, Shahid Sadoughi University of Medical Sciences, Yazd, Iran. E-mail: Hamzian.Nima@gmail.com
2024
28 3 2024
13 9513 7 2022
01 5 2023
Copyright: © 2024 Journal of Education and Health Promotion
2024
https://creativecommons.org/licenses/by-nc-sa/4.0/ This is an open access journal, and articles are distributed under the terms of the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 License, which allows others to remix, tweak, and build upon the work non-commercially, as long as appropriate credit is given and the new creations are licensed under the identical terms.
BACKGROUND:

Due to the effect of coronavirus disease 2019 (COVID-19) outbreak on the continuation, schedule, and efficiency of radiation therapy, this study aimed to investigate the reliability and validity of the COVID-19 Safety Measures (CSM) questionnaire at the radiation therapy center.

MATERIALS AND METHODS:

In this analytical cross-sectional study, which all personnel of the radiation therapy center (20 people) participated, the validity and reliability of the 16-item CSM questionnaire were investigated. Cultural adaptation, face validity, content validity, test-retest reliability, and internal consistency were evaluated. For face and content validity, impact score, content validity ratio, and content validity index (CVR and CVI) were calculated, respectively. Also, internal consistency and stability reliability were calculated with Kuder–Richardson (KR20) alpha and Pearson correlation coefficient and intraclass correlation (ICC), respectively. Data analysis was performed using Statistical Package for the Social Sciences (SPSS) 24 with a significant level of 5%.

RESULTS:

Out of 20 employees, 70% (14 people) were female, 75% (15 people) were married and the mean age (SD) was 32.4 (6.35) years. Scale-based Kuder–Richardson alpha, S-CVI, ICC, and confidence interval were 0.79, 0.97, 0.68, and 0.38–0.89, respectively.

CONCLUSION:

The validity and reliability of the 16-item CSM questionnaire were confirmed. Therefore, the application of this scale is recommended.

Cancer
COVID-19
radiation therapy
safety measures
validity
and reliability
==== Body
pmcIntroduction

In December 2019, pneumonia associated with a new type of coronavirus disease 2019 (COVID-19) began and spread rapidly in Wuhan, China.[1] In Iran, the COVID-19 outbreak was officially confirmed in January 2020 in Qom province, and so far Iran is one of the top 10 countries affected by COVID-19.[2]

COVID-19 has caused many problems for politicians in the economy and beyond. COVID-19 is a major threat to the physical, mental, and emotional health of people, which has caused fear and panic among the people in the society. Most people show negative emotions after hearing about COVID-19, which mostly include stress, anxiety, and anger.[3]

Cancer is one of the leading causes of death in the world. In addition to surgery, chemotherapy, and hormone therapy, radiation therapy is one of the most important and common methods of cancer treatment.[4] The main duties of the officials in radiotherapy centers are to protect the health of patients, patient caregivers, and radiation oncology personnel.[5] Radiation oncologists point to the importance of educating patients and personnel to prevent infection in the radiotherapy environment during the COVID-19 outbreak.[6]

Since radiation therapy for cancer patients may take several weeks, they are more likely to develop acute coronavirus 2 (SARS-CoV-2) respiratory syndrome, and if they do, their treatment program is interrupted for several weeks, which can reduce the efficiency of radiation therapy.[7] Social isolation has also been shown to have a significant effect on cancer survival in general[89] and it can be expected that the psychological disorders caused by COVID-19 will also be able to affect the efficiency and program of treatment for patients undergoing radiation therapy.

Therefore, considering the consequences of the outbreak of COVID-19 on radiation therapy patients and also its potential impact on the quality of staff services, it seems that safety measures in radiotherapy centers are one of the most important measures to deal with COVID-19. According to what was stated, in this study, the investigation of safety measures among the employees of radiation therapy centers was considered through the questionnaire of safety measures. This questionnaire has been developed by the Federation of Africa Medical Physics Organizations (FAMPO) in five main areas including staff, radiation therapy center environment, treatment equipment and protocols, patient schedule, and training.[10] This questionnaire is an efficient tool for measuring safety in the radiation therapy center and is approved by FAMPO. Since it has not been used in Iran until now, it is necessary to carry out standardization in the first step. Therefore, this study aimed to investigate the psychometric properties of the COVID-19 Safety Measures (CSM) questionnaire among the staff of the radiation therapy center of Yazd, Iran.

Materials and Methods

Study design and setting

This analytical cross-sectional study was performed in October 2021 in the Radiation Therapy Center of Yazd, Iran. In this study, a questionnaire was used to check the level of compliance with safety measures. The safety measures questionnaire was prepared by the African Federation of Medical Physics Organizations (FAMPO).[10]

Study participants and sampling

The number of respondents to this questionnaire was 20 and for the test-retest method, 10 participants were selected and for the second time after 2 weeks, the questionnaire was completed.

Data collection tool and technique

In this study, the reliability and validity of a 16-item questionnaire about safety measures were examined in four steps.

Cultural adaptation

For cultural adaptation, the forward-backward method was used in five steps [Figure 1].[1112] First, two official bilingual translators translated the questionnaire from English to Persian (Forward translation). In the second step, the compatibility of the translations was examined by a committee consisting of translators and radiotherapists (radiologists and physicists). After agreeing on the translated words, in the third step, two other English translators, who were completely blind to the main questionnaire, translated the Persian version into English (Backward translation). In the fourth step, the compatibility of the translations was reviewed by the previous committee, and finally, in the fifth step, the native Persian words and terms were replaced. In items where some words were difficult to understand, brief descriptions were added.

Figure 1 Five steps of forward-backward method for cultural adaptation of questionnaire

Face validity

Face validity examines whether the appearance of the questionnaire and its items are suitable for the purpose of the questionnaire or not. Also, spelling, terminology, and punctuation errors are checked. To evaluate the face validity the final questionnaire was given to 10 experts and personnel who worked in medical physics and oncology centers.

To determine the face validity for each item, the impact score (IS) was calculated.[1314] For each item, five-point Likert scale range was considered. The scoring range included very important (score 5), important (score 4), moderately important (score 3), slightly important (score 2), and not important (score 1). After completing the questionnaire, IS was calculated using Equation 1.

Equation 1: IS calculating

IS = Frequency (%) × Importance

Where “Frequency” is the number of panelists who have given the item a score of 4 or 5, and the “Importance” is the average total score of panelists based on the Likert scale. Finally, items with an impact score greater than 1.5 were considered suitable for subsequent analysis.[1314]

Content validity

In this study, content validity of the items was assessed through content validity ratio (CVR) and content validity index (CVI). The CVR and CVI calculation formula are presented below:

Content validity ratio (CVR)

To calculate this index, the final questionnaire was given to 10 raters who were asked to choose one of the three options: “essential,” “useful but not essential,” and “not necessary;” then CVR was calculated using Equation 2[1315]:

Equation 2: CVR calculating

Where Ne is the number of raters who selected “Essential” for items and N is the total number of raters. According to the proposed method of Lawshe, depending on the number of raters involved in determining content validity (10 respondents), a threshold CVR of 0.62 is considered to confirm each item.[15]

Content validity index (CVI)

The item-level content validity index (I-CVI) was used for each item and the scale-level content validity index (I-CVI) was used for the total content validity index. The I-CVI, is the ratio of the number of experts who rated the item 4 or 5 to the total number of participants, After identifying the remaining items in the previous step, the S-CVI was calculated using Equation 3 based on the average method (S-CVI/Ave), which is the average of the I-CVI scores for all scale items on the scale.[1617] I-CVI of 0.79 was considered to be acceptable.

Equation 3: S- CVI calculating

Reliability

Kuder–Richardson (KR20) alpha was used to evaluate the internal consistency. It is a special case of Cronbach's α, computed for dichotomous scores. It is often claimed that a high KR-20 coefficient (e.g., >0.90) indicates a homogeneous questionnaire. For measures containing dichotomous options, the 1937-first-published Kuder–Richardson formulae offer a gauge of internal consistency dependability.[18] and to determine the stability reliability, the test-retest method was used and Pearson correlation coefficient and intraclass correlation coefficient (ICC) were calculated from a two-way mixed statistical model and absolute agreement type of ICC was selected.[19]

For the test-retest method, 10 participants were selected and for the second time after 2 weeks, the questionnaire was completed and then the ICC was calculated. An ICC more than 80% was considered as desirable.[19]

The final CSM questionnaire, after psychometric analysis and cultural adaptation, is presented in the appendix.

Statistical analysis

For descriptive statistics, mean, standard deviation, frequency and percentage were used. For face validity, all analyses were performed in Statistical Package for the Social Sciences (SPSS) software version 24 (SPSS 24.0; SPSS Inc., Chicago, Illinois, USA) with a significance level of 5%.

Ethical consideration

This study was conducted with the permission and support of Shahid Sadoughi University of Medical Sciences, Yazd, Iran (with the ethics code of IR.SSU.REC.1400.119).

Results

Participants

The number of respondents to this questionnaire was 20, of which 70% (14 people) were female. The highest age of the participant was 45 and the lowest was 25 years old. The mean (SD) age of the participants was 32.45 (6.35) years. A total of 75% (15 people) of the staff have less than 10 years of work experience. Furthermore, 50% of participants (10 people) had a history of COVID-19 in family and friends, 40% of participants (8 people) had a history of COVID-19 death in family and friends, and 15% (3 people) had a history of underlying disease. The frequency of demographic variables is listed in Table 1.

Table 1 Descriptive statistics of baseline characteristics

Variable	Levels	Frequency (percentage)	
Gender	Male
Female	6 (30)
14 (70)	
Age	20–30
31–40
41–50	11 (50)
6 (30)
3 (15)	
Underlying disease	I have
I do not have	3 (15)
17 (85)	
Work experience	<10 years
>10 years	15 (75)
5 (25)	
Marital status	Single
Married	5 (25)
15 (75)	
History of COVID-19 in family and friends	Yes
No	10 (50)
10 (50)	
History of death due to COVID-19 in family and friends	Yes
No	8 (40)
12 (60)	
The economic status	Weak
Medium
Good
Excellent	1 (5)
10 (50)
9 (45)
0	
Job	Doctor
Nurse
Physicist
Technologist	1 (5)
4 (20)
5 (25)
10 (50)	

Face validity

Table 2 shows the face validity of each item in the questionnaire. According to the impact score (IS) obtained for each item (more than 1.5), all items were accepted.

Table 2 Impact Score (IS) and I-CVR of each item in the COVID-19 Safety Measures questionnaire (CSM)

Item Number	Evaluation score	Mean	IS	CVR	I-CVI	Acceptability	
	
1	2	3	4	5	6	7	8	9	10	
1	5	5	5	5	5	5	5	4	5	5	4.9	4.9	0.8	1.0	Accepted	
2	5	5	4	5	5	5	5	5	5	5	4.9	4.9	0.8	1.0	Accepted	
3	5	5	4	5	5	5	5	5	5	5	4.9	4.9	0.8	1.0	Accepted	
4	5	5	3	5	5	5	5	5	5	5	4.8	4.3	0.8	0.9	Accepted	
5	5	5	5	5	5	5	5	5	5	5	5.0	5.0	1.0	1.0	Accepted	
6	5	5	5	5	5	5	5	5	5	5	5.0	5.0	1.0	1.0	Accepted	
7	5	5	5	5	5	5	5	5	5	5	5.0	5.0	1.0	1.0	Accepted	
8	5	5	5	5	5	5	5	5	5	5	5.0	5.0	1.0	1.0	Accepted	
9	5	5	5	5	5	5	3	5	5	5	4.8	4.3	0.8	0.9	Accepted	
10	5	5	5	5	5	5	4	5	5	5	4.9	4.9	0.8	1.0	Accepted	
11	5	5	5	5	5	5	5	3	5	5	4.8	4.3	0.8	0.9	Accepted	
12	5	5	5	5	5	5	5	5	5	5	5.0	5.0	1.0	1.0	Accepted	
13	5	5	5	5	5	5	3	5	5	5	4.8	4.3	0.8	0.9	Accepted	
14	5	5	5	5	5	5	4	5	5	5	4.9	4.9	0.8	1.0	Accepted	
15	5	5	5	5	5	5	4	5	5	5	4.9	4.9	0.8	1.0	Accepted	
16	5	5	5	5	5	5	5	5	5	5	5.0	5.0	1.0	1.0	Accepted	

Content validity

According to the proposed method of Lawshe, items with a CVR greater than 0.62 were accepted. As shown in Table 2, the CVR of all items was obtained above 0.62 and all items were accepted. In addition, S-CVI was equal to 0.97, which indicates that the S-CVI of the CSM questionnaire is at a high level.

Reliability

Internal consistency and test-retest method were used to assess reliability of questionnaire. The Kuder–Richardson alpha was obtained equal to 0.79, which is an acceptable value. Pearson correlation coefficient, intracluster correlation coefficient, and confidence interval were 0.64, 0.68, and 0.38–0.89, respectively.

Discussion

Cancer patients who refer to the radiation therapy centers are vulnerable people with weakened immune systems. Most of these patients are also elderly with underlying diseases. Therefore, it is important to ensure maximum safety measures in the face of COVID-19 for patients receiving radiotherapy services.[720] Conventional radiation therapy usually lasts several weeks and this increases the risk of getting acute respiratory syndrome of coronavirus (SARS-CoV-2). On the other hand, COVID-19 can delay their treatment program for a long time and therefore reduce the efficiency of radiotherapy.[7] Quarantine and the resulting social isolation have also shown a significant impact on cancer survival in general.[89]

Therefore, due to the negative consequences of COVID-19 on radiotherapy patients and also the personnel of these centers, safety measures in the face of COVID-19 are of great importance. The aim of this study was to determine the validity and reliability of the questionnaire of Hasford et al.[10] as a tool to assess the safety measures in radiation therapy centers in the face of COVID-19. In this study, face validity, content validity, internal reliability (by calculating Kuder–Richardson alpha), and stability reliability (by test-retest method and calculating ICC) were evaluated.

Following the COVID-19 outbreak, studies have been conducted to assess the knowledge and awareness of technologists, staff, patients, and their companions about the safety measures of health centers, including radiation therapy centers. In this regard, a questionnaire study was conducted in Turkey with the aim of assessing knowledge, attitude, function as well as emotional and psychological concerns related to COVID-19 among radiologists. There was a direct relationship between severe fear and poor mental status caused by COVID-19 and demographic factors such as family history of COVID-19, long working hours, lack of training on preventive measures, and ignorance of safety measures. The Cronbach's alpha of the questionnaire was 0.84.[21]

In 2021, Jazieh et al.[22] also used a questionnaire to assess preventive measures against COVID-19 in cancer patients. The results showed that increasing the knowledge of cancer patients and taking preventive measures are needed not only to reduce the risk of infection but also to minimize interruptions in their medical care. Cronbach's alpha related to the internal reliability of the questionnaire was 0.78. In an analytical cross-sectional study, Sharan Sah et al.[23] in April and May 2020 in Nepal, also examined patients’ adherence to control measures influenced by their knowledge, attitude, and function toward COVID-19 using a questionnaire. Cronbach's alpha coefficient calculated in this questionnaire was 0.85.

In the present study, the Kuder–Richardson coefficient for measuring the internal consistency of the preventive measures questionnaire was 0.79, which indicates the acceptable internal reliability.

In 2020, Jahangiri et al.[24] developed and validated the hospital occupational safety and health risk assessment (HOSHRA) index to assess occupational safety and health risk in 36 hospitals, with a CVI of 0.85 and an ICC of 0.99. Shali et al.[25] developed and evaluated the psychometric properties of a 58-item scale on patient's immunity deficiency in the Iranian medical oncology department. The IS of six items was less than 1.5 and was eliminated. Also, six more items were eliminated due to CVR less than 0.62 and finally, a CVI of 0.72 was computed.

In the present study, the IS of all items were high due to their acceptability, reasonableness, and comprehensibility. In addition, the CVR obtained for all items was higher than 0.62 and S-CVI equal to 0.97 indicating that the content validity index of the CSM questionnaire is at a high level.

To our knowledge, the present study is one of the first studies that have been performed on the reliability and validity of the CSM questionnaire in Iran. This standard study showed that this questionnaire can be used in studies to assess the safety measures of health centers in Iran.

Limitation and recommendation

The limitations of this study include the lack of cooperation of some participants, insufficient information of some personnel about the performance and facilities of the radiation therapy center, and the infection of some personnel with COVID-19, which prolonged the data collection process. Although in this study, sampling was done by census method, the number of staff (20 people) was not enough to perform confirmatory and exploratory factor analyses. In other words, data collection was performed from only one radiation therapy department [Figure 2]; therefore, it is suggested that further studies with larger sample sizes be performed in different areas.

Figure 2 Map of Iran and the region where the study was conducted (Yazd Province)

Conclusion

The validity and reliability of the CSM questionnaire were confirmed in this study. Therefore, the CSM 16-item questionnaire is proposed as a tool to assess the safety measures of radiation therapy and health centers.

Financial support and sponsorship

Shahid Sadoughi University of Medical Sciences has financially supported the work and this is stated in the acknowledgment section of the article.

Conflicts of interest

There are no conflicts of interest.

Acknowledgment

The staff of the Radiation Therapy Center of Yazd who patiently participated in this study is thanked and appreciated.

Appendix: The final COVID-19 Safety Measures (CSM) questionnaire, after psychometric analysis and cultural adaptation

Assessment Field	Item number	Item	Answer choices	
Staffing	1	Placement of staff under a shift system to avoid congestion	Yes□/No□	
2	Use of personal protective equipment (PPE) by radiation therapy personnel	Yes□/No□	
3	Availability of facilities for staff to work virtually/remotely	Yes□/No□	
Radiotherapy environment	4	Availability of screening facilities for patients, caregivers and staff (temperature check, triaging, etc.)	Yes□/No□	
5	Availability of hand washing and disinfection facilities	Yes□/No□	
6	Frequent disinfection of radiotherapy facilities after use	Yes□/No□	
7	Practicing of social/physical distancing protocols in radiotherapy center	Yes□/No□	
8	Implement patient companionship restrictions	Yes□/No□	
9	Availability of isolation unit to hold suspected COVID-19 cases	Yes□/No□	
Radiotherapy equipment and treatment protocols	10	Availability of equipment for suspicious/confirmed COVID-19 cases	Yes□/No□	
11	Availability of developed and/or adopted protocols to guide working procedures in radiotherapy center	Yes□/No□	
Patient condition and scheduling	12	Reorganization of patient presence based on patient condition	Yes□/No□	
13	Patient access to radiotherapy center by prior appointment	Yes□/No□	
14	Availability of remote means of patient follow-up after radiation therapy	Yes□/No□	
Education/ sensitization	15	Education of patients and caregivers on protocols/guidelines to performance when in the radiotherapy center	Yes□/No□	
16	Adequate education of staff on COVID-19 safety measures	Yes□/No□
==== Refs
References

1. Zhan C Tse CK Fu Y Lai Z Zhang H Modeling and prediction of the 2019 coronavirus disease spreading in China incorporating human migration data PloS One 2020 15 e0241171 33108386
2. Shahriarirad R Khodamoradi Z Erfani A Hosseinpour H Ranjbar K Emami Y Epidemiological and clinical features of 2019 novel coronavirus diseases (COVID-19) in the South of Iran BMC Infect Dis 2020 20 1 12
3. Ghebreyesus TA Addressing mental health needs: An integral part of COVID-19 response World Psychiatry 2020 19 129 30 32394569
4. Delaney GP Barton MB Evidence-based estimates of the demand for radiotherapy Clin Oncol 2015 27 70 6
5. Chen HH Kuo MT Improving radiotherapy in cancer treatment: Promises and challenges Oncotarget 2017 8 62742 28977985
6. Wu S Zheng D Liu Y Hu D Wei W Han G Radiation therapy care during a major outbreak of COVID-19 in Wuhan Adv Radiat Oncol 2020 5 531 3 32775770
7. Bese NS Hendry J Jeremic B Effects of prolongation of overall treatment time due to unplanned interruptions during radiotherapy of different tumor sites and practical methods for compensation Int J Radiat Oncol Biol Phys 2007 68 654 61 17467926
8. Moore S Leung B Bates A Ho C Social isolation: Impact on treatment and survival in patients with advanced cancer J Clin Oncol 2018 36 34-Suppl 156
9. Kotronoulas G Papadopoulou C Burns-Cunningham K Simpson M Maguire R A systematic review of the supportive care needs of people living with and beyond cancer of the colon and/or rectum Eur J Oncol Nurs 2017 29 60 70 28720267
10. Hasford F Ige TA Trauernicht C Safety measures in selected radiotherapy centres within Africa in the face of Covid-19 Health Technol 2020 10 1391 6
11. Ghanbari-Firoozabadi M Mirzaei M Nasab MV Grace SL Okati-Aliabad H Madadizadeh F Cross-cultural adaptation and psychometric validation of the Persian version of the Cardiac Rehabilitation Barriers Scale (CRBS-P) BMJ Open 2020 10 e034552 doi: 10.1136/bmjopen-2019-034552
12. Salaree MM Nir MS Sobhani V Delavari AA Mosavi SAR Translation, cross-cultural adaptation and validation of the neck bournemouth questionnaire: Persian version. J Caring Sci 2021 10 205 9
13. MOUSAZADEH Somayeh ; RAKHSHAN Mahnaz ; MOHAMMADI Fateme Investigation of content and face validity and reliability of sociocultural attitude towards appearance questionnaire-3 (SATAQ-3) among female adolescents Iranian journal of psychiatry 2017 12 1 15 28496497
14. Ebadi A Taghizadeh Z Mohammadi E Pourreza A Lili AK Bagherzadeh R Designing and psychometric analysis of a married women's work-family conflict questionnaire Nurs Midwifery Stud 2018 7 24 32
15. Lawshe CH A quantitative approach to content validity Pers Psychol 1975 28 563 75
16. Yusoff MSB ABC of content validation and content validity index calculation Resource 2019 11 49 54
17. Polit DF Beck CT The content validity index: Are you sure you know what's being reported? Critique and recommendations Res Nurs Health 2006 29 489 97 16977646
18. Feldt LS The approximate sampling distribution of Kuder-Richardson reliability coefficient twenty Psychometrika 1965 30 357 70 5216223
19. Ritter NL Understanding a widely misunderstood statistic: Cronbach's Online Submission; 2010
20. Liang W Guan W Chen R Wang W Li J Xu K Cancer patients in SARS-CoV-2 infection: A nationwide analysis in China Lancet Oncol 2020 21 335 7 32066541
21. Akyurt N Knowledge, practice and emotional status related to COVID-19 pandemic among radiology technicians working at pandemic hospitals Eur J Radiol 2021 134 109431 33259991
22. Jazieh AR Bensalem AA Bounedjar A Benbrahim Z Alorabi MO Abusanad AM Predictors of poor precautionary practices towards COVID-19 among cancer patients Future Oncol 2021 17 4871 82 34472365
23. Sah GS Shrestha G Dhakal A Mulmi R Sapkota A Poudel S Knowledge, attitudes, and practices of cancer patients towards COVID-19: A cross-sectional study in central Nepal Cancer Manag Res 2020 12 10173 33116872
24. Jahangiri M Mostafavi A Choobineh A Shakerisn M Tabatabaei HR Zare Derisi F Development and validation of HOSHRA index for occupational safety and health risk assessment in hospitals Shiraz Med J 2020 21 e95357
25. Shali M Ghaffari F Joolaee S Ebadi A Development and psychometric evaluation of the patient safety violation scale in medical oncology units in Iran Asian Pac J Cancer Prev 2016 17 4341 7 27797241
