
==== Front
J Clin Neurol
J Clin Neurol
JCN
Journal of Clinical Neurology (Seoul, Korea)
1738-6586
2005-5013
Korean Neurological Association

39227331
10.3988/jcn.2023.0339
Original Article
Validity and Reliability of the Korean Versions of the 9- and 19-Item Wearing-Off Questionnaires in Parkinson’s Disease
https://orcid.org/0000-0001-8738-5422
Park Jinse a*†
https://orcid.org/0000-0002-4808-7083
Jang Wooyoung b*†
https://orcid.org/0000-0003-3350-5032
Youn Jinyoung cd†
https://orcid.org/0000-0003-2068-3235
Oh Eungseok e†
https://orcid.org/0000-0002-6391-557X
Park Suyeon fg†
https://orcid.org/0000-0002-1566-6265
Oh Yoonsang h†
https://orcid.org/0000-0002-7722-4841
Kim Hee-Tae i
https://orcid.org/0009-0005-2162-8504
Lim Soohyun a
a Department of Neurology, Haeundae Paik Hospital, Inje University College of Medicine, Busan, Korea.
b Department of Neurology, Gangneung Asan Hospital, University of Ulsan College of Medicine, Gangneung, Korea.
c Department of Neurology, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, Korea.
d Neuroscience Center, Samsung Medical Center, Seoul, Korea.
e Department of Neurology, Chungnam National University Hospital, Chungnam National University College of Medicine, Daejeon, Korea.
f Department of Biostatistics, Arcademic Research Office, Soonchunhyang University, Seoul, Korea.
g Department of Applied Statistics, Chung-Ang University, Seoul, Korea.
h Department of Neurology, College of Medicine, The Catholic University of Korea, Seoul, Korea.
i Department of Neurology, Hanyang University Hospital, Hanyang University College of Medicine, Seoul, Korea.
Correspondence: Jinyoung Youn, MD, PhD. Department of Neurology and Neuroscience Center, Samsung Medical Center, Sungkyunkwan University School of Medicine, 81 Irwon-ro, Gangnam-gu, Seoul 06351, Korea Tel +82-2-3410-1034, Fax +82-2-3410-0055, genian@skku.edu
Correspondence: Eungseok Oh, MD, PhD. Department of Neurology, Chungnam National University Hospital, Chungnam National University College of Medicine, 282 Munhwa-ro, Jung-gu,Daejeon 35015, Korea. Tel +82-42-280-7868, Fax +82-42-252-3654, massive@cnuh.co.kr
*These authors contributed equally to this work.

†KOJYP study group

9 2024
19 7 2024
20 5 487492
31 8 2023
27 3 2024
12 4 2024
Copyright © 2024 Korean Neurological Association
2024
Korean Neurological Association
https://creativecommons.org/licenses/by-nc/4.0/ This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (https://creativecommons.org/licenses/by-nc/4.0) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.
Background and Purpose

The wearing-off (WO) phenomenon is the most common motor complication in advanced Parkinson’s disease (PD), but its identification remains challenging. The 9- and 19-item Wearing-off Questionnaires (WOQ-9 and WOQ-19) are self-assessment tools for motor and nonmotor symptoms that are widely used for WO screening. We produced Korean versions of the WOQ-19 and WOQ-9 (K-WOQ-19 and K-WOQ-9) and investigated their validity and reliability.

Methods

We used the translation–back translation method to produce K-WOQ-19 and K-WOQ-9, which were self-administered by 124 patients with PD. We conducted in-depth 10-minute interviews for confirming the presence of the WO phenomenon, and then stratified the participants into groups with and without WO. Diagnostic accuracy was assessed byanalyzing receiver operating characteristic curves. Concurrent validity was assessed using the Movement Disorder Society–Unified Parkinson’s Disease Rating Scale (MDS-UPDRS) andthe Hoehn and Yahr stage with Spearman’s rank correlation analysis. Reliability was assessedbased on test–retest Cohen’s kappa (κ) values and intraclass correlation coefficients (ICCs).

Results

The optimal cutoff scores on the K-WOQ-19 and K-WOQ-9 for WO screening were 4 and 2, respectively. The test–retest ICCs of K-WOQ-19 and K-WOQ-9 were 0.943 and 0.938, respectively. Nineteen of the combined 20 items in K-WOQ-19 and K-WOQ-9 showed moderate-to-substantial agreement (κ=0.412–0.771, p<0.001). The scores on the translated scales were significantly correlated with MDS-UPDRS IV scores.

Conclusions

K-WOQ-19 and K-WOQ-9 are reliable and valid tools for detecting WO, with optimal cutoff scores of 4 and 2, respectively.

Graphical Abstract

Parkinson’s disease
surveys and questionnaires
sensitivity and specificity
Korea Health Industry Development Institute https://doi.org/10.13039/501100003710 HI23C0092
==== Body
pmcINTRODUCTION

Motor fluctuations are one of the motor complications in Parkinson’s disease (PD), and are characterized by unpredictable changes in motor symptoms due to changes in drug efficacy. Up to 80% of patients with PD experience motor fluctuations within 5 years of disease onset.1 Among various motor fluctuations, the most common is the wearing-off (WO) phenomenon, also known as end-of-dose WO, and defined as the recurrence of motor or nonmotor symptoms before the time at which the next levodopa dose is to be taken or during the levodopa-free interval due to WO of the drug effects, particularly among patients experiencing disease progression.2 The WO phenomenon can have a significant impact on quality of life and the ability to perform the activities of daily living, which makes management of this phenomenon an important part of PD management.3

From a clinical perspective, the objective detection and estimation of the WO phenomenon are challenging since they require home-based long-term monitoring. Accordingly, several methods have been introduced for WO assessment, such as keeping a PD diary, clinical assessment, wearable devices, and the use of clinical scales. The PD diary is known to be the most accurate assessment tool, but it has the drawback of poor compliance.4 Wearable sensors are being introduced as a cutting-edge technology, but they have not yet been clinically validated.5 The most commonly used tools for WO assessment in clinical settings are therefore clinical scales.

The 32-item Wearing-off Questionnaire (WOQ-32) was the first self-administered questionnaire introduced for WO assessment.6 WOQ-32 contains detailed items for assessing WO symptoms, and the resulting considerable time required to complete the questionnaire makes it too cumbersome to use in clinical practice. Therefore, 19- and 9-item versions of the WOQ (WOQ-19 and WOQ-9) were subsequently developed to reduce the required screening time. These have a simple self-report questionnaire format for assessing motor and nonmotor symptoms that develop during the off period. Eight of the nine WOQ-9 items overlap those of WOQ-19, with the ninth being unique to WOQ-9. The validity and reliability of the original versions of these two scales have been well documented.7 In addition, both have been translated into many languages and are widely applied clinically.891011 However, there are currently no validated and reliable Korean versions of the WOQ-19 and WOQ-9.

In this study we aimed to produce Korean versions of WOQ-19 and WOQ-9 (K-WOQ-19 and K-WOQ-9) and determine their validity and reliability in patients with PD.

METHODS

Ethics statement

This study conformed with the principles set forth in the Declaration of Helsinki. All patients provided informed consent prior to enrollment, and the study was approved by the Institutional Review Board of Haeundae Paik Hospital (approval number: HP2021-11-006).

Study design and population

This multicenter, cross-sectional, observational study included patients with PD who visited five movement clinics in tertiary hospitals. The inclusion criteria were as follows: 1) diagnosis of PD according to the Movement Disorder Society (MDS) diagnostic criteria for PD,12 and 2) regular dopaminergic treatment. The exclusion criteria were as follows: 1) dementia, 2) presence of any other cause preventing self-administration of the questionnaires, 3) drug-resistant stage of PD, or 4) changing the antiparkinsonian drug before the second visit. Basic demographic and clinical factors, including age, sex, and disease duration, were obtained for all patients.

Translation into Korean

We used the translation–back translation method to produce K-WOQ-19 and K-WOQ-9. Four movement experts participated in the translation process, one of whom was a native Korean speaker who had lived in an English-speaking country. Two experts translated the original versions into Korean, and the other two retranslated them back into Korean. All four experts cross-checked the files for any inconsistencies and adapted the translated versions for Korean culture (Supplementary Material in the online-only Data Supplement).

Validity and reliability assessments

To assess the criterion validity of the translated scales, all patients were initially evaluated using the Movement Disorder Society–Unified Parkinson’s Disease Rating Scale parts I–IV (MDS-UPDRS I–IV), Hoehn and Yahr (H&Y) staging, Non-Motor Symptom Scale (NMSS), and 39-item Parkinson’s Disease Questionnaire.131415 The gold standard for defining WO was the clinical interview with patients. To confirm the presence of the WO phenomenon, the investigators conducted in-depth 10-minute interviews with patients and caregivers. WO was defined as the recurrence of motor or nonmotor symptoms before the next dose of dopaminergic treatment. Clinicians were blinded to the results on the clinical scales including K-WOQ-19 and K-WOQ-9 during the clinical interview. Patients were divided into two groups based on the presence or absence of WO symptoms, and they were then asked to complete K-WOQ-19 and K-WOQ-9. To assess the reliability of the translated scales, a second evaluation (retest) using both scales was performed within 1 month, and the results of the two evaluations were compared.

Statistical analyses

Categorical data of demographic factors were summarized as frequencies and percentages and compared using the chi-square or Fisher’s exact test. Continuous data of demographic factors were summarized as means and standard deviations or medians and interquartile ranges and compared using the independent t-test or the Mann–Whitney U test, respectively, depending on their distribution, which was evaluated using the Shapiro–Wilk test. To assess and compare the diagnostic value of the questionnaires, receiver operating characteristic (ROC) curve analysis was employed and the area under the ROC curve (AUC) was calculated for each questionnaire. Reliability was assessed as the test–retest reliability using Cohen’s kappa (κ) values for each item and intraclass correlation coefficients (ICCs) for the total score. Cohen’s κ was interpreted as follows: 0.2–0.4, fair; 0.4–0.6, moderate; 0.6–0.8, substantial; and 0.8–1.0, almost perfect.16 The test–retest reliability of the two questionnaires was considered acceptable when ICC was >0.5. Concurrent validity was assessed using Spearman’s rank correlation analysis with MDS-UPDRS IV.

All statistical analyses were performed using SPSS Statistics (version 26.0, IBM Corp., Armonk, NY, USA). Probability values of p<0.05 were considered to indicate statistical significance.

RESULTS

Patient characteristics

This study enrolled 124 patients with PD, of whom 73 did and 51 did not have WO symptoms. The mean age was 67.2 years and 58 (46.8%) were male. The patients’ characteristics according to the presence of WO symptoms are presented in Table 1. Among the investigated factors, age, disease duration, NMSS, and score on MDS-UPDRS IV differed between the WO and non-WO groups. Other clinical characteristics did not differ between two groups.

The most common WO symptom was “slowness of movement” (78.2%), followed by “reduced dexterity” (Fig. 1). The “experience of panic attacks” and “abdominal discomfort” were less frequent in the off state.

Diagnostic performance

In the ROC curve analysis, K-WOQ-19 and K-WOQ-9 demonstrated acceptable AUC values of 0.824 (95% confidence interval [CI]=0.746–0.887) and 0.799 (95% CI=0.718–0.866), respectively, for the optimal cutoff scores for detecting WO of 4 and 2, respectively (Fig. 2). Using a K-WOQ-19 cutoff score of 4 produced sensitivity, specificity, positive predictive value, and negative predictive value of 79.5%, 74.5%, 81.7%, and 71.7%, respectively; the corresponding values when the K-WOQ-9 cutoff score was 2 were 86.3%, 62.7%, 76.8%, and 76.2%.

Reliability assessment

The retest was completed by 107 of the 125 patients who underwent the first test, giving a retention rate of 85.6%. Among the combined 20 items in K-WOQ-19 and K-WOQ-9, Cohen’s κ values for 8 items showed moderate agreement and 10 items showed substantial agreement, with values ranging from 0.412 to 0.771 (p<0.001). The “slowness of movement” item yielded the highest κ (0.826, p<0.001), whereas the “numbness” item showed the lowest κ (0.270, p=0.002). The κ values for all items other than “numbness” exceeded 0.4, indicating moderate-to-substantial agreement. The test–retest ICCs for K-WOQ-19 and K-WOQ-9 were 0.943 and 0.938, respectively, indicating strong agreement (Table 2).

Validity assessment

Spearman’s rank correlation analysis revealed that K-WOQ-19 scores were significantly correlated with all parts of MDS-UPDRS and H&Y staging (Table 3). The same significant correlations were observed for K-WOQ-9, except for MDS-UPDRS III and H&Y staging. Both scales showed the strongest significant correlation with MDS-UPDRS IV, which is used to assess motor complications: r=0.657 and 95% CI=0.530–0.755 for K-WOQ-19, and r=0.617 and 95% CI=0.482–0.724 for K-WOQ-9.

DISCUSSION

In this study we translated WOQ-19 and WOQ-9 into Korean and confirmed the validity and reliability of the translated scales. These two scales also demonstrated acceptable diagnostic accuracy in detecting WO.

The K-WOQ-19 and K-WOQ-9 items are written in simple words and do not require an evaluator. However, the two tools are self-administered, and so their validity and reliability could not be evaluated by factor analysis or based on interrater reliability, respectively. Instead, we used Cohen’s κ values and ICCs to assess test–retest reliability and evaluated concurrent validity using clinical scales. Most K-WOQ-19 items and all K-WOQ-9 items showed moderate-to-substantial agreement with κ>0.4, and both scales showed strong agreement since the ICCs were high. Although the AUC analysis indicated that the overall diagnostic accuracy was slightly higher for K-WOQ-19 than for K-WOQ-9, the difference was not statistically significant. The overall diagnostic accuracies of both K-WOQ-19 and K-WOQ-9 were not different significantly.

Among the 20 items across both scales, motor symptoms, particularly cardinal symptoms of PD, were more common manifestations of WO than were nonmotor symptoms. This is consistent with a previous study finding that slowness of movement was the most common WO symptom, while sensory-related nonmotor symptoms were the least common.17 It was particularly interesting that postural instability was the third most common WO symptom in our study, while axial symptoms are known to be unresponsive to levodopa. This finding might have been due the enrolled patients with PD being in the honeymoon phase and in a period of motor complications, and so showed levodopa responses.

In line with our expectations of concurrent validity, K-WOQ-19 and K-WOQ-9 exhibited the strongest correlations with MDS-UPDRS IV, which is the sole test within UPDRS that quantitatively measures motor complications.

In our ROC curve analysis, the optimal cutoff scores on K-WOQ-19 and K-WOQ-9 for WO screening were 4 and 2, respectively, which are higher than those reported for the original and other versions of these scales. For example, a systematic review of studies using other versions of WOQ-19 and WOQ-9 found that their optimal cutoff scores were 2 and 1, respectively.7 Setting the cutoff scores for K-WOQ-19 and K-WOQ-9 to 2 and 1, like in these previous reports, resulted in the sensitivity remaining relatively high while the specificity became extremely low. Setting the K-WOQ-19 and K-WOQ-9 cutoff scores to 4 and 2, respectively, resulted in the sensitivity remaining relatively high at 79.5% and 86.3%, while the specificity increased to 74.5% and 62.7%, respectively.

These findings may be associated with linguistic characteristics of the Korean language. Some expressions used to describe nonmotor symptoms are vague and overlap, resulting in patients selecting multiple items even when their actual symptoms are less severe. When translated into Korean, “aching” and “pain” become similar expressions, making it difficult to distinguish between them as separate words. Additionally, the Korean phrase for “reduced dexterity” is not frequently used in daily life, which could lead to confusion with “slowness of movement.” We presumed that this represents a limitation inherent to using K-WOQ-19 and K-WOQ-9 as simple screening tools.

Previous studies found that WOQ-9 exhibited high sensitivity and low specificity, while WOQ-19 exhibited an acceptable level of specificity.7 Consistent with previous reports, we also found relatively high sensitivity and low specificity values for K-WOQ-9. However, both scales are suitable as screening tools for detecting WO with appropriate levels of sensitivity and specificity.

The limitations of our study include the following: First, the total sample of patients without WO was relatively small, which may have reduced the statistical power in detecting significant differences between groups. Second, we could not assess home-based PD diaries, which are considered the most reliable tool for motor complications. However, such diaries are self-reports completed by patients, and hence also have inherent accuracy limitations.4 To ascertain the precise occurrence of WO, we conducted in-depth interviews lasting 10 minutes in this study.

In conclusion, K-WOQ-19 and K-WOQ-9 are rapid, convenient, and easily applied self-administered tests for use in clinical settings, which we found to show high validity and reliability. The accuracy in detecting WO was similar for the two scales, and both scales were suitable as screening tools. the ICCs were high in reliability assessments of the total scores, but Cohen’s κ values were relatively low in item-specific reliability assessments. Additionally, the redundancy and ambiguity of Korean expressions resulted in the optimal cutoff score being higher than in other language versions of WOQ-19 and WOQ-9. It is recommended to use a suitable screening tool at the stage where motor complications are suspected, and use this cautiously as a monitoring tool. Our results are expected to be widely applicable in both clinical practice and research for the management of WO in patients with PD.

Acknowledgements

We express our gratitude to ACE Statistical Consulting for their contribution to the statistical analyses performed in this study.

Availability of Data and Material

The datasets generated or analyzed during the study are available from the corresponding author on reasonable request.

Supplementary Materials

The online-only Data Supplement is available with this article at https://doi.org/10.3988/jcn.2023.0339.

SUPPLEMENTARY MATERIALS

Fig. 1 Frequency of symptoms on the K-WOQ-19 and K-WOQ-9. Slowness of movement was the most common symptom and experience of panic attacks was the least common symptom on both K-WOQ-19 and K-WOQ-9. *Items of K-WOQ-9. K-WOQ-19, Korean version of 19-item Wearing-off Questionnaire; K-WOQ-9, Korean version of 9-item Wearing-off Questionnaire.

Fig. 2 In the ROC curve analysis, K-WOQ-19 and K-WOQ-9 demonstrated acceptable AUC values of 0.824 (95% CI=0.746–0.887) and 0.799 (95% CI=0.718–0.866), respectively, for the optimal cutoff scores for detecting WO of 4 and 2, respectively. AUC, area under the ROC curve; CI, confidence interval; K-WOQ-19, Korean version of 19-item Wearing-off Questionnaire; K-WOQ-9, Korean version of 9-item Wearing-off Questionnaire; ROC, receiver operating characteristic; WO, wearing-off.

Table 1 Patients’ baseline characteristics

Characteristic	Overall	Presence of WO	No	p	
Yes	
All patients	124 (100.0)	73 (58.9)	51 (41.1)		
Sex				0.129*	
	Male	58 (46.8)	30 (41.1)	28 (54.9)		
	Female	66 (53.2)	43 (58.9)	23 (45.1)		
Age (yr)	67.2±8.0	65.7±8.6	69.3±6.7	0.002†	
Disease duration (yr)	6.7±4.2	7.8±4.4	5.0±3.4	<0.001†	
Education duration (yr)	11.5±3.7	11.3±3.6	11.9±4.0	0.205†	
MDS-UPDRS I score	6.4±5.3	6.8±5.7	5.6±4.4	0.466†	
MDS-UPDRS II score	8.0±6.8	8.7±7.3	6.7±5.7	0.158†	
MDS-UPDRS III score	19.9±12.1	20.9±12.7	18.4±11.2	0.276†	
MDS-UPDRS IV score	3.9±4.1	6.2±3.7	0.5±1.2	<0.001†	
H&Y stage	2.1±0.7	2.2±0.7	2.0±0.7	0.194†	
NMSS score	32.8±27.3	37.9±30.0	25.3±20.7	0.018†	
PDQ-39 score	28.0±27.4	31.1±29.2	21.8±22.7	0.165†	
Data are mean±standard deviation or n (%) values except where indicated otherwise. Consistency with a normal distribution was evaluated using the Shapiro–Wilk test.

*p values were derived from chi-square test; †p values were derived from Mann–Whitney’s U test.

H&Y, Hoehn and Yahr; MSD-UPDRS, Movement Disorder Society–Unified Parkinson’s Disease Rating Scale; NMSS, Non-Motor Symptom Scale; PDQ-39, 39-item Parkinson’s Disease Questionnaire; WO, wearing-off.

Table 2 ICCs for test–retest reliability

	ICC	95% CI	p	
K-WOQ-19	0.943	0.917–0.961	<0.001	
K-WOQ-9	0.938	0.909–0.957	<0.001	
CI, confidence interval; ICCs, intraclass correlation coefficients; K-WOQ-19, Korean version of 19-item Wearing-off Questionnaire; K-WOQ-9, Korean version of 9-item Wearing-off Questionnaire.

Table 3 Spearman’s correlation coefficients

Variable	K-WOQ-19	K-WOQ-9	
Spearman’s r	95% CI	Spearman’s r	95% CI	
MDS-UPDRS I score	0.467**	0.284–0.617	0.421**	0.233–0.579	
MDS-UPDRS II score	0.410**	0.220–0.570	0.338**	0.142–0.509	
MDS-UPDRS III score	0.233**	0.056–0.396	0.158	-0.021–0.327	
MDS-UPDRS IV score	0.657**	0.530–0.755	0.617**	0.482–0.724	
H&Y stage	0.201*	0.023–0.367	0.174	-0.004–0.342	
*p<0.05; **p<0.01.

CI, confidence interval; H&Y, Hoehn and Yahr; K-WOQ-19, Korean version of 19-item Wearing-off Questionnaire; K-WOQ-9, Korean version of 9-item Wearing-off Questionnaire; MDS-UPDRS, Movement Disorder Society–Unified Parkinson’s Disease Rating Scale.

Author Contributions: Conceptualization: Jinyoung Youn, Jinse Park.

Data curation: Soohyun Lim.

Investigation: Suyeon Park, Wooyoung Jang.

Methodology: Eungseok Oh, Yoonsang Oh.

Supervision: Hee-Tae Kim.

Writing—original draft: Jinse Park.

Writing—review & editing: Jinyoung Youn.

Conflicts of Interest: The authors have no potential conflicts of interest to disclose.

Funding Statement: This work was supported by the Korea Health Technology R&D Project through the Korea Health Industry Development Institute (KHIDI), funded by the Ministry of Health & Welfare, Republic of Korea (grant number: HI23C0092).
==== Refs
1 Kim HJ Mason S Foltynie T Winder-Rhodes S Barker RA Williams-Gray CH Motor complications in Parkinson’s disease: 13-year follow-up of the CamPaIGN cohort Mov Disord 2020 35 185 190 31965629
2 Chou KL Stacy M Simuni T Miyasaki J Oertel WH Sethi K The spectrum of “off” in Parkinson’s disease: what have we learned over 40 years? Parkinsonism Relat Disord 2018 51 9 16 29456046
3 Chapuis S Ouchchane L Metz O Gerbaud L Durif F Impact of the motor complications of Parkinson’s disease on the quality of life Mov Disord 2005 20 224 230 15384126
4 Löhle M Bremer A Gandor F Timpka J Odin P Ebersbach G Validation of the PD home diary for assessment of motor fluctuations in advanced Parkinson’s disease NPJ Parkinsons Dis 2022 8 69 35654835
5 Adams JL Lizarraga KJ Waddell EM Myers TL Jensen-Roberts S Modica JS Digital technology in movement disorders: updates, applications, and challenges Curr Neurol Neurosci Rep 2021 21 16 33660110
6 Stacy M Bowron A Guttman M Hauser R Hughes K Larsen JP Identification of motor and nonmotor wearing-off in Parkinson’s disease: comparison of a patient questionnaire versus a clinician assessment Mov Disord 2005 20 726 733 15719426
7 Mantese CE Schumacher-Schuh A Rieder CRM Clinimetrics of the 9- and 19-item wearing-off questionnaire: a systematic review Parkinsons Dis 2018 2018 5308491 29808113
8 Ha ULN Tran TN Le M Dang THT Vũ NA Truong D Validating the Vietnamese version of wearing–off 19 questionnaire for patients with Parkinson’s disease Clin Park Relat Disord 2019 1 37 40 34316597
9 Remigio KC Jamora RDG Philippine translation and validation of the wearing-off questionnaire-19 Clin Park Relat Disord 2019 1 21 24 34316594
10 Santos JG Chien HF Barbosa ER Specificity and sensibility of 9-itens wearing-off questionnaire in Brazilian Parkinson disease patient sample Arq Neuropsiquiatr 2014 72 867 873 25410454
11 Fukae J Higuchi MA Yanamoto S Fukuhara K Tsugawa J Ouma S Utility of the Japanese version of the 9-item wearing-off questionnaire Clin Neurol Neurosurg 2015 134 110 115 25985062
12 Postuma RB Berg D Stern M Poewe W Olanow CW Oertel W MDS clinical diagnostic criteria for Parkinson’s disease Mov Disord 2015 30 1591 1601 26474316
13 Park J Koh SB Kwon KY Kim SJ Kim JW Kim JS Validation study of the official Korean version of the Movement Disorder Society-unified Parkinson’s disease rating scale J Clin Neurol 2020 16 633 645 33029970
14 Chaudhuri KR Martinez-Martin P Brown RG Sethi K Stocchi F Odin P The metric properties of a novel non-motor symptoms scale for Parkinson’s disease: results from an international pilot study Mov Disord 2007 22 1901 1911 17674410
15 Park HJ Sohng KY Kim S Validation of the Korean version of the 39-item Parkinson’s disease questionnaire (PDQ-39) Asian Nurs Res (Korean Soc Nurs Sci) 2014 8 67 74 25030495
16 McHugh ML Interrater reliability: the kappa statistic Biochem Med (Zagreb) 2012 22 276 282 23092060
17 Altavista MC Cassetta E Brusa L Viselli F Denaro A Ventriglia M Wearing-off detection in clinical practice: the wearing off real practice key (WORK-PD) study in Parkinson’s disease Parkinsonism Relat Disord 2015 21 95 100 25443558
