
==== Front
Int J Sports Phys Ther
Int J Sports Phys Ther
2159
International Journal of Sports Physical Therapy
2159-2896
NASMI Website: International Journal of Sports Physical Therapy

35136697
30998
10.26603/001c.30998
Original Research
Adjacent Joint Restriction Differentially Influences Intra- and Inter-rater Reliability and Agreement of Goniometric Measurements
https://orcid.org/0000-0002-0412-4663
Pinto Brendan L. 1
Stankovic Tatjana 2
Frost David M. 2
https://orcid.org/0000-0002-0925-3018
Beach Tyson A. C. 1
1 Department of Kinesiology University of Waterloo https://ror.org/01aff2v68
2 Faculty of Kinesiology and Physical Education University of Toronto https://ror.org/03dbr7087
Corresponding author: Tyson Beach, PhD Department of Kinesiology University of Waterloo 200 University Avenue West Waterloo, ON N2L3G1 CANADA E-mail: tyson.beach@uwaterloo.ca
1 2 2022
2022
17 2 276285
19 7 2021
14 10 2021
© The Author(s)
https://creativecommons.org/licenses/by-nc/4.0/ This is an open access article distributed under the terms of the Creative Commons Attribution-NonCommercial License (4.0) which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and source are credited.

Background

Reliability and agreement of goniometric measurements can be altered by variations in measurement technique such as restricting adjacent joints to influence bi-articular muscles. It is unknown if the influence of adjacent joint restriction is consistent across different range of motion (ROM) tests, as this has yet to be assessed within a single study. Additionally, between-study comparisons are challenged by differences between methodology, participants and raters, obscuring the development of a conceptual understanding of the extent to which adjacent joint restriction can influence goniometric ROM measurements.

Purpose

To quantify intra- and inter-rater reliability and levels of agreement of goniometric measurements across five ROM tests, with and without adjacent joint restriction.

Study Design

Descriptive reliability study

Methods

Three trained and experienced raters made two measurements of bilateral ankle dorsiflexion, first metatarsophalangeal dorsiflexion, hip extension, hip flexion, and shoulder flexion, with and without adjacent joint restriction. Intraclass correlation coefficient (ICC), standard error of measurement (SEM), along with participant, measurement/rater and random error variance were estimated.

Results

Eleven females (age 21.4 ±2.3 years) and 19 males (age 22.1 ±2.8 years) participated. Adjacent joint restriction did not influence the reliability and agreement in a consistent way across the five ROM tests. Changes in the inter-rater reliability and agreement were more pronounced compared to the intra-rater reliability and agreement. Assessing variance components (participant, measurement/rater and random error variance) that are used to calculate the ICC and SEM, improved interpretation of ICC and SEM scores.

Conclusion

The effects of adjacent joint restriction on reliability and agreement of goniometric measurements depend on the ROM test and should be considered when comparing measurements between multiple raters. Reporting variance components that are used to calculate the ICC and SEM can improve interpretation and may improve between-study comparisons, towards developing a conceptual framework to guide goniometric measurement technique.

Level of Evidence

3b

range of motion
repeatability
error
examiner
consistency
==== Body
pmcINTRODUCTION

The universal goniometer is a portable, low cost and easy to use tool which can quantify joint range of motion (ROM). This has favoured its use in clinical settings as well as in research.1,2 Although it can have a close association with technologically advanced techniques,3,4 the measurement reliability and agreement of goniometry can vary due to multiple factors. These factors include the rater, the individual being measured, the joint being measured and differences in ROM measurement technique such as test posture, imposed restraints (e.g. adjacent joint(s) restriction) and assistance (e.g. active vs. passive).5 It is important to understand the extent to which these different factors can influence goniometry to inform best practices in the clinic and research.

Intra-rater reliability and agreement has consistently been shown to be higher than inter-rater reliability and agreement.5–8 Pain and spasticity have the potential to reduce reliability.5 Each joint can have different characteristics that can influence alignment and placement of the goniometer to influence ROM measurements.9 Similarly, different ROM tests measuring different ranges of the same joint (i.e. movement directions such as flexion or extension) can potentially influence reliability and agreement due to different position and goniometer alignment requirements. However, the extent to which different types of techniques can influence goniometry is uncertain. For instance, restricting adjacent joints can be used to assess the flexibility and the contribution of bi-articular and mono-articular muscles to the joint ROM which can inform on function and risk of muscle injury.10 Yet, it is unknown if restricting adjacent joints influences measurement reliability and agreement in a consistent way across different ROM measurements.

Comparing results between studies is challenged by differences between methodology, participants and the raters, as these all have the potential to influence the intraclass correlation coefficient (ICC) and the standard error of measurements (SEM) which have been recommended as the preferred estimate of reliability and agreement.11–13 The ICC is calculated as the ratio of between-participant variance to total variance, where the total variance is the sum of the between participant variance and error variance.14 When there are an equal number of repeated measurements for each participant (at least two per participant), a two-way model can be used to partition the error into systematic error and random error,13,14 such that the absolute agreement, two-way random effects, single measurement ICC (i.e. ICC[2,1]) can be calculated as:

ICC[2,1]=σp2σp2+σr2+σe2

Where σ2p is the variance between participants, σ2r is the systematic error variance due to systematic differences between the measurements or raters being compared, and σ2e is the random error variance.11,13,15 There are other formulas for calculating the ICC that reduce the influence of σ2r and σ2e which results in an increased ICC score (e.g. ICC[3,1] and ICC[2,k]).13 However, the use and reporting of ICCs between studies is inconsistent and often lacks sufficient detail.2,16,17 Additionally, the SEM can be calculated as SEM= SDpooled x 1−ICC.14 Hence, both the ICC and SEM are influenced by any changes that could influence the components of variance within the data. For this reason, differences between the methodology, participants and raters, challenges the comparison of ICC and SEM scores between studies. Further, few studies usually report the components of variance which limit the consolidation of different or conflicting results.11

There is a paucity of research investigating the effect of technique, such as adjacent joint restriction, across more than one joint or ROM test within a single study to overcome the difficulties in comparing results between different studies.5 Additionally, differences between restricting and not restricting the adjacent joint could influence the variance between participants (e.g. due to differences in bi-articular muscle flexibility leading to changes in ROM), variance between measurements/raters and random error in different ways. Thus, understanding if the effect is consistent across different ROM tests and investigating why any changes occur can contribute to the conceptual framework used to guide goniometry in research and practice.

The purpose of this study was to quantify intra- and inter-rater reliability and levels of agreement of goniometric measurements across five ROM tests, with and without adjacent joint restriction. Using the same participants and raters as well as reporting the components of variance may improve understanding of whether adjacent joint restriction consistently influences goniometer measurement reliability and agreement across different ROM tests. This knowledge may contribute to creating a conceptual framework to guide effective use of goniometers.

MATERIALS AND METHODS

A convenience sample of 30 healthy participants provided written informed consent. All procedures were approved by the University of Toronto’s Office of Research Ethics. Two Certified Athletic Therapists and one Registered Physiotherapist rated each participant in a random order. Raters had similar field experience and worked interchangeably within the same clinic. Raters underwent specific training for this study, which included standard instructions for each ROM test (Appendix A) and a practice session involving three participants who were not part of the study. To control for differences in repositioning participants between measurements, a trained research assistant positioned all participants. A second research assistant documented the measured values.

Participants performed a warm up session upon arrival that consisted of five minutes of light stationary cycling and 10 repetitions of bodyweight squats, lunges, calf-raises and arm circles. The ROM tests included ankle dorsiflexion, first metatarsophalangeal (MTPJ1) dorsiflexion, hip extension, hip flexion, and shoulder flexion. Each ROM test was performed twice bilaterally, with and without restraining the adjacent joint position. The left side was measured first, followed by the right side, and then both sides were repeated to obtain two measurements per side. This repositioning between each measurement avoided residual effects from holding the testing position (i.e. stretching). Each rater performed all measurements in the same order but rated each participant in a random order with approximately two minutes of rest between each rater. This allowed for equal time between each rater’s evaluation of each ROM test.

Details of the testing positions and goniometer placements can be found in Appendix A. The following provides a brief description:

Ankle dorsiflexion

Measured in a lunge/split stance. Unrestricted ROM was measured on the front foot, while the rear foot was used to measure restricted ROM by keeping the rear knee extended to influence the gastrocnemius muscles (Figure 1A).

MTPJ1 dorsiflexion

Measured while participants sat on a table with their feet hanging freely above the floor. Unrestricted ROM was measured with the ankle supported in its natural hanging position, whereas the restricted ROM was measured in maximal ankle dorsiflexion to influence flexor hallucis longus (Figure 1B).

Hip extension

Measured with the participant lying supine with their hips on the edge of a table and a rolled towel beneath their lower back. The test leg hung off of the edge while the non-test leg was supported in hip and knee flexion by the research assistant. Unrestricted hip extension was measured by pushing the hanging leg down until the pelvis started to rotate anteriorly, while controlling for leg rotation. Restricted hip extension was performed in the same manner but while holding the hanging leg in 90° of knee flexion to influence rectus femoris (Figure 1C).

Hip flexion

Tested in a supine position. The test hip was flexed with the test side knee either flexed or extended to influence the hamstring muscle group, for the unrestricted and restricted measurements respectively (Figure 1D).

Shoulder flexion

Measured while participants sat with their feet firmly on the ground. The research assistant supported the scapula while raising the participants’ arm with the elbow either extended or flexed to influence the triceps brachii, for the unrestricted and restricted measurements respectively (Figure 1E).

78043 Figure 1. Participant position and goniometer placement for unrestricted and restricted (A) ankle dorsiflexion (B) MTPJ1 dorsiflexion, (C) hip extension, (D) hip flexion and (E) shoulder flexion.

To determine the effects of adjacent joint restriction on intra- and inter-rater reliability and agreement, changes in ICC and SEM scores were assessed between the unrestricted and restricted conditions across the five ROM tests. An absolute agreement, two-way random effects, single measurement ICC (i.e. ICC [2,1]) was used to measure intra- and inter-rater reliability (psych package, RGui Version 4.0.2, The R Foundation, Vienna, Austria). Inter-rater ICC was calculated using the mean of each rater’s two measurements to account for the deviation within their respective measurements. ICC scores were interpreted as <0.40 is poor, 0.41-0.59 is fair, 0.60-0.74 is good, and 0.75-1.0 is excellent.18 SEM was calculated as SEM= SDpooled x 1−ICC. The mean squares from the ICC calculation were used to estimate σ2p, σ2r and σ2e.15

RESULTS

Participant characteristics are provided in Table 1.

78044 Table 1. Participant characteristics.

Participants	N	Height (cm)	Mass (kg)	Age (yrs)	
 	 	Mean	SD	Mean	SD	Mean	SD	
Female	11	167.1	6.4	62.3	9.9	21.4	2.3	
Male	19	179.1	5.8	78.5	7.7	22.1	2.8	

ICC and SEM scores changed with the restricted technique in different ways across ROM tests (Tables 2 and 3). Specifically, the mean intra-rater ICC and SEM scores across raters respectively decreased and increased (left and right ankle and MTPJ1 dorsiflexion), increased and decreased (left and right shoulder flexion and left hip extension), or both increased (left and right hip flexion and right hip extension) with adjacent joint restriction. However, each rater had different changes between unrestricted and restricted techniques. For example, rater 2 had a decrease in ICC and increase in SEM for shoulder flexion (Table 2).

Changes in intra-rater ICC and SEM scores between the unrestricted and restricted techniques matched changes in variance between participants (σ2p) and random error variance (σ2e) respectively. Specifically, σ2p and σ2e respectively decreased and increased (left and right ankle and MTPJ1 dorsiflexion), increased and decreased (left and right shoulder flexion and left hip extension) or both increased (left and right hip flexion and right hip extension) with adjacent joint restriction. Although adjacent joint restriction modified ICC scores, intra-rater ICCs were still good to excellent across the ROM tests even with the associated decreases with or without adjacent joint restriction (Table 2).

Inter-rater ICC and SEM scores showed similar changes to intra-rater scores with adjacent joint restriction for some tests (left and right ankle dorsiflexion, right hip extension, left hip flexion and right shoulder flexion) but not others (increased and decreased with restriction for left and right MTPJ1, decreased and increased with restriction for right side hip flexion and both ICC and SEM increased for left hip extension and left shoulder flexion, Table 3).

Inter-rater ICCs ranged from poor to good and had broader 95% CIs than the intra-rater comparison. Similarly, SEM was higher between raters compared to SEM within each rater’s measurements (Tables 2 and 3). The effects of adjacent joint restriction were more pronounced in the inter-rater comparison. The restricted technique reduced the inter-rater ICC score from fair to poor for left and right ankle dorsiflexion. Conversely, the restricted technique increased ICC scores from poor to fair for left and right hip extension and left shoulder flexion, and improved the ICC score from fair to good for left MTPJ1 flexion. SEM also decreased by approximately 3° with the restricted technique for right MTPJ1 flexion (Table 3).

The systematic variance between measurements/raters (σ2r) and the random error variance (σ2e) were higher for the inter-rater comparison than any of the intra-rater comparisons, but the changes in the ICC and SEM scores due to adjacent joint restriction varied due to differing changes across each component of variance (i.e. σ2p, σ2r and σ2e). The lower inter-rater ICC for the restricted ankle dorsiflexion was due to a decrease in σ2p and increase in σ2r and σ2e compared to unrestricted ankle dorsiflexion. The notable increases in ICC for restricted hip extension and left side shoulder flexion were also due to σ2p, as it increased but σ2r and σ2e remained similar compared to the unrestricted techniques. Conversely, the improved ICC and SEM scores for restricted MTPJ1 were not due to changes in σ2p, but due to decreases in σ2r and σ2e compared to the unrestricted technique (Table 3).

DISCUSSION

No known study has evaluated intra- and inter-rater goniometer measurement reliability and agreement across multiple ROM tests, with and without adjacent joint restriction. Reliability and agreement were differentially influenced by adjacent joint restriction across the five ROM tests. Joint restriction does not appear to substantially influence intra-rater reliability and agreement. Conversely, inter-rater reliability and agreement were more substantially influenced by adjacent joint restriction, but this was due to different changes across the components of variance used to calculate the ICC and SEM.

78679 Table 2. Intra-rater ICC with 95% CI, SEM, variance between participants (σ2p), systematic error variance (σ2r), and random error variance (σ2e).

	Rater 1	Rater 2	Rater 3	Mean	
ROM Test	Side	ICC [95% CI]	SEM	σ2p	σ2r	σ2e	ICC [95% CI]	SEM	σ2p	σ2r	σ2e	ICC [95% CI]	SEM	σ2p	σ2r	σ2e	ICC	SEM	σ2p	σ2r	σ2e	
Ankle Dorsiflexion Unrestricted	Left	0.92
[0.85, 0.96]	2.2	57.7	0.0	5.1	0.93
[0.87, 0.96]	1.8	43.5	0.0	3.3	0.94
[0.90, 0.97]	2.0	61.8	0.0	3.6	0.93	2.0	54.3	0.0	4.0	
Right	0.83
[0.71, 0.91]	3.0	45.2	0.0	9.1	0.93
[0.87, 0.96]	2.0	50.7	0.0	4.0	0.9
[0.83, 0.95]	2.4	50.3	0.1	5.3	0.89	2.5	48.7	0.0	6.1	
Ankle Dorsiflexion Restricted	Left	0.93
[0.87, 0.96]	1.8	43.1	0.0	3.3	0.92
[0.86, 0.96]	2.0	47.2	0.0	4.1	0.87
[0.78, 0.93]	2.8	51.5	0.0	7.6	0.91	2.2	47.3	0.0	5.0	
Right	0.89
[0.81, 0.94]	2.0	31.1	0.1	3.7	0.91
[0.85, 0.95]	2.1	43.9	0.0	4.2	0.81
[0.67, 0.89]	3.0	37.9	0.0	9.0	0.87	2.3	37.6	0.0	5.6	
MTPJ1 Dorsiflexion Unrestricted	Left	0.91
[0.84, 0.95]	3.6	131.9	0.0	12.9	0.83
[0.70, 0.90]	4.6	101.2	0.0	21.2	0.94
[0.90, 0.97]	3.7	215.4	0.0	12.8	0.89	4.0	149.5	0.0	15.6	
Right	0.92
[0.86, 0.96]	4.5	235.3	0.0	20.7	0.9
[0.82, 0.94]	4.2	157.7	0.0	17.5	0.96
[0.93, 0.98]	3.5	296.7	1.0	10.5	0.93	4.1	229.9	0.3	16.2	
MTPJ1 Dorsiflexion Restricted	Left	0.89
[0.80, 0.94]	4.0	131.4	0.0	16.9	0.8
[0.66, 0.89]	5.4	117.3	0.0	28.9	0.8
[0.65, 0.89]	5.9	141.8	1.5	34.5	0.83	5.1	130.2	0.5	26.8	
Right	0.83
[0.70, 0.90]	4.0	79.4	0.0	16.5	0.76
[0.60, 0.86]	6.5	133.5	1.5	40.5	0.88
[0.78, 0.93]	4.1	124.2	0.0	17.5	0.82	4.9	112.4	0.5	24.8	
Hip Extension Unrestricted	Left	0.86
[0.75, 0.92]	2.2	29.5	0.5	4.4	0.74
[0.57, 0.85]	3.3	31.8	0.0	10.9	0.79
[0.65, 0.88]	3.9	57.8	0.0	15.0	0.80	3.1	39.7	0.2	10.1	
Right	0.94
[0.89, 0.97]	1.4	31.4	0.0	2.1	0.8
[0.66, 0.89]	3.1	39.6	0.2	9.6	0.9
[0.81, 0.94]	2.6	59.0	0.5	6.4	0.88	2.4	43.3	0.2	6.0	
Hip Extension Restricted	Left	0.92
[0.85, 0.95]	2.1	48.9	0.0	4.4	0.9
[0.83, 0.95]	3.2	91.8	0.0	9.8	0.92
[0.86, 0.96]	2.7	86.1	0.2	7.3	0.91	2.7	75.6	0.1	7.2	
Right	0.92
[0.83, 0.96]	2.0	48.3	0.8	3.4	0.94
[0.89, 0.97]	2.6	104.1	0.0	6.8	0.85
[0.74, 0.92]	3.5	67.6	0.0	11.7	0.90	2.7	73.3	0.3	7.3	
Hip Flexion Unrestricted	Left	0.96
[0.93, 0.98]	2.0	98.9	0.0	4.2	0.88
[0.79, 0.93]	3.8	106.8	0.0	14.3	0.88
[0.78, 0.94]	4.2	128.1	1.8	15.4	0.91	3.3	111.3	0.6	11.3	
Right	0.95
[0.92, 0.98]	2.3	97.7	0.0	4.6	0.91
[0.84, 0.95]	3.2	103.9	0.0	10.2	0.92
[0.86, 0.96]	3.4	134.2	0.0	11.3	0.93	3.0	111.9	0.0	8.7	
Hip Flexion Restricted	Left	0.98
[0.96, 0.99]	2.2	244.1	0.1	5.8	0.92
[0.85, 0.95]	3.3	121.7	0.2	11.0	0.88
[0.78, 0.93]	5.6	231.5	0.0	32.3	0.93	3.7	199.1	0.1	16.4	
Right	0.95
[0.91, 0.97]	3.7	263.7	0.4	13.6	0.85
[0.73, 0.92]	4.8	131.0	2.6	20.0	0.93
[0.88, 0.96]	4.4	261.2	0.4	18.2	0.91	4.3	218.6	1.2	17.3	
Shoulder Flexion Unrestricted	Left	0.92
[0.86, 0.96]	3.1	112.6	0.0	9.6	0.92
[0.86, 0.96]	3.2	114.4	0.0	10.0	0.78
[0.64, 0.88]	4.8	82.4	0.0	22.6	0.87	3.7	103.1	0.0	14.1	
Right	0.94
[0.89, 0.97]	3.1	153.2	0.0	10.0	0.94
[0.89, 0.97]	2.9	132.3	0.0	8.7	0.41
[0.13, 0.63]	13.4	125.1	0.0	177.2	0.76	6.5	136.9	0.0	65.3	
Shoulder Flexion Restricted	Left	0.93
[0.86, 0.96]	3.4	155.1	1.1	11.2	0.87
[0.78, 0.93]	4.1	110.4	0.0	16.0	0.91
[0.84, 0.95]	3.6	132.6	0.1	12.7	0.90	3.7	132.7	0.4	13.3	
Right	0.96
[0.93, 0.98]	2.5	153.4	0.0	6.3	0.89
[0.80, 0.94]	3.8	115.5	0.4	14.0	0.88
[0.79, 0.93]	4.6	156.4	0.2	20.7	0.91	3.6	141.8	0.2	13.7	

78046 Table 3. Inter-rater ICC with 95% CI, SEM, variance between participants (σ2p), systematic error variance (σ2r), and random error variance (σ2e).

ROM Test	Side	ICC [95% CI]	SEM	σ2p	σ2r	σ2e	
Ankle Dorsiflexion Unrestricted	Left	0.43
[0.18, 0.63]	5.7	47.2	17.6	24.9	
Right	0.52
[0.26, 0.71]	5.0	49.9	12.4	18.2	
Ankle Dorsiflexion Restricted	Left	0.32
[0.10, 0.53]	5.8	34.0	22.1	27.1	
Right	0.28
[0.08, 0.48]	5.4	23.9	17.2	24.5	
MTPJ1 Dorsiflexion Unrestricted	Left	0.55
[0.33, 0.72]	8.4	148.2	23.2	58.5	
Right	0.56
[0.38, 0.72]	10.2	214.8	17.4	94.5	
MTPJ1 Dorsiflexion Restricted	Left	0.66
[0.50, 0.78]	7.0	146.5	5.1	45.9	
Right	0.57
[0.38, 0.73]	7.3	117.2	12.5	46.7	
Hip Extension Unrestricted	Left	0.34
[0.10, 0.56]	5.4	34.3	22.7	21.9	
Right	0.34
[0.14, 0.54]	5.5	29.8	12.0	26.5	
Hip Extension Restricted	Left	0.49
[0.24, 0.69]	6.4	73.5	20.0	30.2	
Right	0.5
[0.27, 0.69]	6.2	70.5	16.5	29.9	
Hip Flexion Unrestricted	Left	0.56
[0.34, 0.73]	7.2	113.2	17.8	41.4	
Right	0.67
[0.52, 0.79]	6.2	120.8	4.1	35.8	
Hip Flexion Restricted	Left	0.59
[0.38, 0.75]	9.2	206.4	25.8	69.7	
Right	0.62
[0.44, 0.77]	9.3	230.1	19.3	73.8	
Shoulder Flexion Unrestricted	Left	0.37
[0.14, 0.58]	8.3	82.9	38.9	54.9	
Right	0.48
[0.26, 0.66]	9.4	144.9	31.9	72.9	
Shoulder Flexion Restricted	Left	0.47
[0.18, 0.68]	8.6	134.9	51.4	49.4	
Right	0.56
[0.32, 0.73]	8.1	149.2	27.3	51.3	

The differing changes in intra-rater reliability and agreement with adjacent joint restriction were primarily influenced by σ2p and σ2e. Although changes in σ2p and σ2e each have the potential to influence the ICC and SEM, it appears as though changes in ICC were more influenced by changes in σ2p and changes in SEM were more influenced by changes in σ2e (Table 2). However, it may not be appropriate to generalize these associations. Regardless, intra-rater reliability and agreement were not substantially influenced by adjacent joint restriction as the ICC scores were still excellent and the differences between the SEM scores were not larger than approximately 1°, with the exception of right-side unrestricted shoulder flexion (Table 2). Upon examining the data, the lower ICC and higher SEM for right side shoulder flexion was likely caused by an outlier trial from rater 3, which measured 270° of shoulder flexion. It appears as though this may be unrealistic but the trial was not removed from the data to maintain equal number of trials across raters.

Inter-rater comparisons provided a more complex scenario due to the higher σ2r, which was almost nonexistent in the intra-rater comparison. The decreased inter-rater ICC and increased SEM can be attributed to the systematic and random error that occurred between raters, as σ2r and σ2e were higher while σ2p was similar to the intra-rater values. Adjacent joint restriction differentially influenced these components of variance and the resulting ICC and SEM. Improvements in reliability and agreement observed with hip extension and shoulder flexion were due to changes σ2p, which suggests that raters were not necessarily better at making the measurements even though ICC and SEM scores improved with adjacent joint restriction. Conversely, the improvements in MTPJ1 with joint restriction, can be attributed to fewer differences between raters and less random error as decreases in σ2r and σ2e caused the observed improvements in ICC and SEM. When multiple raters are involved in the measurement process, it may be beneficial to determine which technique (unrestricted/restricted) provides higher reliability and agreement and standardize it across raters. Although this may also be important with intra-rater comparisons, it may not be as important, because the intra-rater comparisons scores were excellent even with changes due to adjacent joint restriction.

The inter-rater comparison demonstrates the utility of reporting the components of variance used to calculate the ICC and SEM, as it improved interpretation of the scores. Without investigating the components of variance, it is difficult to determine why the changes in ICC and SEM occurred which can lead to misleading conclusions. Reporting the components of variance has the potential to also improve between-study comparisons that use different participants, raters and methodologies, towards enhancing the consolidation of knowledge to guide goniometry. For example, standardizing the measurement protocol and training raters has been investigated as a means to improve reliability and agreement.5 Evaluating the variance components within the data can elucidate how different protocols may differentially influence the systematic and random error, which can be used to inform on how protocols can be tailored to target improvements in reliability and agreement, beyond the constraints of a specific study. Interpreting the data in this way can contribute to developing a conceptual framework that can inform future research and decision making in practice.

Considering the limitations associated with the current investigation, maximal ROM may not have changed between measurements, but it is possible that the repositioning between raters could have contributed to σ2r and σ2e. Although a single researcher positioned participants to control these differences, a criterion measure to confirm repositioning consistency and accuracy was not performed. The study design only provides rater reliability within a single session or day. Although apparently healthy participants were investigated, it is certainly possible that they could have had limitations in bi-articular muscle flexibility which would have influenced their ROM with adjacent joint restriction and the variance between participants (σ2p). Thus, the results from this study could represent the reliability and agreement when assessing heterogeneous groups such as sport teams but would not necessarily be well representative of other homogenous groups such as those with a specific condition that would influence their muscle flexibility. Due to challenges in making comparisons between studies and the scarcity of research on the specific ROM tests and techniques investigated here, it is difficult to make meaningful comparisons to other studies.

CONCLUSION

Restriction of adjacent joints influenced measurement reliability and agreement differentially across the five ROM tests. The changes due to adjacent joint restriction were more pronounced in the inter-rater reliability and agreement, whereas intra-rater reliability and agreement were not substantially influenced. Thus, the effects of adjacent joint restriction on reliability and agreement depends on the ROM test and is of higher importance when multiple raters are involved. Estimating the components of data variance improved the interpretation of the ICC and SEM scores, which demonstrates the utility in reporting variance components in future work, to potentially improve between-study comparisons and towards developing a conceptual framework to guide goniometry.

DECLARATION OF INTEREST

The authors do not have any conflicts of interest to declare.

Supplementary Material

Appendix A

ACKNOWLEDGEMENTS

The authors would like to acknowledge the University of Toronto’s David L. Macintosh Sports Medicine Clinic staff and Dr. Catherine Sabiston for her advice on study design and analyses. The authors would also like to thank Lindsay Musalem, Malinda Hapuarachchi, Phil Toppin, Victor Chan, Rachel Micay, Justine Branco, Pedro Malvar, Joyce Kuang, Izabela Boyaninska, Sunghoon Eric Minn and Mary Claire Geneau for their assistance with data collection.
==== Refs
Chronic effect of different types of stretching on ankle dorsiflexion range of motion: Systematic review and meta-analysis The Foot Medeiros Diulian Muniz Martini Tamara Fenner Elsevier BV 3 2018
34 28 35 0958-2592 10.1016/j.foot.2017.09.006 10.1016/j.foot.2017.09.006 29223884
Inter-rater reliability for measurement of passive physiological movements in lower extremity joints is generally low: a systematic review Journal of Physiotherapy van Trijffel Emiel van de Pol Rachel J. Oostendorp Rob A.B. Lucas Cees 2010
56 4 223 235 1836-9553 10.1016/s1836-9553(10)70005-9 10.1016/s1836-9553(10)70005-9 21091412
Quantifying normal 3D hip ROM in healthy young adult males with clinical and laboratory tools: Hip mobility restrictions appear to be plane-specific Clinical Biomechanics Moreside Janice M. McGill Stuart M. 10 2011
26 8 824 829 0268-0033 10.1016/j.clinbiomech.2011.03.015 10.1016/j.clinbiomech.2011.03.015
Validity of goniometric elbow measurements: Comparative study with a radiographic method Clin Orthop Chapleau Julien Canet Fanny Petit Yvan Laflamme G-Yves Rouleau Dominique M. 11 2011
469 11 3134 3140 0009-921X 10.1007/s11999-011-1986-8 10.1007/s11999-011-1986-8 21779866
Factors influencing the reliability of the universal goniometer in measurement of lower-limb range of motion: A literature review J Prosthet Orthot Mohsin Fatma McGarry Anthony Bowers Roy J. 10 2015
27 4 140 148 1040-8800 10.1097/jpo.0000000000000074 10.1097/jpo.0000000000000074
Intraexaminer reliability, interexaminer reliability, and mean values for nine lower extremity skeletal measures in healthy naval midshipmen Journal of Orthopaedic & Sports Physical Therapy Jonson LCDR Scott R. Gross Michael T. Journal of Orthopaedic & Sports Physical Therapy (JOSPT) 4 1997
25 4 253 263 0190-6011 10.2519/jospt.1997.25.4.253 10.2519/jospt.1997.25.4.253 9083944
Interrater and intrarater reliability in the measurement of ankle joint dorsiflexion is independent of examiner experience and technique used J Am Podiatr Med Assoc Kim Paul Jeong Peace Ruth Mieras Jamie Thoms Tanya Freeman Denise Page Jeffrey 1 9 2011
101 5 407 414 8750-7315 10.7547/1010407 10.7547/1010407
Reliability and agreement of hip range of motion and provocative physical examination tests in asymptomatic volunteers PM Prather Heidi Harris-Hayes Marcie Hunt Devyani M. Steger-May Karen Mathew Vinta Clohisy John C. Wiley 10 2010
2 10 888 895 1934-1482 10.1016/j.pmrj.2010.05.005 10.1016/j.pmrj.2010.05.005 20970757
Reliability and validity of clinically accessible smartphone applications to measure joint range of motion: A systematic review PLOS ONE Keogh Justin W. L. Cox Alistair Anderson Sarah Liew Bernard Olsen Alicia Schram Ben Furness James Müller Juliane 8 5 2019
14 5 e0215806 1932-6203 10.1371/journal.pone.0215806 10.1371/journal.pone.0215806 31067247
Reliability of two methods of clinical examination of the flexibility of the hip adductor muscles Int J Sports Phys Ther Cejudo A. Ayala F. De Baranda P.S. Santonja F. 2015
10 7 976 983 26674136
When to use agreement versus reliability measures Journal of Clinical Epidemiology de Vet Henrica C.W. Terwee Caroline B. Knol Dirk L. Bouter Lex M. 10 2006
59 10 1033 1039 0895-4356 10.1016/j.jclinepi.2005.10.015 10.1016/j.jclinepi.2005.10.015
A Guideline of selecting and reporting intraclass correlation coefficients for reliability research Journal of Chiropractic Medicine Koo Terry K. Li Mae Y. 6 2016
15 2 155 163 1556-3707 10.1016/j.jcm.2016.02.012 10.1016/j.jcm.2016.02.012 27330520
Forming inferences about some intraclass correlation coefficients. Psychological Methods McGraw Kenneth O. Wong S. P. 1996
1 1 30 46 1939-1463 10.1037/1082-989x.1.1.30 10.1037/1082-989x.1.1.30
Quantifying Test-retest reliability using the intraclass correlation coefficient and the SEM The Journal of Strength and Conditioning Research Weir Joseph P. Ovid Technologies (Wolters Kluwer Health) 2005
19 1 231 1064-8011 10.1519/15184.1 10.1519/15184.1 15705040
Intraclass correlation – A discussion and demonstration of basic features PLOS ONE Liljequist David Elfving Britt Skavberg Roaldsen Kirsti Chiacchio Ferdinando 22 7 2019
14 7 e0219854 1932-6203 10.1371/journal.pone.0219854 10.1371/journal.pone.0219854 31329615
Reliability of assessment tools in rehabilitation: an illustration of appropriate statistical analyses Clinical Rehabilitation Rankin Gabrielle Stokes Maria 6 1998
12 3 187 199 0269-2155 10.1191/026921598672178340 10.1191/026921598672178340 9688034
The reliability and measurement error of protractor-based goniometry of the fingers: A systematic review Journal of Hand Therapy van Kooij Yara E. Fink Alexandra Nijhuis-van der Sanden Maria W. Speksnijder Caroline M. 10 2017
30 4 457 467 0894-1130 10.1016/j.jht.2017.02.012 10.1016/j.jht.2017.02.012
Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment Cicchetti Domenic V. 12 1994
6 4 284 290 1939-134X 10.1037/1040-3590.6.4.284 10.1037/1040-3590.6.4.284
