
==== Front
Int J Sports Phys Ther
Int J Sports Phys Ther
2159
International Journal of Sports Physical Therapy
2159-2896
NASMI Website: International Journal of Sports Physical Therapy

36518826
55544
10.26603/001c.55544
Original Research
Psychological Patient-reported Outcomes Cannot Predict a Second Anterior Cruciate Ligament Injury in Patients who Return to Sports after an Anterior Cruciate Ligament Reconstruction
Piussi Ramana 1 2 3
Simonson Rebecca 1 3
Högberg Johan 1 2 3
Thomeé Roland 1 2
Samuelsson Kristian 3 4
Hamrin Senorski Eric 1 2 3 5
1 Sportrehab Sports Medicine Clinic, Gothenburg, Sweden
2 Unit of Physiotherapy, Department of Health and Rehabilitation Institute of Neuroscience and Physiology, Sahlgrenska Academy, University of Gothenburg, Gothenburg, Sweden
3 Sahlgrenska Sports Medicine Center, Gothenburg, Sweden
4 Department of Orthopaedics Institute of Clinical Sciences, Sahlgrenska Academy, University of Gothenburg, Gothenburg, Sweden
5 Swedish Olympic Committee (SOC), Stockholm, Sweden
Corresponding author: Ramana Piussi Sportrehab Sports Medicine Clinic, Stampgatan 14, SE-411 01 Gothenburg, Sweden Ramana.piussi@gu.se +46761677520
1 12 2022
2022
17 7 13401350
20 5 2022
10 10 2022
© The Author(s)
https://creativecommons.org/licenses/by-nc/4.0/ This is an open access article distributed under the terms of the Creative Commons Attribution-NonCommercial License (4.0) which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and source are credited.

Background

Psychological patient-reported outcomes (PROs) are recommended for use in test batteries to aid in decision-making, regarding whether patients are well prepared to return to sports (RTS) after anterior cruciate ligament (ACL) reconstruction. However, the values that should be regarded as “pass” or “fail” are still unclear.

Purpose

This study aimed to identify cut-off values for three commonly used psychological PROs that could differentiate patients who suffer a second ACL injury from patients who do not within two years of RTS in patients after ACL reconstruction with respect to recovery of symmetrical quadriceps strength.

Study design

Diagnostic/prognostic study

Methods

Demographic data, isokinetic strength test data for quadriceps, as well as results for the ACL-Return to Sport after Injury scale (ACL-RSI), Knee Injury and Osteoarthritis Outcome Score (KOOS) Quality of Life, and Function in Sport and Recreation sub-scales, and the 18-item version of the Knee Self-Efficacy Scale (K-SES18) were extracted from a registry. Receiver operating characteristic (ROC) curves were calculated for each PRO. Accuracy of the cut-offs was presented with two summary measures for the ROC: the area under the curve (AUC) and Youden index.

Results

In total, 641 (355 men, 61%) patients (24.8 [SD 7.6] year old at ACL reconstruction) were included. The cut-off values were not able to differentiate patients who suffered a second ACL injury up to 24 months after RTS and ACL reconstruction from patients who did not. Additionally, achieving symmetrical quadriceps strength did not improve the cut-off psychometric properties.

Conclusion

Since cut-off values could not differentiate between patients who suffered a second ACL injury and those who did not, clinicians should not rely only on cut-off values or a single PRO of those analyzed in this study when making decisions on which patients are at risk of experiencing a second ACL injury when returning to sports after ACL reconstruction.

Level of Evidence

Level 3

anterior cruciate ligament
knee self efficacy scale
pateient reported outcomes
psychological outcomes
return to sport
return to sport after injury scale
==== Body
pmcINTRODUCTION

Anterior cruciate ligament (ACL) injury is most commonly treated with rehabilitation, and about 50% of patients undergo surgical reconstruction of the ruptured ligament.1 To sustain a second ACL injury after ACL reconstruction can be devastating; and this further increases the risk for long-term consequences, such as osteoarthritis and lower levels of physical activity and perceived quality of life.2–5 A common goal for patients treated with rehabilitation and ACL reconstruction is to return to sport (RTS).6,7 As participating in sports is associated with a risk of sustaining a second ACL injury,8 a primary concern is to evaluate the patients readiness for RTS, i.e. returning to sports with minimal risk for suffering a second ACL injury.

Responses to psychological patient-reported outcomes (PROs) can differ between patients who suffer a second ACL injury and those who do not, as patients who suffer a second ACL injury have reported greater scores, that is, better responses, on validated PROs.9,10 Collected evidence, including the results from systematic reviews on psychological outcomes after ACL reconstruction and clinical practice guidelines,11–13 as well as patients’ experiences,14 suggest that psychological PROs should be used in test batteries to help identify whether patients are well prepared for RTS after ACL reconstruction.

An issue related to the use of PROs in RTS test batteries is, which cut-off values best identify whether patients “pass” (patients can be cleared to RTS) or “fail” (patients should be advised against RTS)? Different cut-offs for different PROs and outcomes have been proposed.15,16 Notably, some cut-offs commonly used in the decision-making if the patient can RTS have not been anchored against a second ACL injury,15 but are instead based on responses from PROs that were collected several months prior to patients suffering a second ACL injury.16 It is important to acknowledge that the reasons for a second ACL injury are multifactorial and include genetic, physical, psychological, trauma, and surgical factors.17–20 Recovery of symmetrical quadriceps strength has been proposed as a possible protective factor that can reduce the risk of a second ACL injury in high-level athletes with RTS.21 However, the results are inconclusive.22,23

There is a need to better understand which cut-offs for PROs best predict a second ACL injury in patients treated with ACL reconstruction, especially regarding protective factors for a second ACL injury, such as the recovery of symmetrical quadriceps strength. This study aimed to identify cut-off values for three commonly used psychological PROs that could differentiate patients who suffer a second ACL injury from those who do not within two years of RTS after ACL reconstruction with respect to recovery of symmetrical quadriceps strength.

METHODS

Following the guidelines from the Enhancing the Quality and Transparency of Health Research (EQUATOR) network, the Standards for Reporting Diagnostic Accuracy (STARD) statement was used as a checklist to report this study.24

Data for the present study were prospectively collected from a rehabilitation outcome registry, the Project ACL. The registry was established in 2014, which aims to improve the care of patients with ACL injuries. Data in the Project ACL consists of results from muscle function tests and PROs collected prospectively before the surgery (in case of ACL reconstruction), at 10 weeks, 4, 8, 12, 18, and 24 months, and every five years with ACL injury/reconstruction as a baseline.25 Prior to participation in the Project ACL, written consent was collected. Ethical approval was obtained from the Swedish Ethical Review Authority (registration number:2020-02501).

Inclusion and exclusion criteria

All consecutive patients registered in the Project ACL with one ACL injury treated with reconstruction with or without sustaining a subsequent second ACL injury within two years from RTS [Tegner Activity Scale (Tegner)26 ≥6] were eligible for inclusion. The patients included in this study were followed up for two years after RTS. Patients were excluded if one of the following criteria were fulfilled: age <16 or >50 years; not participating in knee-strenuous sports prior to index ACL injury, i.e. reporting a Tegner Activity Scale26 of <6; sustaining a second ACL injury before surgical reconstruction; or not participating in any of the follow-ups of project ACLs. Second ACL injuries are reported by patients themselves, responsible physical therapists, or the test leader at follow-up in Project ACL.

Patient Reported Outcomes

The PROs used in this study were the ACL-Return to Surgery after Injury scale (ACL-RSI),27 Knee Injury and Osteoarthritis Outcome Score subscale Quality of Life (KOOS QoL),28 and Function in sport and recreation (KOOS Sports), the 18-item version of the Knee Self-Efficacy Scale (K-SES18),29 and Tegner Activity Scale.26

The ACL-RSI aims to measure patients’ emotions, confidence, and risk appraisal of RTS after an ACL injury. Herein, the validated 12-item short version was used.27 Each item was graded from 0 to 10, where 10 is the highest response, representing the best possible psychological response to RTS (highest confidence and emotion, and lowest risk appraisal). The final score is calculated by summing the total score of all items (highest score 120), and then normalizing the score to a 0-100 scale as proposed in the original paper.27

The K-SES18 aims to evaluate knee-related self-efficacy,29 that is, the belief in one’s ability to perform a physical task, such as running or jumping. The K-SES18 comprises 18 items divided into two subscales: present (14 items) and future (four items) knee self-efficacies. Each item was graded from 0 to 10, with 10 being the most positive response, representing the greatest belief in carrying out a given physical task. The results from each item were added and divided by the number of items to generate a mean value for the subscales.

The KOOS QoL and KOOS Sports were used in the assessment of patients with ACL injury.28 They comprise four (QoL) and five (Sports) items, respectively. The KOOS QoL assess how often patients are reminded of their knee problems, whether patients can trust their knee function, have to make life changes due to knee function, and whether patients experience problems related to the knee. The KOOS Sports comprises five items assessing a patient’s perceived difficulty in performing different tasks, such as running and jumping during the last week. Each item is rated from 0 to 4 on a 5-point Likert scale, and a normalized score from 0 to 100 is calculated for each subscale, where 0 indicates the most severe symptoms, and 100 indicates no symptoms.

The Tegner Activity Scale (Tegner)26 aims to measure strenuous knee activity. Patients grade their activity based on work and sports activities on a scale from 1 to 10, where one represents disability because of knee problems and 10 represents, for example, national or international level soccer. From level 6 on the Tegner, only sports activities are registered, and therefore when patients rate Tegner ≥6, it is assumable patients are active in a knee strenuous sport.

Strength testing

Strength tests for unilateral concentric knee extension and flexion were performed according to a standardized protocol,30 with an isokinetic dynamometer [Biodex System 4 (Biodex Medical Systems, Shirley, New York, USA)]. The testing procedure started with a standardized warm-up of 10 minutes on a stationary bike and sub-maximum trials on each test. The injured leg was tested first, followed by the uninjured leg. Isokinetic testing was performed at an angular velocity of 90°/s with the patients in a seated position. Three maximum repetitions with approximately 40 s of rest between each repetition were performed, and the greatest peak torque was recorded in the Project ACL database. For this study, the results from the unilateral knee extension test were extracted for analysis.

Study execution

In this study, demographic data, results from PROs, and strength tests for the quadriceps were extracted for analysis from the Project ACL in November 2021.

Time frame for returning to knee-strenuous sports typically varies between 6-13 months31 after primary ACL reconstruction. Results of the PROs and strength tests from the follow-up closest in time to RTS (Tegner Activity Scale, level ≥6) were selected for the included patients as the “index test.”

Primary outcomes of this study were the cut-off values with sensitivity and specificity for predicting a second ACL injury for each of the included PROs. Therefore, patients who did not go on to suffer a second ACL injury were treated as a reference, and patients who suffered a second ACL injury were treated as having the outcome of interest. Clinical reference standard was not applicable, as the analysis aimed to predict the outcome of interest. In order to account for the recovery of symmetrical quadriceps strength as a protective factor,21,32 sensitivity analyses were performed with regard to patients who had and had not recovered ≥90% of their quadriceps strength in the injured limb compared to the uninjured limb.33 Symmetrical quadriceps strength was presented with the limb symmetry index (LSI), where result from the injured limb was divided with result from the uninjured limb and multiplied by 100.

Statistics

Receiver operating characteristics (ROC) were calculated for each PRO at available follow-ups within two years of RTS (Tegner ≥6). The ROC is a graphical method of displaying the discriminatory accuracy of a marker (in this case, responses to PROs) for separating two populations, or distinguishing between patients affected by an outcome, that is, suffering a second ACL injury within two years after RTS (return to Tegner ≥6), and individuals not affected by the outcome of interest, that is, patients who did not suffer a second ACL injury within two years after RTS. A patient was assessed as “positive” if the tested PRO value was greater than a given threshold value; otherwise, the patient was assessed as negative.34 The accuracy of any given threshold value was measured by the probability of a true positive (sensitivity) and true negative (specificity), and presented with two summary measures for ROC: the area under the curve (AUC) and Youden index. The area under the curve (AUC) is a measure of the ability of a classifier to distinguish between outcomes and is used as a summary of the ROC curve. The higher the AUC, the better the performance of the model in distinguishing between positive and negative outcomes. When AUC is comprised between 0.5 and 1, there is a chance that the classifier can be able to distinguish the positive class values from the negative class values, depending on the AUC value, according to the following rule of thumb: 0.5 = no discrimination, with the same value of a coin flip; 0.5-0.7 = poor discrimination, not much better than a coin flip; 0.7-0.8 = acceptable discrimination; 0.8-0.9 = excellent discrimination; and >0.9 = outstanding discrimination.35 Therefore, when interpreting the AUC; values between 0.7 to 0.8 were considered to reflect acceptable accuracy.36 The Youden Index is a frequently used summary measure of the ROC and it measures the effectiveness of a diagnostic marker and enables the selection of an optimal threshold value (cut-off point) for the marker.34 The Youden index value ranges from 0 to 1,37 where 0 means that a diagnostic test gives the same proportion of positive results for groups with and without the disease, i.e. the test is useless, while a value of 1 indicates that there are no false positives or false negatives, i.e. the test is perfect.

Statistical analyses were performed using Statistical Analysis System (SAS) software version 9 (SAS Institute Inc., Cary, North Carolina, USA). Mean values with standard deviations (SD) or medians with minimum and maximum are presented for the demographic data. Significance level was set at 0.05.

RESULTS

In total, 641 patients (355 men, 61%) were included in the present study. Figure 1 shows the inclusion and exclusion process. Table 1 presents the demographic characteristics of the patients.

119789 Figure 1. Flowchart on inclusion and exclusion.

ACL = Anterior Cruciate Ligament; RTS = Return to Sport (return to Tegner Activity level ≥6); n = number

119790 Table 1. Demographic data, mean values, standard deviations (SD), count (n) and proportions (%).

	All patients; n=641	Men; n=355	Women; n=286	
Age at reconstruction, years	24.8 (7.6)	25.8 (7.4)	23.5 (7.6)	
Height, cm	175.7 (9.3)	181.7 (6.4)	168.2 (6.5)	
Weight, kg	73.5 (12.7)	80.9 (10.2)	64.2 (8.9)	
BMI	23.7 (2.8)	24.5 (2.6)	22.7 (2.7)	
Time to RTS, months	11.8 (15.1)	10.3 (13.8)	13.6 (16.4)	
Tegner pre-injury level, n (%)				
6	44 (6.9%)	20 (5.6%)	24 (8.4%)	
7	108 (16.8%)	66 (18.6%)	42 (14.7%)	
8	158 (24.6%)	69 (19.4%)	89 (31.1%)	
9	213 (33.2%)	129 (36.3%)	84 (29.4%)	
10	118 (18.4%)	71 (20.0%)	47 (16.4%)	
Returned to pre-injury Tegner				
Yes, n (%)	442 (69.0%)	248 (69.9%)	194 (67.8%)	
Returned to pre-injury Tegner
Months from index reconstruction				
4	108 (24.4%)	79 (31.8%)	29 (15%)	
5-8	75 (17%)	45 (18.1%)	30 (15.5%)	
9-12	128 (29%)	61 (24.6%)	67 (34.5%)	
12-24	70 (15.8%)	35 (14.1%)	35 (18.0%)	
>24	61 (13.8%)	28 (11.3%)	33 (17%)	
Graft choice				
Hamstring graft, n (%)	465 (72.5%)	260 (73.2%)	205 (71.7%)	
Patellar graft, n (%)	77 (12.0%)	43 (12.1%)	34 (11.9%)	
Other graft, n (%)	6 (0.9%)	2 (0.6%)	4 (1.4%)	
Unknown n (%)	93 (14.5%)	50 (14.1%)	43 (15.0%)	
				
Second ACL injury within 24 months from RTS				
Yes n (%)	64 (10.0%)	31 (8.7%)	33 (11.5%)	
Ipsilateral second ACL injury
n (%)	43 (67.2%)	22 (71.0%)	21 (63.6%)	
Contralateral second ACL Injury n (%)	21 (32.8%)	9 (29.0%)	12 (36.4%)	
ACL = Anterior Cruciate Ligament; BMI = Body Mass Index; cm = centimeters; kg = kilograms; n = number; RTS= Return to Sport; Tegner = Tegner Activity Scale

Figure 2 presents the frequency of a second ACL injury stratified by months from return to knee-demanding activity (Tegner ≥6).

119791 Figure 2. Frequency of a second ACL injury for every month after RTS.

ACL = Anterior Cruciate Ligament; RTS = Return to Sport (defined as return to Tegner Activity level ≥6); n = number

Table 2 presents the Youden index with specificity and sensitivity, as well as the model AUC for all assessed PROs.

119792 Table 2. Cut-off values, Youden index, and specificity and sensitivity for the included PROs in the analysis.

	n	Cut-off	Youden Index	1 - Specificity	Sensitivity	Model AUC	
ACL-RSI	382	71.7	0.13	0.37	0.50	0.553	
KOOS QoL	627	56.0	0.12	0.47	0.59	0.531	
KOOS Sports	627	96.0	0.12	0.07	0.19	0.557	
K-SES18 present	630	9.4	0.09	0.26	0.34	0.520	
K-SES18 future	630	7.0	0.08	0.70	0.78	0.511	
PROs = Patient Reported Outcomes; n = number i.e., patients responding (please note ACL-RSI is not administered at all follow-ups); ACL-RSI = Anterior Cruciate Ligament Return to Sport after Injury Scale; KOOS = Knee injury and Osteoarthritis Outcome Score; QoL = subscale Quality of Life; K-SES = Knee Self-Efficacy Scale; AUC = Area Under the Curve

The cut-offs that best differentiated patients who suffered a second ACL injury after ACL reconstruction from patients who had not were: ≥71.7 for the ACL-RSI, <56 for the KOOS QoL, ≥96.0 for the KOOS Sports, ≥9.4 for the K-SES18 present and ≥7.0 for the K-SES18 future. The AUC and Youden index for each cut-off were below acceptable values, indicating the inability of the cut-offs to properly differentiate between patients who suffered a second ACL injury after ACL reconstruction and those who did not. Table 3 presents the number of patients who suffered a second ACL injury in relation to the achievement of the cut-off values for the different PROs.

119793 Table 3. Proportion and number of patients above or below cut-offs who suffered a second ACL injury.

PROs	Cut off	Second ACL injury, % (n)	95% CI	
ACL RSI	≥71.7	14% (19)	8.4% 20.5%	
<71.7	9% (21)	5.4% 12.9%	
				
KOOS QoL	≥56	10% (40)	7.2% 13.2%	
<56	11% (24)	7.0% 15.6%	
				
KOOS Sports	≥96	23% (12)	12.5% 36.8%	
<96	9% (52)	6.8% 11.7%	
				
K-SES present	≥9.4	23% (12)	12.5% 36.8%	
<9.4	9% (52)	6.8% 11.7%	
				
K-SES future	≥7	23% (12)	12.5% 36.8%	
<7	9% (52)	6.8% 11.7%	
PROs = Patient Reported Outcomes; ACL = Anterior Cruciate Ligament; n = number i.e., patients responding; CI = Confidence Interval; ACL-RSI = Anterior Cruciate Ligament Return to Sport after Injury Scale; KOOS = Knee injury and Osteoarthritis Outcome Score; QoL = subscale Quality of Life; K-SES = Knee Self-Efficacy Scale

Effect of recovery of symmetrical quadriceps strength

Table 4 presents the cut-off analysis stratified according to patients who had and had not achieved an LSI of ≥90% in the quadriceps strength test at the time of RTS.

119794 Table 4. Cut-off values, Youden index, and specificity and sensitivity for the included PROs in the analysis, stratified by whether patients achieved symmetrical quadriceps strength or not

Patients with LSI of ≥90% (n=308)	
	n	Cut-off	Youden Index	1 - Specificity	Sensitivity	Model AUC	
ACL-RSI	220	71.7	0.22	0.35	0.57	0.591	
KOOS QoL	307	56.0	0.24	0.35	0.59	0.580	
KOOS Sports	307	96.0	0.18	0.09	0.27	0.600	
K-SES18 present	308	9.4	0.17	0.34	0.51	0.569	
K-SES18 future	308	8.3	0.05	0.62	0.68	0.495	
Patients with LSI of ≤90% (n=181)	
ACL-RSI	87	46.7	0.23	0.77	1.00	0.561	
KOOS QoL	179	38.0	0.13	0.81	0.94	0.537	
KOOS Sports	179	55.0	0.24	0.61	0.84	0.613	
K-SES18 present	180	6.1	0.21	0.74	0.95	0.537	
K-SES18 future	180	8.0	0.22	0.57	0.79	0.523	
PROs = Patient Reported Outcomes; n = number, i.e., patients responding (please note that ACL-RSI is not administered at all follow-ups); LSI = Limb Symmetry Index; ACL-RSI = Anterior Cruciate Ligament Return to Sport after Injury Scale; KOOS = Knee injury and Osteoarthritis Outcome Score; QoL = subscale Quality of Life; K-SES = Knee Self-Efficacy Scale; AUC = Area Under the Curve

Interpretation of results

Stratifying patients based on achieved symmetrical quadriceps strength (≥90% LSI) or not at the time of RTS did not lead to cut-off values with better psychometric properties, i.e., the Youden index or AUC compared with the cut-offs determined for the entire cohort. In the current results, the best Youden index was 0.24, and the best AUC was 0.613, which is well below acceptable levels.36,37 Therefore, the calculated cut-offs appear unable to separate the two outcomes, with or without a second ACL injury.

Table 5 presents the number of patients who suffered a second ACL injury in relation to the achievement of the cut-offs for the different PROs in patients who achieved symmetrical quadriceps strength at the time of RTS.

119795 Table 5. Proportion and number of patients above or below the determined cut-offs who suffer a second ACL injury were stratified, depending on whether the patients achieved symmetrical quadriceps strength or not at the time of RTS.

	PROs	Cut off	Second ACL injury,
% (n)	95% CI	
Patients with LSI of ≥90% (n=308)	ACL-RSI	≥71.7	18.7% (15)	10.9% 29.0%	
<71.7	9.3% (13)	5.0% 15.4%	
				
KOOS QoL	≥56	11.6% (27)	7.8% 16.5%	
<56	13.3% (10)	6.6% 23.2%	
				
KOOS Sports	≥96	29.4% (10)	15.1% 47.5%	
<96	9.9% (27)	6.6% 14.1%	
				
K-SES18 present	≥9.4	29.4% (10)	15.1% 47.5%	
<9.4	9.9% (27)	6.6% 14.1%	
				
K-SES18 future	≥8.3	29.4% (10)	15.1% 47.5%	
<8.3	9.9% (27)	6.6% 14.1%	
					
					
Patients with LSI of ≤90% (n=181)	ACL RSI	≥46.7	7.2% (9)	3.3% 13.2%	
<46.7	8.1% (3)	1.7% 21.9%	
				
KOOS QoL	≥38	9.1% (24)	5.9% 13.3%	
<38	5.3% (3)	1.1% 14.6%	
				
KOOS Sports	≥55	8.9% (19)	5.5% 13.6%	
<55	7.4% (8)	3.3% 14.1%	
				
K-SES18 present	≥6.1	8.9% (19)	5.5% 13.6%	
< 6.1	7.4% (8)	3.3% 14.1%	
				
K-SES18 future	≥8.0	8.9% (19)	5.5% 13.6%	
<8.0	7.4% (8)	3.3% 14.1%	
PROs = Patient Reported Outcomes; ACL = Anterior Cruciate Ligament; n = number i.e., patients responding; CI = Confidence Interval; ACL-RSI = Anterior Cruciate Ligament-Return to Sport after Injury Scale; KOOS = Knee injury and Osteoarthritis Outcome Score; QoL = subscale Quality of Life; K-SES = Knee Self-Efficacy Scale; RTS = Return To Sport

DISCUSSION

The main finding from this registry study was that the determined cut-off values for different PROs were not able to differentiate patients who suffered a second ACL injury up to 24 months after RTS following ACL reconstruction from patients who did not. Additionally, achieving symmetrical quadriceps strength, i.e., ≥90% LSI did not improve the cut-offs’ psychometric properties. Regardless of recovering symmetrical quadriceps strength, patients with either a higher or lower knee perceived quality of life, knee-related self-efficacy, confidence, emotions, and risk appraisal appear as likely to suffer a second ACL injury. Therefore, it can be assumed that passing the calculated cut-off for PROs used in the present study is not sufficient for safe RTS. Caution must thus be taken if patients are cleared to RTS based on the cut-off values used in the present study, regardless, if they achieve ≥90% LSI in quadriceps strength or not.

The used cut-off values for PROs in this study could not generate an acceptable AUC value (0.7-0.8),36 indicating that the cut-off values are no better than ‘flipping a coin’ when used to assess the risk of suffering a second ACL injury. The poor discriminative ability of the cut-offs can be partly explained by the specificity and sensitivity values. The sensitivity ranged between 0.19-0.59 for all the cut-offs except for the K-SES18 future (0.78). Using the ACL-RSI as an example, a sensitivity of 0.50 means that half of the patients who suffered a second ACL injury had a score above 71.7. Further, when a patient scores above 71.7 on the ACL-RSI, there is approximately a 50% chance to correctly guess if the patient will go on to suffer a second ACL injury (AUC = 0.553); therefore, the psychometric value for the cut-off is very low (Youden index = 0.13). Since the cut-offs that best differentiated patients who suffered a second ACL injury from patients who did not have low AUC values,36 there is a possibility that the PROs used in this study are not suitable for identifying patients who are at increased risk of a second ACL injury at the time of RTS.

One issue related to the inability of PROs to identify patients who will continue to suffer a second ACL injury can be related to the development of the PROs used in this study. The ACL-RSI was developed to assess confidence, emotion, and risk appraisal of RTS after ACL injury,27 which are three different psychological domains (constructs). However, the ACL-RSI is summarized into a single score, ranging between to 0-100; this may be questionable because the scale comprises three domains,38 which can lead to limitations as it induces indirectness towards the outcome when interpreting the PRO as a single score.

When assessing the KOOS, using a stringent psychometric method, i.e. the Rasch analysis, criteria for one-dimensionality are respected only in two out of the five subscales, i.e. KOOS Sports and KOOS Quality of life.39 Furthermore, the KOOS´s development quality has been rated as “inadequate,” and the KOOS´s psychometric properties with regard to patients with an ACL injury have been reported to be “poor,” “insufficient,” and “inconsistent”, but the authors opinions are based on that studies evaluating the psychometric properties are lacking.38,40 The inconsistency reported in the psychometric properties of the KOOS with regard to patients with an ACL injury can be due to issues during scale development, since no patients who suffered an ACL injury participated in the development and were asked whether the items were relevant.

As for the K-SES18, a recent publication reported that it has acceptable reliability and validity for measuring self-efficacy in patients with ACL injury.41 However, no Rasch analysis has yet been performed on the K-SES18, and good psychometric values can be obtained by correlating the K-SES18 with other scales of debatable psychometric properties, such as the KOOS, leading to possible misinterpretation of results. Finally, the inability of PROs to identify patients who would suffer a second ACL injury is likely related to the fact that patients with a second ACL injury were not taken into account when developing PROs.

Limitations

One limitation of this study was the relatively small number of patients in the second ACL injury group (n=64), which could have influenced the statistical power. A post-hoc power calculation was performed, which showed that 33 patients in each group were needed to detect an AUC of 0.7 with a 90% power and an alpha of 0.05. Due to the heterogeneity in the population of patients who suffered an ACL injury, 33 patients per group might still not be sufficient for the statistical power; therefore, the results should be appreciated with caution. Another limitation of the present study could be its conceptual character. A second ACL injury is multifactorial, and both high and low values on PROs can be risk factors for a second ACL injury.20 Assuming that both too high and too low knee related self-efficacy could induce a greater risk for a second ACL injury, using only one cut-off value (reflecting a greater risk for a second ACL injury) for one single scale could be conceptually wrong. If a PRO is sensitive in predicting a second ACL injury, future studies might need to use two different cut-off values: taking K-SES as an example, one cut-off reflecting an extremely high knee-related self-efficacy, and one cut-off reflecting an extremely low knee-related self-efficacy. Notably, some patients returned to the pre-injury Tegner level as early as four months after the ACL reconstruction. It is unclear how this might have affected the results. A further possible limitation of this study concerns age as the ACL-RSI has been reported to be more sensitive for change in younger patients (≤ 20 years) who go on to suffer a second ACL injury.16 Patients in this cohort were on average 24.8 years, and it is not known whether the calculated cut-offs are influenced by age, and if age-specific cut-offs may be better to predict second ACL injuries.

Conclusion

The determined cut-off values for three commonly used psychological PROs could not differentiate patients who suffered a second ACL injury from those who did not within two years from RTS, regardless of whether the patients achieved symmetrical quadriceps strength or not, in a cohort of patients who had undergone with ACL reconstruction. Therefore, clinicians are recommended to not only rely on cut-off values or a single PRO of those analyzed in this study when making decisions on which patients are at risk of experiencing a second ACL injury when returning to sports after ACL reconstruction.

Conflict of interest

Authors declare no conflict of interest.

Acknowledgments

We would like to thank Editage (www.editage.com) for English language editing, and Bengt Bengtsson at Statistiska Konsultgruppen for the help with statistical analysis.
==== Refs
Operative and nonoperative treatment options for ACL tears in the adult patient: a conceptual review The Physician and Sportsmedicine Bogunovic Ljiljana Matava Matthew J. 11 2013
41 4 33 40 0091-3847 10.3810/psm.2013.11.2034 10.3810/psm.2013.11.2034
Risk of secondary injury in younger athletes after anterior cruciate ligament reconstruction: A systematic review and meta-analysis The American Journal of Sports Medicine Wiggins Amelia J. Grandhi Ravi K. Schneider Daniel K. Stanfield Denver Webster Kate E. Myer Gregory D. SAGE Publications 15 1 2016
44 7 1861 1876 0363-5465 10.1177/0363546515621554 10.1177/0363546515621554 26772611
Comparison of concomitant injuries and patient-reported outcome in patients that have undergone both primary and revision ACL reconstruction—a national registry study Journal of Orthopaedic Surgery and Research Svantesson Eleonor Hamrin Senorski Eric Kristiansson Frida Alentorn-Geli Eduard Westin Olof Samuelsson Kristian 10 1 2020
15 1 9 1749-799X 10.1186/s13018-019-1532-z 10.1186/s13018-019-1532-z 31924236
Histopathological changes in the human posterior cruciate ligament during aging and osteoarthritis: correlations with anterior cruciate ligament and cartilage changes Annals of the Rheumatic Diseases Levy Yadin D Hasegawa Akihiko Patil Shantanu Koziol James A Lotz Martin K D'Lima Darryl D 2013
72 2 271 277 0003-4967 10.1136/annrheumdis-2012-201730 10.1136/annrheumdis-2012-201730 22872023
Fear of re-injury: a hindrance for returning to sports after anterior cruciate ligament reconstruction Knee Surgery, Sports Traumatology, Arthroscopy Kvist Joanna Ek Anna Sporrstedt Katja Good Lars 10 2 2005
13 5 393 397 0942-2056 10.1007/s00167-004-0591-8 10.1007/s00167-004-0591-8
Sports participation 2 years after anterior cruciate ligament reconstruction in athletes who had not returned to sport at 1 year: a prospective follow-up of physical function and psychological factors in 122 athletes The American Journal of Sports Medicine Ardern Clare L. Taylor Nicholas F. Feller Julian A. Whitehead Timothy S. Webster Kate E. 12 1 2015
43 4 848 856 0363-5465 10.1177/0363546514563282 10.1177/0363546514563282
Return to sport following anterior cruciate ligament reconstruction surgery: a systematic review and meta-analysis of the state of play British Journal of Sports Medicine Ardern C. L. Webster K. E. Taylor N. F. Feller J. A. 11 3 2011
45 7 596 606 0306-3674 10.1136/bjsm.2010.076364 10.1136/bjsm.2010.076364
Incidence of second ACL Injuries 2 years after primary ACL reconstruction and return to sport The American Journal of Sports Medicine Paterno Mark V. Rauh Mitchell J. Schmitt Laura C. Ford Kevin R. Hewett Timothy E. SAGE Publications 21 4 2014
42 7 1567 1573 0363-5465 10.1177/0363546514530088 10.1177/0363546514530088 24753238
Confidence, ability to meet return to sport criteria, and second ACL injury risk associations after ACL-reconstruction J Orthop Res Paterno M.V. Thomas S. VanEtten K.T. Schmitt L.C. 2021

Greater psychological readiness to return to sport, as well as greater present and future knee-related self-efficacy can increase the risk for an ACL re-rupture: a matched cohort study Arthroscopy: The Journal of Arthroscopic & Related Surgery Piussi Ramana Beischer Susanne Thomeé Roland Thomeé Christoffer Sansone Mikael Samuelsson Kristian Hamrin Senorski Eric 4 2022
38 4 1267 1276.e1 0749-8063 10.1016/j.arthro.2021.08.040 10.1016/j.arthro.2021.08.040 34571186
Psychological predictors of anterior cruciate ligament reconstruction outcomes: a systematic review Knee Surgery, Sports Traumatology, Arthroscopy Everhart Joshua S. Best Thomas M. Flanigan David C. 2015
23 3 752 762 0942-2056 10.1007/s00167-013-2699-1 10.1007/s00167-013-2699-1
Psychosocial factors influencing the recovery of athletes with anterior cruciate ligament injury: a systematic review Scand J Med Sci Sports te Wierike S.C. van der Sluis A. van den Akker-Scheek I. Elferink-Gemser M.T. Visscher C. 2013
23 5 527 540 23121478
How should clinicians rehabilitate patients after ACL reconstruction? A systematic review of clinical practice guidelines (CPGs) with a focus on quality appraisal (AGREE II) British Journal of Sports Medicine Andrade Renato Pereira Rogério van Cingel Robert Staal J Bart Espregueira-Mendes João 2020
54 9 512 519 0306-3674 10.1136/bjsports-2018-100310 10.1136/bjsports-2018-100310 31175108
Perceptions of rehabilitation and return to sport among high school athletes with anterior cruciate ligament reconstruction: A qualitative research study Journal of Orthopaedic & Sports Physical Therapy DiSanti Justin Lisee Caroline Erickson Karl Bell David Shingles Michael Kuenze Christopher Journal of Orthopaedic & Sports Physical Therapy (JOSPT) 12 2018
48 12 951 959 0190-6011 10.2519/jospt.2018.8277 10.2519/jospt.2018.8277
Defining thresholds for the patient acceptable symptom state for the IKDC subjective knee form and KOOS for patients who underwent ACL reconstruction The American Journal of Sports Medicine Muller Bart Yabroudi Mohammad A. Lynch Andrew Lai Chung-Liang van Dijk C. Niek Fu Freddie H. Irrgang James J. SAGE Publications 30 7 2016
44 11 2820 2826 0363-5465 10.1177/0363546516652888 10.1177/0363546516652888 27474383
Psychological readiness to return to sport is associated with second anterior cruciate ligament injuries The American Journal of Sports Medicine McPherson April L. Feller Julian A. Hewett Timothy E. Webster Kate E. SAGE Publications 12 2 2019
47 4 857 862 0363-5465 10.1177/0363546518825258 10.1177/0363546518825258
Risk factors associated with revision and contralateral anterior cruciate ligament reconstructions in the Kaiser Permanente ACLR registry The American Journal of Sports Medicine Maletis Gregory B. Inacio Maria C.S. Funahashi Tadashi T. 2015
43 3 641 647 0363-5465 10.1177/0363546514561745 10.1177/0363546514561745 25548148
Risk Factors and predictors of subsequent ACL injury in either knee after ACL reconstruction: prospective analysis of 2488 primary ACL reconstructions from the MOON cohort The American Journal of Sports Medicine Kaeding Christopher C. Pedroza Angela D. Reinke Emily K. Huston Laura J. Spindler Kurt P. Amendola Annunziato Andrish Jack T. Brophy Robert H. Dunn Warren R. Flanigan David Hewett Timothy E. Jones Morgan H. Marx Robert G. Matava Matthew J. McCarty Eric C. Parker Richard D. Wolcott Michelle Wolf Brian R. Wright Rick W. MOON Consortium 21 4 2015
43 7 1583 1590 0363-5465 10.1177/0363546515578836 10.1177/0363546515578836 25899429
Incidence of second anterior cruciate ligament tears and identification of associated risk factors from 2001 to 2010 using a geographic database Orthopaedic Journal of Sports Medicine Schilaty Nathan D. Nagelli Christopher Bates Nathaniel A. Sanders Thomas L. Krych Aaron J. Stuart Michael J. Hewett Timothy E. 1 8 2017
5 8 2325967117724196 2325-9671 10.1177/2325967117724196 10.1177/2325967117724196 28840155
Clinical risk profile for a second anterior cruciate ligament injury in female soccer players after anterior cruciate ligament reconstruction The American Journal of Sports Medicine Fältström Anne Kvist Joanna Bittencourt Natalia F.N. Mendonça Luciana D. Hägglund Martin SAGE Publications 15 4 2021
49 6 1421 1430 0363-5465 10.1177/0363546521999109 10.1177/0363546521999109
Simple decision rules can reduce reinjury risk by 84% after ACL reconstruction: the Delaware-Oslo ACL cohort study British Journal of Sports Medicine Grindem Hege Snyder-Mackler Lynn Moksnes Håvard Engebretsen Lars Risberg May Arna 9 5 2016
50 13 804 808 0306-3674 10.1136/bjsports-2016-096031 10.1136/bjsports-2016-096031 27162233
What is the evidence for and validity of return-to-sport testing after anterior cruciate ligament reconstruction surgery? A systematic review and meta-analysis Sports Medicine Webster Kate E. Hewett Timothy E. Springer Science and Business Media LLC 23 3 2019
49 6 917 929 0112-1642 10.1007/s40279-019-01093-x 10.1007/s40279-019-01093-x
The association between passing return-to-sport criteria and second anterior cruciate ligament injury risk: A systematic review with meta-analysis Journal of Orthopaedic & Sports Physical Therapy Losciale Justin M. Zdeb Rachael M. Ledbetter Leila Reiman Michael P. Sell Timothy C. Journal of Orthopaedic & Sports Physical Therapy (JOSPT) 2 2019
49 2 43 54 0190-6011 10.2519/jospt.2019.8190 10.2519/jospt.2019.8190
STARD 2015: an updated list of essential items for reporting diagnostic accuracy studies Br Med J Bossuyt P.M. Reitsma J.B. Bruns D.E.. 2015
351 h5527 26511519
Return to knee-strenuous sport after anterior cruciate ligament reconstruction: a report from a rehabilitation outcome registry of patient characteristics Knee Surgery, Sports Traumatology, Arthroscopy Hamrin Senorski Eric Samuelsson Kristian Thomeé Christoffer Beischer Susanne Karlsson Jón Thomeé Roland 2017
25 5 1364 1374 0942-2056 10.1007/s00167-016-4280-1 10.1007/s00167-016-4280-1 27530387
Rating systems in the evaluation of knee ligament injuries Clin Orthop Relat Res Tegner Y. Lysholm J. 1985 198 43 49
Development and validation of a short version of the anterior cruciate ligament return to sport after injury (ACL-RSI) scale Orthopaedic Journal of Sports Medicine Webster Kate E. Feller Julian A. 1 4 2018
6 4 2325967118763763 2325-9671 10.1177/2325967118763763 10.1177/2325967118763763 29662909
The Knee injury and Osteoarthritis Outcome Score (KOOS): from joint injury to osteoarthritis Health and Quality of Life Outcomes Roos Ewa M Lohmander L Stefan 2003
1 64 1477-7525 10.1186/1477-7525-1-64 10.1186/1477-7525-1-64 14613558
Validation of an 18-item version of the Swedish Knee Self-Efficacy Scale for patients after ACL injury and ACL reconstruction Journal of Experimental Orthopaedics Beischer S. Hamrin Senorski E. Thomeé P. Thomeé R. 25 10 2021
8 1 96 2197-1153 10.1186/s40634-021-00414-2 10.1186/s40634-021-00414-2 34693487
Development of a strength test battery for evaluating leg muscle power after anterior cruciate ligament injury and reconstruction Knee Surgery, Sports Traumatology, Arthroscopy Neeter Camille Gustavsson Alexander Thomeé Pia Augustsson Jesper Thomeé Roland Karlsson Jon 14 2 2006
14 6 571 580 0942-2056 10.1007/s00167-006-0040-y 10.1007/s00167-006-0040-y
Eighty-three per cent of elite athletes return to preinjury sport after anterior cruciate ligament reconstruction: a systematic review with meta-analysis of return to sport rates, graft rupture rates and performance outcomes British Journal of Sports Medicine Lai Courtney C H Ardern Clare L Feller Julian A Webster Kate E 2018
52 2 128 138 0306-3674 10.1136/bjsports-2016-096836 10.1136/bjsports-2016-096836 28223305
Can biomechanical testing after anterior cruciate ligament reconstruction identify athletes at risk for subsequent ACL injury to the contralateral uninjured limb? The American Journal of Sports Medicine King Enda Richter Chris Daniels Katherine A.J. Franklyn-Miller Andy Falvey Eanna Myer Gregory D. Jackson Mark Moran Ray Strike Siobhan SAGE Publications 9 2 2021
49 3 609 619 0363-5465 10.1177/0363546520985283 10.1177/0363546520985283
Consensus criteria for defining ‘successful outcome’ after ACL injury and reconstruction: a Delaware-Oslo ACL cohort investigation British Journal of Sports Medicine Lynch Andrew D Logerstedt David S Grindem Hege Eitzen Ingrid Hicks Gregory E Axe Michael J Engebretsen Lars Risberg May Arna Snyder-Mackler Lynn 2015
49 5 335 342 0306-3674 10.1136/bjsports-2013-092299 10.1136/bjsports-2013-092299 23881894
Estimation of the Youden Index and its associated cutoff point Biometrical Journal Fluss Ronen Faraggi David Reiser Benjamin Wiley 8 2005
47 4 458 472 0323-3847 10.1002/bimj.200410135 10.1002/bimj.200410135 16161804
Assessing the Fit of the Model Applied Logistic Regression Lemeshow H. 2013
153 225
Receiver operating characteristic curve in diagnostic test assessment Journal of Thoracic Oncology Mandrekar Jayawant N. 9 2010
5 9 1315 1316 1556-0864 10.1097/jto.0b013e3181ec173d 10.1097/jto.0b013e3181ec173d 20736804
Index for rating diagnostic tests Cancer Youden W. J. Wiley 1950
3 1 32 35 0008-543X 10.1002/1097-0142(1950)3:1 10.1002/1097-0142(1950)3:1 15405679
Psychometric properties of patient-reported outcome measures for use in patients with anterior cruciate ligament injuries: A Systematic Review J Bone Joint Surg Rev Gagnier J.J. Shen Y. Huang H. 2018
6 4 e5
Rasch analysis of the Knee injury and Osteoarthritis Outcome Score (KOOS): a statistical re-evaluation Scandinavian Journal of Medicine & Science in Sports Comins J. Brodersen J. Krogsgaard M. Beyer N. 6 2008
18 3 336 345 0905-7188 10.1111/j.1600-0838.2007.00724.x 10.1111/j.1600-0838.2007.00724.x 18028282
Four of five frequently used orthopedic PROMs possess inadequate content validity: a COSMIN evaluation of the mHHS, HAGOS, IKDC-SKF, KOOS and KNEES-ACL Knee Surg Sports Traumatol Arthrosc Hansen C.F. Jensen J. Odgaard A.. 2021

Validation of an 18-item version of the Swedish Knee Self-Efficacy Scale for patients after ACL injury and ACL reconstruction Journal of Experimental Orthopaedics Beischer S. Hamrin Senorski E. Thomeé P. Thomeé R. 25 10 2021
8 1 96 2197-1153 10.1186/s40634-021-00414-2 10.1186/s40634-021-00414-2 34693487
