
==== Front
BMC Med Educ
BMC Med Educ
BMC Medical Education
1472-6920
BioMed Central London

5932
10.1186/s12909-024-05932-y
Research
Development of a basic evaluation model for manual therapy learning in rehabilitation students based on the Delphi method
Ziyi Wang 12
Supo Zhou 3
Białas Marcin marcin.bialas@awf.gda.pl

1
1 https://ror.org/03rq9c547 grid.445131.6 0000 0001 1359 8636 Department of Sport, Gdansk University of Physical Education and Sport, Gdansk, 80-336 Poland
2 https://ror.org/01jzst437 grid.464489.3 0000 0004 1758 1008 Jiangsu Vocational College of Medicine, Yancheng City, China
3 Jiangsu College of Nursing, Huaian City, China
4 9 2024
4 9 2024
2024
24 96414 5 2024
20 8 2024
© The Author(s) 2024
2024
https://creativecommons.org/licenses/by-nc-nd/4.0/ Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.
Objective

Manual therapy is a crucial component in rehabilitation education, yet there is a lack of models for evaluating learning in this area. This study aims to develop a foundational evaluation model for manual therapy learning among rehabilitation students, based on the Delphi method, and to analyze the theoretical basis and practical significance of this model.

Methods

An initial framework for evaluating the fundamentals of manual therapy learning was constructed through a literature review and theoretical analysis. Using the Delphi method, consultations were conducted with young experts in the field of rehabilitation from January 2024 to March 2024. Fifteen experts completed three rounds of consultation. Each round involved analysis using Dview software, refining and adjusting indicators based on expert opinions, and finally summarizing all retained indicators using Mindmaster.

Results

The effective response rates for the three rounds of questionnaires were 88%, 100%, and 100%, respectively. Expert familiarity scores were 0.91, 0.95, and 0.95; coefficient of judgment were 0.92, 0.93, and 0.93; authority coefficients were 0.92, 0.94, and 0.94, respectively. Based on three rounds of consultation, the model established includes 3 primary indicators, 10 secondary indicators, 17 tertiary indicators, and 9 quaternary indicators. A total of 24 statistical indicators were finalized, with 8 under the Cognitive Abilities category, 10 under the Practical Skills category, and 6 under the Emotional Competence category.

Conclusion

This study has developed an evaluation model for manual therapy learning among rehabilitation students, based on the Delphi method. The model includes multi-level evaluation indicators covering the key dimensions of Cognitive Abilities, Practical Skills, and Emotional Competence. These indicators provide a preliminary evaluation framework for manual therapy education and a theoretical basis for future research.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12909-024-05932-y.

Keywords

Manual therapy
Delphi method
Rehabilitation education
Educational model
issue-copyright-statement© BioMed Central Ltd., part of Springer Nature 2024
==== Body
pmcIntroduction

The term “manual therapy” has traditionally been associated with physical therapists who examine and treat patients who have disorders related to the musculoskeletal system [1]. In vocational colleges in China, manual therapy techniques are an essential part of the rehabilitation education curriculum, integrating traditional Chinese medicine and modern medical teaching methods. These techniques include methods such as neurological rehabilitation, and the level of proficiency in these skills directly impacts the professional capabilities of students after graduation. In documents related to rehabilitation competency by the World Health Organization [2–4], it is noted that traditional teaching implicitly links the health needs of the population to the curriculum content. It also introduces competency-based education, which explicitly connects the health needs of the population to the competencies required of learners. The Rehabilitation Competency Framework (RCF) suggests a methodology for developing a rehabilitation education and training program and curriculum that can support competency-based education [5]. Research indicates that manual therapy education needs reform [6]. The existing evaluation models for manual therapy learning among rehabilitation students face several challenges: the use of equipment for objective assessments is cumbersome, the aspects of evaluation are not comprehensive, and there is a gap between the data from expert practices and the guidance provided to students. Some existing research has proposed models in specific manual therapy instruction. For example, the “Sequential Partial Task Practice (SPTP) strategy” was introduced in spinal manipulation (SM) teaching [7], and studies focusing on force-time characteristics [8, 9] to summarize manual techniques for subsequent teaching. Some approaches apply specific techniques to specific diseases [10]. However, in terms of overall talent development, we may still need a more comprehensive and practical model.

Learning rehabilitation therapy techniques involves comprehensive skill development. Although some studies [11, 12] have addressed the mechanisms of manual therapy, manual therapy based on mechanical actions should be considered one of the most important skills for rehabilitation therapists to focus on [13]. Currently, the training of rehabilitation students in vocational colleges primarily relies on course grades, clinical practice, and final-year exams to assess students before they enter society. However, these assessments often fail to meet the evaluation needs of employers, schools, teachers, patients/customers, and the students themselves regarding their rehabilitation capabilities. We lack a model for evaluating students’ manual therapy skills, especially for beginners. Developing a foundational evaluation model that integrates existing courses and clinical practice, in line with the World Health Organization’s Rehabilitation Competency Framework, holds significant practical and instructional value. This study aims to construct a foundational evaluation model for manual therapy learning among vocational school rehabilitation students through expert consultation. We present the following article following the CREDES reporting checklist (available at https://figshare.com/s/2886b42de467d58bd631) and the survey was performed according to the Delphi studies criteria [14].

Methods

Design

This study employs the Delphi method for the following reasons [5, 15–18]: Different experts have different emphases in manual therapy evaluation, and we need to collect a wide range of opinions and suggestions; unlike a focus group discussion, the anonymity of the Delphi method can reduce some disturbances in achieving consensus; the Delphi method allows for multiple rounds of consultation, facilitating the optimization of the model and flexible adjustment of issues that arise during consultation; the Delphi method is also used in constructing competency models for rehabilitation and has been maturely applied in closely related fields such as nursing. The research is mainly carried out in three stages: (1) Preparatory phase; (2) Delphi phase; (3) Reach consensus (Fig. 1).

Fig. 1 The flow chart of the research

Literature review

We utilized databases from PubMed to search for and collect literature focused on the theme of rehabilitation education. With the MeSH terms related to “manual therapy” and “education” were used in PubMed. We also studied the World Health Organization’s (WHO) guidelines on rehabilitation competencies, gathered score sheets from national rehabilitation skills competitions, and collected training programs for students of rehabilitation therapy technology in vocational colleges in Jiangsu Province. This helped us to identify and organize the indicators that may be involved in the basic manual therapy learning of students.

Design consulting framework

The selection of experts for the study followed the principle of representativeness, considering factors such as educational qualifications, years of professional experience, and the type of workplace, which included schools, hospitals, and studios. It was ensured that each round included at least 15 experts [15]. Each round of questionnaires sent to experts is reviewed and tested. An initial list of 20 experts was created, and after a preliminary survey, the consultation list for the first round was determined randomly. The second round was organized based on the feedback and the collection of expert questionnaires from the first round, and the third round was set up following the second round’s feedback and questionnaire collection, continuing until the criteria for concluding the study were met. Inclusion criteria for experts included: (1) having a bachelor’s degree or higher; (2) at least two years of experience in teaching or mentoring; (3) achievements in provincial or national rehabilitation skills competitions or having guided students to such achievements; (4) high level of enthusiasm; and adherence to the principles of informed consent and voluntariness.

The main contents of expert consultation include the experts’ evaluation of the importance of the basic assessment indicators for students’ manual therapy learning, suggestions for building the model, basic information about the experts, and self-evaluations of the “basis for expert judgment” and “familiarity level”. Importance evaluation follows the Likert five-point rating scale, ranging from “very important” to “not important,” with scores assigned from 5 to 1, respectively. Expert Judgment Basis Coefficient (Ca): This includes aspects of work experience, theoretical analysis, understanding of domestic and international peers, and intuitive feelings, scored at three levels: high, medium, and low, with coefficients of 0.4, 0.3, 0.2 (work experience), 0.3, 0.2, 0.1 (theoretical analysis), 0.2, 0.1, 0.1 (understanding of peers), and 0.1, 0.1, 0.1 (intuitive feelings).Expert Familiarity Score (Cs): Rated over five levels: very familiar (1.0), familiar (0.8), moderately familiar (0.5), unfamiliar (0.2), and very unfamiliar (0.0). Expert Authority Coefficient (Cr): Indicates the level of expert authority, represented by the average of the Expert Judgment Basis Coefficient and Expert Familiarity Score. The prediction accuracy increases with the level of expert authority; an Expert Authority Coefficient ≥ 0.70 is considered acceptable, while this study uses an Expert Authority Coefficient > 0.8.

Statistical analysis

In this study, Excel and Dview software were used to analyze and process the data generated in each round. The degree of agreement among experts was analyzed using the coefficient of concordance and the coefficient of variation. The Kendall’s W coefficient of concordance, calculated through Dview software, is represented by W, which ranges from 0 to 1. A higher W value indicates better agreement among experts, and vice versa. If the P-value corresponding to W is less than 0.05, it can be considered that there is consistency in the experts’ ratings of the indicator system. The coefficient of variation (CV) is the ratio of the mean importance score of a certain indicator to its standard deviation; a smaller CV indicates a higher degree of agreement among experts about this indicator. This paper uses the coefficient of variation (CV) and Kendall’s W (W) to assess the level of agreement among expert opinions. A CV < 0.25 suggests a tendency towards consensus among experts. The concentration of expert opinions is represented by the arithmetic mean and the frequency of maximum scores. The arithmetic mean is the average of the experts’ importance scores for a particular indicator; a higher mean indicates greater importance of the indicator in the system. The frequency of maximum scores is the ratio of the number of experts who gave the highest score to an indicator to the total number of experts who rated that indicator; a higher frequency of maximum scores indicates greater importance of the indicator in the system.

A clear and transparent guide for action

During the indicator selection process, this paper adopts the “threshold method” for selecting indicators. The threshold calculation formulas used are as follows: For maximum score frequency and arithmetic mean, the threshold is calculated as “Threshold = Mean - Standard Deviation.” We will select indicators that score above this threshold. For the coefficient of variation, the threshold is calculated as “Threshold = Mean + Standard Deviation.” We will select indicators that score below this threshold. To ensure that key indicators are not eliminated, we will discard indicators that do not meet all three criteria. For indicators that do not meet one or two criteria, we will modify or discuss selection based on principles of rationality and systematicity. Modifications to the model content are generally confirmed by discussions between two experts. If the two experts cannot reach a consensus, a voting process is introduced for the disputed parts, and consensus is formed through expert voting. The process ends when all consulting experts no longer propose new suggestions for the overall model, and all indicators meet the inclusion criteria.

Basic principles of the model and model presentation

This study establishes two basic principles before constructing the target model. (1) The comprehensiveness of the model, where the dimensions of the assessment indicators built into the model are relatively comprehensive. (2) The flexibility of using the model, allows for flexible application across different scenarios, techniques, and personnel. Additionally, the model can be continuously supplemented and developed through further research. After consensus is reached, use MindMaster software to draw the final model.

Ethical considerations

The assignment for technical design, informed consent form, and data report form were approved by the Research Ethical Committee of Yancheng TCM Hospital Affiliated to Nanjing University of Chinese Medicine according to the World Medical Association Declaration of Helsinki Ethical. Approval number: KY230905-02.

Results

Basic information of experts

In this study, an initial list of 20 experts was drafted. After a preliminary survey of their intentions, one expert who did not respond was excluded, and two with insufficient participation intentions were also excluded. This confirmed a list of 17 experts for the first round of consultation. After the first round, two experts whose authority coefficients were less than 0.8 were excluded, resulting in a final selection of 15 young experts from rehabilitation therapy-related schools, hospitals, and studios in Jiangsu Province (Table 1). The average age was 34.1 ± 6.6 years, and the average teaching tenure was 8.8 ± 7.7 years. Among them, one had an undergraduate degree, and 14 had graduate degrees or higher. All completed all three rounds of the survey. The level of expert engagement was indicated by the response rate of the expert consultation form, reflecting their concern for the study. The effective response rates were 88% for the first round, and 100% for the second and third rounds, all well above the 70% considered excellent. The average familiarity of the experts with the rounds was 0.91, 0.95, and 0.95 respectively, and the judgment basis coefficients were 0.92, 0.93, and 0.93. The authority coefficients were 0.92, 0.94, and 0.94 respectively.

Table 1 Summary of professional information of 17 experts

Expert number	Ca	Cs	Cr	Background	Achievement	
1	1.0	1.0	1.00	Medical college	Provincial level	
2	0.9	1.0	0.95	Medical college	Provincial level	
3	0.9	1.0	0.95	Comprehensive university	Provincial level	
4	1.0	1.0	1.00	Comprehensive university	National level	
5	1.0	0.8	0.90	Top hospital	Provincial level	
6	0.8	0.8	0.80	First-tier market	Provincial level	
7	0.9	1.0	0.95	Medical college	Provincial level	
8	1.0	0.8	0.90	Sports college	National level	
9	1.0	0.5	0.75	Top hospital	Provincial level	
10	0.7	1.0	0.85	Medical college	National level	
11	1.0	1.0	1.00	First-tier market	National level	
12	1.0	0.8	0.90	Top hospital	National level	
13	1.0	0.8	0.90	First-tier market	National level	
14	0.7	1.0	0.85	Special education	Provincial level	
15	0.8	1.0	0.90	Comprehensive university	National level	
16	1.0	1.0	1.00	Sports college	National level	
17	1.0	1.0	1.00	Community hospital	National level	
Remarks: Ca: Expert Judgment Basis Coefficient, Cs: Expert Familiarity Score, Cr: Expert Authority Coefficient. The number 6 and 9 are eliminated

Results of three rounds of the Delphi phase

The experts’ scoring data was organized in Excel and imported into DView software to calculate Kendall’s coefficient of concordance W, the progressive significance P value, chi-square, mean, coefficient of variation, and the frequency of full marks. The degree of opinion coordination and concentration of expert opinions across three rounds was summarized. The threshold method combined with expert views was applied to refine the model after three rounds of indicator screening. The table (Table 2) shows that the experts’ scoring on the indicator system was consistent across all three rounds.

Table 2 Three rounds of consultation expert opinion coordination degree

Phase	W	X2	P	
The first Delphi round	0.215	28.991	< 0.001	
The second Delphi round	0.310	106.971	< 0.001	
The third Delphi round	0.243	83.805	< 0.001	
Remarks: W: Kendall's W coefficient of concordance, X2: Chi-square, P: The progressive significance P value

The first Delphi round results

This round still included input from experts number 6 and 9 (Table 1). After the first round of consultation, according to the threshold principle (Table 3), the arithmetic mean and full score frequency of the primary indicator “Knowledge” in “On-campus” under “Relevant course scores” and “Off-campus” under “Relevant Skills Knowledge” did not meet the threshold. In the primary indicator “Skill”, under “Force” for “Quantitative (Instrument)” the coefficient of variation did not meet the threshold (Table 4). These findings suggest that the indicators set under “Knowledge” and “Skill” require significant modification, combined with the feedback from the consolidated advice of the 17 experts. There were 7 suggestions for optimizing the “Knowledge” indicator, 4 suggestions for “Skill”, 6 suggestions for “Emotion,” and 7 suggestions for the overall framework. We have redefined the “Knowledge” category as “Cognition” to broaden its conceptual scope [19], incorporating the indicator evaluation dimension of “Clinical Reasoning in Rehabilitation“ [20–22]. For the “Skill” category, we included “Proficiency“ [23, 24] and “Subject Evaluation/Effectiveness“ [25] as indicator evaluation dimensions and divided “Applicability Judgment“ [26–29] and “Positioning selection” into four levels of indicators. For the “Emotion” category, we revised the indicators of “Car” and “Respect” to “Conduct and Demeanor” and “Professional Conduct,” dividing “Conduct and Demeanor” into four levels and “Professional Conduct” into three levels [30]. These recommendations were integrated into the design of the second-round consultation form to further explore the scientific nature of the model.

Table 3 First round threshold method table

Indicator	Mean	SD	Threshold	
Arithmetic Mean	4.3400	0.2905	4.0495	
Coefficient of Variation	0.1618	0.0374	0.1992	
Full Score Frequency	0.4867	0.1861	0.3006	
SD Standard Deviation

Table 4 First Round Expert Scoring results

Primary Indicator	Secondary Indicator	Tertiary Indicator	SD	Mean	CV	Full Score Frequency	
Knowledge	On-campus	Core Course Scores

(School)

	0.5164	4.5333	0.1139	0.5333	
Related Course Scores

(School)

	0.6761	3.8000	0.1779	0.1333	
Off-campus	Clinical Practice

(Practice Base)

	0.4880	4.6667	0.1046	0.6667	
Relevant Skills Knowledge

(Self-assessment)

	0.7037	3.9333	0.1789	0.2000	
Skill	Selection	Judgment of Applicability

(Self-assessment/Peer-assessment)

	0.8281	4.4000	0.1882	0.5333	
Positioning

(Patient & Therapist)

	0.6325	4.6000	0.1375	0.6667	
Force	Qualitative

(Evaluator)

	0.8165	4.3333	0.1884	0.4667	
Quantitative

(Instrument)

	0.9411	4.2000	0.2241	0.5333	
Emotion	Care	Command of Manners

(Evaluator)

	0.7237	4.3333	0.167	0.4667	
Respect	Overall Process

(Evaluator)

	0.6325	4.6000	0.1375	0.6667	
SD Standard Deviation, CV Coefficient of Variation

The second Delphi round results

After the second round of consultation, according to the threshold principle (Table 5), the primary indicator “Cognition” under “On-campus” for “Related Course Scores” did not meet the threshold for the arithmetic mean, and the coefficient of variation for “Clinical Practice Site Assessment” under “Off-campus” did not meet the threshold. Additionally, the average and full score frequency for “Related Skills and Knowledge Learning Ability Assessment” under “Off-campus” did not meet the threshold. For the primary indicator “Emotion”, under “Conduct and Demeanor”, the average and full score frequency for “Appearance and Dress” and the coefficient of variation for “Preparation of Materials” did not meet the threshold (Table 6). We consolidated the feedback from 15 experts and optimized the model. There were 11 optimization suggestions for the “Cognition” indicator, 3 for “Skill”, and 3 for “Emotion.” Regarding whether the tertiary indicator “Core Courses Scores” should be detailed into “Theoretical scores” and “Practical scores”, 13 experts chose “yes,” one chose “no,” and one was uncertain, thus it was adopted. Concerning whether to divide the tertiary indicators “Communication” and “Conduct and Behavior” into quaternary indicators, 7 experts chose “yes,” 7 chose “no,” and one was uncertain. Considering the actual application scenario and the simplicity of the model, we retained the quaternary indicators for “Communication” and removed the related quaternary indicators for “Conduct and Behavior”. Additionally, in the “Cognition” part of the “Clinical Reasoning in Rehabilitation”, we added “Science Popularization and Patient Education Awareness“ [31, 32]; in “Skill”, we added “Palpation identification“ [33–35]; and in “Emotion” under “Professional Conduct,” we replaced “Respectful and Compassionate Thinking” with “Benevolent Physician Mindset”. After considering the content covered by nouns and the need for translation understanding, we further adjusted some expressions in the whole framework. The primary indicator “Cognitive”, “Skill” and “Emotion” were changed into “Cognitive Abilities”, “Practical Skills” and “Emotional Competence”. The secondary indicators “On-campus” and “Off-campus” are replaced by “Academic Performance” and “External Assessment”, and some other details are adjusted. These recommendations were integrated into the design of the third-round consultation form.

Table 5 Second round threshold method table

Indicator	Mean	SD	Threshold	
Arithmetic Mean	4.3694	0.3394	4.0300	
Coefficient of Variation	0.1370	0.0390	0.1760	
Full Score Frequency	0.4778	0.2356	0.2422	
SD Standard Deviation

Table 6 Second round expert scoring results

Primary Indicator	Secondary Indicator	Tertiary Indicator	Quaternary Indicator	SD	Mean	CV	Full Score Frequency	
Cognitive	On-campus	Core Course Scores	Theoretical Score	0.6325	4.4000	0.1437	0.4667	
Practical Score	0.3519	4.8667	0.0723	0.8667	
Related Course Scores		0.6172	3.6667	0.1683	0.0667	
Off-campus	Clinical Practice Site Assessment		0.8165	4.3333	0.1884	0.5333	
Related Skills and Knowledge Learning Ability Assessment		0.6399	3.8667	0.1655	0.1333	
Clinical Reasoning in Rehabilitation	Problem Analysis		0.3519	4.8667	0.0723	0.8667	
Problem Solving		0.4140	4.8000	0.0863	0.8000	
Skill	Selection	Applicability Judgment	Appropriateness of Technique Selection (Contraindications and Indications Assessment)	0.3519	4.8667	0.0723	0.8667	
Appropriateness of Progression and Regression	0.5164	4.4667	0.1156	0.4667	
Position Selection	Therapist Positioning	0.6172	4.3333	0.1424	0.4000	
Patient Positioning	0.6325	4.4000	0.1437	0.4667	
Force Application	Qualitative Assessment		0.5606	4.2000	0.1335	0.2667	
Quantitative Evaluation		0.7037	4.0667	0.173	0.2667	
Proficiency	Overall Diagnostic and Treatment Process		0.6399	4.5333	0.1412	0.6000	
Overall Operation Status		0.4880	4.3333	0.1126	0.3333	
Subject Evaluation/Effectivenes			0.6399	4.4667	0.1433	0.5333	
Emotion	Conduct and Demeanor	Communication	Fluent Expression	0.6325	4.4	0.1437	0.4667	
Professional Expression	0.7037	4.0667	0.173	0.2667	
Clear and Comprehensive Response	0.5071	4.4000	0.1152	0.4000	
Conduct and Behavior	Appearance and Dress	0.7988	3.7333	0.214	0.2000	
Preparation of Materials	0.7432	4.1333	0.1798	0.3333	
Observation of Responses to Patient Feedback	0.4140	4.8000	0.0863	0.8000	
Professional Conduct	Respectful and Compassionate Thinking		0.7037	4.2667	0.1649	0.4000	
Scientific Diagnostic and Therapeutic Reasoning		0.6325	4.6000	0.1375	0.6667	
SD Standard Deviation, CV Coefficient of Variation

The third Delphi round results

After the third round of consultation, according to the threshold principle (Table 7), the average for “Related Course Grades” under “Academic Performance” in the primary indicator “Cognition Abilities” did not meet the threshold, nor did the average and full score frequency for “Science Popularization and Patient Education Awareness” under “Clinical Reasoning in Rehabilitation”. Additionally, the coefficient of variation for “Professional Expression” under “Communication” in “Conduct and Demeanor” within “Emotional Competence” did not meet the threshold (Table 8). After expert discussion, it was considered acceptable that these three indicator thresholds were exceptional. The 15 experts did not suggest further modifications to the model’s framework or content of indicators, indicating a stable and ideal concentration of opinions. Consequently, it was decided not to proceed with a fourth round of questionnaire survey.

Table 7 Third round threshold method table

Indicator	Mean	SD	Threshold	
Arithmetic Mean	4.3944	0.2800	4.1144	
Coefficient of Variation	0.1352	0.0336	0.1688	
Full Score Frequency	0.4806	0.2162	0.2644	
SD Standard Deviation

Table 8 Third round expert scoring results

Primary Indicator	Secondary Indicator	Tertiary Indicator	Quaternary Indicator	SD	Mean	CV	Full Score Frequency	
Cognitive Abilities	Academic Performance	Core Course Grades	Theoretical Grades	0.6761	4.2000	0.161	0.3333	
Practical Grades	0.3519	4.8667	0.0723	0.8667	
Related Course Grades

(Overall Performance)

		0.5936	3.7333	0.159	0.0667	
External Assessment	Clinical Practice Site Assessment (Overall Competency)		0.7432	4.5333	0.1639	0.6667	
Related Skills and Knowledge Learning Ability Assessment

(Overall Competency)

		0.7432	4.1333	0.1798	0.3333	
Clinical Reasoning in Rehabilitation	Problem Analysis

(Personalized Plan Development)

		0.3519	4.8667	0.0723	0.8667	
Problem Solving

(Plan Implementation and Problem Fit)

		0.4140	4.8000	0.0863	0.8000	
Science Popularization and Patient Education Awareness		0.5936	4.0667	0.146	0.2000	
Practical Skills	Selection	Applicability Judgment	Appropriateness of Technique Selection

(Contraindications and Indications Assessment)

	0.3519	4.8667	0.0723	0.8667	
Appropriateness of Progression and Regression	0.5164	4.5333	0.1139	0.5333	
Position Selection	Therapist Positioning	0.5936	4.2667	0.1391	0.3333	
Patient Positioning	0.6325	4.4	0.1437	0.4667	
Palpation Identification			0.5164	4.5333	0.1139	0.5333	
Force Application	Qualitative Assessment		0.5606	4.2000	0.1335	0.2667	
Quantitative Evaluation		0.7037	4.2667	0.1649	0.4000	
Proficiency	Overall Diagnostic and Treatment Process		0.6399	4.5333	0.1412	0.6000	
Overall Operation Status		0.488	4.3333	0.1126	0.3333	
Subject Evaluation/Effectiveness			0.6399	4.4667	0.1433	0.5333	
Emotional Competence	Conduct and Demeanor	Communication	Fluent Expression	0.7037	4.2667	0.1649	0.4000	
Professional Expression	0.7746	4.2	0.1844	0.4000	
Clear and Comprehensive Response	0.5164	4.4667	0.1156	0.4667	
Conduct and Behavior

(Attention to Appearance, Preparation of Materials, and Response to Patient Feedback)

		0.6399	4.1333	0.1548	0.2667	
Professional Conduct	Benevolent Physician Mindset		0.7037	4.2667	0.1649	0.4000	
Scientific Diagnostic and Therapeutic Reasoning		0.6399	4.5333	0.1412	0.6000	
SD Standard Deviation, CV Coefficient of Variation

Model presentation and external review

After the third round of research and investigation, we used Mindmaster software to draw the final model diagram (Fig. 2). Ultimately, three primary indicators, ten secondary indicators, seventeen tertiary indicators, and nine quaternary indicators were identified. Six experts evaluated the final model, and all agreed that it is relatively well-developed. Three experts raised concerns about the weighting of indicators, which may be the focus of our next phase of research. Additionally, one expert expressed great anticipation for feedback from the actual teaching application scenarios of this model.

Fig. 2 The final model diagram

Discussion

The design of teaching assessments for manual therapy education

A key aspect of manual therapy education in rehabilitation lies in understanding the “practice and case” paradigm [36–38]. Students transition from classroom learning in school to stage-wise assessment of their learning outcomes before entering the professional sphere, where their clinical practice mindset may evolve [20] but remain consistent in principle throughout. In our model, there is a concept of a “simulated patient”, which involves simulating assessments using standardized patients or cases representing various types of illnesses. This allows beginners to quickly narrow the gap in operational skills compared to experts [25]. The advancement of teaching philosophies has posed challenges in integrating the biopsychosocial model into manual therapy practices [30]. Students’ expectations regarding manual skills in physical therapy, along with reflections on the experiences of touch, both receiving and administering, can foster an understanding of the philosophical aspects of science, ethics, and communication [19]. The COVID-19 pandemic has altered the clinical practice and education of manual therapy globally [39]. Past classical teaching methods, such as Peyton’s four-step approach to teaching complex spinal manipulation techniques, have been found superior to standard teaching methods, effectively imparting intricate spinal manipulation skills regardless of gender [40]. Additionally, other methods involving the integration of teaching with clinical practice [38], interdisciplinary group learning approaches [41], and utilization of instructional videos instead of live demonstrations [42] have also been explored. From the initial use of closed-circuit television in massage education [43], we have progressed to leveraging the internet to learn the operational strategies and steps of exemplary therapists worldwide. This includes practices such as utilizing Computer-Assisted Clinical Case (CACC) SOAP note exercises to assess students’ application of principles and practices in osteopathic therapy [44] or employing interactive interdisciplinary online teaching tools for biomechanics and physiology instruction [45]. Establishing an online practice community to support evidence-based physical therapy practices in manual therapy is also pivotal [46]. Moreover, the integration of real-time feedback tools and teaching aids has significantly enhanced the depth and engagement of learning [9].

Designing teaching assessments is considered an “art”, and with the enrichment of teaching methods and tools, feedback strategies [47] in teaching are continuously optimized. The development of rehabilitation professional courses remains a focal point and challenge for educators. Reubenson A and Elkins MR summarize the models of clinical education for Australian physiotherapy students and analyze the current status of entry-level physiotherapy assessments, along with suggesting future directions for physiotherapy education [48]. Their study underscores the inclusivity of indicator construction in model development, enabling students from different internship sites to evaluate their manual therapy learning progress using the model. Moreover, the model can be utilized for assessment even in non-face-to-face scenarios. Tai J, Ajjawi R, et al.‘s study [49] summarized the historical development of teaching assessment, highlighting the transition of assessment models from simple knowledge or skill evaluation to more complex “complex appraisal.” This reflects the increased dimensions of educational assessment, the evolution of methods, and the emphasis on quality. From the Delphi outcomes, Sizer et al. identified eight key skill sets essential for proficiency in orthopedic manual therapy (OMT), as distilled through principal component factor analysis: manual joint assessment, fine sensorimotor proficiency, manual patient management, bilateral hand-eye coordination, gross manual characteristics of the upper extremity, gross manual characteristics of the lower extremity, control of self and patient movement, and discriminate touch [50]. Additionally, Rutledge CM et al.‘s study [51] focuses on developing remote health capabilities for nursing education and practice. Caliskan SA et al. [52]. established a consensus on artificial intelligence (AI)-related competencies in medical education curricula. These breakthroughs in teaching assessment concepts and formats that transcend spatial limitations are worth noting for the future. While existing research has established quantitative models for some challenging manual therapy operations, such as teaching and assessment of high-speed, low-amplitude techniques for the spine [53], a more comprehensive model is needed to assist beginners in manual therapy education.

The key elements in the manual therapy evaluation model

In 1973, McClelland DC first introduced the concept of competence, emphasizing “Testing for competence rather than for intelligence,” highlighting the importance of distinguishing individual performance levels within specific job contexts [54]. In 2021, the World Health Organization introduced a competence model for rehabilitation practitioners, defining competence in five dimensions: Practice, Professionalism, Learning and Development, Management and Leadership, and Research. Each dimension outlines specific objectives from the perspectives of Competencies and Activities, with requirements for rehabilitation practitioners varying from basic to advanced levels, encompassing simple to more comprehensive skills, under general principles of talent development [2]. Our model draws inspiration and insights from the framework and concepts proposed by the World Health Organization, as well as the scoring criteria of the Rehabilitation Skills Competition. When constructing primary indicators, we initially identified three dimensions: knowledge, skills, and emotions. Subsequently, adjustments were made during three rounds of the Delphi method. The content within the three modules can be independently referenced or utilized for novice practitioners to conduct self-assessment or peer evaluation before entering the workplace.

In the Cognitive Abilities module, the model incorporates Academic Performance, External Assessment, and Clinical Reasoning in Rehabilitation. Apart from the conventional Core Course Grades and Related Course Grades from the school curriculum, it also integrates evaluations from students’ internship processes, including Clinical Practice Site Assessment and Related Skills and Knowledge Learning Ability Assessment. To emphasize the significance of professional course learning in school, we further divide Core Course Grades into Theoretical Grades and Practical Grades, aligning with the current pre-clinical internship assessments at our institution. Regarding health education, this model focuses on areas consistent with some related research directions [32, 55, 56]. The model highlights the importance of Clinical Reasoning in Rehabilitation by emphasizing Problem Analysis and Problem Solving in clinical practice, while also addressing the importance of Science Popularization and Patient Education Awareness.

In the Practical Skills module, this model allows for demonstration assessment based on simulated clinical scenarios, where students perform maneuvers on standardized patients, with evaluation conducted by instructors or other experts. During the operation process, we may involve assessment criteria such as Selection of techniques, Palpation Identification, Force Application, Proficiency, and ultimately, Subject Evaluation/Effectiveness. The selection of techniques involves assessing the condition of the subject, determining specific maneuvers, and appropriateness of progression and regression during maneuvers. Additionally, the selection also considers the positioning of both the operator and the subject. In assessing Force Application, besides traditional subjective evaluations, objective assessments can also be facilitated with the aid of instrumentation. Finally, for assessing Proficiency in operation, evaluations can be provided for the Overall Diagnostic and Treatment Process and Overall Operation Status. This serves as a complement to further standardizing the manual therapy process [16, 53], as the model can be applied in evaluating the procedures of certain specialized manual techniques.

In the Emotional Competence module, the model is divided into Conduct and Demeanor, and Professional Conduct. We believe that the therapeutic process between therapists and patients inherently involves interpersonal communication, hence focusing on Conduct and Behavior. Therefore, in conjunction with score sheets from national rehabilitation skills competitions, we may introduce more detailed requirements for Fluent Expression, Professional Expression, and Clear and Comprehensive Response. Furthermore, from the perspective of rehabilitation therapists’ professional roles and in alignment with the competence model, we emphasize the importance of Professional Conduct. We consider aspects such as Benevolent Physician Mindset and Scientific Diagnostic and Therapeutic Reasoning to be particularly noteworthy.

The scope and prospects of application of manual therapy evaluation model

The assessment model we designed holds relevance for skills or disciplines involving manual manipulation. Reviewing the literature on Manual Therapy [1, 57, 58] reveals that several terms are used interchangeably, such as Manipulative Therapy [59], Hands-on Therapy [31], Massage Therapy [24, 60], Manipulative Physiotherapy [36], the Chiropractic Profession [61], and Osteopathy [62]. Threlkeld AJ once stated that manual therapy encompasses a broad range of techniques used to treat neuromusculoskeletal dysfunctions, primarily aiming to relieve pain and enhance joint mobility [58]. From a professional perspective, practitioners are often referred to as Physical Therapists [30, 59], Manual Therapists [63], Manipulative Physiotherapists [33], and Massage Therapists [24, 37, 64]. Differences between Chiropractors and Massage Therapists have also been discussed in the literature [65]. The evolution of specific manual techniques such as Joint Mobilization [66], Osteopathic Manipulative Treatment (OMT) [67, 68], Spinal Manipulation Therapy (SMT) [69–71], Posterior-to-Anterior (PA) High-Velocity-Low-Amplitude (HVLA) Manipulations [72], and Cervical Spine Manipulation [73] has provided more precise guidance for addressing common diseases and disorders. Furthermore, researchers have highlighted that the development of motor skills is an essential component of clinical training across various health disciplines including surgery, dentistry, obstetrics, chiropractic, osteopathy, and physical therapy [47]. In current rehabilitation education, manual therapy is a crucial component of physical therapy. We categorize physical therapy into physiotherapy and physical therapy exercises. Physiotherapy typically requires the use of special devices to perform interventions involving sound, light, electricity, heat, and magnetism. On the other hand, physical therapy exercises are generally performed manually, with some techniques occasionally requiring the use of simple assistive tools. As researchers have suggested with the concept of motor skills [47], our physical therapy exercises in teaching may not only be beneficial for a single discipline but could also enhance all disciplines that require “hands-on“ [31] or “human touch“ [13] operations.

In the prospects of manual therapy education, the comprehensive neurophysiological model has revealed that manual therapy produces effects through multiple mechanisms [11, 12]. Studies have indicated [12, 74] that the correlation between manual assessments and clinical outcomes, mechanical measurements, and magnetic resonance imaging is poor. As measurement methodologies enrich, our teaching assessment methods will also continuously evolve. Moreover, the close connection of manual therapy with related disciplines such as anatomy and physiology [75–77] provides physical therapists with a comprehensive biomedical background, enhancing their clinical capabilities and multidisciplinary collaboration skills [13]. Secondly, the development of educational resources should emphasize the integration of practice and theory. Drawing on the educational content packaging model of dispatcher-assisted cardiopulmonary resuscitation (DA-CPR) [78], combining e-learning with practical training, and computer-related teaching models will enrich offline teaching [74], providing students with a comprehensive learning experience. This model not only increases flexibility and accessibility but also optimizes learning outcomes through continuous performance assessment. Finally, with the development of artificial intelligence and advanced simulation technologies [79], future manual therapy education could simulate complex human biomechanics and neurocentral processes, providing deeper and more intuitive learning tools. This will further enhance educational quality and lay a solid foundation for the lifelong learning and career development of physical therapy professionals.

Limitations

The panel of experts consulted in this study is relatively concentrated among middle-aged and young professionals and exhibits noticeable regional characteristics. Consequently, the conclusions drawn may exhibit certain regional specificities. Moreover, during the translation process of professional terminology, some terms in the Chinese consultation form were uniform; however, modifications were made to ensure comprehension in the English context.

Conclusions

This study comprehensively utilized theoretical research, literature analysis, and the Delphi expert consultation method. The selected experts are highly authoritative, and there was a good level of activity across three rounds of consultations, with well-coordinated expert opinions. The model includes multi-level evaluation indicators covering the key dimensions of Cognitive Abilities, Practical Skills, and Emotional Competence. This research systematically and preliminarily constructed an evaluation system for foundational manual therapy learning in rehabilitation students.

Supplementary Information

Supplementary Material 1.

Acknowledgements

Thanks to Dong Xinchun, Gu Zhongke, Lu Honggang, Li Le, Sun Wudong, Wang Yudi, Wu Wenlong, Zhao Xinyu and other experts for their assistance and patient analysis during the Delphi consultation process, and some experts chose to remain anonymous, we would like to express our gratitude once again.

Authors’ contributions

Both authors contributed to the creation of the manuscript. WZ designed and conceptualized the review and wrote the draft manuscript. ZS assisted with the Delphi consultation process and article writing. MB was involved in designing and implementing the project as a supervisor.

Funding

This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

Availability of data and materials

The datasets generated and/or analysed during the current study are available in the “figshare” repository, available at https://figshare.com/s/2886b42de467d58bd631.

Declarations

Ethics approval and consent to participate

This research was approved by the Research Ethical Committee of Yancheng TCM Hospital Affiliated to Nanjing University of Chinese Medicine according to the World Medical Association Declaration of Helsinki Ethical. Approval number: KY230905-02. Written Informed consent was obtained from all the study participants.

Consent for publication

Not applicable. This manuscript does not contain any individual person’s data in any form (including individual details, images, or videos).

Competing interests

The authors declare no competing interests.

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
==== Refs
References

1. Riddle DL Measurement of accessory motion: critical issues and related concepts Phys Ther 1992 72 12 865 874 10.1093/ptj/72.12.865 1454862
Riddle DL. Measurement of accessory motion: critical issues and related concepts. Phys Ther. 1992;72(12):865–74.1454862 10.1093/ptj/72.12.865
2. World Health Organization. Rehabilitation Competency Framework [M]. https://www.who.int/publications/i/item/9789240008281
3. World Health Organization. Adapting the WHO rehabilitation competency framework to a specific context [M]. https://www.who.int/publications/i/item/9789240015333.
4. World Health Organization. Using a contextualized competency framework to develop rehabilitation programmes and their curricula [M]. https://www.who.int/publications/i/item/9789240016576.
5. Li Y Zheng D Ma L Luo Z Wang X Competency-based construction of a comprehensive curriculum system for undergraduate nursing majors in China: an in-depth interview and modified Delphi study Ann Palliat Med 2022 11 5 1786 1798 10.21037/apm-22-471 35672895
Li Y, Zheng D, Ma L, Luo Z, Wang X. Competency-based construction of a comprehensive curriculum system for undergraduate nursing majors in China: an in-depth interview and modified Delphi study. Ann Palliat Med. 2022;11(5):1786–98.35672895 10.21037/apm-22-471
6. Kolb WH McDevitt AW Young J Shamus E The evolution of manual therapy education: what are we waiting for? J Man Manip Ther 2020 28 1 1 3 10.1080/10669817.2020.1703315 31964313
Kolb WH, McDevitt AW, Young J, Shamus E. The evolution of manual therapy education: what are we waiting for? J Man Manip Ther. 2020;28(1):1–3.31964313 10.1080/10669817.2020.1703315
7. Wise CH Schenk RJ Lattanzi JB A model for teaching and learning spinal thrust manipulation and its effect on participant confidence in technique performance J Man Manip Ther 2016 24 3 141 150 10.1179/2042618614Y.0000000088 27559284
Wise CH, Schenk RJ, Lattanzi JB. A model for teaching and learning spinal thrust manipulation and its effect on participant confidence in technique performance. J Man Manip Ther. 2016;24(3):141–50.27559284 10.1179/2042618614Y.0000000088
8. Gorrell LM Nyiro L Pasquier M Page I Heneghan NR Schweinhardt P Descarreaux M Spinal manipulation characteristics: a scoping literature review of force-time characteristics Chiropr Man Th 2023 31 1 36 10.1186/s12998-023-00512-1
Gorrell LM, Nyiro L, Pasquier M, Page I, Heneghan NR, Schweinhardt P, Descarreaux M. Spinal manipulation characteristics: a scoping literature review of force-time characteristics. Chiropr Man Th. 2023;31(1):36.10.1186/s12998-023-00512-1
9. Gonzalez-Sanchez M Ruiz-Munoz M Avila-Bolivar AB Cuesta-Vargas AI Kinematic real-time feedback is more effective than traditional teaching method in learning ankle joint mobilisation: a randomised controlled trial BMC Med Educ 2016 16 1 261 10.1186/s12909-016-0789-8 27716215
Gonzalez-Sanchez M, Ruiz-Munoz M, Avila-Bolivar AB, Cuesta-Vargas AI. Kinematic real-time feedback is more effective than traditional teaching method in learning ankle joint mobilisation: a randomised controlled trial. BMC Med Educ. 2016;16(1):261.27716215 10.1186/s12909-016-0789-8
10. Maicki T Bilski J Szczygiel E Trabka R PNF and manual therapy treatment results of patients with cervical spine osteoarthritis J Back Musculoskelet Rehabil 2017 30 5 1095 101 10.3233/BMR-169718 28946528
Maicki T, Bilski J, Szczygiel E, Trabka R. PNF and manual therapy treatment results of patients with cervical spine osteoarthritis. J Back Musculoskelet Rehabil. 2017;30(5):1095–101.28946528 10.3233/BMR-169718
11. Bialosky JE Bishop MD Price DD Robinson ME George SZ The mechanisms of manual therapy in the treatment of musculoskeletal pain: a comprehensive model Man Ther 2009 14 5 531 538 10.1016/j.math.2008.09.001 19027342
Bialosky JE, Bishop MD, Price DD, Robinson ME, George SZ. The mechanisms of manual therapy in the treatment of musculoskeletal pain: a comprehensive model. Man Ther. 2009;14(5):531–8.19027342 10.1016/j.math.2008.09.001
12. Bialosky JE Beneciuk JM Bishop MD Coronado RA Penza CW Simon CB George SZ Unraveling the mechanisms of manual therapy: modeling an approach J Orthop Sports Phys Ther 2018 48 1 8 18 10.2519/jospt.2018.7476 29034802
Bialosky JE, Beneciuk JM, Bishop MD, Coronado RA, Penza CW, Simon CB, George SZ. Unraveling the mechanisms of manual therapy: modeling an approach. J Orthop Sports Phys Ther. 2018;48(1):8–18.29034802 10.2519/jospt.2018.7476
13. Geri T Viceconti A Minacci M Testa M Rossettini G Manual therapy: exploiting the role of human touch Musculoskelet Sci Pract 2019 44 102044 10.1016/j.msksp.2019.07.008 31358458
Geri T, Viceconti A, Minacci M, Testa M, Rossettini G. Manual therapy: exploiting the role of human touch. Musculoskelet Sci Pract. 2019;44:102044.31358458 10.1016/j.msksp.2019.07.008
14. Junger S Payne SA Brine J Radbruch L Brearley SG Guidance on conducting and reporting Delphi studies (CREDES) in palliative care: recommendations based on a methodological systematic review Palliat Med 2017 31 8 684 706 10.1177/0269216317690685 28190381
Junger S, Payne SA, Brine J, Radbruch L, Brearley SG. Guidance on conducting and reporting Delphi studies (CREDES) in palliative care: recommendations based on a methodological systematic review. Palliat Med. 2017;31(8):684–706.28190381 10.1177/0269216317690685
15. de Villiers MR de Villiers PJ Kent AP The Delphi technique in health sciences education research Med Teach 2005 27 7 639 43 10.1080/13611260500069947 16332558
de Villiers MR, de Villiers PJ, Kent AP. The Delphi technique in health sciences education research. Med Teach. 2005;27(7):639–43.16332558 10.1080/13611260500069947
16. O’Donnell M Smith JA Abzug A Kulig K How should we teach lumbar manipulation? A consensus study Man Ther 2016 25 1 10 10.1016/j.math.2016.05.003 27422591
O’Donnell M, Smith JA, Abzug A, Kulig K. How should we teach lumbar manipulation? A consensus study. Man Ther. 2016;25:1–10.27422591 10.1016/j.math.2016.05.003
17. Rushton A Moore A International identification of research priorities for postgraduate theses in musculoskeletal physiotherapy using a modified Delphi technique Man Ther 2010 15 2 142 148 10.1016/j.math.2009.09.003 19889567
Rushton A, Moore A. International identification of research priorities for postgraduate theses in musculoskeletal physiotherapy using a modified Delphi technique. Man Ther. 2010;15(2):142–8.19889567 10.1016/j.math.2009.09.003
18. Keter D Griswold D Learman K Cook C Priorities in updating training paradigms in orthopedic manual therapy: an international Delphi study J Educ Eval Health Prof 2023 20 4 10.3352/jeehp.2023.20.4 36702788
Keter D, Griswold D, Learman K, Cook C. Priorities in updating training paradigms in orthopedic manual therapy: an international Delphi study. J Educ Eval Health Prof. 2023;20:4.36702788 10.3352/jeehp.2023.20.4
19. Perry J Green A Harrison K The impact of masters education in manual and manipulative therapy and the ‘knowledge acquisition model’ Man Ther 2011 16 3 285 90 10.1016/j.math.2010.12.002 21251867
Perry J, Green A, Harrison K. The impact of masters education in manual and manipulative therapy and the ‘knowledge acquisition model.’ Man Ther. 2011;16(3):285–90.21251867 10.1016/j.math.2010.12.002
20. Constantine M Carpenter C Bringing masters’ level skills to the clinical setting: what is the experience like for graduates of the master of science in manual therapy programme? Physiother Theory Pract 2012 28 8 595 603 10.3109/09593985.2012.666333 22432816
Constantine M, Carpenter C. Bringing masters’ level skills to the clinical setting: what is the experience like for graduates of the master of science in manual therapy programme? Physiother Theory Pract. 2012;28(8):595–603.22432816 10.3109/09593985.2012.666333
21. Cruz EB Moore AP Cross V A qualitative study of physiotherapy final year undergraduate students’ perceptions of clinical reasoning Man Ther 2012 17 6 549 553 10.1016/j.math.2012.05.013 22748202
Cruz EB, Moore AP, Cross V. A qualitative study of physiotherapy final year undergraduate students’ perceptions of clinical reasoning. Man Ther. 2012;17(6):549–53.22748202 10.1016/j.math.2012.05.013
22. Yamamoto K Condotta L Haldane C Jaffrani S Johnstone V Jachyra P Gibson BE Yeung E Exploring the teaching and learning of clinical reasoning, risks, and benefits of cervical spine manipulation Physiother Theory Pract 2018 34 2 91 100 10.1080/09593985.2017.1375056 29053390
Yamamoto K, Condotta L, Haldane C, Jaffrani S, Johnstone V, Jachyra P, Gibson BE, Yeung E. Exploring the teaching and learning of clinical reasoning, risks, and benefits of cervical spine manipulation. Physiother Theory Pract. 2018;34(2):91–100.29053390 10.1080/09593985.2017.1375056
23. Przekop PR Jr Tulgan H Przekop A DeMarco WJ Campbell N Kisiel S Implementation of an osteopathic manipulative medicine clinic at an allopathic teaching hospital: a research-based experience J Am Osteopath Assoc 2003 103 11 543 9 14686624
Przekop PR Jr, Tulgan H, Przekop A, DeMarco WJ, Campbell N, Kisiel S. Implementation of an osteopathic manipulative medicine clinic at an allopathic teaching hospital: a research-based experience. J Am Osteopath Assoc. 2003;103(11):543–9.14686624
24. Donoyama N Shibasaki M Differences in practitioners’ proficiency affect the effectiveness of massage therapy on physical and psychological states J Bodyw Mov Ther 2010 14 3 239 244 10.1016/j.jbmt.2009.01.007 20538221
Donoyama N, Shibasaki M. Differences in practitioners’ proficiency affect the effectiveness of massage therapy on physical and psychological states. J Bodyw Mov Ther. 2010;14(3):239–44.20538221 10.1016/j.jbmt.2009.01.007
25. Whitman JM Fritz JM Childs JD The influence of experience and specialty certifications on clinical outcomes for patients with low back pain treated within a standardized physical therapy management program J Orthop Sports Phys Ther 2004 34 11 662 72 10.2519/jospt.2004.34.11.662 15609487
Whitman JM, Fritz JM, Childs JD. The influence of experience and specialty certifications on clinical outcomes for patients with low back pain treated within a standardized physical therapy management program. J Orthop Sports Phys Ther. 2004;34(11):662–72 discussion 672 – 665.15609487 10.2519/jospt.2004.34.11.662
26. Thomson OP Petty NJ Moore AP Clinical decision-making and therapeutic approaches in osteopathy - a qualitative grounded theory study Man Ther 2014 19 1 44 51 10.1016/j.math.2013.07.008 23932101
Thomson OP, Petty NJ, Moore AP. Clinical decision-making and therapeutic approaches in osteopathy - a qualitative grounded theory study. Man Ther. 2014;19(1):44–51.23932101 10.1016/j.math.2013.07.008
27. Hansen BE Simonsen T Leboeuf-Yde C Motion palpation of the lumbar spine–a problem with the test or the tester? J Manipulative Physiol Ther 2006 29 3 208 12 10.1016/j.jmpt.2006.01.009 16584945
Hansen BE, Simonsen T, Leboeuf-Yde C. Motion palpation of the lumbar spine–a problem with the test or the tester? J Manipulative Physiol Ther. 2006;29(3):208–12.16584945 10.1016/j.jmpt.2006.01.009
28. Pool J Cagnie B Pool-Goudzwaard A Risks in teaching manipulation techniques in master programmes Man Ther 2016 25 e1 4 10.1016/j.math.2016.05.335 27319283
Pool J, Cagnie B, Pool-Goudzwaard A. Risks in teaching manipulation techniques in master programmes. Man Ther. 2016;25:e1-4.27319283 10.1016/j.math.2016.05.335
29. Goncalves G Demortier M Leboeuf-Yde C Wedderkopp N Chiropractic conservatism and the ability to determine contra-indications, non-indications, and indications to chiropractic care: a cross-sectional survey of chiropractic students Chiropr Man Th 2019 27 3 10.1186/s12998-018-0227-6
Goncalves G, Demortier M, Leboeuf-Yde C, Wedderkopp N. Chiropractic conservatism and the ability to determine contra-indications, non-indications, and indications to chiropractic care: a cross-sectional survey of chiropractic students. Chiropr Man Th. 2019;27:3.10.1186/s12998-018-0227-6
30. Jones M Edwards I Gifford L Conceptual models for implementing biopsychosocial theory in clinical practice Man Ther 2002 7 1 2 9 10.1054/math.2001.0426 11884150
Jones M, Edwards I, Gifford L. Conceptual models for implementing biopsychosocial theory in clinical practice. Man Ther. 2002;7(1):2–9.11884150 10.1054/math.2001.0426
31. Pesco MS Chosa E Tajima N Comparative study of hands-on therapy with active exercises vs education with active exercises for the management of upper back pain J Manipulative Physiol Ther 2006 29 3 228 235 10.1016/j.jmpt.2006.02.001 16584949
Pesco MS, Chosa E, Tajima N. Comparative study of hands-on therapy with active exercises vs education with active exercises for the management of upper back pain. J Manipulative Physiol Ther. 2006;29(3):228–35.16584949 10.1016/j.jmpt.2006.02.001
32. Eilayyan O Thomas A Halle MC Ahmed S Tibbles AC Jacobs C Mior S Davis C Evans R Schneider MJ Promoting the use of self-management in novice chiropractors treating individuals with spine pain: the design of a theory-based knowledge translation intervention BMC Musculoskelet Disord 2018 19 1 328 10.1186/s12891-018-2241-1 30205825
Eilayyan O, Thomas A, Halle MC, Ahmed S, Tibbles AC, Jacobs C, Mior S, Davis C, Evans R, Schneider MJ, et al. Promoting the use of self-management in novice chiropractors treating individuals with spine pain: the design of a theory-based knowledge translation intervention. BMC Musculoskelet Disord. 2018;19(1):328.30205825 10.1186/s12891-018-2241-1
33. Downey BJ Taylor NF Niere KR Manipulative physiotherapists can reliably palpate nominated lumbar spinal levels Man Ther 1999 4 3 151 6 10.1054/math.1999.0196 10513445
Downey BJ, Taylor NF, Niere KR. Manipulative physiotherapists can reliably palpate nominated lumbar spinal levels. Man Ther. 1999;4(3):151–6.10513445 10.1054/math.1999.0196
34. Harlick JC Milosavljevic S Milburn PD Palpation identification of spinous processes in the lumbar spine Man Ther 2007 12 1 56 62 10.1016/j.math.2006.02.008 16781182
Harlick JC, Milosavljevic S, Milburn PD. Palpation identification of spinous processes in the lumbar spine. Man Ther. 2007;12(1):56–62.16781182 10.1016/j.math.2006.02.008
35. Phillips DR Barnard S Mullee MA Hurley MV Simple anatomical information improves the accuracy of locating specific spinous processes during manual examination of the low back Man Ther 2009 14 3 346 350 10.1016/j.math.2008.02.009 18511330
Phillips DR, Barnard S, Mullee MA, Hurley MV. Simple anatomical information improves the accuracy of locating specific spinous processes during manual examination of the low back. Man Ther. 2009;14(3):346–50.18511330 10.1016/j.math.2008.02.009
36. Rushton A Lindsay G Defining the construct of masters level clinical practice in manipulative physiotherapy Man Ther 2010 15 1 93 9 10.1016/j.math.2009.08.003 19748815
Rushton A, Lindsay G. Defining the construct of masters level clinical practice in manipulative physiotherapy. Man Ther. 2010;15(1):93–9.19748815 10.1016/j.math.2009.08.003
37. Sherman KJ Cherkin DC Kahn J Erro J Hrbek A Deyo RA Eisenberg DM A survey of training and practice patterns of massage therapists in two US states BMC Complement Altern Med 2005 5 13 10.1186/1472-6882-5-13 15955245
Sherman KJ, Cherkin DC, Kahn J, Erro J, Hrbek A, Deyo RA, Eisenberg DM. A survey of training and practice patterns of massage therapists in two US states. BMC Complement Altern Med. 2005;5: 13.15955245 10.1186/1472-6882-5-13
38. Flynn TW Wainner RS Fritz JM Spinal manipulation in physical therapist professional degree education: a model for teaching and integration into clinical practice J Orthop Sports Phys Ther 2006 36 8 577 587 10.2519/jospt.2006.2159 16915979
Flynn TW, Wainner RS, Fritz JM. Spinal manipulation in physical therapist professional degree education: a model for teaching and integration into clinical practice. J Orthop Sports Phys Ther. 2006;36(8):577–87.16915979 10.2519/jospt.2006.2159
39. MacDonald CW Lonnemann E Petersen SM Rivett DA Osmotherly PG Brismee JM COVID 19 and manual therapy: international lessons and perspectives on current and future clinical practice and education J Man Manip Ther 2020 28 3 134 145 10.1080/10669817.2020.1782059 32666912
MacDonald CW, Lonnemann E, Petersen SM, Rivett DA, Osmotherly PG, Brismee JM. COVID 19 and manual therapy: international lessons and perspectives on current and future clinical practice and education. J Man Manip Ther. 2020;28(3):134–45.32666912 10.1080/10669817.2020.1782059
40. Gradl-Dietsch G Lubke C Horst K Simon M Modabber A Sonmez TT Munker R Nebelung S Knobe M Peyton’s four-step approach for teaching complex spinal manipulation techniques - a prospective randomized trial BMC Med Educ 2016 16 1 284 10.1186/s12909-016-0804-0 27809905
Gradl-Dietsch G, Lubke C, Horst K, Simon M, Modabber A, Sonmez TT, Munker R, Nebelung S, Knobe M. Peyton’s four-step approach for teaching complex spinal manipulation techniques - a prospective randomized trial. BMC Med Educ. 2016;16(1):284.27809905 10.1186/s12909-016-0804-0
41. Schuit D Diers D Vendrely A Interdisciplinary group learning in a kinesiology course: a novel approach J Allied Health 2013 42 4 e91 96 24326924
Schuit D, Diers D, Vendrely A. Interdisciplinary group learning in a kinesiology course: a novel approach. J Allied Health. 2013;42(4):e91-96.24326924
42. Seals R Gustowski SM Kominski C Li F Does replacing live demonstration with instructional videos improve student satisfaction and osteopathic manipulative treatment examination performance? J Am Osteopath Assoc 2016 116 11 726 34 27802558
Seals R, Gustowski SM, Kominski C, Li F. Does replacing live demonstration with instructional videos improve student satisfaction and osteopathic manipulative treatment examination performance? J Am Osteopath Assoc. 2016;116(11):726–34.27802558
43. Norkin CC Chiswell J Use of closed-circuit television in teaching massage Phys Ther 1975 55 1 41 42 10.1093/ptj/55.1.41 1109504
Norkin CC, Chiswell J. Use of closed-circuit television in teaching massage. Phys Ther. 1975;55(1):41–2.1109504 10.1093/ptj/55.1.41
44. Chamberlain NR Yates HA Use of a computer-assisted clinical case (CACC) SOAP note exercise to assess students’ application of osteopathic principles and practice J Am Osteopath Assoc 2000 100 7 437 40 10943092
Chamberlain NR, Yates HA. Use of a computer-assisted clinical case (CACC) SOAP note exercise to assess students’ application of osteopathic principles and practice. J Am Osteopath Assoc. 2000;100(7):437–40.10943092
45. Martay JLB Martay H Carpes FP BodyWorks: interactive interdisciplinary online teaching tools for biomechanics and physiology teaching Adv Physiol Educ 2021 45 4 715 719 10.1152/advan.00069.2021 34498937
Martay JLB, Martay H, Carpes FP. BodyWorks: interactive interdisciplinary online teaching tools for biomechanics and physiology teaching. Adv Physiol Educ. 2021;45(4):715–9.34498937 10.1152/advan.00069.2021
46. Evans C Yeung E Markoulakis R Guilcher S An online community of practice to support evidence-based physiotherapy practice in manual therapy J Contin Educ Health Prof 2014 34 4 215 23 10.1002/chp.21253 25530291
Evans C, Yeung E, Markoulakis R, Guilcher S. An online community of practice to support evidence-based physiotherapy practice in manual therapy. J Contin Educ Health Prof. 2014;34(4):215–23.25530291 10.1002/chp.21253
47. Triano JJ Descarreaux M Dugas C Biomechanics–review of approaches for performance training in spinal manipulation J Electromyogr Kinesiol 2012 22 5 732 739 10.1016/j.jelekin.2012.03.011 22542770
Triano JJ, Descarreaux M, Dugas C. Biomechanics–review of approaches for performance training in spinal manipulation. J Electromyogr Kinesiol. 2012;22(5):732–9.22542770 10.1016/j.jelekin.2012.03.011
48. Reubenson A Elkins MR Clinical education of physiotherapy students J Physiother 2022 68 3 153 155 10.1016/j.jphys.2022.05.012 35697597
Reubenson A, Elkins MR. Clinical education of physiotherapy students. J Physiother. 2022;68(3):153–5.35697597 10.1016/j.jphys.2022.05.012
49. Tai J Ajjawi R Boud D Dawson P Panadero E Developing evaluative judgement: enabling students to make decisions about the quality of work High Educ 2017 76 3 467 81 10.1007/s10734-017-0220-3
Tai J, Ajjawi R, Boud D, Dawson P, Panadero E. Developing evaluative judgement: enabling students to make decisions about the quality of work. High Educ. 2017;76(3):467–81.10.1007/s10734-017-0220-3
50. Sizer PS Jr Felstehausen V Sawyer S Dornier L Matthews P Cook C Eight critical skill sets required for manual therapy competency: a Delphi study and factor analysis of physical therapy educators of manual therapy J Allied Health 2007 36 1 30 40 17425189
Sizer PS Jr, Felstehausen V, Sawyer S, Dornier L, Matthews P, Cook C. Eight critical skill sets required for manual therapy competency: a Delphi study and factor analysis of physical therapy educators of manual therapy. J Allied Health. 2007;36(1):30–40.17425189
51. Rutledge CM O’Rourke J Mason AM Chike-Harris K Behnke L Melhado L Downes L Gustin T Telehealth competencies for nursing education and practice: the four P’s of telehealth Nurse Educ 2021 46 5 300 5 10.1097/NNE.0000000000000988 33481494
Rutledge CM, O’Rourke J, Mason AM, Chike-Harris K, Behnke L, Melhado L, Downes L, Gustin T. Telehealth competencies for nursing education and practice: the four P’s of telehealth. Nurse Educ. 2021;46(5):300–5.33481494 10.1097/NNE.0000000000000988
52. Caliskan SA Demir K Karaca O Artificial intelligence in medical education curriculum: an e-Delphi study for competencies PLoS ONE 2022 17 7 e0271872 10.1371/journal.pone.0271872 35862401
Caliskan SA, Demir K, Karaca O. Artificial intelligence in medical education curriculum: an e-Delphi study for competencies. PLoS ONE. 2022;17(7):e0271872.35862401 10.1371/journal.pone.0271872
53. Channell MK Teaching and assessment of high-velocity, low-amplitude techniques for the spine in predoctoral medical education J Am Osteopath Assoc 2016 116 9 610 8 27571298
Channell MK. Teaching and assessment of high-velocity, low-amplitude techniques for the spine in predoctoral medical education. J Am Osteopath Assoc. 2016;116(9):610–8.27571298
54. McClelland DC Testing for competence rather than for intelligence Am Psychol 1973 28 1 1 14 10.1037/h0034092 4684069
McClelland DC. Testing for competence rather than for intelligence. Am Psychol. 1973;28(1):1–14.4684069 10.1037/h0034092
55. Blickenstaff C Pearson N Reconciling movement and exercise with pain neuroscience education: a case for consistent education Physiother Theory Pract 2016 32 5 396 407 10.1080/09593985.2016.1194653 27356079
Blickenstaff C, Pearson N. Reconciling movement and exercise with pain neuroscience education: a case for consistent education. Physiother Theory Pract. 2016;32(5):396–407.27356079 10.1080/09593985.2016.1194653
56. Marcano-Fernandez FA Fillat-Goma F Balaguer-Castro M Rafols-Perramon O Serrano-Sanz J Torner P Can patients learn how to reduce their shoulder dislocation? A one-year follow-up of the randomized clinical trial between the boss-holzach-matter self-assisted technique and the spaso method Acta Orthop Traumatol Turc 2020 54 5 516 8 10.5152/j.aott.2020.19017 33155562
Marcano-Fernandez FA, Fillat-Goma F, Balaguer-Castro M, Rafols-Perramon O, Serrano-Sanz J, Torner P. Can patients learn how to reduce their shoulder dislocation? A one-year follow-up of the randomized clinical trial between the boss-holzach-matter self-assisted technique and the spaso method. Acta Orthop Traumatol Turc. 2020;54(5):516–8.33155562 10.5152/j.aott.2020.19017
57. Farrell JP Jensen GM Manual therapy: a critical assessment of role in the profession of physical therapy Phys Ther 1992 72 12 843 52 10.1093/ptj/72.12.843 1454860
Farrell JP, Jensen GM. Manual therapy: a critical assessment of role in the profession of physical therapy. Phys Ther. 1992;72(12):843–52.1454860 10.1093/ptj/72.12.843
58. Threlkeld AJ The effects of manual therapy on connective tissue Phys Ther 1992 72 12 893 902 10.1093/ptj/72.12.893 1454865
Threlkeld AJ. The effects of manual therapy on connective tissue. Phys Ther. 1992;72(12):893–902.1454865 10.1093/ptj/72.12.893
59. Stephens EB Manipulative therapy in physical therapy curricula Phys Ther 1973 53 1 40 50 10.1093/ptj/53.1.40 4682700
Stephens EB. Manipulative therapy in physical therapy curricula. Phys Ther. 1973;53(1):40–50.4682700 10.1093/ptj/53.1.40
60. Kania-Richmond A Menard MB Barberree B Mohring M Dancing on the edge of research - what is needed to build and sustain research capacity within the massage therapy profession? A formative evaluation J Bodyw Mov Ther 2017 21 2 274 83 10.1016/j.jbmt.2016.06.019 28532869
Kania-Richmond A, Menard MB, Barberree B, Mohring M. Dancing on the edge of research - what is needed to build and sustain research capacity within the massage therapy profession? A formative evaluation. J Bodyw Mov Ther. 2017;21(2):274–83.28532869 10.1016/j.jbmt.2016.06.019
61. Beliveau PJH Wong JJ Sutton DA Simon NB Bussieres AE Mior SA French SD The chiropractic profession: a scoping review of utilization rates, reasons for seeking care, patient profiles, and care provided Chiropr Man Th 2017 25 35 10.1186/s12998-017-0165-8
Beliveau PJH, Wong JJ, Sutton DA, Simon NB, Bussieres AE, Mior SA, French SD. The chiropractic profession: a scoping review of utilization rates, reasons for seeking care, patient profiles, and care provided. Chiropr Man Th. 2017;25:35.10.1186/s12998-017-0165-8
62. Requena-Garcia J, Garcia-Nieto E, Varillas-Delgado D. Objectivation of an educational model in cranial osteopathy based on experience. Med (Kaunas). 2021;57(3):246.
63. Twomey LT A rationale for the treatment of back pain and joint pain by manual therapy Phys Ther 1992 72 12 885 892 10.1093/ptj/72.12.885 1454864
Twomey LT. A rationale for the treatment of back pain and joint pain by manual therapy. Phys Ther. 1992;72(12):885–92.1454864 10.1093/ptj/72.12.885
64. Violato C Salami L Muiznieks S Certification examinations for massage therapists: a psychometric analysis J Manipulative Physiol Ther 2002 25 2 111 5 10.1067/mmt.2002.121413 11896380
Violato C, Salami L, Muiznieks S. Certification examinations for massage therapists: a psychometric analysis. J Manipulative Physiol Ther. 2002;25(2):111–5.11896380 10.1067/mmt.2002.121413
65. Suter E Vanderheyden LC Trojan LS Verhoef MJ Armitage GD How important is research-based practice to chiropractors and massage therapists? J Manipulative Physiol Ther 2007 30 2 109 15 10.1016/j.jmpt.2006.12.013 17320731
Suter E, Vanderheyden LC, Trojan LS, Verhoef MJ, Armitage GD. How important is research-based practice to chiropractors and massage therapists? J Manipulative Physiol Ther. 2007;30(2):109–15.17320731 10.1016/j.jmpt.2006.12.013
66. Petersen EJ Thurmond SM Buchanan SI Chun DH Richey AM Nealon LP The effect of real-time feedback on learning lumbar spine joint mobilization by entry-level doctor of physical therapy students: a randomized, controlled, crossover trial J Man Manip Ther 2020 28 4 201 211 10.1080/10669817.2019.1673953 31593528
Petersen EJ, Thurmond SM, Buchanan SI, Chun DH, Richey AM, Nealon LP. The effect of real-time feedback on learning lumbar spine joint mobilization by entry-level doctor of physical therapy students: a randomized, controlled, crossover trial. J Man Manip Ther. 2020;28(4):201–11.31593528 10.1080/10669817.2019.1673953
67. Johnson SM Kurtz ME Diminished use of osteopathic manipulative treatment and its impact on the uniqueness of the osteopathic profession Acad Med 2001 76 8 821 828 10.1097/00001888-200108000-00016 11500286
Johnson SM, Kurtz ME. Diminished use of osteopathic manipulative treatment and its impact on the uniqueness of the osteopathic profession. Acad Med. 2001;76(8):821–8.11500286 10.1097/00001888-200108000-00016
68. Baker HH Linsenmeyer M Ridpath LC Bauer LJ Foster RW Osteopathic medical students entering family medicine and attitudes regarding osteopathic manipulative treatment: preliminary findings of differences by sex J Am Osteopath Assoc 2017 117 6 387 92 28556861
Baker HH, Linsenmeyer M, Ridpath LC, Bauer LJ, Foster RW. Osteopathic medical students entering family medicine and attitudes regarding osteopathic manipulative treatment: preliminary findings of differences by sex. J Am Osteopath Assoc. 2017;117(6):387–92.28556861
69. Rogers CM Triano JJ Biomechanical measure validation for spinal manipulation in clinical settings J Manipulative Physiol Ther 2003 26 9 539 48 10.1016/j.jmpt.2003.08.008 14673403
Rogers CM, Triano JJ. Biomechanical measure validation for spinal manipulation in clinical settings. J Manipulative Physiol Ther. 2003;26(9):539–48.14673403 10.1016/j.jmpt.2003.08.008
70. Triano JJ Rogers CM Combs S Potts D Sorrels K Developing skilled performance of lumbar spine manipulation J Manipulative Physiol Ther 2002 25 6 353 61 10.1067/mmt.2002.126132 12183693
Triano JJ, Rogers CM, Combs S, Potts D, Sorrels K. Developing skilled performance of lumbar spine manipulation. J Manipulative Physiol Ther. 2002;25(6):353–61.12183693 10.1067/mmt.2002.126132
71. Pasquier M Cheron C Barbier G Dugas C Lardon A Descarreaux M Learning spinal manipulation: objective and subjective assessment of performance J Manipulative Physiol Ther 2020 43 3 189 96 10.1016/j.jmpt.2019.12.010 32951767
Pasquier M, Cheron C, Barbier G, Dugas C, Lardon A, Descarreaux M. Learning spinal manipulation: objective and subjective assessment of performance. J Manipulative Physiol Ther. 2020;43(3):189–96.32951767 10.1016/j.jmpt.2019.12.010
72. Starmer DJ Guist BP Tuff TR Warren SC Williams MG Changes in manipulative peak force modulation and time to peak thrust among first-year chiropractic students following a 12-week detraining period J Manipulative Physiol Ther 2016 39 4 311 7 10.1016/j.jmpt.2016.02.010 27059248
Starmer DJ, Guist BP, Tuff TR, Warren SC, Williams MG. Changes in manipulative peak force modulation and time to peak thrust among first-year chiropractic students following a 12-week detraining period. J Manipulative Physiol Ther. 2016;39(4):311–7.27059248 10.1016/j.jmpt.2016.02.010
73. Van Geyt B Dugailly PA De Page L Feipel V Relationship between subjective experience of individuals, practitioner seniority, cavitation occurrence, and 3-dimensional kinematics during cervical spine manipulation J Manipulative Physiol Ther 2017 40 9 643 8 10.1016/j.jmpt.2017.09.002 29229054
Van Geyt B, Dugailly PA, De Page L, Feipel V. Relationship between subjective experience of individuals, practitioner seniority, cavitation occurrence, and 3-dimensional kinematics during cervical spine manipulation. J Manipulative Physiol Ther. 2017;40(9):643–8.29229054 10.1016/j.jmpt.2017.09.002
74. Bowley P Holey L Manual therapy education. Does e-learning have a place? Man Ther 2009 14 6 709 711 10.1016/j.math.2009.02.001 19261535
Bowley P, Holey L. Manual therapy education. Does e-learning have a place? Man Ther. 2009;14(6):709–11.19261535 10.1016/j.math.2009.02.001
75. Hou Y Zurada JM Karwowski W Marras WS Davis K Estimation of the dynamic spinal forces using a recurrent fuzzy neural network IEEE Trans Syst Man Cybern B Cybern 2007 37 1 100 109 10.1109/TSMCB.2006.881298 17278564
Hou Y, Zurada JM, Karwowski W, Marras WS, Davis K. Estimation of the dynamic spinal forces using a recurrent fuzzy neural network. IEEE Trans Syst Man Cybern B Cybern. 2007;37(1):100–9.17278564 10.1109/TSMCB.2006.881298
76. Erdemir A McLean S Herzog W van den Bogert AJ Model-based estimation of muscle forces exerted during movements Clin Biomech (Bristol Avon) 2007 22 2 131 154 10.1016/j.clinbiomech.2006.09.005 17070969
Erdemir A, McLean S, Herzog W, van den Bogert AJ. Model-based estimation of muscle forces exerted during movements. Clin Biomech (Bristol Avon). 2007;22(2):131–54.17070969 10.1016/j.clinbiomech.2006.09.005
77. Gangata H Porter S Artz N Major K A proposed anatomy syllabus for entry-level physiotherapists in the United Kingdom: a modified Delphi methodology by physiotherapists who teach anatomy Clin Anat 2023 36 3 503 26 10.1002/ca.24001 36601718
Gangata H, Porter S, Artz N, Major K. A proposed anatomy syllabus for entry-level physiotherapists in the United Kingdom: a modified Delphi methodology by physiotherapists who teach anatomy. Clin Anat. 2023;36(3):503–26.36601718 10.1002/ca.24001
78. Claesson A Hult H Riva G Byrsell F Hermansson T Svensson L Djarv T Ringh M Nordberg P Jonsson M Outline and validation of a new dispatcher-assisted cardiopulmonary resuscitation educational bundle using the Delphi method Resusc Plus 2024 17 100542 10.1016/j.resplu.2023.100542 38268848
Claesson A, Hult H, Riva G, Byrsell F, Hermansson T, Svensson L, Djarv T, Ringh M, Nordberg P, Jonsson M, et al. Outline and validation of a new dispatcher-assisted cardiopulmonary resuscitation educational bundle using the Delphi method. Resusc Plus. 2024;17:100542.38268848 10.1016/j.resplu.2023.100542
79. van den Bogert AJ Analysis and simulation of mechanical loads on the human musculoskeletal system: a methodological overview Exerc Sport Sci Rev 1994 22 23 51 10.1249/00003677-199401000-00004 7925545
van den Bogert AJ. Analysis and simulation of mechanical loads on the human musculoskeletal system: a methodological overview. Exerc Sport Sci Rev. 1994;22:23–51.7925545 10.1249/00003677-199401000-00004
