
==== Front
J Med Educ Curric Dev
J Med Educ Curric Dev
MDE
spmde
Journal of Medical Education and Curricular Development
2382-1205
SAGE Publications Sage UK: London, England

10.1177/23821205241278656
10.1177_23821205241278656
Original Research Article
Evaluation of Non-Technical Skills in Students Throughout the Medical Course in Ecuador
Palacios Olga Cecilia Ortiz 1
Pramatárova Mila Inés Moreno 1
Pazin-Filho Antonio 2
1 Facultad de Ciencias de la Discapacidad ,Atencion Prehospitaria y Desastres, Universidad Central del Ecuador, Quito, Ecuador
2 Internal Medicine, University of São Paulo Riberao Preto Medical School, Ribeirao Preto, SP, Brazil
Antonio Pazin Filho, Internal Medicine, University of São Paulo Riberao Preto Medical School, Rua do Professor, 600 apto 143, Ribeirao Preto, SP, Brazil. Email: apazin@fmrp.usp.br
9 9 2024
Jan-Dec 2024
11 238212052412786565 3 2024
12 8 2024
© The Author(s) 2024
2024
SAGE Publications Ltd unless otherwise noted. Manuscript content on this site is licensed under Creative Commons Licenses
https://creativecommons.org/licenses/by-nc/4.0/ This article is distributed under the terms of the Creative Commons Attribution-NonCommercial 4.0 License (https://creativecommons.org/licenses/by-nc/4.0/) which permits non-commercial use, reproduction and distribution of the work without further permission provided the original work is attributed as specified on the SAGE and Open Access page (https://us.sagepub.com/en-us/nam/open-access-at-sage).
OBJECTIVES

This research focuses on assessing non-technical skills (NTS), comprising behavioral aspects that support an individual's expertise in personal and professional performance. Due to varied terminology, including “soft skills” and other terms, NTS is less understood, leading to a potential gap in the literature and limited attention to their development. This study establishes the need for a comprehensive evaluation of NTS to obtain an objective and complete profile of NTS in students pursuing medicine at the Universidad Central del Ecuador (UCE) during 2023, using a standardized and benchmarked psychometric instrument.

METHODS

A cross-sectional study assessed 1035 students from the first to the 12th semester using the CompeTEA® Test, which evaluates 5 factors encompassing 20 competencies and includes sincerity as a reliability condition. Scores were corrected, analyzed, and standardized using TEAcorrige.

RESULTS

The study group comprised 1035 undergraduate medical students (36.8% of students at UCE in 2023), with a mean age of 22.1 years (2.7 standard deviation), and 687 (66.4%) were female. We obtained an average score of 77.7 in sincerity. Factors such as intrapersonal, interpersonal, task development, and managerial skills showed variations throughout the course, with some competencies developing positively in intermediate clinical stages. At the same time, most exhibited a decrease toward the end of the program. Comparisons with population levels revealed 19 competencies at Level 2 and 1 competency predominantly at Level 1.

CONCLUSION

The moderate to low development, with no significant differences based on demographic variables, suggests the need for specific interventions in NTS within comprehensive medical education.

non-technical skills
comprehensive medical education
curriculum
Fundação de Apoio ao Ensino, Pesquisa e Assistência do Hospital das Clínicas da Faculdade de Medicina de Ribeirão Preto da Universidade de São Paulo https://doi.org/10.13039/501100008353 240-CEISH-UCE-2022 typesetterts19
cover-dateJanuary-December 2024
==== Body
pmcIntroduction

Non-technical skills (NTS) are behavioral aspects of performance that support an individual's or professional's expertise. They refer to personal development competencies, constituting an integrated set of knowledge, skills, abilities, aptitudes, personality traits, and motivations inherent or underlying in a person, predisposing them to meet the requirements and demands of their role successfully.1,2 These capabilities reflect cognitive and interpersonal skills that complement technical expertise, ie, technical skills. 3

There is increasing evidence of the importance of developing NTS in all areas, especially for health professions that demand high efficiency, competitiveness, and performance.4–8

Errors in the healthcare environment can be as high as 10%, with human factors responding to 86%. Communication failures, the basis of NTS, comprehends almost half of those human factors. Since healthcare environments are complex systems prone to higher levels of human mistakes, developing NTS could significantly reduce them, increasing patient and professional safety.8,9

The evaluation of the NTS is complex due to the various definitions employed, such as “soft skills,” flexible skills, emotional skills, core skills, and professional competencies. The NTS definition is more inclusive and accurate and, therefore, recommended for standardization. Lozano Fernández 6 and Hinojo Lozano et al 8 provide comprehensive classifications based on performance factors with oneself and others, teamwork, and leadership with various competencies within each factor. Arribas and Pereña Brand 10 offer a complete classification, organizing them into five factors: intrapersonal, interpersonal, task development, environment, and managerial, totaling 20 non-technical competencies.

Several authors emphasize the need for more emphasis on directed learning of NTS in the undergraduate medical curriculum to promote paradigm shifts, particularly among professional educators. Nevertheless, there has yet to be a consensus on evaluating the basal level of NTS or its progression throughout the undergraduate medical curriculum.11–14

In Ecuador, there are no reported evaluations of NTS at the undergraduate medical course level using large samples and standardized instruments, which allow benchmarks to obtain an objective profile. The present study is the first to present an evaluation of such a kind in Ecuador.

Methodology

This descriptive and cross-sectional study aims to obtain an objective profile of NTS in undergraduate medical students from the first up to the 12th semester of 2023 of the Universidad Central del Ecuador (UCE) in Quito, Ecuador.

Inclusion criteria—all students enrolled in undergraduate semesters of the Medicine Program at UCE in Quito during 2023.

Exclusion criteria were (1) failure to complete 100% of responses in the CompeTEA® test; (2) taking more than 25 min to complete the CompeTEA® test; and (3) having perseverative responses, meaning repeatedly answering 1, 1, 1, or 2, 2, 2, 2 more than three times.

After obtaining consent, we presented a dedicated questionnaire to obtain demographic characteristics: age (years), birth gender, number of siblings and hierarchy in the family, nature of funding of fundamental institutions they attended before university admission and previous experience in university courses.

Then, we use a commercially available, standardized psychometric instrument (CompeTEA®) specifically benchmarked for Ecuador. In summary, as described elsewhere, CompeTEA is a self-report questionnaire designed to assess competencies currently understood in the workplace context. 10 The subject has 170 items to complete in up to 30 min, with statements related to various work-related aspects representing multiple behaviors. The individual must respond based on the frequency of these behaviors or the degree of agreement with their content. The assessment covers the 20 most frequently identified competencies in corporate competency dictionaries, grouped into five factors: intrapersonal (self-control and emotional stability, self-confidence, and resilience), interpersonal (communication, relationship building, negotiation, influence, and teamwork), task development (initiative, results orientation, analytical skills, and decision-making), environmental (company knowledge, vision and anticipation, customer orientation, openness, and company identification), and managerial area (leadership, direction, and planning and organization). It also includes a sincerity scale and two qualitative indices: delegation and creativity. Data related to the development of the study (face, content, or construct validity) can be found in the original study by Arribas-Águila. 15 Additionally, the instrument was validated by the Consejo General de la Psicología—España, as evidenced on the website (https://www.cop.es/test/).

We submitted the responses to the online TEA platform. We obtained a direct score and standardized score (SS) for each factor and competency and the classification of NTS development levels for each competency (there is no development level for factors). All the rights of the analyzing process are copyrighted, and the TEA platform provides only the final results.

Statistical analysis

For the calculation of the sample size (students per semester), a 95% confidence level and an estimation error of <7% are considered. With these results, an adjusted sample size of 1103 students to be contacted is obtained.

Descriptive analyses used median and interquartile ranges to measure the central distribution and population dispersion for continuous variables. We used percentages for categorical variables. We used the Kruskal–Wallis and chi-square tests to compare two variables and the Mann–Kendall test to assess the trend of continuous variables over the semesters. We used the Newey–West interrupted time series (ITS) model to determine the impact of curriculum stages (preclinical, clinical, and internship) on competencies and factors. We presented the results as beta coefficients for each semester with their respective 95% confidence intervals. We carried out the analysis and construction of graphs using the statistical software STATA® version 15.

The reporting of this study conforms to the STROBE statement 16 (STROBE_checklist_Supplement).

We obtained ethical approval from the Committee for Research on Human Beings at UCE, endorsed by the Ministry of Public Health in Ecuador (Number 240-CEISH-UCE-2022/code 003-DOCT-FCD-2023).

Results

We enrolled 1443 undergraduate medical students (51.2%) at the Universidad Central del Ecuador (UCE). We excluded 408 students (28.1% out of 1443 enrolled) for inability to answer all questions (205; 50.5% of the 408 students excluded), exceeding the allotted time (152; 37.4% of the 408 students excluded), or exhibiting persistent responses (49; 12.1% of the 408 students excluded). The study group comprised the remaining 1035 students (71.8% of those included and 36.8% of the total students at UCE in 2023). The mean age was 22.1 years (2.7 standard deviations), and 687 (66.4%) were female. We present demographic characteristics in Table 1. Only age and gender were significantly different among the semesters.

Table 1. Socio-demographic characteristics according to semesters in the undergraduate medical program at Universidad Central del Ecuador (UCE).

	01S	02S	03S	04S	05S	06S	07S	08S	09S	10S	11S	12S	p	
Birth gender female N (%)	72 (71.3)	79 (85.8)	61 (71.7)	51 (60.0)	52 (61.2)	61 (71.7)	62 (68.9)	51 (55.4)	51 (57.3)	58 (68.2)	47 (61.8)	42 (60.0)	<0.01	
Age (years)—median (25%, 75%)	18 (18–19)	20 (19–21)	20 (19–21)	20 (20–21)	21 (20–22)	22 (21–23)	22 (21–23)	23 (22–24)	23 (23–24)	24 (23–25)	24 (24–25)	25 (24–27)	<0.01	
Number of siblings—N (%)													0.678	
 0	7 (6.9)	8 (8.7)	6 (7.0)	5 (5.9)	6 (7.0)	4 (4.7)	4 (4.4)	5 (5.4)	5 (5.6)	5 (5.9)	8 (10.5)	3 (4.3)		
 1	25 (24.7)	19 (20.6)	16 (18.8)	18 (21.2)	22 (25.9)	22 (25.9)	27 (30.0)	24 (26.1)	20 (22.4)	11 (12.9)	27 (35.5)	15 (21.4)		
 2	33 (32.6)	29 (31.5)	33 (38.8)	23 (27.0)	24 (28.2)	33 (38.8)	23 (25.5)	25 (27.1)	25 (28.1)	31 (36.4)	11 (14.4)	22 (31.4)		
 3	17 (16.8)	20 (21.4)	19 (22.3)	24 (28.2)	16 (18.8)	16 (18.8)	24 (26.6)	24 (26.1)	22 (24.7)	21 (24.7)	19 (25.0)	16 (22.8)		
 ≥4	19 (18.8)	16 (17.4)	11 (12.9)	15 (17.5)	17 (20.0)	10 (11.7)	12 (13.3)	14 (15.2)	17 (19.1)	17 (20.0)	11 (14.4)	14 (20.0)		
Hierarchy in family—N (%)													0.611	
 1	41 (40.6)	41 (36.4)	31 (48.2)	41 (47.0)	40 (45.9)	39 (53.3)	48 (50.0)	46 (50.0)	42 (47.2)	41 (48.2)	36 (47.3)	34 (48.5)		
 2	38 (37.6)	29 (37.6)	32 (29.4)	25 (27.0)	23 (37.5)	32 (33.3)	30 (28.3)	26 (28.2)	20 (22.4)	21 (24.7)	23 (30.2)	22 (31.4)		
 ≥3	22 (21.8)	22 (25.9)	22 (22.3)	19 (26.0)	22 (16.4)	14 (13.4)	12 (21.7)	20 (21.8)	27 (30.4)	23 (27.1)	15 (19.7)	13 (18.5		
Funding dynamics of fundamental institutions													0.450	
 Public	65 (64.3)	52 (56.5)	48 (56.4)	53 (62.3)	52 (61.2)	60 (70.6)	57 (63.3)	64 (69.5)	61 (68.5)	46 (54.1)	42 (55.2)	42 (60.0)		
 Private	26 (25.7)	27 (29.3)	24 (28.2)	25 (29.4)	26 (30.6)	21 (24.7)	23 (25.5)	22 (23.9)	21 (23.6)	26 (30.6)	22 (28.9)	16 (22.8)		
 Hybrid	10 (9.9)	13 (14.1)	13 (15.3)	7 (8.2)	7 (8.2)	4 (4.7)	10 (11.1)	6 (6.5)	7 (7.8)	13 (15.3)	12 (15.8)	12 (17.1)		
Previous experience in other universities—N(%)	6 (5.9)	12 (13.0)	10 (11.7)	12 (14.1)	9 (10.6)	8 (9.4)	7 (7.8)	10 (10.8)	15 (16.8)	10 (11.7)	13 (17.1)	9 (12.8)	0.499	

We present the SS for every factor and their competencies among the semesters in Table 2.

Table 2. Median and interquartile range of the standardized scores (SS) of factors with their competencies.

FACTORS	COMPETENCES	01S	02S	03S	04S	05S	06S	07S	08S	09S	10S	11S	12S	p	
	Self-control and emotional stability	20 (5, 40)	25 (10, 40)	25 (10, 40)	25 (5, 50)	25 (20, 40)	25 (10, 40)	25 (10, 40)	25 (10, 50)	25 (10, 40)	25 (10, 40)	25 (5, 40)	30 (20, 40)	<0.01	
	Self-confidence	10 (1, 35)	10 (1, 35)	5 (1, 25)	10 (1, 25)	25 (5, 35)	25 (1, 40)	25 (5, 50)	25 (5, 40)	20 (5, 40)	20 (5, 40)	20 (5, 37.5)	20 (10, 40)	<0.01	
	Resilience	20 (5, 35)	25 (10, 35)	20 (5, 35)	20 (10, 35)	25 (10, 35)	20 (10, 35)	20 (5, 35)	25 (10, 38)	20 (10, 35)	25 (5, 35)	25 (10, 40)	25 (10, 40)	0.267	
INTRAPERSONAL		5 (1, 20)	10 (1, 20)	3 (1, 20)	4 (1, 20)	15 (4, 25)	15 (1, 25)	10 (1, 30)	15 (1, 25)	10 (1, 20)	10 (1, 20)	10 (1, 25)	10(1.25)	0.01	
	Communication	20 (10, 40)	20 (10, 45)	20 (15, 40)	30 (10, 40)	30 (10, 40)	20 (10, 40)	30 (15, 45)	30 (15, 45)	20 (10, 40)	30 (15, 45)	30 (15, 40)	30 (20, 45)	0.018	
	Relationship building	25 (10, 35)	30 (10, 35)	25 (10, 35)	25 (10, 35)	25 (10, 35)	30 (15, 35)	27.5 (15, 35)	30 (15, 35)	30 (10, 35)	30 (25, 35)	25 (15, 35)	25 (15, 35)	0.021	
	Negotiation	25 (10, 45)	25 (20, 45)	25 (10, 35)	35 (10, 50)	25 (20, 45)	25 (20, 45)	35 (20, 45)	35 (20, 50)	25 (20, 45)	25 (20, 45)	25 (20, 35)	25 (20, 45)	0.573	
	Influence	30 (5, 50)	30 (25, 50)	30 (15, 30)	30 (15, 40)	30 (25, 50)	30 (15, 50)	40 (25, 55)	30 (25, 50)	30 (15, 50)	30 (15, 50)	30 (25, 40)	30 (15, 40)	0.923	
	Teamwork	20 (5, 35)	30 (15, 40)	30 (15, 35)	20 (15, 35)	20 (15, 35)	30 (15, 35)	30 (15, 40)	32.5 (18, 40)	30 (15, 35)	30 (15, 35)	30 (15, 35)	20 (15, 35)	0.449	
INTERPERSONAL		15 (1, 25)	15 (10, 25)	15 (1, 20)	15 (1, 25)	15 (2, 25)	15 (3, 25)	20 (10, 30)	20 (10, 35)	15 (4, 25)	15 (3, 25)	12.5 (7, 25)	15(3,25)	0.384	
	Initiative	50 (40, 65)	45 (30, 60)	40 (30, 50)	45 (30, 60)	45 (40, 60)	45 (30, 60)	45 (30, 60)	50 (40, 63)	45 (30, 50)	45 (40, 50)	40 (30, 50)	45 (30, 60)	0.098	
	Results orientation	30 (5, 45)	30 (20, 45)	25 (15, 40)	25 (15, 50)	25 (15, 35)	30 (5, 40)	30 (15, 40)	25 (18, 45)	25 (5, 40)	25 (15, 40)	25 (15, 35)	30 (15, 40)	0.799	
	Analytical skills	20 (10, 35)	20 (10, 40)	20 (10, 35)	30 (10, 35)	30 (15, 40)	30 (10, 35)	20 (15, 35)	30 (20, 40)	30 (10, 35)	30 (15, 35)	20 (15, 40)	20 (15, 35)	0.248	
	Decision-making	25 (10, 45)	30 (10, 45)	15 (3, 35)	25 (3, 45)	25 (10, 45)	25 (10, 45)	25 (15, 50)	35 (15, 50)	25 (10, 50)	25 (10, 45)	25 (10, 45)	25 (15, 45)	0.383	
DEVELOPMENT		15 (1, 40)	15 (2, 25)	10 (1, 25)	10 (1, 30)	15 (1, 25)	15 (1, 30)	15 (1, 30)	20 (8, 33)	15 (1, 30)	15 (1, 25)	15 (1, 20)	15 (2, 25)	0.848	
	Company knowledge	1 (1, 15)	5 (1, 15)	1 (1, 10)	5 (1, 10)	1 (1, 10)	5 (1, 10)	1 (1, 10)	5 (1, 15)	1 (1, 15)	5 (1, 15)	10 (1, 15)	10 (5, 15)	<0.01	
	Vision and anticipation	20 (10, 35)	25 (15, 35)	20 (10, 35)	20 (15, 35)	25 (15, 35)	20 (15, 35)	20 (10, 45)	35 (20, 45)	25 (15, 45)	25 (20, 45)	35 (15, 45)	25 (20, 45)	<0.01	
	Customer orientation	15 (10, 35)	25 (10, 40)	25 (15, 35)	25 (10, 40)	25 (10, 35)	15 (10, 35)	25 (10, 35)	25 (15, 40)	25 (10, 35)	15 (10, 35)	20 (10, 35)	15 (10, 25)	0.55	
	Openness	35 (30, 60)	45 (30, 60)	35 (30, 50)	45 (30, 50)	35 (30, 50)	35 (30, 50)	45 (30, 60)	45 (33, 60)	35 (30, 50)	35 (30, 50)	35 (25, 50)	35 (20, 45)	<0.01	
	Company identification	15 (5, 20)	12.5 (5, 30)	10 (4, 20)	10 (4, 20)	5 (1, 15)	5 (4, 15)	10 (4, 20)	10 (4, 20)	5 (1, 15)	5 (1, 15)	10 (4, 20)	10 (4, 15)	<0.01	
ENVIRONMENTAL		1 (1, 5)	1 (1, 10)	1 (1, 1)	1 (1, 10)	1 (1, 2)	1 (1, 5)	1 (1, 10)	1 (1, 10)	1 (1, 3)	1 (1, 10)	1 (1, 10)	1 (1, 10)	0.108	
	Leadership	20 (5, 35)	25 (10, 35)	25 (10, 35)	20 (10, 35)	25 (10, 35)	25 (10, 35)	25 (10, 45)	30 (20, 50)	25 (10, 35)	25 (10, 35)	20 (10, 40)	25 (10, 45)	0.111	
	Direction	30 (15, 55)	35 (18, 50)	35 (20, 50)	35 (20, 50)	30 (15, 50)	35 (20, 55)	40 (25, 55)	40 (30, 65)	35 (20, 55)	35 (20, 50)	40 (22.5)	35 (25, 50)	0.02	
	Planning and organization	15 (5, 30)	25 (10, 35)	25 (10, 35)	25 (10, 35)	25 (15, 35)	20 (10, 35)	30 (15, 45)	30 (15, 45)	25 (10, 35)	30 (15, 45)	25 (15, 40)	25 (15, 35)	<0.01	
MANAGERIAL		10 (1, 20)	10 (2, 23)	10 (2, 25)	10 (2, 20)	10 (2, 20)	10 (1, 25)	15 (3, 30)	15 (10, 30)	10 (1, 25)	10 (1, 25)	10 (2.5, 25)	15 (4, 25)	0.02	
SINCERITY		80 (65, 95)	75 (65, 85)	80 (70, 85)	80 (70, 95)	85 (70, 85)	80 (70, 85)	80 (70, 85)	72.5 (65, 85)	85 (75, 85)	80 (70, 95)	80 (70, 85)	80 (65, 85)	0.744	

The beta coefficients and 95% confidence intervals obtained for ITS were different for the intrapersonal factor between the first two periods (11.5 [5.95, 17.04]); for the interpersonal factor between the first two periods (2 [0.96, 3.03]) and between the last two periods (−2.25 [−3.86, −0.63]); for the task development factor between the first two periods (3.5 [1.24, 5.75]); and the managerial factor between the last two periods (−8.5 [−12.78, −4.21]). We observed no difference in the environmental factor. Competencies nested within the factors demonstrated similar behavior, showing low development in the preclinical period (first to fourth semesters), increasing toward the clinical period (fifth to eighth semesters), and decreasing toward the practical application period (ninth to 12th semesters). The sincerity factor had an average score of 77.7, with no statistically significant difference (p = 0.744) among the semesters. We present the progression of each competency and factor in Table 3, with ITS graphs in Figure 1.

Figure 1. Interrupted time series graphs of standardized scores of each factor with a cut point in the preclinical (x5 xt5) and clinical (x9 xt9) phases.

Table 3. Beta coefficient (95% confidence interval (CI)) for the preclinical (x5 xt5) and clinical (x9 xt9) interrupted time series for each competency and factor.

		NEWEY–WEST	
COEF (95% CI)		t	x5	_xt5	x9	_xt9	
 
INTRAPERSONAL	Self-control and emotional stability	1.5 (−0.5, 3.5)	−2.5 (−6.91, 1.91)	−1.5 (−3.5, 0.5)	−1 (−3.81, 1.81)	1.5 (−0.5, 3.5)	
Self-confidence	−0.5 (−2.98, 1.98)	17.5 (8.26, 26.73)	0.5 (−1.98, 2.98)	−5 (−5, −5)	0 (0, 0)	
Resilience	−0.5 (−2.98, 1.98)	2.5 (−5.88, 1.08)	0.5 (−4.09, 5.09)	−1 (−12.56, 10.56)	1.5 (−2.85, 5.85)	
TOTAL	−1 (−3.58, 1.58)	11.5 (5.95, 17.04)	0.5 (−3.08, 4.08)	−2.5 (−11.73, 6.73)	0.5 (−1.98, 2.98)	
INTERPERSONAL	Communication	3 (−1, 7)	−4 (−22.87, 1.48)	−2 (−8.38, 4.38)	−7 (−19.54, 5.54)	2 (−4.38, 8.38)	
Relationship building	−0.5 (−2.98, 1.98)	1.25 (−5.01, 7.51)	1.75 (−1.21, 4.71)	−0.75 (−4.68, 3.18)	−3.25 (−5.17, −1.32)	
Negotiation	3 (−1, 7)	−11 (−24.86, 2.86)	1 (−3.51, 5.51)	−15 (−21.47, −8.52)	−4 (−6.07, −1.92)	
Influence	0 (0, 0)	1 (1, 1)	1 (1, 1)	−5 (−5, −5)	−1 (−1, −1)	
Teamwork	−0.01 (−7.73, 7.73)	−2.5 (−25.15, 20.15)	3.75 (−4.61, 12.11)	−5.5 (−13.8, 2.8)	−6.75 (−11.86, −1.63)	
TOTAL	−0.01(−0.09, 0.09)	−0.5 (−2.98, 1.98)	2 (0.96, 3.03)	−7.75 (−11.23, −4.26)	−2.25 (−3.86, −0.63)	
DEVELOPMENT	Initiative	−2 (−6, 2)	4 (−9.24, 1.72)	3.5 (−0.97, 7.97)	−5.5 (−12.47, 1.47)	−2 (−5.19, 1.19)	
Results orientation	−2 (−3.03, −0.96)	5 (−2.9, 1.29)	2 (−2, 6)	−3.5 (−14.46, 7.46)	1.5 (−2.85, 5.85)	
Analytical skills	3 (−1, 7)	−1 (−1.49, 1.29)	−4 (−10.38, 2.38)	6 (−1.31, 25.12)	−3 (−8.38, 2.38)	
Decision-making	−1.5 (−6.55, 3.55)	3 (−1.63, 22.31)	4.5 (−1.95, 10.95)	−10 (−22.94, 2.94)	−3 (−7, 1)	
TOTAL	−2(−3.03, −0.96)	6.5 (2.21, 10.78)	3.5 (1.24, 5.75)	−5 (−11.47, 1.47)	−1.5 (−3.5, 0.5)	
ENVIRONMENTAL	Company knowledge	0.8 (−0.86, 2.46)	−3.2 (−9.72, 3.32)	0 (−2.34, 2.34)	−3.3 (−8.86, 2.26)	2.4 (0.04, 4.75)	
Vision and anticipation	−0.5 (−2.98, 1.98)	0.5 (−1.29, 1.39)	3.5 (−4.69, 1.16)	−6.5 (−30.5, 17.5)	−2 (−11.25, 7.25)	
Customer orientation	3 (−1, 7)	−9 (−25.32, 7.32)	−2 (−8.38, 4.38)	−2.5 (−15.03, 10.03)	−3.5 (−9.42, 2.48)	
Openness	2 (−2.15, 6.15)	−11 (−24.86, 2.86)	2 (−2.64, 6.64)	−15 (−21.47, −8.52)	−4 (−6.07, −1.92)	
Company identification	−1.75 (−2.75, −0.74)	−3 (−7.07, 1.07)	3.75 (2.3, 5.19)	−8 (−12.07, −3.92)	0 (−14.68, 1.46)	
TOTAL	0 (0, 0)	0 (0, 0)	0 (0, 0)	0 (0, 0)	0 (0, 0)	
MANAGERIAL	Leadership	0 (−3.86, 3.86)	1.5 (−9.46, 12.46)	1.5 (−2.83, 5.85)	−5.5 (−12.47, 1.47)	−2 (−5.19, 1.19)	
Direction	1.5 (−0.5, 3.5)	−6.5 (−11.73, −1.21)	2 (−0.83, 4.83)	−9.5 (−16.47, −2.52)	−3 (−6.19, 0.19)	
Planning and organization	3 (−1, 7)	−7.5 (−20.27, 5.27)	−0.5 (−5.65, 4.65)	−5.5 (−14.69, 3.69)	−3 (−7.07, 1.07)	
TOTAL	0 (−0.017, 0.017)	−0.5 (−2.98, 1.98)	2 (0.96, 3.03)	−8.5 (−12.78, −4.21)	−0.5 (−2.75, 1.75)	

We show the distribution of the students’ performance in four levels of development (Level 1, the lowest up to the highest in Level 4) according to a reference population produced for Ecuador by CompeTEA® in Table 4. Most students reached only 50% of the expected for an external reference in Level 1 (2 competencies) or Level 2 (17 competencies) development. Less than 21.8% of the students were in Level 3 or 4.

Table 4. Average standardized scores and global competency levels for each skill and competency factor. percentage of students at each level. and the percentage of the total percentage.

			NIVEL (N). NUMBER OF STUDENTS (%)	
			NIVEL 0	NIVEL 1	NIVEL 2	NIVEL 3	NIVEL 4	
FACTORS	COMPETENCIES	MEAN STANDARDIZED SCORES	N (%)	N (%)	N (%)	N (%)	N (%)	
INTRAPERSONAL	Self-control and emotional stability	28	112 (10.8)	359 (34.7)	405 (39.1)	143 (13.8)	16 (1.5)	
Self-confidence	27.7	97 (9.3)	265 (25.6)	412 (39.8)	204 (19.7)	57 (5.5)	
Resilience	24	18 (1.7)	211 (20.4)	491 (47.4)	256 (24.7)	59 (5.7)	
	14.4						
INTERPERSONAL	Communication	28.6	61 (6.0)	272 (26.3)	530 (51.2)	161 (15.5)	11 (1.0)	
Relationship building	27.2	13 (1.2)	51 (4.9)	461 (44.5)	393 (37.9)	117 (11.3)	
Negotiation	30.9	9 (0.8)	176 (17.0)	630 (60.8)	193 (18.6)	27 (2.6)	
Influence	27.1	67 (6.4)	258 (24.9)	529 (51.1)	142 (13.7)	39 (83.7)	
Teamwork	32.3	145 (14.0)	475 (45.9)	334 (32.2)	72 (6.9)	9 (0.8)	
	17.7						
DEVELOPMENT	Initiative	45.1	88 (8.5)	211 (20.4)	512 (49.4)	176 (17.0)	48 (4.6)	
Results orientation	28.6	11 (1.0)	143 (13.8)	507 (49.0)	309 (29.8)	65 (6.3)	
Analytical skills	27	32 (3.1)	191 (18.4)	488 (47.1)	270 (26.1)	54 (5.2)	
Decision-making	29.9	50 (4.8)	543 (52.4)	431 (41.6)	11 (1.0)	0 (0.0)	
	18.3						
ENVIRONMENTAL	Company knowledge	10.2	6 (0.5)	114 (11.0)	541 (52.2)	332 (32.0)	42 (4.0)	
Vision and anticipation	28.4	17 (1.6)	92 (8.9)	510 (49.2)	314 (30.3)	102 (9.8)	
Customer orientation	24	83 (8.0)	305 (26.4)	532 (51.4)	98 (9.4)	17 (1.6)	
Openness	39.4	83 (8.0)	444 (42.9)	444 (42.9)	61 (5.9)	3 (0.3)	
Company identification	12.9	60 (5.8)	189 (18.2)	526 (50.8)	204 (19.7)	56 (5.4)	
	5.6						
MANAGERIAL	Leadership	26	140 (13.5)	381 (36.8)	444 (42.9)	62 (6.0)	8 (0.7)	
Direction	37.9	119 (11.5)	240 (23.2)	412 (39.8)	209 (20.2)	55 (5.3)	
Planning and organization	26.3	118 (11.4)	327 (31.6)	455 (43.9)	115 (11.1)	20 (1.9)	
	16.7						
TOTAL			1329 (6.4)	5247 (25.3)	9594 (46.3)	3725 (18.0)	805 (3.9)	

Discussion

We found out that there were no significant and sustained increments in NTS throughout the undergraduate medical course in the UCE using a reliable and validated instrument. Besides that, the level of development for NTS in comparison with an external reference was below expected.

NTS comprehends only 8 (9.5%) of the 84 signatures of the formal curriculum at the UCE. Of the total time allotted to the medical course (13,160 h), these 8 signatures totalize 324 h (2.4%). Since the time allocated to NTS in the formal curriculum is low, developing these skills should originate from the student's background (family, pre-university education, culture, etc) or the hidden curriculum, as shown in other studies.17,18 We observed no influence from possible upbringing confounding factors such as the number of siblings and hierarchy in the family or external factors such as the type of school attended (a potential indicator for economic level), except for age and birth gender. Data indicate that female students performed better in the last semester of social, administrative, and health careers at 11 universities. Men perform better in leadership and decision-making, while women excel in organization and planning. Nevertheless, considering the instrument selected in this investigation is already adjusted for gender, age, and culture, we did not evaluate these confounders further to avoid overadjustment.17–20

Evaluating the hidden curriculum can be as challenging as evaluating NTS. We tried to assess that with ITS considering the 3 phases of the curriculum (preclinical, clinical, and internship). Some competencies may have been positively influenced by the preclinical phase, developing up to the clinical stages (fifth to eighth semester). Still, they decline toward the end of the course. Any impact of the hidden curriculum in developing competencies throughout the course is not sustainable.12,21,22

Other researchers used CompeTEA® to evaluate the hidden curriculum of different university courses and in professional settings besides medicine.21–24 They found results similar to those we presented.

Having obtained progressive and positive development of NTS toward the clinical period, we observed a decrease in their development toward the end of the course for several competencies and factors. In other words, there is a decrease when interns establish more significant contact with patients, requiring a greater expression of their performance and NTS for patient safety and self-care.

Besides that, we found a different behavior for each NTS factor evaluated in CompeTEA®. The intrapersonal, interpersonal, and managerial factors seem to be more influenced by the phases of the course than the others. We could expect all factors and competencies to develop uniformly throughout the medical course. But up to now, there is no data regarding the validity of this assumption. It would be more intuitive to expect the development of intrapersonal and interpersonal NTS before the student can pay attention to the other factors. If one proves this correct, these findings could impact curriculum development.8,25–27

Our findings are concerning when benchmarking the results obtained at the population level. Out of the 20 assessed competencies, 2 were predominantly at Level 1, 17 competencies predominantly at Level 2, and 1 competency was equally distributed between Level 1 and Level 2. Only 21.8% of students fall into Levels 3 and 4 in a few competencies. Even though there is yet to be an established level the students should achieve, higher levels are desirable when considering the skills they will have to apply in their jobs. Evaluating NTS more systematically, such as in the current investigation, could provide data for curriculum adjustment and follow-up. Moreno-Murcia and Quintero-Pulgar 21 and Rueda Galvis et al 12 obtained similar results, considering students from 3 university courses at different points in the curriculum. They found, on average, scores below the mean in the 5 major areas, obtaining deficient and insufficient scores in terms of skill development: intrapersonal (24.14), interpersonal (28.92), task development (28.48), environment (21.75), and managerial (27.80).

Unfortunately, our findings regarding the time allocated to NTS in medical training are common in Ecuador. Even though academic authorities know the importance of including NTS in the curriculum, we will need more efficient measurements to implement any change. Moreno-Murcia and Quintero-Pulgar 21 and Martínez et al 23 also recommend changes in the curriculum to increase the time allocated to NTS training. Similar to our findings, they argue that there are no statistically significant differences in soft skills concerning the professional cycle and disciplinary training. They emphasize the need for a critical analysis within educational institutions to promote curriculum harmonization or the design of programs for strengthening and developing NTS.

While a new curriculum usually includes innovative strategies, there is no systematic evaluation throughout the course with constant measurement of NTS in subjects exposed to these methodologies. NTS requires significant strengthening for internalization, where both students and teachers acquire theoretical knowledge and practical NTS. 27

Our study opens up several possibilities for enhancing teaching techniques as it adds a quantitative aspect to NTS. Quantitative assessments will enable a more effective awareness among those responsible for curriculum adjustments regarding the importance of including systematic teaching of NTS, although much still needs to be discussed about how and at what point in the curriculum these should be included and evaluated.

Our study has some limitations. First, as we pointed out, studying NTS is very troublesome since there are multiple instruments to measure it. Using an instrument that could evaluate all 4 levels of Kirkpatrick's framework would be desirable. 28 Nevertheless, since it is very difficult and not cost-effective to do so, we chose an instrument with a solid methodology, that includes standards for an external comparison and has been used for more than 10 years.15,22,23,29 Even though this could be a problem when comparing our results to other studies, the available studies are few, and the results are very similar to ours.

Second, all the limitations concerning transversal studies apply, but it is very informative for an initial evaluation of the NTS status in our institution. When we can implement a continuous assessment of NTS throughout the course, we will have data for correlating teaching strategies with NTS outcomes.

Third, we could only enroll approximately one-third of the students at UCE in our study, but this is the largest and possibly the most reliable survey regarding NTS investigation in undergraduate medical students. We also want to note that we excluded some students who underwent the test. We believe this exclusion did not affect our results, as it is expected that a test of this nature would result in some losses.

Conclusion

In conclusion, we need innovative education strategies and a paradigm shift for professional educators to develop NTS in the undergraduate medical curriculum. Objective measurements of NTS will help to evaluate and adapt the strategy available to reach this goal. Since cultural issues are very important in dealing with NTS, evaluating South American countries is paramount.

Supplemental Material

sj-docx-1-mde-10.1177_23821205241278656 - Supplemental material for Evaluation of Non-Technical Skills in Students Throughout the Medical Course in Ecuador

Supplemental material, sj-docx-1-mde-10.1177_23821205241278656 for Evaluation of Non-Technical Skills in Students Throughout the Medical Course in Ecuador by Olga Cecilia Ortiz Palacios, Mila Inés Moreno Pramatárova and Antonio Pazin-Filho in Journal of Medical Education and Curricular Development

Acknowledgments

Not applicable.

Author contributions: OCOP contributed to the research idea, study design, data collection and analysis, and manuscript writing. MIMP contributed to the data collection and analysis and manuscript writing. APF contributed to the research idea, study design, data analysis, and manuscript writing.

Informed consent: All participants signed an informed consent form approved by the ethics committee.

Ethical approval: We obtained ethical approval from the Committee for Research on Human Beings at UCE, endorsed by the Ministry of Public Health in Ecuador (Number 240-CEISH-UCE-2022/code 003-DOCT-FCD-2023).

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

FUNDING: The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was funded by Fundação de Apoio ao Ensino, Pesquisa e Assistência do Hospital das Clínicas da Faculdade de Medicina de Ribeirão Preto da Universidade de São Paulo (240-CEISH-UCE-2022).

Supplemental material: Supplemental material for this article is available online.
==== Refs
References

1 Cabrera Proaño LF García C C J . Hacia una educación de excelencia: el rediseño curricular de medicina aprobado en 2017 genera nuevos cambios. Ciencias_Medicas. 2018;43 (1 ):81–91. doi:10.29166/ciencias_medicas.v43i1.1459
2 Ortiz VHC Prowesk KVS Rodríguez AFU Lesmes CNL Ortiz FA . 2011. Definición y clasificación teórica de las competencias académicas, profesionales y laborales. Las competencias del psicólogo en Colombia.
3 Tseng H Yi X Yeh HT . Learning-related soft skills among online business students in higher education: grade level and managerial role differences in self-regulation, motivation, and social skill. Comput Human Behav. 2019;95 :179–186. doi:10.1016/j.chb.2018.11.035
4 Musicco G . Soft skills & coaching: Motor de la Universidad en Europa [Soft skills & coaching: Engine of the new university in Europe]. Soft skills.
5 Aguinaga Vásquez SJ Sánchez Tarrillo SJ . Énfasis en la formación de habilidades blandas en mejora de los aprendizajes. EEC. 2020;8 (2 ):78–87. doi:10.35383/educare.v8i2.470
6 Lozano Fernández MA Lozano Fernández EN Ortega Cabrejos MY . Habilidades blandas una clave para brindar educación de calidad: Revisión Teórica. Revista Conrado. 2022;18(87) :412–420.
7 Vásquez Villanueva S Vásquez Campos SA Vásquez Villanueva CA Vásquez Villanueva L Castillo Paredes HJ Gomez Miguel JM . Habilidades blandas: su importancia para el desempeño docente. Paidagogo. 2021;3 (2 ):4–16. doi:10.52936/p.v3i2.63
8 Hinojo Lucena FJ Aznar Díaz I Romero JM . Factor humano en la productividad empresarial: un enfoque desde el análisis de las competencias transversales. Innovar. 2020;30 (76 ):51–62. doi:10.15446/innovar.v30n76.85194
9 Portela Romero M Bugarín González R Rodríguez Calvo MS . Error humano, seguridad del paciente y formación en medicina. Educación Médica. 2019;20(S1) :169–174. doi:10.1016/j.edumed.2017.07.027
10 Arribas D Pereña Brand J . Compe TEA: manual. 3a ed., rev. y amp. TEA; 2015.
11 Estévez Abad F Estévez Vélez A . Comunicación efectiva en salud: evaluación de una experiencia docente en estudiantes de medicina de Cuenca, Ecuador. RBD. 2021;52(1) :85–104. doi:10.1344/rbd2021.52.34162
12 Rueda Galvis JF Portilla Castellanos SA . Formación en competencias laborales generales, desafío para laeducación superior. Revinv. 2020;15 :37–44. doi:10.33304/revinv.v15n1-2020004
13 Neves FF Pazin-Filho A . Construindo cenários de simulação: pérolas e armadilhas. Sci Med. 2018;28 (1 ):28579. doi:10.15448/1980-6108.2018.1.28579
14 Lee A Finstad A Gawad N Boet S Raiche I Balaa F . Nontechnical skills (NTS) in the undergraduate surgical and anesthesiology curricula: are we adequately preparing medical students? J Surg Educ. 2021;78 (2 ):502–511. doi:10.1016/j.jsurg.2020.08.001 32839149
15 Arribas-Águila D . Psychometric properties of the TEA personality test: evidence of reliability and construct validity. Eur J Psychol Assess. 2011;27 (2 ):121–126. doi:10.1027/1015-5759/a000057
16 Elm V Altman E Egger DG , et al. The strengthening the reporting of observational studies in epidemiology (STROBE) statement: guidelines for reporting observational studies. Epidemiology. 2007;18 (6 ):800–804. doi:10.1097/EDE.0b013e3181577654 18049194
17 Zhang Y Jiang G Sun Y Zhao X Yu X . Adaptation of the communication skills attitude scale (CSAS) to surgical residents in China. J Surg Educ. 2019;76 (2 ):329–336. doi:10.1016/j.jsurg.2018.07.027 30139694
18 Piedra Durán M . Currículo oculto y no tan oculto de género en la educación superior. Rev Reflexiones. 2021;101 (2 ):1–23. doi: 10.15517/rr.v101i2.45869
19 Brítez G . Importancia del curriculum oculto en el proceso de enseñanza—aprendizaje. Ciencia Latina. 2021;5 (6 ):13859–13870. doi:10.37811/cl_rcm.v5i6.1361
20 Lehmann LS Sulmasy LS Desai S , for the ACP Ethics, Professionalism and Human Rights Committee. Hidden curricula, ethics, and professionalism: optimizing clinical learning environments in becoming and being a physician: a position paper of the American College of Physicians. Ann Intern Med. 2018;168 (7 ):506. doi:10.7326/M17-2058 29482210
21 Moreno-Murcia LM Quintero-Pulgar YA . Relación entre la formación disciplinar y el ciclo profesional en el desarrollo de las habilidades blandas. Form Univ. 2021;14 (3 ):65–74. doi:10.4067/S0718-50062021000300065
22 Gómez GS López OG Vega ID Duarte SR Ponce SL . The impact of gender and academic degrees on the performance of transversal competencies in higher education students. IJHE. 2022;11 (3 ):148. doi:10.5430/ijhe.v11n3p148
23 Martínez PÁL Moreno JJM Brage LB . Analysis of professional competencies in the Spanish public administration management. Rev Psicol Trabajo Y de las Organizaciones. 2014;30 (2 ):61–66. doi:10.1016/j.rpto.2014.06.001
24 Pascual Barrio B Sánchez Prieto L Gomila Grau MA Quesada Serra V Nevot Caldentey L . Formación para la prevención familiar: un análisis de los perfiles profesionales. Ped Soc Rev Int. 2019;34 :31-44.
25 Díaz-Guio DA Henao J Pantoja A Arango MA . Díaz-Gómez AS, Camps Gómez A. Artificial intelligence, applications and challenges in simulation-based education. Colomb J Anesthesiol. 2023;52 (1 ):e1085. doi:10.5554/22562087.e1085
26 Calderón Layedra LR Meza Brito PG Dueñas Matute SE Ruano Rivadeneira TL Erazo Carvajal DA Nunes Aparecido A . Protocolo de evaluación por competencias en la clínica de simulación y robótica. Rev Fac Cien Med (Quito). 2018;43 (2 ):86–100. doi:10.29166/rfcmq.v43i2.2825
27 Scott IA . Errors in clinical reasoning: causes and remedial strategies. Br Med J. 2009;338 :b1860. doi:10.1136/bmj.b1860
28 Cahapay M . Kirkpatrick model: its limitations as used in higher education evaluation. Int J Assess Tools Educ. 2021;8 (1 ):135–144. doi:10.21449/ijate.856143
29 Muñiz J Hernández A Fernández-Hermida JR . Utilización se Los Test en España: El Punto de Vista de los Psicólogos. Pap Psicol. 2020;41 (1 ):1–21. doi: 10.23923/pap.psicol2020.2921
