
==== Front
Sci Rep
Sci Rep
Scientific Reports
2045-2322
Nature Publishing Group UK London

70908
10.1038/s41598-024-70908-3
Article
Development and validation of a higher-order thinking skills (HOTS) scale for major students in the interior design discipline for blended learning
Li Dandan 1
Fan Xiaolei 2
Meng Lingchao lcmeng@must.edu.mo

3
1 https://ror.org/05crr5s63 grid.449626.b 0000 0004 1757 860X Faculty of Education, SEGI University, 47810 Petaling Jaya, Selangor, Malaysia
2 Department of Art and Design, Zhengzhou College of Finance and Economics, Zhengzhou, 450000 Henan China
3 https://ror.org/03jqs2n27 grid.259384.1 0000 0000 8945 4455 Faculty of Humanities and Arts, Macau University of Science and Technology, Avenida Wai Long, 999078 Taipa Macao, Special Administrative Region of China
31 8 2024
31 8 2024
2024
14 2028728 2 2024
22 8 2024
© The Author(s) 2024
2024
https://creativecommons.org/licenses/by-nc-nd/4.0/ Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.
Assessing and cultivating students’ HOTS are crucial for interior design education in a blended learning environment. However, current research has focused primarily on the impact of blended learning instructional strategies, learning tasks, and activities on the development of HOTS, whereas few studies have specifically addressed the assessment of these skills through dedicated scales in the context of blended learning. This study aimed to develop a comprehensive scale for assessing HOTS in interior design major students within the context of blended learning. Employing a mixed methods design, the research involved in-depth interviews with 10 education stakeholders to gather qualitative data, which informed the development of a 66-item soft skills assessment scale. The scale was administered to a purposive sample of 359 undergraduate students enrolled in an interior design program at a university in China. Exploratory and confirmatory factor analyses were also conducted to evaluate the underlying factor structure of the scale. The findings revealed a robust four-factor model encompassing critical thinking skills, problem-solving skills, teamwork skills, and practical innovation skills. The scale demonstrated high internal consistency (Cronbach's alpha = 0.948–0.966) and satisfactory convergent and discriminant validity. This scale provides a valuable instrument for assessing and cultivating HOTS among interior design major students in blended learning environments. Future research can utilize a scale to examine the factors influencing the development of these skills and inform instructional practices in the field.

Keywords

Assessment scale
Higher-order thinking skills
Interior design
Blended learning
Subject terms

Psychology
Environmental social sciences
issue-copyright-statement© Springer Nature Limited 2024
==== Body
pmcIntroduction

In the contemporary landscape of the twenty-first century, students face numerous challenges that necessitate the development of competitive skills, with a particular emphasis on the cultivation of HOTS1–3, this has become a crucial objective in educational reform. Notably, it is worth noting that the National Education Association (NEA, 2012) has clearly identified critical thinking and problem-solving, communication, collaboration, creativity, and innovation as key competencies that students must possess in the current era, which are considered important components of twenty-first century skills4–7. As learners in the fields of creativity and design, students in the interior design profession also need to possess HOTS to address complex design problems and the evolving demands of the industry8,9.

Currently, blended learning has become an important instructional model in interior design education10,11. It serves as a teaching approach that combines traditional face-to-face instruction with online learning, providing students with a more flexible and personalized learning experience12,13. Indeed, several scholars have recognized the benefits of blended learning in providing students with diverse learning resources, activities, and opportunities for interaction, thereby fostering HOTS14–17. For example, blended learning, as evidenced by studies conducted by Anthony et al.10 and Castro11, has demonstrated its efficacy in enhancing students' HOTS. The integration of online resources, virtual practices, and online discussions in blended learning fosters active student engagement and improves critical thinking, problem solving, and creative thinking skills. Therefore, teachers need to determine appropriate assessment methods and construct corresponding assessment tasks to assess students' expected learning outcomes. This decision requires teachers to have a clear understanding of students' learning progress and the development of various skills, whereas students have knowledge of only their scores and lack awareness of their individual skill development18,19.

Nevertheless, the precise assessment of students' HOTS in the blended learning milieu poses a formidable challenge. The dearth of empirically validated assessment tools impedes researchers from effectively discerning students' levels of cognitive aptitude and developmental growth within the blended learning realm20–22. In addition, from the perspective of actual research topics, current studies on blended learning focus mainly on the "concept, characteristics, mechanisms, models, and supporting technologies of blended learning23. " Research on "measuring students' HOTS in blended learning" is relatively limited, with most of the focus being on elementary, middle, and high school students24,25. Few studies have specifically examined HOTS measurement in the context of university students26,27, particularly in practical disciplines such as interior design. For example, Bervell et al.28 suggested that the lack of high-quality assessment scales inevitably impacts the quality of research. Additionally, Schmitt29 proposed the “Three Cs” principle for measurement, which includes clarity, coherence, and consistency. He highlighted that high-quality assessment scales should possess clear and specific measurement objectives, logically coherent items, and consistent measurement results to ensure the reliability and validity of the data. This reflects the importance of ensuring the alignment of the measurement goals of assessment scales with the research questions and the content of the discipline in the design of assessments.

The development of an assessment scale within the blended learning environment is expected to address the existing gap in measuring and assessing HOTS scores in interior design education. This scale not only facilitates the assessment of students' HOTS but also serves as a guide for curriculum design, instructional interventions, and student support initiatives. Ultimately, the integration of this assessment scale within the blended learning environment has the potential to optimize the development of HOTS among interior design students, empowering them to become adept critical thinkers, creative problem solvers, and competent professionals in the field.

Therefore, this study follows a scientific scale development procedure to develop an assessment scale specifically designed to measure the HOTS of interior design students in blended learning environments. This endeavor aims to provide educators with a reliable instrument for assessing students' progress in cultivating and applying HOTS, thus enabling the implementation of more effective teaching strategies and enhancing the overall quality of interior design education. The research questions are as follows:What key dimensions should be considered when developing a HOTS assessment scale to accurately capture students' HOTS in an interior design major blended learning environment?

How can an advanced thinking skills assessment scale for blended learning in interior design be developed?

How can the reliability and validity of the HOTS assessment scale be verified and ensured, and is it reliable and effective in the interior design of major blended learning environments?

Results

Key dimensions of HOTS assessment scale in an interior design major blended learning environment

The research results indicate that in the blended learning environment of interior design, this study identified 16 initial codes representing key dimensions for enhancing students' HOTS. These codes were further categorized into 8 main categories and 4 overarching themes: critical thinking, problem-solving, teamwork skills and practical innovation skills. They provide valuable insights for data comprehension and analysis, serving as a comprehensive framework for the HOTS scale. Analyzing category frequency and assessing its significance and universality in a qualitative dataset hold significant analytical value30,31. High-frequency terms indicate the central position of specific categories in participants' narratives, texts, and other data forms32. Through interviews with interior design experts and teachers, all core categories were mentioned more than 20 times, providing compelling evidence of their universality and importance within the field of interior design's HOTS dimensions. As shown in Table 1.Table 1 The main dimensions of HOTS in interior design.

Codes	Categories	Themes	
Spatial planning	Logical reasoning and judgment	Critical thinking skills	
Application of aesthetic elements	
Design concept and direction	Doubt and reflection	
Design process and results	
Functional requirement issues	Identify and define issues	Problem-solving skills	
The issue of spatial quality	
Design strategy formulation	Develop and implement a plan	
Specific implementation plan	
Information transmission and feedback	Communication and coordination	Teamwork skills	
Resolution of contradictions and reaching consensus	
Task allocation	Division of labor and collaboration	
Resource integration and sharing	
Design inspiration	Creative conception and design expression	Practical innovation skills	
Visual presentation and communication	
Exploration and application of new materials	Innovative application of materials and technology	
Integration and practice of innovative technological means	

Themes 1: critical thinking skills

Critical thinking skills constitute a key core category in blended learning environments for interior design and are crucial for cultivating students' HOTS. This discovery emphasizes the importance of critical thinking in interior design learning. This mainly includes the categories of logical reasoning and judgment, doubt and reflection, with a frequency of more than 8, highlighting the importance of critical thinking skills. Therefore, a detailed discussion of each feature is warranted. As shown in Table 2.Table 2 Categories for the main dimension of critical thinking skills.

Categories	Frequency of occurrences	
Logical reasoning and judgment	10/10	
Doubt and reflection	9/10	

Category 1: logical reasoning and judgment

The research results indicate that in a blended learning environment for interior design, logical reasoning and judgment play a key role in cultivating critical thinking skills. Logical reasoning refers to inferring reasonable conclusions from information through analysis and evaluation33. Judgment is based on logic and evidence for decision-making and evaluation. The importance of these concepts lies in their impact on the development and enhancement of students' HOTS. According to the research results, interior design experts and teachers unanimously believe that logical reasoning and judgment are very important. For example, as noted by Interviewee 1, “For students, logical reasoning skills are still very important. Especially in indoor space planning, students use logical reasoning to determine whether the layout of different functional areas is reasonable”. Similarly, Interviewee 2 also stated that “logical reasoning can help students conduct rational analysis of various design element combinations during the conceptual design stage, such as color matching, material selection, and lighting application”.

As emphasized by interviewees 1 and 2, logical reasoning and judgment are among the core competencies of interior designers in practical applications. These abilities enable designers to analyze and evaluate design problems and derive reasonable solutions from them. In the interior design industry, being able to conduct accurate logical reasoning and judgment is one of the key factors for success. Therefore, through targeted training and practice, students can enhance their logical thinking and judgment, thereby better addressing design challenges and providing innovative solutions.

Category 2: skepticism and reflection

Skepticism and reflection play crucial roles in cultivating students' critical thinking skills in a blended learning environment for interior design. Doubt can prompt students to question and explore information and viewpoints, whereas reflection helps students think deeply and evaluate their own thinking process34. These abilities are crucial for cultivating students' higher-order thinking skills. According to the research findings, most interior design experts and teachers agree that skepticism and reflection are crucial. For example, as noted by interviewees 3, “Sometimes, when facing learning tasks, students will think about how to better meet the needs of users”. Meanwhile, Interviewee 4 also agreed with this viewpoint. As emphasized by interviewees 3 and 4, skepticism and reflection are among the core competencies of interior designers in practical applications. These abilities enable designers to question existing perspectives and practices and propose innovative design solutions through in-depth thinking and evaluation. Therefore, in the interior design industry, designers with the ability to doubt and reflect are better able to respond to complex design needs and provide clients with unique and valuable design solutions.

Themes 2: problem-solving skills

The research findings indicate that problem-solving skills constitute a key core category in blended learning environments for interior design and are crucial for cultivating students' HOTS. This discovery emphasizes the importance of problem-solving skills in interior design learning. Specifically, categories such as identifying and defining problems, as well as developing and implementing plans, have been studied more than 8 times, highlighting the importance of problem-solving skills. Therefore, it is necessary to discuss each function in detail to better understand and cultivate students' problem-solving skills. As shown in Table 3.Table 3 Categories for the main dimension of problem-solving skills.

Categories	Frequency of occurrences	
Identify and define issues	10/10	
Develop and implement a plan	10/10	

Category 1: identifying and defining issues

The research findings indicate that in a blended learning environment for interior design, identifying and defining problems play a crucial role in fostering students' problem-solving skills. Identifying and defining problems require students to possess the ability to analyze and evaluate problems, enabling them to accurately determine the essence of the problems and develop effective strategies and approaches to solve them35. Interior design experts and teachers widely recognize the importance of identifying and defining problems as core competencies in interior design practice. For example, Interviewee 5 emphasized the importance of identifying and defining problems, stating, "In interior design, identifying and defining problems is the first step in addressing design challenges. Students need to be able to clearly identify the scope, constraints, and objectives of the problems to engage in targeted thinking and decision-making in the subsequent design process." Interviewee 6 also supported this viewpoint. As stressed by Interviewees 5 and 6, identifying and defining problems not only require students to possess critical thinking abilities but also necessitate broad professional knowledge and understanding. Students need to comprehend principles of interior design, spatial planning, human behavior, and other relevant aspects to accurately identify and define problems associated with design tasks.

Category 2: developing and implementing a plan

The research results indicate that in a blended learning environment for interior design, developing and implementing plans plays a crucial role in cultivating students' problem-solving abilities. The development and implementation of a plan refers to students identifying and defining problems, devising specific solutions, and translating them into concrete implementation plans. Specifically, after determining the design strategy, students refine it into specific implementation steps and timelines, including drawing design drawings, organizing PPT reports, and presenting design proposals. For example, Interviewee 6 noted, “Students usually break down design strategies into specific tasks and steps by refining them.” Other interviewees also unanimously support this viewpoint. As emphasized by respondent 6, developing and implementing plans can help students maintain organizational, systematic, and goal-oriented problem-solving skills, thereby enhancing their problem-solving skills.

Themes 3: teamwork skills

The research results indicate that teamwork skills constitute a key core category in blended learning environments for interior design and are crucial for cultivating students' HOTS. This discovery emphasizes the importance of teamwork skills in interior design learning. This mainly includes communication and coordination and division of labor and collaboration, which are mentioned frequently in the interview documents. Therefore, it is necessary to discuss each function in detail to better understand and cultivate students' teamwork skills. As shown in Table 4.Table 4 Categories for the main dimension of teamwork skills.

Categories	Frequency of occurrences	
Communication and coordination	10/10	
Division of labor and collaboration	10/10	

Category 1: communication and coordination

The research results indicate that communication and collaboration play crucial roles in cultivating students' teamwork abilities in a blended learning environment for interior design. Communication and collaboration refer to the ability of students to effectively share information, understand each other's perspectives, and work together to solve problems36. Specifically, team members need to understand each other's resource advantages integrate and share these resources to improve work efficiency and project quality. For example, Interviewee 7 noted, “In interior design, one member may be skilled in spatial planning, while another member may be skilled in color matching. Through communication and collaboration, team members can collectively utilize this expertise to improve work efficiency and project quality.” Other interviewees also unanimously believe that this viewpoint can promote students' teamwork skills, thereby promoting the development of their HOTS. As emphasized by the viewpoints of these interviewees, communication and collaboration enable team members to collectively solve problems and overcome challenges. Through effective communication, team members can exchange opinions and suggestions with each other, provide different solutions, and make joint decisions. Collaboration and cooperation among team members contribute to brainstorming and finding the best solution.

Category 2: division of labor and collaboration

The research results indicate that in the blended learning environment of interior design, the division of labor and collaboration play crucial roles in cultivating students' teamwork ability. The division of labor and collaboration refer to the ability of team members to assign different tasks and roles in a project based on their respective expertise and responsibilities and work together to complete the project37. For example, Interviewee 8 noted, “In an internal design project, some students are responsible for space planning, some students are responsible for color matching, and some students are responsible for rendering production.” Other interviewees also support this viewpoint. As emphasized by interviewee 8, the division of labor and collaboration help team members fully utilize their respective expertise and abilities, promote resource integration and complementarity, cultivate a spirit of teamwork, and enable team members to collaborate, support, and trust each other to achieve project goals together.

Themes 4: practical innovation skills

The research results indicate that practical innovation skills constitute a key core category in blended learning environments for interior design and are crucial for cultivating students' HOTS. This discovery emphasizes the importance of practical innovation skills in interior design learning. This mainly includes creative conception and design expression, as well as innovative application of materials and technology, which are often mentioned in interview documents. Therefore, it is necessary to discuss each function in detail to better understand and cultivate students' practical innovation skills. As shown in Table 5.Table 5 Categories for the main dimension of practical innovation skills.

Categories	Frequency of occurrences	
Creative conception and design expression	10/10	
Innovative application of materials and technology	10/10	

Category 1: creative conception and design expression

The research results indicate that in the blended learning environment of interior design, creative ideation and design expression play crucial roles in cultivating students' practical and innovative skills. Creative ideation and design expression refer to the ability of students to break free from traditional thinking frameworks and try different design ideas and methods through creative ideation, which helps stimulate their creativity and cultivate their ability to think independently and solve problems. For example, interviewee 10 noted that "blended learning environments combine online and offline teaching modes, allowing students to acquire knowledge and skills more flexibly. Through learning and practice, students can master various expression tools and techniques, such as hand-drawn sketches, computer-aided design software, model making, etc., thereby more accurately conveying their design concepts." Other interviewees also expressed the importance of this viewpoint, emphasizing the importance of creative ideas and design expression in blended learning environments that cannot be ignored. As emphasized by interviewee 10, creative ideation and design expression in the blended learning environment of interior design can not only enhance students' creative thinking skills and problem-solving abilities but also strengthen their application skills in practical projects through diverse expression tools and techniques. The cultivation of these skills is crucial for students' success in their future careers.

Category 2: innovative application of materials and technology

Research findings indicate that the innovative application of materials and technology plays a crucial role in developing students' practical and creative skills within a blended learning environment for interior design. The innovative application of materials and technology refers to students' exploration and utilization of new materials and advanced technologies, enabling them to overcome the limitations of traditional design thinking and experiments with diverse design methods and approaches. This process not only stimulates their creativity but also significantly enhances their problem-solving skills. Specifically, the innovative application of materials and technology involves students gaining a deep understanding of the properties of new materials and their application methods in design, as well as becoming proficient in various advanced technological tools and equipment, such as 3D printing, virtual reality (VR), and augmented reality (AR). These skills enable students to more accurately realize their design concepts and effectively apply them in real-world projects.

For example, Interviewee 1 stated, "The blended learning environment combines online and offline teaching modes, allowing students to flexibly acquire the latest knowledge on materials and technology and apply these innovations in real projects." Other interviewees also emphasized the importance of this view. Therefore, the importance of the innovative application of materials and technology in a blended learning environment cannot be underestimated. As emphasized by interviewee 1, the innovative application of materials and technologies is crucial in the blended learning environment of interior design. This process not only enables students to flexibly acquire the latest materials and technical knowledge but also enables them to apply these innovations to practice in practical projects, thereby improving their practical abilities and professional ethics.

In summary, through research question 1 research, the dimensions of the HOTS assessment scale in blended learning for interior design include four main aspects: critical thinking skills, problem-solving skills, teamwork skills, and practical innovation skills. Based on the assessment scales developed by previous scholars in various dimensions, the researcher developed a HOTS assessment scale suitable for blended learning environments in interior design and collected feedback from interior design experts through interviews.

Development of the HOTS assessment scale

The above research results indicate that the dimensions of the HOTS scale mainly include critical thinking, problem-solving, teamwork skills and practical innovation skills. The dimensions of a scale represent the abstract characteristics and structure of the concept being measured. Since these dimensions are often abstract and difficult to measure directly, they need to be converted into several concrete indicators that can be directly observed or self-reported38. These concrete indicators, known as dimension items, operationalize the abstract dimensions, allowing for the measurement and evaluation of various aspects of the concept. This process transforms the abstract dimensions into specific, measurable components. The following content is based on the results of research question 1 to develop an advanced thinking skills assessment scale for mixed learning in interior design.

Dimension 1: critical thinking skills

The research results indicate that critical thinking skills constitute a key core category in blended learning environments for interior design and are crucial for cultivating students' HOTS. Critical thinking skills refer to the ability to analyze information objectively and make a reasoned judgment39. Scholars tend to emphasize this concept as a method of general skepticism, rational thinking, and self-reflection7,40. For example, Goodsett26 suggested that it should be based on rational skepticism and careful thought about external matters as well as open self-reflection about internal thoughts and actions. Moreover, the California Critical Thinking Disposition Inventory (CCTDI) is widely used to measure critical thinking skills, including dimensions such as seeking truth, confidence, questioning and courage to seek truth, curiosity and openness, as well as analytical and systematic methods41. In addition, maturity means continuous adjustment and improvement of a person's cognitive system and learning activities through continuous awareness, reflection, and self-awareness42. Moreover, Nguyen43 confirmed that critical thinking and cognitive maturity can be achieved through these activities, emphasizing that critical thinking includes cognitive skills such as analysis, synthesis, and evaluation, as well as emotional tendencies such as curiosity and openness.

In addition, in a blended learning environment for interior design, critical thinking skills help students better understand, evaluate, and apply design knowledge and skills, cultivating independent thinking and innovation abilities44. If students lack these skills, they may accept superficial information and solutions without sufficient thinking and evaluation, resulting in the overlooking of important details or the selection of inappropriate solutions in the design process. Therefore, for the measurement of critical thinking skills, the focus should be on cognitive skills such as analysis, synthesis, and evaluation, as well as curiosity and open mindedness. The specific items for critical thinking skills are shown in Table 6.Table 6 Items in the dimension of critical thinking skills.

No	Specific items	Reference source	
A1	I can effectively integrate online learning resources with face-to-face classroom content for design projects	California Critical Thinking Disposition Inventory (CCTDI) (Sulaiman et al.41, Zhou et al.7)	
A2	I can raise critical questions about online learning content during face-to-face discussions	
A3	I can critically analyze and evaluate design-related cases and materials on online learning platforms	
A4	I can synthesize information from both online and in-person learning sources to analyze design problems	
A5	I can apply feedback received from online discussions to face-to-face design work effectively	
A6	I can identify and challenge assumptions and biases in design within a blended learning environment	
A7	I can critically evaluate design theories from online learning and engage in in-depth discussion during face-to-face classes	
A8	I can propose improvements to design solutions based on data and cases collected from online learning resources	
A9	I can appropriately adjust strategies from online and in-person learning to address design issues	
A10	I can effectively apply critical analysis skills gained from online learning to practical design tasks during face-to-face workshops	
A11	I can identify biases in information from online learning resources and suggest corrections during classroom discussions	
A12	I can integrate different perspectives from online and in-person learning to critically analyze and synthesize design solutions	
A13	I can systematically evaluate the strengths and weaknesses of design solutions by combining online and in-person learning	
A14	I can use feedback from online learning to optimize design solutions and discussions conducted in person	
A15	I can critically analyze the suitability of online learning content in relation to actual design needs	
A16	I can propose evidence-based improvements to design solutions based on theories from online learning resources and actual design needs	
A17	I can apply critical thinking skills acquired from online learning to face-to-face design discussions	
A18	I can assess and suggest improvements for the effectiveness of online and in-person learning in a blended learning environment	
A19	I can identify potential design issues through critical analysis of online course materials	
A20	I can combine innovative ideas from online learning with practical design practices in face-to-face settings to propose new solutions	
A21	I can evaluate and provide feedback on the contribution of online learning resources to actual design projects	
A22	I can critically assess design theories and methods proposed in online learning during in-person classes	
A23	I can effectively address and resolve complex design issues within a blended learning environment	
A24	I can apply results from critical analysis during online learning to design decisions in practical projects	

Dimension 2: problem-solving skills

Problem-solving skills constitute a key core category in blended learning environments for interior design and are crucial for cultivating students' HOTS. Problem-solving skills involve the ability to analyze and solve problems by understanding them, identifying their root causes, and developing appropriate solutions45. According to the 5E-based STEM education approach, problem-solving skills encompass the following abilities: problem identification and definition, formulation of problem-solving strategies, problem representation, resource allocation, and monitoring and evaluation of solution effectiveness7,46. Moreover, D'zurilla and Nezu47 and Tan48 indicated that attitudes, beliefs, and knowledge skills during problem solving, as well as the quality of proposed solutions and observable outcomes, are demonstrated. In addition, D'Zurilla and Nezu devised the Social Problem-Solving Inventory (SPSI), which comprises seven subscales: cognitive response, emotional response, behavioral response, problem identification, generation of alternative solutions, decision-making, and solution implementation. Based on these research results, the problem-solving skills dimension questions designed in this study are shown in Table 7.Table 7 Items in the dimension of problem-solving skills.

No	Specific items	Reference source	
B1	I can effectively combine online design theories with hands-on projects in face-to-face settings to solve design problems	Social Problem-Solving Inventory (SPSI) (D’zurilla and Nezu47)	
B2	I usually conduct classroom and online learning with questions and clear goals	
B3	I can use feedback obtained from online learning to optimize design solutions	
B4	I can propose innovative solutions by integrating resources from both online and in-person learning during the design process	
B5	I can effectively coordinate and integrate problem-solving strategies from online courses and face-to-face discussions in a blended learning environment	
B6	I can develop specific problem-solving plans for design projects based on theoretical knowledge gained from online learning	
B7	In face-to-face discussions, I can raise issues related to online learning content and explore their solutions	
B8	I can identify problems from data and case studies obtained on online learning platforms and propose solutions in practical design tasks	
B9	I can apply technologies and tools from online learning to address design challenges in actual design tasks	
B10	I can effectively utilize different resources from online and in-person learning to solve design problems	
B11	I can apply problem-solving methods from online learning to tackle real-world design challenges in face-to-face settings	
B12	I can improve design projects by integrating feedback from both online and in-person learning experiences	

Dimension 3: teamwork skills

The research results indicate that teamwork skills constitute a key core category in blended learning environments for interior design and are crucial for cultivating students' HOTS. Teamwork skills refer to the ability to effectively collaborate, coordinate, and communicate with others in a team environment49. For example, the Teamwork Skills Assessment Tool (TWKSAT) developed by Stevens and Campion50 identifies five core dimensions of teamwork: conflict management; collaborative problem-solving; communication; goal setting; performance management; decision-making; and task coordination. The design of this tool highlights the essential skills in teamwork and provides a structured approach for evaluating these skills. In addition, he indicated that successful teams need to have a range of skills for problem solving, including situational control, conflict management, decision-making and coordination, monitoring and feedback, and an open mindset. These skills help team members effectively address complex challenges and demonstrate the team’s collaboration and flexibility. Therefore, the assessment of learners' teamwork skills needs to cover the above aspects. As shown in Table 8.Table 8 Items in the dimension of teamwork skills.

No	Specific items	Reference source	
C1	I can effectively allocate design tasks and support my team members to complete the work	Teamwork Skills Assessment Tool (TWKSAT) (Lower et al.50, Rodríguez-Sabiote et al. 2022; Zhou et al.7)	
C2	I take responsibility for my assigned tasks and complete them on time	
C3	I can effectively integrate diverse perspectives and suggestions from online learning into face-to-face team discussions	
C4	I help effectively coordinate online and offline resources, time, and tasks to ensure the smooth progress of the project	
C5	I can effectively resolve conflicts and differences that arise during the project	
C6	I ensure that communication channels within our team are open, and I share information and progress promptly	
C7	I listen to others' opinions and provide constructive feedback during discussions	
C8	When encountering design problems, I help the team brainstorm and propose solutions together	
C9	I exhibit a high sense of responsibility and commitment during the design process	
C10	I help effectively manage task priorities to ensure that important tasks are completed on time	
C11	I proactively assist my team members when needed, demonstrating a strong sense of teamwork	
C12	I consider all opinions and work toward reaching a consensus when making design decisions	
C13	I adapt flexibly to changes and uncertainties during the project	
C14	I clearly articulate my design ideas and viewpoints	
C15	I fully consider the overall goals and requirements of the project when completing tasks	
C16	I participate in conducting effective reviews and reflections at the end of the project to improve in the future	

Dimension 4: practice innovative skills

The research results indicate that practical innovation skills constitute a key core category in blended learning environments for interior design, which is crucial for cultivating students' HOTS. The practice of innovative skills encompasses the utilization of creative cognitive processes and problem-solving strategies to facilitate the generation of original ideas, solutions, and approaches51. This practice places significant emphasis on two critical aspects: creative conception and design expression, as well as the innovative application of materials and technology. Tang et al.52 indicated that creative conception and design expression involve the generation and articulation of imaginative and inventive ideas within a given context. With the introduction of concepts such as 21st-century learning skills, the "5C" competency framework, and core student competencies, blended learning has emerged as the goal and direction of educational reform. It aims to promote the development of students' HOTS, equipping them with the essential qualities and key abilities needed for lifelong development and societal advancement. Blended learning not only emphasizes the mastery of core learning content but also requires students to develop critical thinking, complex problem-solving, creative thinking, and practical innovation skills. To adapt to the changes and developments in the blended learning environment, this study designed 13 preliminary test items based on 21st-century learning skills, the "5C" competency framework, core student competencies, and the TTCT assessment scale developed by Torrance53. These items aim to assess students' practice of innovative skills within a blended learning environment, as shown in Table 9.Table 9 Items in the dimension of practice innovative skills.

No	Specific items	Reference source	
D1	I can propose unique and creative ideas during the design process	21st Century Skills Framework (Chu et al.59)

TTCT assessment scale Torrance53)

	
D2	I can apply different design methods and techniques in my projects	
D3	I am willing to try new design tools and software to enhance my design skills	
D4	I can integrate multidisciplinary knowledge and skills to achieve design innovation	
D5	I can effectively solve complex problems encountered during the design process	
D6	I dare to challenge traditional design concepts and methods	
D7	I can draw inspiration from various design cases and apply it to my own work	
D8	I can effectively translate clients' needs into innovative design solutions	
D9	I can quickly adapt to changes and adjust my design strategies accordingly	
D10	I can effectively evaluate and provide feedback on my innovative design ideas	
D11	I can propose innovative design suggestions during teamwork and actively drive project progress	
D12	I can apply critical thinking to evaluate different design options	
D13	I continuously improve my design innovation skills through blended learning and practice	

The researchers' results indicate that the consensus among the interviewed expert participants is that the structural integrity of the scale is satisfactory and does not require modification. However, certain measurement items have been identified as problematic and require revision. The primary recommendations are as follows: Within the domain of problem-solving skills, the item "I usually conduct classroom and online learning with questions and clear goals" was deemed biased because of its emphasis on the "online" environment. Consequently, the evaluation panel advised splitting this item into two separate components: (1) "I am adept at frequently adjusting and reversing a negative team atmosphere" and (2) "I consistently engage in praising and encouraging others, fostering harmonious relationships. “The assessment process requires revisions and adjustments to specific projects, forming a pilot test scale consisting of 66 observable results from the original 65 items. In addition, there were other suggestions about linguistic formulation and phraseology, which are not expounded upon herein.

Verify the effectiveness of the HOTS assessment scale

The research results indicate that there are significant differences in the average scores of the four dimensions of the HOTS, including critical thinking skills (A1–A24 items), problem-solving skills (B1–B13 items), teamwork skills (C1–C16 items), and practical innovation skills (D1–D13 items). Moreover, this also suggests that each item has discriminative power. Specifically, this will be explained through the following aspects.

Project analysis based on the CR value

The critical ratio (CR) method, which uses the CR value (decision value) to remove measurement items with poor discrimination, is the most used method in project analysis. The specific process involves the use of the CR value (critical value) to identify and remove such items. First, the modified pilot test scale data are aggregated and sorted. Individuals representing the top and bottom 27% of the distribution were subsequently selected, constituting 66 respondents in each group. The high-score group comprises individuals with a total score of 127 or above (including 127), whereas the low-score group comprises individuals with a total score of 99 or below (including 99). Finally, an independent sample t test was conducted to determine the significant differences in the mean scores for each item between the high-score and low-score groups. The statistical results are presented in Table 10.Table 10 Independent-samples T test.

Item	T	Item	T	Item	T	
A1	12.869	A23	12.759	C8	11.252	
A2	13.926	A24	11.871	C9	11.512	
A3	14.914	B1	9.539	C10	10.930	
A4	14.004	B2	9.286	C11	10.114	
A5	13.649	B3	11.759	C12	12.124	
A6	12.664	B4	9.220	C13	12.318	
A7	12.034	B5	11.952	C14	11.434	
A8	13.508	B6	11.464	C15	11.566	
A9	12.972	B7	10.022	C16	10.779	
A10	13.244	B8	11.705	D1	11.675	
A11	11.664	B9	10.255	D2	11.210	
A12	10.873	B10	9.919	D3	12.345	
A13	12.977	B11	10.486	D4	11.532	
A14	11.655	B12	11.680	D5	10.954	
A15	12.409	B13	10.578	D6	11.321	
A16	12.373	C1	10.275	D7	11.261	
A17	10.479	C2	11.371	D8	10.692	
A18	13.516	C3	11.349	D9	13.114	
A19	10.968	C4	13.938	D10	12.289	
A20	13.065	C5	11.205	D11	13.549	
A21	13.836	C6	11.711	D12	11.396	
A22	12.594	C7	11.696	D13	10.449	

The above table shows that independent sample t tests were conducted for all the items; their t values were greater than 3, and their p values were less than 0.001, indicating that the difference between the highest and lowest 27% of the samples was significant and that each item had discriminative power.

In summary, based on previous research and relevant theories, the HOTS scale for interior design was revised. This revision process involved interviews with interior design experts, teachers, and students, followed by item examination and homogeneity testing via the critical ratio (CR) method. The results revealed significant correlations (p < 0.01) between all the items and the total score, with correlation coefficients (R) above 0.4. Therefore, the scale exhibits good accuracy and internal consistency in capturing measured HOTS. These findings provide a reliable foundation for further research and practical applications.

Pilot study exploratory factor analysis

This study used SPSS (version 28) to conduct the KMO and Bartlett tests on the scale. The total HOTS test scale as well as the KMO and Bartlett sphericities were first calculated for the four subscales to ensure that the sample data were suitable for factor analysis7. The overall KMO value is 0.946, indicating that the data are highly suitable for factor analysis. Additionally, Bartlett's test of sphericity was significant, further supporting the appropriateness of conducting factor analysis (p < 0.05). All the values are above 0.7, indicating that the data for these subscales are also suitable for factor analysis. According to Javadi et al.54, these results suggest the presence of shared factors among the items within the subscales, as shown in Table 11.Table 11 KMO and Bartlett’s tests.

Sample suitability quantity		0.964	
Bartlett's sphericity test	Approximate chi square	15120.485	
Degree of freedom	2145	
conspicuousness	0.000	

For each subscale, exploratory factor analysis was conducted to extract factors with eigenvalues greater than 1 while eliminating items with communalities less than 0.30, loadings less than 0.50, and items that cross multiple (more than one) common factors55,56. Additionally, items that were inconsistent with the assumed structure of the measure were identified and eliminated to ensure the best structural validity. These principles were applied to the factor analysis of each subscale, ensuring that the extracted factor structure and observed items are consistent with the hypothesized measurement structure and analysis results, as shown in the table55,58. In the exploratory factor analysis (EFA), the latent variables were effectively interpreted and demonstrated a significant response, with cumulative explained variances of the common factors exceeding 60%. This finding confirms the alignment between the scale structure, comprising the remaining items, and the initial theoretical framework proposed in this study. Additionally, the items were systematically reorganized to construct the final questionnaire. Consequently, items A1 to A24 were associated with the critical thinking skills dimension, items B25 to B37 were linked to problem-solving skills, items C38 to C53 were indicative of teamwork skills, and items D54 to D66 were reflective of practical innovation skills. As shown in Table 12 below.Table 12 Total variance explanation.

Composition	Initial eigenvalue	Extracting the sum of squared loads	Sum of squared rotational loads	
Total	Variance percentage	Accrue %	Total	Variance percentage	Accrue %	Total	Variance percentage	Accrue %	
A1	23.439	35.514	35.514		35.514	35.514		20.526	20.526	
A2	6.599	9.998	45.512	23.439	9.998	45.512	13.547	14.612	35.138	
A3	5.028	7.619	53.131	6.599	7.619	53.131	9.644	12.322	47.460	
A4	4.368	6.618	59.748	5.028	6.618	59.748	8.133	12.288	59.748	
A5	0.894	1.355	61.103	4.368			8.110			
A6	0.822	1.245	62.348							
A7	0.816	1.236	63.585							
A8	0.800	1.213	64.797							
A9	0.770	1.167	65.964							
A10	0.749	1.135	67.098							
A11	0.709	1.075	68.173							
A12	0.687	1.040	69.214							
A13	0.671	1.017	70.230							
A14	0.654	0.990	71.221							
A15	0.649	0.984	72.204							
A16	0.638	0.967	73.171							
A17	0.609	0.923	74.094							
A18	0.590	0.894	74.989							
A19	0.581	0.880	75.869							
A20	0.577	0.874	76.743							
A21	0.571	0.865	77.608							
A22	0.561	0.850	78.458							
A23	0.540	0.819	79.276							
A24	0.513	0.777	80.054							
B25	0.508	0.770	80.824							
B26	0.503	0.763	81.586							
B27	0.498	0.754	82.341							
B28	0.490	0.742	83.083							
B29	0.454	0.688	83.771							
B30	0.451	0.684	84.454							
B31	0.434	0.657	85.111							
B32	0.431	0.653	85.764							
B33	0.425	0.644	86.408							
B34	0.419	0.635	87.043							
B35	0.408	0.619	87.661							
B36	0.399	0.605	88.266							
B37	0.383	0.580	88.846							
C38	0.380	0.576	89.422							
C39	0.368	0.558	89.980							
C40	0.357	0.541	90.521							
C41	0.340	0.515	91.036							
C42	0.338	0.512	91.548							
C43	0.322	0.488	92.036							
C44	0.320	0.484	92.520							
C45	0.315	0.477	92.997							
C46	0.308	0.467	93.464							
C47	0.297	0.450	93.914							
C48	0.283	0.428	94.342							
C49	0.272	0.412	94.754							
C50	0.269	0.408	95.163							
C51	0.260	0.394	95.557							
C52	0.250	0.379	95.936							
C53	0.238	0.361	96.297							
D54	0.237	0.360	96.657							
D55	0.226	0.342	96.999							
D56	0.214	0.324	97.323							
D57	0.211	0.320	97.643							
D58	0.206	0.312	97.954							
D59	0.198	0.300	98.254							
D60	0.191	0.289	98.542							
D61	0.184	0.280	98.822							
D62	0.173	0.261	99.083							
D63	0.166	0.251	99.335							
D64	0.154	0.234	99.569							
D65	0.148	0.225	99.794							
D66	0.136	0.206	100.000							
*A1–A24: critical thinking skills dimension; B25–B37: problem-solving skills; C38–C53: teamwork skills; D54–D66: practical innovation skills.

In addition, the criterion for extracting principal components in factor analysis is typically based on eigenvalues, with values greater than 1 indicating greater explanatory power than individual variables. The variance contribution ratio reflects the proportion of variance explained by each principal component relative to the total variance and signifies the ability of the principal component to capture comprehensive information. The cumulative variance contribution ratio measures the accumulated proportion of variance explained by the selected principal components, aiding in determining the optimal number of components to retain while minimizing information loss. The above table shows that four principal components can be extracted from the data, and their cumulative variance contribution rate reaches 59.748%.

However, from the scree plot (as shown in Fig. 1), the slope flattens starting from the fifth factor, indicating that no distinct factors can be extracted beyond that point. Therefore, retaining four factors seems more appropriate. The factor loading matrix is the core of factor analysis, and the values in the matrix represent the factor loading of each item on the common factors. Larger values indicate a stronger correlation between the item variable and the common factor. For ease of analysis, this study used the maximum variance method to rotate the initial factor loading matrix, redistributing the relationships between the factors and original variables and making the correlation coefficients range from 0 to 1, which facilitates interpretation. In this study, factor loadings with absolute values less than 0.4 were filtered out. According to the analysis results, the items of the HOTS assessment scale can be divided into four dimensions, which is consistent with theoretical expectations.Fig. 1 Gravel plot of factors.

Through the pretest of the scale and selection of measurement items, 66 measurement items were ultimately determined. On this basis, a formal scale for assessing HOTS in a blended learning environment was developed, and the reliability and validity of the scale were tested to ultimately confirm its usability.

Confirmatory factor analysis of final testing

Final test employed that AMOS (version 26.0), a confirmatory factor analysis (CFA) was conducted on the retested sample data to validate the stability of the HOTS structural model obtained through exploratory factor analysis. This analysis aimed to assess the fit between the measurement results and the actual data, confirming the robustness of the derived HOTS structure and its alignment with the empirical data. The relevant model was constructed based on the factor structure of each component obtained through EFA and the observed variables, as shown in the diagram. The model fit indices are presented in Fig. 2 (among them, A represents critical thinking skills, B represents problem-solving skills, C represents teamwork skills, and D represents practical innovation skills). The models strongly support the "4-dimensional" structure of the HOTS, which includes four first-order factors: critical thinking skills, problem-solving skills, teamwork skills, and practical innovation skills. Critical thinking skills play a pivotal role in the blended learning environment of interior design, connecting problem-solving skills, teamwork skills, and innovative practices. These four dimensions form the assessment structure of HOTS, with critical thinking skills serving as the core element, inspiring individuals to assess problems and propose innovative solutions. By providing appropriate learning resources, diverse learning activities, and learning tasks, as well as designing items for assessment scales, it is possible to delve into the measurement and development of HOTS in the field of interior design, providing guidance for educational and organizational practices. This comprehensive approach to learning and assessment helps cultivate students' HOTS and lays a solid foundation for their comprehensive abilities in the field of interior design. Thus, the CFA structural models provide strong support for the initial hypothesis of the proposed HOTS assessment structure in this study. As shown in Fig. 2.Fig. 2 Confirmatory factor analysis based on 4 dimensions. *A represents the dimension of critical thinking. B represents the dimension of problem-solving skills. C represents the dimension of teamwork skills. D represents the dimension of practical innovation skills.

Additionally, χ2. The fitting values of RMSEA and SRMR are both below the threshold, whereas the fitting values of the other indicators are all above the threshold, indicating that the model fits well. As shown in Table 13.Table 13 CFA fitting indicators.

	χ2	RMSEA	SRMR	TLI	CFI	IFI	AGFI	PGFI	PNFI	
Threshold	3	0.08	0.08	0.9	0.9	0.9	0.8	0.5	0.5	
Fitted value	1.156	0.022	0.037	0.977	0.978	0.977	0.816	0.775	0.824	

Reliability and validity analysis

The reliability and validity of the scale need to be assessed after the model fit has been determined through validation factor analysis57. Based on the findings of Marsh et al.57, the following conclusions can be drawn. In terms of hierarchical and correlational model fit, the standardized factor loadings of each item range from 0.700 to 0.802, all of which are greater than or equal to 0.7. This indicates a strong correspondence between the observed items and each latent variable. Furthermore, the Cronbach's α coefficients, which are used to assess the internal consistency or reliability of the scale, ranged from 0.948 to 0.966 for each dimension, indicating a high level of data reliability and internal consistency. The composite reliabilities ranged from 0.948 to 0.967, exceeding the threshold of 0.6 and demonstrating a substantial level of consistency (as shown in Table 14).Table 14 CFA fitting indicators.

Combination reliability and convergent validity	
Dimension	Title	Standardized factor load	P	composite reliability	AVE	Cronbach's Alpha	
A	A1	0.720		0.967	0.551	0.966	
A2	0.761	0.000	
A3	0.736	0.000	
A4	0.783	0.000	
A5	0.777	0.000	
A6	0.717	0.000	
A7	0.733	0.000	
A8	0.747	0.000	
A9	0.756	0.000	
A10	0.755	0.000	
A11	0.711	0.000	
A12	0.721	0.000	
A13	0.750	0.000	
A14	0.738	0.000	
A15	0.720	0.000	
A16	0.763	0.000	
A17	0.729	0.000	
A18	0.760	0.000	
A19	0.703	0.000	
A20	0.756	0.000	
A21	0.748	0.000	
A22	0.721	0.000	
A23	0.734	0.000	
A24	0.730	0.000	
B	B1	0.783		0.949	0.589	0.948	
B2	0.764	0.000	
B3	0.780	0.000	
B4	0.754	0.000	
B5	0.758	0.000	
B6	0.779	0.000	
B7	0.741	0.000	
B8	0.802	0.000	
B9	0.741	0.000	
B10	0.726	0.000	
B11	0.779	0.000	
B12	0.782	0.000	
B13	0.765	0.000	
C	C1	0.742		0.954	0.566	0.954	
C2	0.769	0.000	
C3	0.761	0.000	
C4	0.700	0.000	
C5	0.757	0.000	
C6	0.783	0.000	
C7	0.756	0.000	
C8	0.734	0.000	
C9	0.798	0.000	
C10	0.756	0.000	
C11	0.714	0.000	
C12	0.752	0.000	
C13	0.757	0.000	
C14	0.761	0.000	
C15	0.747	0.000	
C16	0.736	0.000	
D	D1	0.763		0.948	0.586	0.948	
D2	0.769	0.000	
D3	0.759	0.000	
D4	0.793	0.000	
D5	0.768	0.000	
D6	0.757	0.000	
D7	0.760	0.000	
D8	0.753	0.000	
D9	0.763	0.000	
D10	0.768	0.000	
D11	0.787	0.000	
D12	0.755	0.000	
D13	0.748	0.000	
*A represents the dimension of critical thinking. B represents the dimension of problem-solving skills. C represents the dimension of teamwork skills. D represents the dimension of practical innovation skills.

Additionally, the diagonal bold font represents the square root of the AVE for each dimension. All the dimensions have average variance extracted (AVE) values ranging from 0.551 to 0.589, all of which are greater than 0.5, indicating that the latent variables have strong explanatory power for their corresponding items. These results suggest that the scale structure constructed in this study is reliable and effective. Furthermore, according to the results presented in Table 15, the square roots of the AVE values for each dimension are greater than the absolute values of the correlations with other dimensions, indicating discriminant validity of the data. Therefore, these four subscales demonstrate good convergent and discriminant validity, indicating that they are both interrelated and independent. This implies that they can effectively capture the content required to complete the HOTS test scale.Table 15 Discriminant validity.

	A	B	C	D	
A	0.742				
B	0.496	0.767			
C	0.456	0.427	0.753		
D	0.466	0.467	0.506	0.765	

Discussion and conclusion

The assessment scale for HOTS in interior design blended learning encompasses four dimensions: critical thinking skills, problem-solving skills, teamwork skills, and practical innovation skills. The selection of these dimensions is based on the characteristics and requirements of the interior design discipline, which aims to comprehensively evaluate students' HOTS demonstrated in blended learning environments to better cultivate their ability to successfully address complex design projects in practice. Notably, multiple studies have shown that HOTSs include critical thinking, problem-solving skills, creative thinking, and decision-making skills, which are considered crucial in various fields, such as education, business, and engineering20,59–61. Compared with prior studies, these dimensions largely mirror previous research outcomes, with notable distinctions in the emphasis on teamwork skills and practical innovation skills62,63. Teamwork skills underscore the critical importance of collaboration in contemporary design endeavors, particularly within the realm of interior design64. Effective communication and coordination among team members are imperative for achieving collective design objectives.

Moreover, practical innovation skills aim to increase students' capacity for creatively applying theoretical knowledge in practical design settings. Innovation serves as a key driver of advancement in interior design, necessitating students to possess innovative acumen and adaptability to evolving design trends for industry success. Evaluating practical innovation skills aims to motivate students toward innovative thinking, exploration of novel concepts, and development of unique design solutions, which is consistent with the dynamic and evolving nature of the interior design sector. Prior research suggests a close interplay between critical thinking, problem-solving abilities, teamwork competencies, and creative thinking, with teamwork skills acting as a regulatory factor for critical and creative thought processes7,65. This interconnected nature of HOTS provides theoretical support for the construction and validation of a holistic assessment framework for HOTS.

After the examination by interior design expert members, one item needed to be split into two items. The results of the CR (construct validity) analysis of the scale items indicate that independent sample t tests were subsequently conducted on all the items. The t values were greater than 3, with p values less than 0.001, indicating significant differences between the top and bottom 27% of the samples and demonstrating the discriminant validity of each item. This discovery highlights the diversity and effectiveness of the scale's internal items, revealing the discriminatory power of the scale in assessing the study subjects. The high t values and significant p values reflect the substantiality of the internal items in distinguishing between different sample groups, further confirming the efficacy of these items in evaluating the target characteristics. These results provide a robust basis for further refinement and optimization of the scale and offer guidance for future research, emphasizing the importance of scale design in research and providing strong support for data interpretation and analysis.

This process involves evaluating measurement scales through EFA, and it was found that the explanatory variance of each subscale reached 59.748%, and the CR, AVE, Cronbach's alpha, and Pearson correlation coefficient values of the total scale and subscales were in a better state, which strongly demonstrates the structure, discrimination, and convergence effectiveness of the scale57.

The scale structure and items of this study are reliable and effective, which means that students in the field of interior design can use them to test their HOTS level and assess their qualities and abilities. In addition, scholars can use this scale to explore the relationships between students' HOTS and external factors, personal personalities, etc., to determine different methods and strategies for developing and improving HOTS.

Limitations and future research

The developed mixed learning HOTS assessment scale for interior design also has certain limitations that need to be addressed in future research. The first issue is that, owing to the requirement of practical innovation skills, students need to have certain practical experience and innovative abilities. First-grade students usually have not yet had sufficient opportunities for learning and practical experience, so it may not be possible to evaluate their abilities effectively in this dimension. Therefore, when this scale is used for assessment, it is necessary to consider students' grade level and learning experience to ensure the applicability and accuracy of the assessment tool. For first-grade students, it may be necessary to use other assessment tools that are suitable for their developmental stage and learning experience to evaluate other aspects of their HOTS7. Future research should focus on expanding the scope of this dimension to ensure greater applicability.

The second issue is that the sample comes from ordinary private undergraduate universities in central China and does not come from national public universities or key universities. Therefore, there may be regional characteristics in the obtained data. These findings suggest that the improved model should be validated with a wider range of regional origins, a more comprehensive school hierarchy, and a larger sample size. The thirdly issue is the findings of this study are derived from self-reported data collected from participants through surveys. However, it is important to note that the literature suggests caution in heavily relying on such self-reported data, as perception does not always equate to actions66. In addition, future research can draw on this scale to evaluate the HOTS of interior design students, explore the factors that affect their development, determine their training and improvement paths, and cultivate skilled talent for the twenty-first century.

Methods

This study adopts a mixed method research approach, combining qualitative and quantitative methods to achieve a comprehensive understanding of the phenomenon67. By integrating qualitative and quantitative research methods, mixed methods research provides a comprehensive and detailed exploration of research questions, using multiple data sources and analytical methods to obtain accurate and meaningful answers68. To increase the quality of the research, the entire study followed the guidelines for scale development procedures outlined by Professor Li after the data were obtained. As shown in Fig. 3Fig. 3 Scale development program.

Basis of theory

This study is guided by educational objectives such as 21st-century learning skills, the "5C" competency framework, and students' core abilities4. The construction process of the scale is based on theoretical foundations, including Bloom's taxonomy. Drawing from existing research, such as the CCTDI41, SPSI69, and TWKSAT scales, the dimensions and preliminary items of the scale were developed. Additionally, to enhance the validity and reliability of the scale, dimensions related to HOTS in interior design were obtained through semi-structured interviews, and the preliminary project adapted or directly cited existing research results. The preliminary items were primarily adapted or directly referenced from existing research findings. Based on existing research, such as the CCTDI, SPSI, TWKSAT, and twenty-first century skills frameworks, this study takes "critical thinking skills, problem-solving skills, teamwork skills, and practical innovative skills" as the four basic dimensions of the scale.

Participants and procedures

This study is based on previous research and develops a HOTS assessment scale to measure the thinking levels of interior design students in blended learning. By investigating the challenges and opportunities students encounter in blended learning environments and exploring the complexity and diversity of their HOTS, this study aims to obtain comprehensive insights. For research question 1, via the purposive sampling method, 10 interior design experts are selected to investigate the dimensions and evaluation indicators of HOTS in blended learning of interior design. The researcher employed a semi structured interview method, and a random sampling technique was used to select 10 senior experts and teachers in the field of interior design, holding the rank of associate professor or above. This included 5 males and 5 females. As shown in Table 16.Table 16 Demographic Information of Sample Experts and Teachers.

Code	Gender	Position	Years of experience	
1	Male	Asst. Professor	10–12	
2	Female	Professor	20	
3	Female	Asst. Professor	15	
4	Male	Asst. Professor	16	
5	Male	Professor	20–22	
6	Female	Professor	20–21	
7	Female	Asst. Professor	15–16	
8	Male	Asst. Professor	14–15	
9	Male	Professor	20–21	
10	Female	Asst. Professor	15–16	

For research question 2 and 3, the research was conducted at an undergraduate university in China, in the field of interior design and within a blended learning environment. In addition, a statement confirms that all experimental plans have been approved by the authorized committee of Zhengzhou University of Finance and Economics. In the process of practice, the methods used were all in accordance with relevant guidelines and regulations, and informed consent was obtained from all participants. The Interior Design Blended Learning HOTS assessment scale was developed based on sample data from 350 students who underwent one pre-test and retest. The participants in the study consisted of second-, third-, and fourth-grade students who had participated in at least one blended learning course. The sample sizes were 115, 118, and 117 for the respective grade levels, totaling 350 individuals. Among the participants, there were 218 male students and 132 female students, all of whom were within the age range of 19–22 years. Through purposeful sampling, this study ensured the involvement of relevant participants and focused on a specific university environment with diverse demographic characteristics and rich educational resources.

This approach enhances the reliability and generalizability of the research and contributes to a deeper understanding of the research question (as shown in Table 17).Table 17 Demographic data of the participants.

Variable	Demographic	Total	Percentage (%)	Sampling	
Gender	Male	218	62.22	Purposeful	
Female	132	37.71	Purposeful	
Grade	Year two	115	32.85	Purposeful	
Year three	118	33.71	Purposeful	
Year four	117	33.42	Purposeful	
Major	Interior design	350	100	Purposeful	
Education	Blended learning	350	100	Purposeful	

Instruments

The tools used in this study include semi structured interview guidelines and the HOTS assessment scale developed by the researchers. For research question 1, the semi structured interview guidelines were reviewed by interior design experts to ensure the accuracy and appropriateness of their content and questions. In addition, for research question 2 and 3, the HOTS assessment scale developed by the researchers will be checked via the consistency ratio (CR) method to assess the consistency and reliability of the scale items and validate their effectiveness.

Data analysis

For research question 1, the researcher will utilize the NVivo version 14 software tool to conduct thematic analysis on the data obtained through semi structured interviews. Thematic analysis is a commonly used qualitative research method that aims to identify and categorize themes, concepts, and perspectives that emerge within a dataset70. By employing NVivo software, researchers can effectively organize and manage large amounts of textual data and extract themes and patterns from them.

For research question 2, the critical ratio (CR) method was employed to conduct item analysis and homogeneity testing on the items of the pilot test questionnaire. The CR method allows for the assessment of each item's contribution to the total score and the evaluation of the interrelationships among the items within the questionnaire. These analytical techniques served to facilitate the evaluation and validation of the scale's reliability and validity.

For research question 3, this study used SPSS (version 26), in which confirmatory factor analysis (CFA) was conducted on the confirmatory sample data via maximum likelihood estimation. The purpose of this analysis was to verify whether the hypothesized factor structure model of the questionnaire aligned with the actual survey data. Finally, several indices, including composite reliability (CR), average variance extracted (CR), average variance extracted (AVE), Cronbach's alpha coefficient, and the Pearson correlation coefficient, were computed to assess the reliability and validity of the developed scale and assess its reliability and validity.

In addition, exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) are commonly utilized techniques in questionnaire development and adaptation research31,70. The statistical software packages SPSS and AMOS are frequently employed for implementing these analytical techniques71–73. CFA is a data-driven approach to factor generation that does not require a predetermined number of factors or specific relationships with observed variables. Its focus lies in the numerical characteristics of the data. Therefore, prior to conducting CFA, survey questionnaires are typically constructed through EFA to reveal the underlying structure and relationships between observed variables and the latent structure.

In contrast, CFA tests the hypothesized model structure under specific theoretical assumptions or structural hypotheses, including the interrelationships among factors and the known number of factors. Its purpose is to validate the hypothesized model structure. Thus, the initial validity of the questionnaire structure, established through EFA, necessitates further confirmation through CFA57,70. Additionally, a sample size of at least 200 is recommended for conducting the validation factor analysis. In this study, confirmatory factor analysis was performed on a sample size of 317.

Acknowledgements

Thanks to the editorial team and reviewers of Scientific Reports for their valuable comments.

Author contributions

D.L. Conceptualized a text experiment, and wrote the main manuscript text. D.L. and X.F. conducted experiments, D.L., X.F. and L.M. analyzed the results. L.M. contributed to the conceptualization, methodology and editing, and critically reviewed the manuscript. All authors have reviewed the manuscript.

Data availability

All data generated or analyzed during this study are included in this published article. All the experimental protocols were approved by the Zhengzhou College of Finance and Economics licensing committee.

Competing interests

The authors declare no competing interests.

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
==== Refs
References

1. Hariadi B Jatmiko B Sunarto M Prahani BK Sagirani T Amelia T Lemantara J Higher order thinking skills based learning outcomes improvement with blended web mobile learning Model Int. J. Instr. 2022 15 2 565 578
Hariadi, B. et al. Higher order thinking skills based learning outcomes improvement with blended web mobile learning Model. Int. J. Instr. 15(2), 565–578 (2022).
2. Sagala PN Andriani A Development of higher-order thinking skills (HOTS) questions of probability theory subject based on bloom’s taxonomy J. Phys. Conf. Ser. 2019 10.1088/1742-6596/1188/1/012025
Sagala, P. N. & Andriani, A. Development of higher-order thinking skills (HOTS) questions of probability theory subject based on bloom’s taxonomy. J. Phys. Conf. Ser.10.1088/1742-6596/1188/1/012025 (2019).10.1088/1742-6596/1188/1/012025
3. Yudha RP Higher order thinking skills (HOTS) test instrument: Validity and reliability analysis with the rasch model Eduma Math. Educ. Learn. Teach. 2023 10.24235/eduma.v12i1.9468
Yudha, R. P. Higher order thinking skills (HOTS) test instrument: Validity and reliability analysis with the rasch model. Eduma Math. Educ. Learn. Teach.10.24235/eduma.v12i1.9468 (2023).10.24235/eduma.v12i1.9468
4. Leach SM Immekus JC French BF Hand B The factorial validity of the Cornell critical thinking tests: A multi-analytic approach Think. Skills Creat. 2020 10.1016/j.tsc.2020.100676
Leach, S. M., Immekus, J. C., French, B. F. & Hand, B. The factorial validity of the Cornell critical thinking tests: A multi-analytic approach. Think. Skills Creat.10.1016/j.tsc.2020.100676 (2020).10.1016/j.tsc.2020.100676
5. Noroozi O Dehghanzadeh H Talaee E A systematic review on the impacts of game-based learning on argumentation skills Entertain. Comput. 2020 10.1016/j.entcom.2020.100369
Noroozi, O., Dehghanzadeh, H. & Talaee, E. A systematic review on the impacts of game-based learning on argumentation skills. Entertain. Comput.10.1016/j.entcom.2020.100369 (2020).10.1016/j.entcom.2020.100369
6. Supena I Darmuki A Hariyadi A The influence of 4C (constructive, critical, creativity, collaborative) learning model on students’ learning outcomes Int. J. Instr. 2021 14 3 873 892 10.29333/iji.2021.14351a
Supena, I., Darmuki, A. & Hariyadi, A. The influence of 4C (constructive, critical, creativity, collaborative) learning model on students’ learning outcomes. Int. J. Instr. 14(3), 873–892. 10.29333/iji.2021.14351a (2021).10.29333/iji.2021.14351a
7. Zhou Y Gan L Chen J Wijaya TT Li Y Development and validation of a higher-order thinking skills assessment scale for pre-service teachers Think. Skills Creat. 2023 10.1016/j.tsc.2023.101272
Zhou, Y., Gan, L., Chen, J., Wijaya, T. T. & Li, Y. Development and validation of a higher-order thinking skills assessment scale for pre-service teachers. Think. Skills Creat.10.1016/j.tsc.2023.101272 (2023).10.1016/j.tsc.2023.101272
8. Musfy K Sosa M Ahmad L Interior design teaching methodology during the global COVID-19 pandemic Interiority 2020 3 2 163 184 10.7454/in.v3i2.100
Musfy, K., Sosa, M. & Ahmad, L. Interior design teaching methodology during the global COVID-19 pandemic. Interiority 3(2), 163–184. 10.7454/in.v3i2.100 (2020).10.7454/in.v3i2.100
9. Yong SD Kusumarini Y Tedjokoesoemo PED Interior design students’ perception for AutoCAD SketchUp and Rhinoceros software usability IOP Conf. Ser. Earth Environ. Sci. 2020 10.1088/1755-1315/490/1/012015
Yong, S. D., Kusumarini, Y. & Tedjokoesoemo, P. E. D. Interior design students’ perception for AutoCAD SketchUp and Rhinoceros software usability. IOP Conf. Ser. Earth Environ. Sci.10.1088/1755-1315/490/1/012015 (2020).10.1088/1755-1315/490/1/012015
10. Anthony B Kamaludin A Romli A Raffei AFM Phon DNALE Abdullah A Ming GL Blended learning adoption and implementation in higher education: A theoretical and systematic review Technol. Knowl. Learn. 2020 27 2 531 578 10.1007/s10758-020-09477-z
Anthony, B. et al. Blended learning adoption and implementation in higher education: A theoretical and systematic review. Technol. Knowl. Learn. 27(2), 531–578. 10.1007/s10758-020-09477-z (2020).10.1007/s10758-020-09477-z
11. Castro R Blended learning in higher education: Trends and capabilities Edu. Inf. Technol. 2019 24 4 2523 2546 10.1007/s10639-019-09886-3
Castro, R. Blended learning in higher education: Trends and capabilities. Edu. Inf. Technol. 24(4), 2523–2546. 10.1007/s10639-019-09886-3 (2019).10.1007/s10639-019-09886-3
12. Alismaiel O Develop a new model to measure the blended learning environments through students’ cognitive presence and critical thinking skills Int. J. Emerg. Technol. Learn. 2022 17 12 150 169 10.3991/ijet.v17i12.30141
Alismaiel, O. Develop a new model to measure the blended learning environments through students’ cognitive presence and critical thinking skills. Int. J. Emerg. Technol. Learn. 17(12), 150–169. 10.3991/ijet.v17i12.30141 (2022).10.3991/ijet.v17i12.30141
13. Gao Y Blended teaching strategies for art design major courses in colleges Int. J. Emerg. Technol. Learn. 2020 10.3991/ijet.v15i24.19033
Gao, Y. Blended teaching strategies for art design major courses in colleges. Int. J. Emerg. Technol. Learn.10.3991/ijet.v15i24.19033 (2020).10.3991/ijet.v15i24.19033
14. Banihashem SK Kerman NT Noroozi O Moon J Drachsler H Feedback sources in essay writing: peer-generated or AI-generated feedback? Int. J. Edu. Technol. Higher Edu. 2024 21 1 23 10.1186/s41239-024-00455-4
Banihashem, S. K., Kerman, N. T., Noroozi, O., Moon, J. & Drachsler, H. Feedback sources in essay writing: peer-generated or AI-generated feedback?. Int. J. Edu. Technol. Higher Edu. 21(1), 23 (2024).10.1186/s41239-024-00455-4
15. Ji, J. A Design on Blended Learning to Improve College English Students’ Higher-Order Thinking Skills. 10.18282/l-e.v10i4.2553 (2021).
16. Noroozi O The role of students’ epistemic beliefs for their argumentation performance in higher education Innov. Edu. Teach. Int. 2023 60 4 501 512 10.1080/14703297.2022.2092188
Noroozi, O. The role of students’ epistemic beliefs for their argumentation performance in higher education. Innov. Edu. Teach. Int. 60(4), 501–512 (2023).10.1080/14703297.2022.2092188
17. Valero Haro A Noroozi O Biemans H Mulder M First- and second-order scaffolding of argumentation competence and domain-specific knowledge acquisition: A systematic review Technol. Pedag. Edu. 2019 28 3 329 345 10.1080/1475939x.2019.1612772
Valero Haro, A., Noroozi, O., Biemans, H. & Mulder, M. First- and second-order scaffolding of argumentation competence and domain-specific knowledge acquisition: A systematic review. Technol. Pedag. Edu. 28(3), 329–345. 10.1080/1475939x.2019.1612772 (2019).10.1080/1475939x.2019.1612772
18. Narasuman S Wilson DM Investigating teachers' implementation and strategies on higher order thinking skills in school based assessment instruments Asian J. Univ. Edu. 2020 10.24191/ajue.v16i1.8991
Narasuman, S. & Wilson, D. M. Investigating teachers’ implementation and strategies on higher order thinking skills in school based assessment instruments. Asian J. Univ. Edu.10.24191/ajue.v16i1.8991 (2020).10.24191/ajue.v16i1.8991
19. Valero Haro A Noroozi O Biemans H Mulder M Argumentation competence: Students’ argumentation knowledge, behavior and attitude and their relationships with domain-specific knowledge acquisition J. Constr. Psychol. 2022 35 1 123 145
Valero Haro, A., Noroozi, O., Biemans, H. & Mulder, M. Argumentation competence: Students’ argumentation knowledge, behavior and attitude and their relationships with domain-specific knowledge acquisition. J. Constr. Psychol. 35(1), 123–145 (2022).
20. Johansson, E. The Assessment of Higher-order Thinking Skills in Online EFL Courses: A Quantitative Content Analysis (2020).
21. Noroozi O Kirschner PA Biemans HJA Mulder M Promoting argumentation competence: Extending from first- to second-order scaffolding through adaptive fading Educ. Psychol. Rev. 2017 30 1 153 176 10.1007/s10648-017-9400-z
Noroozi, O., Kirschner, P. A., Biemans, H. J. A. & Mulder, M. Promoting argumentation competence: Extending from first- to second-order scaffolding through adaptive fading. Educ. Psychol. Rev. 30(1), 153–176. 10.1007/s10648-017-9400-z (2017).10.1007/s10648-017-9400-z
22. Noroozi O Weinberger A Biemans HJA Mulder M Chizari M Facilitating argumentative knowledge construction through a transactive discussion script in CSCL Comput. Educ. 2013 61 59 76 10.1016/j.compedu.2012.08.013
Noroozi, O., Weinberger, A., Biemans, H. J. A., Mulder, M. & Chizari, M. Facilitating argumentative knowledge construction through a transactive discussion script in CSCL. Comput. Educ. 61, 59–76. 10.1016/j.compedu.2012.08.013 (2013).10.1016/j.compedu.2012.08.013
23. Noroozi O Weinberger A Biemans HJA Mulder M Chizari M Argumentation-based computer supported collaborative learning (ABCSCL): A synthesis of 15 years of research Educ. Res. Rev. 2012 7 2 79 106 10.1016/j.edurev.2011.11.006
Noroozi, O., Weinberger, A., Biemans, H. J. A., Mulder, M. & Chizari, M. Argumentation-based computer supported collaborative learning (ABCSCL): A synthesis of 15 years of research. Educ. Res. Rev. 7(2), 79–106. 10.1016/j.edurev.2011.11.006 (2012).10.1016/j.edurev.2011.11.006
24. Setiawan, Baiq Niswatul Khair, Ratnadi Ratnadi, Mansur Hakim, & Istiningsih, S. Developing HOTS-Based Assessment Instrument for Primary Schools (2019).
25. Suparman, S., Juandi, D., & Tamur, M. Does Problem-Based Learning Enhance Students’ Higher Order Thinking Skills in Mathematics Learning? A Systematic Review and Meta-Analysis 2021 4th International Conference on Big Data and Education (2021).
26. Goodsett M Best practices for teaching and assessing critical thinking in information literacy online learning objects J. Acad. Lib. 2020 10.1016/j.acalib.2020.102163
Goodsett, M. Best practices for teaching and assessing critical thinking in information literacy online learning objects. J. Acad. Lib.10.1016/j.acalib.2020.102163 (2020).10.1016/j.acalib.2020.102163
27. Putra, I. N. A. J., Budiarta, L. G. R., & Adnyayanti, N. L. P. E. Developing Authentic Assessment Rubric Based on HOTS Learning Activities for EFL Teachers. In Proceedings of the 2nd International Conference on Languages and Arts across Cultures (ICLAAC 2022) (pp. 155–164). 10.2991/978-2-494069-29-9_17.
28. Bervell B Umar IN Kumar JA Asante Somuah B Arkorful V Blended learning acceptance scale (BLAS) in distance higher education: Toward an initial development and validation SAGE Open 2021 10.1177/21582440211040073
Bervell, B., Umar, I. N., Kumar, J. A., Asante Somuah, B. & Arkorful, V. Blended learning acceptance scale (BLAS) in distance higher education: Toward an initial development and validation. SAGE Open10.1177/21582440211040073 (2021).10.1177/21582440211040073
29. Byrne D A worked example of Braun and Clarke’s approach to reflexive thematic analysis Qual. Quant. 2022 56 3 1391 1412 10.1007/s11135-021-01182-y
Byrne, D. A worked example of Braun and Clarke’s approach to reflexive thematic analysis. Qual. Quant. 56(3), 1391–1412 (2022).10.1007/s11135-021-01182-y
30. Xu W Zammit K Applying thematic analysis to education: A hybrid approach to interpreting data in practitioner research Int. J. Qual. Methods 2020 19 1609406920918810 10.1177/1609406920918810
Xu, W. & Zammit, K. Applying thematic analysis to education: A hybrid approach to interpreting data in practitioner research. Int. J. Qual. Methods 19, 1609406920918810 (2020).10.1177/1609406920918810
31. Braun V Clarke V Conceptual and design thinking for thematic analysis Qual. Psychol. 2022 9 1 3 10.1037/qup0000196
Braun, V. & Clarke, V. Conceptual and design thinking for thematic analysis. Qual. Psychol. 9(1), 3 (2022).10.1037/qup0000196
32. Creswell, A., Shanahan, M., & Higgins, I. Selection-inference: Exploiting large language models for interpretable logical reasoning. arXiv:2205.09712 (2022).
33. Baron J Thinking and Deciding 2023 Cambridge University Press 155 156
Baron, J. Thinking and Deciding 155–156 (Cambridge University Press, 2023).
34. Silver N Kaplan M LaVaque-Manty D Meizlish D Using Reflection and Metacognition to Improve Student Learning: Across the Disciplines, Across the Academy 2023 Taylor & Francis
Silver, N., Kaplan, M., LaVaque-Manty, D. & Meizlish, D. Using Reflection and Metacognition to Improve Student Learning: Across the Disciplines, Across the Academy (Taylor & Francis, 2023).
35. Oksuz K Cam BC Kalkan S Akbas E Imbalance problems in object detection: A review IEEE Trans. Pattern Anal. Mach. Intell. 2020 43 10 3388 3415 10.1109/TPAMI.2020.2981890
Oksuz, K., Cam, B. C., Kalkan, S. & Akbas, E. Imbalance problems in object detection: A review. IEEE Trans. Pattern Anal. Mach. Intell. 43(10), 3388–3415 (2020).10.1109/TPAMI.2020.2981890
36. Saputra MD Joyoatmojo S Wardani DK Sangka KB Developing critical-thinking skills through the collaboration of jigsaw model with problem-based learning model Int. J. Instr. 2019 12 1 1077 1094
Saputra, M. D., Joyoatmojo, S., Wardani, D. K. & Sangka, K. B. Developing critical-thinking skills through the collaboration of jigsaw model with problem-based learning model. Int. J. Instr. 12(1), 1077–1094 (2019).
37. Imam H Zaheer MK Shared leadership and project success: The roles of knowledge sharing, cohesion and trust in the team Int. J. Project Manag. 2021 39 5 463 473 10.1016/j.ijproman.2021.02.006
Imam, H. & Zaheer, M. K. Shared leadership and project success: The roles of knowledge sharing, cohesion and trust in the team. Int. J. Project Manag. 39(5), 463–473 (2021).10.1016/j.ijproman.2021.02.006
38. DeCastellarnau A A classification of response scale characteristics that affect data quality: A literature review Qual. Quant. 2018 52 4 1523 1559 10.1007/s11135-017-0533-4 29937582
DeCastellarnau, A. A classification of response scale characteristics that affect data quality: A literature review. Qual. Quant. 52(4), 1523–1559 (2018).29937582 10.1007/s11135-017-0533-4
39. Haber J Critical Thinking 2020 MIT Press 145 146
Haber, J. Critical Thinking 145–146 (MIT Press, 2020).
40. Hanscomb S Critical Thinking: The Basics 2023 Routledge 180 181
Hanscomb, S. Critical Thinking: The Basics 180–181 (Routledge, 2023).
41. Sulaiman WSW Rahman WRA Dzulkifli MA Examining the construct validity of the adapted California critical thinking dispositions (CCTDI) among university students in Malaysia Proc. Social Behav. Sci. 2010 7 282 288 10.1016/j.sbspro.2010.10.039
Sulaiman, W. S. W., Rahman, W. R. A. & Dzulkifli, M. A. Examining the construct validity of the adapted California critical thinking dispositions (CCTDI) among university students in Malaysia. Proc. Social Behav. Sci. 7, 282–288 (2010).10.1016/j.sbspro.2010.10.039
42. Jaakkola N Karvinen M Hakio K Wolff L-A Mattelmäki T Friman M Becoming self-aware—How do self-awareness and transformative learning fit in the sustainability competency discourse? Front. Educ. 2022 10.3389/feduc.2022.855583
Jaakkola, N. et al. Becoming self-aware—How do self-awareness and transformative learning fit in the sustainability competency discourse?. Front. Educ.10.3389/feduc.2022.855583 (2022).10.3389/feduc.2022.855583
43. Nguyen TTB Critical thinking: What it means in a Vietnamese tertiary EFL context English For. Language Int. J. 2022 2 3 4 23
Nguyen, T. T. B. Critical thinking: What it means in a Vietnamese tertiary EFL context. English For. Language Int. J. 2(3), 4–23 (2022).
44. Henriksen D Gretter S Richardson C Design thinking and the practicing teacher: Addressing problems of practice in teacher education Teach. Educ. 2020 31 2 209 229 10.1080/10476210.2018.1531841
Henriksen, D., Gretter, S. & Richardson, C. Design thinking and the practicing teacher: Addressing problems of practice in teacher education. Teach. Educ. 31(2), 209–229 (2020).10.1080/10476210.2018.1531841
45. Okes D Root cause analysis: The core of problem solving and corrective action 2019 Quality Press 179 180
Okes, D. Root cause analysis: The core of problem solving and corrective action 179–180 (Quality Press, 2019).
46. Eroğlu S Bektaş O The effect of 5E-based STEM education on academic achievement, scientific creativity, and views on the nature of science Learn. Individual Differ. 2022 98 102181 10.1016/j.lindif.2022.102181
Eroğlu, S. & Bektaş, O. The effect of 5E-based STEM education on academic achievement, scientific creativity, and views on the nature of science. Learn. Individual Differ. 98, 102181 (2022).10.1016/j.lindif.2022.102181
47. Dzurilla TJ Nezu AM Development and preliminary evaluation of the social problem-solving inventory Psychol. Assess. J. Consult. Clin. Psychol. 1990 2 2 156
Dzurilla, T. J. & Nezu, A. M. Development and preliminary evaluation of the social problem-solving inventory. Psychol. Assess. J. Consult. Clin. Psychol. 2(2), 156 (1990).
48. Tan, O.-S. Problem-based learning innovation: Using problems to power learning in the 21st century. Gale Cengage Learning (2021).
49. Driskell JE Salas E Driskell T Foundations of teamwork and collaboration Am. Psychol. 2018 73 4 334 10.1037/amp0000241 29792452
Driskell, J. E., Salas, E. & Driskell, T. Foundations of teamwork and collaboration. Am. Psychol. 73(4), 334 (2018).29792452 10.1037/amp0000241
50. Lower LM Newman TJ Anderson-Butcher D Validity and reliability of the teamwork scale for youth Res. Social Work Pract. 2017 27 6 716 725 10.1177/1049731515589614
Lower, L. M., Newman, T. J. & Anderson-Butcher, D. Validity and reliability of the teamwork scale for youth. Res. Social Work Pract. 27(6), 716–725 (2017).10.1177/1049731515589614
51. Landa R Advertising by design: generating and designing creative ideas across media 2021 Wiley
Landa, R. Advertising by design: generating and designing creative ideas across media (Wiley, 2021).
52. Tang T Vezzani V Eriksson V Developing critical thinking, collective creativity skills and problem solving through playful design jams Think. Skills Creat. 2020 37 100696 10.1016/j.tsc.2020.100696
Tang, T., Vezzani, V. & Eriksson, V. Developing critical thinking, collective creativity skills and problem solving through playful design jams. Think. Skills Creat. 37, 100696 (2020).10.1016/j.tsc.2020.100696
53. Torrance, E. P. Torrance tests of creative thinking. Educational and psychological measurement (1966).
54. Javadi MH Khoshnami MS Noruzi S Rahmani R Health anxiety and social health among health care workers and health volunteers exposed to coronavirus disease in Iran: A structural equation modeling J. Affect. Disord. Rep. 2022 10.1016/j.jadr.2022.100321 35165675
Javadi, M. H., Khoshnami, M. S., Noruzi, S. & Rahmani, R. Health anxiety and social health among health care workers and health volunteers exposed to coronavirus disease in Iran: A structural equation modeling. J. Affect. Disord. Rep.10.1016/j.jadr.2022.100321 (2022).35165675 10.1016/j.jadr.2022.100321
55. Hu L Bentler PM Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives Struct. Equ. Model. Multidiscip. J. 1999 6 1 1 55 10.1080/10705519909540118
Hu, L. & Bentler, P. M. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Struct. Equ. Model. Multidiscip. J. 6(1), 1–55. 10.1080/10705519909540118 (1999).10.1080/10705519909540118
56. Matsunaga M Item parceling in structural equation modeling: A primer Commun. Methods Measures 2008 2 4 260 293 10.1080/19312450802458935
Matsunaga, M. Item parceling in structural equation modeling: A primer. Commun. Methods Measures 2(4), 260–293. 10.1080/19312450802458935 (2008).10.1080/19312450802458935
57. Marsh HW Morin AJ Parker PD Kaur G Exploratory structural equation modeling: An integration of the best features of exploratory and confirmatory factor analysis Ann. Rev. Clin. Psychol. 2014 10 1 85 110 10.1146/annurev-clinpsy-032813-153700 24313568
Marsh, H. W., Morin, A. J., Parker, P. D. & Kaur, G. Exploratory structural equation modeling: An integration of the best features of exploratory and confirmatory factor analysis. Ann. Rev. Clin. Psychol. 10(1), 85–110 (2014).24313568 10.1146/annurev-clinpsy-032813-153700
58. Song Y Lee Y Lee J Mediating effects of self-directed learning on the relationship between critical thinking and problem-solving in student nurses attending online classes: A cross-sectional descriptive study Nurse Educ. Today 2022 10.1016/j.nedt.2021.105227 35797836
Song, Y., Lee, Y. & Lee, J. Mediating effects of self-directed learning on the relationship between critical thinking and problem-solving in student nurses attending online classes: A cross-sectional descriptive study. Nurse Educ. Today10.1016/j.nedt.2021.105227 (2022).35797836 10.1016/j.nedt.2021.105227
59. Chu, S. K. W., Reynolds, R. B., Tavares, N. J., Notari, M., & Lee, C. W. Y. 21st century skills development through inquiry-based learning from theory to practice. Springer (2021).
60. Eliyasni R Kenedi AK Sayer IM Blended learning and project based learning: the method to improve students’ higher order thinking skill (HOTS) Jurnal Iqra': Kajian Ilmu Pendidikan 2019 4 2 231 248 10.25217/ji.v4i2.549
Eliyasni, R., Kenedi, A. K. & Sayer, I. M. Blended learning and project based learning: the method to improve students’ higher order thinking skill (HOTS). Jurnal Iqra’: Kajian Ilmu Pendidikan 4(2), 231–248 (2019).10.25217/ji.v4i2.549
61. Yusuf P Istiyono Blended learning: Its effect towards higher order thinking skills (HOTS) J. Phys. Conf. Ser. 2021 10.1088/1742-6596/1832/1/012039
Yusuf, P. & Istiyono,. Blended learning: Its effect towards higher order thinking skills (HOTS). J. Phys. Conf. Ser.10.1088/1742-6596/1832/1/012039 (2021).10.1088/1742-6596/1832/1/012039
62. Byron K Keem S Darden T Shalley CE Zhou J Building blocks of idea generation and implementation in teams: A meta-analysis of team design and team creativity and innovation Personn. Psychol. 2023 76 1 249 278 10.1111/peps.12501
Byron, K., Keem, S., Darden, T., Shalley, C. E. & Zhou, J. Building blocks of idea generation and implementation in teams: A meta-analysis of team design and team creativity and innovation. Personn. Psychol. 76(1), 249–278 (2023).10.1111/peps.12501
63. Walid A Sajidan S Ramli M Kusumah RGT Construction of the assessment concept to measure students' high order thinking skills J. Edu. Gift. Young Sci. 2019 7 2 237 251
Walid, A., Sajidan, S., Ramli, M. & Kusumah, R. G. T. Construction of the assessment concept to measure students’ high order thinking skills. J. Edu. Gift. Young Sci. 7(2), 237–251 (2019).
64. Alawad A Evaluating online learning practice in the interior design studio Int. J. Art Des. Edu. 2021 40 3 526 542 10.1111/jade.12365
Alawad, A. Evaluating online learning practice in the interior design studio. Int. J. Art Des. Edu. 40(3), 526–542. 10.1111/jade.12365 (2021).10.1111/jade.12365
65. Awuor NO Weng C Militar R Teamwork competency and satisfaction in online group project-based engineering course: The cross-level moderating effect of collective efficacy and flipped instruction Comput. Educ. 2022 176 104357 10.1016/j.compedu.2021.104357
Awuor, N. O., Weng, C. & Militar, R. Teamwork competency and satisfaction in online group project-based engineering course: The cross-level moderating effect of collective efficacy and flipped instruction. Comput. Educ. 176, 104357 (2022).10.1016/j.compedu.2021.104357
66. Noroozi O Alqassab M Taghizadeh Kerman N Banihashem SK Panadero E Does perception mean learning? Insights from an online peer feedback setting Assess. Eval. Higher Edu. 2024 10.1080/02602938.2024.2345669
Noroozi, O., Alqassab, M., Taghizadeh Kerman, N., Banihashem, S. K. & Panadero, E. Does perception mean learning? Insights from an online peer feedback setting. Assess. Eval. Higher Edu.10.1080/02602938.2024.2345669 (2024).10.1080/02602938.2024.2345669
67. Creswell, J. W. A concise introduction to mixed methods research. SAGE publications124–125(2021).
68. Tashakkori, A., Johnson, R. B., & Teddlie, C. Foundations of mixed methods research: Integrating quantitative and qualitative approaches in the social and behavioral sciences. Sage Publications 180–181(2020).
69. Jiang X Lyons MD Huebner ES An examination of the reciprocal relations between life satisfaction and social problem solving in early adolescents J. Adolescence 2016 53 1 141 151 10.1016/j.adolescence.2016.09.004
Jiang, X., Lyons, M. D. & Huebner, E. S. An examination of the reciprocal relations between life satisfaction and social problem solving in early adolescents. J. Adolescence 53(1), 141–151. 10.1016/j.adolescence.2016.09.004 (2016).10.1016/j.adolescence.2016.09.004
70. Orcan F Exploratory and confirmatory factor analysis: Which one to use first Egitimde ve Psikolojide Olçme ve Degerlendirme Dergisi 2018 10.21031/epod.394323
Orcan, F. Exploratory and confirmatory factor analysis: Which one to use first. Egitimde ve Psikolojide Olçme ve Degerlendirme Dergisi10.21031/epod.394323 (2018).10.21031/epod.394323
71. Asparouhov T Muthén B Exploratory structural equation modeling Struct. Eq. Model. Multidiscip. J. 2009 16 3 397 438 10.1080/10705510903008204
Asparouhov, T. & Muthén, B. Exploratory structural equation modeling. Struct. Eq. Model. Multidiscip. J. 16(3), 397–438 (2009).10.1080/10705510903008204
72. Finch, H., French, B. F., & Immekus, J. C. Applied psychometrics using spss and amos. IAP (2016).
73. Marsh HW Guo J Dicke T Parker PD Craven RG Confirmatory factor analysis (CFA), exploratory structural equation modeling (ESEM), and Set-ESEM: Optimal balance between goodness of fit and parsimony Multivar. Behav. Res. 2020 55 1 102 119 10.1080/00273171.2019.1602503
Marsh, H. W., Guo, J., Dicke, T., Parker, P. D. & Craven, R. G. Confirmatory factor analysis (CFA), exploratory structural equation modeling (ESEM), and Set-ESEM: Optimal balance between goodness of fit and parsimony. Multivar. Behav. Res. 55(1), 102–119. 10.1080/00273171.2019.1602503 (2020).10.1080/00273171.2019.1602503
