The effect of ChatGPT on students’ learning performance, learning perception, and higher-order thinking: insights from a meta-analysis

study suggests that: (1) appropriate learning scaffolds or educational frameworks (e.g., Bloom’s taxonomy) should be provided when using ChatGPT to develop students’ higher-order thinking; (2) the broad use of ChatGPT at various grade levels and in different types of courses should be encouraged to support diverse learning needs; (3) ChatGPT should be actively integrated into different learning modes to enhance student learning, especially in problem-based learning; (4) continuous use of ChatGPT should be ensured to support student learning, with a recommended duration of 4–8 weeks for more stable effects; (5) ChatGPT should be flexibly integrated into teaching as an intelligent tutor, learning partner, and educational tool.

Empirical studies of ChatGPT on students’ learning performance, learning perception, and higher-order thinking. Although many studies have examined the use of ChatGPT in the field of education, as of yet no consensus has been reached on whether it effectively supports students’ learning performance, learning perception, and higher-order thinking. Some scholars argue for its benefits (Boudouaia et al. 2024; Gan et al. 2024; Zhou and Kim, 2024). For example, Emran et al. (2024) found that adopting ChatGPT as an assistive tool in an undergraduate academic writing skills course effectively improved the students’ learning performance. Lu et al. (2024) used ChatGPT as an aid in a teacher training program and found that students aided by ChatGPT scored significantly higher on higher-order thinking than those in the control group who were only exposed to traditional teaching methods. Wu et al. (2023) conducted a quasi-experimental study in a high school mathematics course: the experimental group used ChatGPT for blended learning, while the control group used only iPads for blended learning. The results indicated that the experimental group significantly outperformed the control group in terms of intrinsic motivation, emotional engagement, and self-efficacy.

Conversely, other scholars have argued that ChatGPT hinders students’ learning performance, learning perception, and higher-order thinking (Hays et al. 2024; Rahman and Watanobe, 2023). Niloy et al. (2023) conducted a quasi-experimental study with college students, in which the experimental group used ChatGPT 3.5 to assist with writing in the post-test, while the control group relied solely on publicly available secondary sources. Their results showed that the use of ChatGPT significantly reduced students’ creative writing abilities. Yang et al. (2025) conducted a quasi-experimental study with high school students in a programming course. The experimental group used ChatGPT to assist with learning programming, while the control group was only exposed to traditional teaching methods. The results showed that the experimental group had lower flow experience, self-efficacy, and learning performance compared to the control group. Finally, some experimental studies have demonstrated no significant difference between learning with ChatGPT and without it (Bašić et al. 2023a; Farah et al. 2023). Sun et al. (2024) used ChatGPT with a sample of students from a college programming course. Assessment of their performance revealed no significant difference in scores between the experimental and control groups. Donald et al. (2024) conducted a quasi-experimental study in a university programming course. Their results revealed no significant differences in programming performance or learning interest between the control group, which engaged in self-programming, and the experimental group, which used ChatGPT for assisted programming.