Does AI make for better grades or better thinkers? – Psychology Today
Whatever your goals, it’s the struggle to get there that’s most rewarding. It’s almost as if life itself is inviting us to embrace difficulty—not as punishment but as a design feature. It's a robust system for growth.
Self Tests are all about you. Are you outgoing or introverted? Are you a narcissist? Does perfectionism hold you back? Find out the answers to these questions and more with Psychology Today.
Posted September 7, 2025 Reviewed by Kaja Perina
Co-authored by Xiaoyan Dong, Hannah Farrell, and Michael Hogan.
Artificial intelligence (AI) is rapidly changing how we learn and develop knowledge and skills. With the development of AI, more and more students in higher education are using AI like ChatGPT to assist them in completing their assignments. But how do students and AI collaborate in academic tasks? What are the key interaction patterns and dynamics shaping learning?
Researchers Deng et al. (2025) recently asked: Does ChatGPT enhance student learning? They conducted a meta-analysis including 69 studies published from 2022 to 2024. Looking at the overall effects across all studies, they concluded that ChatGPT enhances academic performance, affective-emotional states, and higher-order thinking propensities, while also reducing mental effort. However, Weidlich et al. (2025) identified several flaws in the meta-analysis.
First, different interventions were inappropriately combined (e.g., studies using ChatGPT for medical terminology memorisation, writing feedback, and unrestricted essay assistance were treated as equivalent). Second, control groups were wildly diverse, comparing ChatGPT users to groups receiving no help, human tutoring, or even video games rather than identical instruction without ChatGPT. Third, most studies failed to measure actual learning, instead using collaborative performance during ChatGPT use, unreliable self-reports, or tests given during interventions. Critically, studies failed to measure longer-term learning or knowledge retention after ChatGPT assistance was removed.
The reduced mental effort finding in the Deng et al. meta-analysis is also concerning. Could higher grades come at the cost of diminished cognitive engagement? Roman (2025) warns that although AI offers opportunities for personalized learning through adaptive content delivery and greater accessibility and efficiency in the educational process, over-reliance on AI risks eroding students’ critical thinking skills through reduced cognitive effort. A recent study by Luther and colleagues (2024) highlights these key concerns.
Central to student knowledge and skill development are the closely-coupled activities of thinking and writing, which underpin pedagogy and many classroom, homework, and assessment activities. Luther and colleagues (2024) analyzed student-AI interaction dynamics through a writing task, offering a detailed look at how students actually use ChatGPT in timed writing tasks. Of critical concern are the patterns of student dependency and surface-level engagement with AI outputs observed during ‘thinking and writing’. In the study, 135 participants completed a writing task.
Students wrote 600-1000-word essays discussing alcohol prohibition in public spaces, focusing on risky alcohol consumption. During the task, students could collaborate with ChatGPT and had 40 minutes to write their essay. Participants then completed questionnaires about their ChatGPT experience and technology affinity. Some students also participated in retrospective interviews about the task. Importantly, while students were collaborating with ChatGPT, all screen activity was logged using Camtasia 9 software, including all prompts sent to ChatGPT, all ChatGPT’s responses, and the final texts produced by students. The researchers subsequently coded all prompts from 131 participants who used ChatGPT.
Key findings across the diverse student-AI collaboration dynamic were identified. The number of prompts used by students varied significantly from 0 to 25 (4 students didn’t use ChatGPT at all). Most prompts (95.4%) were content-related prompts, and the vast majority of these were requests for data, facts, and information—essentially using ChatGPT as an information source —but also, to a lesser extent, included prompts related to arguments. Shockingly, 40.5% of prompts also included requests for complete texts, and this pattern of prompting was associated with significantly more copy-paste behaviour. Interestingly, participants with higher technology affinity were more likely to request complete texts and use anthropomorphic language when addressing ChatGPT. Similarly, current ChatGPT users were more likely to request complete texts and individual sections compared to non-users.
Students rated ChatGPT higher on competence than warmth, expressing generally high satisfaction. They reported higher levels of cognition-based trust in ChatGPT compared to affect-based trust. Students were generally confident in ChatGPT’s reliable and consistent functioning, task accuracy, and understandability, while also rating it lower on its warmth as a collaborator (i.e., friendly, kind, compassionate).
The use case scenario in the study by Luther and colleagues mirrors common scenarios of tool-supported thinking and writing in higher education. The key issue in the GenAI era is the nature of the tools students have access to. In Luther and colleagues study, students’ approaches to handling AI output were related to familiarity, affinity, and trust in the technology. Unfortunately, higher familiarity, affinity, and trust in ChatGPT’s competence does little to protect students from over-reliance on ChatGPT outputs and copy-and-paste behaviour when writing. As a generative pre-trained transformer trained on vast text data, ChatGPT’s outputs require cross-verification from other sources rather than immediate acceptance and incorporation.
Students need to maintain academic skepticism and scrutinize the AI’s responses through iterative questioning and verification methods. Researchers continue raising concerns about ChatGPT’s accuracy and reliability, and risks of using it for information retrieval in higher education. Educators need to actively remind students of AI’s limitations. Indeed, rather than prompting for data, facts, and information, ChatGPT could be used more constructively as a collaborative tool to foster critical thinking.
Use cases can be designed where students actively engage in iterative exchanges, questioning and refining AI responses. In these cases, ChatGPT acts as a thinking partner supporting both assignment work and the development of knowledge and essential critical thinking skills. Shifting students from tool-like usage toward collaborative, critical approaches represents a crucial transition that educators and policymakers must actively facilitate.
In our next blog post, we’ll explore a completely different GenAI use case and explain why activity design matters. The study was conducted by Zhang and colleagues (2025) and involved five-person debate groups where only one member accessed ChatGPT. Unlike Luther et al.’s study, these students focused on actively filtering and questioning ChatGPT’s suggestions. A process of group accountability created a gatekeeper effect, whereby the ChatGPT user became responsible for scrutinizing AI outputs before sharing with teammates.
This social pressure prevented blind acceptance and positioned learners as orchestrators who guided AI generation while collaboratively refining results, demonstrating AI’s potential to augment rather than replace human intellectual engagement. Ultimately, our view is that use case design is the critical thing to focus on in efforts to use these new GenAI tools to enhance student learning, knowledge and skill development. This is no easy task, but by adopting a design mindset we can think through our pedagogical options and make the best use possible of these new tools.
Connect with the authors: Xiaoyan Dong, Hannah Farrell, Michael Hogan
Michael Hogan, Ph.D., is a lecturer in psychology at the National University of Ireland, Galway.
Get the help you need from a therapist near you–a FREE service from Psychology Today.
Psychology Today © 2025 Sussex Publishers, LLC
Whatever your goals, it’s the struggle to get there that’s most rewarding. It’s almost as if life itself is inviting us to embrace difficulty—not as punishment but as a design feature. It's a robust system for growth.
Self Tests are all about you. Are you outgoing or introverted? Are you a narcissist? Does perfectionism hold you back? Find out the answers to these questions and more with Psychology Today.
