An AI tutor can raise a homework score and still leave the student worse off in the exam. That is the finding I keep coming back to this week. A large study of students using AI, an employee who walked away from a job rather than opt into it, and a reminder that where AI leadership sits in an organisation seems to change the conversation you get. The common thread is adoption: not whether the tools work, but what they do to learning, to trust, and to the problems leaders feel licensed to take on.
What we mean by an AI tutor is set out on the AI tutor for schools page.
When an AI tutor finishes the homework
A new study of 27,000 students found that those using AI improved their homework scores by 18 per cent and completed assignments much faster. In exams, they scored 20 per cent worse than students who had not used AI.
Wait, before you jump to any conclusions. The interesting part is where the difference came from.
Students who used AI to rush through their homework suffered the biggest drop. Students who used AI but still spent time thinking, working through the problem and understanding the material saw little penalty. In another study, Zara Contractor and Germán Reyes of Middlebury College had undergraduates learn about an unfamiliar topic, either with or without the help of a chatbot. Students using AI scored higher on tests, and the advantage persisted when they repeated the task a week later.
The Economist's write-up is worth sitting with. The tool is not the variable that matters most. The way it is used is. That is an encouraging finding if you are building for learning, and a warning if you are building for answers.
An AI refuser
One AFL employee resigned after being told she could not opt out of the organisation's use of AI. Her concerns ranged from privacy and environmental impact to the effect AI may have on jobs and how people think. The Sydney Morning Herald told her story this week.
AI adoption is moving incredibly quickly, and in many organisations the technology is getting ahead of the people change. I am a strong believer in AI, but I also think we need to take seriously the fact that people will react to this change very differently. Some will run towards it, some will be cautious, some will actively resist it.
The job of leaders is not just to deploy the technology. It is to help people understand what is changing, why it is changing, what it means for their work, and how they can be part of it. The organisations that get AI transformation right will be the ones that bring their people with them.
Where the AI leader sits
I recently saw a Gartner stat that said AI leaders who report directly to the CEO feel 3.3 times more effective than those who sit further down the organisation.
AI touches strategy, operating models, workforce, capability, risk, customer experience and almost every business function. I have noticed that the conversations I have with AI leaders vary significantly depending on where they report. Those closer to the CEO tend to talk about changing the organisation: how work gets done, what capabilities it needs, where value will come from, what the workforce should look like in a few years. Further down the structure, the conversation is much more likely to be about tools, platforms, use cases and delivery.
Both are important. But I think the reporting line has a surprisingly big influence on the size of the problem an AI leader feels empowered to solve. If AI is going to reshape the organisation, its leadership probably needs to sit somewhere that can reshape the organisation too.