CurricuLLM LogoCurricuLLM
SchoolsAI tutorFeaturesPricingTraining HubDevelopers
An AI tutor should not rewrite what a student remembers
3 October 2026

An AI tutor should not rewrite what a student remembers

Dan Hart

Dan Hart

CEO, Co-Founder, CurricuLLM

A school AI tutor that writes the summary can also write the memory. If the summary is wrong in a small way, the student may later remember the summary rather than the lesson.

What we mean by an AI tutor is set out on the AI tutor for schools page.


The summary becomes the memory

I write meeting notes because the act of deciding what mattered is how I remember the meeting. A study on arXiv made me worry about handing that to a model. People watched a short video and, a day or two later, read an AI summary. When the summary included a misleading detail, 44.8 per cent correctly remembered the original. Among people who read an accurate summary, 83.6 per cent did.

One experiment is not a policy. It is enough to stop treating generated notes as a harmless convenience. We are outsourcing both the recording and the summarising, which removes the effort that formed the memory, and then we read the summary back and reinforce whatever version it produced. In a school, a tidy recap from a school AI tutor can do the same thing to a lesson. If the recap is slightly off, it may be the version that sticks.


A tutor configured once, not rediscovered by every user

At DevDay, OpenAI introduced specialist Dots: always-on agents set up for a role or a kind of work and shared across a team, rather than a general model that each person has to wire up alone. The context, the tools, the permissions and the expected way of working are encoded once.

The announcement is here. That is a different shape from "give everyone a chatbot". It starts to look like a capability the organisation owns, which raises ordinary questions. Who owns the agent, how is it measured, and how is it governed once it is inside real work? A school AI tutor should be built on that pattern. The teaching approach, the curriculum and the boundaries belong to the school, not to whichever student happens to be good at prompting.


Measuring the loop, not assuming it

A paper on an intelligence explosion asks what happens if AI does more of the work of building better AI. If better systems do more of the research, and that research produces better systems, the pace can pick up. There is already evidence of a smaller version of this inside frontier labs, where models do a growing share of the coding and the research, and the tasks they can finish are getting harder.

What I like about the paper is that it does not treat an explosion as inevitable. Compute, data, training time, diminishing returns, and the parts of research that are hard to automate are real constraints. Strong benchmark scores do not always show up as productivity. The useful demand is measurement. How much of AI research is actually automated? Is research productivity improving? Are cycles getting shorter? Where are the bottlenecks? How much human oversight is still required? A school adopting a school AI tutor deserves the same discipline. "The models are improving" is not a learning outcome.


Whose values, and whose protection

A New York Times profile of Anthropic co-founder Chris Olah describes an 84-page constitution for the kind of entity the company wants Claude to be. It talks about moral formation rather than only rules. It has explored ideas such as confession as a way of shaping a model's character, and it has discussed the model's "mental health" and whether models might experience something like suffering. Olah does not claim Claude is conscious. His position is that we do not know, and that being wrong could matter enough to take the possibility seriously.

Pope Leo XIV argues close to the reverse. The urgent moral question is not whether machines need protection from humans. It is whether humans need protection from increasingly powerful machines, and specifically from the people who decide which values are built into them.

I care about safety and alignment. I also think a school should be clear about which of these questions it is actually answering. A school AI tutor needs values a parent and a teacher can inspect. It does not need to be treated as a person.

Back to all posts
CurricuLLM Logo
CurricuLLM

AI for schools

Product

FeaturesPricingUse CasesSchoolsAI tutorDevelopersFAQ

Resources

Training hubSupportBlogResearchEvidenceEventsPress

Company

About usTrust & SafetyPrivacy policyTerms of useStatusContact