The evidence behind CurricuLLM

Every CurricuLLM answer is anchored to a specific outcome in the National Curriculum (England), GCSE subject content, and Oak National Academy lesson sequences. In independent benchmarking across approximately 13,500 question-response pairs, CurricuLLM achieved 89 percent accuracy while no mainstream frontier model exceeded 41 percent. You can read more about how we evaluate this on the Research page.

This is the pedagogical hygiene layer that makes every feature above usable in a real classroom. Alignment is not a learning intervention on its own — it does not carry an effect size — but without it none of the effect sizes above transfer into a teacher's real context.

Every CurricuLLM conversation loads the student's position against the curriculum before they type. The tutor pitches at their level and adapts turn by turn. Scaffolded, contingent support is the single most robust finding in K–12 tutoring research, going back to Vygotsky, Bruner, and Wood's original framing of scaffolding.

Ma, Adesope, Nesbit and Liu's meta-analysis of 107 intelligent tutoring system comparisons across 14,321 students found an advantage of g = 0.42 over teacher-led large-group instruction. VanLehn's earlier review found step-based tutoring systems produced d ≈ 0.76, essentially matching human tutors at d ≈ 0.79. The EEF's Metacognition and Self-Regulation strand reports +7 months' additional progress from scaffolded strategy instruction. You can see how this shows up in product on the features overview.

Studio Mode produces flashcards, differentiated quizzes, and exit tickets as low-stakes retrieval opportunities rather than summative tests. The testing effect is one of the best-evidenced findings in cognitive science. Roediger and Karpicke's canonical Test-Enhanced Learning paper showed repeated testing outperforms repeated study on delayed retention. Adesope, Trevisan and Sundararajan's meta-analysis of 272 practice-testing effects reported g ≈ 0.51 over restudy, rising to g ≈ 0.67 in classroom settings, with multiple choice producing g ≈ 0.70. Rowland's meta-analytic review put overall testing at g ≈ 0.50, rising to g ≈ 0.73 when feedback followed retrieval.

For K–12 classrooms specifically, McDermott, Agarwal, D'Antonio, Roediger and McDaniel's Grade 7 science and high school history study found both multiple choice and short-answer quizzes reliably raised unit and semester exam performance over no-quiz controls.

Progression-aware practice also spaces and interleaves. Rohrer, Dedrick and Burgess's Year 7 maths interleaving study reported 72 percent versus 38 percent correct on delayed tests, an effect size of roughly d = 1.05. Dunlosky, Rawson, Marsh, Nathan and Willingham's Psychological Science in the Public Interest review ranked practice testing and distributed practice as the two highest-utility study techniques available.

CurricuLLM's student-facing tutor hints, probes, and asks for self-explanation. It does not hand out answers. This matters because of a specific piece of 2024 evidence teachers should know. Bastani and colleagues at Penn found, in a Turkish high-school maths study with around 1,000 students, that access to raw GPT-4 improved in-task performance by 48 percent but reduced unaided exam performance by 17 percent. A GPT tutor explicitly designed to hint and withhold answers neutralised the harm entirely. The conclusion is direct. AI that answers questions can hurt learning. AI that teaches does not. Our guardrails and safety posture are documented on the Trust & Safety page.

The pedagogical basis is Chi and Wylie's ICAP framework: cognitive engagement goes Passive < Active < Constructive < Interactive. Prompted self-explanation produces effects around g = 0.55 across 64 studies (Bisra, Liu, Nesbit, Salimi and Winne, 2018). Alexander's dialogic teaching framework, tested in the EEF's cluster-randomised trial across 78 primary schools, delivered +2 months' progress in English and science with larger effects for pupils on free school meals.

Recent evidence on well-designed AI tutors is building fast. The World Bank's Nigeria RCT with around 800 senior secondary students found +0.31 SD overall learning gains over six weeks of teacher-facilitated AI tutoring, equivalent to roughly 1.5 to 2 years of schooling, outperforming about 80 percent of rigorously evaluated developing-country education interventions. Henkel and colleagues reported +0.30 SD for the Rori math tutor in Ghana, with effects concentrated among girls. At undergraduate level, Kestin and colleagues found a Socratically-prompted GPT tutor roughly doubled learning gains relative to active-learning classrooms at Harvard.

Every teacher output in Studio Mode, from exit tickets to marking guides, feeds teaching decisions rather than a mark book. Live Mode surfaces who understands, who is stuck, and who is disengaged during a lesson. Black and Wiliam's original Inside the Black Box review defined this field. The EEF's Feedback strand reports +6 months' progress, very low cost, moderate-to-high evidence security.

Kingston and Nash's K–12-specific meta-analysis put formative assessment effects at around d = 0.20, larger in English (0.32) and smaller in maths (0.17). Fuchs and Fuchs's earlier work on systematic formative evaluation reported d = 0.70, rising to 0.90 when graphing and decision rules were added. On feedback itself, Hattie and Timperley's synthesis identified task, process, and self-regulation feedback as effective, with self-level feedback ineffective or negative. The warning from Kluger and DeNisi's 131-study meta-analysis is worth stating: around a third of feedback effects were negative, almost always when feedback targeted the person rather than the task. CurricuLLM feedback is always task- and process-oriented by design.

The EEF's Feedback strand and Dylan Wiliam's ongoing UK Assessment for Learning programme both position formative assessment as non-negotiable. Rosenshine's Principles of Instruction puts checking for understanding at the centre of effective teaching. Less effective teachers ask “any questions?” More effective teachers sample every student. Live Mode operationalises exactly that.

CurricuLLM uses localised UK English accents, generates diagrams, and animates complex concepts. Mayer's multimedia principles are the evidence base here. Mayer and Moreno's Educational Psychologist paper reports median effects of d ≈ 1.39 for the multimedia principle (words plus pictures beats words alone), d ≈ 0.97 for the modality principle, and d ≈ 0.97 for the coherence principle.

Localised accents are not a nicety. Floccia and colleagues' accent perception research found unfamiliar native accents are processed more slowly and less accurately than familiar ones, with the cost larger for younger listeners. UK English voices lower cognitive load for the students listening.

On UDL more broadly, King-Sears and colleagues' meta-analysis of 20 studies, 80 percent K–12, reported g = 0.43 for UDL-based treatments on achievement. This is the first quantitative synthesis of UDL's achievement impact.

When a student is new to a topic and CurricuLLM has no prior progression data to personalise with, the tutor opens with a short quiz. Multiple choice, flashcards, or matching, calibrated to what the student has just asked about. The result sets the difficulty for the rest of the session.

This is doing four evidence-backed things at once.

First, pre-assessment. Ausubel's dictum remains the clearest statement: the most important single factor influencing learning is what the learner already knows.

Second, the pretesting effect. Opening with a quiz is not just a diagnostic, it is a learning event in its own right, even when students get items wrong, provided feedback follows. Pan and Sana's review across five experiments and 1,573 participants reports a d ≈ 0.30 advantage of pretesting over retrieval practice. Recent meta-analytic estimates put the pretesting effect at g = 0.34 to 0.54.

Third, dynamic assessment. The tutor prompts, hints, and mediates during the opening quiz, measuring what the student can do with support rather than only what they can do unaided. This is particularly valuable for students with additional needs, EAL learners, and students from backgrounds that static tests systematically underestimate.

Fourth, adaptive testing. Weiss and Kingsbury showed computer-adaptive tests achieve equivalent measurement precision with around half the items of a fixed test. Roschelle, Feng, Murphy and Mason's cluster RCT of ASSISTments across 43 schools and 2,850 Grade 7 students reported 0.18–0.22 SD gains, meeting What Works Clearinghouse standards without reservations, with larger benefits for low prior achievers.

Using multiple item formats (MCQ, flashcard, matching) is what UDL Guideline 5 prescribes for multiple means of action and expression. It also reflects the retrieval-practice evidence: McDermott and colleagues' K–12 classroom study found both MCQ and short-answer quizzes work, with feedback. Flashcards combine retrieval and spacing, one of the two highest-utility techniques in Dunlosky et al.'s ranking.

For students who want help beyond the classroom, CurricuLLM acts as an always-available, curriculum-aligned tutor — the kind of personal study companion we describe on the Schools page. The evidence base for tutoring is one of the most robust in education. Nickow, Oreopoulos and Quan's systematic review and meta-analysis of 96 PreK–12 RCTs reported a pooled effect of 0.29 SD, with larger effects for teacher or paraprofessional tutors, in-school delivery, and high-frequency sessions. The EEF Toolkit reports +5 months' progress for one-to-one tuition and +4 months' for small-group.

AI tutoring does not replace human tutoring. The 2014 Ma et al. meta-analysis found no reliable difference between intelligent tutoring systems and human one-to-one tutoring. What it does is extend tutoring's reach to every student with a device and a question, at the moment the question arrives.

Studio Mode detects where a class is stuck, generates reteach resources and progression exercises, and holds students at a concept until competence appears. Mastery learning has one of the longest evidence trails in education, starting with Bloom's 2 Sigma paper in 1984. The EEF's Mastery Learning strand reports +5 months' progress on moderate-to-strong evidence. Kulik, Kulik and Bangert-Drowns' meta-analysis of 108 studies reported a mean examination effect of d = 0.52, with larger gains for lower-attaining students.

Studio Mode also produces differentiated versions of a task against a single learning intention, levelled reading that preserves core vocabulary, decodable texts, and EAL vocabulary support. Puzio, Colby and Algeo-Nichols' meta-analysis of differentiated literacy reported g ≈ 0.41. The EEF's guidance for English-as-an-additional-language learners mirrors the evidence base in English classrooms. Worth flagging honestly: differentiation has weaker causal evidence than its prominence in classrooms suggests, and most primary studies are quasi-experimental.

Studio Mode generates fully worked examples, faded worked examples, and practice sets that rise in difficulty. This is the worked-example effect from Cognitive Load Theory (Sweller, 1988; Sweller, van Merriënboer and Paas, 2019). For novices, studying worked solutions frees working memory to build schemas, while unguided problem solving consumes that same capacity on search. Atkinson, Derry, Renkl and Wortham's Review of Educational Research synthesis identified example variability, self-explanation, and fading as the active ingredients.

In Hattie's synthesis, worked examples sit at d ≈ 0.57 and direct instruction at d ≈ 0.59. The EEF's Improving Mathematics in Key Stages 2 and 3 and Tom Sherrington's Rosenshine Masterclass materials are the clearest local translations for English and Welsh teachers, both grounded in Cognitive Load Theory.

Studio Mode generates “what a C looks like” and “what an A looks like” exemplars alongside marking guides, rendered into student-facing language where needed. Sadler's original formulation argued that students improve only when they hold a concept of quality similar to the teacher's, can compare their work against it, and can act to close the gap. Exemplars carry what he called guild knowledge that criterion statements alone cannot.

Panadero, Jonsson and Botella's meta-analyses of self-assessment with rubrics reported gains of g = 0.43–0.65 on self-regulated learning and g = 0.73 on self-efficacy.

Galaxy is deliberately restrained. It rewards progress with a growing constellation, not with points to be farmed or leaderboards to be topped. That restraint is the evidence-based position, not a limitation. The strongest and most consistent finding in the reward literature is that heavy extrinsic reward systems tend to erode the intrinsic motivation they are meant to build. Light, informational, relationship-anchored recognition does the opposite. Galaxy is built on that distinction, and you can see how it works in product in the Galaxy training.

The sections below map each Galaxy design decision to the research that justifies it. Where the evidence is mixed or contested we say so. The through-line is simple. On rewards for children, more is not better. Better is better.

Across the best available meta-analyses, gamification produces a small to moderate positive effect on learning and a moderate effect on motivation and engagement. It is not a magic lever, and poorly designed gamification produces nothing or backfires. Sailer and Homner's 2020 meta-analysis found significant positive effects on cognitive (g ≈ 0.49), motivational (g ≈ 0.36) and behavioural (g ≈ 0.25) outcomes, all in the small-to-moderate range, with the cognitive effect the most stable once only methodologically rigorous studies were counted. Bai, Hew and Huang (2020) report a broadly similar picture and document a large amount of variation between studies, which means the design of the gamification matters more than the fact of it. Huang and colleagues (2020) and Koivisto and Hamari (2019) reach compatible conclusions. Novelty effects are real, so short studies tend to overstate what persists.

Galaxy's low-intensity approach is consistent with a body of evidence showing real but modest gains, where the size of the benefit depends heavily on how the mechanic connects to the actual learning rather than on the presence of game elements for their own sake. The evidence supports gamification that is thoughtfully designed. It warns against treating points and badges as automatically beneficial.

Self-Determination Theory holds that motivation is strongest and most durable when three needs are met. Autonomy is the sense of acting by one's own volition. Competence is the sense of getting better. Relatedness is the sense of being connected to others who care. Ryan and Deci's foundational 2000 paper sets this out, and Cognitive Evaluation Theory (Deci and Ryan, 1985) adds the key nuance for reward design. Rewards experienced as informational, meaning they tell you that you are doing well, support competence and can strengthen motivation. Rewards experienced as controlling, meaning you do this to get that, shift the learner's sense of why they are working from inside to outside, and motivation drops.

Galaxy is designed as informational feedback about competence, not as a currency. A constellation that grows as a learner masters material tells them they are getting better. It does not set up a “study to earn points” transaction. The relatedness need is met by tying recognition back to teachers and the classroom rather than leaving it as an isolated digital score. This is the mechanism that separates good gamification from bad, which is why it carries the highest weight on this page.

The 1999 meta-analysis by Deci, Koestner and Ryan synthesised decades of experiments and found that expected, tangible rewards significantly undermined intrinsic motivation, with the effect strongest for engaging tasks. The effect is muted or absent when rewards are unexpected, or when they are verbal and informational rather than tangible and contingent on task completion. Lepper, Greene and Nisbett's classic 1973 study showed the same effect in children given a reward for an activity they already enjoyed. This body of work is contested at the margins, and some researchers argue the undermining effect is smaller or more conditional than the strong version claims, but the practical lesson is stable. If you make an interesting activity feel like a job you do for points, you tend to make it less interesting.

This is why Galaxy does not hand out escalating points for time spent, does not gate content behind a currency, and does not build a system a learner would optimise for its own sake. The overjustification literature is a direct warning against the heavy gamification model. Galaxy's restraint is not a missing feature. It is the intervention.

Goal-setting theory finds that specific, appropriately challenging goals with feedback on progress produce better performance than vague “do your best” goals (Locke and Latham, 2002). In the schools evidence, the EEF rates the Feedback strand at +6 months' progress and Metacognition and Self-Regulation at +7 months on Evidence for Learning's Australian localisation of the Toolkit, both among the highest-evidence, lowest-cost approaches available.

A constellation is a progress visualisation. It gives a learner a specific, concrete picture of how far they have come and what is next, which is exactly the sub-goal and progress-feedback structure the goal-setting and feedback evidence supports.

Mueller and Dweck (1998) showed that praising children for intelligence, telling them “you are so smart,” led them to avoid challenge and crumble after setbacks, while praising effort and strategy, “you worked hard on that,” built resilience. The lesson is that recognition should be about controllable process, not fixed traits. Separately, the relatedness component of Self-Determination Theory predicts that recognition carried by a real person the learner is connected to is more motivating than an anonymous number on a screen.

This is the research basis for linking edtech progress to real-world acknowledgement. A constellation that a teacher can see and respond to, that surfaces in the classroom rather than staying trapped in the app, converts a digital signal into genuine relatedness. Recognition framed around effort and progress rather than being “smart” is consistent with the process-praise evidence. This is a differentiator worth stating plainly. Points in isolation do not build relationships. Recognition that reaches the people who matter to a child does.

Nicholson (2015) argues that gamification built only on points, badges and leaderboards is shallow and fades once novelty wears off, and that durable gamification connects the mechanics to meaning, mastery and the learner's own goals. The meta-analytic finding that effects are highly design-dependent and vulnerable to novelty decay (Sailer and Homner, 2020; Bai et al., 2020) supports this. It is worth noting the framework has an Australian anchor, since Nicholson is now at Curtin University in Western Australia.

Galaxy is a deliberate rejection of points-maximising design. Every mechanic is meant to serve mastery and recognition, and anything that would push a learner into optimising the system rather than learning is left out on purpose. The restraint is the strategy.

Hattie's Visible Learning effect sizes are cited above for orientation alongside primary meta-analyses. The honest position for teachers to know is that Hattie's synthesis has been methodologically criticised for correlation-to-d conversions and for double-counting meta-analyses. We treat Hattie's numbers as directional benchmarks and privilege specific meta-analyses where they exist.