The evidence behind CurricuLLM

Every CurricuLLM answer is anchored to a specific outcome in the Australian Curriculum v9, Victorian, Western Australian, or New Zealand Curriculum. In independent benchmarking across approximately 13,500 question-response pairs, CurricuLLM achieved 89 percent accuracy while no mainstream frontier model exceeded 41 percent. You can read more about how we evaluate this on the Research page.

This is the pedagogical hygiene layer that makes every feature above usable in a real classroom. Alignment is not a learning intervention on its own — it does not carry an effect size — but without it none of the effect sizes above transfer into a teacher's real context.

Every CurricuLLM conversation loads the student's position against the curriculum before they type. The tutor pitches at their level and adapts turn by turn. Scaffolded, contingent support is the single most robust finding in K–12 tutoring research, going back to Vygotsky, Bruner, and Wood's original framing of scaffolding.

Ma, Adesope, Nesbit and Liu's meta-analysis of 107 intelligent tutoring system comparisons across 14,321 students found an advantage of g = 0.42 over teacher-led large-group instruction. VanLehn's earlier review found step-based tutoring systems produced d ≈ 0.76, essentially matching human tutors at d ≈ 0.79. The EEF's Metacognition and Self-Regulation strand reports +7 months' additional progress from scaffolded strategy instruction. You can see how this shows up in product on the features overview.

Studio Mode produces flashcards, differentiated quizzes, and exit tickets as low-stakes retrieval opportunities rather than summative tests. The testing effect is one of the best-evidenced findings in cognitive science. Roediger and Karpicke's canonical Test-Enhanced Learning paper showed repeated testing outperforms repeated study on delayed retention. Adesope, Trevisan and Sundararajan's meta-analysis of 272 practice-testing effects reported g ≈ 0.51 over restudy, rising to g ≈ 0.67 in classroom settings, with multiple choice producing g ≈ 0.70. Rowland's meta-analytic review put overall testing at g ≈ 0.50, rising to g ≈ 0.73 when feedback followed retrieval.

For K–12 classrooms specifically, McDermott, Agarwal, D'Antonio, Roediger and McDaniel's Grade 7 science and high school history study found both multiple choice and short-answer quizzes reliably raised unit and semester exam performance over no-quiz controls.

Progression-aware practice also spaces and interleaves. Rohrer, Dedrick and Burgess's Year 7 maths interleaving study reported 72 percent versus 38 percent correct on delayed tests, an effect size of roughly d = 1.05. Dunlosky, Rawson, Marsh, Nathan and Willingham's Psychological Science in the Public Interest review ranked practice testing and distributed practice as the two highest-utility study techniques available.

CurricuLLM's student-facing tutor hints, probes, and asks for self-explanation. It does not hand out answers. This matters because of a specific piece of 2024 evidence teachers should know. Bastani and colleagues at Penn found, in a Turkish high-school maths study with around 1,000 students, that access to raw GPT-4 improved in-task performance by 48 percent but reduced unaided exam performance by 17 percent. A GPT tutor explicitly designed to hint and withhold answers neutralised the harm entirely. The conclusion is direct. AI that answers questions can hurt learning. AI that teaches does not. Our guardrails and safety posture are documented on the Trust & Safety page.

The pedagogical basis is Chi and Wylie's ICAP framework: cognitive engagement goes Passive < Active < Constructive < Interactive. Prompted self-explanation produces effects around g = 0.55 across 64 studies (Bisra, Liu, Nesbit, Salimi and Winne, 2018). Alexander's dialogic teaching framework, tested in the EEF's cluster-randomised trial across 78 primary schools, delivered +2 months' progress in English and science with larger effects for pupils on free school meals.

Recent evidence on well-designed AI tutors is building fast. The World Bank's Nigeria RCT with around 800 senior secondary students found +0.31 SD overall learning gains over six weeks of teacher-facilitated AI tutoring, equivalent to roughly 1.5 to 2 years of schooling, outperforming about 80 percent of rigorously evaluated developing-country education interventions. Henkel and colleagues reported +0.30 SD for the Rori math tutor in Ghana, with effects concentrated among girls. At undergraduate level, Kestin and colleagues found a Socratically-prompted GPT tutor roughly doubled learning gains relative to active-learning classrooms at Harvard.

At intervals during a session the tutor breaks out of Socratic questioning and hands the questioning role back to the student, prompting them to plan, monitor, or evaluate their own thinking. The point is to stop students from settling into the respondent seat and give them reps at self-regulation while the session is still live. The EEF Teaching and Learning Toolkit puts metacognition and self-regulation at +7 months' progress across 355 studies, very low cost, high evidence security. The EEF Guidance Report (Quigley, Muijs and Stringer, 2018) recommends explicit strategy instruction, modelling of thinking, and deliberate metacognitive talk, all of which the Mode-Flip operationalises turn by turn.

The meta-analytic evidence is strong. Dignath and Büttner's meta-analysis of 84 K–12 intervention studies reported an overall d = 0.69 across 357 effect sizes, with the largest gains when interventions combined cognitive, metacognitive and motivational strategies. Donker and colleagues' learning-strategy meta-analysis reported g = 0.66 across 58 K–12 studies. Hattie's Visible Learning ranking places metacognitive strategies at d = 0.60. AERO's Supporting Self-Regulated Learning practice guide endorses the same approach for Australian classrooms. The Mode-Flip is built on Zimmerman's three-phase cycle of forethought, performance and self-reflection. Each student-led turn is a rehearsal of the regulatory moves the literature identifies as the active ingredient.

The Wrap button closes a session with a short structured reflection sequence that asks the student what was learned, how the learning happened, and what questions they would ask next. Teaching students to generate their own questions produces reliable comprehension gains. Rosenshine, Meister and Chapman's review of 26 intervention studies reported median effects of d = 0.36 on standardised comprehension tests, rising to d = 0.86 on experimenter-developed tests, when learners were trained on generic question stems and signal-word prompts. King's reciprocal-questioning research showed that generic thinking stems such as “how does X relate to Y?” push learners past recall into integrative and metacognitive processing. Chin and Osborne document the same pattern in science: student-generated questions drive conceptual change through cognitive disequilibrium and elaborated explanation.

Two further evidence strands support the Wrap's design. Bisra, Liu, Nesbit, Salimi and Winne's meta-analysis of self-explanation prompts across 64 studies reports g = 0.55, which is the effect the Wrap's “explain this concept back to me” step draws on. Wagner and colleagues' meta-analysis of prompted monitoring tools across 32 studies reports d = 0.42 on achievement when learners use structured journals, portfolios and reflection prompts at the end of learning episodes. Graesser and Person's classic tutoring study showed it is the quality of student questions, not the frequency, that predicts gains. The Wrap scaffolds question form rather than asking “any questions?” and does it while the conversation is still in working memory.

Every teacher output in Studio Mode, from exit tickets to marking guides, feeds teaching decisions rather than a mark book. Live Mode surfaces who understands, who is stuck, and who is disengaged during a lesson. Black and Wiliam's original Inside the Black Box review defined this field. The EEF's Feedback strand reports +6 months' progress, very low cost, moderate-to-high evidence security.

Kingston and Nash's K–12-specific meta-analysis put formative assessment effects at around d = 0.20, larger in English (0.32) and smaller in maths (0.17). Fuchs and Fuchs's earlier work on systematic formative evaluation reported d = 0.70, rising to 0.90 when graphing and decision rules were added. On feedback itself, Hattie and Timperley's synthesis identified task, process, and self-regulation feedback as effective, with self-level feedback ineffective or negative. The warning from Kluger and DeNisi's 131-study meta-analysis is worth stating: around a third of feedback effects were negative, almost always when feedback targeted the person rather than the task. CurricuLLM feedback is always task- and process-oriented by design.

AERO's Formative Assessment practice guide and the NZ Ministry of Education's Assessment Online both position Assessment for Learning as non-negotiable. Rosenshine's Principles of Instruction puts checking for understanding at the centre of effective teaching. Less effective teachers ask “any questions?” More effective teachers sample every student. Live Mode operationalises exactly that.

CurricuLLM uses localised Australian and New Zealand accents, generates diagrams, and animates complex concepts. Mayer's multimedia principles are the evidence base here. Mayer and Moreno's Educational Psychologist paper reports median effects of d ≈ 1.39 for the multimedia principle (words plus pictures beats words alone), d ≈ 0.97 for the modality principle, and d ≈ 0.97 for the coherence principle.

Localised accents are not a nicety. Floccia and colleagues' accent perception research found unfamiliar native accents are processed more slowly and less accurately than familiar ones, with the cost larger for younger listeners. Australian and New Zealand voices lower cognitive load for the students listening.

On UDL more broadly, King-Sears and colleagues' meta-analysis of 20 studies, 80 percent K–12, reported g = 0.43 for UDL-based treatments on achievement. This is the first quantitative synthesis of UDL's achievement impact.

When a student is new to a topic and CurricuLLM has no prior progression data to personalise with, the tutor opens with a short quiz. Multiple choice, flashcards, or matching, calibrated to what the student has just asked about. The result sets the difficulty for the rest of the session.

This is doing four evidence-backed things at once.

First, pre-assessment. Ausubel's dictum remains the clearest statement: the most important single factor influencing learning is what the learner already knows.

Second, the pretesting effect. Opening with a quiz is not just a diagnostic, it is a learning event in its own right, even when students get items wrong, provided feedback follows. Pan and Sana's review across five experiments and 1,573 participants reports a d ≈ 0.30 advantage of pretesting over retrieval practice. Recent meta-analytic estimates put the pretesting effect at g = 0.34 to 0.54.

Third, dynamic assessment. The tutor prompts, hints, and mediates during the opening quiz, measuring what the student can do with support rather than only what they can do unaided. This is particularly valuable for students with additional needs, EAL learners, and students from backgrounds that static tests systematically underestimate.

Fourth, adaptive testing. Weiss and Kingsbury showed computer-adaptive tests achieve equivalent measurement precision with around half the items of a fixed test. Roschelle, Feng, Murphy and Mason's cluster RCT of ASSISTments across 43 schools and 2,850 Grade 7 students reported 0.18–0.22 SD gains, meeting What Works Clearinghouse standards without reservations, with larger benefits for low prior achievers.

Using multiple item formats (MCQ, flashcard, matching) is what UDL Guideline 5 prescribes for multiple means of action and expression. It also reflects the retrieval-practice evidence: McDermott and colleagues' K–12 classroom study found both MCQ and short-answer quizzes work, with feedback. Flashcards combine retrieval and spacing, one of the two highest-utility techniques in Dunlosky et al.'s ranking.

For students who want help beyond the classroom, CurricuLLM acts as an always-available, curriculum-aligned tutor — the kind of personal study companion we describe on the Schools page. The evidence base for tutoring is one of the most robust in education. Nickow, Oreopoulos and Quan's systematic review and meta-analysis of 96 PreK–12 RCTs reported a pooled effect of 0.29 SD, with larger effects for teacher or paraprofessional tutors, in-school delivery, and high-frequency sessions. The EEF Toolkit reports +5 months' progress for one-to-one tuition and +4 months' for small-group.

AI tutoring does not replace human tutoring. The 2014 Ma et al. meta-analysis found no reliable difference between intelligent tutoring systems and human one-to-one tutoring. What it does is extend tutoring's reach to every student with a device and a question, at the moment the question arrives.

Studio Mode detects where a class is stuck, generates reteach resources and progression exercises, and holds students at a concept until competence appears. Mastery learning has one of the longest evidence trails in education, starting with Bloom's 2 Sigma paper in 1984. The EEF's Mastery Learning strand reports +5 months' progress on moderate-to-strong evidence. Kulik, Kulik and Bangert-Drowns' meta-analysis of 108 studies reported a mean examination effect of d = 0.52, with larger gains for lower-attaining students.

Studio Mode also produces differentiated versions of a task against a single learning intention, levelled reading that preserves core vocabulary, decodable texts, and EAL/D vocabulary support. Puzio, Colby and Algeo-Nichols' meta-analysis of differentiated literacy reported g ≈ 0.41. AERO's Teaching multilingual learners practice guide mirrors the evidence base in Australian classrooms. Worth flagging honestly: differentiation has weaker causal evidence than its prominence in classrooms suggests, and most primary studies are quasi-experimental.

Studio Mode generates fully worked examples, faded worked examples, and practice sets that rise in difficulty. This is the worked-example effect from Cognitive Load Theory (Sweller, 1988; Sweller, van Merriënboer and Paas, 2019). For novices, studying worked solutions frees working memory to build schemas, while unguided problem solving consumes that same capacity on search. Atkinson, Derry, Renkl and Wortham's Review of Educational Research synthesis identified example variability, self-explanation, and fading as the active ingredients.

In Hattie's synthesis, worked examples sit at d ≈ 0.57 and direct instruction at d ≈ 0.59. The NSW Department of Education's Cognitive Load Theory in Practice is the clearest local translation for teachers, reviewed by John Sweller himself. AERO's Managing cognitive load practice guide applies the same evidence to Australian K–12.

Studio Mode generates “what a C looks like” and “what an A looks like” exemplars alongside marking guides, rendered into student-facing language where needed. Sadler's original formulation argued that students improve only when they hold a concept of quality similar to the teacher's, can compare their work against it, and can act to close the gap. Exemplars carry what he called guild knowledge that criterion statements alone cannot.

Panadero, Jonsson and Botella's meta-analyses of self-assessment with rubrics reported gains of g = 0.43–0.65 on self-regulated learning and g = 0.73 on self-efficacy.

Hattie's Visible Learning effect sizes are cited above for orientation alongside primary meta-analyses. The honest position for teachers to know is that Hattie's synthesis has been methodologically criticised for correlation-to-d conversions and for double-counting meta-analyses. We treat Hattie's numbers as directional benchmarks and privilege specific meta-analyses where they exist.