Ninety-six percent of the students tried it. The median student messaged it on about a third of the days they practiced, and in 17 percent of the sessions where they got something wrong.
Those two sentences come from the same abstract, in what its authors call “the largest and longest experimental evaluation of an AI tutoring program in public schools to date” (One Click Away: AI Tutoring with Khanmigo in a Two-Year School Experiment, EdWorkingPaper 26-1551, Philip Oreopoulos and Nina Low, August 2026). Two school years, 18 middle schools in Hamilton County, Tennessee, one or two grades randomly assigned per school to use Khan Academy with its AI tutor Khanmigo inside the district’s existing daily remedial maths block. 6,902 student-terms in the main estimate.
The headline is positive and it holds: assignment raised end-of-term maths scores by 1.3 national percentile ranks per term. The question worth asking of a positive result is what the effect is attributable to, and this paper answers that itself, in a way that announcements of AI tutoring generally do not.
Finding one: the practice is real
The programme moved a quantity that is hard to move. Treated students averaged “about 30 additional minutes of weekly practice on Khan Academy”, rising to “over 45 minutes per week in the second year, as implementation matured”. Appendix Table A2 puts the per-term intent-to-treat effect on learning time between 22.3 and 33.4 minutes a week, every term significant at one percent.
Achievement tracked it. The pooled effect is 1.264 percentile ranks with a baseline control and 1.216 with full controls, both significant at five percent (Table 5). In standard-deviation terms the annual figure is 0.020 in Year 1, statistically indistinguishable from zero, and 0.084 in Year 2 (SE 0.041). The authors note the two years “are not statistically distinguishable from each other (p = 0.61)”, so the Year-2 concentration is a pattern in the point estimates rather than a tested contrast. Worth naming honestly: the no-controls column of Table 5 is -0.244 and insignificant. The result needs the prior-term baseline score to appear.
Finding two: the AI is not what explains it
This is the sentence a vendor would not write. Quasi-experimental estimates of Khan Academy practice without Khanmigo imply returns per hour close to what this trial measured, and the effects sit inside the range computer-assisted learning trials have produced for twenty years. Then: “Nothing in the achievement data requires the AI tutor to explain it.”
The counterfactual cuts in the same direction and the authors say so. Control classes were “not technology-free”: teachers used Waggle, IXL, Zearn and DeltaMath, with four of eighteen schools using no such software, four using it exclusively and ten mixing. So “the treatment-control contrast should not be read as isolating the AI tutor”, because the programme also changed the structure of practice itself, with mastery-based learning paths placed directly from each student’s screener scores.
That concession is a reason to be careful in both directions. It means the measured gain cannot be credited to the conversational tutor. It also means the trial is not evidence that AI tutoring does nothing, because what the trial compared was one whole package against a varied mix of other software.
Finding three: the dosage
Here is why finding two is unsurprising once you read the engagement data. Among 563 activated Year-2 students matched to exercise records, four percent never sent a message at all. Appendix Table A2 puts the treatment effect on Khanmigo chats at 1.2 to 2.3 per week.
The paper’s Table 4 categorises what students did send, and the categories are the finding. Two are substantive: reasoning and questions, and answering the tutor with maths content. Three are not. “Button (help-seeking click)” is a student tapping a canned prompt. “Low-effort / minimal” is illustrated by an exchange the paper gives as K: What do you notice about the rectangle’s rows and columns? S: just give me it I already got it wrong so theres no point doing it again. The paper states these example episodes are “slightly altered to maintain anonymity”, so they are representative rather than raw log lines. The last category is off-task, including a request for an essay on the history of pizza.
The authors’ reading is institutional rather than disparaging: “interventions that rely on student initiative reach fewest of the students who would benefit most”. A tutor that waits to be asked is a tutor that the students least likely to ask will not use.
What the paper refuses to claim
Two restraints are worth crediting, because they are where this kind of paper usually overreaches.
On cost, the marginal licence is “about $15 per student per year”, against several thousand for high-dosage human tutoring at 0.2 to 0.4 SD a year. The obvious move is to declare a win on cost-effectiveness. Instead: “The comparison should not be read as a substitution argument for tutors (or teachers): human tutoring delivers more learning per student per year, and nothing in our results suggests that software closes that gap.”
On the second outcome, Tennessee’s state TCAP assessment was pre-registered as a co-primary outcome and is not reported here; results “will be incorporated as they are finalized”. That matters more than it sounds, because unlike the MAP benchmark, TCAP is not linked to the platform’s own placement system, so it is the check on whether the measured gains are narrowly assessment-aligned practice. The headline currently rests on the assessment with the tighter relationship to the product.
A second trial, pointing the other way
An unrelated randomised trial posted on the same site in October 2026 gave 2,379 undergraduates at a large US public university access to a course-integrated AI tutor. Among sections of the same course, access “reduced final grades by 0.37 standard deviations and learning management system participation by 0.90 standard deviations”, with larger estimated losses among first-generation students (EdWorkingPaper 26-1598). Different population, different age, different design, so this is not a replication and does not rebut the Tennessee result. The reason to read them together is that the mechanism is the same axis: an AI tutor competes for the attention students were already giving something else. In Tennessee, inside a supervised mandatory block, the practice was additive. In the university sections, participation fell.
I read only that paper’s abstract page; the full PDF was not opened for this piece, so treat the two figures as its own summary rather than as numbers I checked in the tables.
The redesign already happened
Buried in a footnote to the setup section: Khanmigo appeared collapsed by default shortly after page load, and “a broader redesign that activates Khanmigo throughout practice, adopted in response to low student-initiated use, was rolled out to district partners after the close of our delivery window”.
So the product this trial measured no longer exists in the form it was measured. Whether the opt-out version produces more conversation, and whether more conversation produces more learning, are two separate open questions, and this trial answers neither. What it establishes is narrower and more useful than the headline: put the practice inside a block students already have to attend and the minutes arrive; leave the conversation to the student’s initiative and it does not.