Does Medly actually raise grades?
An independent randomised controlled trial across 644 GCSE science students. What it found, and what we're testing next.
Every revision app says it works. Very few let someone else check.
The usual evidence can't separate the tool from the student. If someone revises four nights a week and their grades go up, that says as much about them as about us. The only way to answer it properly is to take a group of students, randomly decide who gets the tool, and compare.
So we had a randomised controlled trial run, and we didn't run it. It was designed, randomised and analysed by WhatWorked Education, a research organisation that conducts randomised trials in English schools alongside teachers, and that has worked with the Education Endowment Foundation on RCT projects before, with Professor Steve Higgins of Durham University. They designed the trial, randomised the pupils and ran the analysis. The published research can be found here.
How it worked
Thirty-nine teachers each ran a trial with their own class in Biology, Chemistry or Physics. Within every class, students were randomly assigned to use Medly for a set topic, thirty minutes a week for four weeks, or to revise the same topic for the same time using whatever they normally would.
That second group matters. This wasn't Medly against nothing. Control students used revision guides, notes, videos, and in most classes other online revision platforms. Same time, same content. Only the tool differed.
Everyone sat a test before and a different test of matched difficulty after, five GCSE-style questions marked out of 35. None of them were available on Medly during the trial.
What they found
929 students sat the first test and 644 completed both. Across those 644, students using Medly scored 22% higher on the post-test than comparable students revising normally, adjusted for where they started: 2.56 more marks out of 35. This result was statistically significant.
The effect size was 0.33. On the scale the Education Endowment Foundation uses to translate effect sizes into months of progress, that's roughly four months. It's a rule of thumb rather than something the trial measured, but it's the clearest way to say what 0.33 means.
The gain held in all three sciences: Biology 0.52, Chemistry 0.32, Physics 0.31. Biology came out strongest.
The bit we care about most
Attainment rose consistently with use. The more students engaged with Medly, the greater the effect: every extra question a student answered was worth about 0.18 marks on the post-test, and the most engaged students showed the largest estimated gains.
That pattern is what makes us think this is teaching rather than novelty. Novelty doesn't compound with use. Practice does.
This is an exploratory part of the analysis rather than a randomised comparison, so we'll keep building the evidence base on it. As a pattern, it's the one we most wanted to see.
Did it work for everyone?
The effect size did not differ by disadvantage status. Pupil Premium students gained in line with their peers, which is encouraging on equity and the part of the result we'll be watching most closely as the work continues.
This is the start
We think this is the first randomised controlled trial measuring attainment for an AI revision platform in GCSE science specifically. There's decent evidence for AI tutoring in maths at secondary level, and a strong recent study in university physics, but GCSE science has been a gap.
The reason for doing it this way is simple. Plenty of students have always had someone who marks their work properly and explains what went wrong. That person usually costs £30-40 an hour. If a platform can do part of that job, it should be possible to show it under conditions where it could have failed.
That makes this a positive first data point on AI's potential to widen access to personalised education. We're carrying it into the coming academic year with further studies across more subjects, over larger time horizons for the sciences, and with larger samples.
Read the paper on arXiv: Evaluating AI Tutoring at the Speed of Innovation: Practitioner-Led Micro-Randomised Trials of an AI Tutoring Platform in GCSE Science (Harrison, Khowaja, Dobson, Uwimpuhwe and Higgins, 2026).
Try Medly AI for free
No credit card required. Start learning for free today.