Minding the research-to-practice gap
Following the reading research is more challenging than it sounds. There are thousands of studies and it’s hard to know where to start. For classroom teachers, making sense of studies is extra challenging because a lot (but not all!) of the research has been done in contexts that we can’t replicate — intensive doses of reading intervention, delivered one-on-one, in environments with fewer competing priorities.
So how do we make sense of a study to decide whether it’s worth our effort? The first step is understanding the language researchers use to describe their work.
Making sense of research terms
The language and structure of research papers tends to be pretty consistent. Knowing some basic terms actually goes a long way. (You can download a PDF of this chart here .)
Chart: Research terminology for the classroom teacher
Term | What It Is | Why It Matters |
|---|---|---|
| Sample Size | The number of students included in a study. A larger sample (100+ students) generally gives us more confidence than a smaller one (8-10 students). | Classrooms typically have students with varying levels of language proficiency, decoding abilities, behavioral tendencies, etc. To know if a method is likely to work with our students, we need to know if similar students were included in the study. |
| Control Group | A group of students who didn’t receive the new intervention. Having a control group allows researchers to compare results. The control group might receive regular classroom instruction (“business as usual”) or a diff erent intervention. | A control group can help us determine if an intervention was more effective than another approach. When deciding whether to invest time and resources in a new approach, we want to know, “Would it be better than what we’re already doing? Or is it just better than doing nothing ?” |
| Baseline Levels | The performance of the students in the study before they had the intervention. This includes information about their initial skills, challenges, and relevant characteristics. | Understanding baseline levels helps us know if the students in the study were similar to ours. If a program worked well for students who started at a different baseline than our class, we might not see the same results. |
| Intensity of Intervention | How much instruction students received (minutes per day, days per week) and in what setting (one-on-one, small group, whole class). | If we can’t replicate the intensity of the intervention in our classroom, then we can’t expect the same results as the study. It might not be worth spending 20 min twice a week to deliver an intervention that was intended to be given for 30 min, five times a week |
| Effect Size | A measure of how much impact the intervention had, usually expressed as a number like 0.2 (small effect) to 0.8+ (large effect). | We need to balance an effect size with its cost. For example, having students plan what they’ll write before they draft has a smallish effect size of .32, but it doesn’t require much effort and it may off er additional benefits, like giving us an opportunity to set writing goals with our students (.70). Well worth it! When we’re considering costly materials, intensive training, or an overhaul of our whole schedule, we want to know if big gains are possible. |
| Statistical Significance | Tells us whether results are just due to chance. (Usually marked with p < .05 or similar.) | An intervention can be statistically significant and still not be worth our time or energy. Statistical significance by itself doesn’t tell us about practical importance. |
| Educational Significance | The practical importance of results in a real educational setting, which considers factors like cost, time, resources needed, and the actual impact on student learning. | While statistical significance tells us if results aren’t random, educational significance helps us decide if changes justify implementation. |
| Duration of Study | How long the research followed the students (weeks, months, years). | In addition to a post-test right after the intervention, good studies often follow the students to see how well their gains last. Looking at long-term impact can help us avoid expensive interventions that don’t produce lasting benefits. |
| Outcome Measures | The tools used to measure success (standardized tests, curriculum assessments, teacher observations, etc.). | If an assessment measure is closely matched to the skill targeted by the intervention, big gains are more likely. For example, a phonics intervention might show large gains on DIBELS but smaller gains on a comprehension test because reading comprehension involves many skills beyond decoding. |
| Implementation Fidelity | The degree to which the intervention was delivered as intended. | While we might think fidelity to an approach restricts teacher autonomy, it actually represents having the training, resources, and support necessary to implement a program effectively. Teaching with good fidelity means we’re more likely to get results similar to the study. |
Understanding the terms scientists use can help us make sense of important distinctions in research. I once asked Reid Lyon (an early champion for the science of reading and the former Chief of the Child Development and Behavior Branch at NICHD) to explain the difference between two terms I had thought were synonymous, efficacy and effectiveness. He explained:
An efficacy study is typically run in a highly-controlled environment in order to determine if an intervention has merit. But schools are complex systems… to determine if an intervention will work in classrooms, you need effectiveness trials, and effect sizes are typically smaller in trials. We found efficacy was sometimes reduced to 40% due to normal school challenges, like maintaining program fidelity and achieving sufficient dosage.
I took a minute to think aloud about what he’d explained: “So when teachers read studies and we feel like they are promising us results, that’s not actually a reasonable expectation because the studies aren’t often set in conditions like our own …”
Dr. Lyon added:
You have to make an educated guess — based on knowledge of your school, your youngsters, and the resources you have. Is this instructional method worth trying?
Finding relevant education research
Though I see interesting articles posted on social media, I usually try to select my reading from a peer-reviewed journal or curated set of resources, like those that are on this list.
When I need to quickly determine if a study is worth a close read, I’ll start with the abstract.
For an article like this , I’ll note:
Title: makes me curious and reminds me of Jan Hasbrouck’s quote, “Instruction is brain surgery!”
Authors: reliable researchers (Fletcher, Foorman, etc.)
Sample size:
- small, 16 students
- 8 students, of varied ages, all with severe decoding difficulties
- 8 students who never had reading problems
Intensity of the intervention: 80 hours of intensive remedial intervention in two months
And at this point, I’ll stop and think about the relevance of the study to my classroom instruction. I would need to find two hours a day to work one-on-one with students in order to put this intervention in place. This study shows something important — intensive intervention seems to help rewire the brains of struggling readers, regardless of their age — but it’s not feasible to attempt this instruction in my classroom. I might read and discuss this study with a friend who does tutoring, but it doesn’t offer super relevant guidance for my classroom instruction.
A related note:
If I find an abstract of interest but it’s behind a paywall, there’s usually an email address for the author(s). A short, friendly email does the trick and I use my school address so that they know that I’m a teacher. The author will typically send me a pdf version within a couple of days. Then I can skim the article (or upload it with some targeted questions about the intervention and findings to ClaudeAI) to determine how much time and attention I should devote to reading it.
Valuing meta-analyses
Meta-analyses help me find evidence-based teaching strategies because they review the literature on a particular topic and summarize the findings. For example, the paper below served as a basis for training in SRSD Writing that several of the teachers at my school attended.
If I didn’t have the time to carefully study the whole paper, I would have zeroed in on the strategies that had the greatest effect sizes. I’d read the descriptions of each to see if my teaching seems similar (sentence combining?) and if there are strategies that I should explore (peer assistance! setting product goals!)
Then, I’d skim down to the section labeled “Discussion” to read the specific points that the authors wanted to emphasize. The section called “Limitations” is especially important because it explains how far we can/should go, if we’re drawing our own conclusions about implications. This particular article is especially educator-friendly because it has a section titled “Issues Involved in Implementing the Recommendations.”
I typically hunt for articles that evaluate instructional strategies possible in the classroom environment with students like mine. There’s quite a lot of articles that fit those criteria, but even more that don’t, and so I save myself a lot of time!
Braving the research-to-practice divide
Though I’d love to get a PhD in statistics or methodology, I’ve found I can still sift through publications to find ones that are useful. Articles often connect me with researchers. Sometimes, I’ll ask follow-up questions to make sure I’m on the right track with my interpretation:
- So, am I right in inferring that …?
- Is it safe to say that …?
- How would you want a teacher, like me, to apply what you’ve learned?
Asking our teacher-y questions can be intimidating (see: Teachers Won’t Embrace Research Until It Embraces Them), but I remind myself that it’s my job to balance the evidence with my professional judgment about what’s feasible and worthwhile in my classroom. The more discerning teachers become about research, the more focused and effective we can be in raising our students’ achievement. We couldn’t possibly work any harder. Working smarter is the only option.
About the Author
Publication Date
April 2, 2025