Did Reading First Reveal Phonics Instruction to be Futile?

Teacher Question: I’m a big phonics promoter. Recently, someone challenged me saying that the fact that Reading First didn’t work shows that emphasizing phonics is a bad idea. Can you help?

Shanahan’s response:

In 2001, the President and the U.S. Congress agreed on the creation of a $5 billion program to enhance reading instruction K-3 in especially low performing Title I schools. That program was called Reading First. Every state got a portion of the funds based on their poverty statistics and there was a list of schools and school districts that were eligible for this money based on reading performance on their state tests (3^rd grade scores). The grants were sizeable, often hundreds of thousands of dollars of additional funding only for use in K-3.

This money had to be spent on four things:

(1) professional development for teachers that emphasized phonemic awareness, phonics, oral reading fluency, vocabulary, and reading comprehension;

(2) core reading program adoptions – purchases of programs that were consistent with the research in terms of teaching those 5 things;

(3) an assessment system (typically DIBELS) to screen kids at the beginning of the year and to test them periodically throughout the year to monitor their progress; and

(4) intervention programs aimed at teaching the struggling students those 5 things identified as necessary by those assessments.

Three-quarters (75%) of the money was to be distributed to these underperforming schools to do these things. The other 25% of the money was to be used by the states themselves to incent all of their other schools to do these things, too, but with their own budgets. Simultaneously, Title I nationally encouraged, incented, and mandated certain changes in all Title I schools (including those with no Reading First money). Reading First was the Bush administration’s attempt to codify and implement the findings of the National Reading Panel, the work of which had been conducted under the Clinton administration.

Finally, the U.S. Department of Education was required to evaluate the implementation and effectiveness of Reading First. 

I was a member of the National Reading Panel and served as a consultant on both of the later implementation and the evaluation studies.

The implementation studies revealed that Reading First schools spent a small amount of additional time teaching reading than other Title I schools (it was a statistically significant difference, but not necessarily a pedagogically meaningful one – the $5 billion led to less than 10 more minutes of reading instruction per day). Early on, Reading First schools emphasized phonemic awareness, phonics, fluency vocabulary, and comprehension more than did the non-Reading First schools, but over time the implementation differences shrunk (which makes sense given that states were trying to get everyone else to do what Reading First schools were doing):

“The Implementation Evaluation found that RF teachers reported using instructional practices emphasized by the Reading First Program. It also found that, over time, teachers in other schools increasingly reported using similar practices, and that while significant differences reported between the two types of schools persisted, the differences diminished between 2004–05 and 2006–07.” (Gamse, et al., Reading First Implementation Study 2008: Final Report (opens in a new window). 2011, p. 13)

The outcome evaluation found the following:

• “Reading First produced a positive and statistically significant impact on amount of instructional time spent on the five essential components of reading instruction promoted by the program (phonemic awareness, phonics, vocabulary, fluency, and comprehension) in grades one and two.
• Reading First produced positive and statistically significant impacts on multiple practices that are promoted by the program, including professional development in scientifically based reading instruction (SBRI), support from full-time reading coaches, amount of reading instruction, and supports available for struggling readers.
• Reading First did not produce a statistically significant impact on student reading comprehension test scores in grades one, two or three.
• Reading First produced a positive and statistically significant impact on decoding among first grade students tested in one school year (spring 2007). The impact was equivalent to an effect size of 0.17 standard deviations” (Gamse, et al., Reading First Impact Study: Final Report (opens in a new window). 2008, pp. xv-xvi)

In any event, there were several schools in the evaluation that were doing better under this regime, and some of the state studies (like Pennsylvania, Bean, et al., 2010) showed extensive improvement in early reading achievement, including comprehension, as a result of Reading First.

Unfortunately, in addition to these limited learning outcomes, there was a scandal in the administration of the program (particular instructional products were favored by the Department of Education, a big ethical no-no) and the U.S. invaded Iraq, undermining President Bush’s popularity. When it came time to reauthorize this expensive program, there was no political will among Democrats to support the President on anything, and there was no Congressional support for aligning oneself with a scandal and limited outcomes. 

The implementation study itself shows that by the time the outcome evaluations were being measured, there were few practical differences between Reading First and non-Reading First schools, which should not be that surprising given that the states had spent $1 billion trying to make that happen.

I documented that on my website at the time (Explaining the Reading First Impact Study: New Districts Added (opens in a new window)), listing some school districts (and even states) that had mandated that all of their schools follow the Reading First practices, using the same programs and trainers, etc. For instance, Florida implemented Reading First statewide (using federal money where it was available and Florida tax dollars where it was not) – they made clear gains during that period. The same can be said about the Bureau of Indian Affairs; they did Reading First in all of their schools and had clear improvements.

Researchers worry about this kind of thing (as it might lead to what they refer to as a Type II error) because it means the study isn’t comparing different programs. If everyone is treated with pretty much the same antibiotic regime, then the experimental group shouldn’t outperform the controls, no matter how effective that regime may be (and, then penicillin is out the window). 

I shared this information years ago with some European researchers and told them how the comparison group had been intentionally contaminated by the Reading First policy. They told me that under those circumstances the only possible way you could meaningfully evaluate the program would be to see what happened nationwide in reading achievement over that period. In fact, those years were the the last ones that saw NAEP improvement for our fourth graders. Reading achievement improved significantly during those years and has languished nationwide since its demise.

I would have loved to see follow up studies on those school districts that had been so successful with Reading First as opposed to those who were not. For example, in Chicago we received substantial Reading First money and purchased the Reading First approved programs, but the district told the schools that received the programs that they didn’t have to use them. They often were never even taken out of the boxes!

Now, back to your question. Does the demise of Reading First suggest that a heavy emphasis on phonics instruction is likely to be ineffective?

Since Reading First was not a phonics program, but an overall instructional improvement effort (including phonics) …

And since Reading First did not lead to much more instructional emphasis on phonics (or any other aspect of instruction) than in the comparison schools …

And since Reading First did not lead to any consistent or meaningful superiority in decoding skills …

And since Reading First (and the associated instructional efforts) led to clear national improvements in primary reading achievement according to NAEP …

The idea that the Reading First experience showed the futility of phonics is dopey.

See comments › (opens in a new window)