What Is Scientifically Based Research? A Guide For Teachers

More than ever, educators are expected to make decisions that guarantee quality instruction. As knowledge emerges, so do philosophies, opinions, and rhetoric about definitions of instructional excellence. From policy makers to classroom teachers, educators need ways to separate misinformation from genuine knowledge and to distinguish scientific research from poorly supported claims.

Effective teachers use scientific thinking in their classrooms all the time. They assess and evaluate student performance, develop Individual Education Plans, reflect on their practice, and engage in action research. Teachers use experimental logic when they plan for instruction: they evaluate their students’ previous knowledge, construct hypotheses about the best methods for teaching, develop teaching plans based on those hypotheses, observe the results, and base further instruction on the evidence collected.

In short, teachers use the concepts of rigorous research and evaluation in profoundly practical ways.

Teachers can further strengthen their instruction and protect their students’ valuable time in school by scientifically evaluating claims about teaching methods and recognizing quality research when they see it. This booklet, distilled from the monograph Using Research and Reason in Education: How Teachers Can Use Scientifically Based Research to Make Curricular and Instructional Decisions (opens in a new window), provides a brief introduction to understanding and using scientifically based research.

Teachers can use a simple set of questions to distinguish between research that confirms the effectiveness of an instructional practice and research that does not:

• Has the study been published in a peer-reviewed journal or approved by a panel of independent experts?
• Have the results of the study been replicated by other scientists?
• Is there consensus in the research community that the study’s findings are supported by a critical mass of additional studies?

Independent peer review

Peer review subjects a paper to scrutiny by scientists in the relevant field of specialization. This happens in two ways. In one method, a paper submitted for publication in a peer-reviewed journal is examined by other scientists in the field before an editor (usually an expert in the field) passes judgment on it. The second method is review by an independent panel of experts who, using rigorous criteria, determine whether the findings of the paper are credible.

Peer review provides a baseline of quality control because it exposes ideas and experimentation to examination and criticism by other researchers. Its absence should raise doubt about the quality of the research. Presentations at education conferences that make claims about specific educational practices should also be held to this standard.

It is relatively easy for teachers to determine if a paper has been published in a peer-reviewed journal; it can be more difficult to determine whether a panel review (without publication) has occurred unless it is specified in the paper.

Replication of results by other scientists

Teachers should look for evidence that an instructional technique has been proven effective by more than one study. Knowledge generated by one study without scrutiny and criticism by others cannot be fully scientific. To be considered scientifically based, a research finding must be presented in a way that enables other researchers to reach the same results when they repeat the experiment.

True scientific knowledge is public and open to challenge. It is held tentatively, subject to change based on contrary evidence.

Consensus within a research community

A single experiment rarely decides an issue, supporting one theory and ruling out all others. Issues are most often decided when the community of scientists in a field comes to agreement over time that sufficient evidence has converged to support one theory over another. Scientists do not evaluate data from a single, perfectly designed experiment. They evaluate data from many experiments, each containing some flaws but providing part of the answer.

In many cases, science has not yet provided the answers teachers and others need to make fully informed decisions about adopting, or dropping, particular educational strategies. What if an area of education lacks a research base, has not been evaluated according to the principles of scientific evaluation described above, and no consensus exists? In those cases, teachers have to rely on scientific reasoning to find their way. An important first step is to look at the findings and principles from the established research base.

Imagine, for example, that two untested treatments for children with extreme reading difficulties have emerged. The first treatment suggests a new strategy for teaching phonemic awareness by using only songs and clapping to teach children how language can be broken down into sound segments, or phonemes. The second treatment trains children in vestibular sensitivity by having them walk on balance beams while blindfolded. In this hypothetical case, neither of these new treatment ideas has been tested. Some treatments related to phonemic instruction have, in fact, been scientifically tested and proven effective. This hypothetical example imagines a new strategy — using only songs and clapping — that has not been proven effective. Neither has produced empirical evidence to prove that it is effective.

The lack of such evidence need not automatically lead to the conclusion that the methods do not work. Even without empirical evidence, one might find support for one or both methods from other studies conducted on similar strategies. In this case, the strategy featuring awareness of sound segments merits consideration first, because it makes contact with a broad consensus in the research literature that children with severe reading difficulties are hampered by an insufficiently developed awareness of language’s segmented structure. The second does not have a comparable link to existing research. A teacher thinking scientifically can make a reasonable conclusion that the first method is preferable by knowing that there is a link to the broader research base.

Teachers play a variety of roles in their work — instructor, coach, advocate, and learner — but they also act as scientists in several ways. As they make the important decisions about what and how to teach, they must evaluate the claims associated with educational strategies and programs. And in the classroom, they must constantly assess and reassess the value of programs and their impact on students.

The basic principles of the scientific method

• Science progresses by investigating testable problems.
• A testable theory yields predictions that could possibly be proven wrong.
• Scientific knowledge has passed some minimal tests.
• Data and theories are considered in the public domain, or included in the research base, only after a peer review, either by a journal or a panel.
• Published data and theories allow for replication and criticism by other scientists.
• Theories are tested by systematic observation bound by the logic of true experiments.
• Correlational studies, useful when experiments can not be carried out, only help rule out hypotheses.
• Researchers use many different methods to reach conclusions. Most often, they draw conclusions only after a slow accumulation of data from many studies.