How to Evaluate Conflicting Research About Educational Programs

Picture of David Stanley
David Stanley
Fast ForWord, Educational Neuroscience

Martha Burns.ASHA._1_-1550604536-1I heard from a school about the great results they are getting from a neuroscience program.  Then one of my staff pointed out a meta analysis that is critical of the program. I can’t understand why supposedly gold standard research - a meta analysis - is saying something totally different from what I hear is happening in other schools.” 

That’s what a school principal said to us.

She understood the value of educational neuroscience and was considering whether to use the Fast ForWord neuroscience program in her school to assist teachers to help improve students' learning and reading abilities.

The meta analysis was published in 2010, using research from the previous decade. The authors, Strong et al, selected 5 of 79 published studies they had found on Fast ForWord.

Most people don't bother to read a meta-analysis and its component studies as it takes time and contains complex scientific and statistical concepts. Instead they rely just on the summary for their understanding of it.  And that’s what the school principal had done.

So we referred her to this video review of the meta-analysis by Dr Marty Burns.  

Strong et al meta analysis

In it Dr Burns explains the criteria for valid research about educational programs. Also the importance of checking that a study has used the program correctly - as shown by previous published research.

In the same way, if you join a gym and don’t go, you won’t get the result you wanted.

Here is a one page summary of the video.