Have you considered taking a look at the book Visible Learning by John Hattie? It seems to be permeating and informing reform in our K-12 schools nationwide. Districts are spending a lot of money sending their staffs to conferences by Solution Tree to train their schools to become PLC communities which also use an RTI (Response To Intervention) model. Their powerpoint presentations prominently feature John Hattie’s work. Down the chain, then, if all of these school districts attending are like mine, their superintendents, assistant superintendents, principals, and vice principals are constantly quoting John Hattie’s work to support their initiatives, because they clearly see it as a powerful tool.
I am asking not as a proponent or opponent of Hattie’s work. I’m asking as a high school math teacher who found that there does not seem to have been much critical analysis of his work (except by Arne Kåre Topphol and Pierre-Jérôme Bergeron, as far as I can tell from a cursory search.) This seems strange given its ubiquitous impact on educational leaders’ plans for district and school-wide changes that affect many students and teachers. An old college wrestling teammate of mine, now a statistician, encouraged me to ask you about this.
The reason educational leaders have latched onto this book so much, I believe, is Hattie’s synthesis of over 1,000 meta-analyses. This is, no doubt, a very appealing thing. I’m glad to see educational leaders using data to inform their decisions, but I’m not glad to see them treating it as an educational research bible, of sorts. I wonder about the statistical soundness (and hence value) of synthesizing so many studies of so many designs. I wonder about a book where there’s only two statistics primarily used, one of them incorrectly. And, finally, I wonder about these things b/c this book is functioning as fuel for educational Professional Development conferences over multiple years in multiple states (i.e., it’s a significant component in a very profitable market) as well as the primary resource used by administrators in individual districts to affect change, often without teachers as change-agents. Regardless of these concerns, I also appreciate conversations the book elicits, and am open to the notion that perhaps there are some sound statistical conclusions from the book, ignoring Hattie’s misuse of the CLE stats. (Similarly, I should note, I like a lot about the RTI model that Solution Tree teaches/sells.) I’m sending you this email from a place of curiosity, not of cynicism.