In an email with subject line, “Article full of forking paths,” John Williams writes:
I thought you might be interested in this article by John Sabo et al., which was the cover article for the Dec. 8 issue of Science. The article is dumb in various ways, some of which are described in the technical comment on it that I have submitted, but it also exhibits multiple forking paths, a lack of theory, and abundant jargon. It is also very carelessly written and reviewed. For example, the study analyzed the Mekong River stage (level of the water with respect to a reference point), but refers more often to the discharge (volume per time past a reference point: the relationship between the two is non-linear). It is pretty amazing that something like this got published.
I shared this with a colleague who is knowledgeable about this general area of research, and my colleague wrote that he agreed with Williams’s criticisms.
Williams followed up with a new document listing forking paths in the Sabo analysis.
There’s more to the story but I’ll stop here. I just wanted to share all this because it’s good to be reminded that high-profile statistical errors occur in the physical sciences too, it’s not just a problem in social science.