Kidwell et al. 2017, Badges to Acknowledge Open Practices: A Simple, Low-Cost, Effective Method for Increasing Transparency. Published in Plos Bio, criticized in Plos blog here. Brian Nosek responds at great length therein.
McKiernan et al. 2017, Point of View: How open science helps researchers succeed, claims that there is “evidence that publishing openly is associated with higher citation rates,” but also notes that “some controlled studies have failed to find” an effect. A glance through the citations suggests that all the RCTs find null effects, and the observational studies find substantial effects.
So, one thing that makes me feel sad about this is that they’re all published in journals with a clear stake in open access norms — PLOS, eLife, and F1000. I worry that publishing articles like these discredits the model.
Also, I do think there are large, well-justified benefits to open science practices. Davis 2011 finds that “[a]rticles placed in the open access condition (n=712) received significantly more downloads and reached a broader audience within the first year, yet were cited no more frequently, nor earlier, than subscription-access control articles (n=2533) within 3 yr.” David Donoho (2017) writes that “[w]orking from the beginning with a plan for sharing code and data leads to higher quality work, and ensures that authors can access their own former work, and those of their co-authors, students and postdocs.” But I guess that there is still demand for research showing a strong citation benefit to open scholarship, regardless of what the evidence says.
I don’t have the energy to read these papers—I guess I don’t really care so much if open-science increases citation rates by 17% or whatever—but I agree with the general principle expressed by our correspondent that it’s not good practice to exaggerate evidence, even in a good cause.