My thoughts on the Reinhard and Rogoff coding error:
2. I liked the piece by cyniconomics, which was linked to on MR.
3. I liked the piece by my colleagues Betsey Stevenson and Justin Wolfers
4. Journalists and other economists need to have a reasonable prior here. The Journal of Money, Credit and Banking replication exercise, summarized in this (gated) American Economic Review article and now more than two decades old, probably provides the best evidence. The authors of the summary article conclude "our findings suggest that inadvertent errors in published empirical articles are a commonplace rather than a rare occurrence." Some enterprising journal should repeat the exercise so that priors can be updated.
5. It seems to me professional courtesy that if you replicate someone's paper that has gotten a lot of media attention, and you find something that is likely to draw more media attention, you should send your findings to the authors first and give them a week or two to digest and respond before going to the media yourself. My understanding is that the UMass-Amherst folks did not do this. In my view, they should have.
6. My final (and quite serious) suggestion is that Dan Hamermesh write one of his famous, and very useful, advice papers on the topic of the professional etiquette of replication. Economists do not replicate as much as they should. Part of that is too-low rewards, but part of it is also, I think, that things may go sour in a very public way, as they have in several high profile replications in recent years. Perhaps a formal statement of norms of good behavior in this domain would clarify expectations all around and also encourage more social pressure on both the replicators and the replicated to behave in ways that enhances the scientific credibility of the discipline.
Thanks to Miles Kimball for some encouragement.
0 comments:
Post a Comment