collegerankings

  • Subscribe to our RSS feed.
  • Twitter
  • StumbleUpon
  • Reddit
  • Facebook
  • Digg

Sunday, May 26, 2013

Peer review follies

Posted on 7:43 AM by Unknown
Abstract

A growing interest in and concern about the adequacy and fairness of modern peer-review practices in publication and funding are apparent across a wide range of scientific disciplines. Although questions about reliability, accountability, reviewer bias, and competence have been raised, there has been very little direct research on these variables.

The present investigation was an attempt to study the peer-review process directly, in the natural setting of actual journal referee evaluations of submitted manuscripts. As test materials we selected 12 already published research articles by investigators from prestigious and highly productive American psychology departments, one article from each of 12 highly regarded and widely read American psychology journals with high rejection rates (80%) and nonblind refereeing practices.

With fictitious names and institutions substituted for the original ones (e.g., Tri-Valley Center for Human Potential), the altered manuscripts were formally resubmitted to the journals that had originally refereed and published them 18 to 32 months earlier. Of the sample of 38 editors and reviewers, only three (8%) detected the resubmissions. This result allowed nine of the 12 articles to continue through the review process to receive an actual evaluation: eight of the nine were rejected. Sixteen of the 18 referees (89%) recommended against publication and the editors concurred. The grounds for rejection were in many cases described as “serious methodological flaws.” A number of possible interpretations of these data are reviewed and evaluated.

Gated version of the full paper here.

Hat tip: John Bound
Email ThisBlogThis!Share to XShare to FacebookShare to Pinterest
Posted in | No comments
Newer Post Older Post Home

0 comments:

Post a Comment

Subscribe to: Post Comments (Atom)

Popular Posts

  • Michigan 28, Akron 24
    Dodging a bullet does not even begin to describe this near debacle, with Akron having shots into the end zone on the final four plays! Local...
  • Book: Paying for the Party by Armstrong and Hamilton
    Armstrong, Elizabeth and Laura Hamilton. 2013. Paying for the Party: How College Maintains Inequality . Cambridge, MA: Harvard University P...
  • Assorted links
    1. Rob Mercer on tradition . Why does Canada produce so many good comedians? One suspects it has something to do with living next door to th...
  • In praise of payday lenders
    From the Atlantic, something I never thought I would see: a thoughtful, empirically grounded defense of payday lenders and other alternati...
  • Movie: Jack the Giant Slayer
    Jack the Giant Slayer is, of course, a movie version of the Jack and the Beanstalk story. I pretty much agree with the NYT reviewer down th...
  • Assorted links
    1. Drunken island monkeys . 2. Free speech versus occupational licensing in Kentucky. 3. Virginia Postrel on how to save Barnes and Noble ....
  • Cheaters welcome at Harvard
    This survey of the incoming class , reported in the Harvard Crimson, is a bit troubling. Of course, this is what folks at Chicago suspected ...
  • Assorted links
    1. The goose man . 2. When the mistress of the dead boss sues the company that fired her after he died .  Good stuff. 3. Why I am not a Re...
  • Assorted links
    1. On the history of German restaurants in Ann Arbor. Mainly because it is not within walking distance of campus, I have yet to make it to ...
  • Assorted links
    1. Michigan dorm cafeterias drop their trays . Does this count as an application of behavioral economics? 2. Seattle Public Library breaks ...

Blog Archive

  • ▼  2013 (257)
    • ►  September (18)
    • ►  August (34)
    • ►  July (58)
    • ►  June (4)
    • ▼  May (16)
      • Assorted links
      • Frontiers of academic research: metal music studies
      • Has Obama jumped the shark?
      • On the bells in the Bell Tower at Michigan
      • Peer review follies
      • CDC takedown
      • Assorted links
      • Beer league hockey without the beer?
      • Book: The End of Eternity by Isaac Asimov
      • Frank and Ernest on the difficulty of insuring aga...
      • Audi and Spock
      • A comic about fathers in honor of mother's day
      • Seminary Co-op Grand Re-opening
      • Causal follies: The Atlantic on Citizen Schools
      • Assorted links
      • Reinhart and Rogoff
    • ►  April (25)
    • ►  March (35)
    • ►  February (41)
    • ►  January (26)
  • ►  2012 (243)
    • ►  December (47)
    • ►  November (30)
    • ►  October (28)
    • ►  September (45)
    • ►  August (73)
    • ►  July (20)
Powered by Blogger.

About Me

Unknown
View my complete profile