When I teach research methods, a major focus is peer review. As a process, peer review evaluates academic papers for their quality, integrity and impact on a field, largely shaping what scientists accept as "knowledge"- By instinct, any academic follows up a new idea with the question, "Was that peer reviewed?"
Although I believe in the importance of peer review and I help do peer reviews for several academic journals-I know how vulnerable the process can be.
I had my first encounter with peer review during my first year as a Ph. D student. One day, my adviser handed me an essay and told me to have my -written review back to him in a week. But at the time, I certainly was not a "peer"--I was too new in my field. Manipulated data （不实的数据） or substandard methods could easily have gone undetected. Knowledge is not self-evident. Only experts would be able to notice them, and even then, experts do not always agree on what they notice.
Let's say in my life I only see white swans. Maybe I write an essay, concluding that all swans are white. And a "peer" says, "Wait a minute, I've seen black swans. "I would have to refine my knowledge.
The peer plays a key role evaluating observations with the overall goal of advancing knowledge. For example, if the above story were reversed, and peer reviewers who all believed that all swans were white came across the first study observing a black swan, the study would receive a lot of attention.
So why was a first-year graduate student getting to stand in for an expert? Why would my review count the same as an expert's review? One answer: The process relies almost entirely on unpaid labor.
Despite the fact that peers are professionals, peer review is not a profession. As a result, the same over-worked scholars often receive masses of the peer review requests. Besides the labor inequity, a small pool of experts can lead to a narrowed process of what is publishable or what counts as knowledge, directly threatening diversity of perspectives and scholars. Without a large enough reviewer pool, the process can easily fall victim to biases, arising from a small community recognizing each other's work and compromising conflicts of interest.
Despite these challenges. I still tell my students that peer review offers the best method for evaluating studies aird advancing knowledge. As a process, peer review theoretically works. The question is whether the issues with peer review can be addressed by professionalizing the field.
