Don’t judge a paper until you read it!

I have a dream that my research papers

will one day live in a community where

they will not be judged by the journal they are published in

but by the content of their equations.

This is a post in which I advocate for an abolishment of a form of scientific prejudice that I see too many of my colleagues suffering from, myself included: we often take the journal in which an article is published to be the biggest indicator of the quality of the article.

In the argumentative style of David Hume, I begin by pointing out some facts.

Fact #1: There are lousy articles published in high impact journals.

My favourite example is this one: Information transmission under conditions of sensory shielding: a.k.a. a highly non-rigorous experiment trying to argue for the possible existence of extra-sensory perception. Schroedinger’s rat provides some more examples, though I am sure most of the readers can think of some of their own.

Fact #2: Ground-breaking results have been published in unconventional journals.

Again, my favourite example is John Bell’s original paper introducing his now famous theorem.  Another one, which is very relevant to the research taking place in our group, is this paper by Devetak and Winter, where they establish a very useful security framework for quantum key distribution.

Fact #3: By definition, articles appearing in high-impact journals receive more citations on average than those appearing in lower-impact journals.

This fact is definitely a major reason why many researchers put enormous efforts into publishing their results in prestigious journals. Of course, we could also argue whether the number of citations is a good indicator of scientific merit, but that is a topic for another day.

The three facts I have outlined should be very well known to any researcher, yet easily forgotten. Why do I say this? Because Fact #1 and Fact #2 lead to the following logical conclusion: The scientific quality of a paper cannot be judged solely by the journal that it is published in. Plain and simpleHowever, perhaps because of Fact #3, many people do judge the quality of a paper only in terms of the journal it appears in. It is easy to fall prey to the fallacy: If a paper is cited many times, then it is a good paper and if a paper appears in a high-impact journal, it will be cited many times. Therefore, if a paper appears in a high-impact journal it must be a good paper. This is unjustified prejudice, and like all forms of prejudice, it is unhealthy, discriminatory and dangerous. It is not very different from other forms of prejudice that have been and continue to be poisonous to society. How many people have done fantastic work that was not properly recognized because it wasn’t published in a fancy journal? How many people have fuelled their careers by having questionable articles appearing in prestigious journals? Too many, I suppose.

Look, I am aware that I am not saying anything new here. We know better than to act on prejudice. The real problem is that we do it anyway, over and over. I hear it all the time, research results being judged on where they are published and not on their accomplishments. What I want to do is to persuade you to stop the prejudice. To recognize when you do it and force yourself not to do it again. Do not judge the quality of a paper until you read it! Here are some examples of statements I have actually heard people say, together with what I hope they would have said.

Actual Statement: “I am just happy to have my name appearing in an article in Science”.

Better Statement: “I am happy to have proved X, Y and Z and am hopeful that this will have important repercussions in our field.”

Actual Statement: “This professor in my home university was amazing. She got an article published in Nature!”

Better Statement: “This professor in my home university was amazing. She solved a long-standing problem with a very creative new technique! I want to understand it better, but the paper is hard to read.”

Actual Statement: “This guy has published three PRLs during his PhD. He will surely get a post-doc in a leading research group.”

Better Statement: “This guy has published three PRLs during his PhD. I read them and, honestly, I think they are not significant advances to the field, which is overly hyped anyway.”

Actual Statement: “You become famous by publishing in Nature and Science.”

Better Statement: “You become famous when your research is widely used and recognized by the scientific community.”

I guess I am just offering an invitation: don’t judge a paper until you read it!

Advertisements

One thought on “Don’t judge a paper until you read it!

  1. Pingback: QCMC 2014 | Qonaom

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s