Pay it Backward

I just finished writing peer reviews for an academic conference. The peer review system is the established method of quality control for scientific publications. When work is submitted for publication to an academic journal, experts judge it for correctness, importance, clarity, etc. It is then published or rejected. In computer science, conference publications, which are at least as important as journal publications, also go through the review process.

The question for this post is: how much peer reviewing ought one (researcher) do?

One easy answer is that you can't do more than you're asked to do. So if you're not being asked to review papers, no need to feel guilty! But when can you start to feel like you're doing too much reviewing? How much is the ideal amount?

The average paper gets reviewed by about 4 people, so it's not hard to compute what your "fair share" of reviewing should be. For each paper you submit, whether it is ultimately accepted or not, add to a running total: 4 divided by the number of coauthors on the paper. This sum, perhaps rounded up and say taken yearly, would be your fair share.

But just like taxes are progressive, so is reviewing load. We cannot expect a someone submitting his first paper to review 4 papers in payment. This person isn't yet known in the community and won't be asked to peer review, and either way he wouldn't yet be considered an "expert" in the field. To make up for this, more senior people have to do more reviewing -- it's only fair. Every established scientist had to submit her first paper without having "payed" for it upfront.

Only the situation is even more lopsided. Scientists need to make up for more than the reviewing burden they've placed on others when they were getting started. Research is very bottom heavy; for instance lots of graduate students leave research right after (or even before) finishing their Ph.D.s. These graduate students (at least in computer science) usually submit some publications, but don't review nearly their "fair share." So those who remain in research need to compensate, and they should -- research is their game.

Of course we have to take into account that very senior people do other forms of service that the rest of the researchers benefit from like chairing conferences, editing journals, and serving on committees. This should perhaps reduce their reviewing load.

My feeling is that in computer science a good time for the amount of reviewing you do to become as large as the amount of reviewing you ask of others is when you're a post-doc. You've stayed past graduate school, and you've become an expert in something -- time to pay your dues.

This year I've done at least as much reviewing as I've "used," but that's how it should be. It's not always the most pleasant work, but I realize it needs to be done. And if I'm lucky enough to stay in research, I expect my reviewing load to continue to increase.

Not the worst price to pay for having a job you love.