Author evaluation

Publish or Perish tutorial

Anne-Wil Harzing - Sat 6 Feb 2016 16:10 (updated Tue 26 May 2020 13:53)

Publish or Perish can be used to provide a quick evaluation tool to assess a specific set of academics for a large variety of functions. I would certainly not advocate it to be used as the only evaluation mechanism. However, it can be very useful as a first port of call, because it allows you to quickly reduce your options to a smaller group to assess in more detail.

Evaluate reviewers, examiners, key note speakers, referees

There is a large variety of possible functions one could think of: reviewers for journals or conferences, examiners for Master of PhD theses, key note speakers, discussants or session chairs for conferences, academic mentors, referees, etc. In this section, I will discuss selection as an editorial board member, but the mechanisms involved are very similar for most of these functions. There are several things an editor or evaluator would be interested in.

• Academic credibility: The first question would be: Does the prospective editorial board member (or reviewer or examiner or referee…) have a credible publication record? If one is selecting an editorial board member for a prestigious journal, there should be some evidence of publications that have had an impact on the field and of a sustained stream of research output. This can be easily evaluated looking at the number of publications and citations that PoP reports.
• Expertise in the area in question: The prospective editorial board member should also have expertise in the disciplinary orientation of the journal or the sub-discipline that is currently underrepresented. A quick perusal of the titles of his or her publications should be sufficient to establish this. For some journals it might be important to have a broad orientation so that one is able to review in a range of different, but related areas. Other journals might prefer specialists, either because the journal is a specialist journal itself (e.g. International Journal of Nuclear Desalination), or because the journal has a more general orientation, but only publishes the very best research in each sub-discipline (e.g. Science).
• Experience with the journal: An editor will also want to know whether the prospective editorial board member has experience with the journal. Most journals will keep systematic files on their ad-hoc reviewers. So if the prospective board member has been a successful ad hoc reviewer, they can be expected to have sufficient experience with the journal. However, editors would normally give preference (or in some cases only consider) academics that have published in the journal themselves. Publish or Perish makes it very easy to run a quick search on this using the General citation search.
• Geographical scope: Many journals in the Social Sciences will publish work conducted in different countries. To the extent that the country context matter for the research in question, it is important to have editorial board members with a broad geographical experience. Although it is not always possible to deduce this from the articles titles, in many cases a quick perusal of the PoP results should provide the editor with a feel for the experience the prospective board members has with research in different countries. Looking at their co-authors might also give some clues, to the extent one can deduce nationality from names.

Caveats

None of these factors can be established with absolute certainty through a simple Publish or Perish search. However, the editor should be able to get a pretty good feel for the prospective board members that are worthy of further investigation.

Support Publish or Perish

The development of the Publish or Perish software is a volunteering effort that has been ongoing since 2006. Download and use of Publish or Perish is and will remain free (gratis), but your support toward the costs of hosting, bandwidth, and software development are appreciated. Your support helps further development of Publish or Perish for new data sources and additional features.