Author evaluation: Publication awards

Publish or Perish tutorial

Anne-Wil Harzing - Sat 6 Feb 2016 16:10 (updated Thu 2 Jun 2022 11:00)

Note: This tutorial was originally written for Publish or Perish v4 and all screenshots come from this version. However, the information as such is also applicable for later versions of Publish or Perish.

Citation impact is one factor for publication awards

Many journals give out some type of best paper award on a yearly basis. One of the factors that are often considered when awarding best journal article prizes is the (citation) impact a particular article has had. This is relatively easy to do when the awards are given 10 years after publications, as is for instance the case with the Journal of International Business Studies decade award.

Citation impact within 1-2 years is typically modest

However, most journal article prizes are awarded 1 or 2 years after the articles are published. Especially in the Social Sciences and Humanities there are few articles that gather significant citation impact in such a short time.

Worked example: AMLE outstanding article of the year award

In 2010, my article with Nancy Adler (When Knowledge Wins: the Sense and Nonsense of Academic Ranking) won the 2009 outstanding article of the year for the Academy of Management Learning and Education journal in which it was published. I hope this was mainly because of the article’s content, which cautioned against an exclusive focus on academic rankings and discussed the importance of doing research that has relevance to societal problems. However, it is likely that the decision was at least partially influenced by the article’s citation impact.

Virtually no ISI citations for AMLE at the time of award decision

Unfortunately, around April 2010, when the award decision was taken, the ISI Web of Science had only incorporated articles from the first (March) issue of the Academy of Management Learning & Education, with the three remaining 2009 issues still in process. Of these March articles there was one article with 18 citations (the awarded paper), two articles with 4 citations and several articles with 1-3 citations for a total of 38 citations for AMLE for 2009. Hence, ISI data would have been pretty useless in assessing the impact of articles published in AMLE in 2009.

Google Scholar provides comprehensive citation data

Looking at Google Scholar data in Publish or Perish (see screenshot), the picture is entirely different. All 2009 AMLE papers are included. Although the awarded article is still the most cited article in the journal, there are fourteen (not two) other articles with at least 4 citations and the total number of citations to AMLE articles published in 2009 is 179, not 38. From this, I would conclude that many articles published in this journal do have a fairly substantial immediate impact.

Use Google Scholar data as early indicator of impact

However, even in cases where ISI does have complete data for the journal in question, I would recommend using Google Scholar data instead when evaluating papers for publication awards. Google Scholar citations are a much better indicator of early impact as they include citations in conference papers and working papers, most of which will eventually find their way to published articles.

Support Publish or Perish

The development of the Publish or Perish software is a volunteering effort that has been ongoing since 2006. Download and use of Publish or Perish is and will remain free (gratis), but your support toward the costs of hosting, bandwidth, and software development are appreciated. Your support helps further development of Publish or Perish for new data sources and additional features.

Feedback

PS: If you are using Publish or Perish on a regular basis, please take 5 minutes to provide me with some feedback.