8.4.2 Google scholar spots early cites for best journal articles

Many journals award some type of best paper award on a yearly basis. One of the factors that are often considered when awarding best journal article prizes is the (citation) impact a particular article has had. This is relatively easy to do when the awards are given 10 years after publications, as is for instance the case with the Journal of International Business Studies decade award.

However, most journal article prizes are awarded the 1 or 2 years after the articles are published. Especially in the Social Sciences and Humanities there are few articles that gather significant citation impact in such a short time. For instance this year (2010), my article with Nancy Adler (When Knowledge Wins: the Sense and Nonsense of Academic Ranking) won the 2009 outstanding article of the year for the Academy of Management Learning and Education journal in which it was published.

I hope this was mainly because of the articles content which cautioned against an exclusive focus on academic rankings discussed the importance of doing research that has relevance to societal problems. However, it is likely that the decision was at least partially influenced by the articles citation impact.

Unfortunately the ISI Web of Science general search database has so far neglected to incorporate any articles published in AMLE for 2009, except for articles in the first (March) issue. Of these there was one article with 18 citations (the awarded paper), two articles with 4 citations and several articles with 1-3 citations for a total of 38 citations for AMLE for 2009. Hence, it is clear that ISI data would have been pretty useless in assessing the impact of articles published in AMLE in 2009.

Looking at Google Scholar data (see screenshot below), the picture is entirely different. Although the awarded article is still the most cited article in the journal, there are fourteen (not two) other articles with at least 4 citations and the total number of citations to AMLE articles published in 2009 is 179, not 38. From this, I would conclude that many articles published in this journal do have a fairly substantial immediate impact.

However, even in cases where ISI does have complete data for the journal in question, I would recommend using Google Scholar data instead when evaluating papers for publication awards as they are a much better indicator of early impact. This is true because Google Scholar includes citations in conference papers and working papers, most of which will eventually find their way to published articles. For more detailed information on this, see Chapters 7 and 13.