What makes an article highly cited?

I was recently asked to write a commentary on Arbaugh, Fornaciari and Hwang (2016) article "Identifying Research Topic Development in Business and Management Education Research Using Legitimation Code Theory". The authors use citation analysis – with Google Scholar as their source of citation data – to track the development of Business and Management Education research by studying the field’s 100 most highly cited articles.

Factors influencing an article's level of citations

In their article, the authors distinguish several factors that might impact on an article’s level of citations: the topic it addresses, the profile of the author(s) who wrote it and the prominence of the journal that the article is published in. Although these three factors might seem rather intuitive, and the authors certainly are not the first to identify them, there is a surprising dearth of studies in the bibliometrics literature that attempt to disentangle the relative impact of these factors on citation outcomes.

Why does it matter for academic evaluation?

If citation levels of individual articles are determined more by what is published (topic) and who publishes it (author) rather than by where it is published (journal), this would provide clear evidence that the frequently used practice of employing the ISI journal impact factor to evaluate individual articles or authors is inappropriate. Our regression analysis shows that, when all factors are taken into account at the same time, it is what is published (topic) and who has published it (author) that have the largest impact on citations, not where it is published (journal).

Hence, the commonly used practice of using the prestige of a journal – oftentimes operationalized as the ISI journal impact factor – as a proxy for (citation) impact is clearly not appropriate for the field of Business and Management Education. It is thus rightly condemned by San Francisco Declaration on Research Assessment and should not be used in academic evaluation. Instead, as Arbaugh et al. (2016) do, individual articles should be evaluated on their own merit and – especially in the Social Sciences and Humanities – any analysis of citation impact should use Google Scholar as a preferred data source rather than the ISI Web of Science.

Want to read the paper?

  • Harzing, A.W. (2016) What, who or where? Rejoinder to Identifying Research Topic Development in Business and Management Education Research Using Legitimation Code Theory, Journal of Management Education, vol. 40, no. 6, pp. 726-731. Available online... Publisher's version (free access!)