From h-index to hIa: The ins and outs of research metrics
© Copyright 2015 Anne-Wil Harzing. All rights reserved.
First version, 4 December 2015
This brief summary is largely based on an article in Scientometrics.
- Harzing, A.W.; Alakangas, S.; Adams, D. (2014) hIa: An individual annual h-index to accommodate disciplinary and career length differences, Scientometrics, vol. 99, no. 3, pp. 811-821. Available online...
In the last decade, the use of metrics for research evaluation has become an integral part of the academic landscape. The adverse impact of this “audit culture” is well documented (Adler & Harzing, 2009; Mingers & Willmott, 2013). However, the reversal of this trend is unlikely. Therefore, it is important for academics to be aware that the use of different research metrics can affect their perceived research performance. This brief commentary reviews the h-index and the individual annualised h-index (hIa) and compares three sources of citation data: the Web of Science, Scopus, and Google Scholar.
The single metric that has attracted the highest level of attention in the last decade is the h-index. It is defined as follows (Hirsch, 2005:16569):
A scientist has index h if h of his/her Np papers have at least h citations each, and the other (Np-h) papers have no more than h citations each.
The advantage of the h-index is that it combines an assessment of both quantity (number of papers) and quality (impact, or citations to these papers). An academic cannot have a high h-index without publishing a substantial number of papers. However, this is not enough. These papers need to be cited in order to count for the h-index.
As such the h-index is said to be preferable over the total number of citations as it corrects for “one hit wonders”, i.e. academics who might have authored (or co-authored) one or a limited number of highly cited papers, but have not shown a sustained and durable academic performance. It is also preferable over the number of papers as it corrects for papers that are not cited. Hence the h-index favours academics that publish a continuous stream of papers with lasting and above-average impact.
A shortcoming of the h-index is that it cannot be used to compare academics at different career stages or academics that work in different disciplines. It is obvious that there will be large differences between junior and senior academics in terms of the h-index, as papers of junior academics have not yet had enough time to accumulate citations. Especially in the Social Sciences and Humanities it might take more than five years before a paper acquires a significant number of citations.
However, it might not be immediately obvious that there are also large differences in typical h-values between disciplines. As a general rule of thumb, h-indices are much higher in the Life Sciences and Sciences than in the Social Sciences and Humanities, although there is a large variability even within these fields.
Part of the differences between disciplines are caused by the fact that academics in the Life Sciences and Sciences typically publish more (and often shorter) articles and also publish with a large number of co-authors, while academics in the Social Sciences and Humanities typically published fewer (and longer) articles (or books) and publish with a smaller number of co-authors.
Therefore, a metric that corrects for these differences would provide information that the h-index cannot deliver. Harzing, Alakangas & Adams (2014) introduced such a metric, namely the hI,annual (or hIa for short) which corrects the regular h-index for both career length and differences in the level of co-authorship. It is calculated as follows and is one of the standard metrics reported by Harzing’s Publish or Perish (2007):
- hIa: hI,norm/academic age, where:
- hI,norm: normalize the number of citations for each paper by dividing the number of citations by the number of authors for that paper, and then calculate the h-index of the normalized citation counts
- academic age: number of years elapsed since first publication
The hIa-index thus measures the average number of single-author equivalent h-index points that an academic has accumulated in each year of their academic career. A hIa of 1.0 means that an academic has consistently published one article per year that, when corrected for the number of co-authors, has accumulated enough citations to be included in the h-index.
Someone who co-publishes with others will not need to publish more articles to achieve the same hIa as an academic who publishes single-authored articles. However, the co-authored articles will need to gather more citations to become part of the hIa as the article’s citations will be divided by the number of co-authors.
Harzing, Alakangas & Adams (2014) and Harzing & Alakangas (2016) illustrate the use of the hIa in a study of 146 academics in 37 different disciplines in the Life Sciences, Sciences, Engineering, Social Sciences, and Humanities. To demonstrate the effect of the hIa in terms of correcting for career length differences, Table 1 ranks the top-10 most highly ranked academics for both the h-index and the hIa.
The top-10 by h-index includes mainly full professors, with an average academic age of 34.4 years. The top-10 by hIa is perfectly balanced in terms of level of appointment and the average academic age is much shorter at 20.7 years. This demonstrates that the h-index is strongly influenced by longevity, whereas the hIa provides a more level playing field for younger academics.
Table 1: Level of appointment (Full Professor or Associate Professor) and academic age of the top-10 highest ranked academics by h-index and hIa-index (Scopus data only). Adapted from Harzing, Alakangas & Adams (2014)
Figures 1 and 2 illustrate the effect of using the hIa rather than the h-index across disciplines. They also show the different levels of coverage across disciplines of the three main sources for citation data: Web of Science, Scopus and Google Scholar.
Disciplinary differences between the databases are substantial. In the Web of Science, the h-index of the average Life Sciences academic is nearly 8 times as high as for the average Humanities academic and nearly 3 times as high as for the average Social Scientist. In Google Scholar these differences are reduced to 2.7 times as high for Humanities and only 1.5 times as high for the Social Sciences.
Figure 1: Average h-index per academic for five different disciplines in three different databases, July 2015 (source Harzing & Alakangas, 2016)
An even more striking picture appears when, instead of the regular h-index, we compare disciplines using the hIa. Using this metric dramatically reduces the differences between disciplines for any database. Even in the Web of Science, the difference between Science and Life Science academics on the one hand, and Social Science and Engineering academics on the other hand, is now relatively small, the latter only showing 30% higher metrics.
In Scopus, four of the five disciplines now have very similar scores, whereas in Google Scholar the average for the Social Sciences is marginally higher than the Life Sciences average, and substantially higher than the average for Engineering and the Sciences.
Figure 2: Average hIa per academic for five different disciplines in three different data bases, July 2015 (source Harzing & Alakangas, 2016)
In this brief commentary we reviewed the use of two research metrics: h-index and the annualised individual h-index (hIa). Based on an empirical example of 146 academics in five major disciplines at different career stages, we showed that the hIa attenuates h-index differences attributable to disciplinary co-authorship practices and career lengths.
As for other research metrics, the hIa should never be used as the sole criterion to evaluate academics. However, we conclude that the hIa provides a more reliable comparison between academics in different disciplines and at different career stages than the h-index and would thus recommend it whenever these types of comparisons are deemed necessary.
- Adler, N., & Harzing, A.W. (2009) When Knowledge Wins: Transcending the sense and nonsense of academic rankings, The Academy of Management Learning & Education, 8(1), 72-95. Available online...
- Harzing, A. W. (2007) Publish or Perish, available from http://www.harzing.com/pop.htm
- Harzing, A.W.; Alakangas, S. (2016) Google Scholar, Scopus and the Web of Science: A longitudinal and cross-disciplinary comparison, Scientometrics, 106(2), 787-804. Available online... - Publisher's version - Presentation slides - Video presentation of this article - ESI top 1% most Highly Cited Paper - ESI hot paper
- Harzing, A.W., Alakangas, S., & Adams, D. (2014) hIa: An individual annual h-index to accommodate disciplinary and career length differences, Scientometrics, 99(3), 811-821. Available online...
- Hirsch, J. E. (2005). An index to quantify an individual's scientific research output. Proceedings of the National academy of Sciences of the United States of America, 102(46), 16569-16572.
- Mingers, J., & Willmott, H. (2013). Taylorizing business school research: On the ‘one best way’ performative effects of journal ranking lists. Human Relations, 66(8), 1051-1073.
Copyright © 2017 Anne-Wil Harzing. All rights reserved. Page last modified on Sun 12 Mar 2017 18:29
Anne-Wil Harzing is Professor of International Management at Middlesex University, London. In addition to her academic duties, she also maintains the Journal Quality List and is the driving force behind the popular Publish or Perish software program.