From h-index to hIa: The ins and outs of research metrics

Explains two key research metrics and compares three sources of citation data: the Web of Science, Scopus, and Google Scholar.

Anne-Wil Harzing - Wed 20 Jul 2016 07:48 (updated Sun 14 Jan 2024 16:51)

In the last decade, the use of metrics for research evaluation has become an integral part of the academic landscape. The adverse impact of this “audit culture” is well documented (see e.g. Adler & Harzing, 2009). However, the reversal of this trend is unlikely. Therefore, it is important for academics to be aware that the use of different research metrics can affect their perceived research performance. This blog reviews the h-index and the individual annualised h-index (hIa) and compares three sources of citation data: the Web of Science, Scopus, and Google Scholar.

[A more formal version of this blog post appeared as: Harzing, A.W. (2015) From h-index to hIa: The ins and outs of research metrics, www.harzing.com white paper. Please reference that version if you are citing this material.]

h-index: correcting for one-hit wonders & salami slicers

Unless you have been hiding under a stone in the last ten years, you will probably have heard about the h-index. It is defined as follows (Hirsch, 2005:16569):

A scientist has index h if h of his/her Np papers have at least h citations each, and the other (Np-h) papers have no more than h citations each.

The h-index combines an assessment of both quantity (number of papers) and an approximation of quality (impact, or citations to these papers). An academic cannot have a high h-index without publishing a substantial number of papers. However, this is not enough. These papers need to be cited in order to count for the h-index.

As such the h-index is said to be preferable over the total number of citations as it corrects for “one hit wonders”, i.e. academics who might have authored (or co-authored) one or a limited number of highly cited papers, but have not shown a sustained and durable academic performance. It is also preferable over the number of papers as it corrects for academic who write many papers that are not or rarely cited. Hence the h-index favours academics that publish a continuous stream of papers with lasting and above-average impact.

Career length & disciplinary differences in citation metrics

A shortcoming of the h-index is that it cannot be used to compare academics at different career stages or academics that work in different disciplines. It is obvious that there will be large differences between junior and senior academics in terms of the h-index as papers of junior academics have not yet had enough time to accumulate citations. Especially in the Social Sciences and Humanities it might take more than five years before a paper acquires a significant number of citations.

However, it might not be obvious that there are also large differences in typical h-values between disciplines. Part of the differences between disciplines are caused by the fact that academics in the Life Sciences and Sciences typically publish more (and shorter) articles, and also publish with a large number of co-authors. Academics in the Social Sciences and Humanities typically published fewer (and longer) articles (or books) and publish with a smaller number of co-authors.

hI,annual (hIa) provides a fairer comparison

Therefore, a metric that corrects for these differences provides information that the h-index cannot deliver. Harzing, Alakangas & Adams (2014) introduced such a metric, namely the hI,annual (or hIa for short). This index corrects the h-index for both career length and differences in the level of co-authorship. It is calculated as follows and is one of the standard metrics reported by Harzing’s Publish or Perish (2007):

• hIa: hI _norm/academic age, where:
  • hI _norm: normalize the number of citations for each paper by dividing the number of citations by the number of authors for that paper, and then calculate the h-index of the normalized citation counts
  • academic age: number of years elapsed since first publication

The hIa-index thus measures the average number of single-author equivalent h-index points that an academic has accumulated in each year of their academic career. A hIa of 1.0 means that an academic has consistently published one article per year that, when corrected for the number of co-authors, has accumulated enough citations to be included in the h-index. Someone who co-publishes with others will not need to publish more articles to achieve the same hIa as an academic who publishes single-authored articles. However, the co-authored articles will need to gather more citations to become part of the hIa as the article’s citations will be divided by the number of co-authors.

Comparisons using the h-index and hIa across career length

Harzing, Alakangas & Adams (2014) and Harzing & Alakangas (2016) illustrate the use of the hIa in a study of 146 academics in 37 different disciplines in the Life Sciences, Sciences, Engineering, Social Sciences, and Humanities. To demonstrate the effect of the hIa in terms of correcting for career length differences, Table 1 ranks the top-10 most highly ranked academics for both the h-index and the hIa.

Table 1: Level of appointment (Full Professor or Associate Professor) and academic age of the top-10 highest ranked academics by h-index and hIa-index (Scopus data only). Adapted from Harzing, Alakangas & Adams (2014)

The top-10 by h-index includes mainly full professors, with an average academic age of 34.4 years. The top-10 by hIa is perfectly balanced in terms of level of appointment and the average academic age is much shorter at 20.7 years. This demonstrates that the h-index is strongly influenced by longevity, whereas the hIa provides a more level playing field for younger academics.

Comparisons using the h-index across disciplines

Figures 1 and 2 illustrate the effect of using the hIa rather than the h-index across disciplines. They also show the different levels of coverage across disciplines of the three main sources for citation data: Web of Science, Scopus and Google Scholar. Disciplinary differences between the databases are substantial.

Figure 1: Average h-index per academic for five different disciplines in three different databases, July 2015 (source Harzing & Alakangas, 2016)

In the Web of Science, the h-index of the average Life Sciences academic is nearly 8 times as high as for the average Humanities academic and nearly 3 times as high as for the average Social Scientist. In Google Scholar these differences are reduced to 2.7 times as high for Humanities and only 1.5 times as high for the Social Sciences

Comparisons using the hIa across disciplines

An even more striking picture appears when, instead of the regular h-index, we compare disciplines using the hIa (Figure 2). Using this metric reduces the differences between disciplines for any database.

Figure 2: Average hIa per academic for five different disciplines in three different data bases, July 2015 (source Harzing & Alakangas, 2016)

Even in the Web of Science, the difference between Science and Life Science academics on the one hand, and Social Science and Engineering academics on the other hand, is now relatively small, the latter only showing 30% higher metrics. In Scopus, four of the five disciplines now have very similar scores, whereas in Google Scholar the average for the Social Sciences is marginally higher than the Life Sciences average, and substantially higher than the average for Engineering and the Sciences.

Conclusion

As for other research metrics, the hIa should never be used as the sole criterion to evaluate academics. However, we conclude that the hIa provides a more reliable comparison between academics in different disciplines and at different career stages than the h-index, and would thus recommend it whenever these types of comparisons are deemed necessary.

Watch an interview on the Publish or Perish software

References

• Adler, N., & Harzing, A.W. (2009) When Knowledge Wins: Transcending the sense and nonsense of academic rankings, The Academy of Management Learning & Education, vol. 8, no. 1, pp. 72-95. Available online... - Publisher's version - Related blog post - [Winner of the 2009 AMLE Outstanding article of the year award, free download courtesy of AoM.] - ESI top 1% most Highly Cited Paper
• Harzing, A. W. (2007) Publish or Perish, available from http://www.harzing.com/pop.htm
• Harzing, A.W.; Alakangas, S. (2016) Google Scholar, Scopus and the Web of Science: A longitudinal and cross-disciplinary comparison, Scientometrics, vol. 106, no. 2, pp. 787-804. Available online... - Publisher's version (read for free) - Related blog post - Presentation slides - Video presentation of this article - Blog post - ESI top 1% most Highly Cited Paper - ESI hot paper
• Harzing, A.W., Alakangas, S., & Adams, D. (2014) hIa: An individual annual h-index to accommodate disciplinary and career length differences, Scientometrics, vol. 99, no. 3, pp. 811-821. Available online... - Publisher's version (read for free) - Related blog post
• Hirsch, J. E. (2005). An index to quantify an individual's scientific research output. Proceedings of the National academy of Sciences of the United States of America, 102(46), 16569-16572.