15.3.1 Overall comparison of JIF and h-Index

There are 536 journals in the Journal Quality List that have both an ISI JIF for 2003-2006 and a Google Scholar h-index or g-index. The Spearman correlation used because both the JIF and h/g-index have non-normal distributions - is strong and very significant:

• 0.718 (p < 0.001) between the ISI JIF and the h-index,
• 0.717 (p < 0.001) between the ISI JIF and the g-index, and
• 0.976 (p < 0.001) between the h-index and g-index.

Given that these two sets of indices have different data sources (ISI Thomson JCR versus Google Scholar) and provide different metrics (a mean citations per paper count over 2 years for the ISI JIF and a combined quantity/quality measure over 5 years for the h-index and g-index) this strong correlation is quite remarkable. Given the extremely high correlation between the h-index and g-index, in the remainder of this chapter I will focus on the h-index and provide a comparison between the ISI JIF and the h-index.

Figure 1 (see the end of this chapter) shows a scatterplot of the average ISI JIF for 2003-2006 and Google Scholar h-index for articles published between 2001 and 2005. A line shows the regression equation. Outliers above the line are journals that have a high JIF in comparison to their h-index. Most of the major outliers above the regression line are Psychology journals, which generally similar to journals in the Sciences have very high immediacy index, i.e. a lot of citations to these journals occur quickly after publication.

For example, the 2006 immediacy index for the Annual Review of Psychology (4.091) is more than ten times as high as that of the American Economic Review (0.335). This means that when comparing these two journals over a 2-year period the Annual Review of Psychology will always show a higher impact factor than the American Economic Review, whereas the difference will be much smaller if I consider a 5-year period. This example clearly illustrates the folly of comparing ISI JIFs between disciplines.

The other major outlier is ACM Computing Surveys, which had wildly varying JIFs over the years, from .64 in 2001 and 2.77 in 2002 to impact factor between 7.4 and 10.0 2003-2005. However, the very high impact factors in these years appear to be mainly caused by two very highly cited articles published in 2002 and 2003 (for an even more striking case of this phenomenon see the discussion of SIAM Review below), whilst most of the relatively small number of papers published in this journal (63 over five years) are not particularly well-cited.

The two main outliers below the regression line are The Journal of Finance and The American Economic Review that have a very high h-index in comparison to their JIF. This difference is probably caused by the fact that articles in these journals are cited heavily in working papers (e.g. papers from the National Bureau of Economic Research or the Tinbergen Institute) and government policy documents, neither of which are included in Thomson ISI JIF.