15.4 Discussion and conclusions

In this chapter I systematically compared the ranking of journals based on the traditionally used Thomson ISI JIF and the new Google Scholar based h-index. I have shown that any divergence between the two can generally be explained by either limitations in the way the JIF is calculated or by the more limited coverage of the ISI citation base.

I do acknowledge that our alternative metric disadvantages review journals with a small number of highly cited paper, but I conclude that in the field of Economics and Business in general, the Google Scholar based h-index provides a credible alternative for ranking journals. It addresses some of the statistical limitations underlying the JIF and is more suitable to measure a journals wider economic or social impact rather than its impact on an academic audience only.

As such I argue that the Google Scholar h-index might provide a more accurate and comprehensive measure of journal impact and at the very least should be considered as a supplement to ISI-based impact analyses. However, even though an assessment of journal impact based on the journals Google Scholar h-index might be more accurate and comprehensive than relying only on an ISI-based impact analysis, I would like to express strong caution against a single-minded focus on journal impact in evaluating individual scholars research output.

Whilst journal impact can certainly be used as one of the criteria to evaluate research output, reducing the evaluation to one single number is unlikely to provide a complete picture of a scholars real impact. Recently, many studies have established that highly-cited articles get published in journals that are not considered top journals in the field and a substantial proportion of the articles published in top journals fail to generate a high level of citations (see e.g. Starbuck, 2005, and Singh, Haddad & Chow, 2007).

Hence using journal proxies to evaluate the impact of individual articles can lead to substantial attribution errors. Unfortunately, the impact of individual articles is not generally known until quite some after their publication, making this measure more appropriate for decisions on for instance promotion or appointment to full professorial positions, than for tenure decisions.