15.3.4 Numerical analysis of the divergence between JIF and h-Index

Above, I have documented a number of cases where the JIF and h-index diverged in considerable detail. Table 3 provides a summary of the 50 most prominent cases of divergence. It was constructed by standardizing both the JIF (after giving a 0 score for all journals without a JIF) and the h-index and subtracting the JIF from the h-index. Since Psychology journals differed considerably from all other journals and would make up most of the top 25 on the left-hand side, I excluded them from the analysis.

Table 3: the 50 most prominent cases of divergence between JIF and h-index

JIF exceeds h-index (top 25)   h-index exceeds JIF (top 25)  
ACM Computing Surveys -6.65 American Economic Review 5.30
SIAM review -4.90 Journal of Finance 3.93
Quarterly Journal of Economics -3.81 European Economic Review 2.77
ACM Transactions on Information Systems -3.52 Communications of the ACM 2.70
Human Computer Interaction -3.46 Review of Economics & Statistics 2.38
Journal of Economic Literature -2.74 Research Policy 2.24
Academy of Management Review -2.28 European Journal of Operational Research 2.10
Progress in Human Geography -2.26 Econometrica 2.03
Human Resource Management -2.09 Journal of Banking & Finance 1.89
Journal of Economic Geography -2.08 Management Science 1.79
Annual Review of Sociology -2.03 Journal of Public Economics 1.78
Marketing Science -1.89 Journal of Political Economy 1.76
MIS Quarterly -1.89 World Development 1.65
Journal of Economic Growth -1.72 Journal of Financial Economics 1.63
ACM Trans. on Softw. Eng. and Methodology -1.68 IEEE Trans. on Knowl. & Data Engineering 1.61
Journal of Database Management -1.68 Economic Journal 1.59
Journal of Marketing -1.58 European Journal of Political Economy 1.54
Annals of the Assoc. of American Geographers -1.45 International Organization 1.50
Review of Accounting Studies -1.45 International Jnl of Project Management 1.46
Environment & Planning D -1.44 American Journal of Public Health 1.40
Journal of Org. Behavior Management -1.40 Journal of Econometrics 1.39
Journal of Rural Studies -1.36 Journal of Money, Credit & Banking 1.38
American Sociological Review -1.28 ACM Trans. on Comp. Human Interaction 1.38
Administrative Science Quarterly -1.26 Journal of Knowledge Management 1.38
Structural Equation Modeling -1.15 Review of Economic Studies 1.35

Most of these journals have been discussed in some detail above. Apart from apparent errors in the ISI JIF and one deficiency in Google Scholar coverage, the major reason for a high JIF in comparison to the h-index appears to relate to journals that publish a small number of papers and/or have highly concentrated citation papers, where the top 10 most cited articles provide the bulk of citations. As a result the mean-type the JIF presents a less accurate reflection of a journals overall impact than the h-index.

Reviewing the journal titles in the right-hand column, the single most important determinant of a high h-index in comparison to JIF seems to be the extent to which the journal publishes policy oriented papers that are highly cited in working papers and policy documents. Publishing a large number of papers and being cited in conference papers and non-ISI indexed journals provide secondary reasons for a high h-index. Overall, the h-index might therefore more suitable to measure a journals wider economic or social impact rather than its impact on an academic audience only.

As indicated above there are many journals, especially in Finance & Accounting, Marketing and General Management & Strategy that are not ISI-indexed. So how do these journals compare with journals that are ISI-indexed? As expected, journals that are ISI-indexed in general have a significantly higher h-index (23.5 versus 11.5; t = 15.002, p < 0.001).

But are there any non-ISI indexed journals with a relatively high h-index? In order to assess this I divided the journals by h-index into 4 equal groups. The quartile cut-off points for the h-index were 11, 16 and 24. Journals that ranked in the two top 50% (16 and above) in terms of h-index, but are not ISI-listed are listed in Table 4.

Table 4: Non-ISI indexed journals with a high h-index

Journal Title h-index Published in:
European Journal of Political Economy 28 Europe
International Journal of Project Management 27 Europe
ACM Transactions on Computer Human Interaction 26 USA
Journal of Knowledge Management 26 Europe
Empirical Economics 24 Europe
Accounting Horizons 23 USA
European Management Journal 23 Europe
Journal of Empirical Finance 23 Europe
European Journal of Marketing 22 Europe
Journal of Environmental Management 22 Europe
Journal of Financial Services Research 22 USA
Review of Finance 22 Europe
European Financial Management 21 Europe
Journal of Business Finance & Accounting 21 Europe
Accounting, Auditing & Accountability 20 Europe
Journal Business Strategy & the Environment 20 Europe
International Journal of Physical Distribution & Logistics Management 20 Europe
McKinsey Quarterly 20 USA
Academy of Management Learning & Education 19 USA
Electronic Markets 19 Europe
European Journal of Work and Organizational Psychology 19 Europe
Human Resource Management Journal 19 Europe
Journal of Industrial Ecology 19 USA
Journal of International Development 19 Europe
Management Accounting Research 19 Europe
Economics of Innovation and New Technology 18 Europe
German Economic Review 18 Europe
Industry and Innovation 18 Europe
International Business Review 18 Europe
Applied Financial Economics 17 Europe
Critical Perspectives on Accounting 17 Europe
Electronic Commerce Research 17 USA
Information Technology and People 17 Europe
International Journal of Retail & Distribution Management 17 Europe
Journal of Business Logistics 17 USA
Journal of Consumer Marketing 17 Europe
Journal of Financial Research 17 USA
Journal of Services Marketing 17 Europe
Journal of Supply Chain Management 17 USA
Managing Service Quality 17 Europe
Asia-Pacific Journal of Management 16 Asia
Business & Society 16 USA
European Accounting Review 16 Europe
Human Resource Management Review 16 Europe
Information and Organization 16 Europe
International Journal of Quality & Reliability Management 16 Europe
Journal of Business & Industrial Marketing 16 Europe
Journal of Computational Finance 16 USA
Journal of International Management 16 Europe
Journal of Public Economic Theory 16 USA
Journal of Travel Research 16 USA
Knowledge and Process Management 16 Europe
Management Decision 16 Europe
National Institute Economic Review 16 Europe

Journals with a high h-index that are not ISI listed occur in all disciplines, but are more frequent in the sub-disciplines identified above as having low ISI coverage. However, the single most distinguishing shared characteristic of these journals seems to be that they are published from Europe (usually by Blackwell, Elsevier, or Emerald) and generally have a European editor and a large proportion of non-US academics on the editorial board. Overall, nearly 75% of the non-ISI indexed journals with a high h-index are European journals.

I do not wish to imply that the ISI selection process is biased against non-US journals. Without knowledge of the actual number of European versus US journals submitted for possible inclusion in the database, it is impossible to assess this. European editors might display a self-selection bias and not submit their journals for ISI listing. Of course one reason for this could be the perceived bias against non-US journals. A similar debate is raging with regard to the representation of non-US authors in US-based journals.