15.3.4 Numerical analysis of the divergence between JIF and h-Index
Above, I have documented a number of cases where the JIF and h-index diverged in considerable detail. Table 3 provides a summary of the 50 most prominent cases of divergence. It was constructed by standardizing both the JIF (after giving a 0 score for all journals without a JIF) and the h-index and subtracting the JIF from the h-index. Since Psychology journals differed considerably from all other journals and would make up most of the top 25 on the left-hand side, I excluded them from the analysis.
Table 3: the 50 most prominent cases of divergence between JIF and h-index
| JIF exceeds h-index (top 25) | h-index exceeds JIF (top 25) | ||
|---|---|---|---|
| ACM Computing Surveys | -6.65 | American Economic Review | 5.30 |
| SIAM review | -4.90 | Journal of Finance | 3.93 |
| Quarterly Journal of Economics | -3.81 | European Economic Review | 2.77 |
| ACM Transactions on Information Systems | -3.52 | Communications of the ACM | 2.70 |
| Human Computer Interaction | -3.46 | Review of Economics & Statistics | 2.38 |
| Journal of Economic Literature | -2.74 | Research Policy | 2.24 |
| Academy of Management Review | -2.28 | European Journal of Operational Research | 2.10 |
| Progress in Human Geography | -2.26 | Econometrica | 2.03 |
| Human Resource Management | -2.09 | Journal of Banking & Finance | 1.89 |
| Journal of Economic Geography | -2.08 | Management Science | 1.79 |
| Annual Review of Sociology | -2.03 | Journal of Public Economics | 1.78 |
| Marketing Science | -1.89 | Journal of Political Economy | 1.76 |
| MIS Quarterly | -1.89 | World Development | 1.65 |
| Journal of Economic Growth | -1.72 | Journal of Financial Economics | 1.63 |
| ACM Trans. on Softw. Eng. and Methodology | -1.68 | IEEE Trans. on Knowl. & Data Engineering | 1.61 |
| Journal of Database Management | -1.68 | Economic Journal | 1.59 |
| Journal of Marketing | -1.58 | European Journal of Political Economy | 1.54 |
| Annals of the Assoc. of American Geographers | -1.45 | International Organization | 1.50 |
| Review of Accounting Studies | -1.45 | International Jnl of Project Management | 1.46 |
| Environment & Planning D | -1.44 | American Journal of Public Health | 1.40 |
| Journal of Org. Behavior Management | -1.40 | Journal of Econometrics | 1.39 |
| Journal of Rural Studies | -1.36 | Journal of Money, Credit & Banking | 1.38 |
| American Sociological Review | -1.28 | ACM Trans. on Comp. Human Interaction | 1.38 |
| Administrative Science Quarterly | -1.26 | Journal of Knowledge Management | 1.38 |
| Structural Equation Modeling | -1.15 | Review of Economic Studies | 1.35 |
Most of these journals have been discussed in some detail above. Apart from apparent errors in the ISI JIF and one deficiency in Google Scholar coverage, the major reason for a high JIF in comparison to the h-index appears to relate to journals that publish a small number of papers and/or have highly concentrated citation papers, where the top 10 most cited articles provide the bulk of citations. As a result the mean-type the JIF presents a less accurate reflection of a journals overall impact than the h-index.
Reviewing the journal titles in the right-hand column, the single most important determinant of a high h-index in comparison to JIF seems to be the extent to which the journal publishes policy oriented papers that are highly cited in working papers and policy documents. Publishing a large number of papers and being cited in conference papers and non-ISI indexed journals provide secondary reasons for a high h-index. Overall, the h-index might therefore more suitable to measure a journals wider economic or social impact rather than its impact on an academic audience only.
As indicated above there are many journals, especially in Finance & Accounting, Marketing and General Management & Strategy that are not ISI-indexed. So how do these journals compare with journals that are ISI-indexed? As expected, journals that are ISI-indexed in general have a significantly higher h-index (23.5 versus 11.5; t = 15.002, p < 0.001).
But are there any non-ISI indexed journals with a relatively high h-index? In order to assess this I divided the journals by h-index into 4 equal groups. The quartile cut-off points for the h-index were 11, 16 and 24. Journals that ranked in the two top 50% (16 and above) in terms of h-index, but are not ISI-listed are listed in Table 4.
Table 4: Non-ISI indexed journals with a high h-index
| Journal Title | h-index | Published in: |
|---|---|---|
| European Journal of Political Economy | 28 | Europe |
| International Journal of Project Management | 27 | Europe |
| ACM Transactions on Computer Human Interaction | 26 | USA |
| Journal of Knowledge Management | 26 | Europe |
| Empirical Economics | 24 | Europe |
| Accounting Horizons | 23 | USA |
| European Management Journal | 23 | Europe |
| Journal of Empirical Finance | 23 | Europe |
| European Journal of Marketing | 22 | Europe |
| Journal of Environmental Management | 22 | Europe |
| Journal of Financial Services Research | 22 | USA |
| Review of Finance | 22 | Europe |
| European Financial Management | 21 | Europe |
| Journal of Business Finance & Accounting | 21 | Europe |
| Accounting, Auditing & Accountability | 20 | Europe |
| Journal Business Strategy & the Environment | 20 | Europe |
| International Journal of Physical Distribution & Logistics Management | 20 | Europe |
| McKinsey Quarterly | 20 | USA |
| Academy of Management Learning & Education | 19 | USA |
| Electronic Markets | 19 | Europe |
| European Journal of Work and Organizational Psychology | 19 | Europe |
| Human Resource Management Journal | 19 | Europe |
| Journal of Industrial Ecology | 19 | USA |
| Journal of International Development | 19 | Europe |
| Management Accounting Research | 19 | Europe |
| Economics of Innovation and New Technology | 18 | Europe |
| German Economic Review | 18 | Europe |
| Industry and Innovation | 18 | Europe |
| International Business Review | 18 | Europe |
| Applied Financial Economics | 17 | Europe |
| Critical Perspectives on Accounting | 17 | Europe |
| Electronic Commerce Research | 17 | USA |
| Information Technology and People | 17 | Europe |
| International Journal of Retail & Distribution Management | 17 | Europe |
| Journal of Business Logistics | 17 | USA |
| Journal of Consumer Marketing | 17 | Europe |
| Journal of Financial Research | 17 | USA |
| Journal of Services Marketing | 17 | Europe |
| Journal of Supply Chain Management | 17 | USA |
| Managing Service Quality | 17 | Europe |
| Asia-Pacific Journal of Management | 16 | Asia |
| Business & Society | 16 | USA |
| European Accounting Review | 16 | Europe |
| Human Resource Management Review | 16 | Europe |
| Information and Organization | 16 | Europe |
| International Journal of Quality & Reliability Management | 16 | Europe |
| Journal of Business & Industrial Marketing | 16 | Europe |
| Journal of Computational Finance | 16 | USA |
| Journal of International Management | 16 | Europe |
| Journal of Public Economic Theory | 16 | USA |
| Journal of Travel Research | 16 | USA |
| Knowledge and Process Management | 16 | Europe |
| Management Decision | 16 | Europe |
| National Institute Economic Review | 16 | Europe |
Journals with a high h-index that are not ISI listed occur in all disciplines, but are more frequent in the sub-disciplines identified above as having low ISI coverage. However, the single most distinguishing shared characteristic of these journals seems to be that they are published from Europe (usually by Blackwell, Elsevier, or Emerald) and generally have a European editor and a large proportion of non-US academics on the editorial board. Overall, nearly 75% of the non-ISI indexed journals with a high h-index are European journals.
I do not wish to imply that the ISI selection process is biased against non-US journals. Without knowledge of the actual number of European versus US journals submitted for possible inclusion in the database, it is impossible to assess this. European editors might display a self-selection bias and not submit their journals for ISI listing. Of course one reason for this could be the perceived bias against non-US journals. A similar debate is raging with regard to the representation of non-US authors in US-based journals.