To rank or not to rank
Very shortly after establishing my website in 1999, I posted the first edition of my Journal Quality List, now in its 60th edition. Over the years I have also published a large number of articles dealing in one way or another with rankings. You would thus be forgiven for thinking that I am a big advocate of journal and university rankings. It might surprise you I am not. Just like many of you I would prefer to live in an academic world where rankings and public relations didn't dominate academic substance. However, I recognize that rankings are not going away and that even if we did away with formal rankings, most academics would still create rankings in their own minds.
So my main motivation has been to ensure that if rankings do persist, we at least make sure they are as inclusive as possible and understand that every ranking has its own biases. My first publication in this field thus presented an integrated ranking of the then 16 rankings included in the Journal Quality list. One of its motivations was to help the asessors in the Business & Management panel of the 2008 British Research Assessment Excercise in their assessment of over 10,000 publications by providing a ranking based on a variety of perspectives. Remember, this was before the days of the "one list to rule them all", aka the ABS journal ranking.
- Mingers, J.; Harzing, A.W. (2007) Ranking journals in Business and Management: A statistical analysis of the Harzing Dataset, European Journal of Information Systems, vol. 16, no. 4, pp. 303-316. Available online... - Publisher's version (read for free)
I then moved on to exploring new data sources that provide a fairer assessment of journal impact in the Social Sciences and Humanities, using Google Scholar as an alternative to Scopus and the Web of Science. For a more general introduction into the coverage of Google Scholar and its drawbacks, see here, here, here, here, here and here.
- Harzing, A.W.; Wal, R. van der (2008) Google Scholar as a new source for citation analysis?, Ethics in Science and Environmental Politics, vol. 8, no. 1, pp. 61-73. Available online... - Publisher's version (free access!)
- Harzing, A.W. (2008) On becoming a high impact journal in International Business and Management, European Journal of International Management, 2(2): 115-118. Available online... - Publisher’s version
- Harzing, A.W.; Wal, R. van der (2009) A Google Scholar h-index for journals: An alternative metric to measure journal impact in Economics & Business?, Journal of the American Society for Information Science and Technology, 60(1, pp 41-46. Available online... - Publisher’s version - White paper.
Subsequently, my attention was drawn towards university rankings. A short commentary on the - in my view - arbitrary decisions in one particular ranking of universities study grew into a more provocative piece, co-authored with the wonderful Nancy Adler. We highlight the problematic nature of academic ranking systems and question if such assessments are drawing scholarship away from its fundamental purpose. Ironically, this paper is now my top-ranked publication in terms of citations per year.
- Harzing, A.W. (2008) Arbitrary decisions in ranking studies: A commentary on Xu, Yalcinkaya, and Seggie (2008), Asia Pacific Journal of Management, vol. 25, no. 4, pp. 685-689. Available online... - Publisher's version (read for free)
- Adler, N.; Harzing, A.W. (2009) When Knowledge Wins: Transcending the sense and nonsense of academic rankings, The Academy of Management Learning & Education, vol. 8, no. 1, pp. 72-95. Available online... [Winner of the 2009 AMLE Outstanding article of the year award, free download courtesy of AoM.] - ESI top 1% most Highly Cited Paper
In recent years, I have looked at how problems in citation databases and rankings might reinforce existing academic hierarchies in terms of disciplines and universities. In these articles, I also discuss how these problems might disproportionally disadvantage academics with Asian names, women, and younger academics.
- Harzing, A.W. (2013) Document categories in the ISI Web of Knowledge: Misunderstanding the Social Sciences?, Scientometrics, vol. 93, no. 1, pp. 23-34. Available online... - Publisher's version (read for free) - White paper - Blog post.
- Harzing, A.W. (2015) Health warning: Might contain multiple personalities. The problem of homonyms in Thomson Reuters Essential Science Indicators, Scientometrics, vol. 105, no. 3, pp. 2259-2270. Available online... - Publisher's version (read for free) - [Press coverage in The Times and the Times Higher Education] - Blog post.
- Harzing, A.W.; Mijnhardt, W. (2015) Proof over promise: Towards a more inclusive ranking of Dutch academics in Economics & Business, Scientometrics, vol. 102, no. 1, pp. 727-749. Available online... - Publisher's version (read for free) - Blog post.
My interest in publishing articles in this field has waned a little. There is only so much time in the day and life is short. However, I continue to blog on this and related topics (see below) and give presentations in a variety of fora.
Related blog posts
- Publish or Perish: Realising Google Scholar's potential to democratise citation analysis
- Bibliometrics in the Arts, Humanities, and Social Sciences
- Is Google Scholar flawless? Of course not!
- From h-index to hIa: The ins and outs of research metrics
- Citation analysis for the Social Sciences: metrics and data-sources
- Working with ISI data: Beware of categorisation problems
- Sacrifice a little accuracy for a lot more comprehensive coverage
- Do Google Scholar, Scopus and the Web of Science speak your language?
- Benchmarking research performance
- Bank error in your favour? How to gain 3,000 citations in a week
- Microsoft Academic is one year old: the Phoenix is ready to leave the nest
- Why metrics can (and should?) be used in the Social Sciences
- Running the REF on a rainy Sunday afternoon: Do metrics match peer review?
Copyright © 2018 Anne-Wil Harzing. All rights reserved. Page last modified on Tue 7 Aug 2018 09:24
Anne-Wil Harzing is Professor of International Management at Middlesex University, London and visiting professor of International Management at Tilburg University. In addition to her academic duties, she also maintains the Journal Quality List and is the driving force behind the popular Publish or Perish software program.