Quality and Impact of Academic Research

Research program 6: Editors & editorial board diversity, comparing research output across countries, Google Scholar and bibliometrics for the Social Sciences

Anne-Wil Harzing - (updated Tue 22 Oct 2024 09:13)

This programme grew from an early critical evaluation of received wisdom in The persistent myth of high expatriate failure rates and a later paper in Journal of Organizational Behavior, 2002, which placed the myth in the wider context of how referencing errors undermine our academic credibility. Blogposts summarzing research in this program include:

A lay summary and practical advice for academics in this field is provided in a 2020 book chapter Everything you always wanted to know about impact...

Editors & editorial board diversity

In pursuing research in this area, I have also become interested in critically assessing the impact of academic research as well as the role of editors and editorial boards in the publishing process. First, with Isabel Metz, I studied determinants of editorial board diversity.

Based on data for 60 journals, covering 16,000 editorial board members and nearly 10,000 articles, a paper on gender diversity in editorial boards was published in the Academy of Management Learning and Education, whilst an update with 2009 data analyses the development of female editorial board memberships over time for five management fields, journals of four different ranks, and two geographic regions.

Two papers on geographic diversity were published in Management International Review (2013) and European Journal of International Management (2012). A final paper on the impact of editorial standing on gender diversity was published in British Journal of Management (2016). A related paper in the Dutch professional magazine Economisch Statistische Berichten summarizes the research on gender bias and meritocracy and asks Why are there so few female Economics professors?

Comparing research output across countries

Second, I have developed an interest in comparisons of research output across countries. In this context, my  article on the publication patterns of Australian academics (High Volume, Low Impact?) has generated considerable attention.

A 2014 paper with Axèle Giroud develops this theme by applying theories from International Economics/ Business to explain how nations compete in terms of academic disciplines, and what determinants explain why countries may have a specific advantage in one discipline over another.

Assessment of citation-based impact metrics

A separate project under this programme is a large-scale assessment of citation-based impact metrics.

Several short commentaries look at measuring impact for new journals and the potential arbitrary decisions in the creation of rankings, while an article-length contribution with Nancy Adler discusses the sense and nonsense of academic rankings and won the AMLE best paper award for 2009 and t he AMLE decade award in 2019. A second contribution with Nancy Adler looks at how predatory open-access journals interact with historic assessment systems leading to disruptive distortions.

Google Scholar, Microsoft Academic & bibliometrics for the Social Sciences

Two papers in Scientometrics document how the Social Sciences are disadvantaged in traditional bibliometric analyses. The first paper analyses the shortcomings of ISI Web of Knowledge document categories and argues they show a profound misunderstanding of the Social Sciences. The second paper compares citation scores for Nobel prize winners in ISI and Google Scholar and argues that Google Scholar might provide a less biased comparison across disciplines than the Web of Science. They were two of the five most cited papers out of the more than 250 articles published in the journal in 2013.

Another two papers published in Scientometrics in 2014 provide a comprehensive test of the stability and comprehensiveness of Google Scholar coverage and propose a new metric (hIa: individual annual h-index) to accommodate disciplinary and career length differences. A 2015 paper in Scientometrics presents two alternative rankings of Dutch academics in Economics & Business that are based on citations in Google Scholar rather than publications in ISI listed journals and shows this leads to a more inclusive ranking.

A second 2015 Scientometrics paper critically assesses the problem of homonyms in Thomson Reuters Essential Science Indicators. A 2016 paper in Scientometrics compares four research metrics across Google Scholar, Scopus and Web of Science and shows that both the data source and the specific metrics used change the conclusions that can be drawn from cross-disciplinary comparisons.

My three most recent papers in Scientometrics analyse the reincarnation of Microsoft Academic Search for an individual academic, for a larger sample and longitudinally and finds it to presents an excellent alternative for citation analysis, whereas my latest Scientometrics finds two new data sources – Crossref and Dimensions – also have excellent coverage. A provocative white paper and conference paper suggests the REF could be run on a rainy Sunday afternoon using Microsoft Academic affiliation searches. My 2020 book chapter summarised my work on academic impact in an accessible format.

Aug 2022:

Only £5.95...		 Nov 2022:

Only £5.95...		 Feb 2023:

Only £5.95...		 May 2023:

Only £5.95...		 August 2023:

Only £9.99...

Crafting your career in academia

In August 2022 I launched my latest initiative: a new book series Crafting your career in academia. This series is a collection of books dealing with various aspects of crafting your career in academia. It is based on my popular website Harzing.com and contains edited and curated collections of my white papers, blogposts, and presentations, as well as newly written texts. Five books are currently available, with further volumes to be published in 2024.

Publications

Online papers - Full list of publications

  1. Harzing, A.W. (forthcoming): Everything you always wanted to know about impact..., in: Clark, T., & Ketchen D. (Eds.). How to get published in the best management journals. Edward Elgar Publishing, 3rd edition. Cheltenham: Edward Elgar Publishing, pp.. Available online - Publisher's version (TBP) - Related blog post (TBP)
  2. Harzing, A.W. (2024) SAGEPolicyProfiles: a treasure-trove for discovering policy impact, www.harzing.com white paper.
  3. Harzing, A.W. (2024) Open Syllabus: a treasure-trove for research and teaching, www.harzing.com white paper.
  4. Kim, H.; Harzing, A.W. (2024) Applying IB knowledge to IB researchers: The academic capability building and knowledge creation process, AIB Insights, vol. 24, no. 1, Available online... - Publisher's version (free access) - Related blog post
  5. Harzing, A.W. (2024) The individual annualised h-index: an ecologically rational heuristic?, www.harzing.com white paper.
  6. Harzing, A.W. (2024) Doing a literature review: an 8-step process, www.harzing.com white paper.
  7. Harzing, A.W. (2023) Highly cited academics in Business & Management over the years, www.harzing.com white paper.
  8. Harzing, A.W. (2023) The art of academic writing, www.harzing.com white paper.
  9. Harzing, A.W. (2023) Using the Publish or Perish software: Crafting your career in academia, Published by Tarma Software Research Ltd, London, United Kingdom. More about this book...
  10. Harzing, A.W. (2023) Reflections on staff development, www.harzing.com white paper.
  11. Harzing, A.W. (2023) Measuring and Improving Research Impact: Crafting your career in academia, Published by Tarma Software Research Ltd, London, United Kingdom. More about this book...
  12. Harzing, A.W. (2023) Creating social media profiles: Crafting your career in academia, Published by Tarma Software Research Ltd, London, United Kingdom. More about this book...
  13. Harzing, A.W. (2022) Research Impact 101, www.harzing.com white paper.
  14. Harzing, A.W. (2022) Publishing in academic journals: Crafting your career in academia, Published by Tarma Software Research Ltd, London, United Kingdom. More about this book...
  15. Harzing, A.W. (2022) Writing effective promotion applications: Crafting your career in academia, Published by Tarma Software Research Ltd, London, United Kingdom. More about this book...
  16. Harzing, A.W. (2020): Everything you always wanted to know about impact..., in: Clark, T.; Wright, M.; Ketchen, D. How to get published in the best management journals, 2nd edition. Cheltenham: Edward Elgar Publishing, pp. 127-141. Available online... - Related blog post
  17. Harzing, A.W. (2019) Two new kids on the block: How do Crossref and Dimensions compare with Google Scholar, Microsoft Academic, Scopus and the Web of Science?, Scientometrics, in press. Available online... - Publisher's version (read for free) - Related blog post
  18. Harzing, A.W.; Vinkenburg, C.; van Engen, M. (2018): How to make career advancement in Economics more inclusive, Economisch Statistische Berichten, vol. 103, issue 4767s, 1 November, pp. 42-48. Publisher's version (free access) - Related blog post.
  19. Harzing, A.W. (2018) Running the REF on a rainy Sunday afternoon: Can we exchange peer review for metrics?, paper presented at the 23rd International Conference on Science and Technology Indicators (STI 2018), 12-14 September, Leiden, the Netherlands. Available online... - Presentation slides -  Related blog post
  20. Harzing, A.W. (2017) The mystery of the phantom reference, www.harzing.com white paper.
  21. Harzing, A.W. (2017) Running the REF on a rainy Sunday afternoon: Do metrics match peer review?, www.harzing.com white paper.
  22. Harzing, A.W.; Alakangas, S. (2017) Microsoft Academic is one year old: the Phoenix is ready to leave the nest, Scientometrics, vol. 112, no. 3, pp. 1887-1894. Available online... - Publisher's version (read for free)
  23. Harzing, A.W. (2017) The four C's of getting cited, www.harzing.com white paper.
  24. Harzing, A.W.; Alakangas, S. (2017) Microsoft Academic: Is the Phoenix getting wings?, Scientometrics, vol. 110, no. 1, pp. 371-383. Available online... - Publisher's version (read for free) - Press coverage in Scientific American and Nature
  25. Martin-Martin, A.; Orduna-Malea, E; Harzing, A.W.; Delgado López-Cózar, E. (2017) Can we use Google Scholar to identify highly-cited documents?,  Journal of Informetrics, vol. 11, no. 1, pp. 152-163. Available online... - Publisher's version
  26. Harzing, A.W. (2016) What, who or where? Rejoinder to identifying research topic development in Business and Management education research using legitimation code theory, Journal of Management Education, vol. 40, no. 6, pp. 726-731. Available online...  - Publisher's version
  27. Metz, I.; Harzing, A.W.; Zyphur, M. (2016) Of journal editors and editorial boards: Who are the trailblazers in increasing editorial board gender equality?, British Journal of Management, vol. 27, no. 4, pp. 712-726. [Also incorporated into a virtual issue on Gender in Management Research] Available online... - Publisher's version
  28. Harzing, A.W. (2016) The Publish or Perish tutorial: 80 easy tips to get the best out of the Publish or Perish software, London: Tarma Software Research. US$14.95 - Available from Amazon Amazon as a black & white paperback and Kindle. More about this book...
  29. Harzing, A.W. (2016) Why replication studies are essential: learning from failure and success, Cross Cultural & Strategic Management, vol. 23, no. 4, pp. 563-568. Available online... - Publisher's version
  30. Harzing, A.W. (2016) Microsoft Academic (Search): a Phoenix arisen from the ashes?, Scientometrics, vol. 108, no. 3, pp. 1637-1647. Available online... - Publisher's version (read for free)
  31. Harzing, A.W. (2016) Language and thinking in management and OS field: Main findings and challenges, towards a European multilingual field, panel presentation at the 15th European Academy of Management annual meeting, June 1-4, Paris, France.
  32. Harzing, A.W. (2016) Orchestrating the future of management education and research in Europe, panel presentation at the 15th European Academy of Management annual meeting, June 1-4, Paris, France.
  33. Harzing, A.W.; Adler N.J. (2016) Disseminating knowledge: From potential to reality – New open-access journals collide with convention, Academy of Management Learning & Education, vol 15, no. 1, pp. 140-156. Available online... - Publisher's version
  34. Harzing, A.W.; Alakangas, S. (2016) Google Scholar, Scopus and the Web of Science: A longitudinal and cross-disciplinary comparison, Scientometrics, vol. 106, no. 2, pp. 787-804. Available online... - Publisher's version (read for free) - Presentation slides - Video presentation of this article - ESI top 1% most Highly Cited Paper - ESI hot paper
  35. Harzing, A.W. (2015) From h-index to hIa: The ins and outs of research metrics, www.harzing.com white paper.
  36. Harzing, A.W. (2015) Health warning: Might contain multiple personalities. The problem of homonyms in Thomson Reuters Essential Science Indicators, Scientometrics, vol. 105, no. 3, pp. 2259-2270. Available online... - Publisher's version (read for free) [Press coverage in The Times and the Times Higher Education].
  37. Harzing, A.W.; Alakangas, S. (2015) Citation metrics versus peer review: a longitudinal comparison of Google Scholar, Scopus and Web of Science citation data across disciplines, symposium contribution at the 72nd Academy of Management annual meeting, August 7-11, in Vancouver, Canada.
  38. Harzing, A.W.; Mijnhardt, W. (2015) Proof over promise: Towards a more inclusive ranking of Dutch academics in Economics & Business, Scientometrics, vol. 102, no. 1, pp. 727-749. Available online... - Publisher's version (read for free)
  39. Harzing, A.W.; Alakangas, S.; Adams, D. (2014) hIa: An individual annual h-index to accommodate disciplinary and career length differences, Scientometrics, vol. 99, no. 3, pp. 811-821. Available online... - Publisher's version (read for free)
  40. Harzing, A.W. (2014) A longitudinal study of Google Scholar coverage between 2012 and 2013, Scientometrics, vol. 98, no. 1, pp. 565-575. Available online... - Publisher's version (read for free) - ESI top 1% most Highly Cited Paper
  41. Harzing, A.W.; Giroud, A.L.A. (2014) The competitive advantage of nations: An application to academia, Journal of Informetrics, vol. 8, no. 1, pp. 29-42. Available online...
  42. Harzing, A.W.; Metz, I. (2013) Practicing what we preach: The geographic diversity of editorial boards, Management International Review, vol. 53, no. 2, pp. 169-187. Available online... - Publisher's version (read for free)
  43. Harzing, A.W. (2013) A preliminary test of Google Scholar as a source for citation data: A longitudinal study of Nobel Prize winners, Scientometrics, vol. 93, no. 3, pp. 1057-1075. Available online... - Publisher's version (read for free) - ESI top 1% most Highly Cited Paper
  44. Harzing, A.W. (2013) Document categories in the ISI Web of Knowledge: Misunderstanding the Social Sciences?, Scientometrics, vol. 93, no. 1, pp. 23-34. Available online... - Publisher's version (read for free)
  45. Harzing, A.W.; Metz, I. (2012) Explaining geographic diversity of editorial boards: the role of conference participation and English language skills, European Journal of International Management, vol. 6, no. 6, pp. 697-715. Available online...
  46. Metz, I.; Harzing, A.W. (2012) An update of gender diversity in editorial boards: A longitudinal study of management journals, Personnel Review, vol. 41, no. 3, pp. 283-300. Available online...
  47. Harzing, A.W. (2012) How to become an author of ESI Highly Cited Papers?, www. harzing.com white paper. See also Scams rock academic publishing.
  48. Harzing, A.W. (2011) The Publish or Perish Book, Part 1: A guide to the software, Melbourne: Tarma Software Research.
  49. Harzing, A.W. (2011) The Publish or Perish Book, Part 2: Citation analysis for academics and administrators, Melbourne: Tarma Software Research.
  50. Harzing, A.W. (2011) The Publish or Perish Book, Part 3: Doing bibliometric research with Google Scholar, Melbourne: Tarma Software Research.
  51. Harzing, A.W.; Metz, I. (2011) Gender and geographical diversity in the editorial board of the Journal of International Business Studies, AIB Insights, vol. 11, no. 3, pp. 3-7.
  52. Harzing, A.W.; Metz, I. (2011) Gender and geographical diversity in the editorial board of the Journal of International Business Studies, www.harzing.com white paper.
  53. Harzing, A.W. (2010) The Publish or Perish Book: Your guide to effective and responsible citation analysis, Melbourne: Tarma Software Research. More about this book...
  54. Metz, I.; Harzing, A.W. (2010) Practicing what we preach: The geographic diversity of editorial boards, conference proceedings of The International Conference on Economics, Business Management and Marketing, 28-30 June, 2010, Paris, France.
  55. Metz, I.; Harzing, A.W. (2010) Gender diversity in editorial boards of management journals: An update, Proceedings of Annual London Business Research Conference, 12-14 July 2010, Imperial College, London UK.
  56. Harzing, A.W.; Metz, I. (2010) Geographic diversity in editorial boards: Has the world become smaller?, conference proceedings of the 11th IHRM conference, 9th-12th June 2010, Birmingham, UK.
  57. Harzing, A.W. (2010) Working with ISI data: Beware of Categorisation Problems, www.harzing.com white paper.
  58. Harzing, A.W. (2010) Citation analysis across disciplines: The Impact of different data sources and citation metrics, www.harzing.com white paper.
  59. Metz, I.; Harzing, A.W. (2009) Gender diversity in editorial boards of management journals, The Academy of Management Learning & Education, vol. 8, no. 4, pp. 540-557. Available online...
  60. Adler, N.; Harzing, A.W. (2009) When Knowledge Wins: Transcending the sense and nonsense of academic rankings, The Academy of Management Learning & Education, vol. 8, no. 1, pp. 72-95. Available online... - Publisher's version - Related blog post - Winner of the 2009 AMLE Outstanding article of the year award and the 2019 AMLE Decade Award, free download courtesy of AoM  - ESI top 1% most Highly Cited Paper
  61. Harzing, A.W.; Wal, R. van der (2009) A Google Scholar h-index for Journals: An alternative metric to measure journal impact in Economics & Business?, Journal of the American Society for Information Science and Technology, vol. 60, no. 1, pp 41-46. Available online... - ESI top 1% most Highly Cited Paper
  62. Harzing, A.W.; Wal, R. van der (2008) A Google Scholar h-index for Journals: A better metric to measure journal impact in Economics & Business?, paper presented at the 2008 Academy of Management annual meeting, August 8-13, 2008 - Anaheim, California. Available online...
  63. Harzing, A.W. (2008) Comparing the Google Scholar H-index with the ISI Journal Impact Factor, www.harzing.com white paper.
  64. Harzing, A.W. (2008) Arbitrary decisions in ranking studies: A commentary on Xu, Yalcinkaya, and Seggie (2008), Asia Pacific Journal of Management, vol. 25, no. 4, pp. 685-689. Available online... - Publisher's version (read for free)
  65. Harzing, A.W.; Wal, R. van der (2008) Google Scholar as a new source for citation analysis?, Ethics in Science and Environmental Politics, vol. 8, no. 1, pp. 61-73. Available online... - Publisher's version (free access!)
  66. Harzing, A.W. (2008) On becoming a high impact journal in International Business and Management, European Journal of International Management, vol. 2, no. 2, pp. 115-118. Available online...
  67. Harzing, A.W. (2007) Reflections on norms for the h-index and related indices, www.harzing.com white paper.
  68. Harzing, A.W. (2007) Google Scholar as a new data source for citation analysis, www.harzing.com white paper.
  69. Mingers, J.; Harzing, A.W. (2007) Ranking journals in Business and Management: A statistical analysis of the Harzing Dataset, European Journal of Information Systems, vol. 16, no. 4, pp. 303-316. Available online... - Publisher's version (read for free)
  70. Harzing, A.W. (2007) Reflections on the h-index, www.harzing.com white paper.
  71. Harzing, A.W.K; Metz, I. (2006) An analysis of the gender diversity in editorial boards: How do International Business journals compare? conference proceedings of the ANZIBA annual meeting, November 16-18, Wellington, New Zealand.
  72. Metz, I.; Harzing, A.W. (2006) An analysis of the gender diversity in editorial boards of Management journals, paper presented at the 66th annual meeting of the Academy of Management, Atlanta, August 11-16.
  73. Harzing, A.W. (2005) Australian research output in Economics & Business: High volume, low impact?, Australian Journal of Management, vol. 30, no. 2, pp. 183-200. Available online...
  74. Harzing, A.W. (2002) Are our referencing errors undermining our scholarship and credibility? The case of expatriate failure rates, Journal of Organizational Behavior, vol. 23, no. 1, pp. 127-148. Available online...
  75. Harzing, A.W. (1995) The persistent myth of high expatriate failure rates, International Journal of Human Resource Management, vol. 6, May, pp. 457-475. Available online...

research interests research evaluation reviewing google scholar citation metrics editorial boards

Copyright © 2024 Anne-Wil Harzing. All rights reserved. Page last modified on Tue 22 Oct 2024 09:13

Anne-Wil Harzing is Emerita Professor of International Management at Middlesex University, London. She is a Fellow of the Academy of International Business, a select group of distinguished AIB members who are recognized for their outstanding contributions to the scholarly development of the field of international business. In addition to her academic duties, she also maintains the Journal Quality List and is the driving force behind the popular Publish or Perish software program.

Anne-Wil Harzing's profile and contact details >>