Impact is impact is impact? Well, no...

Invited blogpost on SAGE Social Science Space: disambiguating the concept of impact

Anne-Wil Harzing - (updated Sat 15 Apr 2023 07:35)

[Reprint of an invited blogpost on the SAGE Social Sciences Space, 26 May 2022. This post was also reprinted on the LSE Impact of Social Sciences Blog, 27 May 2022]

It should be easy, shouldn’t it? Impact is impact is impact. Well, no… When talking about research impact, academics often have very different understandings of the concept. Worse still, they might not even be aware that different interpretations are possible.

As a result, any discussion about research impact soon descends in a Babylonian speech confusion. It also means that many academics are struggling to evidence research impact, for instance when making a case for promotion (see: Academic promotion tips (1) – What makes a successful application). In this post, I will provide a brief clarification of the concept of research impact and show how it differs by academic role.

What is impact?

The Oxford dictionary defines impact as “a marked effect or influence”. Research impact thus means that our research has affected or influenced something or someone. Unfortunately, this immediately throws up even more questions:

  • Whom has it impacted, i.e., who is the target audience?
  • How has it made an impact, i.e., what was its ultimate goal?
  • Through what means has this impact occurred, i.e., what are the primary outlets?
  • How do we know this impact has occurred, i.e., how can we measure it?

The answer to these four questions very much depends on the specific academic role we are looking at (See Table 1).

Table 1: How research impact differs by academic role

Research role

In the research role of our academic jobs, our target audience is other academics, and our ultimate goal is to progress scientific knowledge through the incorporation of our work in the scholarly body of knowledge.

We connect with other academics through academic journals, research monographs, and conferences. The preferred types of outlets are highly dependent on disciplinary norms and preferences. In the Life Sciences and Natural Sciences academic journals “rule”. In the Humanities and some of the Social Sciences books are still a popular medium (see: Own your place in the world by writing a book). In some Engineering disciplines, conference papers are a popular way of diffusing knowledge quickly.

Whether or not we have influenced other academics is typically measured by citations in academic journals, books, and conference papers. Although we know that academics are sometimes careless in their referencing (see: Are referencing errors undermining our scholarship and credibility?) and there are many reasons to cite papers (see: On the shoulders of giants? Motives to cite in management research) one would normally expect citations to signify at least some level of impact on the citing academic. Citations can also be field normalised to account for differences in citation practices across disciplines (see: From h-index to hIa: The ins and outs of research metrics).

Teaching role

As academics, we all have a tremendous impact on our students. Obviously, some of this impact will be unrelated to the research we do. However, in any good university, research feeds into the classroom and students benefit from research-informed teaching, allowing them to develop critical thinking skills.

As academics, we facilitate this directly through prescribing our own and other academics’ research as course readings. But in many cases, we will need to “translate” our research to make it more accessible for a student audience. We do so through publishing textbooks or articles in practitioner journals such as Harvard Business Review.

So how do you know whether your research is used beyond your own classroom? You can find out if your research is cited in textbooks by using Google Books. To discover whether your publications (both academic articles and textbooks or practitioner articles) are listed in teaching syllabi, Open Syllabus Explorer (see: Open Syllabus Explorer: evidencing research-based teaching?) is an incredibly useful tool.

Engagement role

So far, we have discussed the two key functions of any university: research and teaching. But there is also a third function: external engagement. This captures the impact of our research on industry, government, and the public/society at large, with the ultimate goal being to address key societal problems. It is the kind of impact incorporated in Impact Case Studies in the UK Research Excellence Framework (REF), defined as “an effect on, change or benefit to the economy, society, culture, public policy or services, health, the environment, or quality of life, beyond academia”.

Making our research accessible to an audience outside academia typically means “translating” our research for non-academic use by writing up articles for practitioner / professional journals and magazines, as well as publishing policy reports. It may also involve the use of (social) media, not just to diffuse our already published work, but also to allow continued engagement of non-academic audiences in our research. Integration of non-academic stakeholders from inception of research projects is increasingly common in the Social Sciences.

Measuring this type of impact is challenging. Recent efforts like Overton – a database that captures policy impact by tracking citations in policy documents – might help. Academics may also be able to evidence changes in government policy or legislation that are linked to their research. A more general case for impact can be made by referring to frameworks such as the United Nations Human Development Index and Sustainable Development Goals.

Horses for courses

Different universities have very different strategies for research impact, especially regarding the research and engagement roles. In the 2021 REF, the national research evaluation in the UK, the field of Business & Management showed a very different ranking of universities depending on which of the constituent criteria you focused on.

London Business School, London School of Economics, and University College London excelled in the perceived quality of their publications, but only had an average performance when looking at their societal impact. In contrast, Middlesex University, the University of Westminster – both modern universities – and SOAS (the School of Oriental and African Studies) topped the list for societal impact. However, they only had an average performance when looking at the perceived quality of their publications.

In sum: measuring impact

It is almost impossible to measure the ultimate effect for these three areas of research impact – progressing scientific knowledge, developing critical thinking, and addressing societal problems – accurately and comprehensively by using citations or other quantitative metrics alone. On the other hand, relying only on testimonials and a fully narrative approach to establishing research impact is unlikely to convince audiences either. So in evidencing impact – for instance when making a case for promotion – academics are advised to rely on a combination of metrics, qualitative evidence such as testimonials, and career narratives.

Resources

Related video

Find the resources on my website useful?

I cover all the expenses of operating my website privately. If you enjoyed this post and want to support me in maintaining my website, consider buying a copy of one of my books (see below) or supporting the Publish or Perish software.

Aug 2022:

Only £5.95...		 Nov 2022:

Only £5.95...		 Feb 2023:

Only £5.95...		 May 2023:

Only £5.95...

academia behind the scenes impact societal impact research evaluation

Copyright © 2023 Anne-Wil Harzing. All rights reserved. Page last modified on Sat 15 Apr 2023 07:35

Anne-Wil Harzing is Emerita Professor of International Management at Middlesex University, London. She is a Fellow of the Academy of International Business, a select group of distinguished AIB members who are recognized for their outstanding contributions to the scholarly development of the field of international business. In addition to her academic duties, she also maintains the Journal Quality List and is the driving force behind the popular Publish or Perish software program.

Anne-Wil Harzing's profile and contact details >>