REF, rankings & reputation

Provides an overview of my work in preparation for REF 2021, research rankings, and reputation management

Anne-Wil Harzing - Tue 7 Mar 2023 09:34 (updated Fri 19 Apr 2024 07:23)

In my position at Middlesex University, my main roles are Staff Development Lead and research mentor. However, inevitably – given my experience with research evaluation – I was drawn into preparation for the Business School’s 2021 REF submission, as well as monitoring of its research rankings and improving its research reputation. This page outlines this work and its outcomes in more detail.

REF submission

I played a major role in crafting our 2021 REF submission, taking responsibility for two of three components – output selection and the research environment statement – allowing the Deputy Dean Research and Knowledge Exchange to focus on impact case studies. This led to a stellar result; we were ranked #1 in the UK for impact.

However, Middlesex Business School also achieved a stellar result overall.

We doubled the proportion of our research ranked 4* (world-leading) to 34%, with another 49% ranked 3* (internationally excellent).
Our GPA (Quality) ranking in Business & Management improved from 51 to 37.
Our weighted Power ranking – the rating that determined our funding; with 4* counting 4 times as much as 3* – improved from 38 to 32.
On a combined GPA/Power ranking, the Business School ranked eight in the Greater London area. We ranked below LBS, City, Imperial, LSE, KCL, Queen Mary, and UCL, but above Royal Holloway, Birkbeck, SOAS, Brunel, and Goldsmith, as well as all Greater London post-92s.

This latter performance is almost identical with our performance on the US News ranking of Global Universities, where - in the London area - we rank below only LSE, LBS, Imperial, UCL, City, KCL, and Queen Mary.

My role in the 2021 REF preparation

I designed the procedures for 2021 REF output selection (around 220 publications), creating a process that maximised both efficiency and effectiveness. At the same time, it avoided the widespread dissatisfaction seen at many other institutions that relied on internal “peer review”.

Our selection process put self-selection based on the REF criteria centre stage; staff ranked their top-5 publications and made a case for their inclusion. These nominations were then matched with journal and article level indicators to create a final output pool.

In addition, I also played a major role in working with the institutional repository staff on compliance and wrote a user's guide for our REF selection system. Our resulting publication pool received a GPA of 2.87, up from 2.51 in 2014, despite doubling the number of FTEs in our submission.

I was also responsible for the research environment statement. I coordinated all necessary data collection, chased university services, conducted all analyses, wrote up most of the RES, and refined it through a process of feedback from key individuals. Our RES was ranked 3.25, an improvement from an already high score of 3.125 in 2014. It ranked #31 in the UK, above two Russell Group universities and 19 unaffiliated universities. Only three post-92 universities (MDX, MMU, and Portsmouth) had a RES GPA above 3.00.

Honouring the collegial spirit at Middlesex University, I also took the initiative to organise and chair a 2-hour Business School REF celebration event in June 2021, just after the submission date (see above). It emphasised the collective nature of our submission and celebrated the effort we had all put into creating the best possible submission.

The event featured presentations by the DD RKE Stephen Syrett, our VC Nic Beech, Impact case study authors, and myself. At a time when pandemic fatigue was running high, it was a major morale boost and our best attended School-wide event since I joined MDX. I edited recordings and made them available to all staff.

Middlesex outgrowing its post-92 reference group

MDX Business School has completely outgrown its post-92 reference group and is now ranking between Russell Group universities and other research-intensive universities. On GPA we moved from being an “above average” post-92 (2008), to being closer to other research-intensive universities than to post-92s (2014) to ranking above the average for other research-intensive universities (2021) (see graph below). Over the years, MDX B&M has improved its GPA more than all three groups.

On FTE submitted we moved from “double the average” for a post-92 (2008) to similar to other research-intensive universities (2014), to nearly mid-way between Russell Group universities and other research-intensives (2021) (see graph below). Over the years MDX B&M has increased its FTE submitted nearly as much as Russell Group universities and much more than other research-intensives and post-92s.

On weighted Power ranking we moved from ranking a bit lower than other research-intensive universities (2008) to ranking similar to them (2014), to ranking exactly mid-way between Russell Group universities and other research-intensive universities (2021) (see graph below). Our power ranking improved nearly as much as that of Russell Group universities, with the two other groups both losing ground.

Middlesex improves more than the sector average

Middlesex Business School has improved its GPA performance on all three components between 2014 and 2021 and showed an overall improvement (0.43) that was nearly 50% higher than the sector’s improvement overall (0.29).

On impact it has beaten the general trend in the sector that showed only very modest levels of improvement between 2014 and 2021. This is remarkable given that institutions now had much better knowledge of what was expected of them and the fact that many used outside services to write up/shape their Impact Case Studies.

We showed only a modest improvement on the research environment statement. This was, however, true for other post-92s and other research-intensive universities too. Only Russell Group universities showed a significant improvement on this component. Middlesex Business School started from a very high base, already ranking higher than other research-intensive universities on the environment statement in 2014.

Middlesex Business School also improved significantly on publications; it even improved (marginally) more than all three groups, evidencing both the quality of our underlying publications and the effectiveness of our selection process. As such it moved further away from other post-92s on publications and closer to other research-intensive universities.

Research rankings and reputation management

As an expert in research evaluation and the major databases used in this area (Web of Science and Scopus), it was only natural that I ended up conducting benchmarking analyses and providing briefings to senior management on the major international research rankings (Times Higher Education, QS, ARWU/Shanghai and USNews).

This includes identifying problems with these rankings, such as the inappropriate definition of “young” universities in the THE Young Universities ranking. Pleasingly – partly through my own work in staff development – Middlesex has risen in each of these rankings since 2016.

However, even when new universities such as Middlesex are performing well on objective metrics such as publications, citations, and funding, their external reputation often lags behind their objective performance.

In the 2022 US News ranking there are 39 universities in the UK that are ranked in Business & Economics. Middlesex ranks ninth in terms of citation metrics, but only 33 in terms of reputation.
In the QS ranking Middlesex ranks identically to the University of Melbourne on citation metrics, but there is a vast difference in their reputation scores. As these reputation scores count for 80% and the metrics only for 20%, Melbourne ranks #26 overall and MDX 351-400.

Hence, I also engage in reputation building, showcasing Middlesex Business School as a research institution. I have written up dozens of blogposts about events I organised at Middlesex see e.g.

Moreover, I opened up my blog to colleagues. This has led to contributions from ten MDX staff, see e.g.

I also maintain a very active social media profile (see Embracing new technologies) where I share MDX successes including our excellent REF 2021 result and promote my colleagues’ achievements. Finally, I use prominent Middlesex branding on my many external speaking engagements and YouTube videos (see below).

Find the resources on my website useful?

I cover all the expenses of operating my website privately. If you enjoyed this post and want to support me in maintaining my website, consider buying a copy of one of my books (see below) or supporting the Publish or Perish software.

Aug 2022:

Nov 2022:

Feb 2023:

May 2023:

August 2023:

Anne-Wil Harzing is Professor of International Management at Middlesex University, London and visiting professor of International Management at Tilburg University. She is a Fellow of the Academy of International Business, a select group of distinguished AIB members who are recognized for their outstanding contributions to the scholarly development of the field of international business. In addition to her academic duties, she also maintains the Journal Quality List and is the driving force behind the popular Publish or Perish software program.

Anne-Wil Harzing's profile and contact details >>