SAGEPolicyProfiles: a treasure-trove for discovering policy impact

How to use SagePolicyProfiles to learn more about, and evidence, the policy impact of your academic research

Anne-Wil Harzing - Mon 21 Oct 2024 16:02 (updated Fri 8 Nov 2024 10:53)

© Copyright 2024 Anne-Wil Harzing. All rights reserved. First version, 22 October 2024

SAGEPolicyProfiles is a free service offered by SAGE Publications in collaboration with, and powered by, Overton. Overton tracks policy impact to help universities, think tanks, and publishers understand the reach and influence of their work. Offering advanced policy discovery tools and a wealth of data, it allows you to search by topic, institution, or author, and filter by policy area, source type, and publisher. It also features Overton Engage which allows you to search for policy engagement opportunities worldwide. Overton offers a free trial, but – not surprisingly given what is on offer – is a paid-for service.

The free SAGEPolicyProfiles is a much simpler tool. It is targeted at individual researchers who want to showcase their policy impact in promotion or tenure applications, as well as funding proposals. You can use it to view your own policy profile and can share a link of your profile with others. However, you cannot search for academics’ policy profiles, let alone peruse profiles for institutions, topics, or countries, which you can do with Overton. Even so, SAGEPolicyProfiles is a potential gamechanger in drawing academics’ attention to the importance of policy impact. In this white paper, I review all of its options as of October 2024.

How to set up your profile?

You can set up your own profile by email or through a link with Google or LinkedIn (see above). You will be asked to provide your ORCID iD (optional) and your full name as it appears on your publications (required). You will then be presented with a list of publications that SAGEPolicyProfiles thinks are yours. I suggest checking them carefully to remove false hits. If any of your publications are missing, you can search for them manually by entering their DOI.

Note that only publications with a DOI – such as journal articles, or individual book chapters that have been registered for a DOI – can be found. Hence, in my case I wouldn’t be able to assess the policy impact of the Publish or Perish software, or that of my many white papers and books without DOIs. This is why it is so important – especially if you are just starting out in academia – to ensure you get a DOI for all your publications. To do this for datasets have a look at these suggestions. For other publications, please consult your research manager or university librarian for the best option as your university might have solutions in place.

Before you start exploring your profile, I suggest you check your account settings and set up alerts to be informed by email of new policy citations and mentions. You can choose weekly or monthly alerts, so even if you are cited often, it will not lead to email overload. You can also choose to receive policy insights and engagement advice, and be the first to know when new app features become available. Below is an example of an email alert I received, which I found very helpful.

What does SAGEPolicyProfiles offer?

SAGEPolicyProfiles provides an intuitive and simple we. interface with three key features: the number of policy citations, citations by location, and citations over time. You can also dive deeper into each of the policy citations and see which of your publications they cite. Vice-versa, you look at the full list of your published research and see if and where it is cited in policy documents. Below I illustrate each of these features using my own profile. You can jump to individual sections using the Table of Contents below.

Table of contents

Policy citations overview

The primary result that SAGEPolicyProfiles  shows on your profile is the number of citations that your publications have generated in policy documents. As an academic who – until fairly recently – has had a relatively “academic” profile, i.e. most of my interactions were within academia, I expected to see a big fat zero in terms of policy impact. To my considerable surprise there were 141 citations to my work across 117 policy documents and even 9 specific mentions of my name in policy documents.

Other academics have had similarly positive surprises and have discovered unexpected policy citations to their work. So, I would highly encourage you to check it out and create your own profile. It is wonderful way to evidence that your research has had an impact outside the world of academia. However, even if – like me – you no longer need this evidence for tenure or promotion applications, performance appraisals, or grant applications, it is still reassuring to learn that your research is having a (positive) impact on society.

Policy citations by location

In addition to the actual number of policy citations SAGEPolicyProfiles also shows where and when these citations happened by means of a map and timeline. When you are logged in to your profile you can see both at the same time. The public version of your profile allows you to “toggle” between them. The default is an attractive looking map.

This map shows you in which countries your work is cited, with the size of the circle reflecting the number of citations. As you can see above, my work has been cited in policy documents in North America, South America, South-Africa, the United Arab Emirates, Indonesia, and Australia, as well as many Western European countries. The big circle in the middle of the Atlantic Ocean reflects 50 policy citations by IGOs (International Governmental Organizations).

As usual in these types of visualizations, Western Europe looks rather crowded as it combines many small countries with a substantial number of policy citations. You cannot zoom in, so it is a bit fiddly to get accurate data for each country, but it is possible. It is worthwhile to do so, as the size of the circles doesn’t fully reflect the differences between countries. For instance, in South America the difference between Brazil (8 citations) and Chile and Peru (1 citation each) is much larger than one might conclude from the modest difference in the size of the circles.

Analysing the citations by country also gives you an indication of your international reach. I wasn’t surprised by the large number of citations in the UK (12), the country where I have worked for the largest part of my career, and the USA (8). The large number of citations by IGOs (50) and to a lesser extent the EU (7) were also to be expected given the large number of policy documents they produce. However, I was surprised by the large number of citations in Germany (11) and Spain (10). You can analyse citations in specific countries in more detail by exporting all your policy citations. We will discuss this in one of the later sections.

Policy citations over time

The map on your Policy Profile page can be toggled to show a timeline instead. Helpfully, this shows both your publications and policy citations over time. It correctly shows my publications starting slowly from 1995 onwards. If you are surprised about my rather large number of publications in some years, rest assured that I don’t typically publish more than two or three articles a year. In my field – Business & Management – I do not think it is possibly to do high-quality research whilst pursuing a “high-volume” publication strategy.

However, the publications covered in SAGEPolicyProfiles report everything that has a DOI. This includes some conference proceedings (e.g. Academy of Management best paper proceedings), letters to the editor, editorials, and also – in some cases – individual book chapters of an authored book. For instance my co-authored book on Managing expatriates in China is represented with no less than nine separate chapters in 2017. Hence, unless you only publish full-length journal articles, I would not recommend using this graph to represent your publication “productivity”. However, it is useful as a “quick and dirty” visualization of sustained publication output over the years.

The line visualizing policy citations indicates that most of my policy impact occurred in the last 10 years. Reviewing my policy citations year by year allowed me to establish that only 41(i.e. less than 30%) occurred before 2014. This may reflect the lack of older policy documents in the Overton database. However, it likely also reflects the fact that citations (whether academic or policy) simply take time to materialise (see also: Making your case for impact if you have few citations).

Dive deeper into the policy citations

By clicking on “Policy Citations” in the blue bar on the left-hand side of your profile you can review each of the policy documents that cite your work and see the specific academic articles that are cited in them. You can sort this list by newest/oldest first, and (reversed) alphabetically. You can also filter both by publication date and cited date, making it easy to create custom reports. SAGEPolicyProfiles even provides you with a link to directly take you to the page where your work is cited. How cool is that? I enjoyed reviewing the large variety of policy documents in which my work was cited. Below are some examples.

You can download a file of all citations in two formats. First, a CSV format, allowing you to do further analyses in a spreadsheet, such as analysing citations by country or by type of citation (government, think-thank, IGO). Second, a Power-Point format that presents every citation on a single slide in a visually attractive format. This could be very useful if you wanted to use it in one of your academic presentations.

Name mentions

I also reviewed my nine names mentions. This is where your full name is mentioned explicitly in the document, rather than your last name only, as will typically be the case for a reference to your academic work. This might be an indication that your body of work or your general standing in an area has been particularly relevant to policy. So, it is useful to review these name mentions carefully.

In my case, however, it turned that 7 of the 9 name mentions were in blogposts on the LSE Impact blog. Four were blogposts written by me (see below for two examples). Interestingly, not all of my seven posts on the LSE blog were included in SAGEPolicyProfiles and – somewhat surprisingly – my post on impact is one of the ones that is not listed.

Three other name mentions were LSE Impact blogposts by other authors that referred to the free Publish or Perish software. The remaining two documents also referred to the software and discussed how it can assist academics in demonstrating their citation impact. Hence, I would not consider these to be exemplars of policy citations.

On the other hand, there are more than twenty name mentions for my name on the LSE Impact blog. Hence, I would argue that at the very least this indicates that my academic work has led to some discussion outside academic journals. Moreover, some of these posts might have contributed to policy changes in universities. Hence, on balance I think counting some of these mentions as policy citations may well be appropriate.

Review your published work for policy citations

This is the mirror image of the previous section. With this option, SAGEPolicyProfiles provides a list of your publications and indicates which policy documents, if any, they are cited in. This is very useful to get a feel for which of your academic articles are more relevant to policy. You can sort this list by most/least cited, newest/oldest first, as well as alphabetically. You can also filter by publication date and cited date, making it easy to create custom reports.

Reviewing this list is likely to be a sobering experience for most academics, even if their total number of policy citations was quite high. Typically, policy citations are linked to a relatively small sub-set of one’s publications. SAGEPolicyProfiles indicates that, overall, only 5% of published work attracts policy citations.

In my case, about a third of my articles had one or more policy citations. However, this varied substantially by age. Whereas only 10-15% of my articles published in the last 5 years displayed policy impact, this was true for 40-45% of the older articles. As indicated above, this may reflect the fact that citations in the Social Sciences (whether academic or policy) take time to materialise (see also: Making your case for impact if you have few citations). There were three specific areas of my research that generated significant policy citations.

International Management Research

There were more than 50 policy citations to my academic articles on topics such as country-of-origin effects in MNCs, HQ and subsidiary typologies, knowledge flows within MNCs, cultural distance, and expatriation.

My most highly cited articles dealt with country-of origin effects on internationalization strategies and corporate control in MNEs (see: Country of origin matters for multinationals too (11 citations), Testing key IB typologies: Bartlett & Ghoshal and Gupta & Govindarajan (7 and 5 citations) and my work on the role of expatriates in coordinating foreign subsidiaries (see Of bears, bumble-bees and spiders & who's in charge?) (7 citations). Language differences and the use of corporate languages within MNCs (see: Language barriers in multinational companies) constituted a distinct area within my work with around a dozen policy citations in total, spread fairly equally among five articles on the topic.

Finally, although it received only one policy citation, I was very pleased to see my 2004 article – which was little more than a curiosity project – being cited in a document on improving participation in the Erasmus programme.

Bibliometric research

There were nearly 50 policy citations to my bibliometric research into different data sources and research metrics, as well as my research into university rankings.

Highly cited articles in this category were my 2008 article on Google Scholar as a new source for citation analysis (10 citations), my 2016 comparison of Google Scholar with Scopus and the Web of Science (15 citations), my 2017 trilogy on Microsoft Academic (7 citations in all), and my critique of international research rankings When Knowledge Wins (4 citations). For a summary of these and other articles in this area (such as the image below) see my blogpost: To rank or not to rank.

International mail surveys

The next cluster was formed by my research on international mail surveys, and in particular differences in response rates across countries, differences between the use of ranking and rating as scale anchors, the effect of the language of the questionnaire on how people responded, as well as cross-national differences in response styles.

This cluster of nine publications attracted around 30 policy citations. My most-cited article here was on responses styles in cross-national surveys (see: What if fully agree doesn't mean the same thing across cultures?) with 16 citations, but several other articles also attracted 3-6 citations. The article below provides a summary of all my work in this area.

You can download a file with all of these citations in two formats. First, a CSV format, allowing you to do further analyses in a spreadsheet, such as analysing citations by country or by type of citation (government, think-thank, IGO). Second, a Power-Point format that presents every article on a single slide in a visually attractive format, very useful if you want to use it in one of your academic presentations.

Knowing which areas of your academic work generate policy impact might be a good way to assess which elements of your research are most likely to resonate in the policy arena. This may help you tailor your external engagement efforts and/or funding applications.

Want to know more about research impact?

If you would like to learn more about research impact, SAGEPolicyProfiles also links to SAGE’s Social Science Space (S3) Network, a treasure trove of materials related to research impact. From their website:

“Social Science Space brings social scientists together to explore, share and shape the big issues in social science, from funding to impact. This online social network features blogs with the most current thinking from key players in social science, a forum for discussions, a resource center with free videos, reports and slides that support these discussions, as well as funding and job opportunity notices.”

“We believe Social Science Space can bring together key players who are engaging in higher level debates about social science issues, from funding to impact including chief funders, societies, think tanks, policy makers and government as well as researchers. By bringing these key players together and promoting the work that they are doing, we aim to facilitate discussion about the value of social sciences with a wider community.”

S3 includes an very interesting exchange “On Measuring Social Science Impact”, in which six scholars (including myself) comment on an essay by Ziyad Marar, President, Global Publishing at SAGE Publishing published in the new Agora [Aγoρά] series in Organization Studies. My own commentary (see Impact is impact is impact? Well, no... ) formed the basis for my – much longer – white paper Research impact 101.

Conclusion

SAGEPolicyProfiles provides individual academics with a quick and easy route to find out whether their work is cited in policy documents. It is a major step forward in bringing home the importance of policy impact to a wider academic audience. As such, I highly recommend you create your own policy profile and study the information it provides.

However, we should be careful not to get seduced by the fact that we can now put a number on policy impact. Citations in policy documents do not necessarily mean that your work has had an actual and direct impact on policy. For that a much more detailed, and most likely qualitative, analysis would be needed. That said, citations in policy documents are a “pathway to impact”, a route to possible policy impact.

In sum, SAGEPolicyProfiles is an excellent service that allows individual academics to assess which parts of their research are most likely to resonate in the policy arena. This may motivate you to shape your research in this direction so that its policy impact can be realised, whilst maintaining the rigour and independence of academic research.