Open Syllabus: a treasure-trove for research and teaching

How to use Open Syllabus to evidence your own teaching impact, conduct curiosity research, and search for suitable textbooks

Anne-Wil Harzing - Thu 17 Oct 2024 14:11 (updated Sun 20 Oct 2024 11:33)

© Copyright 2024 Anne-Wil Harzing. All rights reserved. First version, 17 October 2024

[Note: I first discussed this service in November 2019. You can find the original blogpost here: Open Syllabus Explorer: evidencing research-based teaching?]

Open Syllabus was officially launched in the Summer of 2019 as Open Syllabus Explorer. This white paper reports on the latest Open Syllabus service as of October 2024. You can explore this version for yourself here.

Note: This white paper reports findings from the 'full' Analytics function, using a free trial account. If you do not use the 'full' Analytics version, results will be different from those reported in this white paper as the free version caps data at 2019. Your trial account will be valid for one month, more than enough to prepare your application for tenure or promotion or do a small research project.

Table of contents

This white paper discusses Open Syllabus Analytics and Open Syllabus Galaxy. You can use the table of contents to jump to specific sections.

What is Open Syllabus?

[From their own website; more information can be found here. Note that just like my free Publish or Perish software, Open Syllabus is only as accurate as its underlying data. Publication data often suffer from a lack of author/publisher/institutional disambiguation (see also: Health warning: Might contain multiple personalities). Hence, the results should be taken as an indication only. OS estimates that 90-95% of its data are correct.]

"Open Syllabus currently has a corpus of 21 million English-language syllabi from 140 countries. We use machine learning and other techniques to extract citations, dates, fields, and other metadata from these documents. The resulting data is made available through (currently) three online tools:

• Analytics provides free and open exploration of most of the data, while reserving the most recent data and advanced capabilities for schools, publishers, and other institutional subscribers. You can try the 'full' Analytics by signing up for a free trial account.
• The Coursematcher predicts 'course equivalance' across the catalogs of hundreds of schools as a way to support the course transfer process for students and college staff. [Note: this part of the service only appears to cover North America; it is thus is less relevant to my audience and not discussed in this white paper.]
• The Co-Assignment Galaxy is a massive plot of the top million titles in the Open Syllabus dataset, grouped by how often they are assigned together. It is the closest thing available to a unified representation of the project of higher education."

What disciplines/countries/publishers does it cover?

The field with the largest representation is Business, with more than 2 million syllabi included. Mathematics, Computer Science, English Literature, Education, Engineering and Biology are all represented with around 1 million syllabi. History, Psychology, Fine Arts, Political Sciences, Medicine and Law feature around 500,000-700,000 syllabi, with a further 30-odd disciplines represented by between 120,000 and 450,000 syllabi and another 20 with fewer than 100,000 syllabi.

The countries with the largest representation are the USA (9 million), the UK (2.1 milion), Australia (1.8 million), and Canada (1.2 million). There are another 18 countries represented with more than 100,000 syllabi. In terms of publishers, Oxford University Press, Cambridge University Press, and Routledge have more than 1 million appearances, with McGraw Hill, SAGE, Wiley, and Pallgrave MacMillan reporting more than 500,000 appearances.

Open Syllabus Analytics

This section of Open Syllabus is the most extensive of the three and - as the name says - allows you to conduct a wide range of analyses, searching by title, author, country, field, school, and publisher. With the 'full' version you can also dive into individual syllabi and learning outcomes. In this white paper, I focus on using Open Syllabus Analytics to evidence the teaching impact of your research and to conduct small curiosity-driven research projects.

Evidencing teaching impact: Do my publications feature in syllabi? (1)

Many academics need to evidence that their research has had "impact". Academic impact, i.e. impact on the field of knowledge, is usually evidenced through citations (see also: Measuring and improving research impact). The free SAGE Policy Profiles are a great way to assess your policy impact. Open Syllabus, however, allows you to evidence that your research has had an impact in the classroom. Academics are usually able to make the case that their own teaching is research-informed or research-based. However, showing that your research has made it into someone else's classroom is a much more powerful way to evidence the teaching impact of your research. It really demonstrates that your research helps the next generation to understand the world around them.

To demonstrate this feature, I searched for my own name and was pleasantly surprised. No less than 36 of my academic articles appeared in syllabi, with a total of more than 400 appearances. The top-10 most listed are shown below. Although not in the top-10, I am proud that my very first article (see: What's the story behind your first paper?) was listed too. Its follow-up article (see: Expatriate Failure revisited) even appeared eleven times in syllabi, ranking just outside my top-10. Clicking through, OS showed me the actual syllabi in which my work was used, including the countries and universities that listed my work. I could see all of this being really useful if I had to go up for promotion and had to evidence my contribution to research-based teaching.

Not surprisingly, my edited textbook on International Human Resource Management was by far my most frequently included publication in syllabi, reporting more than 800 appearances. This textbook was first published in 1994 and is now in its 6th edition. Two new co-editors (Sebastian Reiche and Helene Tenzer) joined the editorial team for the 5th edition. Unfortunately, it was linked only to the first co-editor's name (Sebastian Reiche), so I had to search for the title instead. Remember that like any service based on publication data, Open Syllabus isn't flawless. So use it intelligently.

Reviewing the articles that were included in syllabi, I was quite surprised not to see my papers on academic referencing, transcending the sense and nonsense of university rankings, country-of-origin effects in HRM policies, and staffing policies in MNCs listed. Anecdotally, I had heard that colleagues were using these articles in their teaching. However, when searching for my name in Open Syllabus Galaxy (see an extended discussion later in this white paper) I found that all four of them were included, some even with a very large number of appearances. So if you are using Open Syllabus for your tenure or promotion application, make sure you use both sources.

Which journals are featured in syllabi in your field?

Open Syllabus can be used to find out what academic journals feature most frequently in syllabi in your field. You can do this in the publishers section by filtering for "articles". The filtering for journals isn't perfect as in some cases publishers - such as PubMed, Harvard Business School Publishing or the Social Science Resarch Network - are shown rather than journals. However, it gives you a pretty good idea of what journals are frequently listed in syllabi. You can narrow these results down by country. Below you can see the top-10 for Business in the UK and here is the complete list. [Note: the first two are publishers rather than journals, so I have shown the first 12 entries.]

I did the same search for the three other countries with strong representation in Open Syllabus. I found the geographical differences fascinating: the top journals differ substantially between the UK [see above], Australia [see below], and the USA / Canada [that are similar to each other].

The USA/Canada mostly feature only the very top academic journals in the field of Business, those that academics on a tenure track at elite North American universities would be expected to publish in. Examples are: Administrative Science Quarterly, Academy of Management Review/Journal, Strategic Management Journal, Journal of Finance, Organization Science, and Journal of Management. The only journals listed in the top-10 in the USA/Canada that are outside this elite group are Academy of Management Perspectives, which publishes rigorous original analyses that inform managerial practice or policy, and California Management Review, which publishes evidence-based research that not only engages academics, but also educates students, and contributes to the practice of management

In the UK and Australia, Academy of Management Review and Strategic Management Journal are ranked in the top-10 too. However, there is a far greater representation of more critically oriented journals such as Human Relations, Work, Employment & Society, and Organization Studies. The Journal of Business Ethics is also ranked more highly in Australia (#1) and the UK (#6) than it is in Canada (#8) and especially the USA (#14). Finally, the UK and Australia both feature Tourism journals in the top-10/15, whereas in the USA/Canada these barely make it to the top-50.

Do countries differ in their disciplinary focus?

Journal coverage in the four most represented countries in Open Syllabus reflects their differential disciplinary focus. In my own field of International Business, the most prestigious journal (Journal of International Business Studies) nearly made it to the top-10 in the UK (#11) and is ranked in the top-20 in the USA, but only made it to the top-50 in Australia and top-70 Canada. The field's 2nd ranked journal, Journal of World Business, featured in the top-30 in the UK and the top-40 in Australia, but only just made it to the top-100 in the USA, and is outside it for Canada. Based on Open Syllabus data, International Business as a discipline appears to be seen as more important in the UK universities than in Australia and North America.

Courses dealing with the human aspects of Business typically draw from a variety of sub-disciplines, including Human Resource Management (HRM) and Industrial Relations (IR), but also Applied Psychology. We find very interesting patterns here with HRM and especially IR journals being rather unpopular in North America, whereas Psychology journals are more popular. The reverse is true for the UK and Australia.

No less than two HRM journals feature in the top-10 in both the UK and Australia, whereas only one of them even made it to the top-30/40 in the USA and Canada. An IR journal made it to the top-30 in both the UK and Australia. The UK featured no less than four IR journals in the top-150. Work, Employment, and Society - a sociology of work journal - even featured at #4. In Canada, two IR journals made it to the top-50 and a further three featured in the top-150. The USA only featured one IR journal and this journal just made it into the top-100.

In contrast, Psychology journals are more popular in North America than in the UK and Australia. The USA and Canada both featured three Psychology journals in the top-50/60, including one in the top-10/15. Australia also features three journals in the top-50/60, but they are all lower ranked than in the USA and Canada. In the UK, only one Psychology journal makes it to the top-50.

These differences clearly reflect different societal and research traditions that - in turn - are translated into differences in course syllabi. The USA, and to a lesser extent Canada, is characterised by a dualistic perspective on the employment relationship, concerning the employer and employee only and without regulation by external parties. In the UK and Australia collective actors such as the government and labour unions have traditionally played a much bigger role in regulating the employment relationship.

That said, Industrial Relations as a field of study is clearly in decline across the world. The trend analysis above shows that the use of this topic/concept in syllabi has faced a steady decline since 2008, when it equalled Psychology. In contrast, Psychology has largely maintained its position in syllabi and in the last five years has had 5-6 times more coverage than Industrial Relations. For more on trend analyses see a later section in this white paper.

Gender statistics: how does the gender ratio of authors vary by discipline?

The 'full' version of Open Syllabus Analysis includes a section on gender statistics that explores the gender ratio of authors and instructors across the various fields in the dataset. Below are the fields with the highest and lowest proportion of female authors in the syllabus set. In total, more than 60 disciplines are covered.

Open Syllabus shows gender ratios for both assigned authors and instructions. Hence, we can establish that in virtually every field covered in Open Syllabus the proportion of female instructors is higher than the proportion of female authors. The only field in which this is not the case is women's studies, with 58% female instructors and 64% female authors.

As most academics have both a teaching and a research role, this may allow us to conclude that publications by female authors are underrepresented in syllabi. However, this may also be partly caused by the fact that syllabi often feature classic older works that were published at a time where female researchers were less common. In any case, these gender statistics are another interesting data source for small curiosity research projects as they can also be narrowed down by country, institution and time period.

Trends: does teaching of specific topics/concepts vary over time?

A final interesting feature - available in the 'full' version of OS Analytics only - is a longitudinal analysis of the frequency of specific keywords or concepts over time. This featured can be narrowed down by country, allowing for a cross-country comparison of teaching (and research) interests. Below I compared the UK, Australia, and the Netherlands for three topics I have taught in these countries: international business, international management, and cross-cultural management. I added the USA for comparative purposes.

In the USA, all three topics appear to have become less popular in the last 15 years, possibly due to the decline of freestanding International Business courses in the USA. In the UK, the pattern is less clear, with a decline followed by an uptick in recent years for International Business, bringing it back to the early days of the period. International management and cross-cultural management also show an uptick in the last 3-4 years and now have a higher representation in syllabi than ever.

Australia combines the UK's volatile pattern for International Business, with the USA's steady decline for international and cross-cultural management. The Netherlands shows the most international orientation of the four countries with the highest reported frequency of all three topics, and a strong increase in frequency in International Business in recent years, partly compensated by a decline in international management and cross-cultural management.

Trends can also be analysed by institution or by institution type, allowing you to assess changing coverage in your own or a benchmark institution or a set of institutions. Obviously, the quality of this data is highly dependent on the level of coverage of individual institutions, which can be sparse, thus leading to volatile results. Hence, I found playing around with concepts in the full data-set the most helpful use of this service. However, for countries with larger coverage, analyses such as the above might reveal interesting patterns that would not easily be discernible in other ways.

Open Syllabus Galaxy

This part of the service provides a plot of the top one million titles in the Open Syllabus dataset, grouped by how often they are assigned together. It allows for a wide range of searches. I have included some representative examples here, but I would encourage you to experiment a little to get the most out of this service.

Evidencing teaching impact: Do my publications feature in syllabi? (2)

First, you can use the Galaxy as another way to assess the extent to which your research publications are covered in syllabi, thus evidencing your teachin impact. This service only reports on the one million most frequently assigned texts overall. So, it will normally provide a sub-set of what you found in OS Analytics. However, in my case it found four additional highly represented articles that were not included in my primary profile in OS Analytics.

Overall, I was intrigued to see that no less than 17 of my publications had made it to the most frequently assigned texts overall (see image below). This included the four publications on academic referencing, transcending the sense and nonsense of university rankings, country-of-origin effects in HRM policies, and staffing policies in MNCs that weren't found in my OS Analytics search. They appeared in 24, 12, 17, and 6 syllabi respectively.

In which disciplines is your work used?

The Galaxy makes for a fascinating interactive service. You can zoom in into different parts of the Galaxy to figure out which other publications your or others' work is most frequently co-assigned with. It is similar to a citation network, but set in the context of higher education as reflected in teaching syllabi.

Much of my most featured work was co-cited in the Business area (see image below), reflecting publications from my  three research programmes in International Business (International HRM & Staffing Policies, HQ-Subsidiary Relationships in MNCs and Language in International Business). However, there were also significant clusters in the research methods (e.g. Are referencing errors undermining our scholarship and credibility? and the Library and Information Science area, reflecting my research programme on the Quality and Impact of Academic Research.

How are topics/concepts spread over different disciplines?

The Galaxy can also be used to search for topics/concepts, allowing you to assess to what extent they cluster by discipline or are spread over different disciplines. The image below shows the result of searching for expatriate (a topic I have been researching and teaching in). I was fascinated to see that it registered in a wide variety of disciplines.

As reflected at the top of the figure, there was a big concentration in Business. Expatriate management is an important topic within the field of International Business and International HRM. However, it also featured in disciplines as diverse as History, Sociology, English Literature, Political Science, Migration, East Asian Studies, Education, Psychology, and even Research Methods. The latter featured my own article about academic referencing which used expatriate failure rates as a case study: Are referencing errors undermining our scholarship and credibility?

What textbook to use for my course?

You can also use the Galaxy as a tool if you are teaching a new course and are looking for inspiration for which textbooks to use. You can conduct search both in books/articles and in syllabi. The two options provide slightly different results, and I recommend you use both sources in order to triangulate your findings. Here are two searches which show the most frequently assigned textbooks for two areas I have taught in the past. Of course, the most frequently used book is not necessary the best for your purpose, but at least it is a start!

Conclusion

Open Syllabus is a versatile tool, which can be used for a wide variety of purposes.

• academics who need to evidence research-based teaching or are simply curious whether their research is used in the classroom at all,
• curiosity driven searches or even little research projects such as:
  • which journals are used most often in the classroom in different countries for specific disciplines,
  • the gender distribution in assigned literature compared to the gender distribution in instructors,
  • the development of teaching topics/concepts over time in different countries,
  • the representation of teaching topics/concepts in different disciplines.
• teaching preparation, such as searching for a new textbook and related readings.