10.2.1 Worked example: Accounting journals
For a detailed example of doing a Google Scholar based impact analysis, see Chapter 15 on Journal Impact Analysis, which compares the Thomson ISI JIF and the Google Scholar h-index for some 800 journals. Below I will provide a smaller-scale example of journals in the field of Accounting.
Because of differences in accounting rules across countries, Accounting is a localized discipline. As a result, not many of the fields journals are listed in the Thomsons Journal Citation Reports. Only 30% of the journals in Finance & Accounting listed on the JQL are included in ISI. In contrast, three quarters or more of the journals on the JQL in Economics or Management Information Systems are ISI listed. Hence, if one wants to compare the citation impact of Accounting journals, using Google Scholar is often the only alternative.
The table below lists a random selection of Accounting journals, including the journals generally recognized as the top-5 accounting journals. The table first lists the ISI Journal Impact Factor for 2009 (where available) and the ABDC (Australian Business Deans Council) rank, a popular journal ranking list in Australia. It then reports on the results of a Publish or Perish impact analysis for papers published in the journals between 2005 and July 2010. I report the average number of citations per paper, the Google Scholar h-index and g-index.
In order to get a realistic citations per paper count, I merged duplicate papers, removed book reviews, commentaries, obituaries, conference announcements, call for papers, etc. as these items rarely ever attract citations. Including them would distort comparisons between journals that include these items and journals that do not.
| Journal Name | ISI JIF 2009 | ABDC ranking | GS cites pp | GS h-index | GS g-index |
|---|---|---|---|---|---|
| Jnl of Accounting Research | 2.350 | A* | 24.20 | 43 | 74 |
| Review of Accounting Studies | 1.500 | A* | 23.64 | 28 | 51 |
| Contemporary Accounting Research | 1.087 | A* | 16.87 | 28 | 45 |
| Accounting, Organizations and Society | 1.803 | A* | 15.28 | 34 | 49 |
| Accounting Review (The) | 1.920 | A* | 15.20 | 37 | 56 |
| European Accounting Review | 0.633 | A | 10.88 | 23 | 37 |
| Accounting and Business Research | Not listed | A | 9.62 | 15 | 25 |
| Jnl of Accounting and Public Policy | Not listed | A | 9.23 | 19 | 30 |
| Accounting Horizons | Not listed | A | 8.45 | 17 | 27 |
| Accounting, Auditing & Accountability Jnl | Not listed | A | 7.63 | 22 | 31 |
| Int Jnl of Accounting Inf Systems | Not listed | A | 7.20 | 16 | 21 |
| British Accounting Review | Not listed | A | 6.85 | 15 | 22 |
| International Jnl of Accounting | Not listed | A | 6.63 | 15 | 21 |
| Critical Perspectives on Accounting | Not listed | A | 5.42 | 20 | 29 |
| Behavioral Research in Accounting | Not listed | A | 4.78 | 9 | 14 |
| Issues in Accounting Education | Not listed | A | 3.20 | 11 | 17 |
The top-5 accounting journals (all A* ranked in the ABDC ranking) stand apart in terms of their Journal Impact Factor and Google Scholar metrics. However, there is quite a difference in terms of citations per paper for the remaining journals, with European Accounting Review, Accounting and Business Research and Journal of Accounting and Public Policy having citation per paper rates that are 2-3 times as high as the journals towards the bottom of the list. This is true despite the fact that they were all ranked A on the Australian journal ranking list.
The GS h-index and GS g-index also show similar differences. It is, however, interesting to see that some journals (e.g. Critical Perspectives on Accounting) publish a fairly large number of impactful papers, as evidenced by the relatively high h-index and g-index, even though the average number of citations per paper is not very high. Overall though, there is a very strong correlation between the three GS-based impact measures (0.86 between GS cpp and GS h-index; 0.92 between GS cpp and GS g-index; 0.98 b/w the h-index and g-index).
In conclusion, this example shows that even when comparing journals that score similarly in stated preference (peer review) rankings can have very different impact scores. Given that very few Accounting journals have ISI journal impact factors, a Google Scholar based impact analysis is an excellent way to assess the impact of non-IS listed journals. It allows you to make a more-informed choice when you chose a journal to submit your paper to.