16.2.2 Google Scholar versus ISI and Scopus general search

As can easily be verified in Table 1, Google Scholar is the most democratic of the three data sources in that it provides the highest level of citations for all ten academics in our sample, with the exception of our Pharmacist.

Sciences

Even for the Pharmacist in our sample, Google Scholar only underestimates his citations by 10% and does much better than Scopus in this respect. The underestimation is most likely caused by Google Scholar's weaker coverage of older materials, which are not always available on the web. Since our Pharmacist has been publishing for over 40 years, some of his older work might not be covered in Google Scholar. This assumption is confirmed if I compare ISI and Google Scholar's citations to his work in the last decade only. For the last decade alone, citations to his work in Google Scholar are twice as high as ISI citations.

As Figure 3 shows, overall, academics working in the Sciences do not show very significant differences between their ISI and Google Scholar citation scores. For the Pharmacist they are 10% lower, for the Mathematician and Cell Biologist they are only 1-3% higher. For the Physicist, citations in Google Scholar lie 25% above ISI citations.

As before, the Computer Scientist is the exception to this rule. In his case, Google Scholar citations are no less than 7.5 times as high as ISI citations. As for Scopus, this is partly caused by the much broader journal coverage of Google Scholar. However, in this particular case, a very significant chunk of the Google Scholar citations (more than 40%) are citations to a book that as discussed below is completely ignored in ISI.

Figure 3: Citations for ISI and Scopus General Search compared to Google Scholar: Science disciplines

We have seen above that Scopus citation scores for the Scientists were generally lower than their ISI citation scores. Therefore, it is not surprising that their Google Scholar citation scores are substantially higher than their Scopus citation scores. For the Cell Biologist and the Mathematician Google Scholar citation scores are one third higher than Scopus citation scores, for the Physicist this is two thirds, whilst the Pharmacist has nearly twice as many citations in Google Scholar than in Scopus.

For the Computer Scientist Google Scholar provides five times as many citations as ISI, again reflecting the very significant number of book citations. So overall, although Google Scholar still has a slightly lower coverage of older publications than ISI, it is doing much better than Scopus in this respect.

Social Sciences and Humanities

As is readily apparent form Figure 4, for the academics working in the Social Sciences and Humanities, the differences between Google Scholar on the one hand, and ISI or Scopus on the other hand are much larger than for academics working in the Sciences. When comparing Google Scholar and ISI citation scores, the Business academic has six times as many citations in Google Scholar than in ISI. However, this difference is dwarfed by the increases for the Political Scientist (15 times), the Cinema Studies Academic (25 times), the Linguist (32 times) and the Educationalist (33 times). It is clear that for academics working in the Social Sciences and Humanities, a focus on citations to ISI listed publications only, severely underestimates their citation impact.

Figure 4: Citations for ISI and Scopus General Search compared to Google Scholar: Social Science and Humanities disciplines

With regard to the differences between Google Scholar and Scopus, the Social Science and Humanities academics again follow a different pattern from most of the Science academics. Whilst for the Science academics differences between Google Scholar and Scopus are slightly larger than between Google Scholar and ISI, for academics in the Social Sciences and Humanities, they are slightly smaller, but still substantial in absolute terms. Differences run from 3.5 times as many citations in Google Scholar for the Business academic to 27 times as many for the Linguist. The Cinema Studies academic shows a 180 times increase in citations, but that is caused entirely by the very incomplete coverage of her publications in Scopus.

Conclusion

Comparing Google Scholar on the one hand and ISI and Scopus on the other hand provides mixed results. For the academics working in the Sciences, Google Scholar's advantage over Scopus is larger than over ISI (except for the Computer Scientist). For the academics working in the Social Sciences and Humanities, this pattern is reversed in that Google Scholar's advantage over ISI is larger than over Scopus. However, in virtually all cases Google Scholar provide the highest citation count, reflecting its broader coverage in terms of sources compared to both ISI and Scopus and its longer coverage in time compared to Scopus.

Publication patterns in the Social Sciences and Humanities include a heavy emphasis on books and many of the journals in these fields are not listed in either ISI or Scopus. Publication patterns in Computer Science include a heavy emphasis on conference proceedings. Google Scholar therefore provides a much fairer assessment of citation impact for academics working in these disciplines. It includes citation in and to books, book chapters, government reports, conference proceedings, as well citations to and in academic journals not listed in ISI or Scopus.

So whilst in the ISI database academics working in the Sciences have on average 17.5 times as many citations as academics working in the Social Sciences and Humanities, in Google Scholar this difference is reduced to a mere 1.5 times. In addition, three of the Social Scientists (Business, Education & Political Science) have citation records that are higher than or comparable with three of the Scientists (Cell Biology, Mathematics, Physics).