9.1.1 Not everything published on the Internet counts in Google Scholar

Some academics are under the misplaced impression that anything posted on the Internet that includes citations will be counted in Google Scholar. This might also be the source behind the misconception that one can put simply put phantom papers online to improve ones citation count. However, Google Scholar only indexes scholarly publications. As their website indicates “We work with publishers of scholarly information to index peer-reviewed papers, theses, preprints, abstracts, and technical reports from all disciplines of research.”

Some non-scholarly citations, such as student handbooks, library guides or editorial notes slip through. However, incidental problems in this regard are unlikely to distort citation metrics, especially robust ones such as the h-index. An inspection of my own papers shows that two thirds to three quarters of the citations are in academic journals, with the bulk of the remainder occurring in books, conference papers, working papers and student theses.

Very few non-scholarly citations were found. Moreover, I would argue that even a citation in student handbooks, library guides, textbooks or editorial notes shows that the academic has an impact on their field. In a similar vein, Vaughan and Shaw (2008) argue that 92% of the citations identified by Google Scholar in the field of library and information science represented intellectual impact, primarily citations from journal articles.

Hence, although there might be some overestimation of the number of non-scholarly citations in Google Scholar, for many disciplines this is preferable to the very significant and systematic under-estimation of scholarly citations in ISI or Scopus. Moreover, as long as one compares like with like, i.e. compares citation records for the same data source, this should not be a problem at all.