9.5 Conclusion: what sensible administrators should do

So what advice can I give on citation analysis to administrators who want to maintain rigorous standards, but also want to be fair and equitable? First, especially if you work in the Social Sciences and Humanities or in Engineering and Computer Science, give up your reservations about Google Scholar.

These days, many universities allow or even recommend the use of Google Scholar and Publish or Perish. Institutions such World Bank and Microsofts research laboratory use it to evaluate their broader impact beyond ISI listed journals. Many US government departments use it to evaluate the impact of particular research projects. You can always ask applicants to provide ISI or Scopus data as well, but do take Google Scholar data seriously.

Yes, some of Google Scholar's results are nonsensical. Yes, occasionally Google Scholar will double-count citations. Yes, some of the citations are not as scholarly as one would want. However, these minor errors are not worth worrying about when comparing them with the systematic and very large underestimation of citations in ISI or even Scopus. For an introduction of the three data sources see Chapter 1. For more detail about their respective advantages and disadvantages, see Chapters 13, 14 and 15.

Second, give up your fixation with self-citations. Yes, occasionally academics abuse the system and systematically cite their own work. However, there are safeguards against this system as journal editors and reviewers will not like gratuitous citations. Moreover, if academics are so unethical, they probably have bigger problems than their inflated citation record. More importantly, for the majority of academics self-citations do not distort their citation records in any significant way. The hassle and possible inconsistency in excluding self-citations is not worth the small possible gain in accuracy.

Third, be very hesitant in applying citation analysis for junior academics, especially again in the Social Sciences and Humanities. It can easily take 5-10 years after an academics first publication for a significant number of citations to flow in. Hence, if an early career academic shows a large number of citations, make sure you promote them and keep them happy (assuming other aspects of their performance are also at least satisfactory). However, if academics going up for tenure have very few citations, dont hold it against them.

Fourth, realize that citations can vary dramatically between or even within disciplines. Never compare citation (or publication) records across disciplines. If for some inexplicable reason you have to do so, use Google Scholar, not ISI or Scopus. Be very hesitant to prescribe norm scores for the number of citations to be accumulated before someone is considered for tenure or promotion to a certain level. If you feel you do need to prescribe norm scores, make sure they reflect current practice in the field and for academics at the same level the candidates apply for.

Fifth, if you have any doubts about what you are doing, consult an expert. Academics and administrators at this level are far too busy to try to understand the minutia of citation analysis. Get a proper bibliometric expert involved! You might have access to a librarian with good skills in this respect or you might have academics in your staff who do bibliometric research. If you do not, read the rest of this book as well. If you read it closely, at the end youll probably know more about citation analysis than 99.9% of the academics.