The mystery of the phantom reference: a detective story

Short summary of white paper that shows how sloppy writing and sloppy quality control lead to a non-existing article being cited nearly 400 times

Anne-Wil Harzing - Wed 8 Nov 2017 08:11 (updated Thu 2 Jun 2022 14:31)

Picture: Paris Musee Cluny by Pieter Kroonenberg

Through my work with Publish or Perish I get in touch with many academics who are doing bibliometric work, oftentimes as a “research hobby”. In one of these exchanges, Pieter Kroonenberg, a Dutch emeritus professor in Statistics, told me about an interesting puzzle he had come across. When looking at the author guidelines for an Elsevier journal that he intended to submit to he noticed the following reference:

• Van der Geer, J., Hanraads, J.A.J., Lupton, R.A., 2000. The art of writing a scientific article. J Sci. Commun. 163 (2) 51-59. [The journal name can also be found with its full title Journal of Science Communications]

He was intrigued to see that one of his former colleagues Prof. John van de Geer had a “hidden side”, publishing about the art of academic writing in addition to his work on experimental psychology and multivariate analysis. But, wait a minute…, this reference referred to Van der Geer instead of Van de Geer. Still…, the paper looked interesting so he ventured to look it up. However, despite strenuous efforts he was unable to find it. An (Italian) journal with a similar name did exist, but its full name was Journal of Science Communication rather than Communications and it had only started in 2002. Looking at the original reference again, it did strike him as a little odd for a journal to have published 163 volumes in a discipline that normally equates volumes to years. Moreover, the second author seemed to have only ever published this particular article, which obviously is rather strange for someone writing about the art of writing a scientific article.

To cut a long story short, the article appeared to be completely made up and did not in fact exist. It was a “phantom reference” that had been created merely to illustrate Elsevier's desired reference format. Even so, Pieter found that in the Web of Science there were nearly 400 articles citing this non-existing reference and many more citing articles appeared in the more comprehensive Google Scholar. The fact that academics don’t always take the necessary care in their referencing behaviour is something that is not unfamiliar to me. Early on in my career, I even wrote an article about this: Are referencing errors undermining our scholarship and credibility? But even so, how could authors cite a publication that didn’t in fact exist?

Thus, I was instantly intrigued as this is just the kind of puzzle I like solving. Yes I know I am a bit of a forensic bibliometrics nerd. Maybe it has something to do with me wanting to be a detective when I was young? Although I did have one of Enid Blyton’s girly “Pitty at boarding school” books, my favourite books were her “Famous Five” detective stories. And of course, I always wanted to be Dick, the one who knew all the detective tricks, not one of the two rather meek girls. Well in a way academics are the ultimate detectives aren’t they? They follow both hunches and systematic research to get to the bottom of tricky problems. For the full detective story, refer to the white paper:

• Harzing, A.W.; with Kroonenberg, P. M. (2017) The mystery of the phantom reference, harzing.com white paper.

Caveat

As I wrote to a journalist from CBC who contacted me with further questions, the pressure on scientists to "publish or perish" might well have contributed to this phenomenon. That said, nearly every academic is under the same pressure, but the vast majority of academics do behave both conscientiously and ethically. As always, it is only a small minority that is giving the profession a bad name.

Press coverage for the white paper

• Retraction Watch: The “phantom reference:” How a made-up article got almost 400 citations, 14 November 2017
• Gizmodo: At Least 400 Science Articles Cite Nonexistent ‘Phantom’ Study, 15 November 2017
• IFL Science: The Mystery Of The Study That's Been Cited 400 Times Despite Not Existing, 15 November 2017
• Indicator: Несуществующую статью процитировали сотни раз, 15 November 2017
• Science Alert: Over 400 Scientific Papers Have Cited a 'Phantom' Reference That Never Existed, 16 November 2017
• The Times: Professor and paper that don’t actually exist are copybook way to win a citation, 16 November 2017
• Gizmodo Spain: Más de 400 trabajos científicos han citado un estudio sobre el arte de escribir ciencia que jamás existió, 16 November 2017.
• Tekniikan Maailma: Tämä artikkeliviittaus on päässyt yli 400 tiedeartikkelin lähdeluetteloon – Ongelma on, että artikkelia ei ole olemassakaan, 16 November 2017
• CBC News: SECOND OPINION | 'Phantom' research paper cited at least 400 times by scientists, 18 November 2017
• Focus Magazine: Il caso dell'articolo fantasma citato 400 volte, 18 November 2017
• Der Standard: Nie geschriebenes Paper wurde in 400 Fachartikeln zitiert, 18 November 2017
• Leids Universitair Weekblad Mare: Slordigheid veroorzaakt spookcitaties, 23 November 2017
• soL: Hiç yazılmamış 'hayali' bir bilimsel makale 400 kereden fazla kaynak gösterildi, 27 November 2017
• NRC: Spookartikel wordt ‘toevallig’ 400 keer geciteerd, 5 December 2017
• Com Ciência: Falsa ciência e pós-ciência?, 17 April 2018
• Referenced in Wikipedia article on Procedia, the conference series that was responsible for 90% of the references to our Phantom article.

Related blogposts

• Are referencing errors undermining our scholarship and credibility?
• Sacrifice a little accuracy for a lot more comprehensive coverage
• Is Google Scholar flawless? Of course not!
• Bank error in your favour? How to gain 3,000 citations in a week
• Transcending the (non)sense of academic rankings
• Health warning: Might contain multiple personalities
• Publish or Perish General Search - a Swiss Army Knife?