Accuracy of the results

The citation analysis is based on the results returned by Google Scholar. These are not always 100% accurate. Here are some issues to be aware of. Please also note that Google Scholar limits its results to 1000. The results are ranked by number of citations, so the 1000 shown are the most-cited results.

Note: See More about citation analysis for an in-depth discussion of the validity, assumptions, and limitations of the underlying sources and methods used by Publish or Perish.

Results that differ from Google Scholar

If the Publish or Perish results differ from the ones you get by using Google Scholar directly, this is typically caused by the fact that Publish or Perish uses the Advanced Scholar Search capabilities of Google Scholar, whereas your manual search probably used the standard Google Scholar search. The latter is equivalent to an All of the words search, which matches the search terms anywhere in the searched documents (author, title, source, abstract, references etc.) and usually provides too many irrelevant results for an effective citation analysis.

However, if you do want to get the same results in Publish or Perish as with a standard Google Scholar search, do the following.

  1. Go to the General citation search page.
  2. Empty all text fields except All of the words.
  3. Enter your query terms in the All of the words field.
  4. Set the Year of publication fields both to 0.
  5. Clear the Title words only field.
  6. Click on Lookup.
  7. When the results appear, click on the Rank column header to sort the results in the order in which Google Scholar returned them.

Ineffective queries

Not all queries return the results you would expect. You might have to refine your queries to get the most accurate results. See Author impact analysis, Journal impact analysis, and General citation search for tips to refine your query.

Some tell-tale signs of ineffective queries are:

Too many results that you are not interested in
This is typically caused by criteria or search terms that are too broad.
Mixed up ranking order
Sometimes results appear plausible, but the Rank column appears strangely jumbled. In the screen shot below, note that the most cited works rank very low, and that the #1 and #2 ranked works only appear some way down the list:

This indicates that the most cited works are not the most relevant to the query. It is caused by search terms that are inappropriate, too wide-ranging, and that therefore catch highly-cited but mostly irrelevant works as well.

For other potential problem areas, read on.

Mixed-up title and source fields

Some references contain mixed-up fields as illustrated in the second reference below:

Mixed-up results

This is typically caused by garbled information returned by Google Scholar, presumably because its sources were inaccurate or difficult to parse automatically by Google's web crawler.

The effect on the citation analysis is similar to having duplicates (see above), because some works end up as separate entries instead of being included with the correct title.

Author of publication listed under title

For some references Google Scholar lists the author name as part of the title, rather than including the author’s name in the author field, presumably because its sources were inaccurate or difficult to parse automatically by Google's web crawler.

As a result the reference in question does not show up if you search using the author’s name in an author query, since this query only lists results where the name is listed in the author field. For example, the following highly cited paper by Anbulagan does not show up when searching for his name in the author field.

[CITATION] Anbulagan. Heuristics based on unit propagation for satisfiability problems
CM Li - Proceedings of the International Joint Conference on …, 1997
Cited by 237

If you have reason to believe that certain publications of a particular author are missing, you might want to repeat the search using a general query with the author’s name in the Any of the words field. This query searches for the author’s name in all parts of the database. Whilst it normally provides a large range of irrelevant results (especially if an author’s name is also a common noun, e.g. Robert Wood), it does allow you to find publications where Google Scholar has accidentally misplaced the author’s name.

Matching of journal titles

The journal impact analysis implements the Advanced Scholar Search "return articles published in". Google Scholar interprets this search broadly and returns matches for both the publication and the publisher. This means that a search for a relatively generic journal title such as Information Systems Research might get additional matches in the publisher field, such as Center for Information Systems Research or even the eventual co-sponsor/co-publisher of MIS Quarterly (Management Information Systems Research Center, University of Minnesota), even though this is not directly visible in the Google Scholar output.

Duplicate results

Occasionally you might notice duplicate or near-duplicate articles in the Results list. These duplicates may be due to one or more of the following:

The effect on the citation analysis is that: