10.2 Step 2: Comparing journals for impact

After creating a “short-list” of journals that you might want to publish your paper in, one of the criteria to make your final choice could be the standing or rank of the journal. Of course, the most important criterion should be that your paper suits the focus and editorial aims of the journal. However, if there are a set of journals that meet this criterion, you might as well submit your paper to a higher-ranked journal first.

In general we can distinguish two broad approaches to ranking journals: stated preference (or peer review) and revealed preference (Tahai & Meyer, 1999). Stated preference involves members of particular academic community ranking journals on the basis of their own expert judgments. These are often undertaken by particular universities or departments in order to help make decisions about, for example, library budgets, promotion or tenure, and national research evaluations, such as the Research Assessment Exercise in the UK.

As a result there are hundreds of individual university journal rankings in existence and integrated journal ranking lists have sprung up that combine a range of rankings (see e.g. the British ABS Journal Quality Guide (ABS, 2010) and Harzings Journal Quality List (Harzing, 2010). Opinions might be based on anything from a large-scale worldwide survey of academics to a small group of individuals with decision-making power, but will always contain some element of subjectivity.

Revealed preference rankings are based on actual publication behavior and generally measure the citation rates of journals using Thomson ISI's Web of Knowledge. Most commonly used are the ISI Journal Citation Reports (JCR), which provide the yearly Journal Impact Factors (JIF). For more details on the JIF, see Section 1.4.1. However, any source of citation data can be used. Publish or Perish is ideally suited to measure the impact of journals with Google Scholar data.

Mingers and Harzing (2007) show that there is a high degree of correlation between journal rankings based on stated and revealed preference. However, as Tahai & Meyer (1999) point out, stated preference studies have long memories: perceptions of journals normally change only slowly in these rankings. As such, revealed preference studies provide a fairer assessment of new journals or journals that have recently improved their standing. Therefore, revealed preference studies present a more accurate picture of journal impact.