7.1 Create your own reference group
First, it is a good idea to compare your case to a relevant group of peers. Many evaluators have very little idea of what typical norm scores for the various metrics are. So unless you make an explicit comparison, they will explicitly or implicitly use their own reference group, which might not work to your advantage.
As the analysis in Chapter 16 shows, there are vast differences in typical citation scores between disciplines, especially when using Thomson ISI citation data. Therefore, if your university has a tenure or promotion process in which decisions are made by committees composed of people in related or even unrelated disciplines, it is even more important to frame your case for tenure or promotion with an appropriate reference group.
Citation behaviors can also vary dramatically within disciplines or even within sub-disciplines. The area of Human Resource Management as a sub-discipline of Management includes scholars working on industrial relations and labor unions as well as scholars working on more psychologically oriented topics such as motivation or job attitudes. The latter academics might be able to publish in a mainstream Psychology journal such as Psychological Bulletin, whilst the former academics would feel most fortunate if they published in the top US journal in their field: Industrial Relations. At 12.85, the journal impact factor of the former is six times as high as the journal impact factor of the latter (2.05).
Moreover, many Industrial Relations academics will not be able to publish in mainstream US-American Industrial Relations journals as their research is very contextual. Hence they might need to publish their work in even lower impact journals such as British Journal of Industrial Relations (1.38), European Journal of Industrial Relations (1.00), Asia Pacific Journal of Human Resources (0.78), or the Australian journal Labour History (0.15). Hence, any articles in the area of Industrial Relations can be expected to be cited far less frequently than articles in the area of organizational behavior, which is more closely related to psychology.
Therefore, it is very important to pick your reference group wisely. Your reference group should be narrow enough to reflect any differences in citation behaviors across disciplines. However, it should not be so narrow that it leads your committee to discard your selection as biased or irrelevant. Generally, I have found two strategies to be particularly effective: the international discipline-based strategy and the institution-based strategy.
For the first strategy, you compare your record with a representative selection of other academics at the level you are applying for. To make your case convincing, it is usually best to pick academics at institutions of similar or higher level of prestige. If you can show you are performing at the same level as academics in more prestigious institutions who have been in position for a while, you have a very strong case.
The second strategy is a more local strategy. Here you compare your record with academics in your own institution at the level you are applying for. If you have access to the length of tenure of your academic colleagues, you might be able to compare your performance with that of both long-established academics and that of academics recently promoted to the same level. This strategy might be particularly effective if your institution has more stringent norms for promotion than comparable institutions.
A combination of both strategies might even be more effective. The table below was taken from my own (successful) application for full professor in 2007. I used three reference groups, focused on my strengths (single/first authored articles) and included ISI citations for panel members who dont like Google Scholar.
Bibliometric comparison with other professors, mean and range are given for each indicator*.
| Reference Group | h-index | 1st authored papers in h-index | Single-author papers in h-index | Number of ISI citing articles (2006 only) | Years as professor |
|---|---|---|---|---|---|
| 2005/2006 promotions in the same department | Mean: 6.3 Range: 4-8 |
Mean: 4.3 Range: 3-5 |
Mean: 3.0 Range: 2-4 |
Mean: 15 Range: 3-34 |
Recently appointed |
| IB professors at top Australian universities | Mean: 9.0 Range: 4-16 |
Mean: 3.3 Range: 1-6 |
Mean: 1.5 Range: 0-3 |
Mean: 10 Range: 3-21 |
15 years (4-28 years) |
| Established professors in the same department | Mean: 14.0 Range: 6-22 |
Mean: 5.0 Range: 0-11 |
Mean: 2.0 Range: 0-4 |
Mean: 46 Range: 8-102 |
14 years (10-19 years) |
| Anne-Wil Harzing | 13 | 13 | 10 | 63 | N/A |
* Please note that the specific Google Scholar data this table is no longer representative because of a substantial increase in GS coverage. The general principle however still applies.
Please note that you will normally need at least 3-4 academics in your reference group to be able to make a credible comparison and larger numbers are advisable. I would generally advise against listing names of individuals as this can easily lead to antagonistic responses. However, be prepared to substantiate your averages if so requested. You might wish to create folders for your reference groups in the Publish or Perish multi-query center (see Chapter 6), so that you can store and update analyses easily.
In Section 7.7 you will find some “norm” scores for management, marketing and social psychology. Please note that many of these norm scores are aspirational scores as they refer to top academics in the respective fields. Of course if you can show you meet even these norm scores, your case for research impact is very strong.
In general, please realize that it is your job to convince and educate your tenure or promotion panel of the impact of your research. Many senior academics, having grown up in an age in which citation metrics were relatively unimportant, have a very limited knowledge of their own or other academics citation records. Moreover, in my experience many academics have the tendency to subconsciously overestimate what their own records were when they went up for tenure or promotion and hence use an inappropriate reference group.
You might think this is unfair and senior academics should know better, but remember that they are only human and are very busy people as well. Moreover, many processes in academia (e.g. further promotions, job applications, grant applications, applications for research awards, and applications for fellowships) depend on you making the case for the impact of your research. Hence it is not a bad idea to get some skills in “selling” your record!