Nature對影響因子的評論
Editorial
Nature 435, 1003-1004 (23 June 2005)

Not-so-deep impact

Research assessment rests too heavily on the inflated status of the impact factor.

Every year at the end of June, scientific publishers' eyes turn to Philadelphia, where the Institute for Scientific Inxxxxation (ISI) releases a snippet of data that they crave: the impact factor of each journal. In due course, bureaucrats in research agencies will roll the impact figures into their perxxxxance indicators, and those scientists who worry about such things will quietly note which journal's number wins them the most brownie points.

Attempts to quantify the quality of science are always fraught with difficulty, and the journal impact factors are among the few numbers to persist. The result is an overemphasis of what is really a limited metric.

To obtain the latest impact factors, which were released last week, the ISI number-crunchers added the total number of citations from all the monitored journals during 2004 to items in the journal of interest that were published in 2002 and 2003. They then divided that total by the number of 'citable items' — loosely, papers and review articles — that were published in the journal during those same two years.

The impact factor is taken by some administrators as a measure of the typical citation rate for the journal. But for many journals, it isn't 'typical' at all. Nature's latest impact factor is 32.2, an increase on last year and a high number that we're proud of, but it's one that merits a closer look.

For example, we have analysed the citations of individual papers in Nature and found that 89% of last year's figure was generated by just 25% of our papers.

The most cited Nature paper from 2002??03 was the mouse genome, published in December 2002. That paper represents the culmination of a great enterprise, but is inevitably an important point of reference rather than an expression of unusually deep mechanistic insight. So far it has received more than 1,000 citations. Within the measurement year of 2004 alone, it received 522 citations. Our next most cited paper from 2002??03 (concerning the functional organization of the yeast proteome) received 351 citations that year. Only 50 out of the roughly 1,800 citable items published in those two years received more than 100 citations in 2004. The great majority of our papers received fewer than 20 citations.

These figures all reflect just how strongly the impact factor is influenced by a small minority of papers — no doubt to a lesser extent in more specialized journals, but significantly nevertheless. However, we are just as satisfied with the value of our papers in the 'long tail' as with that of the more highly cited work.

The citation rate of our papers also varies sharply between disciplines. Many of Nature's papers in immunology published in 2003 have since received between 50 and 200 citations. Significant proportions of those in cancer and molecular and cell biology have been in the 50??150 range. But papers in physics, palaeontology and climatology typically achieved fewer than 50 citations. Clearly, these reflect differences in disciplinary dynamics, not in quality.

The impact factor also mixes citations to diverse types of content: unsurprisingly, review articles are typically the most highly cited, but citations of our Commentaries, News Features and News & Views articles also contribute in a minor way to the numerator (although these items are not counted in the denominator).

The net result of all these variables is a conclusion that impact factors don't tell us as much as some people may think about the respective quality of the science that journals are publishing. Neither do most scientists judge journals using such statistics; they rely instead on their own assessment of what they actually read.

None of this would really matter very much, were it not for the unhealthy reliance on impact factors by administrators and researchers' employers worldwide to assess the scientific quality of nations and institutions, and often even to judge individuals. There is no doubt that impact factors are here to stay. But these figures illustrate why they should be handled with caution.

--