上海交大世界大学排行榜被质疑‘造假’

近日罗马尼亚学者Razvan V. Florian攥文质疑上交排行榜‘造假’。Razvan在其论文中指出，如果只根据上交在其网站提供的排名方法来处理原始数据，根本没可能重获上交的排行结果。其论文概要为

Abstract. I discuss the difficulties that I encountered in reproducing the results of the Shanghai ranking of world universities. In the Shanghai ranking the dependence between the score for the SCI indicator and the weighted number of considered articles obeys a power law, instead of the proportional dependence that is suggested by the official methodology of the ranking. Discrepancies from proportionality are also found in some of the scores for the N&S and Size indicators.

其后，作者根据上交网站阐述的方法试着计获SCI，N&S ，SIZE指标，结果都令人怀疑。Razvan曾写信质问上交原始数据和最后得分非线性关系的存在性，上交回信默认非线性关系是存在的。上交在其FAQ也曾提到:“the distribution of data for each indicator is examined for any significant distorting effect; standard statistical techniques are used to adjust the indicator if necessary”.

这么就好了，Razvan在论文就间接质问到：“It is however unclear why the authors of the Shanghai ranking considered that there was a “significant distorting effect” in the original data, and why they chose to distort the data in this particular nonlinear fashion.”

阴谋论:上交如果可以操纵所谓的"significant distorting effect"，上交排行榜又是权威（因为唯一）的话，那么就是说上交在世界高校界获有了一定的monopoly power。而其他中国大学争先发行所谓“具有世界影响力”的排行榜的动机则更是一目了然。

Razvan V. Florian最后在其总结中写道：

The data presented here suggests that the results of the Shanghai ranking are irreproducible. At least the data concerning the SCI indicator suggests that the authors of the Shanghai ranking deviated from the official published methodology when computing the scores of the universities.

......

In correspondence through email with the authors of the Shanghai ranking, they insist (in response to a draft version of this paper) that the results found here do not mean that the ranking results are irreproducible, since “any two institutions having the same amount of SCI articles, they should get the same scores in our ranking”. I believe that reproducibility means that, given correct raw data for one university inside top 500, one should be able to compute its score according to the published methodology, and this score should be equal to the score used in Shanghai ranking.

上交在其网站，曾自诩到

3. Is our academic ranking objective?

The quality of universities cannot be precisely measured by mere numbers. Therefore, any ranking is controversial and no ranking is absolutely objective. People should be cautious about any ranking including our Academic Ranking of World Universities.

It would be impossible to have a comprehensive ranking of universities worldwide, because of the huge differences of universities in the large variety of countries and the technical difficulties in obtaining internationally comparable data. Our ranking is using carefully selected indicators and internationally comparable data that everyone could check.

现在有人check了，大家看看上交又是怎么回复的。

From: Razvan Florian
To:
Sent: Wednesday, February 08, 2006

Hello,

I am sending you attached a paper that I wrote about the impossibility
of reproducing the results of the Shanghai ranking. I intend to publish
it in the Ad Astra online journal, http://www.ad-astra.ro/journal/?lang=en

We invite you to write a reply, that we will publish in the same journal.

With best regards,

Razvan Florian
Ad Astra association of Romanian scientists
www.ad-astra.ro

From: ranking
To: Razvan Florian
Date: 08/02/2006 08:31
Dear Razvan Florian,

Thank you very much for sending us your paper.

As you have mentioned in your paper, the data is fitted well with some non-linear power-law relations, which indicates that our data is reproducible. As you have already noticed, distribution of data has been examined for distorting effect and statistical treatment has been applied. You should question why the non-linear treatment is done, not the reproducibility itself. Therefore, I strongly urge you to change the title and conclusion of your paper.

The best way to verify the data of an institution in our ranking is to compare its scores and raw data on a particular indicator with its peer institutions. That is to say, if any two institutions having the same amount of SCI articles, they should get the same scores in our ranking. In fact, numerous institutions have done that and clarified with us. In any case, our data is reproducible. Again, I strongly urge you to revise your paper focusing on the methodology question of treatment methods, since that has nothing to do with reproducibility.

Sincerely yours,
The Ranking Team
Institute of Higher Education
Shanghai Jiao Tong University
Shanghai, China

相关连接：

1.Irreproducibility of the results of the Shanghai academic ranking of world universities

http://www.ad-astra.ro/journal/8/florian_shanghai_irreproducibility.pdf

2.国际权威数据大师A.F.J. van Raan狠批上交弱智榜

http://www.daiwen.com/news/shownews.asp?id=1050