上海交大世界大學排行榜被質疑‘造假’

近日羅馬尼亞學者Razvan V. Florian攥文質疑上交排行榜‘造假’。Razvan在其論文中指出，如果只根據上交在其網站提供的排名方法來處理原始數據，根本沒可能重獲上交的排行結果。其論文概要為

Abstract. I discuss the difficulties that I encountered in reproducing the results of the Shanghai ranking of world universities. In the Shanghai ranking the dependence between the score for the SCI indicator and the weighted number of considered articles obeys a power law, instead of the proportional dependence that is suggested by the official methodology of the ranking. Discrepancies from proportionality are also found in some of the scores for the N&S and Size indicators.

其後，作者根據上交網站闡述的方法試着計獲SCI，N&S ，SIZE指標，結果都令人懷疑。Razvan曾寫信質問上交原始數據和最後得分非線性關係的存在性，上交回信默認非線性關係是存在的。上交在其FAQ也曾提到:“the distribution of data for each indicator is examined for any significant distorting effect; standard statistical techniques are used to adjust the indicator if necessary”.

這麼就好了，Razvan在論文就間接質問到：“It is however unclear why the authors of the Shanghai ranking considered that there was a “significant distorting effect” in the original data, and why they chose to distort the data in this particular nonlinear fashion.”

陰謀論:上交如果可以操縱所謂的"significant distorting effect"，上交排行榜又是權威（因為唯一）的話，那麼就是說上交在世界高校界獲有了一定的monopoly power。而其他中國大學爭先發行所謂“具有世界影響力”的排行榜的動機則更是一目了然。

Razvan V. Florian最後在其總結中寫道：

The data presented here suggests that the results of the Shanghai ranking are irreproducible. At least the data concerning the SCI indicator suggests that the authors of the Shanghai ranking deviated from the official published methodology when computing the scores of the universities.

......

In correspondence through email with the authors of the Shanghai ranking, they insist (in response to a draft version of this paper) that the results found here do not mean that the ranking results are irreproducible, since “any two institutions having the same amount of SCI articles, they should get the same scores in our ranking”. I believe that reproducibility means that, given correct raw data for one university inside top 500, one should be able to compute its score according to the published methodology, and this score should be equal to the score used in Shanghai ranking.

上交在其網站，曾自詡到

3. Is our academic ranking objective?

The quality of universities cannot be precisely measured by mere numbers. Therefore, any ranking is controversial and no ranking is absolutely objective. People should be cautious about any ranking including our Academic Ranking of World Universities.

It would be impossible to have a comprehensive ranking of universities worldwide, because of the huge differences of universities in the large variety of countries and the technical difficulties in obtaining internationally comparable data. Our ranking is using carefully selected indicators and internationally comparable data that everyone could check.

現在有人check了，大家看看上交又是怎麼回復的。

From: Razvan Florian
To:
Sent: Wednesday, February 08, 2006

Hello,

I am sending you attached a paper that I wrote about the impossibility
of reproducing the results of the Shanghai ranking. I intend to publish
it in the Ad Astra online journal, http://www.ad-astra.ro/journal/?lang=en

We invite you to write a reply, that we will publish in the same journal.

With best regards,

Razvan Florian
Ad Astra association of Romanian scientists
www.ad-astra.ro

From: ranking
To: Razvan Florian
Date: 08/02/2006 08:31
Dear Razvan Florian,

Thank you very much for sending us your paper.

As you have mentioned in your paper, the data is fitted well with some non-linear power-law relations, which indicates that our data is reproducible. As you have already noticed, distribution of data has been examined for distorting effect and statistical treatment has been applied. You should question why the non-linear treatment is done, not the reproducibility itself. Therefore, I strongly urge you to change the title and conclusion of your paper.

The best way to verify the data of an institution in our ranking is to compare its scores and raw data on a particular indicator with its peer institutions. That is to say, if any two institutions having the same amount of SCI articles, they should get the same scores in our ranking. In fact, numerous institutions have done that and clarified with us. In any case, our data is reproducible. Again, I strongly urge you to revise your paper focusing on the methodology question of treatment methods, since that has nothing to do with reproducibility.

Sincerely yours,
The Ranking Team
Institute of Higher Education
Shanghai Jiao Tong University
Shanghai, China

相關連接：

1.Irreproducibility of the results of the Shanghai academic ranking of world universities

http://www.ad-astra.ro/journal/8/florian_shanghai_irreproducibility.pdf

2.國際權威數據大師A.F.J. van Raan狠批上交弱智榜

http://www.daiwen.com/news/shownews.asp?id=1050