信息不少，摘要如下：

Visit to the National University for Defense Technology Changsha, China

June 3, 2013

At the IHPCF workshop Xiangke Liao from NUDT presented details on the new Tianhe-
2 (TH-2) also called the Milkyway-2 supercomputer. The project is sponsored by the 863
High Technology Program of the Chinese Government, the Government of Guangdong
province, and the Government of Guangzhou city. The system will be in the National
Supercomputer Center in Guangzhou (NSCC-GZ) by the end of the year. It will provide
an open platform for research and education and provide high performance computing
service for southern China. At the end of the first day of IHPCF there was a tour of the
TH-2 computer room. The IHPCF participants had an opportunity to see the system and
receive addition information on various components of the computer and its operation.

。。。

For comparison, the next large acquisition of a supercomptuer for the US Department of
Energy will not be until 2015.

第一能保持到15年？

。。。

While the TH-2 system is based on Intel multicore (Ivy Bridge) and coprocessors (Xeon
Phi), there are a number of features of the TH-2 that are Chinese in origin, unique and
interesting, including the TH-Express 2 interconnection network, the Galaxy FT-1500 16-
core processor, the OpenMC programming model,their high density package, the
apparent reliability and scalability of the system.

。。。

Power
and
Cooling
The peak power consumption under load for the system is at 17.6 MWs. This is just for
the processors, memory and interconnect network. If cooling is added the total power
consumption is 24 MWs.

。。。

The Frontend processors
In addition to the compute nodes there is a frontend system composed of 4096 Galaxy
FT-1500 CPUs. These processors were designed and developed at NUDT. They are not
considered as part of the compute system. The FT-1500 is 16 cores and based on
SparcV9.

。。。

The Interconnect
NUDT has built their own proprietary interconnect called the TH Express-2 interconnect
network. The TH Express-2 uses a fat tree topology with 13 switches each of 576 ports at
the top level. This is an optoelectronics hybrid transport technology. Running a
proprietary network. The interconnect uses their own chip set. The high radix router
ASIC called NRC has a 90 nm feature size with a 17.16x17.16 mm die and 2577 pins.
The throughput of a single NRC is 2.56 Tbps. The network interface ASIC called NIC
has the same feature size and package as the NIC, the die size is 10.76x10.76 mm, 675
pins and uses PCI-E G2 16X. A broadcast operation via MPI was running at 6.36 GB/s
and the latency measured with 1K of data within 12,000 nodes is about 9 us.

互聯網絡是關鍵之一，俺們超算中心用的是infiniBand (300gbps)，TH Express-2看來牛多了。

。。。

The Software Stack
The Tianhe-2 is using Kylin Linux as the operating system. Kylin is an operating system
developed by the National University for Defense Technology, and successfully
approved by China’s 863 Hi-tech Research and Development Program office in
2006. See http://en.wikipedia.org/wiki/Kylin_(operating_system) for addition details.
Kylin is compatible with other mainstream operating systems and supports multiple
microprocessors and computers of different structures. The Kylin packages all include
standard open source and public packages. This is the same OS used in the Tianhe-1A.
Resource management is based on SLURM. They have a power-aware resource
allocation and use multiple custom scheduling polices.
There are Fortran, C, C++, and Java compilers, OpenMP, and MPI 3.0 based on MPICH
version 3.0.4 with custom GLEX (Galaxy Express) Channel support. They can do multichannel
message data transfers, dynamic flow control and have offload collective
operations. In addition, they are developed something called OpenMC. It is a directive
based intra-node programming model. Think of it as a way to use OpenMC instead of
Open-MP and either CUDA, OpenACC, or OpenCL. This new abstraction for hardware
and software provides for a unified logical layer above all computing including CPU
cores and Xeon Phi processors but could be extended to architectures with similar ISA
and heterogeneous processors. They provide directives for high efficient SIMD
operations and directives for high efficiency data locality exploitation and data
communication. Open-MC is still a work in progress.

麒麟操作系統，剛出來時被不少人嚴重鄙視之，原來是為天河配套的。呵呵。

失望，軟件還是MPI主打，討厭那東西，只能湊合用。OpenMC是directive based，跟OpenMP一個意思。不合口味。

They are using the Intel ICC 13.0.0 compiler. They claim to have a math library, which
are based on Intel’s MKL 11.0.0 and BLAS for the GPU based on Xeon Phi and
optimization by the NUDT.

再次看到軟件生態環境的重要，用人家的芯片，人家的軟件支持好。選用那款芯片是綜合考慮的。

接下來一張表，對比橡樹嶺的cray xk7，天河2各指標完爆之

。。。

Applications
NUDT claims to have a number of applications that are being ported to the TH-2. The list
includes:
• High-order CFD Simulation：HostA
o Complex flow simulation of the full plane：C919
o WCNS - Weighted Compact Nonlinear Scheme
o 10 billion grid points
o Over 1000 compute nodes
o 1 MIC= 70% of 2 Ivy Bridge

大飛機真是整個產業鏈的提升啊，短板太多肯定搞不定的。

• Gyrokinetic Toroidal Code：GTC
o Porting to TH-2 using CPU+MIC （TH-1A: CPU+GPU）
o Over 4096 compute nodes
o 1 MIC= 80% of 2 Ivy Bridge

• Business Opinion Analysis
o Store and process 600TB structured/non structured data with Hadoop on
1024 nodes
o Process 100 Million data records per day

決策支持，呵呵，不會是中國稜鏡吧？Hadoop是美國稜鏡也用的

• Security e-Government Cloud
o 512 FT-1500 nodes
o Increase server utility from 30% to 71%

這是啥？

後記：哪位同學想要完整PDF，寫信給俺，通報馬甲。