Compute Nodes and Storage

CPU compute node: approximately $3,300 
There are 16 physical CPU cores, 64 GB memory, and 1 TB local disk space for a CPU compute node (see technical specifications). The compute node cost includes onefourth of the chassis, which houses four nodes. In practice, nodes will probably be purchased in groups of four. As hardware becomes more modern, we expect the specific models of the components to change, but for the overall configuration and cost to stay roughly the same.

GPU compute node: approximately $10,200 
GPU compute nodes have the same components as a CPU node and eight PNY GeForce GTX 980Ti GPUs with 2816 CUDA cores and 6GB of memory. The increase in price compared to the CPU node includes the GPUs and the differences in motherboard, chassis, power supply.

Storage: approximately $3,000 per 22 TB partition 
The base cluster includes two file servers that cost approximately $18,600 each and hold 208 TB of data. With 8.5% tax, the total was about $40,400.

Infrastructure

The cluster also contains important infrastructure components which were purchased using a mix of MPS and individual PI funds. For the base cluster, the Division of Mathematical and Physical Science contributed approximately $44,000. The shared equipment includes:

Head node: approximately $6,500 
The head node has 16 physical CPU cores, 276 GB memory, and 2 TB local disk space, and 10 GbE connection to the internet.

Networking: The head node and CPU nodes are networked using FDR Infiniband. For the initial configuration of the cluster, the networking components costs were approximately $39,700 (4 switches, 4 power supplies, 24 x 0.5 meter cables, 37 x 1 meter cables, 33 x 2 meter cables, 6 x 2 meter cables). The approximate cost for each of these components is: 
● Mellanox MSX6025F1SFS SwitchX2 FDR 36Port InfiniBand Switch , $7,145 ea. 
● Mellanox 300W Power Supply , $735 ea . 
● Mellanox FDR InfiniBand QSFP Passive Copper Cable, 0.5 meter , $50 ea. 
● Mellanox FDR InfiniBand QSFP Passive Copper Cable, 1 meter , $56 ea. 
● Mellanox FDR InfiniBand QSFP Passive Copper Cable, 2 meters , $64 ea. 
● Mellanox FDR InfiniBand QSFP Passive Copper Cable, 3 meters , $72 ea.

PDU (Power Distribution Units): approximately $530 
The power distribution units for the GPU compute nodes are given below. (I’m not sure what PDUs we are using for the other nodes.) 
● TrippLite PDU3VN3L1530 (for GPU nodes).