Difference between revisions of "Node Provider Machine Hardware Guide"

From Internet Computer Wiki
Jump to: navigation, search
(→‎Gen 1 Node Machine: Clarified which section to use)
(Adjusted node names)
 
(23 intermediate revisions by 3 users not shown)
Line 1: Line 1:
−
== What are the Hardware Requirements for Node Machines? ==
+
Node Providers operate one or more node machines than run in the Internet Computer network.  
−
Node providers operate one or more node machines than run in the IC network. Gen1 Hardware requirements have been used by Node Providers to set up node machines during Genesis launch.
 
  
−
The Gen2 Hardware requirements have been defined for the further growth of the IC network. The specifications for the Gen2 node machines are generic (instead of vendor specific) and support VM memory encryption and attestation, which will be needed in future features on the IC.
+
Gen1 hardware requirements have been used by Node Providers to set up node machines during the Genesis launch.  
  
−
Below are the up-to-date specifications for both the Gen2 node machines and Gen1 node machines.
+
The Gen2 hardware requirements have been defined for the further growth of the Internet Computer network. The specifications for the Gen2 node machines are generic (instead of vendor specific) and support VM memory encryption and attestation, which will be needed in future features on the Internet Computer.
−
== Gen 2 Node Machine ==
+
 
−
=== Generic specification ===
+
While Gen2 Node Providers are only strictly required to follow the generic specifications '''(the IC-OS installation will fail if the generic specifications are not met)''', it is strongly recommended for Node Providers to purchase one of the validated configurations listed below.  
 +
== Gen 2 Node Machine requirements ==
 +
==== Gen-2.3 Generic specification ====
 
{| class="wikitable"
 
{| class="wikitable"
 
|-
 
|-
Line 14: Line 15:
 
| 16x 32GB RDIMM, 3200MT/s, Dual Rank
 
| 16x 32GB RDIMM, 3200MT/s, Dual Rank
 
|-
 
|-
−
| 5x 6.4TB NVMe Mixed Mode (DWPD >= 3)
+
| 5x 6.4TB NVMe Mixed Mode (DWPD >= 3) with U.2 or U.3 interface
 
|-
 
|-
 
| Dual Port 10G SFP or BASE-T
 
| Dual Port 10G SFP or BASE-T
Line 20: Line 21:
 
| TPM 2.0
 
| TPM 2.0
 
|}
 
|}
 +
'''Important:''' Do NOT order a RAID controller for your node machine. If a RAID controller is included in your setup, it will cause issues with the installation of IC-OS on your machine.
  
−
=== Validated Configurations ===
+
=== Dfinity-validated configurations ===
 
DFINITY has [https://forum.dfinity.org/t/draft-motion-proposal-new-hardware-specification-and-remuneration-for-ic-nodes/14202/14?u=garym validated] the following Gen2 hardware configurations.
 
DFINITY has [https://forum.dfinity.org/t/draft-motion-proposal-new-hardware-specification-and-remuneration-for-ic-nodes/14202/14?u=garym validated] the following Gen2 hardware configurations.
 
==== Dell ====
 
==== Dell ====
Line 59: Line 61:
 
|TPM 2.0*
 
|TPM 2.0*
 
|}
 
|}
−
<nowiki>*</nowiki> Note Gen2 machines require TPM 2.0 hardware but this specific test machine did not include it.  
+
<nowiki>*</nowiki> We (Dfinity) forget to order the TPM 2.0 on the ASUS test machine we validated. Your machine DOES need the TPM 2.0. 
  
 
==== Supermicro ====
 
==== Supermicro ====
Line 81: Line 83:
 
| 1 || Trusted Platform Module 2.0
 
| 1 || Trusted Platform Module 2.0
 
|}
 
|}
−
==== Currently pending validation: Gigabyte ====
+
<nowiki>*</nowiki> Only one dual port network adaptor is required.
 +
 
 +
==== Gigabyte ====
 
{| class="wikitable"
 
{| class="wikitable"
 
|-
 
|-
Line 103: Line 107:
 
<nowiki>*</nowiki> Only one dual port network adaptor is required.
 
<nowiki>*</nowiki> Only one dual port network adaptor is required.
  
−
== Gen 1 Node Machine ==
+
=== Community-validated configurations ===
−
If you're a Node Provider acquiring machines to join the IC, use the Gen2 section above. This section is for Gen1 Node Providers who joined before 2022.  
+
Node Providers have validated the following Gen2 hardware configurations:
 +
 
 +
==== Dell PowerEdge R7525 ====
 +
{| class="wikitable"
 +
|2
 +
|AMD 7313 3.0GHz,16C/32T,128M,155W,3200
 +
|-
 +
|16
 +
|32GB RDIMM, 3200MT/s, Dual Rank 16Gb BASE x8
 +
|-
 +
|5
 +
|6.4TB Enterprise NVMe Mixed Use AG Drive U.2 Gen4 with carrier
 +
|-
 +
|1
 +
|PowerEdge R7525 Motherboard, with 2 x 1Gb Onboard LOM (BCM5720)MLK V2
 +
|-
 +
|2
 +
|Dual, Hot-plug, Power Supply Redundant (1+1), 800W, Mixed Mode, NAF
 +
|-
 +
|1
 +
|Intel X710 Dual Port 10GbE SFP+, OCP NIC 3.0
 +
|-
 +
|1
 +
|Trusted Platform Module 2.0 V3
 +
|}
 +
 
 +
== Gen 1 Node Machine requirements ==
 +
If you're a Node Provider acquiring machines to join the IC, use the Gen2 specifications listed above. This section is for Gen1 Node Providers who joined before Genesis.  
  
−
=== Node Machine Type 1 - Dell ===
+
==== SuperMicro - Gen-1.1  ====
 +
{| class="wikitable"
 +
|-
 +
| 1 || AS - 1023US - TR4
 +
|-
 +
| 2 || Rome 7302 DP/UP 16C/32T 3.0
 +
|-
 +
| 16 || 32GB DDR4-3200 2Rx4 ECC REG DIMM
 +
|-
 +
| 5 || Samsung PM983 3.2TB NVMe PCIE/SATA Hybrid M.2 & 1 PCIE
 +
|-
 +
| 2 || 800W Power Supply
 +
|-
 +
| 1 || Std LP 2-port 10G RJ45, Intel x540
 +
|-
 +
| 5 || Micron 5300 PRO 7.4TB, SATA, 2.5', 3D TLC, .6DWPD (with Caddie)
 +
|-
 +
| 1 || C13/C14 13A Power Cord
 +
|}
 +
 
 +
==== Dell - Gen-1.1  ====
 
{| class="wikitable"
 
{| class="wikitable"
 
|-
 
|-
Line 135: Line 186:
 
|}
 
|}
  
−
=== Node Machine Type 1 - SuperMicro ===
+
==== Dell - Gen-1.2 ====
−
{| class="wikitable"
 
−
|-
 
−
| 1 || AS - 1023US - TR4
 
−
|-
 
−
| 2 || Rome 7302 DP/UP 16C/32T 3.0
 
−
|-
 
−
| 16 || 32GB DDR4-3200 2Rx4 ECC REG DIMM
 
−
|-
 
−
| 5 || Samsung PM983 3.2TB NVMe PCIE/SATA Hybrid M.2 & 1 PCIE
 
−
|-
 
−
| 2 || 800W Power Supply
 
−
|-
 
−
| 1 || Std LP 2-port 10G RJ45, Intel x540
 
−
|-
 
−
| 5 || Micron 5300 PRO 7.4TB, SATA, 2.5', 3D TLC, .6DWPD (with Caddie)
 
−
|-
 
−
| 1 || C13/C14 13A Power Cord
 
−
|}
 
−
 
 
−
=== Node Machine Type 2 - Dell ===
 
 
{| class="wikitable"
 
{| class="wikitable"
 
|-
 
|-
Line 184: Line 215:
 
| 2 || Jumper Cord - C13/C14, .6M, 250V, 13A
 
| 2 || Jumper Cord - C13/C14, .6M, 250V, 13A
 
|}
 
|}
 +
 +
== Purchasing Hardware ==
 +
# '''Choose your node machine configuration'''
 +
#* Stick with the configurations above. Anything else is unsupported.
 +
# '''Use local and trusted supply chains'''
 +
#* If possible, purchase hardware locally from a trusted vendor to avoid global single points of failure and to reduce the risk that somebody tampers with your hardware during delivery
 +
# '''Where to buy?'''
 +
#*[https://www.supermicro.com/en/wheretobuy Supermicro]
 +
#*[https://www.dell.com/en-us/shop/dell-poweredge-servers/sc/servers Dell]
 +
#*[https://www.gigabyte.com/WheretoBuy/Enterprise Gigabyte]
 +
#*[https://servers.asus.com/support/wheretobuy/117 ASUS]
 +
# '''Place your order'''
 +
#* Double check the components meet the generic specs above.
 +
 +
 +
 +
Lead times can be weeks to '''months'''. Plan accordingly with respect to onboarding.
 +
 +
Note: Ordering the hardware directly after the acceptance of your Node Provider and Data Center registration by the NNS is the safest option. However, in practice, if you are fully aware of the associated risks, ordering the hardware at an earlier stage can be done as well.

Latest revision as of 16:58, 29 September 2023

Node Providers operate one or more node machines than run in the Internet Computer network.

Gen1 hardware requirements have been used by Node Providers to set up node machines during the Genesis launch.

The Gen2 hardware requirements have been defined for the further growth of the Internet Computer network. The specifications for the Gen2 node machines are generic (instead of vendor specific) and support VM memory encryption and attestation, which will be needed in future features on the Internet Computer.

While Gen2 Node Providers are only strictly required to follow the generic specifications (the IC-OS installation will fail if the generic specifications are not met), it is strongly recommended for Node Providers to purchase one of the validated configurations listed below.

Gen 2 Node Machine requirements

Gen-2.3 Generic specification

Dual Socket AMD EPYC Milan CPU - Recommended: 7313 (16C/32T 3 Ghz)

optionally 7343, 7373, 73F3

16x 32GB RDIMM, 3200MT/s, Dual Rank
5x 6.4TB NVMe Mixed Mode (DWPD >= 3) with U.2 or U.3 interface
Dual Port 10G SFP or BASE-T
TPM 2.0

Important: Do NOT order a RAID controller for your node machine. If a RAID controller is included in your setup, it will cause issues with the installation of IC-OS on your machine.

Dfinity-validated configurations

DFINITY has validated the following Gen2 hardware configurations.

Dell

2 AMD EPYC 7343 3.2GHz, 16C/32T, 128M Cache (190W)
16 32GB RDIMM, 3200MT/s, Dual Rank 16Gb BASE x8
5 6.4TB Enterprise NVMe Mixed Use AG Drive U.2 Gen4 with carrier
1 PowerEdge R6525 Motherboard, with 2 x 1Gb Onboard LOM (BCM5720)MLK V2
2 Dual, Hot-plug, Redundant Power Supply (1+1) 1100W, Mixed Mode Titanium
1 Intel X710 Dual Port 10GbE SFP+, OCP NIC 3.0
1 Trusted Platform Module 2.0 V3

ASUS

2 AMD EPYC 7313 (3,00 GHz, 16-Core, 128 MB)
16 32GB ECC Reg ATP DDR4 3200 RAM
5 6.4 TB NVMe Kioxia SSD 3D-NAND TLC U.3 (Kioxia CM6-V)
1 Asus Mainboard KMPP-D32 Series (without OCP 3.0, without Pike)
2 1600 Watt redundant PSU
1 Broadcom 25 Gigabit P225P SFP28 Dual Port Network Card
0 TPM 2.0*

* We (Dfinity) forget to order the TPM 2.0 on the ASUS test machine we validated. Your machine DOES need the TPM 2.0.

Supermicro

2 AMD EPYC 7313 (3,00 GHz, 16-Core, 128 MB)
16 32GB ECC Reg DDR4 3200 RAM
5 6.4 TB NVMe TLC SSD, PCIe 4.0 x4, U.3 2.5", 3 DWPD
1 SuperMicro H12DSU-iN
2 1200 Watt redundant PSU
1 Dual Port 10GBase-T Network Adapter Intel® X710-TM4*
1 Dual Port 10GbE Network Adapter Intel® X710-TM4, SFP+*
1 Trusted Platform Module 2.0

* Only one dual port network adaptor is required.

Gigabyte

2 AMD EPYC 7313 (3,00 GHz, 16-Core, 128 MB)
16 32GB ECC Reg DDR4 3200 RAM
5 6.4 TB NVMe TLC SSD, PCIe 4.0 x4, U.3 2.5", 3 DWPD
1 Gigabyte MZ92-FS1 Rev. 3.0
2 1200 Watt redundant PSU
1 Dual Port 1000Base-T Network Adapter Intel® I350-AM2*
1 Dual Port 10GbE Network Adapter, Broadcom NetXtreme E-Series*
1 Trusted Platform Module 2.0

* Only one dual port network adaptor is required.

Community-validated configurations

Node Providers have validated the following Gen2 hardware configurations:

Dell PowerEdge R7525

2 AMD 7313 3.0GHz,16C/32T,128M,155W,3200
16 32GB RDIMM, 3200MT/s, Dual Rank 16Gb BASE x8
5 6.4TB Enterprise NVMe Mixed Use AG Drive U.2 Gen4 with carrier
1 PowerEdge R7525 Motherboard, with 2 x 1Gb Onboard LOM (BCM5720)MLK V2
2 Dual, Hot-plug, Power Supply Redundant (1+1), 800W, Mixed Mode, NAF
1 Intel X710 Dual Port 10GbE SFP+, OCP NIC 3.0
1 Trusted Platform Module 2.0 V3

Gen 1 Node Machine requirements

If you're a Node Provider acquiring machines to join the IC, use the Gen2 specifications listed above. This section is for Gen1 Node Providers who joined before Genesis.

SuperMicro - Gen-1.1

1 AS - 1023US - TR4
2 Rome 7302 DP/UP 16C/32T 3.0
16 32GB DDR4-3200 2Rx4 ECC REG DIMM
5 Samsung PM983 3.2TB NVMe PCIE/SATA Hybrid M.2 & 1 PCIE
2 800W Power Supply
1 Std LP 2-port 10G RJ45, Intel x540
5 Micron 5300 PRO 7.4TB, SATA, 2.5', 3D TLC, .6DWPD (with Caddie)
1 C13/C14 13A Power Cord

Dell - Gen-1.1

1 R6525
1 Chassis - Supports Up to 10 NVMe drives, 12 drives total
1 Dual 1 GB on Motherboard
3 Low Profile PCIe Slots
- 3 Year Basic NBD Support
- iDrac Enterprise
1 Dual port 10GbE Base - T Adapter Broadcom, PCIe Low Profile
10 3.2TB NVMe, Mixed Use, 2.5" with Carrier
16 32GB RDIMM (3200MT/s)
2 AMD 7302 3GHz, 16C/32T, 128M, 155W, 3200
1 Single Power Supply (800W)
1 C13-C14, 3M, 125V 15A Power Cored

Dell - Gen-1.2

1 R6515
1 3.5" Chassis with up to 4 Hot-Plug Hard Drives and OS RAID
1 Dual 1 Gb on Motherboard
3 Low Profile PCIe Slots
1 Standard Fan
- 3 Year Basic NBD Support
- iDrac Enterprise
2 Dual Port 10GbE Base - T Adapter Broadcom, PCIe LOw Profile
2 480GB SSD SATA Mix Use 6Gbps 512 2.5in Hot-Plug AG Drive, 3.5in
4 8GB RDIMM, 3200 MT/s, Single Rank
1 AMD EPYC 7232P 3.10GHz, 8C/16T, 64M Cache (120W) DDR4-3200
1 Dual Hot-Plug Redundant Power Supply (1+1), 550W
2 Jumper Cord - C13/C14, .6M, 250V, 13A

Purchasing Hardware

  1. Choose your node machine configuration
    • Stick with the configurations above. Anything else is unsupported.
  2. Use local and trusted supply chains
    • If possible, purchase hardware locally from a trusted vendor to avoid global single points of failure and to reduce the risk that somebody tampers with your hardware during delivery
  3. Where to buy?
  4. Place your order
    • Double check the components meet the generic specs above.


Lead times can be weeks to months. Plan accordingly with respect to onboarding.

Note: Ordering the hardware directly after the acceptance of your Node Provider and Data Center registration by the NNS is the safest option. However, in practice, if you are fully aware of the associated risks, ordering the hardware at an earlier stage can be done as well.

Retrieved from "https://wiki.internetcomputer.org/w/index.php?title=Node_Provider_Machine_Hardware_Guide&oldid=6470"