Difference between revisions of "Node Provider Machine Hardware Guide"

Latest revision as of 15:45, 28 May 2024

Node Providers operate one or more node machines than run in the Internet Computer network.

Gen1 hardware requirements have been used by Node Providers to set up node machines during the Genesis launch.

The Gen2 hardware requirements have been defined for the further growth of the Internet Computer network. The specifications for the Gen2 node machines are generic (instead of vendor specific) and support VM memory encryption and attestation, which will be needed in future features on the Internet Computer.

While Gen2 Node Providers are only strictly required to follow the generic specifications (the IC-OS installation will fail if the generic specifications are not met), it is strongly recommended for Node Providers to purchase one of the validated configurations listed below.

Gen 2 Node Machine requirements

Gen-2.3 Generic specification

Quantity	Item description
2	Dual Socket AMD EPYC Milan CPU - Recommended: 7313 (16C/32T 3 Ghz) optionally 7343, 7373, 73F3
16	32GB RDIMM, 3200MT/s, Dual Rank
5	6.4TB NVMe Mixed Mode (DWPD >= 3) with U.2 or U.3 interface
1	Dual Port 10G SFP or BASE-T
1	TPM 2.0

Important: Do NOT order a RAID controller for your node machine. If a RAID controller is included in your setup, it will cause issues with the installation of IC-OS on your machine.

Dfinity-validated configurations

DFINITY has validated the following Gen2 hardware configurations.

Dell

Quantity	Item description
2	AMD EPYC 7343 3.2GHz, 16C/32T, 128M Cache (190W)
16	32GB RDIMM, 3200MT/s, Dual Rank 16Gb BASE x8
5	6.4TB Enterprise NVMe Mixed Use AG Drive U.2 Gen4 with carrier
1	PowerEdge R6525 Motherboard, with 2 x 1Gb Onboard LOM (BCM5720)MLK V2
2	Dual, Hot-plug, Redundant Power Supply (1+1) 1100W, Mixed Mode Titanium
1	Intel X710 Dual Port 10GbE SFP+, OCP NIC 3.0
1	Trusted Platform Module 2.0 V3

ASUS

Quantity	Item description
2	AMD EPYC 7313 (3,00 GHz, 16-Core, 128 MB)
16	32GB ECC Reg ATP DDR4 3200 RAM
5	6.4 TB NVMe Kioxia SSD 3D-NAND TLC U.3 (Kioxia CM6-V)
1	Asus Mainboard KMPP-D32 Series (without OCP 3.0, without Pike)
2	1600 Watt redundant PSU
1	Broadcom 25 Gigabit P225P SFP28 Dual Port Network Card
0	TPM 2.0*

* We (Dfinity) forget to order the TPM 2.0 on the ASUS test machine we validated. Your machine DOES need the TPM 2.0.

Supermicro

Quantity	Item description
2	AMD EPYC 7313 (3,00 GHz, 16-Core, 128 MB)
16	32GB ECC Reg DDR4 3200 RAM
5	6.4 TB NVMe TLC SSD, PCIe 4.0 x4, U.3 2.5", 3 DWPD
1	SuperMicro H12DSU-iN
2	1200 Watt redundant PSU
1	Dual Port 10GBase-T Network Adapter Intel® X710-TM4*
1	Dual Port 10GbE Network Adapter Intel® X710-TM4, SFP+*
1	Trusted Platform Module 2.0

* Only one dual port network adaptor is required.

Gigabyte

Quantity	Item description
2	AMD EPYC 7313 (3,00 GHz, 16-Core, 128 MB)
16	32GB ECC Reg DDR4 3200 RAM
5	6.4 TB NVMe TLC SSD, PCIe 4.0 x4, U.3 2.5", 3 DWPD
1	Gigabyte MZ92-FS1 Rev. 3.0
2	1200 Watt redundant PSU
1	Dual Port 1000Base-T Network Adapter Intel® I350-AM2*
1	Dual Port 10GbE Network Adapter, Broadcom NetXtreme E-Series*
1	Trusted Platform Module 2.0

* Only one dual port network adaptor is required.

Community-validated configurations

Node Providers have validated the following Gen2 hardware configurations:

Dell PowerEdge R7525

Quantity	Item description
2	AMD 7313 3.0GHz,16C/32T,128M,155W,3200
16	32GB RDIMM, 3200MT/s, Dual Rank 16Gb BASE x8
5	6.4TB Enterprise NVMe Mixed Use AG Drive U.2 Gen4 with carrier
1	PowerEdge R7525 Motherboard, with 2 x 1Gb Onboard LOM (BCM5720)MLK V2
2	Dual, Hot-plug, Power Supply Redundant (1+1), 800W, Mixed Mode, NAF
1	Intel X710 Dual Port 10GbE SFP+, OCP NIC 3.0
1	Trusted Platform Module 2.0 V3

Gen 1 Node Machine requirements

If you're a Node Provider acquiring machines to join the IC, use the Gen2 specifications listed above. This section is for Gen1 Node Providers who joined before Genesis.

SuperMicro - Gen-1.1

Quantity	Item description
1	AS - 1023US - TR4
2	Rome 7302 DP/UP 16C/32T 3.0
16	32GB DDR4-3200 2Rx4 ECC REG DIMM
5	Samsung PM983 3.2TB NVMe PCIE/SATA Hybrid M.2 & 1 PCIE
2	800W Power Supply
1	Std LP 2-port 10G RJ45, Intel x540
5	Micron 5300 PRO 7.4TB, SATA, 2.5', 3D TLC, .6DWPD (with Caddie)
1	C13/C14 13A Power Cord

Dell - Gen-1.1

Quantity	Item description
1	R6525
1	Chassis - Supports Up to 10 NVMe drives, 12 drives total
1	Dual 1 GB on Motherboard
3	Low Profile PCIe Slots
-	3 Year Basic NBD Support
-	iDrac Enterprise
1	Dual port 10GbE Base - T Adapter Broadcom, PCIe Low Profile
10	3.2TB NVMe, Mixed Use, 2.5" with Carrier
16	32GB RDIMM (3200MT/s)
2	AMD 7302 3GHz, 16C/32T, 128M, 155W, 3200
1	Single Power Supply (800W)
1	C13-C14, 3M, 125V 15A Power Cored

Dell - Gen-1.2

Quantity	Item description
1	R6515
1	3.5" Chassis with up to 4 Hot-Plug Hard Drives and OS RAID
1	Dual 1 Gb on Motherboard
3	Low Profile PCIe Slots
1	Standard Fan
-	3 Year Basic NBD Support
-	iDrac Enterprise
2	Dual Port 10GbE Base - T Adapter Broadcom, PCIe LOw Profile
2	480GB SSD SATA Mix Use 6Gbps 512 2.5in Hot-Plug AG Drive, 3.5in
4	8GB RDIMM, 3200 MT/s, Single Rank
1	AMD EPYC 7232P 3.10GHz, 8C/16T, 64M Cache (120W) DDR4-3200
1	Dual Hot-Plug Redundant Power Supply (1+1), 550W
2	Jumper Cord - C13/C14, .6M, 250V, 13A

Purchasing Hardware

Choose your node machine configuration
- Stick with the configurations above. Anything else is unsupported.
Use local and trusted supply chains
- If possible, purchase hardware locally from a trusted vendor to avoid global single points of failure and to reduce the risk that somebody tampers with your hardware during delivery
Where to buy?
- Supermicro
- Dell
- Gigabyte
- ASUS
Place your order
- Double check the components meet the generic specs above.

Lead times can be weeks to months. Plan accordingly with respect to onboarding.

Note: Ordering the hardware directly after the acceptance of your Node Provider and Data Center registration by the NNS is the safest option. However, in practice, if you are fully aware of the associated risks, ordering the hardware at an earlier stage can be done as well.

@@ Line 6: / Line 6: @@
 While Gen2 Node Providers are only strictly required to follow the generic specifications '''(the IC-OS installation will fail if the generic specifications are not met)''', it is strongly recommended for Node Providers to purchase one of the validated configurations listed below.
-'''Important:''' Do NOT order a RAID controller for your node machine. If a RAID controller is included in your setup, it will cause issues with the installation of IC-OS on your machine.
 == Gen 2 Node Machine requirements ==
-==== Generic specification ====
+==== Gen-2.3 Generic specification ====
 {| class="wikitable"
+|'''Quantity'''
+|'''Item description'''
 |-
+|2
 | Dual Socket AMD EPYC Milan CPU - Recommended: [https://en.wikichip.org/wiki/amd/epyc/7313 7313] (16C/32T 3 Ghz)
 [https://en.wikichip.org/wiki/amd/epyc#7003_Series_.28Zen_3.29 optionally] 7343, 7373, 73F3
 |-
-| 16x 32GB RDIMM, 3200MT/s, Dual Rank
+|16
+| 32GB RDIMM, 3200MT/s, Dual Rank
 |-
-| 5x 6.4TB NVMe Mixed Mode (DWPD >= 3)
+|5
+| 6.4TB NVMe Mixed Mode (DWPD >= 3) with U.2 or U.3 interface
 |-
+|1
 | Dual Port 10G SFP or BASE-T
 |-
+|1
 | TPM 2.0
 |}
+'''Important:''' Do NOT order a RAID controller for your node machine. If a RAID controller is included in your setup, it will cause issues with the installation of IC-OS on your machine.
 === Dfinity-validated configurations ===
@@ Line 28: / Line 34: @@
 ==== Dell ====
 {| class="wikitable"
+|'''Quantity'''
+|'''Item description'''
 |-
 | 2 || AMD EPYC 7343 3.2GHz, 16C/32T, 128M Cache (190W)
@@ Line 46: / Line 54: @@
 ==== ASUS ====
 {| class="wikitable"
+|'''Quantity'''
+|'''Item description'''
 |-
 | 2 || AMD EPYC 7313 (3,00 GHz, 16-Core, 128 MB)
@@ Line 62: / Line 72: @@
 |TPM 2.0*
 |}
-<nowiki>*</nowiki> Note Gen2 machines require TPM 2.0 hardware but this specific test machine did not include it.
+<nowiki>*</nowiki> We (Dfinity) forget to order the TPM 2.0 on the ASUS test machine we validated. Your machine DOES need the TPM 2.0.
 ==== Supermicro ====
 {| class="wikitable"
+|'''Quantity'''
+|'''Item description'''
 |-
 | 2 || AMD EPYC 7313 (3,00 GHz, 16-Core, 128 MB)
@@ Line 86: / Line 98: @@
 <nowiki>*</nowiki> Only one dual port network adaptor is required.
-==== Currently pending validation: Gigabyte ====
+==== Gigabyte ====
 {| class="wikitable"
+|'''Quantity'''
+|'''Item description'''
 |-
 | 2 || AMD EPYC 7313 (3,00 GHz, 16-Core, 128 MB)
@@ Line 113: / Line 127: @@
 ==== Dell PowerEdge R7525 ====
 {| class="wikitable"
+|'''Quantity'''
+|'''Item description'''
+|-
 |2
 |AMD 7313 3.0GHz,16C/32T,128M,155W,3200
@@ Line 136: / Line 153: @@
 == Gen 1 Node Machine requirements ==
-If you're a Node Provider acquiring machines to join the IC, use the Gen2 specifications listed  above. This section is for Gen1 Node Providers who joined before 2022.
+If you're a Node Provider acquiring machines to join the IC, use the Gen2 specifications listed above. This section is for Gen1 Node Providers who joined before Genesis.
-==== SuperMicro ====
+==== SuperMicro - Gen-1.1  ====
 {| class="wikitable"
+|'''Quantity'''
+|'''Item description'''
 |-
 | 1 || AS - 1023US - TR4
@@ Line 158: / Line 177: @@
 |}
-==== Dell - type 1 ====
+==== Dell - Gen-1.1  ====
 {| class="wikitable"
+|'''Quantity'''
+|'''Item description'''
 |-
 | 1 || R6525
@@ Line 187: / Line 208: @@
 |}
-==== Dell - type 2 ====
+==== Dell - Gen-1.2 ====
 {| class="wikitable"
+|'''Quantity'''
+|'''Item description'''
 |-
 | 1 || R6515
@@ Line 217: / Line 240: @@
 |}
-== Purchasing Hardware Guide ==
+== Purchasing Hardware ==
+# '''Choose your node machine configuration'''
+#* Stick with the configurations above. Anything else is unsupported.
+# '''Use local and trusted supply chains'''
+#* If possible, purchase hardware locally from a trusted vendor to avoid global single points of failure and to reduce the risk that somebody tampers with your hardware during delivery
+# '''Where to buy?'''
+#*[https://www.supermicro.com/en/wheretobuy Supermicro]
+#*[https://www.dell.com/en-us/shop/dell-poweredge-servers/sc/servers Dell]
+#*[https://www.gigabyte.com/WheretoBuy/Enterprise Gigabyte]
+#*[https://servers.asus.com/support/wheretobuy/117 ASUS]
+# '''Place your order'''
+#* Double check the components meet the generic specs above.
+Lead times can be weeks to '''months'''. Plan accordingly with respect to onboarding.
+Note: Ordering the hardware directly after the acceptance of your Node Provider and Data Center registration by the NNS is the safest option. However, in practice, if you are fully aware of the associated risks, ordering the hardware at an earlier stage can be done as well.