Node Provider Onboarding

To participate in the Internet Computer network as a Node Provider and receive the rewards for supporting the network.

Requirements

• Node Hardware
• Rack space with a 10GB connectivity, RJ45 terminated on the nodes
• Public /29 IPv4 range and /64 IPv6 range
• Hardware wallet
• NitroKey HSM
• 11 ICP (10 of which are to be staked for the NNS proposal deposit)
• Basic understanding of neurons, staking, and governance proposals. For instance, understanding what it means to stake a neuron for 8 years.
• The technical knowledge to understand some minor steps that are not explicitly mentioned in these instructions. For instance, when to insert an HSM.

Install the required tools

Install ic-admin

ic-admin is the tool used to create and submit NNS proposals.

MacOS

curl "https://download.dfinity.systems/blessed/ic/0ef2aebde4ff735a1a93efa342dcf966b6df5061/nix-release/x86_64-darwin/ic-admin.gz" -o - | gunzip > ./ic-admin
chmod +x ./ic-admin

Verify the binary

diff <(shasum -a 256 ./ic-admin | cut -d' ' -f1) <(echo c18d55b3961bcca6d0bfa0a7e7d9a4e6d4daf74ced64b0767db13d53c1f16cb4) && echo "ic-admin checksum matches" || echo "***ERROR***: ic-admin checksum does not match"

Verify that the version is 1.0 or greater

./ic-admin --version
ic-admin 1.0

Linux

NOTE: The instructions below have been tested with the Ubuntu 20.04 release.

curl "https://download.dfinity.systems/blessed/ic/0ef2aebde4ff735a1a93efa342dcf966b6df5061/release/ic-admin.gz" -o - | gunzip > ./ic-admin
$ chmod +x ./ic-admin

Verify the binary

diff <(shasum -a 256 ./ic-admin | cut -d' ' -f1) <(echo c4c93df7b015742ecadf62d44f8287ba3ee960f98832fc5a0c648bfd9acf834e) && echo "ic-admin checksum matches" || echo "***ERROR***: ic-admin checksum does not match"

Verify that the version is 1.0 or greater

./ic-admin --version
ic-admin 1.0

Install dfx

dfx allows generating a neuron hotkey, among other things

$ sh -ci "$(curl -fsSL https://sdk.dfinity.org/install.sh)"

Verify that the version is 0.8.1 or greater

$ export PATH=$HOME/bin:$PATH
$ dfx --version
dfx 0.8.1

Create a wallet hotkey principal

$ dfx identity new node-provider-hotkey

Creating identity: "node-provider-hotkey".
Created identity: "node-provider-hotkey".

$ NODE_PROVIDER_PRINCIPAL=$(dfx --identity node-provider-hotkey identity get-principal)
$ echo $NODE_PROVIDER_PRINCIPAL
fharn-5vyi2-4xb4a-64yyi-3jpmj-pga23-mxy25-d5uim-fqcro-eoefh-tae

Create and Manage Neuron via NNS Frontend Dapp and Internet Identity

1. Setup your hardware wallet: https://medium.com/dfinity/integrating-ledger-nano-with-the-nns-front-end-dapp-user-manual-9c5600925e16
2. Send at least 11 ICPs to the hardware wallet address.
3. Navigate to Neurons tab and create a Neuron by staking at least 10 ICP from your hardware wallet. Staking more ICP works as well, but 10 is the minimum.
4. IMPORTANT! Confirming the transaction on your hardware wallet.

   

5. After the Neuron has been created successfully, confirm to add NNS Dapp as hotkey in the dialogue and on your hardware wallet, and close the dialog after the action completes.

   

6. Set the dissolve delay to at least 6 months, and confirm the choice in the dialogue and on your hardware wallet. After the action completes, you can close the "Follow Neurons".

   

7. You will now see a Neuron listed with its ID. Copy the Neuron ID, since you will need the it in the next steps to place the necessary proposals.

   

Add a hotkey

1. Select the Neuron you just created to open Neuron management view and press “Add hotkey” button.

   

2. A dialog will pop up where you can enter the principal you generated in step 2 (output from command dfx --identity node-provider-hotkey identity get-principal). Press the confirm button and confirm the transactions on your hardware wallet.
   

   

3. Get the hardware principal id

Navigate back to ICP page and select your hardware wallet account.

Here you can get your node provider principal by clicking on the copy icon after the principal id. You'll need it in the next steps.

Configure your HSM

It's first necessary to install the necessary tools.

MacOS

1. Download this OpenSC binary: https://github.com/OpenSC/OpenSC/releases/download/0.22.0/OpenSC-0.22.0.dmg
2. Double click the DMG image that you downloaded and then double click the OpenSC PKG file.
3. If your system doesn't allow the installation software from an unidentified developer please follow these steps or contact your system administrator:

   Choose the Apple menu > System Preferences > click Security and Privacy.

   Click the lock Icon to unlock it, then enter an administrator name and password.

   Ensure that you're on the tab named “General”.

   You should see the OpenSC app and you should be able to enable its installation by choosing “Open anyway”.

4. Click continue and install until the installation is complete.

Linux

NOTE: The instructions below have been tested with the Ubuntu 20.04 release.

1. Install pcscd and opensc

   sudo add-apt-repository universe
   sudo apt update
   sudo apt install pcscd opensc
   

Setup the HSM

Initialize the HSM.

sc-hsm-tool --initialize --so-pin 3537363231383830 --pin 358138

Change the HSM so pin.
WARNING: The new HSM so pin must have 16 hexadecimal digits. This is not very well known, and some HSM users have lost access to a Nitrokey HSM because they tried using regular characters and the command below accepted it.

pkcs11-tool --login --login-type so --so-pin 3537363231383830 --change-pin

Create a keypair on the HSM. Enter the default pin 358138 when prompted.
Note: Before initializing the HSM key please refer to the Nitrokey HSM documentation if you wish to create a backup. Creating a backup of the HSM device is NOT possible after the key has already been created.

pkcs11-tool -k --key-type EC:prime256v1 --login -d 01

Get the node operator principal from the HSM

Configure dfx identity (skip this step if you already configured it for an other HSM).

Note: Depending on your installation, the path to the --hsm-pkcs11-lib-path might be different on your platform. You can locate the correct path with the following command:

find / -name opensc-pkcs11.so 2> /dev/null

MacOS:

dfx identity new node-operator-hsm --hsm-key-id 01 --hsm-pkcs11-lib-path /Library/OpenSC/lib/opensc-pkcs11.so

Linux:

dfx identity new node-operator-hsm --hsm-key-id 01 --hsm-pkcs11-lib-path /usr/lib/x86_64-linux-gnu/opensc-pkcs11.so

Get the principal.

$ NODE_OPERATOR_PRINCIPAL=$(DFX_HSM_PIN=358138 dfx --identity node-operator-hsm identity get-principal)
$ echo $NODE_OPERATOR_PRINCIPAL

uqquy-76uhn-2mys5-xa3j3-oynfv-rrorc-pygn7-dldbd-4dr6n-lbhz7-zqe

Register your NP principal to the network

Replace the --proposer argument value with your neuron ID from the NNS Frontend Dapp, --node-provider-pid with your NP principal that you got from the command dfx --identity node-provider-hotkey identity get-principal, and "My Company", with the name of the entity that will provide the nodes.

IMPORTANT: Please make sure that in the --summary you include a link to the forum discussion, your company's web page, and/or to another place that can convince the voting community that you are making a legitimate request.

NODE_PROVIDER_NAME="My Company"
NODE_PROVIDER_PRINCIPAL=$(dfx --identity node-provider-hotkey identity get-principal)
NEURON_ID=13419667327548602649  # Coming from the NNS FE dapp https://nns.ic0.app/
./ic-admin \
        --nns-url https://nns.ic0.app \
        -s ~/.config/dfx/identity/node-provider-hotkey/identity.pem \
    propose-to-add-or-remove-node-provider add \
        --proposer $NEURON_ID \
        --proposal-title "Register a node provider '${NODE_PROVIDER_NAME}'" \
        --summary "Register a node provider '${NODE_PROVIDER_NAME}', in line with the announcement and discussion at https://forum.dfinity.org/t/..." \
        --node-provider-pid "$NODE_PROVIDER_PRINCIPAL"

Find the proposal on https://dashboard.internetcomputer.org/governance and wait until it is executed before proceeding to next step.

Ensure that your datacenter is registered in the network

Search for your data center on https://dashboard.internetcomputer.org/centers. If you found the datacenter that is hosting your nodes, remember its ID, and skip the following section. Otherwise, proceed with the registrion of a new DC.

Create a data center record

Replace the --proposer argument value with your neuron ID from the NNS Frontend Dapp. Replace the JSON fields from –data-centers-to-add argument and their corresponding values in --summary with: "id"

The ID should be combination of two letters representing a city that your datacenter is in, and an incrementing number. Search data center IDs on https://dashboard.internetcomputer.org, and find a combination of two letters and a number that’s not yet registered.

Examples:

• dl1 (Dallas, no IDs with “dl” prefix)
• zh10 (Zurich, numbers 0-9 are already registered)

"region"

Region represents the local region of a datacenter and is formulated as a three-part string divided by commas. The three parts making the string are continent, country code, and region, in the given order.

Examples:

• North America,US,Florida
• Europe,DE,Bavaria
• Asia,SG,Singapore

"owner" The entity that provides your datacenter facilities. Search https://dashboard.internetcomputer.org for existing data center providers. If there’s match, make sure you use the same exact some name for your datacenter. Otherwise, name the data center owner to your best knowledge.

"gps" Find your datacenter on https://www.google.com/maps/. Right click on location, and select the GPS coordinates (first item in the menu) in order to copy them.

NEURON_ID=13419667327548602649  # Coming from the NNS FE dapp https://nns.ic0.app/
$ ./ic-admin \
        --nns-url https://nns.ic0.app \
        -s ~/.config/dfx/identity/node-provider-hotkey/identity.pem \
    propose-to-add-or-remove-data-centers \
        --summary "Register a Flexential datacenter as dl1 in North America,US,Texas" \
        --skip-confirmation \
        --proposer $NEURON_ID \
        --data-centers-to-add '{
            "id": "dl1",
            "region": "North America,US,Texas",
            "owner": "Flexential",
            "gps": [
                33.00803, -96.66614
            ]
        }'

Find the proposal on https://dashboard.internetcomputer.org/governance and wait until it's executed before proceeding to next step.

Create a node operator record

Replace the NEURON_ID variable value with your neuron ID obtained from the NNS frontend dapp.
Replace the DC_ID variable value with id of your datacenter.
Replace the NODE_ALLOWANCE variable value with number of nodes you are providing.

NODE_PROVIDER_PRINCIPAL=$(dfx --identity node-provider-hotkey identity get-principal)
NODE_OPERATOR_PRINCIPAL=$(DFX_HSM_PIN=358138 dfx --identity node-operator-hsm identity get-principal)
NEURON_ID=13419667327548602649  # Coming from the NNS FE dapp https://nns.ic0.app/
NODE_PROVIDER_NAME="My Company"
NODE_ALLOWANCE=28
DC_ID=dl1

./ic-admin \
        --nns-url https://nns.ic0.app \
        -s ~/.config/dfx/identity/node-provider-hotkey/identity.pem \
    propose-to-add-node-operator \
        $NODE_PROVIDER_PRINCIPAL \
        --summary "Node provider '$NODE_PROVIDER_NAME' is adding $NODE_ALLOWANCE nodes in the $DC_ID data center" \
        --proposer $NEURON_ID \
        --node-operator-principal-id $NODE_OPERATOR_PRINCIPAL \
        --node-allowance $NODE_ALLOWANCE \
        --dc-id $DC_ID

Find the proposal on https://dashboard.internetcomputer.org/governance and wait until it's executed before proceeding to next step.

Configure firewall rules

Note: Until we provide a better way to configure the firewall rules, feel free to reach out to us e.g. on the public forum and ask for assistance in submitting proposal for the firewall configuration.

Replace the NEURON_ID variable value with your neuron ID obtained from the NNS frontend dapp.
Replace the NODES_IPV6_PREFIX variable value with IPv6 prefix of the network of your nodes.

NEURON_ID=13419667327548602649  # Coming from the NNS FE dapp https://nns.ic0.app/
NODES_IPV6_PREFIX=2001:4d78:700:10a::/64
NODE_OPERATOR_PRINCIPAL=$(DFX_HSM_PIN=358138 dfx --identity node-operator-hsm identity get-principal)

NFTABLES=$(./ic-admin --nns-url "https://nns.ic0.app" get-firewall-config | grep "firewall_config" | cut -d':' -f2 | cut -c2- | rev | cut -c2- | rev | xargs printf)
IPV6_PREFIXES=$(./ic-admin --nns-url "https://nns.ic0.app" get-firewall-config | tr -d '\n' | grep -oE 'ipv6_prefixes: \[[^]]+' | cut -d'[' -f2 | tr -d '"' | tr -d ' ' | tr -d '\n'; echo $NODES_IPV6_PREFIX)

./ic-admin \
        --nns-url https://nns.ic0.app \
        -s ~/.config/dfx/identity/node-provider-hotkey/identity.pem \
    propose-to-set-firewall-config \
        --proposer $NEURON_ID \
        --summary "Set the firewall rules for node operator ${NODE_OPERATOR_PRINCIPAL}" \
        <(echo $NFTABLES) \
        - \
        $IPV6_PREFIXES

Find the proposal on https://dashboard.internetcomputer.org/governance and wait until it’s executed before proceeding to next step.

Onboard nodes

Follow the instructions to onboard new nodes.

• For Dell Servers
• For Supermicro Servers

Verify that all the nodes were successfully onboarded by checking their status on the dashboard is set to either “Up” or “Unassigned”, or by checking the output from ic-admin get-topology command.

The internal dashboard can be searched by your provider principal.

Set the reward configuration for your nodes

Replace the NEURON_ID variable value with your neuron ID obtained from the NNS frontend dapp.
Replace the <NODE_X_PRINCIPAL> placeholders with your node principals.
Replace the <number-of-nodes> placeholder with the number of nodes you listed.

Note: The current maximum number of nodes per node operator are 28.

NEURON_ID=13419667327548602649  # Coming from the NNS FE dapp https://nns.ic0.app/
NODE_OPERATOR_PRINCIPAL=$(DFX_HSM_PIN=358138 dfx --identity node-operator-hsm identity get-principal)

./ic-admin \
        --nns-url https://nns.ic0.app \
        -s ~/.config/dfx/identity/node-provider-hotkey/identity.pem \
    propose-to-update-node-operator-config \
        --proposer $NEURON_ID \
        --summary "Set rewards for the following nodes:

        * <NODE_1_PRINCIPAL>
        * <NODE_2_PRINCIPAL>
        * ...
        " \
        --node-operator-id $NODE_OPERATOR_PRINCIPAL \
        --rewardable-nodes '{"type0": <number-of-nodes>}'