Manual Node Recovery Guide

This runbook describes what steps node providers need to take during an NNS recovery.

Security warning

⚠️⚠️⚠️ Don’t get tricked into compromising your nodes. Only complete a manual node recovery if all of the following conditions are met:

• A subnet recovery is announced on the Internet Computer Status Page
• The DFINITY team reached out on the dedicated Matrix channel #ic-node-providers-incident-response:matrix.org.
  • Only the DFINITY team is able to send messages on this channel. In case of an incident, permissions are adapted so that everyone can send messages.

Prerequisites

• The recovery coordinator should have communicated with you the following:
  • The recovery parameters:
    • The VERSION: the commit ID of the recovery-GuestOS update image
    • The VERSION-HASH: the SHA256 sum of the recovery-GuestOS update image.
    • The RECOVERY-HASH: the SHA256 sum of the recovery.tar.zst
  • The node(s): which specific nodes managed by the NP/NO are part of the target subnet.
• Obtain console access to all nodes you run that are part of the target subnet.
  • Note that the recovery can be completed from a physical console or from the node's remote BMC virtual console view.