IC message routing layer

Overview

The Internet Computer (IC) achieves its security and fault tolerance by replicating computation across node machines located in various independent data centers across the world. For scalability reasons, the Internet Computing Protocol (ICP) composes the IC of multiple independent subnets. Each subnet can be viewed as an independent replicated state machine that replicates its state over a subset of all the available nodes.

Roughly speaking, replication is achieved by having the two lower ICP layers (P2P & Consensus) agree on blocks containing batches of messages to be executed, and then having the two upper ICP layers (Message Routing & Execution) execute them. Blocks are organized as a chain, where each block builds on the previous block. Each block has an associated height in the chain and one can look at execution of a batch of messages corresponding to the agreed upon block at height [math]\displaystyle{ x }[/math] by the upper layers as taking the replicated state of version [math]\displaystyle{ x-1 }[/math], and "applying" the batch to it to obtain replicated state of version [math]\displaystyle{ x }[/math].

In this document we describe the role of the Message Routing layer in deterministic batch processing. Its responsibilities are:

• Coordinating the deterministic processing of batches: Fetching the right versions of the replicated state and the registry view to process the batch, triggering the deterministic processing, and committing the resulting replicated state.

• Deterministic processing of batches: Deterministic processing of batches relative to some replicated state and some registry view, resulting in an updated replicated state.

• Transferring message streams from one subnet to another: Moving streams from one subnet to another.

Remarks and Required Prior Knowledge

• The goal of this document is to provide the next level of detail compared to the material in the "How it works" section of internetcomputer.org. So it is recommended to study the material available there first.
• This page builds upon definitions made in the page describing the state manager. Please refer to this page for missing definitions related to the replicated state etc.
• Also see this and this blog post for some relevant and easier to digest background information.
• The documentation provided in this page may slightly deviate from the current implementation in terms of API as well as naming of functions, variables, etc. However, it still conveys the high-level ideas required to understand how the component itself works and how it interacts with other components. The implementation also contains several optimizations which are, however, not important for the conceptual overview here and therefore skipped.
• The notation used in this page is described here.

Replicated vs. Canonical State

While the external API functions defined in this document will always take state in its implementation specific representation, i.e., as ReplicatedState, we describe the operation the message routing component performs on the state based on its canonical representation, i.e., the CanonicalState. Given the relations between ReplicatedState and CanonicalState as defined in the specification of the state manager, this will implicitly define how an implementation needs to act on the respective parts of the ReplicatedState. We assume an implicit conversion from ReplicatedState to CanonicalState whenever we access some state passed to this component via an API function.

Guarantees Provided by Message Routing

Intuitively, the goal of the message routing layer is to enable transparent communication of canisters across subnets. This means that this layer formally does not add any guarantees the system provides, but simply needs to make sure that system invariants are preserved. Those system invariants include

• guaranteed replies (each canister-to-canister request will eventually receive a reply),

• canister-to-canister ordering (the order of canister-to-canister requests sent from one canister to another canister is preserved), and

• authenticity (only messages that come from canisters on the IC are processed).

To ensure that the system invariants hold, message routing needs to provide the following guarantees:

• Canister-to-canister messages will eventually be passed to the execution layer at the subnet the destination canister lives on exactly once.

• If a message can not be delivered, a synthetic reject response must be produced.

• If a canister [math]\displaystyle{ A }[/math] sends two messages [math]\displaystyle{ m_1 }[/math] and [math]\displaystyle{ m_2 }[/math] to a canister [math]\displaystyle{ B }[/math], then, if none of them gets synthetically rejected, it must be guaranteed that they are put in canister [math]\displaystyle{ B }[/math]'s input queue from [math]\displaystyle{ A }[/math] in that order.