Internet Computer Wiki - User contributions [en]

IC Smart Contract Memory

2023-12-18T18:26:35Z

Sb: Correct number of GiB of stable storage

==Overall Architecture==
[[File:Screen Shot 2022-12-01 at 14.41.33.png|512px|thumb|Figure 1. The two memories that can be accessed by the canister smart contracts.]]
Canister smart contracts running on the Internet Computer (IC) store data just like most other programs would. To this end, the IC offers developers two types of memory where data can be stored, as depicted in Figure 1. The first is the regular '''heap memory''' that is exposed as the Web Assembly virtual machine heap. This should be used as a scratch, temporary memory that will be cleared after any canister upgrade. The second type of memory is the '''stable memory''', which is a larger memory used for permanent data storage.

==Orthogonal Persistence==

<syntaxhighlight lang="rust">
use ic_cdk_macros::{query, update};
use std::{cell::RefCell, collections::HashMap};

thread_local! {
static STORE: RefCell<HashMap<String, u64>> = RefCell::default();
}

#[update]
fn insert(key: String, value: u64) {
STORE.with(|store| store.borrow_mut().insert(key, value));
}

#[query]
fn lookup(key: String) -> u64 {
STORE.with(|store| *store.borrow().get(&key).unwrap_or(&0))
}
</syntaxhighlight>

The IC offers orthogonal persistence, an illusion given to programs to run forever: the heap of each canister is automatically preserved and restored the next time it is called. For that, the execution environment needs to determine efficiently which memory pages have been dirtied during message execution so that the modified pages are tracked and periodically persisted to disk. The listing above shows an example key-value store that illustrates how easy it is to use orthogonal persistence. The key-value store in this case is backed by a simple Rust HashMap stored on the heap by means of a thread-local variable. A RefCell is used to provide interior mutability. The example would also be possible without it, but mutating the thread-local variable would be unsafe in that case, as the Rust compiler cannot guarantee exclusive access to it.

==Heap Memory==
Canisters running on the IC are programmed either in Rust or Motoko. The canisters are then compiled down to web assembly (Wasm). All the variables and data structures defined in these higher-level languages are then stored in the Wasm heap. All accesses to data structures and variables defined in the higher-level languages are then translated to memory copy operations in Wasm (e.g., load, store, copy, grow). The Wasm heap memory is a 4GiB, 32-bit address space that backs the Wasm programs. Due to possible changes in data structures and in Wasm (and high-level language) compilers, the heap should not be used as a permanent memory, but rather as a (faster) scratch, temporary memory. This is because during canister upgrades, the heap layout might change (i.e., data structure layouts) which could leave the canister in an unusable state. However, in-between updates the heap memory is persisted thanks to orthogonal persistence.

==Stable Memory==
Next to the heap memory, canister developers can make use of the stable memory. This is an additional 64-bit addressable memory, which is currently 96GiB in size, with plans to increase it further in the future. Programs written in either Rust or Motoko need to explicitly use stable memory by using the API. This API offers primitives to copy memory back and forth between the Wasm heap and the stable memory. An alternative to using this lower level API directly is to use the stable structures API, which offers developers a collection of Rust data structures (e.g., B-trees) that operate directly in stable memory. Next to using the stable memory through stable data structures, a pattern often used by developers is to persist heap state between canister upgrades. This is achieved via serializing heap memory (or data structures), saving it to stable memory and applying the opposite operations (copying back and deserializing) when the upgrade is done.

==Behind the scenes: Implementation==
To serve memory contents to canister smart contracts, the IC software stack has the following design. First, it is important to mention that every N (consensus) rounds, canister state (heap, stable memory and other data structures) are checkpointed on disk. This is called a checkpoint file. Whenever a canister executes messages after a checkpoint, all its memory resides in the checkpoint file. Therefore, all memory requested will be served from the checkpoint file. Memory modifications (i.e., dirtied pages in terms of operating systems) are saved in a data structure called the heap delta. The following paragraphs describe how this design enables orthogonal persistence.

[[File:Screen_Shot_2022-12-01_at_14.30.58.png|512px|thumb|Figure 2. The memory faulting architecture which encompasses the checkpoint file and the heap delta.
.]]

Any implementation of orthogonal persistence has to solve two problems: (1) How to map the persisted memory into the Wasm memory?; and (2) How to keep track of all modifications in the Wasm memory so that they can be persisted later. Page protection is used to solve both problems.The entire address space of the Wasm memory is divided into 4KiB pages. All pages are initially marked as inaccessible using the page protection flags of the operating system.

The first memory access triggers a page fault, pauses the execution, and invokes a signal handler. The signal handler then fetches the corresponding page from persisted memory and marks the page as read-only. Subsequent read accesses to that page will succeed without any help from the signal handler. The first write access will trigger another page fault, however, and allow the signal handler to remember the page as modified and mark the page as readable and writable. All subsequent accesses to that page (both r/w) will succeed without invoking the signal handler.

Invoking a signal handler and changing page protection flags are expensive operations. Messages that read or write large chunks of memory cause a storm of such operations, degrading performance of the whole system. This can cause severe slowdowns under heavy load.

==Versioning: Heap Delta and Checkpoint Files==

A canister executes update messages sequentially, one by one. Queries, in contrast, can run concurrently to each other and to update messages. The support for concurrent execution makes the memory implementation much more challenging. Imagine that a canister is executing an update message at (blockchain) block height H. At the same time, there could still be a previous long-running query that started earlier, at block height H-K. This means the same canister can have multiple versions of its memory active at the same time; this is used for the parallel execution of queries and update calls.

A naive solution to this problem would be to copy the entire memory after each update message. That would be slow and use too much storage. Thus, our implementation takes a different route. It keeps track of the modified memory pages in a persistent tree data-structure called Heap Delta that is based on Fast Mergeable Integer Maps. At a regular interval (i.e., every N rounds), there is a checkpoint event that commits the modified pages into the checkpoint file after cloning the file to preserve its previous version. Figure 2 shows how the Wasm memory is constructed from Heap Delta and the checkpoint file.

====Memory-related performance optimizations====
'''Optimization 1:''' Memory mapping the checkpoint file pages.
This reduces the memory usage by sharing the pages between multiple calls being executed concurrently. This optimization also improves performance by avoiding page copying on read accesses. The number of signal handler invocations remains the same as before, so the issue of signal storms is still open after this optimization.

'''Optimization 2:''' Page Tracking in Queries
All pages dirtied by a query are discarded after execution. This means that the signal handler does not have to keep track of modified pages for query calls. As opposed to update calls, queries saw the introduction of a fast path that marks pages as readable and writable on the first access. This low-hanging fruit optimization made queries 1.5x-2x faster on average.

'''Optimization 3:''' Amortized Prefetching of Pages
The idea behind the most impactful optimization is simple: to reduce the number of page faults, more work is needed per signal handler invocation. Instead of fetching a single page at a time, the signal handler tries to speculatively prefetch pages. The right balance is required here because prefetching too many pages may degrade performance of small messages that access only a few pages. The optimization computes the largest contiguous range of accessed pages immediately preceding the current page. It uses the size of the range as a hint for prefetching more pages. This way the cost of prefetching is amortized by previously accessed pages. As a result, the optimization reduces the number of page faults in memory intensive messages by an order of magnitude.

A downside of this approach is that prefetched page content needs to be compared with previous content after message execution to determine if a page was modified instead of relying on tracking write accesses via signal handlers.

These optimizations bring substantial benefits for the performance of the memory faulting component of the execution environment. The optimizations allow the IC to improve its throughput for memory-intensive workloads.

==See Also==
* '''The Internet Computer project website (hosted on the IC): [https://internetcomputer.org/ internetcomputer.org]'''

Node Provider Machine Hardware Guide

2023-04-24T16:50:49Z

Sb:

== What are the Hardware Requirements for Node Machines? ==
Node providers operate one or more node machines than run in the IC network. Gen1 Hardware requirements have been used by Node Providers to set up node machines during Genesis launch.

The Gen2 Hardware requirements have been defined for the further growth of the IC network. The specifications for the Gen2 node machines are generic (instead of vendor specific) and supports VM memory encryption and attestation which will be needed in future features on the IC.

Below are the up-to-date specifications for both the Gen2 node machines and Gen1 node machines.

== Gen 2 Node Machine ==

=== Generic specification Gen2 Node Machine ===
{| class="wikitable"
|-
| Dual Socket AMD EPYC 7313 Milan 16C/32T 3 Ghz, 32K/512K/128M
optionally 7343, 7373, 73F3
|-
| 16x 32GB RDIMM, 3200MT/s, Dual Rank
|-
| 5x 6.4TB NVMe Mixed Mode (DWPD >= 3)
|-
| Dual Port 10G SFP or BASE-T
|-
| TPM 2.0
|}
 
Note the NVMe drives should be recognized by Linux as NVMe (i.e., show up as `/dev/nvme*` devices). SATA backplanes or any other hardware which prevents this should not be used.
 
=== Validated Configurations ===
DFINITY has [https://forum.dfinity.org/t/draft-motion-proposal-new-hardware-specification-and-remuneration-for-ic-nodes/14202/14?u=garym validated] the following Gen2 hardware configurations.
 
==== Validated configuration: Dell ====
{| class="wikitable"
|-
| 2 || AMD EPYC 7343 3.2GHz, 16C/32T, 128M Cache (190W)
|-
| 16 || 32GB RDIMM, 3200MT/s, Dual Rank 16Gb BASE x8
|-
| 5 || 6.4TB Enterprise NVMe Mixed Use AG Drive U.2 Gen4 with carrier
|-
| 1 || PowerEdge R6525 Motherboard, with 2 x 1Gb Onboard LOM (BCM5720)MLK V2
|-
| 2 || Dual, Hot-plug, Redundant Power Supply (1+1) 1100W, Mixed Mode Titanium
|-
| 1 || Intel X710 Dual Port 10GbE SFP+, OCP NIC 3.0
|-
| 1 || Trusted Platform Module 2.0 V3
|}
 

==== Validated configuration: ASUS ====
{| class="wikitable"
|-
| 2 || AMD EPYC 7313 (3,00 GHz, 16-Core, 128 MB)
|-
| 16 || 32GB ECC Reg ATP DDR4 3200 RAM
|-
| 5 || 6.4 TB NVMe Kioxia SSD 3D-NAND TLC U.3 (Kioxia CM6-V)
|-
| 1 || Asus Mainboard KMPP-D32 Series (without OCP 3.0, without Pike)
|-
| 2 || 1600 Watt redundant PSU
|-
| 1 || Broadcom 25 Gigabit P225P SFP28 Dual Port Network Card
|}
 
==== Validated configurations: Supermicro & Gigabyte ====
Validation is being re-run on Supermicro and Gigabyte machines which match the spec. This section will be updated when those results are ready.

== Gen 1 Node Machine ==
=== Node Machine Type 1 - Dell ===
{| class="wikitable"
|-
| 1 || R6525
|-
| 1 || Chassis - Supports Up to 10 NVMe drives, 12 drives total
|-
| 1 || Dual 1 GB on Motherboard
|-
| 3 || Low Profile PCIe Slots
|-
| - || 3 Year Basic NBD Support
|-
| - || iDrac Enterprise
|-
| 1 || Dual port 10GbE Base - T Adapter Broadcom, PCIe Low Profile
|-
| 10 || 3.2TB NVMe, Mixed Use, 2.5" with Carrier
|-
| 16 || 32GB RDIMM (3200MT/s)
|-
| 2 || AMD 7302 3GHz, 16C/32T, 128M, 155W, 3200
|-
| 1 || Single Power Supply (800W)
|-
| 1 || C13-C14, 3M, 125V 15A Power Cored
|-
|}
 
=== Node Machine Type 1 - SuperMicro ===
{| class="wikitable"
|-
| 1 || AS - 1023US - TR4
|-
| 2 || Rome 7302 DP/UP 16C/32T 3.0
|-
| 16 || 32GB DDR4-3200 2Rx4 ECC REG DIMM
|-
| 5 || Samsung PM983 3.2TB NVMe PCIE/SATA Hybrid M.2 & 1 PCIE
|-
| 2 || 800W Power Supply
|-
| 1 || Std LP 2-port 10G RJ45, Intel x540
|-
| 5 || Micron 5300 PRO 7.4TB, SATA, 2.5', 3D TLC, .6DWPD (with Caddie)
|-
| 1 || C13/C14 13A Power Cord
|}
 
=== Node Machine Type 2 - Dell ===
{| class="wikitable"
|-
| 1 || R6515
|-
| 1 || 3.5" Chassis with up to 4 Hot-Plug Hard Drives and OS RAID
|-
| 1 || Dual 1 Gb on Motherboard
|-
| 3 || Low Profile PCIe Slots
|-
| 1 || Standard Fan
|-
| - || 3 Year Basic NBD Support
|-
| - || iDrac Enterprise
|-
| 2 || Dual Port 10GbE Base - T Adapter Broadcom, PCIe LOw Profile
|-
| 2 || 480GB SSD SATA Mix Use 6Gbps 512 2.5in Hot-Plug AG Drive, 3.5in
|-
| 4 || 8GB RDIMM, 3200 MT/s, Single Rank
|-
| 1 || AMD EPYC 7232P 3.10GHz, 8C/16T, 64M Cache (120W) DDR4-3200
|-
| 1 || Dual Hot-Plug Redundant Power Supply (1+1), 550W
|-
| 2 || Jumper Cord - C13/C14, .6M, 250V, 13A
|}

Internet Computer performance

2021-11-10T16:42:26Z

Sb:

This post describes the DFINITY Foundation's performance evaluation of the Internet Computer. We will periodically update the numbers on this page to reflect performance improvements realized over time.

Scalability of the Internet Computer comes from partitioning the IC into subnetworks, aka subnets. Subnets process update calls from ingress messages independently from other subnets. They can scale up by adding more subnets at the cost of having more network traffic (as applications then need to potentially communicate across subnets). In its current form, the IC should be able to scale out to hundreds of subnets.

Query calls are read-only calls that are processed locally on each node. Scalability comes from adding more nodes, either to an existing subnet (at the cost of making consensus i.e. update calls more expensive) or as new subnet.

== Test setup ==

We are running all of our experiments concurrently against all subnets other than the NNS and some of the most utilized application subnets to avoid disturbance of active IC users.
We send load against those subnets directly and are not using boundary nodes for those experiments. Boundary nodes have additional rate limiting which is currently set slightly more conservative compared to what the IC can handle and running against them therefore is unsuitable for performance evaluation.
We are targeting all nodes in every subnet concurrently, much the same as what boundary nodes would be doing if we would use them.

We have installed one counter canister in every subnet. This counter canister is essentially a no-op canister. It only maintains a counter, which can be queries via a query call and incremented via update call. The counter value is not using orthogonal persistence, so the overhead for the execution layer of the IC is minimal. Stressing the counter canister can be seen as a way to determine the system overhead or baseline performance.

== Measurements ==
=== Update calls ===

The Internet Computer can currently sustain more than '''11_000 updates/second''' for a period of four minutes, with peaks over '''11_500 updates/second.'''

The update calls we have been measuring here are triggered from Ingress messages sent from outside the IC.

[[File:update-call-performance.png|1024px|Update Call Performance]]

=== Query calls ===
Arguably more important are query calls, since they contribute with more than 90% of the traffic we are observing on the IC.

[[File:query-call-performance.png|1024px|Query Call Performance]]

The Internet Computer can currently process up to '''250,000 queries per second.'''
During our experiments, we increment the load incrementally and run each load for a period of 5 minutes.

== Conclusion and next steps ==

The Internet Computer today already shows impressive performance. On top of that, it should be possible to further scale out the IC by:

* More subnets: This will immediate increase the query and update throughput. While adding subnets might eventually lead to other scalability problems, the IC in its current shape should be able to support hundreds of subnets.
* Performance improvements: Performance can also be improved by better single machine, network and consensus performance tuning. Increasing the performance by at least an order of magnitude should be possible.