Difference between revisions of "Not all transactions are equal"
(Update the numbers and add links as references) |
|||
| (6 intermediate revisions by 2 users not shown) | |||
| Line 1: | Line 1: | ||
Whilst it’s typical for blockchains to flaunt metrics around transactions per second (TX/s) or transactions per day (TX/d), comparisons between blockchains only make sense when those transactions are roughly equivalent i.e. TX/s comparisons only make sense to compare within a single problem domain. | Whilst it’s typical for blockchains to flaunt metrics around transactions per second (TX/s) or transactions per day (TX/d), comparisons between blockchains only make sense when those transactions are roughly equivalent i.e. TX/s comparisons only make sense to compare within a single problem domain. | ||
| − | The ICP is a blockchain which aims to provide a general purpose world computer, capable of serving real web services directly to user's browsers without any need for web2 cloud providers. Individual transactions on the ICP can be considered to be richer and | + | The ICP is a blockchain which aims to provide a general purpose world computer, capable of serving real web services directly to user's browsers without any need for web2 cloud providers. Individual transactions on the ICP can be considered to be richer and typically do more computation than most other blockchains. |
This page aims to explain the differences between the work performed by transactions on the ICP vs those on Ethereum. | This page aims to explain the differences between the work performed by transactions on the ICP vs those on Ethereum. | ||
| Line 9: | Line 9: | ||
To compare the overall throughput of the two blockchains (i.e., how many ops per second can be handled), one needs to make several assumptions. First, it is assumed that the simpler EVM instructions (e.g., add, mul, div etc.) are roughly equivalent to the Wasm instructions of the same type, both kinds being translated to a similar x86 instruction executed by the hardware. The comparison is much more complex and not apples-to-apples for the more complex operations. For a proper comparison here one would need to either (1) thoroughly understand the design of both execution layers or (2) run a similar program/benchmark on both blockchains and compare their overall performance. These two options are time-consuming and would lead to longer-term research efforts. | To compare the overall throughput of the two blockchains (i.e., how many ops per second can be handled), one needs to make several assumptions. First, it is assumed that the simpler EVM instructions (e.g., add, mul, div etc.) are roughly equivalent to the Wasm instructions of the same type, both kinds being translated to a similar x86 instruction executed by the hardware. The comparison is much more complex and not apples-to-apples for the more complex operations. For a proper comparison here one would need to either (1) thoroughly understand the design of both execution layers or (2) run a similar program/benchmark on both blockchains and compare their overall performance. These two options are time-consuming and would lead to longer-term research efforts. | ||
| − | For a quicker comparison, it can be assumed that all EVM instructions are equal in terms of gas cost (and also assume no fees are involved). Since ETH is currently burning approximately ''' | + | For a quicker comparison, it can be assumed that all EVM instructions are equal in terms of gas cost (and also assume no fees are involved). Since ETH is currently burning approximately '''108.3B gas units per day''' (https://ycharts.com/indicators/ethereum_gas_used_per_day, on Sept 16 2024), and assuming each instruction costs 1 gas unit (which vastly ''underestimates'' the costs of memory access operations), it is clear that the ETH blockchain is running less than 109B instructions per day. |
| − | The IC | + | The IC executed more than 110 billion replicated Wasm instructions per second on Sept 16 2024. Under the simplifying assumption that all instructions are comparable, this means the IC runs the daily number of ETH instructions in less than 1 second. |
| − | + | Ethereum executed about about 1.102M transactions on September 16, 2024 ([https://ycharts.com/indicators/ethereum_transactions_per_day#:~:text=Ethereum%20Transactions%20Per%20Day%20is,completed%20on%20the%20Ethereum%20network. https://ycharts.com/indicators/ethereum_transactions_per_day.] ), which means that there are on average 0.098M instructions per transaction that day. On the IC, the dashboard shows that 110 billion instructions/s were executed for about 8500 replicated calls/s, amounting to an average of more than 12.9M instructions per call. Comparing the work intensity of the two blockchains, taking the ratio of instructions per transaction ICP performs around 130x the amount of computational work of Ethereum per transaction. It's important to note that the multiplier is calculated only considering replicated calls as these are the interactions that carry out ETH equivalent work. | |
| − | To compare the two networks in terms of efficiency, one also needs to consider the replication factor. In ICP the typical replication factor is 13 versus approximately | + | To compare the two networks in terms of efficiency, one also needs to consider the replication factor. In ICP the typical replication factor is 13 versus approximately 1.5M for Ethereum (a number that is steadily increasing https://beaconscan.com/stat/validator). Concluding, ICP is at least 15.5 million times more efficient than Ethereum. |
| − | Concluding, ICP is at least | ||
| − | == EdDSA verification | + | == ETH vs ICP EdDSA verification == |
To get a view on the validity of the above calculations in a real-world setting, comparisons can be made by running a given function. A realistic function that is used often in the blockchain setting is signature verification. | To get a view on the validity of the above calculations in a real-world setting, comparisons can be made by running a given function. A realistic function that is used often in the blockchain setting is signature verification. | ||
| + | |||
| + | Previous work from the Ethereum Foundation estimates the validation of an EdDSA signature to cost ~500k in Gas [[https://ethresear.ch/t/verify-ed25519-signatures-cheaply-on-eth-using-zk-snarks/13139 source]]. One way to get a comparison on the IC is to create a canister, import the [https://docs.rs/ed25519/latest/ed25519/ Rust ed25519 library] and test verification by creating a signature on a hash of an arbitrary message and using that for verification. Counting cycles burned before and after this call, discounting the base cost (i.e., cycles charged for ingress and for running an update call) results in a cycle cost of 4,211,120. | ||
| + | |||
| + | Putting a dollar cost on this comparison, with a conservative assumption that 1 Gas costs 40GWEI and 1 ETH being ~USD1800 means the cost of an EdDSA verification on Ethereum currently costs USD36. Considering the cycle cost (4,211,120) on the IC with an XDR rate of USD1.3476 yields a cost of USD0.00000567490 to run an EdDSA verification on the IC. Overall, this suggests that the IC is 6,343,718 times less costly for a standard computation. | ||
[[L1 comparison]] | [[L1 comparison]] | ||
Latest revision as of 17:54, 17 September 2024
Whilst it’s typical for blockchains to flaunt metrics around transactions per second (TX/s) or transactions per day (TX/d), comparisons between blockchains only make sense when those transactions are roughly equivalent i.e. TX/s comparisons only make sense to compare within a single problem domain.
The ICP is a blockchain which aims to provide a general purpose world computer, capable of serving real web services directly to user's browsers without any need for web2 cloud providers. Individual transactions on the ICP can be considered to be richer and typically do more computation than most other blockchains.
This page aims to explain the differences between the work performed by transactions on the ICP vs those on Ethereum.
ETH vs. ICP execution throughput
Both ETH and ICP are able to run (general-purpose) smart contracts. At the execution layer, contracts are translated to a lower-level virtual machine interpretable language. These are EVM in the case of ETH and a Wasm-compatible runtime in the ICP case. Both EVM and Wasm instructions include arithmetic instructions (e.g., add, mul, div), but also more smart-contract specific instructions (e.g., reading and writing memory). The latter are in general more expensive operations in terms of consumed resources, which is then translated to the amount of gas used for each opcode of ETH and cycles used for ICP.
To compare the overall throughput of the two blockchains (i.e., how many ops per second can be handled), one needs to make several assumptions. First, it is assumed that the simpler EVM instructions (e.g., add, mul, div etc.) are roughly equivalent to the Wasm instructions of the same type, both kinds being translated to a similar x86 instruction executed by the hardware. The comparison is much more complex and not apples-to-apples for the more complex operations. For a proper comparison here one would need to either (1) thoroughly understand the design of both execution layers or (2) run a similar program/benchmark on both blockchains and compare their overall performance. These two options are time-consuming and would lead to longer-term research efforts. For a quicker comparison, it can be assumed that all EVM instructions are equal in terms of gas cost (and also assume no fees are involved). Since ETH is currently burning approximately 108.3B gas units per day (https://ycharts.com/indicators/ethereum_gas_used_per_day, on Sept 16 2024), and assuming each instruction costs 1 gas unit (which vastly underestimates the costs of memory access operations), it is clear that the ETH blockchain is running less than 109B instructions per day.
The IC executed more than 110 billion replicated Wasm instructions per second on Sept 16 2024. Under the simplifying assumption that all instructions are comparable, this means the IC runs the daily number of ETH instructions in less than 1 second.
Ethereum executed about about 1.102M transactions on September 16, 2024 (https://ycharts.com/indicators/ethereum_transactions_per_day. ), which means that there are on average 0.098M instructions per transaction that day. On the IC, the dashboard shows that 110 billion instructions/s were executed for about 8500 replicated calls/s, amounting to an average of more than 12.9M instructions per call. Comparing the work intensity of the two blockchains, taking the ratio of instructions per transaction ICP performs around 130x the amount of computational work of Ethereum per transaction. It's important to note that the multiplier is calculated only considering replicated calls as these are the interactions that carry out ETH equivalent work.
To compare the two networks in terms of efficiency, one also needs to consider the replication factor. In ICP the typical replication factor is 13 versus approximately 1.5M for Ethereum (a number that is steadily increasing https://beaconscan.com/stat/validator). Concluding, ICP is at least 15.5 million times more efficient than Ethereum.
ETH vs ICP EdDSA verification
To get a view on the validity of the above calculations in a real-world setting, comparisons can be made by running a given function. A realistic function that is used often in the blockchain setting is signature verification.
Previous work from the Ethereum Foundation estimates the validation of an EdDSA signature to cost ~500k in Gas [source]. One way to get a comparison on the IC is to create a canister, import the Rust ed25519 library and test verification by creating a signature on a hash of an arbitrary message and using that for verification. Counting cycles burned before and after this call, discounting the base cost (i.e., cycles charged for ingress and for running an update call) results in a cycle cost of 4,211,120.
Putting a dollar cost on this comparison, with a conservative assumption that 1 Gas costs 40GWEI and 1 ETH being ~USD1800 means the cost of an EdDSA verification on Ethereum currently costs USD36. Considering the cycle cost (4,211,120) on the IC with an XDR rate of USD1.3476 yields a cost of USD0.00000567490 to run an EdDSA verification on the IC. Overall, this suggests that the IC is 6,343,718 times less costly for a standard computation.