Blockchain Expert Proposes TPS/$ Metric to Combat Misleading Performance Claims

The metrics used to measure blockchain performance can often be misleading, leading to exaggerated claims about the efficiency and capabilities of various networks. As the number of blockchain networks continues to grow, there is an increasing need for clear, evidence-based metrics that can accurately differentiate between them. Steven Pu, the co-founder of Taraxa, has called for a new metric to combat these misleading performance claims, proposing a metric called TPS/$.

Pu highlighted the challenges in accurately comparing blockchain performance due to the reliance on overly optimistic assumptions rather than real-world results. He emphasized that the industry lacks reliable benchmarks for real-world efficiency, with many projects relying on hype-driven popularity and exaggerated performance figures. This makes it difficult for developers and users to make informed decisions about which blockchain networks to use or invest in.

One of the key issues Pu addressed is the misuse of the Transactions Per Second (TPS) metric. TPS is often used as a marketing tactic to make blockchain performance sound more appealing than it might be under real-world conditions. Pu explained that decentralized networks are complex systems that need to be considered as a whole, and that focusing on a single metric like TPS can be misleading. He noted that blockchain projects with extreme claims on single metrics may have compromised decentralization, security, and accuracy.

To illustrate the extent of these inflated metrics, Pu conducted a study that found a significant discrepancy between the theoretical and real-world TPS of 22 permissionless and single-shard networks. On average, the theoretical metric was 20 times higher than the maximum observed mainnet TPS. This discrepancy is a result of the highly speculative and narrative-driven nature of the crypto market, where projects often use theoretical estimates or unrealistic assumptions to arrive at inflated metrics.

In response to these exaggerated metrics, Pu developed his own performance measure called TPS/$. This metric assesses performance based on verifiable TPS achieved on a network’s live mainnet while also considering hardware efficiency. Pu and his team determined the cost per validator node for each network and estimated their monthly cost, paying particular attention to their relative sizing when used to compute the TPS per dollar ratios. This metric aims to correct two of the most egregious categories of misinformation by forcing the TPS performance to be on mainnet and revealing the inherent tradeoffs of extremely expensive hardware.

Pu also stressed the importance of considering whether a network is permissionless and single-sharded. Permissioned blockchains, where access and participation are restricted to a predefined group of users, are at odds with the philosophy of decentralization. In contrast, permissionless blockchains allow anyone to participate and are a hallmark of decentralization. Pu noted that networks with centralized authorities tend to be more vulnerable to certain weaknesses and that understanding whether a network employs state-sharding versus maintaining a single, sharded state reveals how unified its data management is.

State-sharded networks divide the blockchain’s data into multiple independent parts called shards, while non-state-sharded networks have a single, shared state across the entire network. Pu noted that state-sharded networks aim to increase storage and transaction capacity but often face longer finality latencies due to the need to process transactions across multiple independent shards. He added that many projects adopting a sharding approach inflate throughput by simply replicating their network rather than building a truly integrated and scalable architecture.

Pu’s insights present a notable alternative in a Layer-1 blockchain space where misleading performance metrics increasingly compete for attention. Reliable and effective benchmarks are essential to counter these false representations and foster informed decision-making. Adopting standardized and transparent benchmarks will drive genuine progress beyond merely promotional claims as the industry matures. Pu concluded that having standardized, transparent measurements allows simple comparisons across product options so developers and users understand what it is they’re using and what tradeoffs they’re making. This is a hallmark of any mature industry, and the crypto industry still has a long way to go in this regard.