Solana Network Near Finality Failure Due to Routing Error

icon币界网
Share
AI summary iconSummary
A routing error at Teraswitch caused 28.83% of staked SOL to go offline, nearly disrupting Solana’s Proof of Stake (PoS) finality. The misconfigured default route in Miami propagated to 12 global locations, affecting over 90 validators. Engineers resolved the issue within 10 minutes, but 333 SOL in rewards were lost. The incident highlighted concentration risks, as more than 27% of staked SOL went offline simultaneously—exceeding the 25% threshold. Marinade Finance noted its staking was overly concentrated, prompting a review of diversification strategies.
CoinDesk reports:

On Wednesday morning, the Solana network briefly neared loss of transaction finality. A routing configuration issue at the custodial service Teraswitch caused 28.83% of staked SOL to go temporarily offline, and Solana halts block confirmation when approximately 33.34% of staked tokens are unreachable.

This means the network was just 4.51 percentage points away from completing finality. According to Marinade Finance’s calculations, this event reached approximately 86% of the “stall threshold.”

The outage was caused by a single point of routing failure.

Teraswitch later stated that the issue originated from a default route issued by its Miami site. This route lost its original metric and community attributes during propagation and was subsequently disseminated to European and Asia-Pacific nodes via an Amsterdam route reflector.

The local edge router misclassified it as a locally generated route, giving it higher priority than the normal exit; however, the data center core network deemed it invalid. As a result, 12 locations—including London, Amsterdam, Dublin, Frankfurt, Singapore, and Tokyo—temporarily lost valid forwarding paths, while North America remained unaffected.

The engineering team identified the issue within approximately 10 minutes, and service was restored at 04:16:15 UTC.

Nearly 30% of staked SOL went briefly offline

Marinade reported that approximately 90 validators were affected, collectively losing 333 SOL in rewards. Their analysis showed that the outage was highly concentrated on a single autonomous system, AS20326.

The system holds 118.89 million SOL, accounting for approximately 27.34% of the total network stake, with 94% of it going offline during the same period. This percentage exceeds the Solana Foundation’s delegated program limit of 25% for a single autonomous system.

Additionally, 59 validators with approximately 80.2 million SOL recovered within the same time window in Amsterdam, Frankfurt, and Tokyo, indicating that these nodes were primarily waiting for routing to reconverge rather than switching to alternative available paths. Solana’s second-largest validator, Helius, was offline for a full 33 minutes.

Concentration risk has been exposed again

Marinade also noted that an additional 14.1 million SOL were offline during the same period in托管 environments such as latitude.sh, Limestone, Butterfly Research, and Allnodes, a phenomenon that cannot be explained solely by the available data.

This means that measuring dispersion solely based on the custodian's口径 may underestimate the risk of validators becoming simultaneously offline during the same failure.

Marinade also disclosed concentration in its own allocation model: two-thirds of its staked tokens are concentrated in four autonomous systems, with AS395201 alone accounting for 36.94%. The protocol stated that it will reassess concentration limits by network and data center and begin disclosing whether validators have hot-swapping and automatic failover capabilities.

Additional information: The 333 SOL reward that was missed is expected to be covered by validator stakes after the current epoch ends. However, if the offline rate exceeds one-third at that time, transaction confirmations for all SOL holders will halt, and no staking mechanism exists to mitigate such systemic consequences. Solana’s last full network outage occurred in February 2024, when it took nearly five hours to restore operations.

Disclaimer: The information on this page may have been obtained from third parties and does not necessarily reflect the views or opinions of KuCoin. This content is provided for general informational purposes only, without any representation or warranty of any kind, nor shall it be construed as financial or investment advice. KuCoin shall not be liable for any errors or omissions, or for any outcomes resulting from the use of this information. Investments in digital assets can be risky. Please carefully evaluate the risks of a product and your risk tolerance based on your own financial circumstances. For more information, please refer to our Terms of Use and Risk Disclosure.