How far is the road to the popularization of Decentralization storage?

Storage has been one of the popular tracks in the blockchain industry. Filecoin, as the leading project of the last bull market, once had a market value exceeding 10 billion USD. Arweave focuses on permanent storage, with a peak market value of 3.5 billion USD. However, as the availability of cold data storage has come into question, the development prospects of decentralized storage have also been put under a cloud. Recently, the emergence of Walrus has brought new attention to the long-quiet storage track, while the Shelby project launched by Aptos and Jump Crypto aims to make breakthroughs in the field of hot data storage. This article will analyze the narrative changes in decentralized storage from the development paths of four projects: Filecoin, Arweave, Walrus, and Shelby, and explore its future development direction.

Filecoin: The Essence of Mining Coins Behind the Storage Facade

Filecoin is one of the early rising cryptocurrency projects, and its development direction revolves around Decentralization. Filecoin attempts to combine storage with Decentralization, addressing the trust issues associated with centralized data storage service providers. However, certain trade-offs made to achieve Decentralization have become pain points that later projects sought to solve. To understand that Filecoin is essentially a mining coin project, one needs to be aware of the limitations of its underlying technology IPFS in handling hot data.

IPFS: Decentralization architecture transmission bottleneck

IPFS(, the InterPlanetary File System, was introduced around 2015 with the aim of disrupting traditional HTTP protocols through content addressing. However, the biggest problem with IPFS is its extremely slow retrieval speed. In an era where traditional data services can achieve millisecond-level responses, retrieving a file from IPFS still takes over ten seconds, making it difficult to promote in practical applications. Apart from a few blockchain projects, IPFS is rarely adopted by traditional industries.

The underlying P2P protocol of IPFS is mainly suitable for "cold data," which refers to static content that does not change frequently. When it comes to handling hot data, such as dynamic web pages, online games, or AI applications, the P2P protocol does not have any significant advantages over traditional CDNs.

Although IPFS itself is not a blockchain, its directed acyclic graph )DAG( design is highly compatible with many public chains and Web3 protocols, making it inherently suitable as a foundational framework for blockchains. Therefore, even in the absence of practical value, IPFS is sufficient as a foundational framework for carrying blockchain narratives. Early altcoin projects only needed a functioning framework to initiate their grand plans, but with the development of Filecoin, the issues brought by IPFS have also begun to hinder its progress.

) Coin logic under the storage cloak

The original intention of IPFS's design is to allow users to become part of the storage network while storing data. However, in the absence of economic incentives, it is difficult for users to voluntarily use this system, let alone become active storage nodes. This means that most users will only store files on IPFS without contributing their own storage space or storing others' files. It is against this backdrop that Filecoin was born.

The token economic model of Filecoin includes three main roles: users pay fees to store data; storage miners receive token rewards for storing user data; retrieval miners provide data when users need it and receive rewards.

This model has potential malicious space. Storage miners may fill garbage data after providing storage space to gain rewards. Since this garbage data will not be retrieved, even if lost, it will not trigger the penalty mechanism for storage miners. This allows storage miners to delete garbage data and repeat this process. Filecoin's proof of replication consensus can only ensure that user data has not been privately deleted, but it cannot prevent miners from filling garbage data.

The operation of Filecoin largely relies on miners' continuous investment in the token economy, rather than on the real demand from end-users for distributed storage. Although the project is still undergoing continuous iteration, at this stage, the ecological construction of Filecoin is more in line with "mining coin logic" rather than the "application-driven" positioning of storage projects.

Arweave: Born of Long-termism, Defeated by Long-termism

If Filecoin's goal is to build an incentivized, verifiable decentralized "data cloud" framework, then Arweave has taken another extreme direction in storage: providing permanent storage capabilities for data. Arweave does not attempt to build a distributed computing platform; its entire system revolves around a core assumption - important data should be stored once and preserved on the network forever. This extreme long-termism makes Arweave vastly different from Filecoin in terms of mechanisms, incentive models, hardware requirements, and narrative perspectives.

Arweave takes Bitcoin as its learning object, attempting to continuously optimize its permanent storage network over long periods measured in years. Arweave does not care about marketing, nor does it care about competitors and market trends. It is simply moving forward on the path of iterating its network architecture, indifferent to whether anyone is paying attention, because this is the essence of the Arweave development team: long-termism. Thanks to long-termism, Arweave was warmly embraced in the last bull market; and because of long-termism, even after falling to the bottom, Arweave might still survive several rounds of bull and bear markets. The only question is whether there will be a place for Arweave in the future of decentralized storage. The existence value of permanent storage can only be proven over time.

The Arweave mainnet has evolved from version 1.5 to the recent version 2.9. Although it has lost market attention, it has been dedicated to allowing a broader range of miners to participate in the network at minimal cost, and to incentivizing miners to maximize data storage, continuously enhancing the robustness of the entire network. Arweave is well aware that it does not align with market preferences, taking a conservative approach, not embracing the miner community, resulting in a complete stagnation of the ecosystem, upgrading the mainnet at minimal cost, and continuously lowering hardware thresholds without compromising network security.

A Review of the Upgrade Journey from 1.5 to 2.9

Arweave version 1.5 exposed a vulnerability where miners could rely on GPU stacking instead of actual storage to optimize block production chances. To curb this trend, version 1.7 introduced the RandomX algorithm, limiting the use of specialized computing power and requiring general CPUs to participate in mining, thereby reducing the centralization of computing power.

In version 2.0, Arweave adopts SPoA, transforming data proofs into a concise path of Merkle tree structure, and introduces format 2 transactions to reduce synchronization burdens. This architecture alleviates network bandwidth pressure, significantly enhancing the collaborative capabilities of nodes. However, some miners can still evade the responsibility of holding real data through centralized high-speed storage pool strategies.

To correct this bias, version 2.4 introduced the SPoRA mechanism, which incorporates global indexing and slow hash random access, requiring miners to genuinely hold data blocks to participate in valid block creation, thus weakening the stacking effect of computing power at a mechanistic level. As a result, miners began to focus on storage access speeds, promoting the application of SSDs and high-speed read/write devices. Version 2.6 introduced hash chain control for block creation rhythm, balancing the marginal benefits of high-performance equipment and providing a fair participation space for small and medium miners.

Subsequent versions further enhance network collaboration capabilities and storage diversity: 2.7 adds collaborative mining and pool mechanisms to improve the competitiveness of small miners; 2.8 introduces a composite packaging mechanism that allows large-capacity low-speed devices to participate flexibly; 2.9 introduces a new packaging process in replica_2_9 format, significantly improving efficiency and reducing computational dependencies, completing the closed-loop of data-oriented mining models.

Overall, Arweave's upgrade path clearly presents its long-term strategy oriented towards storage: while continuously resisting the trend of computing power concentration, it lowers the participation threshold to ensure the possibility of the protocol's long-term operation.

Walrus: Is Embracing Hot Data Hype or Something Deeper?

The design philosophy of Walrus is completely different from that of Filecoin and Arweave. Filecoin starts from creating a decentralized and verifiable storage system, at the cost of cold data storage; Arweave aims to build an on-chain library of Alexandria that can permanently store data, at the cost of too few application scenarios; Walrus, on the other hand, aims to optimize the storage costs of hot data storage protocols.

Magic Modified Error Correction Code: Cost Innovation or Old Wine in a New Bottle?

In terms of storage cost design, Walrus believes that the storage overhead of Filecoin and Arweave is unreasonable. The latter two both adopt a fully replicated architecture, whose main advantage is that each node holds a complete copy, providing strong fault tolerance and independence between nodes. This type of architecture ensures that even if some nodes go offline, the network still maintains data availability. However, this also means that the system requires multiple copies for redundancy to maintain robustness, which in turn increases storage costs. Especially in the design of Arweave, the consensus mechanism itself encourages nodes to store redundant data to enhance data security. In contrast, Filecoin is more flexible in cost control, but at the cost of potentially higher data loss risks for some low-cost storage options. Walrus attempts to find a balance between the two, with its mechanism controlling replication costs while enhancing availability through structured redundancy, thereby establishing a new compromise between data availability and cost efficiency.

The Redstuff created by Walrus is a key technology for reducing node redundancy, originating from Reed-Solomon###RS( coding. RS coding is a very traditional erasure coding algorithm, and erasure coding is a technique that allows the dataset to be doubled by adding redundant fragments, which can be used to reconstruct the original data. From CD-ROMs to satellite communications to QR codes, it is frequently used in daily life.

Erasure codes allow users to take a block, for example, 1MB in size, and then "amplify" it to 2MB, with the additional 1MB being special data known as erasure codes. If any byte in the block is lost, users can easily recover those bytes through the codes. Even if up to 1MB of the block is lost, the entire block can still be recovered. The same technology allows computers to read all the data on a CD-ROM, even if it has been damaged.

The most commonly used is RS coding. The implementation method starts from k information blocks, constructs the relevant polynomial, and evaluates it at different x coordinates to obtain the encoded blocks. Using RS erasure codes, the probability of randomly sampling large amounts of lost data is very small.

For example: Dividing a file into 6 data blocks and 4 parity blocks, totaling 10 parts. As long as any 6 of them are retained, the original data can be completely restored.

Advantages: Strong fault tolerance, widely used in CD/DVD, fault-tolerant hard disk arrays ) RAID (, and cloud storage systems ) such as Azure Storage, Facebook F4(.

Disadvantages: Decoding calculations are complex, and the overhead is relatively high; it is not suitable for scenarios with frequent data changes. Therefore, it is usually used for data recovery and scheduling in off-chain centralized environments.

Under the Decentralization architecture, Storj and Sia have adjusted traditional RS coding to meet the practical needs of distributed networks. Walrus has also proposed its own variant based on this - the RedStuff coding algorithm, to achieve lower costs and a more flexible redundancy storage mechanism.

What is the most significant feature of Redstuff? By improving the erasure coding algorithm, Walrus can quickly and robustly encode unstructured data blocks into smaller shards, which are distributed and stored in a network of storage nodes. Even if up to two-thirds of the shards are lost, the original data block can be quickly reconstructed using partial shards. This is made possible while maintaining a replication factor of only 4 to 5 times.

Therefore, it is reasonable to define Walrus as a lightweight redundancy and recovery protocol redesigned around the Decentralization scenario. Compared to traditional erasure codes ) like Reed-Solomon (, RedStuff no longer pursues strict mathematical consistency, but instead has made realistic trade-offs regarding data distribution, storage verification, and computational costs. This model abandons the immediate decoding mechanisms required for centralized scheduling, opting instead to verify whether nodes hold specific data copies through on-chain Proof, thereby adapting to a more dynamic and marginalized network structure.

The core design of RedStuff is to split data into two categories: primary slices and secondary slices. Primary slices are used to restore the original data, and their generation and distribution are strictly constrained, with a recovery threshold of f+1, requiring 2f+1 signatures as availability endorsement; secondary slices are generated through simple operations such as XOR combinations, serving to provide elastic fault tolerance and enhance the overall robustness of the system.