Sharding - Seeing the big picture

Given how popular blockchain has become, both as the support for cryptocurrency and for the development of decentralized apps, it was only a matter of time until scalability became an important topic of discussion. With more and more transactions taking place simultaneously, each node has more processing to do in order to validate them.

But what are the implications of this situation? Can this become a big issue for users, or is it already a problem? And why is sharding being constantly proposed as a possible solution?

The issue at hand

Proof-of-Work blockchains are the most popular model out there, and thus the more prone to fall victims of their own growth and lack of scalability solutions. With PoW, each node of the blockchain has to process all of the transactions in order to validate them before generating the block. Once the majority of the nodes reaches a consensus, the block is added, and they begin to work on the next one.

But when does this issue starts to actively hurt blockchain? Some argue that it already has. Currently, Bitcoin can only handle 3 to 7 transactions per second, and Ethereum varies between 12 to 30. With hundreds of thousands of transactions taking place every second all around the world, this degree of latency can be a deterrent for users.

Resource_03.jpg

The solution most technical experts are focusing on is developing ways to distribute the workload of each node so the validation of transactions can be done in a faster way, but some argue that this approach goes against the main point of blockchain: the mass distribution of data that makes it secure against falsifying and tampering.

Thus, many discussions have arisen regarding what possible solutions there are for this issue, and many believe that sharding is the way that better reduces the load of each node while not sacrificing security in a drastic way, which is way there has been many different proposals on how to approach sharding for PoW blockchains.

Divide and conquer

Sharding is a simple way to distribute the workload among the different nodes of a blockchain, so more transactions can be processed simultaneously, while a way of communication is kept open between different shards to maintain stability.

Big names like Vitalik Buterin, the creator of Ethereum, have already expressed their support for the development of sharding protocols, saying that this method would keep “most of the desired decentralization and security properties of a blockchain”, but others are more cautious about this subject. It would be simpler to take over a single shard and work up from there than disrupting into a whole blockchain.

Ethereum’s model offers a solution via random redistribution of nodes between shards, as it allows for an additional measure of security where attacking certain nodes will not result in the loss of a whole shard. David Huseby, from Hyperledger, said “the idea is to make it difficult for an attacker to predict, or force, which shard their node gets assigned to.”

He then went on to explain how implementing sharding in a Hyperledger blockchain would be different from the Ethereum model, given that they don’t need to split up an address, which gives them more freedom to experiment with other sharding methods. And yet, the communication issue remains.

Stay connected

The greatest efforts are now focused on solving the problems that arise when communication between shards comes into play. One of the most interesting ones is Devvio’s “Devv” protocol.




Resource_01.jpeg
Scalability issues are simply solved by adding more and more shards to the blockchain (above), while the T1 network keeps growing to connect the shards together (below). Image from Tom Anderson.

Scalability issues are simply solved by adding more and more shards to the blockchain (above), while the T1 network keeps growing to connect the shards together (below). Image from Tom Anderson.

According to Devvio’s CEO, Tom Anderson, this protocol consists of each node representing its own blockchain ledger, which means that each node takes care of a separate group of transactions, up to 3,000. And with more nodes, this number grows linearly. Each of these shards becomes a source of information for a larger network, dubbed the T1 network; and a separate transactional network is used for communication between shards, the T2 network.

Another possible solution comes from a joint effort from the MIT, Stanford, Berkeley and many others, with the support of the Distributed Technology Research Foundation (DTR). Calling themselves Unit-e, they are using an “entirely new way of sharding” they call PolyShard, which differs from traditional sharding solutions by mixing the information that’s assigned to each shard, while still allowing for accurate recovery and reconstruction of the original data.

David Huseby has said that sharding may not be a definitive and immediate solution to scalability issues. He argues that "there are lots of details that have to be considered and we're going to need some empirical experimentation to go along with the theory before we can call it secure”, and given that blockchain is being used for more important tasks and assets, security will remain the main priority right after efficiency.