Erasure Coding Explained
How Erasure Coding Splits and Protects Data
Erasure coding is a data protection method that splits data into a fixed number of data fragments and adds parity fragments calculated from them. The fragments are spread across different disks, servers or sites. If some fragments are lost, the original data can be rebuilt from the remaining ones, as long as enough of them survive.
A common notation is k+m. In a 4+2 scheme, data is split into four data fragments and two parity fragments, six in total. Any four of the six are enough to reconstruct the data, so the scheme tolerates the loss of any two. The parity is produced with mathematics that makes this recovery possible without storing a full extra copy.
Erasure Coding Compared With Replication and RAID
Replication keeps complete copies. Three copies of a file tolerate the loss of two copies, but cost three times the raw capacity. A 4+2 erasure code tolerates the same two losses with 1.5 times the raw capacity, which is why large storage systems use it to reduce overhead.
RAID also uses parity, but it works within the disks of one server or array. Erasure coding applies the same principle across independent nodes, so it can protect against the loss of a whole machine or site, not only one drive. The common RAID layouts are compared in RAID levels compared. Erasure coding has costs of its own: more computation, more network traffic when data is written or rebuilt, and slower handling of small writes.
When Erasure Coding Makes Sense
Erasure coding suits large, mostly static data such as archives, media libraries and object storage backends, where writes are batched and capacity efficiency matters. Systems often use plain replication for data that changes frequently, because updating an erasure-coded object requires recalculating parity and touching several fragments.
Repair is also more expensive. Rebuilding a lost fragment means reading several surviving fragments over the network, which can slow the system while repair runs. Planners weigh the storage savings against this repair traffic and the extra CPU use. Object-based systems that often use this method are described in what object storage is.
A 4+2 Scheme Across Six Storage Nodes
A storage provider keeps archive objects across six nodes using a 4+2 scheme, so each object has one fragment on each node. When the disks in one node fail, four of the surviving nodes supply the fragments needed to rebuild the lost data, and customers keep reading their files throughout. The repair is slower than copying a file, because every rebuilt fragment depends on data read from several places.
Frequently asked questions
Is erasure coding better than RAID?
It depends on the goal. RAID is simple, fast for local disks and well understood within one server. Erasure coding is designed for distributed systems, where protecting against the loss of a server or site matters and capacity overhead must stay low. Many systems use both, with RAID or similar protection inside a machine and erasure coding across machines.
Does erasure coding replace backups?
No. Erasure coding protects against hardware failures, but it does not protect against deleted files, ransomware, software bugs or mistakes that spread to every fragment. Separate copies, ideally stored in another location, remain necessary for recovering from those events. Distributed systems that use this method are covered in Ceph storage explained.