File Corruption And Consensus

The Morning Paper blog continues to deliver with on overview of how file corruption causes data loss on consensus systems such as Zookeeper and etcd:

Protocol aware recovery for consensus-based storage

etcd is used by Kubernetes (which is eating the cloud), and Zookeeper is a banks best friend for managing distributed systems configuration, so this is a major problem.

Better yet the paper retrofits a solution called CTRL onto those popular open source work horses with only a 4% overhead. It seems highly likely that CTRL will be coming to your part of the cloud any day soon.