slash dev slash null

simbo1905’s ramblings about computers

One More Frown Please? (UPaxos Quorum Overlaps)

There was some discussion around UPaxos safety on a gist where Dave Turner was kind enough to clarify a confusion of mine. I had said that we needed an overlap between prepare quorums to avoid a split-brain. This was incorrect and I am greatful for Dave for correcting my misunderstanding. Yet there was something about not having that overlap that was bugging me… This morning I had an “Aha!” moment: if it exists then Trex will perform an optimisation. Yet in Trex this overlap is not enforced.

Read the rest of this entry »

Blockchain for Finance is Bollocks

Blockchain was invented in 2008 and folks have claimed that it might be the most important invention since the internet. I have worked at a number of companies and organizations and seen them throw millions of dollars at ”doing Blockchain”. I am familiar with multiple attempts to use block in banking and insurance. All of the time and money invested were wasted. I am going to call it: Blockchain for Finance is bollocks.

Read the rest of this entry »

Agile and Domain Driven Design (Part 4): Microservice Sagas

The last post discussed breaking up a digital service into bounded contexts that can be enhanced by small autonomous teams. A microservice architecture allows teams to focus on domain driven design by creating aggregate entities that enforce the invariants of a business. The catch to such autonomy is the price of the operational complexity of running a distributed system. Technologies such as Kubernetes and a Service Mesh help. Yet there remains an additional complexity that you have to abandon global transactions and the “I” of ACID updates. You also need to use asynchronous messaging and deal with eventual consistency. Why? Read the rest of this entry »

How not to structure your database-backed web applications: a study of performance bugs in the wild

The case against ORM continues in this study of performance bugs in the wild.

the morning paper

How not to structure your database-backed web applications: a study of performance bugs in the wild Yang et al., ICSE’18

This is a fascinating study of the problems people get into when using ORMs to handle persistence concerns in their web applications. The authors study real-world applications and distil a catalogue of common performance anti-patterns. There are a bunch of familiar things in the list, and a few that surprised me with the amount of difference they can make. By fixing many of the issues that they find, Yang et al., are able to quantify how many lines of code it takes to address the issue, and what performance improvement the fix delivers.

To prove our point, we manually fix 64 performance issues in [the latest versions of the applications under study] and obtain a median speed-up of 2x (and up to 39x max) with fewer than 5 lines of…

View original post 1,507 more words

File Corruption And Consensus

The Morning Paper blog continues to deliver with on overview of how file corruption causes data loss on consensus systems such as Zookeeper and etcd:

Protocol aware recovery for consensus-based storage

etcd is used by Kubernetes (which is eating the cloud), and Zookeeper is a banks best friend for managing distributed systems configuration, so this is a major problem.

Better yet the paper retrofits a solution called CTRL onto those popular open source work horses with only a 4% overhead. It seems highly likely that CTRL will be coming to your part of the cloud any day soon.

Lean Means Your Never Done

I have just enjoyed reading the book The Lean Startup by Eric Ries. This talks about the minimum viable product (“MVP”) and deploying code continually to test hypotheses using the scientific method (e.g. A/B Testing). Anyone who has worked on Government Digital Services (GDS) in the UK who has heard of those terms but who hasn’t read the book should read it immediately. What is fascinating about the book is how in 2011 it was trying to start a “movement” which by 2016 was mainstream on large government digital services doing user-centric design. Of course like all Big Ideas™ much of the spirit of the approach has been lost in translation was it was codified into a process. Read the rest of this entry »

Thinbus PHP is now on Packagist providing zero-knowledge password-proofs

I finally got around to releasing Thinbus PHP onto Packagist. The PHP demo app is also released there to show how to use the library. The JavaScript at that repo allows a browser to register and login to the PHP server. It is shared with the Thinbus Java version. Next up will be a demo app for the Thibus Node version. Now there are fewer excuses for transmitting a password to the server to be checked.

Brute Force Attacks On Browser based Secure Remote Password Protocol

Mattias Siø Fjellvang contacted me to discuss brute force attacks on SRP such as the thinbus-srp JavaScript library. I thought to write down the things that came out of the discussion.  Read the rest of this entry »

Pre-voting in distributed consensus

Another top notch Paxos post from the inventor of UPaxos covers leader election in Paxos. The outlined approach is similar to that used in TRex which is based on a sloppy timeout mechanism. This post will get into why this is a must read for consensus fans.

Read the rest of this entry »

Observability in Paxos clusters

David Turner the inventor of UPaxos has posted a great blog post about monitoring Paxos clusters. It’s a must read for consensus aficionados.