slash dev slash null

stuff about puters

How not to structure your database-backed web applications: a study of performance bugs in the wild

The case against ORM continues in this study of performance bugs in the wild.

the morning paper

How not to structure your database-backed web applications: a study of performance bugs in the wild Yang et al., ICSE’18

This is a fascinating study of the problems people get into when using ORMs to handle persistence concerns in their web applications. The authors study real-world applications and distil a catalogue of common performance anti-patterns. There are a bunch of familiar things in the list, and a few that surprised me with the amount of difference they can make. By fixing many of the issues that they find, Yang et al., are able to quantify how many lines of code it takes to address the issue, and what performance improvement the fix delivers.

To prove our point, we manually fix 64 performance issues in [the latest versions of the applications under study] and obtain a median speed-up of 2x (and up to 39x max) with fewer than 5 lines of…

View original post 1,507 more words

File Corruption And Consensus

The Morning Paper blog continues to deliver with on overview of how file corruption causes data loss on consensus systems such as Zookeeper and etcd:

Protocol aware recovery for consensus-based storage

etcd is used by Kubernetes (which is eating the cloud), and Zookeeper is a banks best friend for managing distributed systems configuration, so this is a major problem.

Better yet the paper retrofits a solution called CTRL onto those popular open source work horses with only a 4% overhead. It seems highly likely that CTRL will be coming to your part of the cloud any day soon.

Lean Means Your Never Done

I have just enjoyed reading the book The Lean Startup by Eric Ries. This talks about the minimum viable product (“MVP”) and deploying code continually to test hypotheses using the scientific method (e.g. A/B Testing). Anyone who has worked on Government Digital Services (GDS) in the UK who has heard of those terms but who hasn’t read the book should read it immediately. What is fascinating about the book is how in 2011 it was trying to start a “movement” which by 2016 was mainstream on large government digital services doing user-centric design. Of course like all Big Ideas™ much of the spirit of the approach has been lost in translation was it was codified into a process. Read the rest of this entry »

Thinbus PHP is now on Packagist providing zero-knowledge password-proofs

I finally got around to releasing Thinbus PHP onto Packagist. The PHP demo app is also released there to show how to use the library. The JavaScript at that repo allows a browser to register and login to the PHP server. It is shared with the Thinbus Java version. Next up will be a demo app for the Thibus Node version. Now there are fewer excuses for transmitting a password to the server to be checked.

Brute Force Attacks On Browser based Secure Remote Password Protocol

Mattias Siø Fjellvang contacted me to discuss brute force attacks on SRP such as the thinbus-srp JavaScript library. I thought to write down the things that came out of the discussion.  Read the rest of this entry »

Pre-voting in distributed consensus

Another top notch Paxos post from the inventor of UPaxos covers leader election in Paxos. The outlined approach is similar to that used in TRex which is based on a sloppy timeout mechanism. This post will get into why this is a must read for consensus fans.

Read the rest of this entry »

Observability in Paxos clusters

David Turner the inventor of UPaxos has posted a great blog post about monitoring Paxos clusters. It’s a must read for consensus aficionados.

Rust and Embedded Databases for Paxos 

So far on my spike into Rust we have been on a roll. Next on the bucket list is an embedded disk backed B-tree database to act as the Paxos journal for TRex. This is where I have hit my first bump in the road. Read the rest of this entry »

Rust Extension Methods

Seven years ago I was working on a system that had a fat client desktop app written in C# and a Java backend. I signed up for a week-long C# training programme. I was fascinated about the bits of C# that had evolved away from Microsoft J++. A lot of the differences that still stuck in my mind were all about boilerplate removal. One of those was C# Extension Methods. I have posted some sample code of Rust Extention Methods over on GitHub.    Read the rest of this entry »

Rust Pickling 

The first thing item to research on my Rust spike is picking. A quick survey of the landscape indicates that serde appears to be the current defacto standard framework. Pickling was an area that I chose to hand code in the TRex Scala implementation. Why? Because Scala’s pickling engine crashed my JVM and wasn’t yet the stage of a stable disk format. I want to have no external dependencies for the inner Paxos library and as few as possible elsewhere. As Scala’s official pickling engine wasn’t ready for prime time, I was on my own. I had some fun writing my own ByteChain based pickling where I wrote a compact binary wire encoding for unsigned integers. With Rust does serde put me in a better place? Read the rest of this entry »