The End Of Agile Estimation: Flying Without Estimates
by simbo1905
In the past twelve months I have had the privilege of working across more than a half-dozen teams building digital services. It was interesting that only one of them was attempting to do estimates and it wasn’t clear to anyone, let alone them, exactly why they were attempting it. In this post I will sketch out how a successful large scale team abandoned estimations entirely.
You must have been hiding under a rock the past decade if you haven’t seen a burndown chart or witnessed a game of planning poker. It may seem a bit odd that I am brushing off the idea of estimates given that “Agile estimation” is a thing. Heck, I even wrote an opensource planning poker webapp. Rather than get into why I believe burndown charts might be asking the wrong questions let me present a case study of a successful “no estimations” team.
For about a year and a half through to the end of 2015, I was working on a very large platform. When I left it had thirty folks working on it in a globally distributed team. When I arrived the platform was live and in the process of scaling up. Attempts to do scrum were failing. It was back-end heavy work and a highly technical platform. Everything that was being built was new, only built once, and had complex dependencies. The core engineers simply stopped engaging with ceremonies and just doubled-down on shipping working code.
The users of the platform were a large number of teams building business services. Their execs started to get really annoyed that features (platform technical capabilities) were late or stalled. They were being charged for the platform build and felt little control. This seems like a job for a certified scrum master to get in there and crack the whip on Agile estimations, right? Nope.
So how did the situation go from angry internal customers to high fives all round? The bloke in charge of the platform (who was also the platform chief architect) read The Pheonix Project and the team switched over to Kanban without estimations. Team ceremonies for Kanban got down to as low as 3 minutes a day. That was how quickly we could identify the tickets that needed collective focus or intervention. Anyone idle (due to Kanban board work limits) could stay on the line to swarm blocked, starved or urgent production support issues. Everyone else hung up and got on with their work.
As most of the time developers only had meetings totalling 15 minutes a week they were happy. Happy developers with less work in progress, and with better coordination due to fewer fires, gives high productivity. Better yet it was agile in the sense of responding and adapting quickly when things weren’t working. Rather than platitudes to “being bold” and “failing fast” the team was living the dream.
As blocking issues were dealt with pro-actively fewer things burst into flames. That freed up management time to do more proactive work. More often than not when the situation was more complex than had been anticipated the customers themselves were engaged. Usually, quick and practical steps (aka compromises) were found so that their business features were delivered on time.
After some months the team of thirty had many man years of statistical data. The data showed that the team was very good and breaking down the prioritised features into chunks that they could ship inside a couple of weeks. Inevitably there were some outliers of tickets that ran over spectacularly; but since we had been proactive about managing issues the few big outliers were all genuine knotty issues. Only with hindsight were the knots unpicked.
Given that a big agile team can be very successful, and very productive, without estimates why exactly do we feel we need estimations again? More on that in the next post.
Interesting little article you have here. I am currently running a DevOps team (actually it’s broader than that) but deals in large part with production issues, which by their nature are not very estimable (especially against large strategic pieces which are several orders of magnitude larger in effort/complexity, so story points don’t make sense for us). However, I do have one question: How does a team that needs to deliver products over time know that they are on track, if they don’t know their velocity and therefore how many sprints it would take to get through the current scope of their work? Wouldn’t they need to use this to determine whether they need to cut scope of lower priority stories to ensure a deliverable product by a deadline? For my team, this matters less as the turn-around for our Kanban items is much smaller, with no large project work that may take several sprints to complete.
There is a book “Agile IT Organization Design: For Digital Transformation and Continuous Delivery” by Sriram Narayan which basically explains that there is a paradigm effect at work here. The paradigm is from project management of manufacturing where it is hugely expensive to correct things once they are in production. It’s all about optimizing and tracking activities to achieve a predefined outcome. So you must to do a huge amount of up-front planning and tracks that things are “on course” to bring a physical product to production. As manufacturing deals with physical items then its not a flawed approach. Yet it is a flawed approach when it comes to software. The book goes though the whole thing in detail. To give a very approximate and tongue in cheek description the new way goes like this:
Business: “OMG! Our competitiors are killing us. We are going to go out of business unless we are agile to our customer needs and stay head of the challengers and startups cutting into our market share”. IT: “Okay we need to have have a small agile team who can rapidly build a minimal viable product to pilot. Then we need to have a standing agile capability who can continuously improve on it based on qualitative and quantatative user feedback.” Business: “How much will it cost?”. IT: “How much do you want to spend as a percentage of budget to have an agile IT capability which keeps you in business?” Business: “Er, let me get you those figures…” IT: “Whoa! That’s *way* too much money. Spending that much wont be a small focused team and it will be a disaster. We only need a fraction of that money.” Business: “Er.. Okay. What are we going to get for the money?” IT: “Happy customers? You get to stay in business?” Business: “Well yes, but I mean what is the scope of the product range we are building?” IT: “That’s not the way agile works. You define the resource, and the time we have, and the scope is whatever we learn is working well with users. Too much time actually makes agile harder. So we are going to ship something we can internally test in twelve weeks or less from a standing start. Thats actually much faster than you usually take to approve your annual budgets.” Business: “Wow… But the trolls in accounting say you cannot do it that way it violates all these accounting standards! It this even legal!” IT: “Actually I took an accounting qualification myself and the trolls are about two decades out of date on this one. Let me pull up all the facts about that one and how you can run you business as a software business not a manufacturing one …”.
If you are doing agile and you want to know if you are “on-track” look at the qualitative and quantitive user research feedback. You should aim to get positive feedback within weeks of formulating an agile team. You should aim to have working private beta software do remote user testing on within a handful of months. Measure the ramp up of users who you are testing with and whether they are happy as thats measuring the outcome of an agile build. Measuring the input or activities is a waste of time and effort and it mostly counterproductive.
> Wouldn’t they need to use this to determine whether they need to cut scope of lower priority stories to ensure a deliverable product by a deadline?
In my experience, I have rarely see stakeholders reprioritising the backlog in a fine-grained manner based on velocity or estimates. Usually, there are things they really want to happen which stay at the top of the list based on their needs only. Stuff that stays near the top but doesn’t get done because it is too big needs to be broken up or some compromise needs to be found. Stuff that has been started which is going on for too long needs specific management focus to bash into shape. I have never seen any situation where accurate estimates or velocity was in any way need to know “do we need to do this next?” or “can we do this in a reasonable amount of time?”. When things went wrong it is because we didn’t know up front what was the true size of the work. The estimates would have been wrong. Given that estimates only work when you are right, when you don’t need them, I really don’t understand why we need to we expend time on them.
The only thing you really need is that the team commits to do the work and can break it down into manageable chunks. That’s far more about skill as a developer and making compromises. If you find that a feature goes on for many weeks as the scope is blowing up then in my experience the best approach is to go back to the stakeholders and say “Oops. This is a nightmare. Can we renegotiate the scope of this feature?” If the team has a good track record of delivering a lot of functionality, such that there is a lot of stakeholder trust, a compromise or solution will be found.
Once again I don’t think that estimates and velocity are any part of dealing with the exceptions. If we are running Kanban we should only manage the exceptions (i.e. blocked, starved, or “scope blew up” tickets) and we should avoid any management overhead with tickets which don’t run into problems.