I was fortunate to hear a talk recently by Martin Aspeli, Head of Engineering at Deloitte Digital, explaining how and why to use metrics to predict software delivery rather than estimates. Here are my brief notes:
tl;dr: there is no need to spend time estimating — with discipline and data we can forecast!
Predictability is more important than pure top speed for most organisations.
Humans are notoriously bad at estimating. This is a facet of human nature — as a species we are bad at estimating. See Hofstadter’s Law.
We tend to ignore Wait Time when estimating, yet Wait Time makes up the majority of Cycle Time in most organisations doing knowledge work like software development.
With some discipline in how we use Jira, we can start to use metrics to help track how long things actually take — this increases predictability.
We need to take care to ensure that the Arrival Rate of new work items does not exceed the Departure Rate. Treat our team and its context as a system to be cared for.
Use a Cumulative Flow Diagram (CFD) to track the system health.
Measure end-to-end Cycle Time of actual stories. No need to size the stories with Story Points or t-shirt sizes. The ensemble of stories provides confidence measures.
As people are “loaded” beyond about 80% capacity, their capacity to respond drops dramatically. This is basic queueing theory.
Forecast the delivery dates for batches of work using Monte Carlo simulations. This gives confidence measures (50%, 85%, 95%, etc.) allowing stakeholders to choose dates based on likelihood and risk.
I’ve been working in the software industry for just short of 20 years. In that time we have seen incredible advances in digital technologies along with huge advances in software engineering and systems engineering approaches to be able to deal with web-scale systems. Here are 4 key things that I think we’ve learned in the past 20 years in the software industry that we can offer to other industries, especially those industries now adopting digital approaches like manufacturing and making.
Design for change & failure — we have useful patterns
One of software’s key properties is its malleability: we can change it and re-shape it easily. This sets software engineering apart from most other engineering fields because we have come to explicitly expect change and design our software systems to accommodate not just occasional change but rapid, regular, relentless change.
This drive for relentless change in software has forced us to discover and adopt many useful patterns for working with this kind of change: service discovery, robust routing protocols, stateless scaling, search algorithms, public key infrastructure (PKI) and its related encryption patterns like public-private keys, and many more.
The speed of change enabled by software forces rapid discovery of failure modes too, along with patterns to deal with failure: fault-tolerant networking, fault-tolerant clustering and data replication patterns,
Learning 1: design for relentless change.
Iterative delivery works — Agile/Lean approaches
Software is cheap to experiment with, and easy to adapt time and time again. This has made it amenable to the discovery of techniques that focus ruthlessly on early delivery of customer value: user stories, thin slicing, MVP, and so on, many of which derive from principles in the Agile Manifesto. Done well, agile and lean approaches help us to “zoom in” on the core of the problem we’re solving, without getting side-tracked in “nice to have” features and over-engineered “reusable” solutions. When we combine these value-focused approaches with the practices of Continuous Delivery and a focus on operability, we have a powerful way to deliver sustainable innovation.
Getting the thing (software, product, solution) in the hands of real users (whether at scale or at least in a beta launch) is such a key validation for our assumptions that we want to do this as early and as often as possible. In fact, we should expect to be wrong and learn from the real world. This is very far from the “genius inventor” of yesteryear, sitting in a dark room, tinkering away until the masterpiece is ready. With software we use regular real-world validation of our systems, together with rich digital telemetry, to tell us what works and what does not work.
Learning 2: frequent iterative delivery with engaged stakeholders works.
Design for version control — full digital change tracking is powerful
I think that modern version control systems like Git and Mercurial are close to wonders of the world, particularly when combined with browser-based user interfaces like Github, Bitbucket, Gitlab, etc. Version control systems for software (code, configuration, documentation, etc.) enable us to employ powerful reasoning about changes without which those changes would be incredibly error-prone and fraught with doubt. We can drive very specific automation from a change change to a single file in version control, all due to the fact that we store our specifications (code, config, etc.) as plain text files that can be parsed and interpreted by software.
Having experienced version control working so effectively, I cannot now imagine wanting to engineer any kind of commercial system (digital or mechanical) without using version control to store and track the digital specifications for the system: it would feel very wrong.
The speed of innovation in the software sector is underpinned by a solid foundation of richly-featured version control systems. The manufacturing and making industries can take what we’ve learned and built in the software industry with version control to build a secure foundation for specification management.
Learning 3: put every specification in version control: code, instructions, documentation.
The system is socio-technical — people and machines together
When we look at web-scale software development — many hundreds of people deploying thousands of software changes every day to live systems in just a single organisation — we see system effects that are neither purely social (human beings) nor technical (machines), but a combination of humans and machines: socio-technical. This is particularly noticeable in the studies that have confirmed Conway’s Law, originally stated in 1968 but since validated (more or less) especially with teams building software:
Digital technologies will augment human involvement in manufacturing and making: better telemetry will enable better decisions and fewer dull, repetitive activities. For example, I was recently speaking to someone from Leeds Hackspace who was interested in using a Raspberry Pi to monitor the temperature of a fluid bath for dyeing yarn by hand: she was engaged in a very old tradition but was happy to get help from digital technologies to deal with the boring aspects (getting the temperature just right).
Learning 4: The system is socio-technical: people and machines working together
Assembly Conference 2018 — the conference for digital in manufacturing and making — will take place on Tuesday 02 October 2018 in Leeds. Tickets are now on sale: http://assemblyconf.com/