It is a capital mistake to theorize before one has data*

I believe that banking software can be developed in such a way to be sustainable.

Sustainable software stays up to date with new client demands and with the latest innovations in cloud computing and beyond. It allows for the possibility of cost-effective change. It is written with the future in mind, expecting that it will be maintained or expanded on by someone else.

Welcome back! Have a read of my introductory blog if you haven’t yet seen it. 

We’re going to dive into the first key principle enabling sustainable software development. 

Shift left

Shift left is the first in my list of principles for a reason. This is the practice of collecting timely, empirical data about the behaviour of the software as early in the development process as possible. 

Without this, it’s hard to know if we’re making the right design decisions. We would have to rely on best practices and our experience alone.

History holds a long list of well-engineered systems ‘on paper’ that have failed dramatically, even with some of the most talented and famed engineers working on them. 

Collecting timely, empirical data means getting feedback about the behaviour of the system as fast as possible and with minimal cost. And receiving it before it gets into the customer's hands. 

Shift left requires investing engineering effort in developing platforms around the product itself to be able to generate, collect and analyse the data both before and after the software reaches users. 

All software is unique to a various degree, and ambitious. Differentiated software products require ambitious, unique platforms. At Uber, more than 30% of engineering investment is dedicated to developing the underlying platform promoting the shift left principle.

In practical terms, the foundation for shift left in software is a strong CI/CD pipeline. This makes it cost effective and fast to collect data via automated tests covering both non-functional and functional requirements.

The mirror aspect of a strong CI/CD test automation tooling is a quality observability stack. Without tracers, logs and metrics, it is hard – almost impossible – to analyse the results of tests and make informed decisions.

At Oneiro, shift left means automating all aspects of the software delivery process. From automating the underlying software infrastructure, security configuration and deployments, even the versioning process, to implementing an entire battery of tests including production-like testing.

We are data hungry, and our goal is to extract as much information from the software delivery process as possible using automated means.

This enables us to make informed design decisions based on actual data and focus our resources on what is important. This is a fundamental prerequisite on which we build the actual product.

The shift left principle also makes us rethink what is the essence of software engineering…

At its core, like any other engineering discipline, software engineering is the application of the scientific process to solve a problem in an economical and efficient way.

As David Farley correctly mentions in his influential book Modern Software Engineering: Doing What Works to Build Better Software Faster, there are no manufacturing costs in software as the cost of replication is zero. Though the principles of production engineering do not apply to software engineering; the principles of engineering design through a scientific process of discovery do. And science is in essence progress through measured facts.

Similar to building a new model of a car or a plane that has not been built before, a lot of experimentation is required as well as testing of models before the product is ready for mass production.

In software engineering, the product itself is the model which makes testing and measuring easier compared with other disciplines, as it will end up being the final product. However, like other engineering design disciplines, what is still key is collecting empirical data through experiments on the product before it goes into production, and iteratively improving on it based on hard evidence.

Automated testing can be considered a form of scientific experiments where we control some variables to test for specific behaviours in terms of correctness, performance or resilience. 

By following our scientific experiment metaphor to its conclusion, the tests have to be designed for one purpose only, and kept deterministic and hermetic, i.e. measuring the variables.

A note on building banking software systems…

For a long time banking systems have lacked the quality needed for this business. They have been built almost as an antithesis of applying scientific and empirical principles. 

Systems that track tweets and social media posts (arguably far less essential) have been better built due to the fact that they have followed these shift left principles, which slowly permeated the Silicon Valley engineering culture.

The fact that management in financial institutions and legacy vendors have seen engineering as a production plant for a long time, where the goal is to ship mediocre features at the lowest cost possible, has not helped. 

What hasn’t solved the problem either is the quasi-religious application of whatever engineering process was formalised in the company, regardless if it’s kanban, scrum or waterfall (which in essence are manufacturing processes applied to solving design problems). 

I hope, in this essential business of handling money, that software engineering approaches would adopt the shift left principle: collecting timely, empirical data about the behaviour of the software as early in the development process as possible, and making rational decisions based on facts. 

Let me know your thoughts or experiences. And coming up next… the 80/20 rule.