Model Software

Today, I had to stop myself writing “solving the problem” about developing software. Why do we say that? Why do software people call any bounded area of reality “the problem domain”?

My change of mind has been fermenting for a while, due to modelling business processes, learning about incremental, agile software development and more recently writing and learning functional programming. In the shower this morning, I finally concluded that I think software is primarily a modelling medium. We solve problems using the models we build.

Wanting to create another first-person shooter game or to model the fluids in a thermo-nuclear reactor are challenges, not problems. We build models of systems we have defined and the systems don’t even have to be real. I read a couple of days ago that a famous modern philosopher said our world is made of both reality and our ideas. Assuming the computer hardware is real, the software can model either reality or our imagination; our chosen narrative.

‘Digital’ gets everyone working with software models instead of reality. Once everyone lives inside the shared model, when does it become our reality?

Or when did it?

Advertisement

Agility vs Momentum

[ This post is aimed at readers with at least basic understanding of agile product development. It doesn’t explain some of the concepts discussed.]

We often talk of software development as movement across a difficult terrain, to a destination. Organisational change projects are seen as a lightening attack on an organisation, though in reality, have historically proved much slower than the speed of light. Large projects often force through regime change for ‘a leader’. Conventionally, this leader has been unlikely to travel with the team. Someone needs to “hold the fort”. There may be casualties due to friendly firings.

Project Managers make ‘plans’ of a proposed ‘change journey’ from one system state to another, between points in ‘change space’, via the straightest line possible, whilst ignoring the passage of time which makes change possible. Time is seen as distance and its corollary, cost. The language of projects is “setting-off”, “pushing on past obstacles” and “blockers” such as “difficult customers”, along a fixed route, “applying pressure” to “overcome resistance”. A project team is an army on the march, smashing their way through to a target, hoping it hasn’t been moved. Someone must pay for the “boots on the ground” and their travel costs. This mind-set leads to managers who perceives a need to “build momentum” to avoid “getting bogged down”.

Now let us think about the physics:

•  momentum = mass x velocity, conventionally abbreviated to p = mv.
  At this point it may also be worth pointing out Newton’s Second Law of Motion:

• force = mass x acceleration, or F = ma
  (Interpretted by Project Managers as “if it gets stuck, whack it hard with something heavy.”)

What about “agile software developments”? There is a broad range of opinion on precisely what those words mean but there is much greater consensus on what agility isn’t.

People outside the field are frequently bemused by the words chosen as Agile jargon, particularly in the Scrum framework:
A Scrum is not held only when a product development is stuck in the mud.
A Scrum Master doesn’t tell people what to do.
Sprints are conducted at a sustainable pace.
Agility is not the same as speed. Arguably, in agile environments, speed isn’t the same thing as velocity either.

Many teams measure velocity, a crude metric of progress, only useful to enable estimation of how much work should be scheduled for the next iteration, often guessed in ‘story-points’, representing relative ‘size’ but in agile environments, everything is optional and subject to change, including the length of the journey.

If agility isn’t speed, what is it? It is lots of things but the one that concerns us here is the ability to change direction quickly, when necessary. Agile teams set off in a direction, possibly with a destination in mind but aware that it might change. If the journey throws up unexpected new knowledge, the customer may wish to use the travelling time to reach a destination now considered more valuable. The route is not one straight line but a sequence of lines. It could end anywhere in change-space, including where it started (either through failing fast or the value of the journey being exploration rather than transportation.) Velocity is therefore current progress along a potentially windy road of variable length, not average speed through change-space to a destination. An agile development is really an experiment to test a series of hypotheses about an organisational value proposition, not a journey. Agile’s greatest cost savings come from ‘wrong work not done’.

Agility is lightweight, particularly on up-front planning. Agile teams are small and aim to carry everything they need to get the job done. This enables them to set off sooner, at a sensible pace and, if they are going to fail, to fail fast, at low cost. Agility delivers value as soon as possible and it front-loads value. If we measured velocity in terms of value instead of distance, agile projects would be seen to decelerate until they stop. If you are light, immovable objects can be avoided rather than smashed through. Agile teams neither need nor want momentum, in case they decide to turn fast.

Software Development Science

I’ve tried to write this post before https://andywootton.wordpress.com/2015/07/07/software-life-cycle-part-2-from-craftsmanship-to-computational-science/
but I just answered a question about ‘software engineering’ on LinkedIn and I’m pleased with the brevity:

Agile software development is the application of the scientific method to understanding the requirements for a software product. Each iteration is an experiment to confirm a hypothesis of what the customer should really have wanted at the beginning, if they’d known what they know now.

Lean or Agile? Pick any two.

I still see many people writing about adopting Lean and/or Agile software development. I can remember how difficult it was for my team to work out what ‘Agile’ was and I think it has got harder since, as growing popularity has drawn charlatans into the area. I see two main types of useful articles.

1. What (theory) : “It’s a philosophy” articles which usually point first towards the different values of agile and lean practitioners. But you can’t “do” a philosophy, so we get:
2. What (practice) : Methodology – the study of methods that embody the philosophies. Many will say that Lean & Agile are not processes but I disagree; I think they are ‘software development process’ change processes.

I’d like to try something different: WHY?

The old ways of planning engineering projects, used for building a tower block, didn’t work for software. We don’t know enough, with sufficient certainty at the beginning of development to design top-down and are rarely sufficiently constrained by physics to be forced to build bottom-up.

Unusually for computing, the words ‘lean’ and ‘agile’ have useful meanings.

Lean is about ‘travelling light’, by avoiding waste in your software development process. It uses observation and incremental changes of your current process, while you incrementally deliver business value via working software.

Agile is about doing only valuable work, being nimble and able to change direction, in response to changed requirements or better understanding. It recognises that there are very few completely stable business processes, so software developers need to identify changes that will have impact on the software under development and apply effort in a new direction.

I recommend that you consider both approaches, as they are complementary. Neither removes the need for appropriate engineering practices. We’re only throwing out hard-engineering stuff we packed but didn’t prove useful on a software journey. We throw out what we don’t need, to prevent the weight of unecessary baggage holding us back.

Agility at Human Scale

[ I wrote the post below on LinkedIn, as a reaction to something I read which annoyed me. One person whose opinion I respect liked it, so I thought I’d share it here. I don’t understand where old LinkedIn posts go and you never know when you might need a positive thought. ]

I’m always surprised how many people believe software development is about turning what they think someone wants into code, when the really hard part is understanding what they will turn out to have really needed, at the end.

OK, coding is hard too; so don’t waste effort writing the wrong thing. The answer is not a better specification, a more demanding project manager or more people. The answer is to embrace your human limitations and think small. Be lean and agile. Work as a team. Go back for the wounded. Be a human.

Remoting-in to the Virtual Box

As no-one spoke up yesterday, I’ll assume everyone has accepted the notion that all software runs in a virtual universe, free from the laws of physics. That’s why we don’t need to run software developments like an engineering project. They are not subject to all of the constraints that make engineering hard. We can ‘build the roof first’ and worry about ‘how strong the foundations need to be’ later, when we understand more about the model we have built of our incomplete idea. Like this blog post, Agile products are almost free-floating in a world of our invention, until they need to communicate with people. We are stuck here, interacting with this parallel universe, using our big, heavy ape arms and clumsy interfaces. We drag behind them like tired children.

Did you see the original ‘Tron’ film? Do you remember how the programmers’ personalities were represented by the programs? That was a true story. Programs can be gentle, kind, beautiful but shallow, or bullying ego-maniacs, just like their creators. They can appear to have a certain character while actually being something else entirely. Software reflects aspects of the personalities of it’s creators, as expressed within their self-imposed cultural boundaries.

I think most people reading this will accept that evolution theory is most likely true and that genes carry the necessary code to make new life. I want to propose my own hypothesis. I don’t know if I’ve re-invented an old idea so please tell me if you’ve heard of it before. I think that every form of life has its own culture and that DNA and culture have evolved together in a symbiotic relationships, like a third interlocked spiral. The main difference is the speed at which the invisible cultural strand can change. We may still have the emotional responses of cave dwellers due to our DNA but we can change our political and religious opinions in a day. Every system that survives, protects itself, so we have evolved early-adopters, fashion-victims, people who want to fit in and reactionaries, to quality-check dangerous ideas. As a species we resists change, because change has proved to be really dangerous. At the same time, we constantly strive to try something new because that has been proved to give evolutionary advantage, if you don’t die trying. The variation in the attitude of humans is one of our evolutionary advantages. The two are kept in balance by death of the over adventurous and economic failure of the over-cautious.

“Where’s he going with this?”, you may ask. Well: just as Richard Dawkins put forward the idea that we are carriers of our selfish genes, I’m in turn proposing the idea that selfish us and our selfish genes are carriers of our cultures and that if we can project human culture into software, we can free it, and ourselves from the rules of physics and the constraints of limited resources and thereby, finally, from the drive to be selfish.

Computational Science + Informatics = Software Development

1970s At university, I studied Physics and:

• Computer Science

but it had very little to do with computers. It was far more about becoming a

• Computer Programmer

1980s When I started work I heard about the increasing formalisation of the software development process and I wanted to be a

• Software Engineer

1990s I’d moved into server management by the time I qualified as a

• Chartered Information Systems Engineer

which fit in with my thinking that information is the important resource in any organisation but took me no closer to knowing how to make great software efficiently. I became disillusioned by watching people trying to apply the rigorous methods of hard engineering to the uncertainties and unknown complexities of software. I became interested in prototyping, incremental delivery and Agile.

2000s I was increasingly drawn to the idea that constructing software is a design discipline and that a software developer needed to be a

• Software Craftsman

2010s After a few years working with an agile software team, I decided I wanted to try writing software again myself. A few weeks ago, an online MIT course on Python programming introduced me to an idea that I felt very comfortable with: a good developer is a

• Computational Scientist (I’d settle for ‘Computing Scientist’)

and I’d add the option

• Informatician, which overlaps with what librarians do now

Agile development processes are adaptations of the scientific method, to research what customers want and work out how to give it to them in the way that meets their opinion of best value, and I don’t see why software developers can’t be their own customer. Every great new software component starts off as someone’s experiment.

Ordering Scrum sprint-backlog by user-story ‘benefit:cost ratio’ of equal sized jobs, to deliver better business value

WARNING: this entry is in the ‘really quite geeky / technical specialist’ category. It assumes knowledge of an Agile software development framework called Scrum. If that isn’t you then you’ve probably got better things to do with your life. Maybe I need a new blog for things THIS weird?

[ The title of this article has been changed but it needs to be revised further to explain that ‘smallest job first’ is assumed in the examples before ordering of equal size jobs by user-story benefit:cost ratio. 29/4/2014 ]

I was recently introduced to a management fashion for calculating the http://en.wikipedia.org/wiki/Net_present_value of projects. It appears to offer an excellent mechanism for demonstrating the greater value delivered by Agile, if we knew the business value of each requirement.

I started to think about how we might estimate the approximate benefit that would delivered by each user-story and whether, within a Scrum sprint, the order that user-stories were tackled would change the total benefit delivered. In the LinkedIn Agile group I proposed the following thought experiment:

3 user-stories, worth 2, 1 & 1 units of business value and estimated at 20, 10 & 10 story-points respectively. Assuming no dependencies, the same priority and 1 developer, what order do you tackle them in?

“Working in the lab, late one night” these were the results of that experiment.

SprintByValueOrder (only 3 pages, including 2 diagrams)

I would welcome any feedback. Was that obvious? Has it been said before or is it new? Should I develop a mathematical proof and publish in a technical journal? Please don’t make me do a literature. I read REALLY slowly.