Agility vs Momentum

[ This post is aimed at readers with at least basic understanding of agile product development. It doesn’t explain some of the concepts discussed.]

We often talk of software development as movement across a difficult terrain, to a destination. Organisational change projects are seen as a lightening attack on an organisation, though in reality, have historically proved much slower than the speed of light. Large projects often force through regime change for ‘a leader’. Conventionally, this leader has been unlikely to travel with the team. Someone needs to “hold the fort”. There may be casualties due to friendly firings.

Project Managers make ‘plans’ of a proposed ‘change journey’ from one system state to another, between points in ‘change space’, via the straightest line possible, whilst ignoring the passage of time which makes change possible. Time is seen as distance and its corollary, cost. The language of projects is “setting-off”, “pushing on past obstacles” and “blockers” such as “difficult customers”, along a fixed route, “applying pressure” to “overcome resistance”. A project team is an army on the march, smashing their way through to a target, hoping it hasn’t been moved. Someone must pay for the “boots on the ground” and their travel costs. This mind-set leads to managers who perceives a need to “build momentum” to avoid “getting bogged down”.

Now let us think about the physics:

•  momentum = mass x velocity, conventionally abbreviated to p = mv.
  At this point it may also be worth pointing out Newton’s Second Law of Motion:

• force = mass x acceleration, or F = ma
  (Interpretted by Project Managers as “if it gets stuck, whack it hard with something heavy.”)

What about “agile software developments”? There is a broad range of opinion on precisely what those words mean but there is much greater consensus on what agility isn’t.

People outside the field are frequently bemused by the words chosen as Agile jargon, particularly in the Scrum framework:
A Scrum is not held only when a product development is stuck in the mud.
A Scrum Master doesn’t tell people what to do.
Sprints are conducted at a sustainable pace.
Agility is not the same as speed. Arguably, in agile environments, speed isn’t the same thing as velocity either.

Many teams measure velocity, a crude metric of progress, only useful to enable estimation of how much work should be scheduled for the next iteration, often guessed in ‘story-points’, representing relative ‘size’ but in agile environments, everything is optional and subject to change, including the length of the journey.

If agility isn’t speed, what is it? It is lots of things but the one that concerns us here is the ability to change direction quickly, when necessary. Agile teams set off in a direction, possibly with a destination in mind but aware that it might change. If the journey throws up unexpected new knowledge, the customer may wish to use the travelling time to reach a destination now considered more valuable. The route is not one straight line but a sequence of lines. It could end anywhere in change-space, including where it started (either through failing fast or the value of the journey being exploration rather than transportation.) Velocity is therefore current progress along a potentially windy road of variable length, not average speed through change-space to a destination. An agile development is really an experiment to test a series of hypotheses about an organisational value proposition, not a journey. Agile’s greatest cost savings come from ‘wrong work not done’.

Agility is lightweight, particularly on up-front planning. Agile teams are small and aim to carry everything they need to get the job done. This enables them to set off sooner, at a sensible pace and, if they are going to fail, to fail fast, at low cost. Agility delivers value as soon as possible and it front-loads value. If we measured velocity in terms of value instead of distance, agile projects would be seen to decelerate until they stop. If you are light, immovable objects can be avoided rather than smashed through. Agile teams neither need nor want momentum, in case they decide to turn fast.

Advertisement

Women’s Day Intuition

The first thing I did yesterday, on International Women’s Day 2017, was retweet a picture of Margaret Hamilton, allegedly the first person in the world to have the job title ‘Software Engineer’. The tweet claimed the pile of printout she was standing beside, as tall as her, was all the tweets asking “Why isn’t there an International Men’s Day?” (There is. It’s November 19th, the first day of snowflake season.) The listings were actually the source code which her team wrote to make the Apollo moon mission possible. She was the first virtual woman on the Moon.

I followed up with a link to a graph showing the disastrous decline of women working in software development since 1985, by way of an explanation of why equal opportunities aren’t yet a done deal. I immediately received a reply from a man, saying there had been plenty of advances in computer hardware and software since 1985, so perhaps that wasn’t a coincidence. This post is dedicated to him.

I believe that the decade 1975 – 1985, when the number of women in computing was still growing fast, was the most productive since the first, starting in the late 1830s, when Dame Ada Lovelace made up precisely 50% of the computer software workforce worldwide. It also happens to approximately coincide with the first time I encountered computing, in about 1974 and stopped writing software in about 1986.

1975 – 1985:
As I entered: Punched cards then a teletype, connected to a 24-bit ICL 1900-series mainframe via 300 Baud accoustic coupler and phone line. A trendy new teaching language called BASIC, complete with GOTOs.

As I left: Terminals containing a ‘microprocessor’, screen addressable via ANSI escape sequences or bit-mapped graphics terminals, connected to 32-bit super-minis, enabling ‘design’. I used a programming language-agnostic environment with a standard run-time library and a symbolic debugger. BBC Micros were in schools. The X windowing system was about to standardise graphics. Unix and ‘C’ were breaking out of the universities along with Free and Open culture, functional and declarative programming and AI. The danger of the limits of physics and the need for parallelism loomed out of the mist.

So, what was this remarkable progress in the 30 years from 1986 to 2016?

Good:

Parallel processing research provided Communicating Sequential Processes and the Inmos Transputer.
Declarative, non-functional languages that led to ‘expert systems’. Lower expectations got AI moving.
Functional languages got immutable data.
Scripting languages like Python & Ruby for Rails, leading to the death of BASIC in schools.
Wider access to the Internet.
The read-only Web.
The idea of social media.
Lean and agile thinking. The decline of the software project religion.
The GNU GPL and Linux.
Open, distributed platforms like git, free from service monopolies.
The Raspberry Pi and computer science in schools

Only looked good:

The rise of PCs to under-cut Unix workstations and break the Data Processing department control. Microsoft took control instead.
Reduced Instruction Set Computers were invented, providing us with a free 30 year window to work out the problem of parallelism but meaning we didn’t bother.
In 1980, Alan Kay had invented Smalltalk and the Object Oriented paradigm of computing, allowing complex real-world objects to be simulated and everything else to be modelled as though it was a simulation of objects, even if you had to invent them. Smalltalk did no great harm but in 1983 Bjarne Stroustrup left the lab door open and C++ escaped into the wild. By 1985, objects had become uncontrollable. They were EVERYWHERE.
Software Engineering. Because writing software is exactly like building a house, despite the lack of gravity.
Java, a mutant C++, forms the largely unrelated brand-hybrid JavaScript.
Microsoft re-invents DEC’s VMS and Sun’s Java, as 32-bit Windows NT, .NET and C# then destroys all the evidence.
The reality of social media.
The writeable Web.
Multi-core processors for speed (don’t panic, functions can save us.)

Why did women stop seeing computing as a sensible career choice in 1985 when “mine is bigger than yours” PCs arrived and reconsider when everyone at school uses the same Raspberry Pi and multi-tasking is becoming important again? Probably that famous ‘female intuition’. They can see the world of computing needs real functioning humans again.

Change Time

After some time trying to think about almost nothing, the last 24 hours have been an alarm call. As others come out of hibernation too, they post interesting stuff and Radio 4 provoked me with a discussion on facts and truth. Now Marc Cooper is at it, with difficult  links about computation and I’m all on Edge https://www.edge.org/response-detail/26733
Before I read about “discrete tensor networks”, I need to write down my own ideas about time, so I will know in the future what I thought, before my mind was changed.

I am ill-equipped for this task, having only 1 term of university maths to my name so I intend to talk in vague, abstract terms that are hard to argue with.

Much of physics is very dependent on Time, like almost all of computer science and business management theory. You can’t have change without time, it seems. Einstein talked about space-time, mostly in the language of mathematics. I can just about order a beer in math(s) but I can’t hold a whole conversation. I know what the first 3 dimensions are: left-right, up-down and back-forward. My personal model of the 4th dimension is that same space in continuous state-change through time. There are a few things I’m not happy about:

• There is no evidence that time is either continuous or constant.
• We only have evidence of time being a one-way dimension.
• What the heck does ‘continous state-change’ mean? Is state a particle or a wave? Make your mind up, physics!
• There’s that troubling many-worlds interpretation of the universal ‘WAVE’function (which I don’t understand either) which says that everything that might have happened did, in other universes. I don’t like this. Yes, that’s my entire justification – I don’t like the conclusion of a thought process I don’t even understand. It doesn’t feel right.

I’ve been learning about the functional programming language Clojure which does not ‘mutate (change) state’. It doesn’t have ‘variables’ like the more common imperative languages such as FORTRAN, BASIC, C, Java or Python. In Clojure, data flows through functions and is transformed from one form to another on the way. It is basically magic. In a pure functional program, no state is changed. State-change is called a “side-effect”. Sadly, side-effects are required to make a program do anything useful in the real world. Arguably, the purest magic is encapsulated in the world of mathematics and the physical world is a messy place that breaks things.

Clojure models time. It does not model the real world by replacing the current value in a variable and throwing the old value away but by chaining a new value onto the end of a list of all previous values.

Now let us extend this idea ‘slightly’ in a small thought-experiment, to a 3-D network of every particle state in the universe.

Space-time now has 2 regions:

1. The past – all historic states of those particles as a theoretical chain of events
2. The future – all possible future states of the universe; effectively an infinity of all possible future universes that could exist, starting from now.

Which brings us to what I mean by ‘now’ – a moving wave at the interface between the past and the future, annihilating possible future universes. Time becomes a consequence of the computation of the next set of states and the reason for it being a one-way street becomes obvious: the universe burned its bridges. Unless the universe kept a list, or we do, the past has gone. Time doesn’t need to be constant in different parts of the universe, unless the universe state ticks are synchronous but it seems likely to be resistant to discontinuities in the moving surface. I imagine a fishing net, pulled by current events.

It’s just an idea. Maybe you can’t have Time without change.

[ Please tell me if this isn’t an original idea, as I’m not very well read.
I made it up myself but I’m probably not the first. ]

2017: Fighting back

2016 was an excellent year for the progress of Evil. It’s time for those of us who believe there ‘is such a thing as society’ to start pushing back harder, for a happier 2017.

I plan to take on fascism, corrupt media, lying politicians, hierarchy in general, fixing democratic reform, climate chaos and replacing the failing socialism and capitalism systems, as the year warms up; but for now lets warm up with open communication protocols. This article recommends the Franz messaging application:
http://www.omgubuntu.co.uk/2016/12/21-must-have-apps-ubuntu. It says:

“The days of multi-protocol instant messengers are long gone, with (mostly) proprietary mobile-first services now ruling the roost.

Want to chat on WhatsApp, Facebook Messenger, Telegram or another well-known messaging service on Ubuntu, without using your browser? Try Franz.”

Or don’t. Ask why Google needed to assimilate XMPP into Google Talk, in order to destroy it, so it could launch Hangouts to compete with Skype and Apple and control yet another market. It got beaten and left us with the current fragmented mess. When will we all learn the lesson to choose open protocols not products and suppliers that do what is good for us rather than for themselves? We will only progress in software if we stop throwing everything away every few years and starting again with something worse. Customers and governments must start demanding this. We must NOT Brexit or Trump on international communication. Tribal thinking is Bad, kids!

Can anyone recommend a good XMPP service provider that serves the UK which REALLY believes in open standards and privacy, like Google, Facebook, Apple and (occasionally) Microsoft pretend to? Maybe with an old-fashioned SMTP/POP email service that isn’t Gmail. Liars must be punished, or they’ll keep doing it… and we’re back to politics.

This week I mentioned a product in a closed Slack group and received targeted marketing next time I logged into Facebook. What else is being snooped on?

A Functional Mindset

When I started learning Clojure, I thought I knew what functional programming was but I’ve learned that the functional paradigm is now more than I expected.

Everyone agrees that it’s a computational model based on evaluation of mathematical functions, which return values. This is generally contrasted with imperative programming languages such as FORTRAN, C, JavaScript or Python, which are also procedural and some of which are object-oriented but may make functional coding possible, in a hybrid style. I wouldn’t recommend learning functional concepts in a language that gives you short-cuts to stray back  to more familiar territory.

Clojure is a member of the Lisp family, first specified in 1958. The unusual feature of Lisps is their homoiconicity – code and data are the same thing. Learning Clojure has informed my thinking about business process change.

Some modern, functional languages such as Clojure use immutable data whenever possible, to eliminate side-effects. This allows better use of multi-core processors but requires a complete change in thinking, as well as programming style. ‘Variables’ are replaced by fixed ‘values’, so loops have to be replaced by recursive functions. New data can be created but it doesn’t replace old data. Yesterday’s “today’s date” isn’t automatically wiped when we decide today has happened.

Objects with their methods and local data were designed for simulating the current state of real-world objects by changing (mutating) object data state. The object model, like relational databases, has no inbuilt representation of time. Functional programming splits these objects back into separate functions and data structures and because values can’t change, they may be transformed by flowing through networks of functions, some recursive, to keep doing something until a condition is satisfied. Eventually, code must have a side effect, to tell us the answer.

Rather than computation being a conditional to-do list with data being moved between boxes, it becomes a flow of data through a network of ‘computing machines’; and the data and machines can be transformed into each other.

I hear that map, reduce & filter data transformation functions will change my world again.

Lispbian Pi. A Lambda Delta.

I’m conflicted. Part of me says that ‘us old timers’ shouldn’t assume ‘the way things were when we were kids’ were better but we know the Raspberry Pi was an attempt to recapture the spirit of the BBC Micro Model B and that seems to have gone quite well. I got a Pi 2 and I’ve worked out that it is more powerful than the first computer I worked on, a DEC VAX-11/780 which supported about 16 terminals, most used for teaching college level computing. Having that machine to myself would have been an unimaginable amount of processing power for one developer. Banks ran their financial modelling software on boxes like that. So why does the Pi feel so slow? We wasted our gains on GUI fluff.

When I started computing you learned just enough of the command language to get going. So, that’s bash on a pi. Then an editor. For reasons that should become obvious, let us choose emacs. When I first used the VAX/VMS operating system, it didn’t have command line editing. If you made an error, you typed it all again. Getting the facility to press up-arrow, edit the command and re-execute it was a big advance. We should keep it. Bash has that, using a sub-set of the emacs keys, so that’s a way into emacs.

The next big improvements I remember were X windowing and symbolic debugging. We got debugging first but it became far more powerful with multiple terminal windows. The GUI was OK, I guess but DEC didn’t give us many free toys so the main advantage to a developer was having lots of terminal windows. emacs can do that, without the overhead of X.

When I decided to re-learn coding a while back, I got my shortlist of languages down to Python, Java and JavaScript but picked Python because I was already learning a new language and the Object paradigm, so I didn’t want to have to learn web at the same time. I heard about the modern Lisp dialect Clojure and changed horses mid-stream. I’m convinced by the argument that functions and immutability can save the universe from the parallel dimension.

Last night I deep-dived into emacs and found myself in an editor session with 4 windows. Why do I need more than that to learn about computation and data transformation? This guy seems to have come to a similar conclusion http://hackaday.com/2015/09/23/old-lisp-languaged-used-for-new-raspberry-pi-os/ I’ve also wondered whether a purely functional OS might make Sun’s ‘the network is the computer’ dream a little easier. emacs is written in Lisp.

I think a dedicated Lisp machine may be a step too far back. How would you browse in the world-wide hypertext library when you got stuck? But a Linux with bash, emacs, the Java Virtual Machine and libraries, Clojure via Leiningen and Cider to plug everything together might make a fine Lispbian Pi! Is all this chrome and leather trim completely necessary in the engine compartment?

It is unfortunate that the Raspbian upgrade left Leiningen broken.

A Page Break in Markdown (ReText)

Since I started to use Markdown, I’ve found it helpful to use the ReText editor in ‘Live Preview’ mode (Ctrl-L) so I can write Markdown markup language in the left-hand window and see What I Got in the right-hand window. It’s easy to forget that there is no official standard for Markup, so when you want to do something ‘none standard’, different implementations may have handled your problem their own way. Some answers I found didn’t work.

I wanted to force a page-break when I print out a cheat-sheet I’ve written and intend to update, as I  learn emacs key-bindings. There are several already but things weren’t arranged in an order that was helping me to learn which keys work in the Linux bash shell (another command-line editor option is available but emacs is the default.)

ReText supports something that appears to be based on HTML/CSS. Other interpreters use LaTeX syntax. In ReText, page-break-before style is applied after the text you want to appear at the top of the new page.

"...some text from the previous page

## New Section Title {: style="page-break-before: always" }

Text on new page"

Licensing Pain

It’s a very long time since I’ve used any software that requires a licence but I decided to try the patented MPEG2 codec licence for my Raspberry Pi, to decode MPEG2 in hardware. Now I remember why I disliked software licensing so much. This was the process I followed for the Pi’s default Raspbian operating system.

Step 1: Find my Pi’s serial number. At the command line, type:  cat /proc/cpuinfo
The last line, starting Serial: is the unique number of your Pi.

Step 2: Find where to buy it. http://www.raspberrypi.com/license-keys/
Enter the serial numer of the Pi you wish to buy the licence for..
Pay £2-40 and wait for an email. It may take up to 72 hours. I assume this is because a unique(ish) key must be generated.

Step 3: Add the following line to “the config.txt file in the FAT partition of your SD card:

decode_MPG2=0xfdb4a3ac” You may note that this is not the 10 digit hex you were expecting. That doesn’t matter. The first two digits must be implied zeros.

Others told me this meant sudo my chosen editor to add the line to /boot/config.txt then save and reboot.

and “If you want to verify that the codecs are now enabled, the following commands will report their status:

vcgencmd codec_enabled MPG2”

Step 4: Wonder why that didn’t work.

In my case, I was typing:
“vcgencmd codec_enabled mpg2”. It said mpg2 was disabled. In reality, there was no “mpg2”, because it’s “MPG2”. Case matters.

 

Moderately Grouped

One of the rules I try to live my life by is: “Small pieces, loosely joined”

Then this happened.

http://phys.org/news/2015-06-social-networks-group-boundaries-ideas.html

I don’t know who I am any more. I already feared de-selection from the cult of Unix and now this.

Then I realised that although I favour hi-fi separates, I don’t  design my own amplifiers and hand-wire the components. I don’t compile Linux from source every time. I’m not a fanatic.

Tooling-up for agile state-transition

This post started out in life as an answer to a question about ‘backlog tooling’ on the LinkedIn ‘Lean & Agile’ group. Someone had given the culturally acceptable answer that the best solution is simple cards or post-it notes on a board or wall. I normally just let that pass because I don’t have a better solution to offer but this time, THIS happened:

I’m about to be intentionally provocative. We know that we are engaged in transforming a multidimensional network of business functions from one poorly understood and transient state to another, currently ill-defined, future state that we hope will emerge from the mist as we travel in it’s general direction. In organisations of any size, this change process is likely to run in parallel with other change programmes, some of them probably deliberately kept secret by people whose pay grade exceeds their ability to make rational judgements about the basis of who “needs to know”.

Amongst this chaos, the chosen tool of ‘the Agile Community’ is a single, 2-dimensional view of a ‘list of lists’, sometimes known as ‘a tree’ or ‘a star’, all of which are topologically equivalent representations of items’ states in the backlog of each Product development. Our best software tools are little more than a model of cards on a board.

Why do we expect the complex, dynamic agile change process to map any more adequately onto a tree of cards than it does onto a hierarchical management structure? Have we learned nothing from our mistakes of modelling within the limitations of the filing cabinet and the typewriter? Perhaps agilists don’t value tools because our tools aren’t fit for purpose.

If all you have is lists representing vague descriptions of changes between two mental models you hope your whole team all share, perhaps the limited nature of the backlog tool isn’t your biggest problem. The backlog items reference changes to an implicit model of roles and the objects in the business domain of your product. My advice is to make it explicit.