Software Life-cycle. Part 1 – From Engineering to Craftsmanship

I graduated just after the Structured Programming War was won. I was probably the first generation to be taught to program by someone who actually knew how; to be warned of the real and present danger of the GOTO statement and to be exposed first to a language that didn’t need it. I didn’t need to fall back to assembler when the going got tough or to be able to read hex dumps or deal with physical memory constraints. I entered the computing profession just as people were starting to re-brand their programmers as ‘software engineers’ and academics were talking of ‘formal methods’ then ‘iterative development’ and ‘prototyping’ as we lost faith and retreated, as the techniques borrowed from other engineering disciplines continued to disappoint when applied to software.

After some years away from software development, I returned to find ‘Agile’, ‘Lean’ and ‘Software Craftsmanship’. We’d surrendered to the chaos, accepted that we weren’t designers of great engineering works but software whittlers. I was pleased that we’d dropped the pretence that we knew what we were doing but disappointed that we’d been reduced to hand-weaving our systems like hipsters.

There had been another good change too: The Object Model. The thrust of software engineering had often been decomposition but our model had been the parts breakdown structure, the skeletal parts of a dead system. Objects allowed us to model running systems, the process network at the heart of computation. I’ve recently seen a claim that the Unix command line interface with its pipes and redirection was the first object system. Unix begat GNU and Free software and Linux and close to zero costs for the ‘means of production’ of software. I don’t believe that Agile or Lean start-ups could have happened in a world without objects, the Internet or Free software. Do you know how much work it takes to order software on a tape from the US by post? I do.

So here we are, in our loft rooms, on a hand crafted loom made of driftwood and sweat, iterating towards a vague idea emerging out of someone’s misty musings and watching our salary eroded towards the cost of production. Is this why I studied Computer Science for 3 years? Who turned my profession into a hobby activity?

Advertisement

IF socialist THEN IF democratic AND distributed_power THEN Green_Party

I always like ‘crossing the streams’ of my apparently disparate obsessions. Last night my long term fascination in whether the Free Software movement can survive a war with software capitalists, collided with my recent interest in the Green Party.

I have struggled for years to find any political party in the UK that comes close to my political ideals. I am economically Left, Right in terms of Liberty and think the environment is sending us very strong signals that capitalism has been a more destructive failure than communism. I believe in equality of opportunity rather than equality and in distributed rather than centralised power. I prefer incremental change to the unpredictability of revolution. I see little difference between nationalism and racial or religious hatred. No party quite fits my shopping list but at the recent General Election, I decided that the current Green Party comes closest, so far.

Richard M. Stallman, instigator of the GNU Project, who kicked off the the GNU GPL (General Public Licence) and indirectly, the CopyLeft movement has turned his attention to politics in recent years. Arguably, his life’s work, Free Software, is the practical application of ‘social ownership of the means of production’ but RMS is the ‘Marmite’ of the Free Software community. His almost total lack of pragmatism and slightly abrasive personality towards anyone who disagrees with him divides opinions but I have learned over the years to never question his basic logic. He has a habit of being right, even when it is inconvenient.

I have become increasingly suspicious of large corporations and hierarchical power structures. RMS’s idea on ‘too big to fail’ is the best economic solution to monopolies I’ve ever seen: http://blogs.reuters.com/great-debate/2013/02/04/fixing-too-big-to-fail/

In his ‘political notes’, at (https://www.stallman.org) on the day of the UK election, RMS wrote this:

“18 May 2015 (Revitalizing the Labour Party)

Making the Labour Party good for something depends on bottom-up community organizing. Acting like a right-wing party produces a right-wing party.

Perhaps instead of revitalizing the Labour Party, Britons should go Green.”

I also noticed that his preferred US presidential candidate is an independent who describes himself as a ‘Democratic Socialist’. This is surprisingly different to the ‘Social Democrat’ “…view of reform through state intervention within capitalism” https://en.wikipedia.org/wiki/Democratic_socialism

“Decentralised socialism”. “…seeing capitalism as incompatible with the democratic values of freedom, equality and solidarity.” Maybe I’m a Democratic Socialist now. Are you?

If RMS thinks the British Labour Party has lost touch with socialism, then I’m a little more comfortable about having thought the same for the last few years too. I was unhappy with the LibDems prioritising economic growth over environmental danger and perhaps I understand better now where my intuitive reaction to that came from.

Social vs Capital Part 2

To recap:

Unix happened because companies trying to run their businesses using software didn’t like being dependent on the whims of hardware manufacturers. Each manufacturer defined their own hardware architecture. Customers wanted hardware-independence.

GNU/Linux and the Free & Open-Source Software movement happened because coders didn’t like waiting for someone else to fix their problems or to decide not to fix them. They wanted access to the source code and the legal right to change software and share their changes. They wanted software-supplier independence.

I simplified last time. BSD, GNU & Linux weren’t the only game in town. An important book was published: ‘Operating Systems, Design And Implementation’ by Andrew Tanenbaum. Tanenbaum believed in giving his students access to working source code. He had used Unix but when AT&T pulled up their draw-bridge, he needed a replacement that his readers could use and change – so he wrote one. ‘Minix’ was a minimal rewrite  of the key functions of Unix v7 which ran on a twin-floppy IBM PC (just about.) Unfortunately, the publishers, Prentice-Hall, insisted on retaining Copyright to the software. To get a copy, you had to buy a computer science book you probably didn’t want but. But I bought it, as did a young man called Linus Torvald. Tanenbaum was also instrumental in the production of the Amsterdam Compiler Kit which was the starting point for the GNU C compiler. My first sight of the GNU effort was on a listing for a 9-inch tape from the DEC User Society: It included emacs, gcc & the ACK, to be run on your own commercial Unix system.

Stallman and the GNU organisation were writing another replacement for Unix, free of copyrighted code. Their aim was to ensure that if you used their software, no-one else would ever prevent you from using code, particularly if you had contributed to it. Copyleft was born. ‘Free’ (as in beer) licences were not new. The BSD licence allows anyone to take BSD code and do what they wish with it, including building non-Free code on top and selling it.  OS X is built on FreeBSD but Apple sells licences and protects “its” intellectual property from re-use by competitors, including those companies that contributed code that Apple used. ‘Strangely’ Stallman didn’t think this was fair so worked towards creating the GNU licence, the GPL. The original idea has been described as “viral”. If you used GNU-licensed code then your code was required to be GNU-licensed too and you were required to make it available to anyone who wanted it, for only the cost of reproduction. THe GPL was arguably ‘less free’ because it enforced sharing and prevented commercial exploitation. GPL supporters point out that the code you contribute becomes a marketing tool to sell a future service. You offer a service (typically to write other software) rather than sell a software product licence. There is downward market pressure on price and upward on quality, to provide the best value service.
Compromises were made to the GPL  later, leading to the ‘Lesser’ LGPL. This allowed  software libraries to be used in conventionally licensed commercial software.

Having established the new ground-rules, GNU started work, from the top downwards. The bottom layer, the HURD kernel, has still not been delivered. Fortunately, that guy Linus started at the bottom and worked up. When he and his Internet recruits started needing to test their kernel, the GNU tools were ready. Because GNU & Linux were both copying the same open system interfaces, they worked together.

The Free Software movement happened because all these individuals knew they couldn’t do everything on their own. If they wrote something new or fixed a problem, they gave it away. In return, someone else would have fixed a future problem before they found it and shared the solution with them, free.

People sometimes question what happened to ‘the hippy generation’. It appears that many of them went into computer science and carried right on with implementing a community based on freedom & love, inside any institution that would pay them a salary. ‘The Community’ developed a culture and distributed processes and tools that anyone was allowed to use. When people who had grown up in this community started their own companies, they didn’t follow the IBM model, as Bill Gates and Steve Jobs had. They adopted the Free tool-kits of the hippies. Facebook, Google, the Nokia Maemo/MeeGo team, Red Hat, Canonical (Ubuntu Linux) and the Steam gaming platform come from a new breed of entrepreneur. They are not in business to sell you hardware or software as a product. They sell you a service on the Internet. Many of these services are ‘free at point of delivery’.

But there are costs and someone has to pay. You are no longer locked into a hardware or software supplier but to a single service provider and they have your information, probably the most valuable asset of your organisation.

Social vs Capital Part 1

When I joined ‘the computing industry’ (or was it ‘the data processing trade’?,) there were two kinds of computers: those made by IBM and the others. The others came in two flavours: IBM mainframe clones and ‘trying to be different’. Trying to be different was so successful that IBM were eventually forced to try being different to themselves.  The various hardware families all ran different operating systems. Changing hardware required all your software to be rewritten. Moving from IBMish mainframes meant your data had to be translated into ASCII. The proposed solution to the operating system problem was Unix. Unix was created to give hardware independence, through software portability. It was made easier to ‘port’ Unix by writing it in the C programming language rather than the specific assembly language of the hardware.

This revolution happened within AT&T, a company prevented from competing with IBM by anti-trust legislation. Freed from the profit motive, other than the desire to save costs, they did with Unix what was best for everyone. They gave it and its source code away free to anyone who wanted it. More importantly, they allowed its improvement by universities.

Later, the US government started to allow commercial exploitation of Unix by AT&T. Key source code became subject to non-disclosure agreements and the fastest period of cooperative computing innovation up to that point was closed down.

Two important things came out of this disaster – 1) PCs and hence Microsoft and 2) the Berkeley System Distribution (BSD) of Unix and GNU’s Not Unix (GNU), led by Richard Stallman, whose frustration at not having the source code to fix his own printer gave him such a mighty itch, he kick-started the whole Free and Open Source Software (FOSS) movement and it’s biggest success, the Linux Operating System, recently made popular by Google. Bill Gates’ biggest competitor was never Steve Jobs; it was an idea set loose by idealistic academics – that people are stronger when they share the product of their labours, that you pay people for producing, not for the product. This was a harmless ideal at first because large organisations owned the computers that were a key part of the means of production.

I am indebted to Robin Ince again, for pointing out in his TEDx Dublin talk ‘The Mind is a Chaos of Delight’ http://www.youtube.com/watch?v=0pfOHaWeTr8 that Evolution doesn’t predict only “survival of the fittest” but ‘survival of the just good enough not to die’, which I think explains Microsoft’s success, and for poking me in the profit-motive with his blog entry http://robinince.wordpress.com/2013/11/26/i-was-going-to-jump-in-the-canal-to-save-the-drowning-man-but-then-i-thought-whats-in-it-for-me/, to finally start this troubled tale of open software.

FOSS has been running around in the background, largely unnoticed by the lumbering beasts, much like the early mammals. Apple OS X is built on FreeBSD and Google Android and Chrome OS are based on the Linux kernel. In the long run, Apple and Google may look like the last of the small, fast raptors rather than the first intelligent apes, because somewhere along the way, the sharing became one-directional, and their essentially predatory nature struggled to survive as their more social competitors saw the danger and drove them into the swamp.

In forthcoming posts, I plan to look at the dangers the FOSS communities’ dreams of Freedom are facing in the current collision with Capitalism.