Tag Archives: Information Theory

The Unvexing

This post comes to you courtesy of Graham Lee at: http://www.sicpers.info/2016/11/the-vexing-problems-in-programming/ who reminded me about Value. We agilists (people who believe in agile software development) talk about value a lot but haven’t  agreed on a consistent definition of the word. As an illustration of two of the options:

“The value of this car is £9,500.” Is that to a seller or a buyer? Unless there is a difference, no trade will take place. Or is it a market rate? The long acquisition dance between Yahoo! and Google makes an interesting case study. This is: a value. A ‘fact’, right or wrong. It is the kind of thing developers in imperative programming languages put into variables and mutate but pure functional programmers squirrel away for ever, to embarrass themselves at their past Wrongness.

In contrast, “This car cost £9,500 pounds. I think that was quite good value but the model that costs £1,000 more is even better value.”, is a relative benefit:cost calculation, yet the manufacturer may market a ‘value’ model, which is just cheap. This is: a value judgement. An explicit or implicit comparison relative to something else, followed by a decision. It is a computation.

When Graham says, the most vexing problem of software product  developers is their inability to “compare the expected value of their work to the expected cost of the work.”, I think he means  ‘business benefit of their work’, and cost normally equates closely to development time. All the customer really wants to know is: “Is this the best investment I could make now?”

Graham goes on to say that we are very bad at estimating how long something will take. This is true but we are much better at estimating small jobs accurately than large ones.  Uncertainty increases exponentially with length of sequence of actions, to slightly corrupt Shannon’s Information Theory. This is why the Fibonacci sequence is often used as a sizing tool.

Agility accepts this reality. It addresses the list of things the customer currently wants, in highest benefit:cost ratio order (guessed by a business domain expert, based on guesses by an agile develooment team.) It doesn’t yet know whether the ‘whole job’ is worth doing. It decides only whether to risk the next small, cheap step and keeps doing that, as long as the ‘value’ is Good Enough. While value is high and risk is low enough, keep going.

A journey of any length starts with the first step, so why worry about whether or not it is going to be 1000 Miles? The hard part is to make it a journey, rather than simply wandering about, lost.

Then there are: our values. Our personal decisions about what matters most to us. Do we go home to read our children a bedtime story or work late and win that promotion so they have greater financial security in the future? What do we really care about and what are we willing to pay for it? This too is relative. Politics is the art of persuading you to modify your personal values. Currently, cheating is allowed.

But is IT art?

What is IT? It’s Information Technology, clearly but why is it that rather than “Information and Communication Technology”, as UK schools call it? Is communication simply information on the move? Are messages only data in transit? Conversely, ‘are’ data messages that have been frozen? Is communication more than messages? Perhaps communication is related to messages as information is related to data?

What are data, information, knowledge, wisdom and enlightenment? Most of modern computer science is built upon Claude Shannon’s Information Theory: http://en.wikipedia.org/wiki/Information_theory

“Information theory is a branch of applied mathematics, electrical engineering, and computer science” How would the worlds of applied mathematics, electrical engineering, and computer science take it if I publish a book that points out that Shannon’s definition of Information is, at best, unhelpful and possibly wearing holey clothes? Well, I’m sure that because of the enquiring minds of scientists etc.

In ‘A Mathematical Theory of Communication’:
http://cm.bell-labs.com/cm/ms/what/shannonday/shannon1948.pdf, Shannon said,

“The fundamental problem of communication is that of reproducing at one point either exactly or approximately a message selected at another point. Frequently the messages have meaning; that is they refer to or are correlated according to some system with certain physical or conceptual entities. These semantic aspects of communication are irrelevant to the engineering problem. The significant aspect is that the actual message is one selected from a set of possible messages.”

Shannon’s theory appears not to relate to what we would now call ‘information’ at all but to data or even pattern.

Since we are questioning the foundations of computer science, what is art?  Is art everywhere?

Donald Knuth produced a 3-volume ‘Art of Computer Programming’. Clearly he saw the creation of new algorithms as a creative act.

Does Software Engineering deserve it’s name? Or is it Software Craftsmanship? Agile software development is a recognition that the project management methods from hard engineering cannot be applied to software, where the butterfly effect of uncertainties in requirements can lead to chaos. Agile works by leaving decisions later in the process when a feedback loop from incrementally delivered software better informs the requirements and allows emergent designs. But are Agile/Lean teams doing incremental delivery ever able to create beautiful design? Does great art/design need an architect who owns and drives the creative vision?

I remember going to a talk about Smalltalk where the speaker claimed that “No worthwhile piece of software has ever been written by a team of more than 2, maybe 3 people.” to an audience largely made up of the Inland Revenue IT department.

How to make friends and influence people. You know that Shannon bloke? Pah!