Things I used to be Wrong about – Part 1

I get very annoyed about politicians being held to account for admitting they were wrong, rather than forcefully challenged when they were wrong in the first place. Unless they lied, if someone was wrong and admits it, they should be congratulated. They have grown as a human being.

I am about to do something very similar. I’m going to start confessing some wrong things I used to think, that the world has come to agree with me about. I feel I should congratulate you all.

You can’t design a Database without knowing how it will be used

I was taught at university that you could create a single abstract data model of an organisation’s data. “The word database has no plural”, I was told. I tried to create a model of all street furniture (signs and lighting) in Staffordshire, in my second job. I couldn’t do it. I concluded that it was impossible to know what was entities and what was attributes. I now know this is because models are always created for a purpose. If you aren’t yet aware of that purpose, you can’t design for it. My suspicion was confirmed in a talk at Wolverhampton University by Michael ‘JSD’ Jackson. The revelation seemed a big shock to the large team from the Inland Revenue. I guess they had made unconscious assumptions about likely processes.

Relations don’t understand time

(They would probably say the same about me.) A transaction acting across multiple tables is assumed to be instantaneous. This worried me. A complex calculation requiring reads could not be guaranteed to be consistent unless all accessed tables are locked against writes, throughout the transaction. Jackson also confirmed that the Relational Model has no concept of time. A dirty fix is data warehousing which achieves consistency without locking by the trade-off of guaranteeing the data is old.

The Object Model doesn’t generalise

I’d stopped developing software by the time I heard about the Object Oriented Programming paradigm. I could see a lot of sense in OOP for simulating real-world objects. Software could be designed to be more modular when the data structures representing the state of a real-world object and the code which handled state-change were kept in a black box with a sign on that said “Beware of the leopard”. I couldn’t grasp how people filled the space between the objects with imaginary software objects that followed the same restrictions, or why they needed to.

A new wave of Functional Programming has introduced immutable data structures. I have recently learned through Clojure author Rich Hickey’s videos that reflecting state-change by mutating the value of variables is now a sin punishable by a career in Java programming. Functional Programmers have apparently always agreed with me that not all data structures belong in an object

There are others I’m still waiting for everyone to catch up on:

The Writable Web is a bad idea

The Web wasn’t designed for this isn’t very good at it. Throwing complexity bombs at an over-simplified model rarely helps.

Rich Hickey’s Datomic doesn’t appear to have fixed my entity:attribute issue

Maybe that one is impossible.

Advertisement

Change Time

After some time trying to think about almost nothing, the last 24 hours have been an alarm call. As others come out of hibernation too, they post interesting stuff and Radio 4 provoked me with a discussion on facts and truth. Now Marc Cooper is at it, with difficult  links about computation and I’m all on Edge https://www.edge.org/response-detail/26733
Before I read about “discrete tensor networks”, I need to write down my own ideas about time, so I will know in the future what I thought, before my mind was changed.

I am ill-equipped for this task, having only 1 term of university maths to my name so I intend to talk in vague, abstract terms that are hard to argue with.

Much of physics is very dependent on Time, like almost all of computer science and business management theory. You can’t have change without time, it seems. Einstein talked about space-time, mostly in the language of mathematics. I can just about order a beer in math(s) but I can’t hold a whole conversation. I know what the first 3 dimensions are: left-right, up-down and back-forward. My personal model of the 4th dimension is that same space in continuous state-change through time. There are a few things I’m not happy about:

• There is no evidence that time is either continuous or constant.
• We only have evidence of time being a one-way dimension.
• What the heck does ‘continous state-change’ mean? Is state a particle or a wave? Make your mind up, physics!
• There’s that troubling many-worlds interpretation of the universal ‘WAVE’function (which I don’t understand either) which says that everything that might have happened did, in other universes. I don’t like this. Yes, that’s my entire justification – I don’t like the conclusion of a thought process I don’t even understand. It doesn’t feel right.

I’ve been learning about the functional programming language Clojure which does not ‘mutate (change) state’. It doesn’t have ‘variables’ like the more common imperative languages such as FORTRAN, BASIC, C, Java or Python. In Clojure, data flows through functions and is transformed from one form to another on the way. It is basically magic. In a pure functional program, no state is changed. State-change is called a “side-effect”. Sadly, side-effects are required to make a program do anything useful in the real world. Arguably, the purest magic is encapsulated in the world of mathematics and the physical world is a messy place that breaks things.

Clojure models time. It does not model the real world by replacing the current value in a variable and throwing the old value away but by chaining a new value onto the end of a list of all previous values.

Now let us extend this idea ‘slightly’ in a small thought-experiment, to a 3-D network of every particle state in the universe.

Space-time now has 2 regions:

1. The past – all historic states of those particles as a theoretical chain of events
2. The future – all possible future states of the universe; effectively an infinity of all possible future universes that could exist, starting from now.

Which brings us to what I mean by ‘now’ – a moving wave at the interface between the past and the future, annihilating possible future universes. Time becomes a consequence of the computation of the next set of states and the reason for it being a one-way street becomes obvious: the universe burned its bridges. Unless the universe kept a list, or we do, the past has gone. Time doesn’t need to be constant in different parts of the universe, unless the universe state ticks are synchronous but it seems likely to be resistant to discontinuities in the moving surface. I imagine a fishing net, pulled by current events.

It’s just an idea. Maybe you can’t have Time without change.

[ Please tell me if this isn’t an original idea, as I’m not very well read.
I made it up myself but I’m probably not the first. ]

“Agile Project” is an Oxymoron

From http://en.wikipedia.org/wiki/Project:

“In contemporary business and science a project is defined as a collaborative enterprise, involving research or design, that is carefully planned to achieve a particular aim.

Projects can be further defined as temporary rather than permanent social systems or work systems that are constituted by teams within or across organizations to accomplish particular tasks under time constraints”

Projects are risky, slippery things, particularly when made out of software, which is notoriously difficult to get a grip on. Large companies take big risks with projects so have to protect themselves with governance systems and project portfolio management and project managers to take the blame. Project managers become highly skilled at batting blame away.

The effect of all this insurance is that projects become even more expensive. The smallest software project may have a start-up cost of £50K, used to build the corporate control structures necessary to get permission to begin. This makes it a very bad idea to run any small software projects, however good their apparent benefit:cost ratio. You want nice chunky projects to spread that £50K of overheads across.You’ll think of something.

Assume you didn’t – say you ran a project that only cost another £50K to complete. You would have spent £50K to insure yourself against £50K of risk, while exposing yourself to the new risk that your project might be rejected, which would have cost you £50K to do nothing. That would be CRAZY!

What is project management? You write a specification that precisely defines your aim. This is generally described as ‘the scope’. Then ALL YOU HAVE TO DO is achieve ‘the aim’ (which is hopefully in some way related to ‘the scope’ – no-one is quite sure) while balancing the famous ‘Time, Quality, Cost triangle’ on your head. It is unlikely that you will get any actual definition of the Quality that is required, so if the triangle falls off, that is clearly the side you should aim for it to land on. Luckily, quality is unlikely to be tested until the end of the project when either the time or the money will have already run out, so it will be too late to do anything about it and you win. We now refer to this method as ‘Traditional’, much like Morris dancing or cock-fighting.

What the cool-kids are doing these days, in their fancy Internet start-ups, is called ‘Agile’. They don’t have a whole load of cash to spend on insuring themselves against risk so they cunningly avoid it; or at least minimise it so that it doesn’t need to be managed. They don’t do a year of work to decide and plan what they are going to do over the next 5 years. They say “What do we need right now?” and “What can we achieve in a month?” and then they do it. They might make a big list of what they’d like and guess it could take 2 years with the resources they have but after a month they have some software that they can hopefully use. If they are happy with it then they decide what to do next month.

So there isn’t really anything you could call a “particular aim” or “a scope”, or indeed any long-term “time-constraint”. As long as each month of work promises to deliver value, you can carry on. Or you can stop because you can think of better things to do with your money now.

In total it is not, in fact, a project.

Clearly there is a down-side to all of this. There must be, mustn’t there? You can’t build a house or a bridge or a car without planning! But software isn’t a house, or a bridge or a car. Modern software is built out of objects, like Lego bricks in zero-gravity. You can change things later, at reasonable cost.