Things I used to be Wrong about – Part 1

I get very annoyed about politicians being held to account for admitting they were wrong, rather than forcefully challenged when they were wrong in the first place. Unless they lied, if someone was wrong and admits it, they should be congratulated. They have grown as a human being.

I am about to do something very similar. I’m going to start confessing some wrong things I used to think, that the world has come to agree with me about. I feel I should congratulate you all.

You can’t design a Database without knowing how it will be used

I was taught at university that you could create a single abstract data model of an organisation’s data. “The word database has no plural”, I was told. I tried to create a model of all street furniture (signs and lighting) in Staffordshire, in my second job. I couldn’t do it. I concluded that it was impossible to know what was entities and what was attributes. I now know this is because models are always created for a purpose. If you aren’t yet aware of that purpose, you can’t design for it. My suspicion was confirmed in a talk at Wolverhampton University by Michael ‘JSD’ Jackson. The revelation seemed a big shock to the large team from the Inland Revenue. I guess they had made unconscious assumptions about likely processes.

Relations don’t understand time

(They would probably say the same about me.) A transaction acting across multiple tables is assumed to be instantaneous. This worried me. A complex calculation requiring reads could not be guaranteed to be consistent unless all accessed tables are locked against writes, throughout the transaction. Jackson also confirmed that the Relational Model has no concept of time. A dirty fix is data warehousing which achieves consistency without locking by the trade-off of guaranteeing the data is old.

The Object Model doesn’t generalise

I’d stopped developing software by the time I heard about the Object Oriented Programming paradigm. I could see a lot of sense in OOP for simulating real-world objects. Software could be designed to be more modular when the data structures representing the state of a real-world object and the code which handled state-change were kept in a black box with a sign on that said “Beware of the leopard”. I couldn’t grasp how people filled the space between the objects with imaginary software objects that followed the same restrictions, or why they needed to.

A new wave of Functional Programming has introduced immutable data structures. I have recently learned through Clojure author Rich Hickey’s videos that reflecting state-change by mutating the value of variables is now a sin punishable by a career in Java programming. Functional Programmers have apparently always agreed with me that not all data structures belong in an object

There are others I’m still waiting for everyone to catch up on:

The Writable Web is a bad idea

The Web wasn’t designed for this isn’t very good at it. Throwing complexity bombs at an over-simplified model rarely helps.

Rich Hickey’s Datomic doesn’t appear to have fixed my entity:attribute issue

Maybe that one is impossible.

Advertisement

Objects vs Functions

I learned to ‘programme computers’ long ago, almost before there was no “me” in ‘program’ and certainly before I knew how to ‘team’. I had a very brief and unsuccessful exposure to functional programming in LISP (not Lisp) then stopped. I did other ‘Data Processing’ things.

In recent years I’ve been working as an analyst, alongside people who write code according to the object model. I think I have a feel for objects but never having written code in an object-oriented language, I can’t be sure. I decided to try, in the Python language, then got distracted by the shiny Clojure language which is functional. I feel that right now I’m approximately equally confused by objects and functions, so I thought I’d write this quickly before I know what I’m talking about. I can come back later to laugh at my naivety, along with the rest of you.

Like the person who wrote this https://news.ycombinator.com/item?id=4246018 ,
“I’ve been watching some talks online recently by Rich Hickey of Clojure fame”
The post asks “So if I follow Hickey’s advice, how am I supposed to represent a book? As a vector of vectors of vectors of vectors of strings? If so, then how do I prevent a change in the representation of the Book from breaking client code?”

I found the question very interesting because representing ‘books’ in a functional language is exactly what I want to do. I think differently to the author because  I’m not yet trapped inside the object paradigm. I can see that ‘book’ is a real-world class of objects, a very specific and limited implementation of the representation of a small subset of all the information in the world. That’s what my ‘book’ was going to be about and why I’m now playing with functions instead of writing it.

Objects are good at simulation of real life systems. They encapsulate small sub-systems of a process and it’s local data into an object. What I always struggled to understand was what you did with the data that didn’t want to be enclosed – “information wants to be free”. People seem to cope by inventing objects that don’t really exist: to be data shepherds.

Functions are good at abstraction. A book is a single output format from something much richer. That’s what I want to write. Data and processes are complex. Objects and functions are simplifying models; there may be others.

p.s. (not Lisp) Get it?

Why I want Clojure

In the final year of my degree, I had to complete a major piece of work; a kind of dissertation equivalent for the illiteratti of Computer Science. I chose to implement a neural network [ this originally said “semantic network” but that was Freudian slippage. ] My project supervisor, a PhD researcher, suggested that I use Lisp with a library for fuzzy logic.

We didn’t have The Interwebs in those days, so I went to a book shop to order a Lisp book, well, THE only Lisp book, which I had to order from the US:

(LISP 1.5 PRIMER (BY (CLARK WEISSMANN)))

Do you see what he did there, apart from the obvious shouting? All computers shouted, in those days.

It mostly went wrong from there on. My mate Mark wrote a comparison between Algol 68 which we’d been taught and BCPL, the forerunner of C; another block-structured imperative language but typeless. I had chosen to do something difficult in a language I knew nothing about. I tend to take the path less travelled, but Mark’s path led to a doctorate. “That’ll learn him”, as Badger would say.

I learned that it is easy to be taught a computer language by someone who knows how (if you think in the right way) and it’s quite easy to move to a slightly different language. I’ve done it several times, but Lisp has a completely different computing paradigm and the book didn’t arrive in time. I was beaten by Lisp once but I didn’t feel it was a fair fight. I arrived back at the Clojure shore of Lisp recently, determined on wreaking revenge. This time it was going to be different!

I started work seriously last week but had a few bad days. I still wasn’t “getting it”, but this weekend, I found myself a new sensei, Rich Hickey. He seems to understand my need for a conceptual framework to hang up the weapons I’m being handed. I may be able to function soon. I now know what “programming to abstractions” means. It doesn’t involve my head and a solid, vertical surface.

UPDATE: In Weismann’s introduction, he writes: “In learning a symbolic programming language such as LISP, however, we cannot call on our experience, because the formal skills for representing and manipulating symbolic data processing is not part of this experience. Thus, we have the added task of learning a basic set of formal skills for representing and manipulating symbolic data before we can study the syntax of the LISP 1.5 programing language.”
I guess that hasn’t got any easier with Clojure