Change Time

After some time trying to think about almost nothing, the last 24 hours have been an alarm call. As others come out of hibernation too, they post interesting stuff and Radio 4 provoked me with a discussion on facts and truth. Now Marc Cooper is at it, with difficult  links about computation and I’m all on Edge https://www.edge.org/response-detail/26733
Before I read about “discrete tensor networks”, I need to write down my own ideas about time, so I will know in the future what I thought, before my mind was changed.

I am ill-equipped for this task, having only 1 term of university maths to my name so I intend to talk in vague, abstract terms that are hard to argue with.

Much of physics is very dependent on Time, like almost all of computer science and business management theory. You can’t have change without time, it seems. Einstein talked about space-time, mostly in the language of mathematics. I can just about order a beer in math(s) but I can’t hold a whole conversation. I know what the first 3 dimensions are: left-right, up-down and back-forward. My personal model of the 4th dimension is that same space in continuous state-change through time. There are a few things I’m not happy about:

• There is no evidence that time is either continuous or constant.
• We only have evidence of time being a one-way dimension.
• What the heck does ‘continous state-change’ mean? Is state a particle or a wave? Make your mind up, physics!
• There’s that troubling many-worlds interpretation of the universal ‘WAVE’function (which I don’t understand either) which says that everything that might have happened did, in other universes. I don’t like this. Yes, that’s my entire justification – I don’t like the conclusion of a thought process I don’t even understand. It doesn’t feel right.

I’ve been learning about the functional programming language Clojure which does not ‘mutate (change) state’. It doesn’t have ‘variables’ like the more common imperative languages such as FORTRAN, BASIC, C, Java or Python. In Clojure, data flows through functions and is transformed from one form to another on the way. It is basically magic. In a pure functional program, no state is changed. State-change is called a “side-effect”. Sadly, side-effects are required to make a program do anything useful in the real world. Arguably, the purest magic is encapsulated in the world of mathematics and the physical world is a messy place that breaks things.

Clojure models time. It does not model the real world by replacing the current value in a variable and throwing the old value away but by chaining a new value onto the end of a list of all previous values.

Now let us extend this idea ‘slightly’ in a small thought-experiment, to a 3-D network of every particle state in the universe.

Space-time now has 2 regions:

1. The past – all historic states of those particles as a theoretical chain of events
2. The future – all possible future states of the universe; effectively an infinity of all possible future universes that could exist, starting from now.

Which brings us to what I mean by ‘now’ – a moving wave at the interface between the past and the future, annihilating possible future universes. Time becomes a consequence of the computation of the next set of states and the reason for it being a one-way street becomes obvious: the universe burned its bridges. Unless the universe kept a list, or we do, the past has gone. Time doesn’t need to be constant in different parts of the universe, unless the universe state ticks are synchronous but it seems likely to be resistant to discontinuities in the moving surface. I imagine a fishing net, pulled by current events.

It’s just an idea. Maybe you can’t have Time without change.

[ Please tell me if this isn’t an original idea, as I’m not very well read.
I made it up myself but I’m probably not the first. ]

Advertisement

To Be Derived

I thought I’d said something about this but I can’t find it, so sorry if you’ve heard this before:

Professor Brian Cox turned to Dr. Hannah Fry and said, “You have 30 seconds to define ‘calculus'”.
She replied, “It’s the study of change.”

What we call “The IT Industry” deals almost exclusively with change. As a business analyst, I’ve worked in business process re-engineering which is the process of changing the processes which cause those changes. I’ve worked in agile software development teams that try to change software while the requirements for the target they’re travelling towards are changing, so the changes have to change.

I’ve met calculus in maths and physics but never in computer science. Why on earth aren’t computer scientists up to their necks in calculus?

Concerns

I learned to write ‘computer code’ in the era of Structured Programming. In the last few months I have come to question how much science I was exposed to in my Computer Science education. I was taught facts and current best practice but that wasn’t enough.

I’d stopped writing code professionally by the time Software Engineering became trendy, so I skipped relational databases, object orientation and coding for the web before I decided to reconnect with software development via the Business Analysis, UML modelling and Scrum Agile, Product Owner route. I THINK I know what objects are now.

When I decided to do some coding again, I at first decided to learn Python but quickly jumped ship to Clojure. I’m finding the functional model new and exciting but also unfamiliar and strange. I’ve made a huge leap into the dark, from a direction that that the text books I’m reading weren’t expecting. This post represents me taking a breath of air.

I thought I had my head around ‘separation of concerns’ into code modules, in a world made of objects. An object is a model of a real-world entity in a software simulation of reality. It represents the state of an object’s data and via calls to its methods, implements message passing between objects. What functional programming texts have shown me is that OO also invented objects that had no equivalent in the real world. In the functional world, concerns are implemented in stateless functions and state is represented by the flow of change over data structures, outside the functions.

What I haven’t yet worked out is what the “logically discrete functions, interacting through well-defined interfaces” of ‘Top-Down Design’ and ‘Step-wise refinement’ were supposed to represent. Anything we liked, I suspect, because no-one else knew either. I feel now that I was equipped with excellent knowledge of woodworking tools and the idea of furniture without being shown any woodworking joints. At least I recognised at the time that I was clueless and stopped. Many didn’t. OK, I think I’m ready to carry on.

Why I want Clojure

In the final year of my degree, I had to complete a major piece of work; a kind of dissertation equivalent for the illiteratti of Computer Science. I chose to implement a neural network [ this originally said “semantic network” but that was Freudian slippage. ] My project supervisor, a PhD researcher, suggested that I use Lisp with a library for fuzzy logic.

We didn’t have The Interwebs in those days, so I went to a book shop to order a Lisp book, well, THE only Lisp book, which I had to order from the US:

(LISP 1.5 PRIMER (BY (CLARK WEISSMANN)))

Do you see what he did there, apart from the obvious shouting? All computers shouted, in those days.

It mostly went wrong from there on. My mate Mark wrote a comparison between Algol 68 which we’d been taught and BCPL, the forerunner of C; another block-structured imperative language but typeless. I had chosen to do something difficult in a language I knew nothing about. I tend to take the path less travelled, but Mark’s path led to a doctorate. “That’ll learn him”, as Badger would say.

I learned that it is easy to be taught a computer language by someone who knows how (if you think in the right way) and it’s quite easy to move to a slightly different language. I’ve done it several times, but Lisp has a completely different computing paradigm and the book didn’t arrive in time. I was beaten by Lisp once but I didn’t feel it was a fair fight. I arrived back at the Clojure shore of Lisp recently, determined on wreaking revenge. This time it was going to be different!

I started work seriously last week but had a few bad days. I still wasn’t “getting it”, but this weekend, I found myself a new sensei, Rich Hickey. He seems to understand my need for a conceptual framework to hang up the weapons I’m being handed. I may be able to function soon. I now know what “programming to abstractions” means. It doesn’t involve my head and a solid, vertical surface.

UPDATE: In Weismann’s introduction, he writes: “In learning a symbolic programming language such as LISP, however, we cannot call on our experience, because the formal skills for representing and manipulating symbolic data processing is not part of this experience. Thus, we have the added task of learning a basic set of formal skills for representing and manipulating symbolic data before we can study the syntax of the LISP 1.5 programing language.”
I guess that hasn’t got any easier with Clojure

Applying the Science Process to Process Change (see Recursion)

Now that we’ve established that very few people who work in “IT” have anything to do with technology, except as a tool, and that “computer science” isn’t about computers, we’re back at the original reason I began to write a book. I was once an ‘Information Systems Engineer’ and wasn’t very sure what any of those words meant, particularly “information”.

Software development teams often see their role as solving their customer’s problems. Software package providers say they are “solution providers”. What does the ‘unspeakable profession’ actually do? We got some clues from Hal Abelson in the video I linked to in my last post, that new areas of intellectual endeavour often confuse the ‘essence’ of their subject with their tools; so do we really engineer software?

Hal said that writing software is the process (or function) of formalising our intuitions about process (function.) Our software is a speculative formal abstraction of our intuitive understanding of a process we may not entirely understand. No wonder software projects so often fail. Like the rest of science, software is built on ideas that haven’t been proved wrong yet.

Software developers are presented with, or attempt to discover, experts’ (declarative) knowledge of the business process in the ‘domain’ we are about to change. This is ‘the abstract requirements’. Some of this may have to be implied from imperative knowledge embedded in existing software. It may be presented as imperative solutions. It may be incomplete.

We then follow our own process (function) for: ‘the way we do software’, in order to design a new process, some of which is also likely to be have to be embedded in software. Applying functions to change functions? That’s what functional programming does, isn’t it?

I believe that the ‘stuff’ of ‘computer science’, is state-change of systems of process and data. Who remembers ‘Data Processing’? Functional programming points out that the processes themselves are data and dynamic state-change is unpredictable and therefore dangerous.

Engineering would apply ‘project thinking’ to this unstable, poorly defined change. I may once have tried to be an information systems engineer but I saw that it didn’t work and took a 20 year break from software development.

Agile frameworks recognise such changes as risky experiments and carefully apply the scientific method, incrementally with feedback loops, to check assumptions.

We may need to return another time to see what functional programming has to say about 2 projects making concurrent changes from the same initial state. Until then, good luck with those.

Like magic, the laws of physics do not apply to software

For a few months I’ve been arguing that Agile product/service development only works for classes of systems that are not subject to gravity, like software and business processes. After attending a talk on the functional language Clojure by Paul Williams of @birminghamfp at Agile Staffordshire, I discovered this MITOpenCourseWare 6-001 introductory video.

http://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-001-structure-and-interpretation-of-computer-programs-spring-2005/video-lectures/1a-overview-and-introduction-to-lisp/

I particularly recommend about the first 10 minutes. After that you may accidentally learn Lisp and I’ll deny all responsibility for subsequent brain damage if that happens. Hal Abelson argues that ‘computer science’ is not bound by the rules of physics. Like me, he sees the design of business processes and procedures being within its boundaries. Business procedures are just an attempt to implement processes on people rather than in software, sacrificing reliability for flexibility.

I agree that ‘computer science’ isn’t about computers but I think Lean & Agile methods re-take computation for science, rather than magic or engineering. I’m sticking with ‘computational science’ for now.

Many software people are frustrated by their managers’ attempts at business change. Why do organisations allow general managers with no training in the re-factorisation of complex processes to run business change programmes when they wouldn’t trust them to run a small software project? It’s the same thing, but much harder.

Computational Science + Informatics = Software Development

1970s At university, I studied Physics and:

• Computer Science

but it had very little to do with computers. It was far more about becoming a

• Computer Programmer

1980s When I started work I heard about the increasing formalisation of the software development process and I wanted to be a

• Software Engineer

1990s I’d moved into server management by the time I qualified as a

• Chartered Information Systems Engineer

which fit in with my thinking that information is the important resource in any organisation but took me no closer to knowing how to make great software efficiently. I became disillusioned by watching people trying to apply the rigorous methods of hard engineering to the uncertainties and unknown complexities of software. I became interested in prototyping, incremental delivery and Agile.

2000s I was increasingly drawn to the idea that constructing software is a design discipline and that a software developer needed to be a

• Software Craftsman

2010s After a few years working with an agile software team, I decided I wanted to try writing software again myself. A few weeks ago, an online MIT course on Python programming introduced me to an idea that I felt very comfortable with: a good developer is a

• Computational Scientist (I’d settle for ‘Computing Scientist’)

and I’d add the option

• Informatician, which overlaps with what librarians do now

Agile development processes are adaptations of the scientific method, to research what customers want and work out how to give it to them in the way that meets their opinion of best value, and I don’t see why software developers can’t be their own customer. Every great new software component starts off as someone’s experiment.

You kids had one job: to make everything amazing

In between me starting to learn to ‘computer science’ and stopping writing compiled code in about 1985/6, the world of information systems moved from punched-cards and teletypes to on-line editing and batch-processing to full screen editing then windows and code management and source-code level debugging. I moved from mainframes that made you choose between upper-case letters or maths, to super-minis and graphical workstations. I didn’t move to PCs because they were so obviously THE WRONG WAY to go. This took less than 10 years.

Shortly after I stopped coding, I became aware that the portability of C and Unix wasn’t just snake-oil, and of the availability of relational databases, network filing systems, the object model and parallel computing: Hoare’s Communicating Sequential Processes.

Later, lost in server-land I became a ‘user’ of hypertext and browsers for documentation, a little later of the Internet. I use ‘social’ tools and believe that they are an enabling tool for a new society based on networks rather than hierarchy. What a pity they’re so clunky.

Inspired to make things better, I decided to learn to code again. It must all be so great by now!
What do you mean, “Do I want to be a front-end or a back-end developer?”
Well, since you ask, I want to be both and sideways and up and down, virtually travelling freely through multi-dimensional networks, working with 3D graphical representations of algorithms and business objects moving in object pipelines, using standardised Free tools in the Cloud, with the display and persistence mechanisms abstracted away, out of sight as implementation details to worry about later, but with automated parallelism and fault tolerance.

What? What’s THIS? Why am I expected to work with a tagging language designed for sequential documents? Seriously, you want me to specify a font? I have to write different code for web browsers or phones?! And where is my hover-board? What the hell have you people been doing?
Did Microsoft lead you into the woods to see their unicorn foals? Were they goats in a dunce’s cap?

Are we all back now? Hold someone’s hand. We can get through this if we all stay together and look out for the traps. Don’t get too close to the googles. They seem friendly but they’ll pick your packets.

Software Life-cycle. Part 2 – From Craftsmanship to Computational Science

I decided to learn the programming language Python. I was steered towards the MIT OpenCourseware ‘Introduction to Computer Science and Programming’ 6.00 course, taught by Prof. Eric Grimson and Prof. John Guttag (they say it is a course about computational thinking.)

http://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-00-introduction-to-computer-science-and-programming-fall-2008/video-lectures/

As the first lecture felt a bit basic, at great personal risk of uncovering ‘a spoiler’, I skipped on to the course summary in the last lecture to see if it was worth sticking around. I found it inspirational. Prof. John Guttag explains computation in the context of ‘The Scientific Method’. I’ve since realised that his explanation maps with great accurately onto Agile iterative methods. Agilists aren’t engineers, we’re scientists again. Engineering Project Management uses experience of similar previous projects. Why would you ever write similar software twice? Most of the work is already done. Every change to a computational system should be R&D.

Every Scrum Sprint is a suite of computational experiments. The Product Owner is our test subject. This feels right. I never felt like a computer scientist. In James Gleich’s ‘The Information’, he explains that Alan Turing introduced Babbage’s mechanical Difference Engine when talking to non-specialist, to emphasise that computing is an abstract concept, independent from computers and electricity. I’ve always had a computational scientist trying to get out <Woo asplodes>.

Software Life-cycle. Part 1 – From Engineering to Craftsmanship

I graduated just after the Structured Programming War was won. I was probably the first generation to be taught to program by someone who actually knew how; to be warned of the real and present danger of the GOTO statement and to be exposed first to a language that didn’t need it. I didn’t need to fall back to assembler when the going got tough or to be able to read hex dumps or deal with physical memory constraints. I entered the computing profession just as people were starting to re-brand their programmers as ‘software engineers’ and academics were talking of ‘formal methods’ then ‘iterative development’ and ‘prototyping’ as we lost faith and retreated, as the techniques borrowed from other engineering disciplines continued to disappoint when applied to software.

After some years away from software development, I returned to find ‘Agile’, ‘Lean’ and ‘Software Craftsmanship’. We’d surrendered to the chaos, accepted that we weren’t designers of great engineering works but software whittlers. I was pleased that we’d dropped the pretence that we knew what we were doing but disappointed that we’d been reduced to hand-weaving our systems like hipsters.

There had been another good change too: The Object Model. The thrust of software engineering had often been decomposition but our model had been the parts breakdown structure, the skeletal parts of a dead system. Objects allowed us to model running systems, the process network at the heart of computation. I’ve recently seen a claim that the Unix command line interface with its pipes and redirection was the first object system. Unix begat GNU and Free software and Linux and close to zero costs for the ‘means of production’ of software. I don’t believe that Agile or Lean start-ups could have happened in a world without objects, the Internet or Free software. Do you know how much work it takes to order software on a tape from the US by post? I do.

So here we are, in our loft rooms, on a hand crafted loom made of driftwood and sweat, iterating towards a vague idea emerging out of someone’s misty musings and watching our salary eroded towards the cost of production. Is this why I studied Computer Science for 3 years? Who turned my profession into a hobby activity?