Tag Archives: Clojure

Tooling for Clojure and ClojureScript

[:ChangeLog
(:v0.2
“recognise that ‘resolving dependencies’ and ‘build’ are different operations.”
“Add conversion tools between project.clj & deps.edn”)
(:v0.3
“Remove CLI tools described as a build tool comment.”)]

WARNING
I still have much to learn about ClojureScript tooling but I thought I’d share what (I think) I’ve learned, as I have found it difficult to locate advice for beginners that is still current. This is very incomplete. It may stay that way or I may update it into a living document. I don’t actually have much advice to give and it’s only about the paths that have interested me.

Clojure development requires:

  • a text editor,

and optionally,

  • a REPL, for a dynamic coding environment
  • dependency and build tool(s).

The absolute minimum Clojure environment is a Java .jar file, containing the clojure.main/main entry point, which can be called with the name of your file.clj as a parameter, to read and run your code. I don’t think anyone does that, after they’ve run it once to check it flies.

Based on 2 books, ‘Clojure for the Brave & True’ and ‘Living Clojure’, my chosen tools are emacs for editing, with CIDER connecting a REPL, and Leiningen as dependency & build tool. ‘lein repl’ can also start a REPL.
Boot is available as an alternative to Leiningen but I got the impression it might be a bit too ‘exciting’ for a Clojure noob like me, so I haven’t used it yet.
CIDER provides a client-server link between an editor (I’m learning emacs) and a REPL.

If you use Leiningen, it comes with a free set of assumptions about development directory structure and the expectation that you will create a file, project.cli in the root directory of each ‘project’, containing a :dependencies vector. Then magic happens. If your change the dependencies of your project, the config fairies work out everything else that needs changing.

Next, I wanted to start using ClojureScript (CLJS.) I assumed that the same set of tools would extend. I was wrong to assume.
Unfortunately, CLJS tooling is less standardised and doesn’t seem to have reached such a stable state.

In ‘Living Clojure’, Carin Meier suggests using cljsbuild. It uses the lein-cljsbuild plugin and the command:

lein cljsbuild auto

to start a process which automatically re-compiles whenever a change is saved to the cljs source file. If the generated JavaScript is open in a browser, then the change will be shown in the browser window. This is enough to get you going. It is my current state.

I’ve read that there are other tools such as Figwheel, now transitioning to ‘Figwheel Main’ which hot-load the transcribed code into the browser as you change it.
There is a lein-figwheel as well as a lein-cljsbuild, which at least sounds like a drop-in replacement. I suspect it isn’t that simple.

There are several REPLs, though there seems to be some standardisation around nrepl.
It was part of the Clojure project but now has its own nrepl/nrepl repository on Github. It is used by Clojure, ‘lein repl’ and by CIDER.

There is something called Piggieback which adds CLJS support to NREPL. There is a CIDER Piggieback and an NREPL Piggieback. I have NO IDEA! (yet.)
shadow-cljs exists. Sorry, that’s all I have there too.

At this point in my confusion, a dependency issue killed my tool-chain.
I think one of the config fairies was off sick that day. The fix was a re-install of an emacs module. This forced me to explore possible reasons. I discovered the Clojure ‘Getting Started’ page had changed (or I’d never read it.)
https://clojure.org/guides/getting_started

There are also (now?) ‘Deps and the CLI Tools’ https://clojure.org/guides/deps_and_cli and https://clojure.org/reference/deps_and_cli

I think these are new and I believe they are intended to be the beginners’ entry point into Clojure development, before you dive into the more complex tools. There are CLI commands: ‘clojure’ and a wrapper that provides line-editing, ‘clj’
and a file called ‘deps.edn’ which specifies the dependencies, much as ‘projects.clj’ :depencies vector does for Leiningen but with a different syntax.

I’m less clear if these are also intended as new tools for experts, to be used by ‘higher order’ tools like Leiningen and Figwheel, or whether they will be adopted by those tools.

[ On the day I wrote this, I had a tip from didibus on clojureverse.org that there are plugins for Leiningen and Boot to use an existing deps.edn,

so perhaps this is the coming future standard for specifying & resolving dependencies, while lein and boot continue to provide additional build capabilities. deps.edn refers to Maven. I discovered elsewhere that Maven references existed but were hidden away inside Leiningen. It looks like I need to learn a little about Apache Maven. I didn’t come to Closure via Java but I can see the advantages to Java practitioners of using their standard build tool. I may need to drop down into Java one day, so I guess I may as well learn about Java-land now.

Also via: https://clojureverse.org/t/has-anyone-written-a-tool-to-spit-out-a-deps-edn-from-a-project-clj/2086, there is a https://github.com/hagmonk/depify, which ‘goes the other way’, trying it’s best to convert a project.clj to a deps.edn. Hopefully that would be a ‘one-off’? ]

I chose the Clojure language for its simplicity. The tooling journey has been longer than I expected, so I hope this information cuts some corners for you.

[ Please let me know if I’m wrong about any of this or if there are better, current documents that I should read. ]

Reality has Levels

It’s been a while since I blogged. I’ve been busy.

A major theme emerging from ‘writing my book’ is that we humans are very bad at confusing our models of reality with the reality we are modelling.

I started planning with the ‘Freemind mind-mapping tool for hierarchical brains’ before finding my own creative process had a network architecture and discovering ‘concept mapping’ which uses graphs to represent concepts and propositions. I saw that graphs were what I needed and decided to experiment with building my own software tools from bits I had lying around.

I didn’t have a current programming language, so I set out to learn Clojure. Being a Lisp, Clojure uses tree-structures internally to represent lists and extends the idea to abstractions such as collections but the only native data structures available to me appeared to be 1-dimensional.  I confidently expected to be able to find ways to extend this to 3 or more dimensional graphs but despite much reading and learning lots of other things, I’d failed to find what I was looking for. I had in mind the kind of structures you can build with pointers, in languages like ‘C’. There are graph libraries but I was too new to Clojure to believe my first serious program needed to be dependent on language extensions, when I haven’t securely grasped the basics.

This morning, I think I ‘got it’. I am trying to build a computational model of my graphical view of a mathematical idea which models a cognitive model of reality. There was always scope for confusion. Graphs aren’t really a picture, they are a set of 1-dimensional connections and potentially another set of potential views of those connections, constructed for a particular purpose. I was trying to build a data structure of a view of a graph, not the graph itself and that was a really bad idea. At least I didn’t get software confused with ‘actual’ reality, so there’s still hope for me.

Yesterday, I used Clojure/Leiningen’s in-built Test-Driven Development tool for the first time. It looks great. The functional model makes TDD simple.

Change Time

After some time trying to think about almost nothing, the last 24 hours have been an alarm call. As others come out of hibernation too, they post interesting stuff and Radio 4 provoked me with a discussion on facts and truth. Now Marc Cooper is at it, with difficult  links about computation and I’m all on Edge https://www.edge.org/response-detail/26733
Before I read about “discrete tensor networks”, I need to write down my own ideas about time, so I will know in the future what I thought, before my mind was changed.

I am ill-equipped for this task, having only 1 term of university maths to my name so I intend to talk in vague, abstract terms that are hard to argue with.

Much of physics is very dependent on Time, like almost all of computer science and business management theory. You can’t have change without time, it seems. Einstein talked about space-time, mostly in the language of mathematics. I can just about order a beer in math(s) but I can’t hold a whole conversation. I know what the first 3 dimensions are: left-right, up-down and back-forward. My personal model of the 4th dimension is that same space in continuous state-change through time. There are a few things I’m not happy about:

  • There is no evidence that time is either continuous or constant.
  • We only have evidence of time being a one-way dimension.
  • What the heck does ‘continous state-change’ mean? Is state a particle or a wave? Make your mind up, physics!
  • There’s that troubling many-worlds interpretation of the universal ‘WAVE’function (which I don’t understand either) which says that everything that might have happened did, in other universes. I don’t like this. Yes, that’s my entire justification – I don’t like the conclusion of a thought process I don’t even understand. It doesn’t feel right.

I’ve been learning about the functional programming language Clojure which does not ‘mutate (change) state’. It doesn’t have ‘variables’ like the more common imperative languages such as FORTRAN, BASIC, C, Java or Python. In Clojure, data flows through functions and is transformed from one form to another on the way. It is basically magic. In a pure functional program, no state is changed. State-change is called a “side-effect”. Sadly, side-effects are required to make a program do anything useful in the real world. Arguably, the purest magic is encapsulated in the world of mathematics and the physical world is a messy place that breaks things.

Clojure models time. It does not model the real world by replacing the current value in a variable and throwing the old value away but by chaining a new value onto the end of a list of all previous values.

Now let us extend this idea ‘slightly’ in a small thought-experiment, to a 3-D network of every particle state in the universe.

Space-time now has 2 regions:

  1. The past – all historic states of those particles as a theoretical chain of events
  2. The future – all possible future states of the universe; effectively an infinity of all possible future universes that could exist, starting from now.

Which brings us to what I mean by ‘now’ – a moving wave at the interface between the past and the future, annihilating possible future universes. Time becomes a consequence of the computation of the next set of states and the reason for it being a one-way street becomes obvious: the universe burned its bridges. Unless the universe kept a list, or we do, the past has gone. Time doesn’t need to be constant in different parts of the universe, unless the universe state ticks are synchronous but it seems likely to be resistant to discontinuities in the moving surface. I imagine a fishing net, pulled by current events.

It’s just an idea. Maybe you can’t have Time without change.

[ Please tell me if this isn’t an original idea, as I’m not very well read.
I made it up myself but I’m probably not the first. ]

A Functional Mindset

When I started learning Clojure, I thought I knew what functional programming was but I’ve learned that the functional paradigm is now more than I expected.

Everyone agrees that it’s a computational model based on evaluation of mathematical functions, which return values. This is generally contrasted with imperative programming languages such as FORTRAN, C, JavaScript or Python, which are also procedural and some of which are object-oriented but may make functional coding possible, in a hybrid style. I wouldn’t recommend learning functional concepts in a language that gives you short-cuts to stray back  to more familiar territory.

Clojure is a member of the Lisp family, first specified in 1958. The unusual feature of Lisps is their homoiconicity – code and data are the same thing. Learning Clojure has informed my thinking about business process change.

Some modern, functional languages such as Clojure use immutable data whenever possible, to eliminate side-effects. This allows better use of multi-core processors but requires a complete change in thinking, as well as programming style. ‘Variables’ are replaced by fixed ‘values’, so loops have to be replaced by recursive functions. New data can be created but it doesn’t replace old data. Yesterday’s “today’s date” isn’t automatically wiped when we decide today has happened.

Objects with their methods and local data were designed for simulating the current state of real-world objects by changing (mutating) object data state. The object model, like relational databases, has no inbuilt representation of time. Functional programming splits these objects back into separate functions and data structures and because values can’t change, they may be transformed by flowing through networks of functions, some recursive, to keep doing something until a condition is satisfied. Eventually, code must have a side effect, to tell us the answer.

Rather than computation being a conditional to-do list with data being moved between boxes, it becomes a flow of data through a network of ‘computing machines’; and the data and machines can be transformed into each other.

I hear that map, reduce & filter data transformation functions will change my world again.

Becoming Functional

I’ve been playing with the idea of doing some functional programming for a while now. I’ve been trying to learn and paddling around in the shallows but this week I dived right in the emacs/CIDER pool. I was aware of some dangers lurking beneath the surface: recursion, immutable data structures and the functional holy trinity of map, reduce & filter, so I came up with some ideas to face my fears. I’ve also realised my maths has got rusty so: Some of That Too.

  1. I’ve ‘done recursion’ before but I thought I’d read that my chosen weapon Clojure didn’t do tail-end recursion. This isn’t true. What it can’t do is automatic optimisation of tail-end  recursion, to stop it blowing the stack after a few thousand iterations but Clojure has a ‘recur’ expression to manually signal tail recursion and fix that. I knocked off the programme in a couple of hours and went to bed happy. My code was happily printing the first n numbers of the Fibonacci sequence but a day later I still couldn’t get it the return the numbers as a sequence.
  2. I was finding out about immutable data the hard way. You can’t build up an immutable vector, 1 element at a time. You get to keep the empty vector you created first. It’s a big mind-set change to not have variables that can vary. In my next post, I’ll try to say what I’ve learned. On this occasion it was lazy sequences.
  3. I mentioned the Algorave in my last post. I only found out about that because of an idea I had for improving my theoretical understanding of music. I realised that I could write, for example, a function that would return the 1st, 3rd and 5th notes in a major scale, using a map function.While working the theory out, I found out that Lisps are already popular in the live-coding world.
  4. At Algorave, I was inspired by the live-coded graphics to try automatically generating some graphics too, to work out the maths of mapping triangular grids onto Cartesian co-ordinates. I need that for another idea.

Three basic working programmes in about a week. They aren’t ‘finished’ but is software ever? They have delivered value via increased Clue.

My First Algorave

@algobbz

On Saturday night I went to ‘Algorave Birmingham’, curated  by Antonio Roberts at Vivid Projects. I said I might write ‘a review’ but I’m not going to, because I wouldn’t know how. This is ‘a reaction’ – a digital feedback loop, an emission from the event horizon (should have worn my ‘Big Bang’ T-Shirt – the noughties Brum band, not the nerd show.)

My background is information technology. My current work is writing. I use the word ‘work’ in the artistic sense: something I spend my time on but may never get paid for. Themes recur. Are science and art actually different things? Is maths real or a model? Is software any different to magic, existing only outside the physical realm and communicating via intermediary objects?

Q: How much can you strip away from music and it still exist as an idea: melody, scales, pitch?

I came to Algorave via my functional programming experiments. I’m trying to learn Clojure, a member of the Lisp family of languages but with added time-travel. It messes with whether time is a wave or a set of discrete steps that can be retraced. Not real time, obviously but the model of time our software deals with. Time travel outside of the magical realm would be crazy-talk.

Dance music is often first. Drum machines. I got really frustrated the first time I saw how hard it was to programme beats. Where was the programmatic interface? Sampling, pitch-shifting, the ‘sound’ being manipulated by code. Digits being manipulated by digits, like the higher order functions of functional programming. I wondered a few weeks ago if processors had got fast enough to generate live noises. They have. A Raspberry Pi has http://sonic-pI noti.net/http://sonic-pi.net/. From there I discovered Clojure has, via ‘Overtone’ on ‘SuperCollider’ http://sam.aaron.name/, which resonates with my theory of a super-massive idea colider to mash-up memes.

Algorave Birmingham presented live coders generating sound and visuals. At times I felt that the graphics were pulsing to the beats but I don’t know if that really happened. I saw two pixelated women on the screen typing on ‘real’ laptops and a live drummer on digital drums. Virtuality virtuosos. I had a chat about how to make a hit record and forgot the name of the Kaiser Chiefs but remembered Black Wire who were the first band with a drum machine that I actually liked, because it didn’t sound mechanical, then The Kills who insisted everything was analogue, but now I’m looping.

A: I enjoyed the pulsing white noise. Software can do things that aren’t possible in Reality.

Lispbian Pi. A Lambda Delta.

I’m conflicted. Part of me says that ‘us old timers’ shouldn’t assume ‘the way things were when we were kids’ were better but we know the Raspberry Pi was an attempt to recapture the spirit of the BBC Micro Model B and that seems to have gone quite well. I got a Pi 2 and I’ve worked out that it is more powerful than the first computer I worked on, a DEC VAX-11/780 which supported about 16 terminals, most used for teaching college level computing. Having that machine to myself would have been an unimaginable amount of processing power for one developer. Banks ran their financial modelling software on boxes like that. So why does the Pi feel so slow? We wasted our gains on GUI fluff.

When I started computing you learned just enough of the command language to get going. So, that’s bash on a pi. Then an editor. For reasons that should become obvious, let us choose emacs. When I first used the VAX/VMS operating system, it didn’t have command line editing. If you made an error, you typed it all again. Getting the facility to press up-arrow, edit the command and re-execute it was a big advance. We should keep it. Bash has that, using a sub-set of the emacs keys, so that’s a way into emacs.

The next big improvements I remember were X windowing and symbolic debugging. We got debugging first but it became far more powerful with multiple terminal windows. The GUI was OK, I guess but DEC didn’t give us many free toys so the main advantage to a developer was having lots of terminal windows. emacs can do that, without the overhead of X.

When I decided to re-learn coding a while back, I got my shortlist of languages down to Python, Java and JavaScript but picked Python because I was already learning a new language and the Object paradigm, so I didn’t want to have to learn web at the same time. I heard about the modern Lisp dialect Clojure and changed horses mid-stream. I’m convinced by the argument that functions and immutability can save the universe from the parallel dimension.

Last night I deep-dived into emacs and found myself in an editor session with 4 windows. Why do I need more than that to learn about computation and data transformation? This guy seems to have come to a similar conclusion http://hackaday.com/2015/09/23/old-lisp-languaged-used-for-new-raspberry-pi-os/ I’ve also wondered whether a purely functional OS might make Sun’s ‘the network is the computer’ dream a little easier. emacs is written in Lisp.

I think a dedicated Lisp machine may be a step too far back. How would you browse in the world-wide hypertext library when you got stuck? But a Linux with bash, emacs, the Java Virtual Machine and libraries, Clojure via Leiningen and Cider to plug everything together might make a fine Lispbian Pi! Is all this chrome and leather trim completely necessary in the engine compartment?

It is unfortunate that the Raspbian upgrade left Leiningen broken.