Pig, satisfied

a blog belonging to Ben Butler-Cole

Visualizing CloudFormation Templates

I’ve been using CloudFormation a lot recently to manage AWS resources. I’m impressed with a lot of the functionality it provides and it has allowed us to delete a lot of code on my current project which was needed to handle edge cases in provisioning AWS resources (eventual consistency issues, for example).

After a couple of months using it seriously, I only have four complaints about CloudFormation:

  • Stack lifecycle operations (create/update and delete) are not idempotent.
  • Notification topics can only be set up at stack-creation time, so if the topic gets accidentally deleted there is no way to get notifications from the stack.
  • Creation and modification of resources in the stack is serialized, which makes operations on large stacks very slow.
  • The template syntax is horrible: hard to write and even harder to read.

A Thin Layer Over Enlive

In my last post I showed a simple use of Enlive to create a web application with a common layout for all pages. I resisted the temptation to introduce any abstractions because I wanted to make it absolutely clear how to use the building blocks that Enlive provides.

But my fingers were itching the whole time to abstract away some of the wrinkles and I couldn’t let it rest until I’d had a play to see what it looks like. So here is a very thin layer over Enlive that manifests some of the structure that I saw.

A Simple Enlive Example

Enlive is a Clojure library for generating HTML that uses transformations instead of templates. Rather than starting with templates containing code which are then executed to produce the final output, it starts with plain HTML which is subjected to a series of transformations; the transformations are ordinary functions, targeted to the right part of the DOM by standard CSS selectors.

This approach allows a clean separation between template and code; it avoids creating a novel hybrid language—often crippled and always nasty—for the templates.

As you can tell, I am pretty sold on the benefits of selector-based templating, so I was excited to come across Enlive when I started playing with Clojure. Enlive is powerful and somewhat complex; there are two good tutorials but I found it hard to get straight in my head which bits were really necessary to do something simple.

So, having wrestled with this for a while (and having surface after a longish dive in the Enlive source code), I’ve put together a basic example, which I think takes a reasonable, minimal approach to using Enlive for a simple web application. This post walks through it, explaining how it fits together. It is not intended to be a complete introduction to what Enlive can do; for that, read the tutorials and documentation.

Why Software Development Methodologies Rock

(This post was insprired by a sneak preview of a blog post by Jez Humble.)

Methodologies or other defined practices can have value in so far as they cause people to reflect on and criticise what they are doing. Often it’s the adoption of a new approach that brings benefits, rather than the approach itself.

This is why consultants are all convinced that their approach is the one-true-way. They tend to meet teams that have stopped reflecting and so are in trouble. They introduce change and see that everything magically starts to improve. A spot of attributional bias and they’re convinced.

The subjects of their advice, however, see that things get worse again after a while (because they’ve stopped reflecting). So they conclude that methodology X is all very well, but in practice it doesn’t work in the long term.

It takes a special kind of person and team to keep reflecting even when there is no change agent. People who can do this without any prompting are gold dust and don’t need a methodology. People who can nearly do it benefit from any methodology that they can be convinced by, because the existence of an ideal keeps them thinking about how they are deviating from it (and it’s the thinking that matters, not the extent of the deviation).

My experience has informed my opinion. My first job was for a small software shop that followed a rigorous waterfall approach: strong hierarchy, lots of documentation, estimation-by-loc-guessing, separate development phases. They were extremely successful and repeatedly delivered on-time and within-budget; I was miserable. My second job was as a consultant (with ThoughtWorks).