Erlang

Just bought Joe Armstrong's book on Erlang and have started working through it. Erlang's an interesting language: although it's been around for years it's recently seen a sudden rise in popularity. Why the sudden interest? Concurrent programming. Moore's law is disintegrating for single core processors, and multi core/multi-processor machines are seen as the way forward. For those used to programming in traditional imperative languages (C, C#, Java, ...) multi-threading has always been difficult and error prone. Erlang was conceived from the beginning to support hundreds, thousands, even millions of concurrent processes, communicating with each other by passing messages. Armstrong contends this makes concurrency much, much simpler than trying to force it on top of the inherently sequential, shared memory model underpinning imperative languages.

Erlang is a functional language which, among other things, means it doesn't support mutable state. 'Variables' aren't; once assigned a value, it's not possible to re-assign a variable to a new value. So variables would be more accurately described as 'immutable single assignment references', although that's admittedly a bit less catchy.

As someone schooled in mutable state since Amstrad CPC464 basic, that's a violent, disruptive change to the mental model. In fact, it's a disruptive change for anyone used to observing the world around them as entities with properties that change over time: bank account balances that vary from day to day, people whose age increases every birthday.

It's a practical problem for functional languages too, since for most any program to be useful it needs to persist or change the state of the world around it. (That's why Haskell has its monad system).

Erlang supports programs that need to store, update and use mutable state; it has its own relational database system (Mnesia) which - since it's written in Erlang - is fully ditributable, concurrent and fault tolerant. Oh, and it doesn't need an object-relational mapping layer either since it's built to work natively with Erlang's type system.

I'm looking forward to getting into Erlang. The concurrent, message passing model is very appealing conceptually and offers lots of possibilities in the new multi-core world. But more than anything else in the book, I'm looking forward to getting my head around marrying the functional model with mutable state.

Textual Input / Graphical Output - the best of both worlds?

Textual vs. Graphical representation was a recurring theme at this years Code Generation Conference. Thankfully, we seem to have moved beyond the “models are graphical, code is textual” misconception. The discussions were more focused on the relative strengths and weaknesses of each.

Arno Hasse and Sven Efftinge categorised it really nicely: graphics are best for visualisation, text is best for editing. There's been some work to integrate the two; for example, it's now possible in Eclipse to generate both graphical and textual editors that act on the same underlying model simultaneously. Marcus Volter has also demonstrated generating graphical visualisations automatically from models using a couple of auto-generation tools (graphviz and prefuse).

Outside of software development tools, this is nothing new: Autocad has for years allowed designers to create drawings with textual commands. Whilst users can draw directly on the canvas, they can also use a textual dsl enter commands - such as lineto(20,100) - with the results rendered graphically.

It would be good to think software tooling will catch on to these possibilities - and it looks like Openarchitectureware may be in the vanguard.

WebDSL and the missing abstractions

“drinking from the firehose” must be on everyone's buzzword bingo all-time greats list. Nevertheless, it's a very apt description of how I felt in Eelco Visser's talk on WebDSL during the code generation 2008 conference.

The core subject matter has been occupying my thoughts in the last few months; there are myriad lightweight web frameworks around now, all of which offer easy definition of the core domain objects and most of which will generate a standard CRUD GUI. All good stuff. But move beyond the auto-generated GUI into something custom and you're on your own. It's down to hacking some form of web template language (JSP/RHTML/Django templates/...) and patching up the link to the domain model through controllers.

I'd been thinking there must be a better, higher level set of abstractions for describing the UI – which could in turn be mapped onto the underlying technology. WebDSL addresses exactly that problem. At least I think it does. There was so much good stuff in Eelco's talk that I couldn't take it all in. I'll need to dig deeper – but right now it looks promising.

Editing Generated code - an anti-pattern?

In the 1970s, Ivor Tiefenbrun revolutionised the hi-fi world. Perceived wisdom held that, since the loudspeakers produced the sound, they were the most important part of the system. So, for a given budget, the largest portion should be assigned to the speakers.

Tiefenbrun disagreed. He believed the most important part was the input. Since the sound originated from the record (vinyl in those days), the most important component was therefore the turntable. So he invented the Linn Sondek, revolutionising the industry with a product that is still for sale today.

Tiefenbrun's lesson came to mind at the code generation 2008 conference in Cambridge – stimulating content in a fantastic setting.

During a discussion in Anneke Kleppe's session, a delegate asked about good practice for editing / modifying / augmenting generated code. The usual solutions came up (Generation Gap pattern, protected blocks) but for me the answer is clear: any kind of modification to the generated code is bad. It's an anti-pattern, there only because of deficiencies in the input and/or lack of customisation in the generator. It's often necessary, in many cases simply because the input language doesn't allow the full model to be expressed (this is true for most mainstream UML tools). But we should be striving to fix the real problem – insufficient or incorrect input – instead of trying to fix the symptom. We need to apply the 'Tiefenbrun principle'.

Agile and the pragmatists

I listened to the Naked Agilists latest podcast the other day.

Brian Marick's pitch in particular sparked my interest. He compares agile adoption against Geoffrey Moore's technology adoption curve and observes a discrepancy; namely the seeming dearth of pragmatists. As I listened, a fundamental question immediately sprung to mind:

- Do we really have a case where the standard curve doesn't fit, or
- Are we looking at the data incorrectly?

Moore's curve is of course statistical and hence variation is to be expected. Nevertheless, I can't help wonder if there's something else going on.

Software projects can be viewed in terms of 4 dimensions: cost, time, quality and function (per e.g. Kent Beck in "Extreme Programming"). The software industry can also be split roughly into IT and Product Development (PD) - the difference being that IT delivers solutions for use within a business; product development builds products that are sold to the market.

I've worked in and/or experienced a variety of both IT and PD companies. Despite both being ostensibly focused on software delivery, there are some notable differences. In particular, there seems to a different bias on the 4 project dimensions:

- IT seems to focus more on cost and schedule;
- PD tends to focus more on function and quality.

I've some theories about why that might be, but will save for later - it's not relevant here. The interesting question is whether this difference in any way explains Brian's observation.

Here's my thoughts:

• Agile primarily manages delivery of working software --> focuses on function & quality over cost & schedule
• IT projects are often implementations of purchased products
• The majority of Agile use is associated with building systems rather than implementing products
• IT is generally more conservative than PD

Therefore:

Agile's lack of penetration into the pragmatist market is a consequence of the increased IT representation in that segment.

Some observations arising from that assertion:

• What would the curves look like if we separated IT & PD? My gut feeling is each would more or less follow Moore's model. Agile probably has crossed the chasm for PD; and the size of the segment is probably in line with Moore's curve. Agile in IT is however much earlier in the lifecycle, and crucially hasn't crossed the chasm yet.
• For agile to cross the IT chasm, two things are required:
1. There needs to be much more support for agile product implementation to complement that for agile system building
2. It needs to be presented in a way that resonates with people whose primary focus is cost and schedule over function and quality. That means IT project/programme managers brought up on a diet of MS Project and interpreting PRINCE2 as a waterfall process.
   

I'm aware the assertions above are based on a very narrow - and therefore statistically insufficient - sample. I know, for example, of at least two IT organisations locally who've embraced agile principles. Thoughts and comments welcome.

UML vs. Domain-Specific Languages - a false dichotomy?

I have just listened to the panel discussion at code generation 2007 entitled UML vs. Domain-Specific Languages - a false dichotomy?

It's very interesting. The panel includes some very well known luminaries from the modelling world (e.g Steve Cook) and some less well know but equally capable (such as Tony Clarke of xactium).

Perhaps the heart of the discussion is given over to the subject of modelling versus programming languages. This is something I've come across in various contexts, where the following views are not uncommon:

• "Modelling languages are for drawing pictures, you build software with programming languages"
• "If it's graphical, it's a modelling language; programming languages are textual"

Both views are as understandable as they are wrong. Promulgating them is perhaps the most heinous crime of the uml juggernaut. Even today, Grady Booch - one of the uml's originators - maintains it should be used for sketching things out; clarifying your thoughts before writing code. There are plenty of others who also support the view (e.g. Scott Ambler).

As a result the majority body of opinion equates uml - and therefore graphical notation - with sketches. Something that precedes the real work (coding).

That's a great shame - and baseless. In other industries (mechanical engineering and construction for example) graphical notations are standard. Not for sketching, but as the normative specification. The code if you like.

Graphical notations can be used in software too. Some companies have formalised a subset of the uml into a language that is precise, executable and graphical. Conversely this post is textual - but definitely not executable.

Abstract Syntax, Concrete Syntax and Semantics

UML 2.0 wasn't all bad. One of the good things it popularised was a way to look at languages as consisting of three parts: abstract syntax, concrete syntax and semantics. (I prefer to name them concepts, representations and meanings, but that's personal preference):

• Concepts (or Abstract Syntax) are the things we want to talk about: 'Cars', 'Accounts', 'Dogs', 'Equations', whatever.
• Representations (or Concrete Syntax) are how we depict those concepts; textual, graphical, aural, olfactory...
  
• Meanings (Semantics) are, well, what we mean when we discuss concepts.

It's often difficult to separate representation from concept, since we can only communicate concepts by giving them representation. Using different representations for the same concepts is so intrinsic to our way of life we don't even notice it. (Imagine the confusion if we didn't know the sound 'car' referred to the same concept as the word 'car').

But analysing languages in terms of their concepts, representations and meaning provides a useful framework for classifying languages.

So is UML vs DSLs a false dichotomy?

Let's consider it from the 3 perspectives.

• Concepts. At its (most useful) core, the uml is intended for describing concepts, their relationships and behaviour. So its concepts - classes, relationships, states, events, etc. - are those useful for describing other concepts. The concepts in a DSL are, by definition, specific to the domain in question. So perhaps Dogs, Breeds and Owners in the domain of pedigree competition. Is that a dichotomy? No. Just a different focus. In fact the raison d'etre of UML's core is to enable description of other domains. The uml is, in effect, a DSL for describing other domains.
  
• Representations. UML ascribes a graphical notation to many of its concepts - perhaps most obviously the boxes and lines in class diagrams or state charts. Many would argue - and with good cause - this is in fact the most useful thing about UML. DSLs usually enable a representation appropriate for the domain. It might be graphical (perhaps an image for each dog appropriate to its breed), symbolic (the extended alphabet used in mathematical equations) - whatever suits the problem. Is that a dichotomy? Again, no. However it does expose a limitation in UML, in that it doesn't explicitly allow for representations to be ascribed to concepts described in a UML model.
• Meaning. The semantics of UML is a subject of life study, not a paragraph in a blog. Despite its supposed formalisation, UML does not have a precise semantics in any useful meaning of the term. That belies its origins as a sketching tool. While some have divined a precise subset it's been done despite the language rather than because of it. DSLs, by comparison, assume that the results of modelling will be machine processed: translated into some other executable form or interpreted directly. Is that a dichotomy? Fundamentally yes. Those are two very different philosophies.

So UML and DSLs do share significant properties. DSLs extend UML in their support for domain specific representation, but it's an extension rather than a conflict. Were UML to allow the assignment of domain-specific notation to the models this difference would disappear.

Where the two fundamentally differ is on semantics. As long as the majority influence on UML supports the standpoint of a sketching toolkit, the dichotomy will remain. There will continue to be those who provide a formal semantics for some subset of uml and in such cases there is no real dichotomy; however those are the exception and not the rule.