Sunday, 29 June 2014

How we think about software and the process of it's creation -

Writing code and developing software is a creative act. A programmer who finds herself in her work wields her tools and applies and refines her craft with as much care and discipline as any other type of artist. Anyone who has ever written software and/or developed an artwork in anger will be familiar with the dual processes of working toward a clear goal or idea, and coaxing an expression into existence through incremental experimentation and failure. 

To illustrate this notion I invite you to take a look at Picasso’s artistic process, dear reader:

["Why You Need to Fail", by Derek Sivers - Picasso is between approximately 9:09 and 10:40]

Perhaps you are not comfortable with an analogy of art/painting for programming? Then the following analogy for programming as gardening may be easier:

["Rather than construction, programming is more like gardening" - source:]

In any case, the idea that I want to express is that – although it makes sense that software development initially be lumped with other engineering disciplines, since it essentially emerged in it's modern form from electrical and electronic engineering – the perception of engineering as the final resting place of software development as a discipline is not right. Software development is bigger than that, and I want to talk about that here…

Software is as-if-not-more malleable than any other living thing or artwork. And – unless you somehow take into account the mass and velocity of the electrons involved – software is not realised through any specific physical structure. It manifests itself as patterns of electronic signals, through which data flows and is transformed. Ultimately, physical activity always results from and/or produces data that is fed into software, but the software itself is purely electronic.

Here’s Fred Brook’s (author of "The Mythical Man Month") take on programming:

“The programmer, like the poet, works only slightly removed from pure thought-stuff. He builds his castles in the air, from air, creating by exertion of the imagination. Few media of creation are so flexible, so easy to polish and rework, so readily capable of realizing grand conceptual structures...”

One of the factors that makes it difficult for the idea of software development to grow beyond the engineering analogy is that working software is hosted by and dependent on physical structure – i.e. hardware. The development and arrangement of hardware naturally runs on far longer cycles than that of software and therefore certainly fits the classical engineering model.

Another factor that I would suggest keeps software development closely linked to engineering is the perception that from a business perspective, art is perhaps not seen as being as serious-of-a matter or as predictable or rigorous as engineering. Software is ever-increasingly important to business; in-fact, much of the world’s modern business process – from the smallest scale through to the very largest – is no longer capable of operating without software (think point-of-sale systems, telecommunications, share-markets and stock-exchanges). The idea that business is heavily dependent on (the result of) a largely artistic process (software development) though would probably make a good percentage of executives feel ill. So maybe from a business perspective, software development’s happy place is engineering.

[Repeat after me "Everything seems to be in ooorderrr" - source:]

Furthermore, for reasons that perhaps justify another blog post – it seems to me that art and engineering are in-fact inextricably linked. All of engineering, from a Mars rover to stone footsteps laid across a stream, could be seen to be forms of installation artwork; each subsequent piece incrementally extending the ideas of its predecessors. And art-for-art’s-sake is apparently a necessity – artistic expression is as old as human history – so it becomes a chicken-and-egg question; which came first, art or engineering?

So I have come to think that art and engineering have a symbiotic relationship. And I think that software development more than (almost?) any other discrete discipline to emerge from history to-date, straddles art and engineering most evenly. One of the ways that technology emerges is when we figure out how to effectively commoditise an art form. That process of commoditisation is still active with software and is perhaps only just beginning.

Tuesday, 28 January 2014

A lap around the "single page web app"...

What's this about then?

2014 will perhaps be the year of the "internet of things" - and the WWW will surely play a big part in that.
Front-end in WWW (HTML5 in particular) is rapidly becoming the vast majority for UI development - consider the fact that even many (most?) successful smartphone apps are running in HTML5 containers like phonegap: 
Having an understanding of the way the web works and what the state of the art is on the web today could save us (as in - software developers working with the WWW and their customers) a lot of time and money.
So I have written this post to help myself better understand. Perhaps you may also find it useful, dear reader!
Please just note that this post (or more accurately, links listed within) occasionally swings wildly from relatively simple to deeply technical...

The single page web app

According to @peterwayner, the JavaScript MV* framework and the single page web app are both now hot:
But web sites/applications used to have lots of HTML files, site maps, etc. So what what's happened? How does one write a web application on a single HTML page?
Basically it seems this trend is supported by HTML5, but has largely been enabled by the rise of the JavaScript async framework.
The evolution has more-or-less been like this:
[Bernard's Interpretation of the Evolution of the "Dynamic" Web Application]

In reference to the "XMLHttpRequest + JavaScript = RYO AJAX script" - if you interested in seeing some source code that does this, take a look at a previous post I made on this subject:

The JavaScript MVC

So let's take a closer look at JavaScript MVC. Basically these are frameworks that enable us to treat a web server and an appropriately designed suite of web services on that server, as an application back-end.
The following section of the (freely available) book "Backbone.js Applications" provides a more detailed overview:
These WWW client-side MVC frameworks have really taken off in popularity over the past couple of years - they are generally based on the likes of jQuery and provide beautiful functionality for UI and back-end integration development purposes.
The following article (I was referred to this compliments of @josh_robb) is what inspired me to write this post and provides an extremely interesting (IMHO!) overview of why JavaScript MVC frameworks like backbone.js' days may be numbered...
Perhaps this is the article to read if you are considering using a JavaScript MVC framework in a project currently: 

Some references

Get Backbone.js:
Get Knockout.js:
Take a look at "Om":


Finally another interesting addition to the mix is node.js; this is not really a web development framework, but in-fact this is JavaScript turned into a server-side language. Indeed, a node.js program can initialize it's own self-hosting web server - i.e. a "node". 
Get node.js:

The enabling substrate

And what does all of this jumble of stuff depend on for it's very existence? It is of course HTTP; currently in version 1.1, or 1.0, depending on which browser you are running. This year though, we will see HTTP 2.0 start to make inroads.
This event is a bit of a big deal and carries with it the potential to turn "conventional" web development on it's head...again. Take a look at this video for an overview:

["Making HTTP realtime with HTTP 2.0" - Source: Realtime Conf 2013]
This video referred to me via @petegoo. Here's the Wikipedia page for HTTP 2.0 also, for reference:

Diagnostic tooling

What would an enthusiast of any genre - including web applications (or "internet of things") development - be without their tools? Probably not an enthusiast...
There are a range of diagnostic tools - that is, tools for analysing the performance/behaviour of your app, as opposed to developing it - available supporting web development. Tooling ranges from on-board (actually within the browser) to IDE integrated, to server-side diagnostics, to bits-and-bytes-over-the-wire-level. 

So here is a non-exhaustive list for your reading pleasure:

On-board: Chrome

Follow these links and update your instance of Chrome to the latest version to take a look at some of the newer tools available for this browser, including the fantastic "flame chart" for JavaScript performance analysis:

On-board: Firefox

Browser-independent: Fiddler

Bits-and-bytes: Wireshark


In summary - have fun!

Monday, 27 January 2014

What’s your view of purpose of life?

I was recently asked during an on-going conversation what my view on the purpose of life was. This is something that I consider here-and-there and since my response was written I figured I'll post it here. I have accompanied it with a couple of photos that I have taken recently.

[Sunset Over the Waitakeres, from Glenfield, Auckland]

So, my initial response to this question these days is usually:
"Honestly I have no idea."
However I followed this up with:

"It seems to me though that there are certain themes in life, and although some of the themes change with time, the one that is constant is to "overcome adversity" in whatever form it comes in.
Engaging the challenges that life puts in-front of me seems to be the source of all of my joy and frustration. When I’m not actively engaging challenges, before too long life seems to lose it’s meaning. I tend to encounter many challenges at once – some I "choose", others arrive of their own accord and demand my attention! Some of my time is spent thinking about how to balance the amount of my time I invest in each. And also there is time spent relaxing, meditating/praying, basically re-centring myself.
I do think that life is infinitely bigger than my own experience though and sometimes it feels as though I am just passively observing the universe and everything I do is automatic."
...and provided the following disclaimer:
"Phew! That’s the best I can do right now – ask me again next month/year though and you might get a different answer!"
...then finally:
"How about you?"
So over to you then, dear reader.

[Rangitoto Island, from Cheltenham Beach, Auckland]

Thursday, 9 January 2014

Software architecture; the lay of the land -


Given the plethora of genres of architect that are sought on [insert your favourite job search tool].com, I thought it would be worth writing a post on this subject, to help myself and anyone else who may be inclined to ponder software architecture, better understand the lay-of-the-land within and outside of their own organisation.
This post is actually a loose assembly of my own thoughts and previous posts that I have made on the matter of programming and architecture; I have copied/pasted, and stitched together to form a somewhat coherent document.


"An architect is a person trained and licensed to plan, design, and oversee the construction of buildings. [...]"
Here's a picture of a Saab – according to Top Gear, all construction type architects own one of these:

[A Saab]

But this post is about software architecture...

So how does architecture relate to programming?

To be clear, architecture is by no means a final or inevitable destination in the career of a programmer. It is more a way of thinking than a "job", and in-fact, a good architect may not necessarily be (or have been) a good programmer.
Architects like to think about the big picture – business implications, what the customer thinks, how a design will fit with company/product strategy, project scope, etc – hence their work is much "fuzzier" than the programmer's. By contrast, a programmer may love the intricacy and precision of their work; designing a complex system at the "nuts and bolts" level and seeing it come to life.
Fred Brooks puts it like this in his influential book The Mythical Man Month:
"[programming is] the fascination of fashioning complex puzzle-like objects of interlocking moving parts and watching them work in subtle cycles, playing out the consequences of principles built in from the beginning. The programmed computer has all the fascination of the pinball machine or the jukebox mechanism, carried to the ultimate."
Neither programmer nor architect is "better" than the other, but they are different and can be expected to have different interests.
As the profession and industry of software development matures, we have an ever-increasing volume of historical accomplishments and/or issues to look back on and pat ourselves on the back about, or alternatively facepalm – depending on how you look at it!


We now understand that it's not how much code is written that is important; what's really important is the quality of the code and whether the code implements a product that meets the customer's requirements. Movements like Agile and Lean (as it applies to software development) reflect this thinking – but that's another story...
Successful software architecture is to a large extent guided by looking to and applying learning from our own history; certainly architecture is more about this, than it is about trying to find new and exciting ways to apply the latest/greatest technologies known to humankind.
Depending on the type of organisation, architecture may be more-or-less important. An organisation that aspires to be a product company for example, may put more emphasis on architecture than a services or utility type company (power, telecommunications, etc). 
To elaborate; imagine a suite of products with implementations geographically/physically distributed across many countries, sites and customers. Design decisions in this situation made around software development can have a far reaching impact. By comparison, "internal" software systems (the services or utility company, for example) maintained by an internal team of programmers, can be manipulated and modified with relative ease; the implication being, priority of design may be overridden by convenience in this situation.

Types of architect/ure

So what are the various "flavours" of architecture that we may come across, within and outside of the organisations we work with?

Solution architecture

The goal of solution architecture is to produce a customised solution for a specific customer. The solution may end up being part of a product, but initially at least, is tailored to a specific customer's (or group of customer's) requirements to solve a specific problem.
Wikipedia sum up solution architecture quite nicely:

Product architecture

This is a relatively old discipline that is applicable to areas outside of software development.
One definition is as follows: "The arrangement of functional elements into physical chunks which become the building blocks for the product or family of products."
Here's a slideshow that Berkley provides online, relating to PA:
Basically the role of the product architect is to think about the products in a company's portfolio and how they can be developed, improved and streamlined in order to make them more marketable, maintainable and effective.

Enterprise architecture (edited 20140714)

Enterprise architecture (EA) departs from other types of architecture in terms of definition and function - it is more of an introspective business discipline that leans on technology (business process design/mapping, automation/improvement, etc) than the other way around.
With the world becoming more-and-more interconnected and automated, and with new ideas being able to be brought to market faster and faster, there is a growing interest in EA as a means to improve internal business process. 
Several EA "frameworks" have sprung up in recent years - here's Microsoft's comparison:
Effective internal business process can be seen as a way to gain competitive advantage, since internal process is harder to imitate (by competitors) and easier to control (by the enterprise) than external factors. Examples of external business factors would be product marketing and development, price-based competition with other companies, etc.

EA's goal is to change and improve internal business process, which is in-turn expected to result in improved ability to successfully negotiate and manage external factors, thus enhancing and enterprise's overall competitive advantage.

Agile architecture

Another more recent addition to the list of types of architecture, and a growing movement, is the concept of "Agile" architecture:
The Agile architecture concept essentially says that architecture is a collective and emergent thing.
The role of the architect in an Agile team is one of bringing ideas together, making sure people feel comfortable sharing their ideas and then assembling the team's thinking to form a coherent "way forward" from a combination of the group's collective thinking and the architect's personal judgement. Sounds a bit "new agey"...but Agile is a bit like that.
Indeed, the role of "Agile Architect" in a team may not necessarily be held by an individual, but exist in the collective thoughts and opinions of the team, as a means to achieve a specific goal.

Concluding thoughts

Although we use titles to distinguish functions, within a single company/industry many of the above "flavours" of architecture may bleed into each other.
People's titles reflect their various backgrounds and interests, and what they are expected to be able to contribute in a team context. Despite this, we may find that the acting architect on a project/team may not hold the title of architect, but rather, be a senior developer, team leader (perhaps even a PM!), etc – essentially, someone who knows and/or has more experience than most about a particular problem/domain.
In Patterns of Enterprise Application Architecture (recommended reading for anyone who is in software development/design-esque line-of-work) Martin Fowler describes software architecture as being:
"The highest-level breakdown of a system into its parts; the decisions that are hard to change; there are multiple architectures in a system; what is architecturally significant can change over a system's lifetime; and, in the end, architecture boils down to whatever the important stuff is."
This seems to fit – basically, in order to deal with architectural decisions and problems, it helps to have the desire to be involved with this type of fuzzy stuff, and also the experience to be able to make informed/effective decisions in the face of fuzzyness – or at least, to get yourself (and the system) out of the schtuk if you make the wrong decision.

[Stakes is high: the "Ford Edsel" -]

Sunday, 24 November 2013

Engineering to management/leadership - you're doing it wrong


Imagine you’re a long-time and/or hard working engineer who has been promoted to manager/leader...

Let me tell you the story of how the transition from engineering to management/leadership often plays out in NZ...and paint the picture of what I think is a national problem faced by NZ, and a bug bear of mine.

The choice of who to promote is right – as a nation, we do this well; by merit. Rather than just promoting the most senior person on a team, we will usually promote someone has demonstrated leadership and/or a propensity to take care of their team mates; make sure their colleagues are happy and have the right tools, etc. I think that this is a reflection of our low rating on the power-distance index and perhaps our perceived relatively low level of corruption - couple of links for reference:

So back to our story - where you find yourself in a leadership/management position. This part is great – as I mention, in New Zealand I think we choose our leaders wisely. Unfortunately from there, I think we tend to drop the ball. Too often, we fail our emerging leaders and managers; sustaining sub-optimal practices.

Here’s how it works – we blame our parents

Because you have probably been given no indication to suggest otherwise – other than a pat on the back and a few gladly received “well deserved” comments – you are left to think that your promotion is attributable to your being a “naturally gifted” manager/leader and that you will be able to take care of a team easily and/or “learn on the job”. But you’re trained and experienced as an engineer; you probably have no formal training/coaching in leadership or management, what you know, you have most likely learned on the job and picked up from others. So when the rubber hits the road and management decisions start falling to you, you fall back to emulating the style of the most influential managers you have worked with in the past. And in most cases (and this is purely in my opinion) those people are probably – guess who – your parents!

When you (and/or your siblings) were born, your parents were also thrown into a management/leadership role and learned on the job, cutting their teeth and learning their lessons on – guess who – you!

And, hey, you are successful person. After all, you have just been promoted – so your parents must have been pretty good managers, right?! Well, unfortunately, unless you had some remarkably enlightened and/or thoroughly experienced parents, in my opinion the majority of the techniques you (and I) are most likely to have learned from them are of the command-and-control genre – remember all those “don’t touch that!”, “go over there!”, “because I said so!”, etc*...? That's what sticks - the more enlightened parenting we get during our teens (once our parents have learned better how to manage and lead), mostly goes in one ear and out the other. At the end of the day - kids are most vulnerable when they are little and are not able to make decisions for themselves (wisely) - arguably, command-and-control works best in this case.

So that’s probably where you start with your own staff; and unless you seek out training/coaching yourself (or an enlightened manager coaxes you in to taking some) – that may well be where you stay. You treat your staff like young children. They get stuff done, but they may well feel dissatisfied, patronised and perhaps even bullied.

Sustainability gone wrong

Basically, this is a vicious cycle; management comes to be seen as something that does not in itself merit dedicated, specific, learning or study – it comes to be seen as something that we are expected to know how to do inherently. Habits are formed and sub-optimal management/leadership styles and patterns are sustained.

Despite this, NZ companies certainly punch above their weight and many are doing rather well currently. In my opinion though, this is not thanks to good management, but despite of relatively poor (or "patchy") management. Our strong current run-rate I reckon is more attributable to our grit and ingenuity. Yes, NZ is doing well currently. In my opinion though, if our organisations were to pay a little more attention to the quality of their leadership and management, rather than throwing our emerging managers/leaders in the deep end and hoping they learn on the job (by testing and refining their potentially flawed/primitive philosophies on their staff), we could be doing much, much better.

All this being said - I certainly think that we are improving. In my relatively short career (to-date) I have seen gradual improvement and recognition that there are smarter, more productive ways to manage and lead people. After all, it's a journey, not a destination!

A prescribed medication

Be assured that there is no doubt we can resolve this situation. I am no pessimist; like I say, I have simply sought here to describe a problem as I perceive it - a bug bear of mine. See my previous post entitled “Agile as an Antidepressant” for part of the antidote. Other things I think we could do are to identify and groom potential leaders and managers earlier, encourage on-going development and training, etc. The whole solution though I think it multi-dimensional and requires on-going attention. It's a cultural issue - there's no easy fix; acknowledgement though, I think is a fair enough start.


The inspiration for this article - which is perhaps a little on the bleak side (and will be followed up with more writing on "the solution" for balance) - comes from two sources:

  1. The Catholic mass ceremony that I attended on Sunday 24/11/2013, in which we heard about Jesus the "servant leader" and the conversation that he had with the two thieves he was crucified next to. There was an emphasis in the story on the style of leadership that Jesus demonstrated. Servant leadership being one of the key tenets of Agile.
  2. A 2011 article (admittedly, much bleaker times for the global and NZ economy) that was relatively critical of NZ managers. Here is a PDF version and here is the original article. This article was was presented to me as a reference by a trainer named Daniel Vidal at a course I am currently attending on business strategy
To be clear, this is not a story about religion - there is no ulterior motive. I like the values of the Catholic Church and/but I am not a “man of faith”. That’s another story.


* There is recognition that the quality of parenting leaves a lot to be desired in NZ and elsewhere in the world also – institutions like “The Parenting Place” can help – like any other form of management and leadership, parenting is an important role, which I think produces the best results when it's well supported, coached and consumes the occasional bit of formal training.


Saturday, 28 September 2013

Buying a House in Auckland - Mortgages and Interest Rates

This post retraces a section of the house hunting and buying "journey" that my Wife and I have been on recently; having bought a house in Auckland. Many aspects of the process were both trying and interesting - and I reckon, worth writing about. In this post I ramble through a few of my thoughts on mortgages and mortgage repayment, interest rates, and loosely how they relate to the 2008 Global Financial Crisis (GFC).

Interest rates

We have found a few particularly useful resources for coming up-to-speed with what's gone on historically with interest rates in NZ and globally, including the Reserve Bank of New Zealand's (RBNZ) website, which provides a great overview of what's happened with fixed and floating mortgage interest rates since 1990. Also, Tony Alexander's weekly overview (includes a useful "If I were a borrower what would I do?" section), which is quite candid, and very useful. I found the following chart on the RBNZ website...

[Source - The Reserve Bank of New Zealand:]

Interest rates and the GFC

On about this date five years ago, the US stock market crashed. Apparently there were actually a few hours on 29/09/2008, when the US stock market went into free-fall, that it was thought the entire US economy could melt down - not just the stock market.

I find the RBNZ chart (above) pretty interesting. My layman's interpretation (which could be entirely misguided!) of that the huge drop in interest rates in 2009 is that it was a response to the US Government flooding the market with cheap credit to keep the economy going following the GFC. And the steady hike in interest rates prior to that, is presumably what led to the sub-prime meltdown that triggered the GFC. Vast numbers of sub-prime borrowers in the USA no-longer being able to afford their mortgage repayments - presumably due to escalating interest rates(?) - defaulted on their payments, causing the whole securitised house of cards to collapse. Happy times...not!

["A Metaphor for The Global Financial Crisis" - Source - not provided (I like the image, but I don't like the site it came from)]

Mortgages and mortgage repayment

Thinking about the the GFC, and looking at the amount of money we will be borrowing on our mortgage, the repayments required on the mortgage, and what's happened with interest rates in NZ over the past few somewhat unnerving. Still, this process (of borrowing a large amount of money to purchase a home) seems to be an entirely acceptable financial risk, a long standing tradition, and perhaps even a rite of passage - at least, to most first-home-buyers in NZ, and to the banks that lend them the cash. That being said, provided nothing terrible happens with the market, our income, or our health - and we have our insurances setup right - we will be able to make our repayments and have a house to call our own. That is a pleasing thought. Here's hoping!

["Our House!" - Source, TradeMe:]

Mortgage repayment structure

So anyway, our research has essentially reinforced our thinking that trying to predict what's going to happen with interest rates is probably a mix of black magic and luck, and at best, an educated guess. 

With regard to our repayment structure; we think that it's best for the most part to map out we think is going to happen with our own circumstances over the next few years, and assemble a fixed repayment structure around that. The theory being that we may have somewhat more control over what happens with our own situation than the global financial markets (at least, so we hope!).

Some useful links for Kiwi home-buyers looking at mortgage interest rates: 

Parting thoughts

Our "house hunt" has been a pretty interesting journey. I'm thinking it'll be useful to document our experience further and I hope to bang out a couple further blog posts on related matters such as the house hunt process, etc, also. For our own future reference and perhaps to help others who may stumble across this blog.

Monday, 9 September 2013

Rewriting and/or Refactoring Software - Recommended Reading

As time goes on, the world is becoming more-and-more automated by software. This essentially also means that there is an exponentially growing surface-area of legacy application code. This matter is especially pertinent to older software product companies, which may have deep bases of legacy of code to grapple with.

Here are a few recommended readings for those who deal with legacy code...

Refactor vs. rewrite

Joel Spolsky wrote the following article way back in 2000, entitled "things you should never do, part 1":

Spolsky is a very good writer,  and expresses simple ideas very well. This article says that it can easily become a strategically disastrous move to rewrite an application from the ground-up. Spolsky recommends against doing that, if at all possible.

On the other hand, there Dan Milstein prodives a counter-argument to Spolsky's, in the following recent response (only 13 years later!):

The "strangler application"

Martin Fowler also gives us some options around how we can better manage the replacement of legacy software with new software, by applying the "Strangler Application" pattern:

Coined by Martin Fowler, this pattern enables the gradual phase-out of a pre-existing codebase into a new one. Fowler of course makes it sound far, far easier than it actually is (or would be). Like Spolsky though, his thinking is clear and he makes a good point.

A recent article (July 2013) written by Paul Hammant discusses a few "strangler application" initiatives that he has been involved with at ThoughtWorks:

If you read nothing else in Paul Hammant's article, go to the the "best practices" section at the end and just read that.

Technical and innovation debt

Ward Cunningham introduced us to the concept of technical debt as early as 1992 - really this is just a version of the old economics mantra of "no free lunch", translated to software engineering:

Then Peter Bell reminds us of the perils of not taking the opportunity to aspire to the cutting edge and resting on our laurels (that is, current/former market success), with his article on "Innovation Debt":

Innovation debt is essentially technical debt, but applies to people, processes and businesses - instead of nuts, bolts and code.

Refactoring legacy code

Michael Feathers provides us with some highly recommended reading on refactoring and working with legacy code in his book of the same name (I've bogged/reviewed on this book here previously):

Feathers defines "legacy" code as essentially being any code that is untested or currently untestable.

The book is written in the style of the "Gang of Four" Design Patterns book - there are three sections; the first sets the stage, introducing Feathers' reasoning, definitions, and some tools. The second section is scenarios (essentially suggestions as to how to apply patterns to deal with problems, like "My Application Is All API Calls") and the third is a reference for a stack of dependency breaking patterns.

Of course, there is also Martin Fowler's original book on refactoring:

To me, Feather's book however seems to be a much easier read.