Simplicity, Flexibility, and Agility

Agile programming is supposed to be about flexibility in the face of changing requirements. It’s supposed to be about rapid development and iteration. But all too often it ends up being like classical methodologies in many ways. Many agile methodologies drown developers in process, taking time away from development. Test-driven development is a brilliant concept, but it puts more time between planning and iteration, making it more difficult to deal with changing requirements, not easier, and increasing the burden of change and the cost of refactoring.


Every developer wants carefully, precisely defined requirements. Developers often try to handle changing requirements by developing for flexibility, but flexibility often comes at the cost of added complexity. Trying to write-in endless flexibility to allow for changing requirements is very much akin to premature optimization. You end up doing a whole lot of work to make some code “better” – more flexible in this case, versus more performant in the case of optimization – when you don’t yet know which code really needs it and which code doesn’t.

“There are two ways of constructing a software design: One way is to make it so simple that there are obviously no deficiencies, and the other way is to make it so complicated that there are no obvious deficiencies.” –C. A. R. Hoare 

Often the best way to maintain flexibility is through simplicity. A program that meets its requirements with the simplest possible implementation is one that will be naturally flexible, maintainable, stable, and manageable.Of course, intelligent development plays a major role; making appropriate use of design patterns can do a lot to improve both flexibility and simplicity. Key design tenets like DRY, YAGNI, separation of concerns, and avoiding premature optimization (including overarchitecting for flexibility) help keep complexity down and productivity up.

What if requirements change? What if the simplest possible solutions isn’t as extensible? Good news: having invested in the simplest possible solution, you’ve lost little in development. You haven’t built anything that wasn’t strictly necessary. The new solution will still aim for simplicity, still reap the same rewards. By keeping things simple, you’ve kept down the cost of change.

Software development is a learning process by nature; you’re building something that’s never been done before, or at least building something in a way that’s never been done before. Innovation is at its heart, and learning is the personal experience of innovation. That being the case, every iteration has value in what’s learned, even if the code is later removed or replaced. The experience of writing it adds value to the team, and the team defines the value of the end product.


Many readers may think of targeting simplicity as a given, but it truly isn’t; while simplicity is often a goal, all too frequently it takes a back seat to other concerns, or is abandoned altogether because simplicity becomes far more difficult to achieve with many of the popular frameworks and libraries available today. Frameworks have to aim for maximum flexibility in order to be successful; in order to be general-purpose, they can’t make many assumptions, and they have to account for a whole host of different usage scenarios and edge cases. This increases the complexity of the framework, and accordingly, the complexity of any implementation using the framework. The fewer assumptions the framework can make, the more effort a developer has to put in just telling the framework what she’s trying to accomplish.


I can’t count how many implementations I’ve seen that are drastically more complex than necessary, simply because they have been forced to apply the conventions required by their chosen framework; and even following those conventions, they’re still left managing arcane configuration files, and tracking down bugs becomes an epic undertaking requiring delving deep into the inner workings of the framework and libraries. All too often, the quick-start “hello world” app is far more complex with a framework than without it, and adding functionality only makes the situation more bleak.


So, what does all this add up to? Here’s the bullet-point version:

  • If you’re aiming for agility – the ability to adapt quickly to changing requirements – don’t invest too much time on nailing down requirements. Get as much detail as you can, and start iterating. If you’re planning for requirements to change, go all the way – assume that your initial requirements are wrong, and think of each iteration as an opportunity to refine the requirements. All code is wrong until proven right by user acceptance.
  • Use interface mockups (for software with a UI) or API documentation (for libraries and services) as a tool to give stakeholders a chance to revise requirements while looking at a proposed solution and thinking about using it in real-world scenarios.
  • Don’t choose flexibility or modularity over simplicity. Choose a framework that won’t get in your way; if there isn’t one, then don’t use a framework at all. Don’t write what you don’t need. Don’t turn a piece of code into a general-purpose API just because you might need to use it again. If you need it in multiple places now, separate it out; otherwise, you can refactor it when it’s appropriate.
  • Think about separation of concerns early in the game, but don’t sacrifice simplicity for the sake of compartmentalization. Simple code is easier to refactor later if refactoring turns out to be necessary. Overarchitecting is the same sin as premature optimization, it’s just wearing a nicer suit.
  • The simplest solution isn’t always the easiest. The simplest solution often requires a lot of thought and little code. Don’t be a code mason, laying layer after layer of brick after brick; be a code poet, making every line count. If a change could be implemented by increasing the complexity of existing code, or by refactoring the existing code to maintain simplicity, always take the latter route; you’ll end up spending the same amount of time either way, but will reap far more benefits by maintaining simplicity as a priority.
  • Simplicity carries a great many implicit benefits. Simpler code is very often faster (and easier to optimize), more stable (and easier to debug), cleaner (and easier to refactor), and clearer to read and comprehend. This reduces development, operational, and support costs across the board.
  • Simplicity doesn’t just mean “less code”, it means better code. SLOC counts don’t correlate directly to complexity.
  • Don’t reinvent the wheel – unless you need to. All wheels aren’t created equal; there’s a reason cars don’t use the same wheels as bicycles.
What are your experiences? What ways have you found to keep hold of simplicity in the face of other pressures? Feedback is welcome in the comments!

Urgent Apps – Mac Development Kit

After yesterday’s post, I got to thinking. I had pulled some items out of that list because they were highly developer-centric applications. However, that does mean that some really top-notch programs didn’t make the list, and I think that’s unfair. There are some apps that I really can’t live without when it comes to development work.

Before we begin, I should point out that my particular development tasks typically include the administration of a MySQL database, editing PHP, Java, HTML, CSS, and JavaScript files, and operating revision control; the tools laid out here are centered around those tasks.

Now, without further ado, the list:

jEdit

jEdit calls itself “the programmer’s text editor”, but that’s selling it short. jEdit is, to put it lightly, a god among executables. I’ve never seen another program come close to its level of flexibility, modularity, and customizability. The sacrifice for all this goodness is that it’s a bit of a RAM-hog, particularly running under the MacOS JRE (I highly recommend updating to the 1.6 JRE available on apple’s website, and completely switching over to 1.6; it provides some vast performance and footprint improvements.)

NetBeans
NetBeans is, of course, the Java IDE, unless you’re one of those people that thinks that Eclipse is the Java IDE, but I’m not.

SmartSVN (payware)
The best SVN client I’ve seen for the Mac. Unfortunately, there aren’t many good free options.

MySQL GUI Tools
The real deal, straight from the source.

Platypus
This nifty little app lets you take any shell or other script file and turn it into a Mac application package.

ArgoUML
Java-based UML designer.

Subversion
Revision control extraordinaire. Any box I do development on has a local Subversion server for anything I happen to want to keep a history for.

Trac
I’ve recently fallen in love with Trac, which is why it made the list. It’s not exactly an application – it’s a web application. However, it can be installed on a Mac, so it made the cut, and I do love it dearly. Go check out their page; the Trac site runs on Trac.

Anything I’m missing? Post in the comments!!

Development Tactics

I recently set up an account with hosted-projects.com, because I wanted a Subversion repository more accessible & stable than the one running on my home desktop. I shopped around for a while, and decided on this place – it’s a small project, and a starter account is only $7/month, so I figure, what the heck.

My account was set up within a few minutes, even though I ordered after business hours – I’m guessing they’ve got a pretty good automation system going. I get fast, secure access for unlimited users to unlimited projects in 100M of space, plus a free Trac – not a bad deal. As far as reliability and support, well – only time will tell.

The host is all well and good, but what I really wanted to talk about is Trac. I had looked Trac up some time ago, and decided to take a pass on it – it just wasn’t mature enough at the time, and didn’t have most of the features I was looking for.

Now, however – after some time, and a few bug tracking schemes – I find myself with a free Trac page sitting around, and I figure, what the hey, I’ll give it a shot. And you know what? It still doesn’t have some of the features I was looking for. But it works so well, it doesn’t matter.

The whole thing runs on a Wiki engine. This Wiki engine identifies all CamelCase as wiki links, which I find a bit annoying, but I got used to it pretty quickly. It lets you easily link to pretty much anything, and inline, too: #123 is ticket 123, r456 is revision 456, etc. It hooks up to your Subversion repo and lets you keep an eye on changelogs and browse the repo; plus, this means if you put properly formatted notes in your commit messages (which isn’t hard), you get links in the changelog, for free.

While not quite as versatile as MediaWiki, for example, in terms of page layout and design, it’s probably easier to use – and programmers tend to go for form over function anyway. It’s a developer’s tool. Developers probably won’t spend all day perfecting page templates and macros.

The system provides for a roadmap of milestones, a list of issue tickets, the wiki, and the repository. That’s it. What’s the big deal? How insanely easy it is to wire them all together. With some really basic formatting, you can turn a simple list of milestones into this.

It’s got some rough edges, and there are definitely some huge opportunities yet to be taken advantage of – particularly, I have yet to discover decent, proper JavaDoc support, with full wiki integration. I may just have to learn enough Python to write a plugin for it. I’d also really like to see automatic backlinks added to all the internal links.

I know it’s still version “0.10.3”, but it’s pretty stable so far, and everything works pretty well. I have yet to run into any bugs or bad behavior – however, you should keep in mind that this is bleeding-edge software if you’re considering deploying it. Don’t let that scare you off though: if you don’t mind the under-heavy-development label, you really should give this little application a try and see what you think. At the very least, check out Trac’s own website to see what it can do.