Truly Agile Software Development

Truly agile software development has to, by nature, allow for experimentation. In order to quickly assess the best option among a number of choices, the most effective method is empirical evidence: build a proof of concept for each option and use the experience of creating the proof, as well as the results, to determine which option is the best for the given situation.

While unit tests are valuable for regression testing, a test harness that supports progression testing is at least as useful. Agile development methodologies tend to focus on the idea of iterating continuously toward a goal along a known path; but what happens when there’s a fork in the road? Is it up to the architect to choose a path? There’s no reason to do so when you can take both roads and decide afterward which you prefer.

Any large development project should always start with a proof of concept: a bare-bones, quick-and-dirty working implementation of the key functionality using the proposed backing technologies. It doesn’t need to be pretty, or scaleable, or extensible, or even maintainable. It just has to work.

Write it, demo it, document what you’ve learned, and then throw the code away. Then you can write the real thing.

It may seem like a waste of time and effort at first.  You’ll be tempted to over-engineer, you’ll be tempted to refactor, you’ll be tempted to keep some or all of the code. Resist the urge.

Why would you do such a thing? If you’re practicing agile development, you might think your regular development is fast enough that you don’t need a proof. But that’s not the point; the point is to learn as much as you can about what you’re proposing to do before you go all-in and build an architecture that doesn’t fit and that will be a pain to refactor later.

Even if it takes you longer to build the proof,it’s still worth it – for one thing, it probably took longer because of the learning curve and mistakes made along the way that can be avoided in the final version, and second because again, you’ve learned what you really need and how the architecture should work so that when you make the production version you can do it right the first time, with greater awareness of the situation.

This approach allows much greater confidence in the solutions chosen, requiring less abstraction to be built in to the application, which allows for leaner, cleaner code, and in less time. Add to that the value of building a framework that is flexible enough to allow for progression testing, and you’ve got the kind of flexibility that Agile is really all about.

Note: Yes, I understand that Scrum calls prototypes “spikes”. I think this is rather silly – there are already terms for prototypes, namely, “prototype” or “proof of concept”. I’m all for new terms for things that don’t have names, but giving new names to things that already have well-known names just seems unnecessary.

HTML5 Grid Layouts

I have to take issue with the swarm of “responsive grid layout” systems that have been cropping up lately. Yes, they’re great for wireframes and prototypes. No argument there. And yes, they take care of a lot of the legwork involved in producing a responsive layout. Great. But in the process, they throw semantic markup and separation of concerns out the window.

The idea of semantic markup is that your document structure, IDs, and classes should describe the content of the document. Separation of concerns, in HTML and CSS, means using classes and IDs to identify what something is (not how it should appear), and using CSS to identify content and determine how it should appear; this allows you to change content without having to change appearance, and vice versa: the concerns of document structure and appearance are kept separate.

That means, as far as I’m concerned, as soon as you put a ‘class=”two column”‘ into your HTML, you’ve lost the game. You’ve chained you structure to your presentation. Can you change your presentation without modifying the markup? Not any more. All we’ve achieved in this is bringing back the days of nested tables for layout, with a pretty CSS face on it. With one dose of “clever” we’ve traveled back in time 15 years. Only this time, there *are* other ways to do it. There’s no excuse. It’s just plain laziness.

Is building a truly semantic, responsive, attractive layout possible? Absolutely. Difficult? Yes. Is it worth the effort? In the long run, I think it is – except for those cases mentioned above, prototypes and wireframes, code that’s meant to be disposable. But any code that has to be maintained in the future will be hamstrung by these systems.

Web development has made tremendous strides over the last 10 years. It’s amazing how far we’ve come in terms of what can be done and how. Don’t take all those advances and use them to regress all the way back to clunky old table-based layouts. Try using them to do something new, and interesting instead. There’s no reason the idea of software craftsmanship should be missing from the web design world.

Assumptions

We all make assumptions. It’s the only way we can get anything done. If every time you found a bug you started at the iron – testing the CPU to make sure every operation returns an expected result – it’d take you months to troubleshoot the simplest issue. So we make assumptions to save us time, when we know that the likelihood of something being the cause of a problem is far less than the time it would take to verify it.

We also make assumptions out of sheer bloody-mindedness. You can spot these assumptions by phrases like “that couldn’t possibly be it” or “it’s never been a problem before” or “I wrote that code, I know it works”. These are the kinds of assumptions that can get us into trouble, and they’re the exact reason why it’s important to have developers from different backgrounds, with different perspectives, who make different assumptions.

Since we all make assumptions, the best way to challenge those assumptions is to have someone who makes different assumptions look at the issue. They’ll bring their perspective and experience to the matter, challenge your assumptions where they don’t make sense, and make you prove those assumptions to be accurate or not. This stands as a strong incentive to hire a team with diverse backgrounds and areas of expertise. They bring not just talent to your team, but a different perspective.

It’s also a good reason to invest the time in learning different technologies, languages, and development philosophies. Getting outside of your comfort zone can open your eyes to things you might not otherwise have considered, and help you to gain new perspective on your work – helping you to challenge your own assumptions.

The Semantic Web: Practical Semantic Markup

There’s been a lot of talk, for many years about the coming of “the semantic web” – all markup will include semantics that automated systems can read and understand for as-yet-undefined purposes, though prognosticators will speculate on all manner of technical advances that could come from semantics. What about the here and now, though? Right now, today, semantic markup can help you. Semantic markup does one very useful thing: it makes building and styling web pages a heck of a lot easier, if done right.

So, how do you do it right, and reap those benefits? Starting from a blank slate, start filling in your content. Don’t even think about layout or styling – worry only about organizing the content in a clean and sensible way. Your headings should be in h* tags, with lower-level headings using lower-level heading tags. Your content should be in p tags, with no br’s. Enclose the main content in an article tag, enclose sidebars in aside tags, and so on. Enclose your header, navigation, and footer in the appropriate tags.

Load the page. It’ll look like crap. All you’re looking for right now is a sensible document flow. The page should read cleanly from top to bottom with no styling. If not, reorganize your markup until it does.

Now that you have a well-organized, semantically-tagged document, start identifying the parts of the page that are specific to your document. Add id’s to unique elements on the page. Add classes to just about every container on the page to identify its purpose – even if it already has an ID (more on this later.) Name your IDs and classes based on what they identify, not how it’s supposed to look. For example, don’t use “small” or “bold” as class names; if you want your copyright footer to be small, name it “copyright” and worry about the appearance later. If you want text to be bold, use the strong tag if it’s appropriate (e.g. a bold segment of body text), or use a class name that says what the thing is that you want to be bold (e.g. class=”announcement” or class=”specialOffer”.)

Try to use a consistent naming scheme. I use CamelCase for all classes and IDs, with IDs starting with a capital letter and classes starting with a lowercase letter. This is just what makes sense to me personally; it doesn’t matter what your standard is, as long as you find it intuitive and you stick to it.

After all this, your page looks exactly like it did before. Excellent. Now that you’ve got semantic tags identified with semantic classes and IDs, you’re ready to start styling your document. It doesn’t really matter what you start with, but I tend to start with typographic styling. The reason behind this is that typographic styling will change the font metrics, and many parts of a responsive design will be relative to your font metrics, so starting with typography gives you a solid foundation on which to build your layout.

For typography, start at the bottom and work your way up: start by applying your default font style to body, and then use that as a base to style any other elements you need to style – headers, paragraphs, strong/emphasis, a, blockquote, and so on. Start with the most generic styles, where you can apply the style to the tag globally, with no class name or ID specified. Work your way in deeper, first with those cases where you can still identify with only tag names, but based on ancestry; for example, you may want list elements inside nav to look one way, list elements inside article to look another way, and list elements inside an aside to have a third, different styling. This is still global based on document structure, not based on classes or IDs.

View your document again; the layout still sucks, but the document should be readable, and your typography should be pretty close to what you want in the finished product. Identify the places where certain uses of an element – which should already be identified by semantic classes and IDs – should be styled a certain way, and start defining those styles in CSS. Avoid using IDs in your CSS; identifying elements by class rather than by ID lends more flexibility to your code.

Once you have your typography more or less like you want it (at least the font families and sizes), start thinking about layout. Your document is already well-organized, but the layout is very 1995. Now is the time to fix that. Presumably you already have a final design in mind, but if not, take the time to quickly sketch out a rough layout for the page, where you want everything to be, and how you want the document to flow in its final incarnation.

You should conveniently already have all of the blocks that you want to lay out in appropriate tags with appropriate classes, so it should be easy to identify them in CSS. If not, review your markup and clean it up. Again, start with the big chunks and work your way deeper from there. Adjust the layout of the main page elements: header, footer, body, columns/grid. View your page, and start tweaking the layout of the elements within those main containers; adjust the layout of inline elements like sidebars and images, adjust the layout of your navigation items, and so on.

Now that your typography is set, and your layout is looking good, you can start on the fancy stuff, like borders, backgrounds, rounded corners, drop shadows, spriting, and so on and so forth: all of the interface fluff that takes a site from usable to beautiful. We’re on the home stretch now!

If you’re building a modern website, you’re probably going to be implementing some fancy UI behaviors using something like jQuery. Depending on the complexity of what you want to achieve, this may be quick and easy, or it may be weeks worth of iteration. Regardless, you’ve already given yourself a significant advantage: all of that semantic markup, the careful selection and classing of elements, gives you a huge boost using a tool like jQuery, for a couple of reasons. First, it makes it easier to identify the elements you’re trying to control in your scripts. Second, it makes your code more readable automatically, because you can quickly tell from the way you’ve identified an element what you’re trying to do. “$(‘p.boldRed’)” doesn’t tell you much, but “$(‘p.callToAction’)” tells anyone reading the code that you’re manipulating the call to action paragraph on the page. They know what to look for in the HTML, they know what to look for when they’re looking at the page in the browser, it’s all immediately clear from the identifier used.

This is the basic process for building a semantic web page. This doesn’t cover the finer points of responsive design, which is a whole can of worms that I look forward to opening in a future post.

A Programmer’s Comparison of Java and C#/.NET

I’ve been developing in Java for almost ten years, and in C# for only a few months, so this comparison may not be as thorough as it could be. If I’ve missed something please feel free to comment.

Java is far more portable than C# for one. I’ve built applications on Windows, and ported them to Linux and Mac with minimal effort. With Android running on a Java-based platform, porting to Android is made far easier. There’s no fussing with Mono and the like, no making sure you don’t use any API functions that are only available in .NET.

Java also has a wide array of available IDEs with excellent features, and a huge, active community. There are libraries available for just about any technology you can think of, usually more than one, such that one of the options will likely fit your specific situation.

Java’s runtime generics seem to be easier to learn and work with than C#’s compile-time generics; however, compile-time generics are likely better performing. Java’s overridable-by-default scheme also makes development a lot easier in many cases. While I do understand the idea behind C#’s final-by-default scheme, I prefer a language that leaves those kinds of choices up to the developer rather than enforcing good development practices through language features.

The JVM is also now expanding to support other scripting languages, including PHP, Python, Ruby, Scala, and others.

C# has some excellent language features that I would like to see in Java, however. Extension methods are extremely useful for adding functionality to classes without having to subclass, particularly useful in adding functionality to library classes. C#’s delegate features are really excellent, and beat any workaround Java has for its lack of closures for callbacks and event handlers. The upcoming Java closure functionality looks like it will still pale in comparison to C#’s delegates.

LINQ is something I would love to see come to Java; the ability to query collections like databases is extraordinarily useful and eliminates a lot of tedious code iterating over collections. I’ve yet to use it for querying against a database, but it seems more than adequate for that purpose and likely much friendlier than JDBC. And while porting is more complicated, Mono is a very strong platform, and there’s even a Mono module for hosting web applications through Apache.

Speaking of web applications, I have no experience so far with building web applications in C# .NET, but I have done some research. Based on that research, I have to say I significantly prefer JSP/JSTL/EL over ASP.NET. I prefer the syntax, the workflow, and JSP’s tag library system over ASP.NET, which reminds me a little too much of PHP or old-school JSP (pre-JSTL/EL) for my tastes.

All in all, I can’t say one is superior to the other; like any development decision, it comes down to which option is best suited to the situation and the developer. If you’ve had the opportunity to make that choice, please leave a note in the comments and let me know what you chose and why, I’d be happy to hear it!

Convenience Languages

I’ve come to see the uncertainty of untyped and interpreted languages as something of a curse. In a strongly typed, compiled language, you know ahead of time that the code is at least trying to do what you want it to; you know you didn’t typo any variable, function, method, or class names. You know you didn’t misuse or misunderstand a function, passing one type when another is required. Sanitizing inputs and returns is a matter of checking bounds, not types. Type-safe comparison is a non-issue.

After working with PHP and JavaScript extensively, as well as dabbling in Perl, Python, and Ruby, I miss the basic assurance you get from a language like C/C++, C#, or Java that if it compiles, nothing is completely wrong. Even in HTML, you can validate the markup. But in PHP or JavaScript, you probably don’t know about even a major, simple error until run-time testing (unit or functional).

To me, that’s a nightmare. I miss knowing. I miss that little bit of certainty and stability. With an untyped interpreted language, you may never be 100% certain that you’ve not made a silly but fatal mistake somewhere that your tests just didn’t happen to catch.

These are languages of convenience: easy to learn, quick to implement small tasks, ubiquitous. But they just aren’t professional-grade equipment.

Developing software is both an art and a science. I make an effort every day not to just be a coder, but to be a code poet. That’s hard to do on the unsure footing of a dynamic language. I won’t argue that these languages let you do some neat tricks; on the other hand, I also won’t discuss the performance issues. My concern is purely quality.

Is it possible to write quality code in a dynamic language? Absolutely. Unfortunately, it’s harder, and far more rare – not just because it’s challenging. It’s mainly temptation. Why would the language offer global variables if you weren’t supposed to use them? Why have dynamic typing at all if you aren’t going to have variables and function return values that could have various types depending on the context? Even with the best intentions, you can commit these Crimea against code accidentally, without even knowing it until you finally track down that pesky bug 6 months down the road.

Using (and abusing) these sorts of language features makes for messy, sloppy, confusing, unreadable code that can be an extraordinary challenge to debug. Add to that the fact IDEs are severely handicapped with these languages, unable to offer much – if any – information on variables and functions, and unable to detect even the simplest of errors. That’s because variable types and associated errors only exist at runtime; and while an IDE can rapidly attempt to compile a source file and use that to detect errors, it can’t possibly execute every possible code path in order to determine what type(s) a variable might contain, or function might return.

I know most of this has been said before, and every new language will inspire a new holy war. I’m writing this more because all of the above leads me to wonder about the growing popularity of dynamic languages like Python, Ruby and JavaScript, and the continued popularity of PHP. Anyone care to shed some light on the subject in the comments?

Simplicity, Flexibility, and Agility

Agile programming is supposed to be about flexibility in the face of changing requirements. It’s supposed to be about rapid development and iteration. But all too often it ends up being like classical methodologies in many ways. Many agile methodologies drown developers in process, taking time away from development. Test-driven development is a brilliant concept, but it puts more time between planning and iteration, making it more difficult to deal with changing requirements, not easier, and increasing the burden of change and the cost of refactoring.


Every developer wants carefully, precisely defined requirements. Developers often try to handle changing requirements by developing for flexibility, but flexibility often comes at the cost of added complexity. Trying to write-in endless flexibility to allow for changing requirements is very much akin to premature optimization. You end up doing a whole lot of work to make some code “better” – more flexible in this case, versus more performant in the case of optimization – when you don’t yet know which code really needs it and which code doesn’t.

“There are two ways of constructing a software design: One way is to make it so simple that there are obviously no deficiencies, and the other way is to make it so complicated that there are no obvious deficiencies.” –C. A. R. Hoare 

Often the best way to maintain flexibility is through simplicity. A program that meets its requirements with the simplest possible implementation is one that will be naturally flexible, maintainable, stable, and manageable.Of course, intelligent development plays a major role; making appropriate use of design patterns can do a lot to improve both flexibility and simplicity. Key design tenets like DRY, YAGNI, separation of concerns, and avoiding premature optimization (including overarchitecting for flexibility) help keep complexity down and productivity up.

What if requirements change? What if the simplest possible solutions isn’t as extensible? Good news: having invested in the simplest possible solution, you’ve lost little in development. You haven’t built anything that wasn’t strictly necessary. The new solution will still aim for simplicity, still reap the same rewards. By keeping things simple, you’ve kept down the cost of change.

Software development is a learning process by nature; you’re building something that’s never been done before, or at least building something in a way that’s never been done before. Innovation is at its heart, and learning is the personal experience of innovation. That being the case, every iteration has value in what’s learned, even if the code is later removed or replaced. The experience of writing it adds value to the team, and the team defines the value of the end product.


Many readers may think of targeting simplicity as a given, but it truly isn’t; while simplicity is often a goal, all too frequently it takes a back seat to other concerns, or is abandoned altogether because simplicity becomes far more difficult to achieve with many of the popular frameworks and libraries available today. Frameworks have to aim for maximum flexibility in order to be successful; in order to be general-purpose, they can’t make many assumptions, and they have to account for a whole host of different usage scenarios and edge cases. This increases the complexity of the framework, and accordingly, the complexity of any implementation using the framework. The fewer assumptions the framework can make, the more effort a developer has to put in just telling the framework what she’s trying to accomplish.


I can’t count how many implementations I’ve seen that are drastically more complex than necessary, simply because they have been forced to apply the conventions required by their chosen framework; and even following those conventions, they’re still left managing arcane configuration files, and tracking down bugs becomes an epic undertaking requiring delving deep into the inner workings of the framework and libraries. All too often, the quick-start “hello world” app is far more complex with a framework than without it, and adding functionality only makes the situation more bleak.


So, what does all this add up to? Here’s the bullet-point version:

  • If you’re aiming for agility – the ability to adapt quickly to changing requirements – don’t invest too much time on nailing down requirements. Get as much detail as you can, and start iterating. If you’re planning for requirements to change, go all the way – assume that your initial requirements are wrong, and think of each iteration as an opportunity to refine the requirements. All code is wrong until proven right by user acceptance.
  • Use interface mockups (for software with a UI) or API documentation (for libraries and services) as a tool to give stakeholders a chance to revise requirements while looking at a proposed solution and thinking about using it in real-world scenarios.
  • Don’t choose flexibility or modularity over simplicity. Choose a framework that won’t get in your way; if there isn’t one, then don’t use a framework at all. Don’t write what you don’t need. Don’t turn a piece of code into a general-purpose API just because you might need to use it again. If you need it in multiple places now, separate it out; otherwise, you can refactor it when it’s appropriate.
  • Think about separation of concerns early in the game, but don’t sacrifice simplicity for the sake of compartmentalization. Simple code is easier to refactor later if refactoring turns out to be necessary. Overarchitecting is the same sin as premature optimization, it’s just wearing a nicer suit.
  • The simplest solution isn’t always the easiest. The simplest solution often requires a lot of thought and little code. Don’t be a code mason, laying layer after layer of brick after brick; be a code poet, making every line count. If a change could be implemented by increasing the complexity of existing code, or by refactoring the existing code to maintain simplicity, always take the latter route; you’ll end up spending the same amount of time either way, but will reap far more benefits by maintaining simplicity as a priority.
  • Simplicity carries a great many implicit benefits. Simpler code is very often faster (and easier to optimize), more stable (and easier to debug), cleaner (and easier to refactor), and clearer to read and comprehend. This reduces development, operational, and support costs across the board.
  • Simplicity doesn’t just mean “less code”, it means better code. SLOC counts don’t correlate directly to complexity.
  • Don’t reinvent the wheel – unless you need to. All wheels aren’t created equal; there’s a reason cars don’t use the same wheels as bicycles.
What are your experiences? What ways have you found to keep hold of simplicity in the face of other pressures? Feedback is welcome in the comments!