Truly Agile Software Development

Truly agile software development has to, by nature, allow for experimentation. In order to quickly assess the best option among a number of choices, the most effective method is empirical evidence: build a proof of concept for each option and use the experience of creating the proof, as well as the results, to determine which option is the best for the given situation.

While unit tests are valuable for regression testing, a test harness that supports progression testing is at least as useful. Agile development methodologies tend to focus on the idea of iterating continuously toward a goal along a known path; but what happens when there’s a fork in the road? Is it up to the architect to choose a path? There’s no reason to do so when you can take both roads and decide afterward which you prefer.

Any large development project should always start with a proof of concept: a bare-bones, quick-and-dirty working implementation of the key functionality using the proposed backing technologies. It doesn’t need to be pretty, or scaleable, or extensible, or even maintainable. It just has to work.

Write it, demo it, document what you’ve learned, and then throw the code away. Then you can write the real thing.

It may seem like a waste of time and effort at first.  You’ll be tempted to over-engineer, you’ll be tempted to refactor, you’ll be tempted to keep some or all of the code. Resist the urge.

Why would you do such a thing? If you’re practicing agile development, you might think your regular development is fast enough that you don’t need a proof. But that’s not the point; the point is to learn as much as you can about what you’re proposing to do before you go all-in and build an architecture that doesn’t fit and that will be a pain to refactor later.

Even if it takes you longer to build the proof,it’s still worth it – for one thing, it probably took longer because of the learning curve and mistakes made along the way that can be avoided in the final version, and second because again, you’ve learned what you really need and how the architecture should work so that when you make the production version you can do it right the first time, with greater awareness of the situation.

This approach allows much greater confidence in the solutions chosen, requiring less abstraction to be built in to the application, which allows for leaner, cleaner code, and in less time. Add to that the value of building a framework that is flexible enough to allow for progression testing, and you’ve got the kind of flexibility that Agile is really all about.

Note: Yes, I understand that Scrum calls prototypes “spikes”. I think this is rather silly – there are already terms for prototypes, namely, “prototype” or “proof of concept”. I’m all for new terms for things that don’t have names, but giving new names to things that already have well-known names just seems unnecessary.

Driving Algorithm

Yes, I drive by algorithm. I’m a programmer, and generally have an analytical mind. I can’t help it. Here’s what goes on in my head when I’m behind the wheel.

My top priority is awareness. I may drive like it’s a video game, but I do realize it’s the sort of game in which you only get one life. I’m constantly looking around as if I’m about to make a lane change. I try to have a picture in my head of where every car is, and how they’re moving in relation to each other and myself, so I can predict where they’ll be. If a car appears where I didn’t expect it to be, I’ve already failed.

Beyond that, it’s very much like a video game, in several different ways. Like many games, I need to assess my fellow players: who is being aggressive? They’re likely to accelerate suddenly, and take advantage of small openings to make lane changes. I need to be careful around them and keep an extra close eye on them, but I also know they’re not a bad person to be behind. Who is being cautious? They’re likely to drive slower and leave more following distance. I don’t want to be behind them, but I know their habits mean that they’ll provide ample safe lane change opportunities in front of them. Who’s being reckless, or not paying attention, or not maintaining lane? They’re a hazard, and I need to keep my distance and pay close attention to them. I also pay attention to what they’re driving: I know a sporty car is going to accelerate more quickly than an older car, a large truck, or a minivan.

I also need to know the map: is there an on ramp people will be merging from? Is there a lane that ends or becomes turn only? People will be leaving that lane, possibly suddenly if they aren’t familiar with the area. Is there a lane that backs up due to people turning? That lane will be slower, and people are likely to dodge into the next lane to get around the traffic. Any situation with a split, where one lane goes one way and the next lane goes another way (a fork, a turn-only lane, an exit-only lane, etc.)? Both of those lanes will be slower the closer they get to the split, because of people making last-minute maneuvers when they realize their lane doesn’t go where they want to be.

I also need to maintain tactical awareness. If a lane is slower, is there a reason? A slow driver or a big truck that’s slow to accelerate? If a lane is faster, is there a reason for that? Is there genuinely less or faster traffic, or will the lane slow to the same speed after a few hundred feet? If two lanes are moving the same speed, will one become faster or slower? Maybe many people are leaving that lane, or entering that lane. One lane could have more gaps between cars; as those gaps close (I think of this as the traffic compressing), the lane will advance further than a lane without large gaps. One lane could also end up being slower, if there’s a large truck, or a stalled vehicle, or an accident.

Generally, I try to avoid changing lanes unless there’s some reason that the lane I’m moving to will end up being faster than the lane I’m moving from. All things being equal, I try to be in the lane I’m going to need to be in for my next turn/exit/etc. If I’m going to make a lane change, I’m going to take all of the above factors into account: will the lane actually end up being appreciably faster? Is there an opportunity to make a lane change, or will there be soon? Will there be an opportunity to get back into the lane I need to be in, by the time I need to be there? Or, if I’m moving into a lane that ends, will I be able to get out before it does? Would I be getting behind a slower driver that will keep me from advancing, even if the lane itself should be faster?
If I’m trying to get around one or two particularly slow vehicles, I try to plan the entire maneuver: getting out of my lane, getting ahead of them, and getting back into the lane once I’ve passed them. This requires awareness of my lane and the passing lane, and being able to line up both the lane exit and re-entry maneuvers.
Generally, in light traffic, knowing the other drivers and their cars tends to be the most important, because they have the biggest impact on speed of travel in each lane. In moderate traffic, all three factors are fairly equal, but map awareness has a bigger impact in general. In very heavy traffic, situational awareness becomes more important, in order to know which lane is (or will be) the least bogged down.
Yes, I honestly think this way while I’m driving. Also, I tend to sing along to the stereo. Be glad you don’t have to hear it.

A Programmer’s Comparison of Java and C#/.NET

I’ve been developing in Java for almost ten years, and in C# for only a few months, so this comparison may not be as thorough as it could be. If I’ve missed something please feel free to comment.

Java is far more portable than C# for one. I’ve built applications on Windows, and ported them to Linux and Mac with minimal effort. With Android running on a Java-based platform, porting to Android is made far easier. There’s no fussing with Mono and the like, no making sure you don’t use any API functions that are only available in .NET.

Java also has a wide array of available IDEs with excellent features, and a huge, active community. There are libraries available for just about any technology you can think of, usually more than one, such that one of the options will likely fit your specific situation.

Java’s runtime generics seem to be easier to learn and work with than C#’s compile-time generics; however, compile-time generics are likely better performing. Java’s overridable-by-default scheme also makes development a lot easier in many cases. While I do understand the idea behind C#’s final-by-default scheme, I prefer a language that leaves those kinds of choices up to the developer rather than enforcing good development practices through language features.

The JVM is also now expanding to support other scripting languages, including PHP, Python, Ruby, Scala, and others.

C# has some excellent language features that I would like to see in Java, however. Extension methods are extremely useful for adding functionality to classes without having to subclass, particularly useful in adding functionality to library classes. C#’s delegate features are really excellent, and beat any workaround Java has for its lack of closures for callbacks and event handlers. The upcoming Java closure functionality looks like it will still pale in comparison to C#’s delegates.

LINQ is something I would love to see come to Java; the ability to query collections like databases is extraordinarily useful and eliminates a lot of tedious code iterating over collections. I’ve yet to use it for querying against a database, but it seems more than adequate for that purpose and likely much friendlier than JDBC. And while porting is more complicated, Mono is a very strong platform, and there’s even a Mono module for hosting web applications through Apache.

Speaking of web applications, I have no experience so far with building web applications in C# .NET, but I have done some research. Based on that research, I have to say I significantly prefer JSP/JSTL/EL over ASP.NET. I prefer the syntax, the workflow, and JSP’s tag library system over ASP.NET, which reminds me a little too much of PHP or old-school JSP (pre-JSTL/EL) for my tastes.

All in all, I can’t say one is superior to the other; like any development decision, it comes down to which option is best suited to the situation and the developer. If you’ve had the opportunity to make that choice, please leave a note in the comments and let me know what you chose and why, I’d be happy to hear it!

Dynamic Playlists

There is a feature I miss from Audion, which they removed before they retired the app entirely, that I have yet to see recreated in any other music player. It was simple. It was brilliant. I want it back.
Basically, Audion used to allow you to group tracks in your playlists into folders, and – this is the important part – check or uncheck folders to include or exclude them from the playlist, temporarily.
Big deal, right? iTunes lets you check or uncheck songs, and you can multi-select and do a bunch at once. Except that it unchecks them everywhere, not just in that playlist, and you still have to do your multi-select by hand every time.
The beauty of this was it gave playlists a sort of dynamic quality: I could have a playlist with folders for happy tracks, funny tracks, angry tracks, sad tracks, tracks with a good beat, tracks with a fast beat, etc. and so on. Then, depending on my mood, I could check, say, happy songs and songs with a good beat, when I’m in a good mood. Or angry songs and songs with a fast beat, when I’m looking to play some first-person shooters. And if my mood changes an hour later, I can uncheck parts and check other parts and it will just keep shuffling through whatever is active when it comes time to pick the next track. It was brilliant.
And I miss it, and I want it back. iTunes doesn’t do it, WinAmp doesn’t do it, Songbird doesn’t do it, VLC doesn’t do it… nobody does. And it doesn’t seem like it should be necessary to write an entire music player just to get this one feature; maybe one day I’ll get up the nerve to modify Songbird or VLC to do it. Or maybe, sometime between now and then, some kind-hearted developer will hear my pleas and implement it in their player. Who knows.
If anybody out there knows of a player that does have this functionality, please, let me know in the comments… I’d be forever grateful!

XML: No, it isn’t.

XML, well, just isn’t. It’s a raging misnomer. XML is, in theory, the eXtensible Markup Language. I have a couple of problems with that idea.

First, it’s not extensible. It just isn’t. You can’t extend it. I can’t extend it. No one can extend it. You know how I know? Because there aren’t any extensions. Not a single one. Go ahead. Go find an extension to XML. I’d love to hear about it.

But it’s just as well – there’s no reason to extend it. XML defines very little; it’s a syntax definition, nothing more. DTDs and Schemas are what make XML useful. They aren’t extensions to XML, they’re applications of XML. What’s more, the DTDs and Schemas can be combined in a single document, but even they can’t be extended.

Second, while it can be used as a markup language, it almost never is. XHTML is a markup language based on XML. There are a few others that are (debatably) markup languages, like DocBook, but even the likes of DocBook are more on the side of data structure definition than markup. A database file isn’t markup. A Java properties file isn’t markup. It’s a data structure. Per Wikipedia:

A markup language is an artificial language using a set of annotations to text that describe how text is to be structured, laid out, or formatted.

Does that sound like most of the XML formats you’ve encountered? How many XML config files have you had to deal with? Do they fit that description, even a little bit? Of course not. You don’t care about the structure, layout, or formatting of the text in a config file. All you care about is getting at the particular block of text you want. So, what is XML then? Something of a generic hierarchical data file format – though I suppose GHDFF just isn’t as catchy as XML.

Now, besides being aggregiously misnamed, it’s also a wretched tool for nearly every purpose to which it is applied. It’s a language that aims for the middle ground between human-readable and machine-readable, and while it achieves both, it does so very poorly. XML is annoying to read, tedious to write, and resource-intensive to process.

I’m not suggesting dumping XML entirely, not at all. The angle-bracket tax is a fee worth paying for actual markup – you need syntax to seperate the markup from the text. XML is a flexible and effective format for marking up text. What it isn’t is an effective format for storing arbitrary data. It’s usable, but nowhere near optimal. What’s the solution? Something else.

Programmers have a tendency to cling to standards, to try to apply them as much as possible. “Don’t reinvent the wheel,” we say. And that’s a perfectly reasonable mantra – but that doesn’t mean all wheels are created equal. When’s the last time you saw a bicycle wheel on a car? Would the world be a better place if every wheel were the same? Sure, they’d be interchangeable – but they wouldn’t be anywhere near as effective.

We need to step back sometimes, and think about whether there is, or could be, a better wheel for any given situation.

More on this to come.

Ultimate OS Wishlist

You’ll find below my ultimate OS feature wishlist. This is from years of use of Windows and Mac OS, as well as some Linux use. Now, I know, some of these may be covered by one or more of the countless Linux distros out there, however, Linux isn’t ready for my desktop (whether it’s ready for the desktop is a matter for people smarter than I).

The ultimate operating system would feature:
• A completely vector-based UI with full transparency, allowing users to select not only a monitor resolution, but also a display DPI, so that large, high-density displays could be used to display the same size features at higher quality.
• An auto-update system that’s open to all developers to use. Currently, modern OS’s (Mac OS and Windows) supply a system auto-update feature that cannot be used by installed applications, forcing application developers to write their own, separate auto-update libraries.
• A bug/crash reporting system that’s open to all developers to use. We sort of have this now, but it could get a whole lot better.
• A complete skinning/theming system that can be used to apply complete OS-wide interface makeovers, as well as skinning specific applications. Currently, operating systems typically require a seperate piece of software to skin the OS, and many individual applications provide a means to skin that application.
• Filesystem-level local revision control, and integrated access to remote revision control repositories. Revision control isn’t just for programmers any more.
• A tightly-integrated, professional-quality Personal Information Manager (PIM), integrated into the system’s clipboard and drag-and-drop functionality. Apple almost has the right idea here, except that the PIM applications themselves are dreck. It should be simple to use my computer, without any 3rd-party software, to store contacts, set reminders and appointments, create a to-do list, and so forth.
• Multiple clipboards and clipboard history.
• Solid remote command-line and remote desktop capabilities. Only *nix really has this nailed down. I want to be able to throw away my KVM in favor of my LAN.
• System self-optimization based on usage statistics. C’mon, guys, this can’t be that hard. I shouldn’t have to do much, if any, of my own optimization; operating systems should be smart enough to monitor how I use my PC and adjust system settings accordingly.
• Window-manager-level support for tabbed interfaces. It should be up to the user, not the developers, what windows and applications can be run in tabs rather than a slew of individual windows.
• A decent application launcher. Seriously, this is the core functionality of all operating systems – running applications. But still, with every OS I’ve ever used, I’ve had to install a 3rd-party application launcher to really get the most out of my system. I should have a customizable solution that completely eliminates the need for programs like QuickSilver, Colibri, Katapult, DragThing, and so on.
• Easy management of startup items. For crying out loud, this is still a pain in the ass on both Windows and Mac OS. Why?!?
• A cappuccino maker.

So, what’s your OS wishlist? Post in the comments!

Wired vs. Wireless

I was talking to my mom today, and she wanted to set up a wireless network. I advised against it,
and her argument was that wired networks seem archaic.

Now, while I can certainly understand the idea that the same cables we’ve been using for decades are still just fine today may be hard to swallow, but in reality, wired networking is advancing far faster than wireless.

Wireless networking, over the last decade, has gone from 11mbit (802.11b) to 54mbit (802.11g) to “up to 700mbit” (802.11n; effectively 100 – 200 mbit). This bandwidth is per airspace – multiple clients on a network, and multiple networks in the same airspace, must share the available bandwidth.

Meanwhile, wired networks have gone from 10 to 100 mbit, then to 1 gbit and now 10 gbit, all on copper cable. And each client on a wired network gets a dedicated, full-bandwidth pipe all to its own.

So, while the cables may not have changed much (cat 6 is hard to tell from cat 5 to the average person), wired networks are advancing far beyond wireless, and all the while, they provide greater reliability, security, ease of use, and power efficiency.

All in all, I’ll keep my wired network.

Urgent Apps – Mac Development Kit

After yesterday’s post, I got to thinking. I had pulled some items out of that list because they were highly developer-centric applications. However, that does mean that some really top-notch programs didn’t make the list, and I think that’s unfair. There are some apps that I really can’t live without when it comes to development work.

Before we begin, I should point out that my particular development tasks typically include the administration of a MySQL database, editing PHP, Java, HTML, CSS, and JavaScript files, and operating revision control; the tools laid out here are centered around those tasks.

Now, without further ado, the list:

jEdit

jEdit calls itself “the programmer’s text editor”, but that’s selling it short. jEdit is, to put it lightly, a god among executables. I’ve never seen another program come close to its level of flexibility, modularity, and customizability. The sacrifice for all this goodness is that it’s a bit of a RAM-hog, particularly running under the MacOS JRE (I highly recommend updating to the 1.6 JRE available on apple’s website, and completely switching over to 1.6; it provides some vast performance and footprint improvements.)

NetBeans
NetBeans is, of course, the Java IDE, unless you’re one of those people that thinks that Eclipse is the Java IDE, but I’m not.

SmartSVN (payware)
The best SVN client I’ve seen for the Mac. Unfortunately, there aren’t many good free options.

MySQL GUI Tools
The real deal, straight from the source.

Platypus
This nifty little app lets you take any shell or other script file and turn it into a Mac application package.

ArgoUML
Java-based UML designer.

Subversion
Revision control extraordinaire. Any box I do development on has a local Subversion server for anything I happen to want to keep a history for.

Trac
I’ve recently fallen in love with Trac, which is why it made the list. It’s not exactly an application – it’s a web application. However, it can be installed on a Mac, so it made the cut, and I do love it dearly. Go check out their page; the Trac site runs on Trac.

Anything I’m missing? Post in the comments!!

Urgent Apps – "20" Mac Software Picks

So, I just reformatted my laptop and reinstalled the OS, which got me to thinking about which applications I installed first, reflexively, as I can’t stand to be without them – and then I progress on through the stuff that I need infrequently, but I do still need nonetheless. These are my “Urgent Applications”.

Firefox
The best browser, period. Well, okay, Camino might actually be better – I haven’t used it, because it doesn’t support FireFox plugins. Bust.

Adium
The ultimate multi-messenger application. Handles MSNM, Y!IM, AIM, ICQ, GTalk, Jabber, IRC, and a whole host of smaller services. Moreover, it’s just about the most customizable instant messaging app I’ve ever used, on any patform.

Quicksilver
Quicksilver is a launcher and then some. It doesn’t just let you quickly find and open documents and applications; it lets you do anything to them, with just a few key presses. I still have yet to fully discover its potential.

TextWrangler
Nice text editor, especially for programmers, webmasters, and power-users.

Punakea
MacOS file tagger and tag-based file browser. Stores tags in meta data so they can still be searched with Spotlight.

Growl
Global notification app. Many of the programs on this list support Growl notifications, and more applications add Growl support every day.

7zX
On windows, 7Z has always been one of my “urgent apps” whenever I reinstall. 7zX holds the same spot for OS X.

Xcode
I know, I know, it’s cheating, but hey, you do have to manually install it. And besides, it’s required for Fink.

X11
Again, I know, it comes with MacOS, but again, it must be installed seperately. X11 is found in the “optional installs” package on your MacOS disk, and it allows you to run graphical Linux applications on your Mac, side-by-side with Mac applications (programs such as OpenOffice, the GIMP, and Inkscape.)

NeoOffice
A pure Aqua port of OpenOffice. It’s got some quirks, and it tends to be a couple of steps behind the official OpenOffice tree, but it’s still a very solid port.

VLC
VLC isn’t just a multimedia player; it’s an omnimedia player. I’ve only run into a couple of files in my life that VLC can’t play; and even when multiple players will play the same file, VLC usually does so with better quality, less resource usage, in fullscreen (unlike unregistered Quicktime), and for free.

Fink
Fink is the Darwin package manager, like apt or yum (in fact, it’s a forked port of apt to Darwin.) It’s a command-line tool you can use to install and update the thousands of standard (free) packages that have been ported to Darwin.

FinkCommander
For those squeemish at the command line, there’s FinkCommander, which puts a nice graphical interface over top of Fink. Heck, I love the command line, but I still use this instead of fink itself 90% of the time.

FruitMenu (payware)
One of the few pay apps on my list, FruitMenu is well worth the price. It lets you turn your Apple menu into, well, something that’s actually useful for stuff. There’s a free demo, so check it out.

FileZilla 3.0 beta
The famous FTP client for Windows has in its third generation finally been ported to the Mac – and the people rejoiced. My all-time favorite Windows FTP client is finally available, and completely free, for the Mac (and, for that matter, Linux as well!)

Chmox
Mac OS X app that lets you view Windows CHM helpfiles, which are prolific throughout the open source community in providing packaged online documentation. Works very well, with an interface similar to Preview.

OnyX
OnyX is a system tweaking, tinkering, optimizing, and maintenance tool. And it’s free. Go get it. I run the full suite about once a week.

SuperDuper (payware)
The other pay app on my list, SuperDuper lets you quickly back up your Users directory, your entire disk, or any selection of files, to a disk image.

hfsdebug

Hfsdebug is a command-line utility you can use to quickly get information about an HFS drive, such as file size and fragmentation. The only free way I know of to determine file fragmentation on an HFS+ disk.

Dashboard Widgets
iStat Pro
Shows various system stats, such as memory use, network info, disk use, fan speeds, temperature readouts, CPU usage, uptime, battery status, and more.

Delivery Notification
The best package tracking widget I’ve ever seen, bar none – and I’ve used many of them. If you do a lot of online shopping – or even just occasionally – check this little widget out.

Color LS
Yes, yes, this would make #22 (#21 if you count the dashboard widgets as one item, which I do, because I’m a cheater), but it’s not exactly just an application, so I’m not counting it as one. In order to get LS in color, you have to install a version of LS which supports color output. You can do this from Fink by installing the “fileutils” package. This gets you a color-capable ls, but it’s not in color by default; you have to use the –color=always flag. However, you can change this by editing your ~/.bash_profile and adding a line like so:

alias ls=’ls –color=always”

You can find more info here: http://kung-foo.tv/xtips.html#9.

Open Terminal Here Workflow
(See note from Color LS about how this doesn’t put me over 20 items.) Sometimes you’re browsing around the Finder and you need to get to the current directory in a Terminal window – only to discover there’s no easy way to do it; you can’t even quickly copy and paste your current path. So, what is a power-user to do? Script the action!

Right-click on a folder’s background (or the desktop background) and choose Automator -> Create Workflow. This will open Automator with a new Workflow, with “Get Selected Finder Items” already inserted as Step 1. Choose Automator as the library, find Run AppleScript, and add it as Step 2. For the script body, use:

on run {input, parameters}

tell application “Terminal”
set firstpath to item 1 of input
do script “cd ” & (quoted form of POSIX path of firstpath)
activate
end tell

return input
end run

Now save the file as a Finder Plugin, and name it anything you want. Now, to get a Terminal that’s where you are in the Finder, just right-click, and choose your script name from the Automator menu.

Know of a superior alternative to one of my picks? Or something that should be on the list but isn’t? Post a comment and let me know!!

Click "I Agree" To Proceed…

Okay, this is just getting under my skin. I “sign” agreements all the time, every day. You do to. I agree to things I never read. You do that too. Everyone does it. All the time.

I’m talking about EULAs, ToSs, T&C’s, and, of course, “privacy policies”. What is this stuff? Well, the end-user license agreement, or EULA, tells you what you can and can’t do with a piece of software you’re about to install, and it probably mentions anything shady that the software is about to do. Why be so upfront? You’re not going to read it, and it absolves them legally. Unfortunately, perfectly legitimate software has EULAs too, and it’s too hard to figure out which is which that way.

Terms of Service, or ToS, are what service providers, like ISPs and web hosts and such, use to lay down the litany of things which are, should they unfortunately occur, not in any way their problem. It also likely details the things which you are not, under any circumstances, to do with the service in question, regardless of whether it might seem like a perfectly viable use to you or the general public.

Terms and Conditions, or T&Cs, are usually required right before you sign up for something, like internet or cell phone service. Again, this often outlines exactly what you’re not to do with the service, and exactly how much it would cost you if you did.

Privacy policies are my favorite one of all. Why? Because most of the time, you aren’t actually agreeing to the privacy policy. You’re just required to read it. I find this odd – typically, a company requires their employees to know their policies, not their customers. I find this particularly frightening since the only reason I’d be required to know about it is if there’s something shady in there.

Of course, just like you, I still check the box and click the button and make my merry way toward whatever courtroom doom might befall me.

Click “I Agree” to proceed.