Category Archives: software craftsmanship

Standard

Posted by

Posted on

2016-05-07

Posted under

devops, golang, software craftsmanship

Go for Dev & Ops

Go (or golang, as it is often called, because “Go” is a bit vague when typed into a search engine), if you haven’t heard of it, is a relatively new language for cross-platform development. It is a modern, C-style, statically-typed language that produces platform-native binaries, but still includes memory management and garbage collection.

Go is open-source, stable (currently at version 1.6, with its first stable release dating back to 2011), and it’s not going anywhere. Written by a couple of Google’s brilliant engineers, and used internally at Google at the levels of scale and reliability you would expect from a tech giant, it is an entirely different beast from something like Ruby or NodeJS. It encourages the kind of development and deployment pipeline that we should all strive for.

Software development has been moving more and more toward interpreted, dynamically-typed, single-threaded languages like Ruby and Node. It’s been moving more toward mountains of external dependencies – I’ve seen projects where more than 90% of the code was in third-party libraries. That’s a lot of other people’s technical debt and generalized implementations to absorb into your project.

Go encourages the exact opposite. It’s statically-typed, though it offers the convenience of type inference. It’s compiled, and not just to bytecode, but to native binary. It’s innately and intuitively concurrent. Its dependency management, like everything else about it, is intentionally minimalist, implicitly discouraging overuse of external libraries and frameworks.

Go, by design, at the language level, encourages a lot of really good programming practices:

  • Because the language and standard library aim for “exactly enough and no more”, developers are more likely to follow that lead with their own code. Go is the embodiment of KISS.
  • Go makes it easy and intuitive to handle concurrency through “communication instead of sharing”, keeping code cleaner and reducing the rate of defects. It also has a built-in race condition detector that can be activated when running tests or benchmarks.
  • The core Go tool chain includes support for unit tests and benchmarks, and will even include example code from your documentation to ensure it compiles and works.
  • Go has a language-level style guide, and included in the core Go tool chain is a tool (gofmt) that reformats files to fit the standard style.
  • Go omits many facets of OOP, including inheritance (though it has mix-ins), abstract classes, method overloading, operator overloading, and multi-level access control. Though useful tools in some cases, these are most frequently used to vastly over-complicate implementations. Go really wants you to just write the simplest thing that will work.

After the code is written, Go makes it very easy to set up continuous integration, including unit tests, benchmarks, linting, and test coverage. It’s trivial to have a build go on to cross-compile for every platform and architecture you might want to deploy to.

How do you deploy a Go app? Drop the executable on a server and execute it. There is no interpreter or virtual machine to install. In fact, there are no external dependencies at all unless you create them in code. Performance is excellent and resource usage is minimal – unlike, for example, Java or Ruby. The deployment footprint is about as small as it gets, and deployments are highly reliable. You suffer none of the fragility brought on by differences between versions of Java, Ruby, Python, or NodeJS – your binary runs the same regardless of the system it’s deployed to.

I won’t say that Go is perfect, or universally applicable. There are certainly use cases it is not well-suited to. For starters, Go produces console applications, not GUIs. It is not intended for building desktop applications – though there are libraries for doing so. Go also has a focus on simplicity and minimalism, and it could get difficult to manage very large code bases in Go unless you’re very careful and very smart about it.

Where Go really shines is in producing small, self-contained, single-purpose command-line tools and microservices. The Go standard library includes a multi-threaded HTTP server perfect for building a REST API server in a hurry, while still being able to stand up to production use.

What sort of things are built in Go? Let’s see:

 

Standard

Posted by

Posted on

2015-07-31

Posted under

programming, software craftsmanship

A Good Case for Third Party Libraries

Developers have a tendency to over use third party libraries. They bring them in because they want some new technical advance that’s in the spotlight – ORM or IOC or AOP. They don’t stop to consider that they’re importing all that library’s technical debt and defects, along with a black box they won’t understand when it fails.

However, there are some excellent cases for third party libraries. Unfortunately, these are the same cases w where developers tend not to reach for an outside solution; cases where developers think, hey, this is easy, I’ve got this.

The best case I can think of for a third party library is functionality that is simple, well understood, and stable, but with a lot of edge cases. The canonical example is date and time handling.

Date and time handling is easy, right? You’ve understood clocks and calendars since grade school. You even grok Unix timestamps. You could do date and time handling in your sleep, with your hands tied behind your back – if you can manage to sleep with your hands tied behind your back in the first place, but that’s neither here nor there.

But wait – you wrote this software to use the user’s local time zone with no offset. Interoperability issues ensue. OK, no big deal, time zones are tricky but they’re not rocket science. You make a fix a move on.

Damn. Daylight savings time. No one hates daylight savings time more than developers. Another patch. But wait, some places don’t do daylight savings time. Another patch. Leap years. Forgot about leap years. Another patch. But they aren’t every four years – the year 2000, for example, was not a leap year. Another patch. Wait, leap seconds? That’s a thing? OK, another patch. What do you mean different people put the day, month, and year in a different order? Alright, alright, another patch. 24-hour time? No problem, patch. Wait, is midnight 00:00 or 24:00? What if someone inputs 24:05? What if someone increments a time by 10 minutes at five till midnight? Or at 1:55 the day daylight savings starts or ends? Or before a leap second is applied? What if a user is mobile and their time zone changes while they’re using the application?

Time and date are simple, every day parts of our lives. We take them for granted. We haven’t changed the way clocks and calendars work in any significant way in our lifetimes. But they’re absolutely rife with details and complexities and edge cases that are very difficult to enumerate off the top of your head – you’re bound to miss some of them. A library is perfect for this: because the functionality doesn’t need to change, a good library will be stable, with very infrequent updates, and few defects. Why kill yourself with patch after patch when there’s bound to be a solid solution waiting out there for you to use?

These are the sorts of cases where libraries are the right answer. Not for adding new bells and whistles, but for using an existing solution to a thoroughly solved problem. Character encoding is a good case. Handling any common file format is definitely a good case – XML, JSON, YAML, CSV. If I never see another hand-written CSV parser it’ll be too soon. Sorting – for Petes sake, if your language doesn’t have sorting built in, find a library. I don’t care if you learned to write a bubble sort in college. Don’t write one at work. Ever.

Anything even remotely related to cryptography you should absolutely solve with a library. Hashing, random number generation, and encryption are solved, hard problems. Even when security isn’t critical, you may as well lean on the work of other smart people.

Solved problems are where you want a library. Solutions in search of a problem may be more exciting, and come with more interesting acronyms and blog posts and conference keynotes, but they often create at least as many problems as they solve. Don’t get sucked in.

Standard

Posted by

Posted on

2015-06-08

Posted under

development, project management, quality, software craftsmanship

Code Reviews

“Code reviews are the worst! All the code I have to review is terrible, and people always take offense when I point out problems. And being reviewed is even worse – people always think they know better than I, can you believe it? And it’s all such a waste of time!”

It can be difficult to get a team started on code reviews, especially an established team without an established culture of reviews. Developers can be very defensive about their code, and often don’t see the value in code reviews. But the value is undeniable, and good developers will come to appreciate code reviews once they get used to them. Why?
  • Reviews reduce defects. This is the primary purpose of reviews, and they’re very effective.
  • Reviewing code builds familiarity for the reviewer. The reviewer is exposed to code they might not have worked with before, giving them a broader base of knowledge in their own development work.
  • Reviewing code improves developers ability to self-review. The more code you review, the better you get at reviewing code. The better you are at reviewing code, the better you can review your own code before you commit it. The better reviewed code is before it’s committed, the fewer defects are found in peer review, and the faster peer reviews are.
  • Expecting code to be reviewed encourages developers to self-review. Knowing that someone else is going to go over your changes after you submit them encourages you to self-review to save yourself the embarrassment, however slight, of a failed peer review.
  • Peer review improves consistency. When developers submit code without anyone else looking at it, they tend to follow their own styles and practices. As they review more of each others’ code, they will naturally tend to converge on a fairly similar set of styles and practices.
  • Peer review helps to breed a sense of collective ownership. After peer review, code is no longer “his code” or “her code” or “their code” it’s “our code”.
  • In situations where everyone is an architect – which I strongly support – peer review is even more critical. Collective design is only collective if everyone is looking over each other’s shoulders, seeing how problems are being solved, and suggesting possible alternative solutions. It really helps close the gap between a group of individuals and a true development team.
Countless tools exist for performing code reviews; the review handling built into pull requests in Atlassian’s Bitbucket and Stash is excellent, though there are many solutions, from simple things like adding a review step to your ticket workflow and putting review comments in the ticket, to using a dedicated review tool like Crucible or Reviewboard.
Standard

Posted by

Posted on

2013-05-10

Posted under

development, programming, software craftsmanship

The Importance of Logging

Add more logging. I’m serious.
Logging is what separates an impossible bug report from an easy one. Logging lets you replace comments with functionality. I’d even go so far as to say good logging separates good developers from great ones.
Try this: replace your inline comments with equivalent logging statements. Run your program and tail the log file. Suddenly, you don’t need a step wise debugger for the vast majority of situations, because you can see, in the log, exactly what the program is doing, what execution path it’s taking, where in the source where each logging statement is coming from, and where execution stopped in the event of a crash.
My general development process focuses on clean, readable, maintainable, refactorable, self-documenting code. The process is roughly like this:
  1. Block out the overall process, step by step, in comments.
  2. Any complex step (more than five or ten lines of code), replace the comment with a clearly-named method or function call, and create a stub method/function.
  3. Replace comments with equivalent logging statements.
  4. Implement functionality.
    • Give all functions, methods, classes, parameters, properties, and variables clear, concise names, so that the code ends up in some semblance of readable English.
    • Use thorough sanity checking, by means of assertions or simple if blocks. When using if blocks, include logging for any failed checks, including what was expected and what was found. These should be warnings.
    • Include logging in any error/exception handling code. These should be errors if recoverable, or fatal if not. This is all too often the only logging a developer includes!
  5. Replace inline comments with equivalent logging statements. These should be debug or info/trace level; major section starts should be higher level, while mid-process statements should be lower level.
  6. Add logging statements to the start of each method/function. These should also be debug or info/trace level. Use higher-level logging statements for higher-level procedures, and lower-level logging statements for more deeply-nested calls.
  7. For long-running or resource-intensive processes, particularly long loops, add logging statements at regular intervals to provide progress and resource utilization details.
Make good use of logging levels! Production systems should only output warnings and higher by default, but it should always be possible to enable deeper logging in order to troubleshoot any issues that arise. However, keep the defaults in mind, and ensure that any logging you have in place to catch defects will provide enough information in the production logs to at least begin an investigation.
Your logging messages should be crafted with dual purpose in mind: first, to provide useful, meaningful outputs to the log files during execution (obviously), but also to provide useful, meaningful information to a developer reading the source – i.e., the same purpose served by comments. After a short time with this method you’ll find it’s very easy to craft a message that serves both purposes well.
Good logging is especially useful in an agile environment employing fast iteration and/or continuous integration. It may not be obvious why at first, but all the advantages of good logging (self-documenting code, ease of maintenance, transparency in execution) do a lot to facilitate agile development by making code easier to work with and easier to troubleshoot.
But wait, there’s more! Good logging also makes it a lot easier for new developers to get up to speed on a project. Instead of slogging through code, developers can execute the program with full logging, and see exactly how it runs. They can then review the source code, using the logging statements as waypoints, to see exactly how the code relates to the execution.

If you need a tool for tailing log files, allow me a shameless plug: try out my free log monitor, Rogue Informant. It’s been in development for several years now, it’s stable, it’s cross-platform, and it’s completely free to use privately or commercially. It allows you to monitor multiple logs at once, filter and search logs, and float a log monitoring window on top of other applications, to make it easier to watch the log while using the program to see exactly what’s going on behind the scenes.Give it a try, and if you find any issues or have feature suggestions, feel free to let me know!