Simple Entity Framework Model Structure

I’ll say right up front, I don’t have a lot of experience with Entity Framework, and this could either be a well-known solution or a completely foolish one. All I know is that, so far, it has worked extremely well for my purposes.

Coming from the Java world, I’m used to using DAO’s to serve as an abstraction layer between the controllers and the database, with the basic CRUD methods, plus additional methods for any specific queries needed for a given entity type.

Conveniently, entity framework provides a fairly complete DAO in the form of DbSet. DbSet is very easy to work with, and provides full CRUD functionality, and acts as a proxy for more complex queries. I wanted to keep queries out of my logic, however, and in the model.

Looking at it, I didn’t want to have to write an entire wrapper for DbSet, and subclassing it seemed like asking for trouble. That’s when it occurred to me to use extension methods for queries. It turns out you can define extension types against a generic type with a type argument specified (e.g. this IEnumerable). This not only allowed me to abstract out the queries and keep them in the model, without having to wrap or subclass anything; but by defining the extensions on IEnumerable instead of DbSet, I have access to my queries on any collection of the appropriate entity type, not just DbSet. I can then chain my custom queries in a very intuitive and fluid way, keeping all of the code clean, simple, and separate.

For example, I have a table of tags. I’ve created extension methods on IEnumerable to filter to tags used by a given user, and to filter by tags starting with a given string. I can chain these to get tags used by a given user and starting with a given string. I can also use these queries on the list of tags associated with another entity, as IList implements IEnumerable, and thus inherits my query extension methods.

I don’t know if this is the best way – or even a good way – but it’s worked for me so far. I do see some possible shortcomings; mainly, the extensions don’t have access to the context, so they can’t query any other DbSets, only the collection it’s called against. This means that only explicit relationships can be queried against, which hasn’t been a roadblock so far in my (admittedly simple) application. I’m not sure this is really a drawback though – you can still add a parameter to pass in an IEnumerable to query against, which again offers the flexibility to pass a DbSet or anything else.

Pragmatic Prioritization

The typical release scheduling process works something like this:

  1. Stakeholders build a backlog of features they’d like to see in the product eventually.
  2. The stakeholders decide among themselves the relative priority of the features in the backlog.
  3. The development team estimates the development time for each feature.
  4. The stakeholders set a target feature list and ship date based on the priorities and estimates.
The problem here is primarily in step 2; this step tends to involve a lot of discussion bordering on arguing bordering on in-fighting. Priorities are set at best based on a sense of relative importance, at worst based on emotional attachment. Business value is a vague and nebulous consideration at most.
I propose a new way of looking at feature priorities:

  1. Stakeholders build a backlog of features they’d like to see in the product eventually.
  2. The stakeholders estimate the business value of each feature in the backlog.
  3. The development team estimates the development time for each feature.
  4. The stakeholders set a target feature list and ship date based on the projected return of each feature – i.e., the estimated business value divided by the estimated development time.
This turns a subjective assessment of relative priorities into an objective estimate of business value, which is used to determine a projected return on investment for each feature. This can then be used to objectively prioritize features and schedule releases.
I’ve been using this workflow recently for one of my upcoming projects, and I feel like it’s helped me to more objectively determine feature priorities, and takes a lot of the fuzziness and hand-waving out of the equation.

Shameless self-promotion: Pragmatic prioritization is a feature of my project scheduling and estimation tool, Rogue Prognosticator

Building a Foundation

It’s been said that pharmaceutical companies produce drugs for pennies per pill – except the first pill, which costs millions. Things aren’t so different in the land of software development: the first usage of some new functionality might take hours, building the foundation and related pieces. But it could be re-used a hundred times trivially, and usually expanded or modified with little effort as well (assuming it was well-written to start with).

This is precisely what you should be aiming for: take the time to build a foundation that will turn complex tasks into trivial ones as you progress. This is the main purpose behind design concepts like the single responsibility principle, the Hollywood principle, encapsulation, DRY, and so on.

This isn’t to be confused with big upfront design; in face, it’s especially important to keep these concepts in mind in an agile process, where you’re building the architecture as you go. It can be tempting to just hack together what you need at the moment. That’s exactly what you should be doing for a prototype, but not for real development. For lasting functionality, you should assemble a foundation to support the functionality you’re adding now, and similar functionality in the future.

It can be difficult to balance this against YAGNI – you don’t want to build what you don’t need, but you want to build what you do need in such a way that it will be reusable. You want to save yourself time in the future, without wasting time now.

To achieve a perfect balance would require an extraordinary fortune teller, of course. Experience will help you get better at determining what foundation will be helpful, though. The more experience you have and the more projects you work on, the better sense you’ll have of what can be done now to help out future you.

The Importance of Logging

Add more logging. I’m serious.
Logging is what separates an impossible bug report from an easy one. Logging lets you replace comments with functionality. I’d even go so far as to say good logging separates good developers from great ones.
Try this: replace your inline comments with equivalent logging statements. Run your program and tail the log file. Suddenly, you don’t need a step wise debugger for the vast majority of situations, because you can see, in the log, exactly what the program is doing, what execution path it’s taking, where in the source where each logging statement is coming from, and where execution stopped in the event of a crash.
My general development process focuses on clean, readable, maintainable, refactorable, self-documenting code. The process is roughly like this:
  1. Block out the overall process, step by step, in comments.
  2. Any complex step (more than five or ten lines of code), replace the comment with a clearly-named method or function call, and create a stub method/function.
  3. Replace comments with equivalent logging statements.
  4. Implement functionality.
    • Give all functions, methods, classes, parameters, properties, and variables clear, concise names, so that the code ends up in some semblance of readable English.
    • Use thorough sanity checking, by means of assertions or simple if blocks. When using if blocks, include logging for any failed checks, including what was expected and what was found. These should be warnings.
    • Include logging in any error/exception handling code. These should be errors if recoverable, or fatal if not. This is all too often the only logging a developer includes!
  5. Replace inline comments with equivalent logging statements. These should be debug or info/trace level; major section starts should be higher level, while mid-process statements should be lower level.
  6. Add logging statements to the start of each method/function. These should also be debug or info/trace level. Use higher-level logging statements for higher-level procedures, and lower-level logging statements for more deeply-nested calls.
  7. For long-running or resource-intensive processes, particularly long loops, add logging statements at regular intervals to provide progress and resource utilization details.
Make good use of logging levels! Production systems should only output warnings and higher by default, but it should always be possible to enable deeper logging in order to troubleshoot any issues that arise. However, keep the defaults in mind, and ensure that any logging you have in place to catch defects will provide enough information in the production logs to at least begin an investigation.
Your logging messages should be crafted with dual purpose in mind: first, to provide useful, meaningful outputs to the log files during execution (obviously), but also to provide useful, meaningful information to a developer reading the source – i.e., the same purpose served by comments. After a short time with this method you’ll find it’s very easy to craft a message that serves both purposes well.
Good logging is especially useful in an agile environment employing fast iteration and/or continuous integration. It may not be obvious why at first, but all the advantages of good logging (self-documenting code, ease of maintenance, transparency in execution) do a lot to facilitate agile development by making code easier to work with and easier to troubleshoot.
But wait, there’s more! Good logging also makes it a lot easier for new developers to get up to speed on a project. Instead of slogging through code, developers can execute the program with full logging, and see exactly how it runs. They can then review the source code, using the logging statements as waypoints, to see exactly how the code relates to the execution.

If you need a tool for tailing log files, allow me a shameless plug: try out my free log monitor, Rogue Informant. It’s been in development for several years now, it’s stable, it’s cross-platform, and it’s completely free to use privately or commercially. It allows you to monitor multiple logs at once, filter and search logs, and float a log monitoring window on top of other applications, to make it easier to watch the log while using the program to see exactly what’s going on behind the scenes.Give it a try, and if you find any issues or have feature suggestions, feel free to let me know!

The Problem with Responsive Design

A huge problem I see with responsive/adaptive design today is that, all too often, it treats “small viewport” and “mobile” as being synonymous, when the two concepts are orthogonal. A mobile device can have a high-resolution display, just as a desktop user can have a small display, or just a small browser window.

Responsive designs need to design for viewport size, and nothing more. It’s not mobile, it’s a small display. Repeat that to yourself about a thousand times.

What’s holding back single-design philosophies isn’t display size, it’s user interface; for decades, web designers have counted on there being a mouse cursor to generate events – mouseovers, clicks, drags. That’s not how it works on touchscreen devices, and we need some facility – JavaScript checks, CSS media queries – to cater to touch-based devices as opposed to cursor-based devices.

Sanity Checks: Assumptions and Expectations

Assertions and unit tests are all well and good, but they’re too narrow-minded in my eyes. Unit tests are great for, well, testing small units of code to ensure they meet the basic requirements of a software contract – maybe a couple of typical cases, a couple of edge cases, and then additional cases as bugs arise and new test cases are created for them. No matter how many cases you create, however, you’ll never have a test case for every possible scenario.

Assertions are excellent for testing in-situ; you can ensure that unacceptable values aren’t given to or by a piece of code, even in production (though there is a performance penalty to enabling assertions in production, of course.) I think assertions are excellent, but not specific enough: any assertion that fails is automatically a fatal error, which is great, unless it’s not really a fatal error.

That’s where the concept of assumptions and expectations come in. What assertions and unit tests really do is test assumptions and expectations. A unit test says “does this code behave correctly when given this data, all assumptions considered?” An assertion says “this code assumes this thing, and will not behave correctly if it gets another, so throw an error.”

When documenting an API, it’s important to document assumptions and expectations, so users of the API know how to work with your code. Before I go any further, let me define what I mean by these very similar terms: to me, code that assumes something operates as if its assumptions are correct, and will likely fail if its assumptions turn out to be incorrect. Code that expects something operates as if its expectations are met, but will likely still operate correctly even if they aren’t. It’s not guaranteed to work, or guaranteed to fail; it’s likely to work, but someone should probably know about it and look into it.

Therein lies the rub: these are basically two types of assertions, one fatal, one not. What we need is an assertion framework that allows for warning-level assertion failures. What’s more, we need an assertion framework that is performant enough to be regularly enabled in production.

So, any code that’s happily humming along in production, that says:

Assume.that(percentage).isBetween(0,100);

will fail immediately if percentage is outside those bounds. It’s assuming that percentage is between zero or one hundred, and if it assumes wrong, it will likely fail. Since it’s always better to fail fast, any case where percentage is outside that range should trigger a fatal error – preferably even if it’s running in production.

On the other hand, code that says:

Expect.that(numRows).isLessThan(1000);

will trigger a warning if numRows is over a thousand. It expects numRows to be under a thousand; if it isn’t, it can still complete correctly, but it may take longer than normal, or use more memory than normal, or it may simply be that if it got more rows than that, something may be amiss with the query that got the rows or the dataset the rows came from originally. It’s not a critical failure, but it’s cause for investigation.

Any assumption or expectation that fails should of course be automatically and immediately reported to the development team for investigation. Naturally a failed assumption, being fatal, should take priority over a failed expectation, which is recoverable.

This not only provides greater flexibility than a simple assertion framework, it also provides more explicit self-documenting code.

Real Sprints

Agile methodologies talk about “sprints” – workloads organized into one to four week blocks. You schedule tasks for each sprint, you endeavour to complete all of it by the end of the sprint, then you look back and see how close your expectations (schedule) were to reality (what actually got done).

Wait, wait, back up. When I think of a sprint, I think short and fast. That’s what sprinting means. You can’t sprint for a month straight; you’ll die. That’s a marathon, not a sprint.

There are numerous coding competitions out there. Generally, you get around 48 hours, give or take, to build an entire, working, functional game or application. Think about that. You get two days to build a complete piece of software from scratch. Now that’s what I call sprinting.

Of course, a 48 hour push is a lot to ask for on a regular basis; sure, your application isn’t in a competition, this is the real world, and you need to get real work done on an ongoing basis. You can’t expect your developers to camp out in sleeping bags under their desks. But that doesn’t mean turning a sprint into a marathon.

The key is instilling urgency, while moderating burnout. This is entirely achievable, and can even make development more fun and engaging for the whole team.Since the term sprint has already been thoroughly corrupted, I’ll use the term “dash”. Consider this weekly schedule:

  • Monday: Demo last week’s accomplishments for stakeholders, and plan this week’s dash. This is a good week to schedule any unavoidable meetings.
  • Tuesday and Wednesday: your 48 hours to get it done and working. These are crunch days, and they will probably be pretty exhausting. These don’t need to be 18-hour days, but 10 hours wouldn’t be unreasonable. Let people get in the zone and stay there as long as they can.
  • Thursday: Refactoring and peer reviews. After a run, athletes don’t just take a seat and rest; they slow to a jog, then a walk. They stretch. The cool off slowly. Developers, as mental athletes, should do the same.
  • Friday: Testing. QA goes through the application with a fine-toothed comb. The developers are browsing the web, playing games, reading books, propping their feet up, and generally being lazy bums, with one exception: they’re available at a moment’s notice if a QA has any questions or finds any issues. Friday is a good day for your development book club to meet.
  • By the end of the week, your application should be ready again for Monday’s demo, and by Tuesday, everyone should be well-rested and ready for the next dash.
Ouch. That’s a tough sell. The developers are only going to spend two days a week implementing features? And one basically slacking off? Balderdash! Poppycock!

Think about it, though. Developers aren’t factory workers; they can’t churn out X lines of code per hour, 40 hours per week. That’s not how it works. A really talented developer might achieve 5 or 6 truly productive hours per day, but at that rate, they’ll rapidly burn out. 4 hours a day might be sustainable for longer. Now, mind you, in those four hours a day, they’ll get more done, better, with fewer defects, than an army of incompetent developers could do in a whole week. But the point stands: you can’t run your brain at maximum capacity eight hours straight, five days a week. You just can’t – not for long, anyway.

The solution is to plan to push yourself, and to plan to relax, and to keep the cycle going to maximize the effectiveness of those productive hours. It’s also crucial not to discount refactoring as not being productive; it sets up the following weeks’ work, and reduces the effort required to get the rest of the development done for the rest of the life of the application. It’s a critical investment in the future.

Spending a third of your development time on refactoring may seem excessive, and if it were that simple, I’d agree. But if you really push yourself for two days, you can get a lot done – and write a lot of code to be reviewed and refactored. In one day of refactoring, you can learn a lot, get important work done, and still start to cool off from the big dash.

That lazy Friday really lets you relax, improve your craft, and get your product ready for next week, when you get to do it all over again.