Standard

Posted by

Posted on

2016-10-06

Posted under

development, team management

Slow is Smooth, Smooth is Fast

​I think it originally comes from the military, probably the US Navy SEALs, but it’s a popular trope in the racing world, and equally applicable to software development: “slow is smooth, smooth is fast”. It’s intentionally conflicting, but it succinctly states the idea that taking the time to do something well allows you to accomplish the end goal faster overall. 

In autocross, it’s often the run that looks or feels slow that turns out to be fast. There are a few reasons why, and the allegory to development is not hard to see. A lack of excitement or surprises may seem slow, but it’s a sign of tight control and keeping within limits. Smooth, gentle, practiced driver inputs can appear almost casual in their ease. 

On the other hand, the very active driver, who is constantly manipulating the controls and struggling to maintain control and riding the very razor’s edge of traction (or past it) appears to be fast. There’s a lot going on, a lot of action, some squealing and sliding and tire smoke. But the timer doesn’t lie: these runs are not fast. They only appear fast. 

The key is separating being busy from being productive. Not only are they not the same, they’re often in direct opposition to one another. But it may be the busy team that appears productive on the surface. 

The team that fills calendars with meetings and holds war room sessions and works late nights and fixes critical bugs at the last minute looks fast. They can’t waste time on careful thought or refactoring or automation, they’ve got features to ship! But it’s the team that works their eight hours each day with no fuss and no panic, that spends time on automating everything they can, that takes steps to detect and mitigate problems early, that thinks through problems and solutions before writing code, that will accomplish the most at the end of the week. 

Urgency is not a requirement of productivity. Remember: slow is smooth, smooth is fast. 

Standard

Posted by

Posted on

2016-05-07

Posted under

devops, golang, software craftsmanship

Go for Dev & Ops

Go (or golang, as it is often called, because “Go” is a bit vague when typed into a search engine), if you haven’t heard of it, is a relatively new language for cross-platform development. It is a modern, C-style, statically-typed language that produces platform-native binaries, but still includes memory management and garbage collection.

Go is open-source, stable (currently at version 1.6, with its first stable release dating back to 2011), and it’s not going anywhere. Written by a couple of Google’s brilliant engineers, and used internally at Google at the levels of scale and reliability you would expect from a tech giant, it is an entirely different beast from something like Ruby or NodeJS. It encourages the kind of development and deployment pipeline that we should all strive for.

Software development has been moving more and more toward interpreted, dynamically-typed, single-threaded languages like Ruby and Node. It’s been moving more toward mountains of external dependencies – I’ve seen projects where more than 90% of the code was in third-party libraries. That’s a lot of other people’s technical debt and generalized implementations to absorb into your project.

Go encourages the exact opposite. It’s statically-typed, though it offers the convenience of type inference. It’s compiled, and not just to bytecode, but to native binary. It’s innately and intuitively concurrent. Its dependency management, like everything else about it, is intentionally minimalist, implicitly discouraging overuse of external libraries and frameworks.

Go, by design, at the language level, encourages a lot of really good programming practices:

  • Because the language and standard library aim for “exactly enough and no more”, developers are more likely to follow that lead with their own code. Go is the embodiment of KISS.
  • Go makes it easy and intuitive to handle concurrency through “communication instead of sharing”, keeping code cleaner and reducing the rate of defects. It also has a built-in race condition detector that can be activated when running tests or benchmarks.
  • The core Go tool chain includes support for unit tests and benchmarks, and will even include example code from your documentation to ensure it compiles and works.
  • Go has a language-level style guide, and included in the core Go tool chain is a tool (gofmt) that reformats files to fit the standard style.
  • Go omits many facets of OOP, including inheritance (though it has mix-ins), abstract classes, method overloading, operator overloading, and multi-level access control. Though useful tools in some cases, these are most frequently used to vastly over-complicate implementations. Go really wants you to just write the simplest thing that will work.

After the code is written, Go makes it very easy to set up continuous integration, including unit tests, benchmarks, linting, and test coverage. It’s trivial to have a build go on to cross-compile for every platform and architecture you might want to deploy to.

How do you deploy a Go app? Drop the executable on a server and execute it. There is no interpreter or virtual machine to install. In fact, there are no external dependencies at all unless you create them in code. Performance is excellent and resource usage is minimal – unlike, for example, Java or Ruby. The deployment footprint is about as small as it gets, and deployments are highly reliable. You suffer none of the fragility brought on by differences between versions of Java, Ruby, Python, or NodeJS – your binary runs the same regardless of the system it’s deployed to.

I won’t say that Go is perfect, or universally applicable. There are certainly use cases it is not well-suited to. For starters, Go produces console applications, not GUIs. It is not intended for building desktop applications – though there are libraries for doing so. Go also has a focus on simplicity and minimalism, and it could get difficult to manage very large code bases in Go unless you’re very careful and very smart about it.

Where Go really shines is in producing small, self-contained, single-purpose command-line tools and microservices. The Go standard library includes a multi-threaded HTTP server perfect for building a REST API server in a hurry, while still being able to stand up to production use.

What sort of things are built in Go? Let’s see:

 

Standard

Posted by

Posted on

2015-07-31

Posted under

programming, software craftsmanship

A Good Case for Third Party Libraries

Developers have a tendency to over use third party libraries. They bring them in because they want some new technical advance that’s in the spotlight – ORM or IOC or AOP. They don’t stop to consider that they’re importing all that library’s technical debt and defects, along with a black box they won’t understand when it fails.

However, there are some excellent cases for third party libraries. Unfortunately, these are the same cases w where developers tend not to reach for an outside solution; cases where developers think, hey, this is easy, I’ve got this.

The best case I can think of for a third party library is functionality that is simple, well understood, and stable, but with a lot of edge cases. The canonical example is date and time handling.

Date and time handling is easy, right? You’ve understood clocks and calendars since grade school. You even grok Unix timestamps. You could do date and time handling in your sleep, with your hands tied behind your back – if you can manage to sleep with your hands tied behind your back in the first place, but that’s neither here nor there.

But wait – you wrote this software to use the user’s local time zone with no offset. Interoperability issues ensue. OK, no big deal, time zones are tricky but they’re not rocket science. You make a fix a move on.

Damn. Daylight savings time. No one hates daylight savings time more than developers. Another patch. But wait, some places don’t do daylight savings time. Another patch. Leap years. Forgot about leap years. Another patch. But they aren’t every four years – the year 2000, for example, was not a leap year. Another patch. Wait, leap seconds? That’s a thing? OK, another patch. What do you mean different people put the day, month, and year in a different order? Alright, alright, another patch. 24-hour time? No problem, patch. Wait, is midnight 00:00 or 24:00? What if someone inputs 24:05? What if someone increments a time by 10 minutes at five till midnight? Or at 1:55 the day daylight savings starts or ends? Or before a leap second is applied? What if a user is mobile and their time zone changes while they’re using the application?

Time and date are simple, every day parts of our lives. We take them for granted. We haven’t changed the way clocks and calendars work in any significant way in our lifetimes. But they’re absolutely rife with details and complexities and edge cases that are very difficult to enumerate off the top of your head – you’re bound to miss some of them. A library is perfect for this: because the functionality doesn’t need to change, a good library will be stable, with very infrequent updates, and few defects. Why kill yourself with patch after patch when there’s bound to be a solid solution waiting out there for you to use?

These are the sorts of cases where libraries are the right answer. Not for adding new bells and whistles, but for using an existing solution to a thoroughly solved problem. Character encoding is a good case. Handling any common file format is definitely a good case – XML, JSON, YAML, CSV. If I never see another hand-written CSV parser it’ll be too soon. Sorting – for Petes sake, if your language doesn’t have sorting built in, find a library. I don’t care if you learned to write a bubble sort in college. Don’t write one at work. Ever.

Anything even remotely related to cryptography you should absolutely solve with a library. Hashing, random number generation, and encryption are solved, hard problems. Even when security isn’t critical, you may as well lean on the work of other smart people.

Solved problems are where you want a library. Solutions in search of a problem may be more exciting, and come with more interesting acronyms and blog posts and conference keynotes, but they often create at least as many problems as they solve. Don’t get sucked in.

Standard

Posted by

Posted on

2015-06-08

Posted under

development, project management, quality, software craftsmanship

Code Reviews

“Code reviews are the worst! All the code I have to review is terrible, and people always take offense when I point out problems. And being reviewed is even worse – people always think they know better than I, can you believe it? And it’s all such a waste of time!”

It can be difficult to get a team started on code reviews, especially an established team without an established culture of reviews. Developers can be very defensive about their code, and often don’t see the value in code reviews. But the value is undeniable, and good developers will come to appreciate code reviews once they get used to them. Why?
  • Reviews reduce defects. This is the primary purpose of reviews, and they’re very effective.
  • Reviewing code builds familiarity for the reviewer. The reviewer is exposed to code they might not have worked with before, giving them a broader base of knowledge in their own development work.
  • Reviewing code improves developers ability to self-review. The more code you review, the better you get at reviewing code. The better you are at reviewing code, the better you can review your own code before you commit it. The better reviewed code is before it’s committed, the fewer defects are found in peer review, and the faster peer reviews are.
  • Expecting code to be reviewed encourages developers to self-review. Knowing that someone else is going to go over your changes after you submit them encourages you to self-review to save yourself the embarrassment, however slight, of a failed peer review.
  • Peer review improves consistency. When developers submit code without anyone else looking at it, they tend to follow their own styles and practices. As they review more of each others’ code, they will naturally tend to converge on a fairly similar set of styles and practices.
  • Peer review helps to breed a sense of collective ownership. After peer review, code is no longer “his code” or “her code” or “their code” it’s “our code”.
  • In situations where everyone is an architect – which I strongly support – peer review is even more critical. Collective design is only collective if everyone is looking over each other’s shoulders, seeing how problems are being solved, and suggesting possible alternative solutions. It really helps close the gap between a group of individuals and a true development team.
Countless tools exist for performing code reviews; the review handling built into pull requests in Atlassian’s Bitbucket and Stash is excellent, though there are many solutions, from simple things like adding a review step to your ticket workflow and putting review comments in the ticket, to using a dedicated review tool like Crucible or Reviewboard.
Standard

Posted by

Posted on

2015-06-02

Posted under

development, Uncategorized, user experience

Never Be More Precise Than Accurate

User experience design is a tricky thing, full of tiny, seemingly insignificant pitfalls that can end up causing major frustration for users. One common pitfall is being more precise than you are accurate. The typical example uses pi: 3 is accurate, but not precise, while 3.6789 is more precise, but less accurate. Accuracy in a system is controlled by a wide array of factors, but generally you’re aware of the limitations in place and you have a general idea of your accuracy. Precision you can control directly through interface design, so you should always have it match your accuracy, never exceed it. Any estimation, extrapolation, aggregation, or rounding can introduce a loss of precision. Digital floating point math is inherently imprecise.

Users naturally presume that any number they’re looking at is as accurate as it is precise. If you show four decimal places, they assume that number is accurate to four decimal places, and rightly so. If you show numbers in millions (37M), users will assume this is accurate to the nearest million. They naturally trust you to present them with accurate information, so they assume that whatever information they’re given is accurate. This is exactly why you should ensure that you don’t present information that you don’t know to be accurate.

Standard

Posted by

Posted on

2014-03-21

Posted under

development, programming

A Wrinkle in Time

You’ve built a prototype, everything is going great. All your dates and times look great, they load and store correctly, everything is spiffy. You have your buddy give it a whirl, and it works great for them too. Then you have a friend in Curaçao test it, and they complain that all the times are wrong – time zones strike again!

But, you’ve got this covered. You just add an offset to every stored date/time, so you know the origin time zone, and then you get the user’s time zone, and voila! You can correct for time zones! Everything is going great, summer turns to fall, the leaves change, the clocks change, and it all falls apart again. Now you’re storing dates in various time zones, without DST information, you’re adjusting them to the user’s time zone, trying to account for DST, trying to find a spot here or there where you forgot to account for offsets…

Don’t fall into this trap. UTC is always the answer. It is effectively time-zone-less, as it has an offset of zero and does not observe daylight savings time. It’s reliable, it’s universal, it’s always there when you need it, and you can always convert it to any time you need. Storing a date/time with time zone information is like telling someone your age by giving your birthday and today’s date – you’re dealing with additional data and additional processing with zero benefit.

When starting a project, you’re going to be better off storing all dates as UTC from the get-go; it’ll save you innumerable headaches later on. I think it is atrocious that .NET defaults to system-local time for dates; one of the few areas where I think Java has a clearly better design. .NET’s date handling in general is a mess, but simply defaulting to local time when you call DateTime.Now encourages developers to exercise bad practices; the exact opposite of the stated goals of the platform, which is to make sure that the easy thing and the correct thing are, in fact, the same thing.

On a vaguely related note, I’ve found a (in my opinion) rather elegant solution for providing localized date/time data on a website, and it’s all wrapped up in a tiny Gist for your use: https://gist.github.com/aprice/7846212

This simple jQuery script goes through elements with a data attribute providing a timestamp in UTC, and replaces the contents (which can be the formatted date in UTC, as a placeholder) with the date/time information in the user’s local time zone and localized date/time format. You don’t have to ask the user their time zone or date format.

Unfortunately it looks like most browsers don’t take into account customized date/time formatting settings; for example, on my computer, I have the date format as yyyy-mm-dd, but Chrome still renders the standard US format of mm/dd/YYYY. However, I think this is a relatively small downside, especially considering that getting around this requires allowing users to customize the date format, complete with UI and storage mechanism for doing so.

Standard

Posted by

Posted on

2014-03-13

Posted under

development, programming

On Code Comments

I’ve been seeing a lot of posts lately on code comments; it’s a debate that’s raged on for ages and will continue to do so, but for some reason it’s been popping up in my feeds more than usual the last few days. What I find odd is that all of the posts generally take on the same basic format: “on the gradient of too many to too few comments, you should aim for this balance, in this way, don’t use this type of comments, make your code self-documenting.” The reasoning is fairly consistent as well: comments get stale, or don’t add value, or may lead developers astray if they don’t accurately represent the code.

And therein lies the rub: they shouldn’t be representing the code at all. Code – clean, self-documenting code – represents itself. It doesn’t need a plain-text representative to speak on its behalf unless it’s poorly written in the first place.

It may sound like I’m simply suggesting aiming for the “fewer comments” end of the spectrum, but I’m not; there’s still an entity that may occasionally need representation in plain text: the developer. Comments are an excellent way to describe intent, which just so happens to take a lot longer to go stale, and is often the missing piece of the puzzle when trying to grok some obscure or obtuse section of code. The code is the content; the comments are the author’s footnotes, the director’s commentary.

Well-written code doesn’t need comments to say what it’s doing – which is just as well since, as so many others have pointed out, those comments are highly likely to wind up out-of-sync with what the code is actually doing. However, sometimes – not always, maybe even not often, but sometimes – code needs comments to explain why it’s doing whatever it’s doing. Sure, you’re incrementing Frobulator.Foo, and everybody is familiar with the Frobulator and everybody knows why Foo is important and anyone looking at the code can plainly see you’re trying to increment it. But why are you incrementing it? Why are you incrementing it the way you’re doing it in this case? What is the intent, separate from its execution? That’s where comments can provide value.

As a side note (no pun intended), I hope we can all agree that doc comments are a separate beast entirely here. Doc comments provide meta data that can be used by source code analyzers, prediction/suggestion/auto-completion engines, API documentation generators, and the like; they provide value through some technical mechanism and are generally intended for reading somewhere else, not for reading them in the source code itself. Because of this I consider doc comments to be a completely separate entity, that just happen to be encoded in comment syntax.

My feelings on doc comments are mixed; generally speaking, I think they’re an excellent tool and should be widely used to document any public API. However, there are few things in the world more frustrating that looking up the documentation for a method you don’t understand, only to find that the doc comments are there but blank (probably generated or templated), or are there but so out of date that they’re missing parameters or the types are wrong. This is the kind of thing that can have developers flipping desks at two in the morning when they’re trying to get something done.

Standard

Posted by

Posted on

2014-02-26

Posted under

hardware, reviews

New Laptop! ASUS ROG G750JW

I recently received the generous gift of an ASUS ROG (Republic of Gamers) G750JW laptop, and let me tell you, the thing is a beast. Seriously, it’s huge.

It’s a 17″ widescreen laptop (1920×1080 TN panel, no touch thankyouverymuch), with an extra two inches or so of chassis behind the hinge. It also weighs just short of ten pounds.

But, I wasn’t looking for an ultraportable. I wanted something that I could use around the house and on the road, primarily for software development, but also for occasional gaming. That meant I needed a comfortably-sized keyboard, trackpad, and display; that meant a 17″ laptop. I wanted decent battery life and decent performance, which meant it would be heavy for its size. And I got exactly what I asked for.

The G750JW runs a Core i7 at 3.2GHz, 12GB of RAM, an NVidia GeForce 765m, and a 750GB HDD. Step one was replacing the HDD with a 240GB Crucial M500 SSD I picked up for $135 on Amazon – less than half what I paid for a nearly identical drive just over a year ago. The difference in speed is truly staggering, going from a 5400 RPM laptop hard drive to a full-tilt SSD. It also cut a few ounces off the weight, and added a good half hour to hour of working time on the battery, so a win across the board.

I tried installing Windows 7 on it as I despise Windows 8, but kept running into an error during the “extracting files” stage of the installation. I found numerous posts online from people with the same problem, some of them with solutions, but none of those solutions worked for me; from what I can tell, it appears to be some conflict between the latest-and-greatest UEFI in the G750’s motherboard and the aging Windows 7 OS. It’s a shame, but I suppose being forced to gain more familiarity with Windows 8 isn’t all bad; I just wish I had the option to use something more, well… usable.

Other than the OS though, it’s been a joy. It performs extremely well, it has all the features and specs I need for what I’m using it for, and it’s a beast for gaming – more horsepower than I really need considering I’m not a huge gamer and gaming was not the primary purpose of the laptop to begin with. Part of its bulk comes from the two huge rear-venting fans in the thing, which do a good job of keeping it cool – something I’ve had problems with when using other laptops, and which was the ultimate bane of my wife’s old MacBook Air. I don’t think I need to worry about it overheating and locking up while playing video like the MBA did on a regular basis.

My only gripe at the moment is that it seems to be impossible to find a decent Bluetooth mouse. Sure, the market is flooded with wireless laptop mice; but 95% of them use a proprietary receiver (I’m looking at you, Logitech!) rather than native Bluetooth, which requires you to use the provided USB dongle. That seems like an utter waste considering the laptop has a built-in transceiver capable of handling mice without any USB dongle.

All I really want is a decent-sized (I have large hands) Bluetooth wireless mouse, with a clickable scroll wheel and back/forward thumb buttons. That doesn’t seem like too much to ask, but as far as I can tell, it just doesn’t exist. Thankfully the laptop has a very generous touchpad with multi-touch, and clicking both the left and right buttons together generates a middle-click. Still, I really hope Logitech gives up on the proprietary wireless idea and gets on board with the Bluetooth standard, because I’d like to have a decent mouse to use with it.

It’s telling that, on Amazon, you can find a discontinued Logitech Bluetooth mouse that meets my requirements – selling in new condition for a mere three hundred dollars. That’s three times what Logitech’s finest current proprietary wireless mouse costs, for an outdated, basic mouse. That’s how much standard Bluetooth wireless is worth to people. Wake up Logitech!

Any suggestions on a suitable mouse in the comments would be greatly appreciated…

Standard

Posted by

Posted on

2014-02-07

Posted under

c#, entity framework, optimization, programming, sql

Optimizing Entity Framework Using View-Backed Entities

I was profiling a Web application built on Entity Framework 6 and MVC 5, using the excellent Glimpse. I found that a page with three lists of five entities each was causing over a hundred query executions, eventually loading a huge object graph with hundreds of entities. I could eliminate the round trips using Include(), but that still left me loading way too much data when all I needed was aggregate/summary data.

The problem was that the aggregates I needed were complex and involved calculated properties, some of which were based on aggregates of navigation collection properties: a parent had sums of its children’s properties, which in turn had sums of their children’s properties, and in some cases parents had properties that were calculated partly based on aggregates of children’s properties. You can see how this quickly spun out of control.

My requirements were that the solution had to perform better, at returning the same data, while allowing me to use standard entity framework, code first, with migrations. My solution was to calculate this data on the server side, using entities backed by views that did the joining, grouping, and aggregation. I also found a neat trick for backward-compatible View releases:

IF NOT EXISTS (SELECT Table_Name FROM INFORMATION_SCHEMA.VIEWS WHERE Table_Name = 'MyView')
    EXEC sp_executesql N'create view [dbo].[MyView] as select test = 1'
GO
ALTER VIEW [dbo].[MyView] AS
SELECT ...

It’s effectively upsert for views – it’s safe to run whether or not the view already exists, doesn’t ever drop the view if it does exist (leaving no period where a missing view might cause an error), and it doesn’t require keeping separate create and alter scripts in sync when changes are made.

I then created the entities that would represent the views, using unit tests to ensure that the properties now calculated on the server matched expected values the same way that the original, app-calculated properties did. Creating entities backed by views is fairly straightforward; they behave just like tables, but obviously can’t be modified – I made the property setters protected to enforce this at compile time. Because my View includes an entry for every “real” entity, any query against the entity type can be cast to the View-backed type and it will pull full statistics (there is no possibility of an entity existing in the base table but not in the view).

Next I had to create a one to one association between the now bare entity type and the view type holding the aggregate statistics. The only ID I had for the view was the ID of the raw entity it was connected to. This turned out to be easier said than done – entity framework expects that, in a one to one relationship, it will be managing the ID at one end of the relationship; in my case, the ID’s at both ends were DB-generated, even though they were guaranteed to match (since the ID in the view was pulled directly from the ID in the entity table).

I ended up abandoning the one-to-one mapping idea after a couple days’ struggle, instead opting to map the statistics objects as subclasses of the real types in a table per type structure. This wound up being relatively easy to accomplish – I added a table attribute to the sub type, giving the name of the view, and it was off to the races. I went through updating references to the statistics throughout LINQ queries, views, and unit tests. The unit and integration tests proved very helpful in validating the output of the views and offering confidence in the changes.

I then ran my benchmarks again and found that pages that had required over a hundred queries to generate now used only ten to twenty, and were rendering in half to a third the time – a one to two hundred percent improvement, using views designed purely to mimic the existing functionality – I hadn’t even gone about optimizing them for performance yet!

After benchmarking, it looks even better (times are in milliseconds, min/avg/max):
EF + LINQ EF + Views
3 lists of 5 entities (3 types) 360/785/1675 60/105/675
2 lists of 6 entities (1 type) 325/790/1935 90/140/740
1 entity’s details + 1 list of 50 entities 465/975/2685 90/140/650
These tests were conducted by running Apache JMeter on my own machine against the application running on Windows Azure, across a sampling of 500 requests per page per run. That’s a phenomenal 450 to 650 percent improvement across the board on the most intensive pages in the application, and has them all responding to 100% of requests in under 1 second. The performance gap will only widen as data sets grow; using views will make the scaling much more linear.
I’m very pleased with the performance improvement I’ve gotten. Calculating fields on the app side works for prototyping, but it just can’t meet the efficiency requirements of a production application. View-backed entities came to the rescue in a big way. Give it a try!
Standard

Posted by

Posted on

2014-02-07

Posted under

networking

You’re Being Held Hostage and You May Not Even Know It

To me, net neutrality isn’t about fair business practices between businesses. That’s certainly part of it, but it’s not the crux of the issue. To me, net neutrality is about consumer protection.

Your broadband provider would like to charge companies – particularly content companies – extra in order to bring you their content. Setting aside the utterly delirious reasoning behind this for the moment, let’s think about this from the consumer’s perspective. You’re paying your ISP to provide you access to the internet – the whole thing. When you sign up for service, you’re signing up for just that: access to the internet. Period. What your ISP fails to disclose, at least in any useful detail, is how they intend to shape that access.

For your $40, $50, $60 or more each month, you might get high-speed access to some things, and not to others. You don’t get to know what ahead of time, or even after you sign up – the last thing your ISP wants is for you to be well-informed about your purchase in this regard. They’ll do whatever they can to convince you that your service is plain, simple, high-speed access to the whole internet.

Then, in negotiations behind closed doors, they’re using you as a hostage to extort money from the businesses you’re a customer of. Take Netflix as an example: you pay your ISP for internet service. Netflix also has an ISP, or several, that they pay for internet service. Those ISPs have what are called “peering arrangements” that determine who, if anyone, pays, and how much, when traffic travels between their networks on behalf of their customers. This is part and parcel of what you and Netflix pay for your service. You pay Netflix a monthly fee to receive Netflix service, which you access using your ISP. Netflix uses some part of that monthly fee to pay for their own internet service.

Your ISP has gone to Netflix and said “hey, if you want to deliver high-definition video to your customers who are also my customers, you have to pay me extra, otherwise my customers which are also your customers will receive a sub-par experience, and they might cancel their Netflix account.” They’re using you as a bargaining chip without your knowledge or consent, in order to demand money they never earned to begin with; everyone involved is already paying their fair share for their connection to the global network, and for the interconnections between parts of that global network.

To me, when a company I do business with uses me, and degrades my experience of their product, without my knowledge or consent, that’s fraud from a consumer standpoint. Whatever Netflix might think about the deal, whether Netflix is right or wrong in the matter, doesn’t enter into it; I’m paying for broadband so that I can watch Netflix movies, I’m paying for Netflix so that I can watch movies over my broadband connection, and my ISP is going behind my back and threatening to make my experience worse if Netflix doesn’t do what they want. Nobody asked me how I feel about it.

Of course, they could give full disclosure to their customers (though they never would), and it wouldn’t matter a whole lot, because your options as a broadband consumer are extremely limited; in the majority of cases, the only viable solution is cable, and when there is competition, it comes from exactly one place: the phone company. The cable companies and phone companies are alike in their use of their customers as hostages in negotiations.

What about fiber broadband? It’s a red herring – it’s provided by the phone company anyway. Calling fiber competition is like saying Coke in cans competes with Coke in bottles – it’s all Coke, and whichever one you buy, your money goes into Coke’s pocket.

What about wireless? Wireless will never, ever be able to compete with wired service, due to simple physics. The bandwidth just isn’t there, the spectrum isn’t there, there’s noise to contend with, and usage caps make wireless broadband a non-starter for many cases, especially streaming HD video. Besides, the majority of truly high-speed wireless service is provided by the phone companies anyway; see the previous paragraph.

Why aren’t they regulated? The FCC is trying, in its own way, but there’s little traction; the cable and telephone companies have the government in their collective pockets with millions of dollars of lobbying money, and We The People haven’t convinced Our Government that we care enough for them to even consider turning down that money.

In the United States, we pay many, many times what people pay in much of the developed world, and we get many, many times less for what we spend. On top of that, our ISPs are using us as bargaining chips, threatening to make our already overpriced, underpowered service even worse if the companies we actually chose in a competitive market – unlike our ISPs – don’t pay up. This is absolutely preposterous, it’s bordering on consumer fraud, and you should be angry about it. You should be angry enough to write your congressman, your senator, the president, the FCC, and your ISP (not that the last will do you much good, but it can’t hurt.)

Some excellent places to find more information: