This seems to happen quite often when programmers try to save time when writing tests, instead of writing very simple tests and allowing the duplication to accumulate before removing it. I understand how they feel: they see the pattern and want to skip the boring parts.
No worries. If you skip the boring parts, then much of the time you’ll be less bored, but sometimes this will happen. If you want to avoid this, then you’ll have to accept some boredom then refactor the tests later. Maybe never, if your pattern ends up with only two or three instances. If you want to know which path is shorter before you start, then so would I. I can sometimes guess correctly. I mostly never know, because I pick one path and stick with it, so I can never compare.
This also tends to happen when the code they’re testing has painful hardwired dependencies on expensive external resources. The “bug” in the test is a symptom of the design of the production code. Yay! You learned something! Time to roll up your sleeves and start breaking things apart… assuming that you need to change it at all. Worst case, leave a warning for the next person.
If you’d like a simple rule to follow, here’s one: no branching in your tests. If you think you want a branch, then split the tests into two or more tests, then write them individually, then maybe refactor to remove the duplication. It’s not a perfect rule, but it’ll take you far…
the code they’re testing has painful hardwired dependencies on expensive external resources
I’ve told this story elsewhere, but I had a coworker who wrote an app to remote-control a baseball-throwing machine from a PDA (running WinCE). These machines cost upwards of $50K so he only very rarely had physical access to one. He loved to write tests, which did him no good when his code fired a 125 mph knuckleball a foot over a 10-year-old kid’s head. This resulted in the only occasion in my career when I had to physically restrain a client from punching a colleague.
Wow. I love that story and I’m glad nobody was hurt.
I wonder whether that happened as a result of unexpected behavior by the pitching machine or an incorrect assumption about the pitching machine in that coworker’s tests.
I find this story compelling because it illustrates the points about managing risk and the limits of testing, but it doesn’t sound like the typical story that’s obviously hyperbole and could never happen to me.
It happened because the programmer changed the API from a call that accepted integer values between 0 and 32767 (minimum and maximum wheel speeds) to one that accepted float values between 0.0 and 1.0. A very reasonable change to make, but he quick-fixed all the compiler errors that this produced by casting the passed integer parameters all through his code to float and then clamping the values between 0.0 and 1.0. The result was that formerly low-speed parameters (like 5000 and 6000, for example, which should have produced something like a 20 mph ball with topspin) were instead cast and clamped to 1.0 - maximum speed on both throwing wheels and the aforesaid 125 mph knuckleball. He rewrote his tests to check that passed params were indeed between 0.0 and 1.0, which was pointless since all input was clamped to that range anyway. And there was no way to really test for a “dangerous” throw anyway since the machine was required to be capable of this sort of thing if that’s what the coach using it wanted.
This seems to happen quite often when programmers try to save time when writing tests, instead of writing very simple tests and allowing the duplication to accumulate before removing it. I understand how they feel: they see the pattern and want to skip the boring parts.
No worries. If you skip the boring parts, then much of the time you’ll be less bored, but sometimes this will happen. If you want to avoid this, then you’ll have to accept some boredom then refactor the tests later. Maybe never, if your pattern ends up with only two or three instances. If you want to know which path is shorter before you start, then so would I. I can sometimes guess correctly. I mostly never know, because I pick one path and stick with it, so I can never compare.
This also tends to happen when the code they’re testing has painful hardwired dependencies on expensive external resources. The “bug” in the test is a symptom of the design of the production code. Yay! You learned something! Time to roll up your sleeves and start breaking things apart… assuming that you need to change it at all. Worst case, leave a warning for the next person.
If you’d like a simple rule to follow, here’s one: no branching in your tests. If you think you want a branch, then split the tests into two or more tests, then write them individually, then maybe refactor to remove the duplication. It’s not a perfect rule, but it’ll take you far…
I’ve told this story elsewhere, but I had a coworker who wrote an app to remote-control a baseball-throwing machine from a PDA (running WinCE). These machines cost upwards of $50K so he only very rarely had physical access to one. He loved to write tests, which did him no good when his code fired a 125 mph knuckleball a foot over a 10-year-old kid’s head. This resulted in the only occasion in my career when I had to physically restrain a client from punching a colleague.
Wow. I love that story and I’m glad nobody was hurt.
I wonder whether that happened as a result of unexpected behavior by the pitching machine or an incorrect assumption about the pitching machine in that coworker’s tests.
I find this story compelling because it illustrates the points about managing risk and the limits of testing, but it doesn’t sound like the typical story that’s obviously hyperbole and could never happen to me.
Thank you for sharing it.
It happened because the programmer changed the API from a call that accepted integer values between 0 and 32767 (minimum and maximum wheel speeds) to one that accepted float values between 0.0 and 1.0. A very reasonable change to make, but he quick-fixed all the compiler errors that this produced by casting the passed integer parameters all through his code to float and then clamping the values between 0.0 and 1.0. The result was that formerly low-speed parameters (like 5000 and 6000, for example, which should have produced something like a 20 mph ball with topspin) were instead cast and clamped to 1.0 - maximum speed on both throwing wheels and the aforesaid 125 mph knuckleball. He rewrote his tests to check that passed params were indeed between 0.0 and 1.0, which was pointless since all input was clamped to that range anyway. And there was no way to really test for a “dangerous” throw anyway since the machine was required to be capable of this sort of thing if that’s what the coach using it wanted.