Death by Design

July 16, 2006 by Isaac

One of the greatest things about testing is posting bug reports. While I'm not a sadistic software bug witch hunter, I do enjoy finding a good bug.

A few weeks ago one of the CORBA components in our system was turned over to me for testing. So, after looking over the specifications and the IDL describing the interfaces I designed some tests to exercise the range of inputs, do boundary testing, and cause the component to generate exceptions.

The first bug I found was that no matter what values I used in the method call it never threw an INVALID_VALUE exception, even though that was in the spec. Bug!

One of the particular parameters this component accepts is the time that it should take to process data. If this time is expired then the component is to go idle again. I bet you can guess the second bug; this parameter had no effect.

This was great! Within the first few minutes with this component I found two bugs. The only catch? I was flabbergasted to find that they were designed into the system. Although the IDL specifies an exception to be thrown on invalid input, this was never implemented. The explanation I got was that this component must never throw an exception and must continue to work no matter what. Ok, that's fine, but if its in the component specifications then it must be implemented or else it is a bug. Either the spec or the component must change. The explanation for the second bug was the same, it was never meant to be implemented.

Since both things were written into the component spec, but weren't implemented, they are bugs, so I wrote them up.

One of the great problems of testing is the disconnect between the specifications and what is implemented. Since documentation is official and verbal communication is not I always go with the written spec. Of course this doesn't always bode well with the project leaders or the software developers who insist they have an understanding with the customer that certain features just don't work. Being the messenger is not always fun, but someone has to do it.

Posted in Software Testing