Genuinely surprised when software fails?


As last night’s
SASQAG meeting, James
Whittaker
again presented his Future of Testing talk (GTAC
version
still not on YouTube).

A statement near
the end piqued me. 

            "Users will be genuinely surprised
when it fails."

I think James is
tainted by his past experiences.  There are past and current companies that
already work like that.  My first job out of college at Tandem Computers warped
my expectations forever.   As a fault tolerant computer company, not failing
(hardware or software) was important.   In my first week there I
found a trivial bug in their editor with a recursive macro called that crashed. 
I was told to report the bug and get it fixed.  That was the attitude in
general, all bugs reported and generally all bugs
fixed.  There was of course triage and minor bugs deferred until next release,
but I never saw bug tracking systems for tens of thousands of bugs until I came
to Microsoft.   Long ago Tandem saw the criticality of avoiding data corruption
and a fail fast philosophy.  Code reviews of the kernel were keen on what could
go wrong.  Many others from that crucible ended up at Microsoft with an
expectation for quality in their soul.  Many worked on Microsoft SQL with an
expectation for outstanding quality.

The Agile
community with their “Test Driven Development” and automated regression runs
every check in help instill a quality expectation in developers.  But many
already had it.  I have my Software Quality Engineering big red buttons from the
mid 1980’s – “TEST then CODE”.  My favorite was a developer who bet me a dinner
I couldn’t find a bug in his code.   Why the challenge?  Mutual respect and
improvement.   I didn’t take the challenge.  Why?  The developer was good; they
had done the right things.  I was not currently on their project (but had been
months previously).    I could probably find a bug, but it would probably cost
me my entire weekend and that wasn’t worth a dinner to me.   But the real point
is the proper confidence of the developer.  Not arrogance or bravado.  He knew
if I worked hard enough I would find a bug and he would be happy to receive it
and improve his product.  To him, a dinner was a small price to pay for a better
product.

 

My expectation is
always that when I am asked to test a developer’s work product it is because they
believe it is ready to ship to the best of their ability.  They’ve code reviewed
(peer review, pair programming, etc.) it and unit tested and run the standard
tests available to them.  The only bugs I should be finding are those requiring
resources outside the scope of a single developer, such as large scale stress
testing or full system (integration) testing.

 


Advertisements

About testmuse

Software Test Architect in distributed computing.
This entry was posted in software testing. Bookmark the permalink.

One Response to Genuinely surprised when software fails?

  1. Unknown says:

    Love the button !! Why do you think quality has declined since then – systems too complex, too much emphasis on getting sytems out ?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s