Info buying instead of bug buying; Simple MBT; conferences

Random musings:

Testing is getting paid by the bug?

“a unique pay-per-performance business model, to provide our customers with a cost effective solution to test any product,”
            pay you based on your approved bugs submitted.”

Attended Risk Based Testing talk by Paul Gerrard as part of MS internal Engineering Excellence and Trustworthy Computer Forum (EE&TwC – as the invited speaker Scott Berkun  said – this contorted name shows a turf fight between groups that compromised on “&” rather than a useful, catchy phrase or acronym for people to remember.  The EE&TwC theme was design!).

Most interesting thing to me was in a list of Risk Reduction Strategies:          Information Buying
What is “Information Buying”?     It’s testing!
What a great way to really hit home about what testing is really about.
We test to gain information about a system to reduce risk.  You pay for the testing to get that information.

Note that paying only for bugs, means you have no idea what parts of your system are good.  Does no bugs mean it is good, or just that nobody tried it?

The other nugget from Gerrard was we should measure not by deliverables, was the code checked in, but by test evidence.   What was the pass rate, coverage, etc. of the checked in code.    If you have test evidence, you can infer the item being tested has been delivered.

I just gave a basic Model Based Testing (MBT) introductory talk to the VSTS (Visual Studio Team System) Test SIG (Special Interest Group).  Always amazed how few people have heard of MBT before.  I was asked how big a model you need to discover bugs.  The questioner had seen his own simple example of just 2 states.  I related the 2 state model I saw find bugs.  It was used by a tester on Indigo we had just taught MBT.  They were testing the .Net asynchronous calling pattern for several APIs.  Their simple model was:


This has just 2 states (invoked or not) and 2 actions (Begin or End).   Each action can conceptually occur from each state (4 transitions).   The ones in red should result in an error without changing state.    The tester then ran this simple model over about 50 APIs and found about 1/3 of them failed doing NotInvoked – EndInvoke ->

You can also see my article on Testing For Exceptions for why a sequence of more than one transitions might be useful.

I’ll be moderating a panel about Collaborative Quality at PNSQC and presenting Thursday, 9:45 “Patterns and Practices for Model-Based Testing” at StarWest.   Maybe I’ll see you at ISSTA in Seattle?

More conferences past and present about the MBT work of my Protocols Team.

Wolfgang Grieskamp : ETAPS MBT 2008Using Model-Based Testing for Quality Assurance of Protocol Documentation
Wolfgang Grieskamp : ICST 2008 Model-Based Quality Assurance of Windows Protocol Documentation
Nico Kicillof : QSIC 2008Model-Based Quality Assurance of the SMB2 Protocol Documentation


About testmuse

Software Test Architect in distributed computing.
This entry was posted in software testing. Bookmark the permalink.

One Response to Info buying instead of bug buying; Simple MBT; conferences

  1. Unknown says:

    Hi Keith
    If we refer to the following links, apparently utest have a "Maturity Indicator" to give the information about the application’s readiness.
    Inder P Singh

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s