Scenario vs Functional/Component testing


I’ve not done the research to support this, but my gut feel has always been that scenario testing (especially when done with models) is an initial big win over simple feature testing.  I drew some graphs for a colleague to attempt to illustrate the point.
 
Scenario testing is only cost effective when you don’t find too many bugs.  There is a cost to diagnosing and reproducing bugs which is higher than fine grained unit or component tests.   However, more bugs can be found faster with fewer tests.   So if there aren’t too many bugs, the savings in test writing will far outweigh the slightly increased cost of reporting bugs.
 

Because it takes a lot of little tests to cover a product, it takes time to create and automate all those functional or component tests.
 
Scenario tests cover more of the product faster at the beginning providing better
  • High quality tests.
  • Focus on improving test coverage. Improving test coverage doesn’t mean blindly adding test cases but adding tests with clear goals in mind and being aware of what is being tested and how we’re testing.

A problem with Scenario tests, in my mind, it they are not cost effective for reaching the highest levels of code coverage (whether that should even be a goal, is another separate, big, debate). 
 
But a combination strategy can leverage each techniques strengths for optimal testing.
Advertisements

About testmuse

Software Test Architect in distributed computing.
This entry was posted in software testing. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s