Getting managers to understand trade-offs in testing is critical. I loved my Manager’s explanation to her test leads. Mary Beam presented the analogy of insurance.
You can buy insurance for almost anything. When an organization is buying testing, it is like buying insurance. Personally anybody could spend their entire salary on insurance.
Besides standard home, life, disability, and auto, you can get earthquake, flood, war and other perils insurance, including umbrella insurance. Maybe you also need dog bite insurance, ID theft insurance, computer insurance, kidnap ransom extortion insurance,
alien abduction insurance, etc.
Almost nobody spends all their money on insurance and neither should a company spend all its money on testing. Software testers talk about risk and so does insurance. Insurance has a long history with lots of data in most cases, e.g. actuarial life tables for life insurance. Unfortunately software testing has few calculated data, but still we must try.
So Mary implored her managers to think about which risks were truly worth the expense (of testing).
Along the same lines, I was re-reading Marick’s, Testing for Programmers and still find his “Tests Are Economic Entities” a good introduction to automation value (overestimated) and cost (underestimated). Every tester and test lead needs to be aware of this. Should I manually explore this test (one shot), manually script it, automatically explore it, or automate a script? Many tests aren’t worth running more than once. Numerous other authors and articles have discussed this.
After writing this, I searched and found a few other blog entries related to this which may be of interest: