什麼是ET, ST, 和Test Automation正確的組合?
My theory of software testing - I
http://xndev.blogspot.com/2007/12/my-theory-of-software-testing-i.html
常常有人問作者:
What's the right mix of exploratory testing, "planned" manual testing, and test automation?
作者認為這個答案是"看狀況", 要看你遇到的問題種類是什麼.
那什麼時候適合做Test Automation呢?
1) The build verification test, or BVT.
- These are tests you run immediately after every build to make sure the software isn't completely hosed.
- You add more tests over time, but the point is - if these tests fail, it was probably one big massive regression error, any additional testing information is suspect.
- BVT tests are incredibly boring, and often worth automating.
2) The Fahrenheit-To-Celsius Conversion test.
- Sure, you could have a human being test a hundred examples manually, but if you have a spreadsheet that you can use as a source of Truth, why not write a program that loops through all values from -10,000 to 10,000, by increments of 0.001, calls the function, does the math in the spreadsheet, and compares the two?
(也就說有需要大量計算的時候, 便很適合使用test program)
- Note that this does not provide information about Boundaries, which may be best explored - but it can help you build some confidence about the happy middle.
3) Model-Based Testing.
- Similar to #2, if the application is simple enough you can create a model of how the software should behave, then write tests to take "random walks" through the software.
- Despite the appeal of Model-Based Testing, it requires someone with true, deep testing expertise, true development expertise, modeling skill, and, usually, a gui-poking tool.
- You can get a lot of informaiton here.
http://www.geocities.com/model_based_testing/
4) Unit Testing.
- This is as simple as "low-level code to test low-level code" - often much further down in the bowels than a manual tester would want to go, it provides the kind of "safety net" test automation suite makes a developer comfortable refactoring code.
- And devs need to do this; otherwise, maintenance changes are just sort of hacked onto the end, and, five years later, you've got a big ball of mud.
5) Isolation Problems.
- If you have a system that requires a stub or simulator to test (embedded system), you may need to want/need to write automated tests for it.
6) Macros.
- Any time you want to do something multiple times in a row to save yourself typing, you may want to do automation.
那什麼時候我不該做Test Automation?
1) To find boundaries.
- As I hinted at above, automated system-level tests are usually pretty bad at finding bounds.
- So if you've got a lot of boundary errors, by all means, test manually.
2) As Exploratory Testing.
- Sometimes when we test our goal is investigation; we play "20 questions" with the software. We learn about the software as we go.
- This kind of "exploratory testing" can be extremely effective; much more effective than mine-field automated testing.
3) When we don't need to repeat the tests.
- Some things only need to be tested once - Tab Order.
- Or may be best tested by a human, because they require some kind of complex analysis.
所以人和程式的差別是什麼呢?
- Human beings can be sapient; they can think.
- Humans can look at a screen and say "Something's not right ..." where a bitmap compare would simply fail or pass.
- Humans can look at a screen and intuit the most likely parts to fail.
因此對於slow, boring, dumb tasks, 這些會是你automation的重點tasks.
留言列表
