Test Automation ROI (Part II)
Posted by Bj Rollison
Posted in I. M. Testy
1. Free up my time,
2. Re-verify baseline assessments (BVT/BAT, regression, acceptance test suites)
3. Increase test coverage (via increased breadth of data or state variability),
4. Accomplish tasks that can’t easily be achieved via manual testing approaches.
接著大家可以先去參考Brian Marick所寫的"When Should a Test Be Automated"
1. How much more will this test cost to automate versus running it manually?
2. What is the potential lifetime of this automated test?
3. Does the automated test have some probability of exposing new issues?
- 作者並不太認同這一點, 因為有些自動化測試無法找到新的bug,但是仍然很有價值.
- 所以作者建議把它改成: “Does this automated test have some probability of exposing new issues, providing additional information that increases confidence, or increases test coverage?”
1. What exactly is being evaluated?
- If the test is evaluating functional or non-functional (stress, perf, security, etc.) capabilities then automation may be worthwhile.
- But, behavioral tests such as usability tests and content testing are generally not good candidates for automated testing.
2. What is the reliability of automating this test?
- On one wants to have to constantly massage a test in order to get it to run.
3. What are the oracles for this test and can they be automated?
- If the most effective oracle is a human reviewing the results then that test should probably not be automated using current technologies.
- 如果你想要知道什麼是test oracle, 你可以參考這裡: http://en.wikipedia.org/wiki/Oracle_(testing)