回歸測試的作法
The Minefield Myth (Part 2) – The value of regression testing
http://blogs.msdn.com/imtesty/archive/2009/01/30/the-minefield-myth-part-2-the-value-of-regression-testing.aspx
January 30, 2009
Posted by Bj Rollison
Published in I. M. Testy
作者和朋友討論到regression testing, 很多人認為regression testing 幫助不大, 無法找到一些有用的bug出來.
但是作者認為regression testing是有價值的, 不過在以下這樣的狀況下,regression testing的效果確實有限
retesting scripted, or simple procedural programs
retesting programs that are relatively static (unchanging),
retesting programs with no internal or external dependencies
rapid testing approaches where the test strategy is to simply sample the program behavior in order to provide a quick assessment and find some bugs
也就是說單純只是重新做以前人做的事情, 那自然不會有什麼新的價值產生.
但是一個有效率的regression test strategy應該不是這樣的, 它不但要能幫助確定重要的,潛在的不穩定的功能能運作正常, 必且也可以提高整體信心的產品, 特別是在關鍵的地方
那要怎麼做到呢? 作者認為至少有以下的錯誤觀念需要被修正:
1. 所以的測試個案都是相等, 都需要被放入regreion test中
- Not all tests are equal, and not all tests need to be re-executed repeatedly (every build) throughout the development lifecycle.
- When redundant test cases or tests that dont provide great value to the overall testing effort are
* slovenly added to the regression test suite to simply increase code coverage or to artificially inflate the number of tests for 'feel-good’generally only results in an overloaded test suite that rapidly grows out of control and becomes a management nightmare.
2. Unit tests已經足夠去確保程式碼的品質
- It is simply a fool-hearty assumption and idealistic.
- The primary purpose of unit testing is to verify a method or function does what it is supposed to do, and may include ancillary negative tests as well.
- However, unit tests generally do not include
* comprehensive data coverage
* data permutations
* combinatorial analysis of parameters
- If unit testing were the answer for effectively evaluating code churn I suspect
* a lot of testers would be out of a job and
* testing would simply be a rudimentary process of behavioral-type acceptance testing performed by non-technical 'end-user-like' individuals.
3. 回歸測試不太有價值, 因為他們不能有效識別或揭露'新'的錯誤
- This is a myopic view of testing that focuses on the current version of the product, and on 'bug-finding' as the main purpose of testing.
- Value is very subjective, and industry reports suggest that less than 15% of the bugs reported during a product development lifecycle are detected via regression testing.
- But, if even half that number are critical issues that would cause the company to pay a lot of money to release a patch then regression testing is most probably worth the time and effort.
- An effective suite of well-designed regression test cases will not only help provide a baseline assessment of the current version,
* but a significant portion of the test suite will/should be reused during maintenance or sustained engineering of that product for upwards of 10 years,
* and some subset of those tests in that suite will also be reused on the next release or version of the product.
一些有效的regression testing strategies
Regression Test基本上是一組測試案例提供了一個基本的衡量, 以確保預期或重要的功能運作正常, 並驗證以前的錯誤沒有再次發生.
根據這些說法，測試案例必須是經過精心挑選, 以防止超多的測試個案, 或rerunning不重要測試個案.
我通常建議以下四種類型的測試案例, 需列入regression test中：
Test cases designed to verify critical functional attributes and capabilities of the software program
Test cases designed to validate baseline functionality/behavior specified in project requirements or user acceptance criteria
Test cases designed to verify functional anomalies that are found and fixed during the software development lifecycle
Test cases designed to evaluate collateral or dependent areas associated with a code fix
但是這樣應該很是很多case要跑, 這時候可以再根據test cases和feature/component 的關係來分類.
然後先執行通過修改部分的測試個案, 其次是執行經過和被修改部分相關聯的測試個案, 最後是其餘的測試個案.
最後盡可能的自動化, 這樣才會比較有效率.
留言列表
