David Ko的學習之旅

Exploratory Testing的優缺點及適用時機

from Exploratory Testing: Evolution or Revolution?, SVEN SAMBAER
http://www.ctg.com/pdf/exploratory-testing.pdf

Exploratory Testing的優點
(1) 它鼓勵創造性
(2) 它增加機會去找到新的, 未知的bugs
(3) 它允許花較多的時間去測試一些有趣或是複雜的cases
(4) 它的最大好處是使tester能在短時間發現bug, 並且對受測系統做快速的評量
(5) 它能讓你知道系統是否容易使用(如果很難用, 在短時間是不容易找出問題)
(6) 它是可變通, 有彈性的
(7) 它比scripted testing有趣, 因為它不會一成不變

Exploratory Testing的缺點
(1) 它的有限度管理性, 使它很困難被協調和調節
(2) 它只能提供有限的情報, 無法對風險，測試覆蓋，或測試的深度有深入了解
It provides only limited project intelligence, with little insight into risks, test coverage, or test depth.
(3) 它提供有限的test reusability, 以及對於bug的reproducibility也不是很好
(4) 它大量依賴tester的testing skills和domain knowledge
(5) 如果和scripted testing合併使用時, 可能會有redudant test的風險
(6) 它並不提供絕對保證, 最重要的bug一定會被發現
(7) 對於像security testing, performance testing等有特別目的的測試項目, 它並不適用.
(8) 對於需要花時間做複雜的設定, 或是要很久才能看到feedback的測試(例如要跑整夜的測試), 它並不適用.
(9) 當test object是available時, 才適合開始使用它

什麼時候適用Exploratory Testing
(1) 當有新人進來
- ET can make the learning phase an active testing and exploration experience.
(2) 需要做快速的評估
- ET offers fast insight into product quality in the short term when there is no time for test preparation.
(3) 在做scripted testing時, 發現有新東西要去了解
- The new information might suggest another test strategy that would warrant switching to an exploratory mode.
(4) 你需要去確認另外一位tester的工作狀況
- ET lets you explore the feature that he/she has tested.
(5) 當你的team裡面有tester有很豐富的domain knowledge
- Those individuals can be trusted to perform effective testing using ET.
(6) 當smoke test被需要時
- ET lets you determine the maturity and stability of the test object before progressing to large-scale test execution.
(7) 當沒有任何測試的依據時
- ET is useful when there is no documentation or other sources that can define clear, expected results for the tests.
(8) 當你在run agile project時
(9) 當你想要深入調查某個特別的bug
(10) You want to determine the status of a particular risk in order to evaluate the need for ST in that area.
(11) 當在早期, 受測產品還不穩定到可以執行scripted testing時
(12) 在beta時, 你希望customer提供early feedback, 你可以請他們做exploratory testing
(13) 你想要擴大scripted testing的多樣性
- The combination of ST and ET is appropriate for testing features with a high risk and/or priority profile.