Bug Verification Checklist
http://blog.abakas.com/2008/10/bug-verification-checklist.html
Verifying bug是QA一定會面臨的情況, 但是它不是一件容易做的事情. 套一句周星馳獎的話, 這種事情是很講天份的. 有些QA只是很單純的, 照本宣科的重新跑一次. 有些QA卻能找出RD是否真正解掉它, 是否會有side effect.
作者提出他自己的一些經驗法則
1. 確定我所run的build有包含修正的程式碼
- In particular when there are a number of branches this is something that needs double-checking.
- Rely on check-ins and build tags for this, not on bug comment time stamps.
2. 重複產生問題的步驟, 確認會產生我想要的結果
- This is the obvious part. I try the thing that broke before and see if the behavior has changed.
- If there's zero change in behavior (i.e., the exact same thing happens), I'm really suspicious - after all the fix attempt is likely to have at least modified the system behavior, even if the fix is complete.
不要認為這個步驟很簡單, 有時候當bug的reproduce 步驟很複雜時, 要能很準確重建原始狀況很不容易.
若是你當初你在找到bug時, 沒有很認真確認, 這時候你就會很痛苦.
或者像是一些hardly reproduce的case, 你更要花很多時間來確認是否真的已經被解決了
3. 確認所有的方式都能過關
- I can't prove this bug is resolved unless I can prove I exercised the thing that used to cause the bug.
我想這裡你需要知道root cause 是什麼, 否則你是無法找出所有的可能性. 因此我會強烈建議QA要去看RD所寫的解法或是root cause分析, 這不但讓你對受測系統更了解, 並且你也能再三確認這解法是否正確, 以及那些scenarios是要再加以驗證的
4. 找尋問題被解決的記號
- Often a bug fix will include a mark or a note that is a secondary way to know the bug was fixed.
- Usually this is in the form of a message that does appear in the log (XX complete) or that does not appear in the log (failure message "blah" does not appear).
- Look for the positive indicators of a fix - success message - in addition to the negative indicators of a fix - lack of prior error.
5. 重新再思考這個bug.
- Think I got it? Great.
- I'm going to read the bug one more time, start to finish, especially with a really long bug.
- Maybe the behavior morphed over time, or maybe there is a reference to another problem that sometimes hides this problem.
- Maybe there's a reference to some documentation change that should be opened as a separate request.
- Once this bug is closed out, it's unlikely to be read again, so make sure you get everything out of it that you need.
但是在時間的壓力下, 我也知道大部分的人, 都只是簡單的確認原先的scenario是否已經解掉了. 不過我想妳若是要更上一層樓, 成為不可取代的QA, 是要有所改變的.
http://blog.abakas.com/2008/10/bug-verification-checklist.html
Verifying bug是QA一定會面臨的情況, 但是它不是一件容易做的事情. 套一句周星馳獎的話, 這種事情是很講天份的. 有些QA只是很單純的, 照本宣科的重新跑一次. 有些QA卻能找出RD是否真正解掉它, 是否會有side effect.
作者提出他自己的一些經驗法則
1. 確定我所run的build有包含修正的程式碼
- In particular when there are a number of branches this is something that needs double-checking.
- Rely on check-ins and build tags for this, not on bug comment time stamps.
2. 重複產生問題的步驟, 確認會產生我想要的結果
- This is the obvious part. I try the thing that broke before and see if the behavior has changed.
- If there's zero change in behavior (i.e., the exact same thing happens), I'm really suspicious - after all the fix attempt is likely to have at least modified the system behavior, even if the fix is complete.
不要認為這個步驟很簡單, 有時候當bug的reproduce 步驟很複雜時, 要能很準確重建原始狀況很不容易.
若是你當初你在找到bug時, 沒有很認真確認, 這時候你就會很痛苦.
或者像是一些hardly reproduce的case, 你更要花很多時間來確認是否真的已經被解決了
3. 確認所有的方式都能過關
- I can't prove this bug is resolved unless I can prove I exercised the thing that used to cause the bug.
我想這裡你需要知道root cause 是什麼, 否則你是無法找出所有的可能性. 因此我會強烈建議QA要去看RD所寫的解法或是root cause分析, 這不但讓你對受測系統更了解, 並且你也能再三確認這解法是否正確, 以及那些scenarios是要再加以驗證的
4. 找尋問題被解決的記號
- Often a bug fix will include a mark or a note that is a secondary way to know the bug was fixed.
- Usually this is in the form of a message that does appear in the log (XX complete) or that does not appear in the log (failure message "blah" does not appear).
- Look for the positive indicators of a fix - success message - in addition to the negative indicators of a fix - lack of prior error.
5. 重新再思考這個bug.
- Think I got it? Great.
- I'm going to read the bug one more time, start to finish, especially with a really long bug.
- Maybe the behavior morphed over time, or maybe there is a reference to another problem that sometimes hides this problem.
- Maybe there's a reference to some documentation change that should be opened as a separate request.
- Once this bug is closed out, it's unlikely to be read again, so make sure you get everything out of it that you need.
但是在時間的壓力下, 我也知道大部分的人, 都只是簡單的確認原先的scenario是否已經解掉了. 不過我想妳若是要更上一層樓, 成為不可取代的QA, 是要有所改變的.
全站熱搜
留言列表
