目前分類:Exploratory Testing (14)

瀏覽方式: 標題列表 簡短摘要
    公司在五月時, 請了邰曉梅老師來台灣教授探索性測試, 那次的課程收穫良多, 推翻了不少我對探索性測試的認識. 因此, 之前也花些時間, 整理了探索性測試的相關資料.
 
 
接著我也在專案中, 讓團隊成員試試看探索性測試的做法, 幾個上過課的成員, 覺得效果不錯, 值得在專案內繼續使用, 並且也值得讓剩下的成員在明年來上這門課. 因此, 趁著剛做完回顧會議, 把一些重點給記錄下來.
 

kojenchieh 發表在 痞客邦 PIXNET 留言(0) 人氣()

在碧血劍中, 有一場劇情是這樣的, 男主角袁承志和其師兄的弟子過招, 在過程中他告訴對方華山派拳法的起手式也是能在實戰上被使用的. 小說是這樣寫: 
 
l_p1003201870   
 
袁承志道:「你以為起手式只是客套禮數,臨敵時無用的麼?要知咱們祖師爺創下這套拳來,沒一招不能克敵制勝。你瞧著。」

kojenchieh 發表在 痞客邦 PIXNET 留言(0) 人氣()

Exploratory Testing 相關資料

A. 書籍
1. Exploratory Software Testing: Tips, Tricks, Tours, and Techniques to Guide Test Design
http://www.amazon.com/Exploratory-Software-Testing-Tricks-Techniques/dp/0321636414/ref=sr_1_1?s=books&ie=UTF8&qid=1305614121&sr=1-1


kojenchieh 發表在 痞客邦 PIXNET 留言(0) 人氣()

Exploratory Testing 相關文章

最近看了James A Whitttaker所寫的Exploratory Software Testing, 覺得exploratory testing比我想像的還有趣, 因此收集了一些Exploratory testing相關文章, 之後好好地再回味一下

1. Exploratory Testing Explained
http://www.satisfice.com/articles/et-article.pdf

2. Session-Based Test Management,

kojenchieh 發表在 痞客邦 PIXNET 留言(0) 人氣()

Exploratory Testing 新書介紹


等了很久終於有人為exploratory testing 出書了, 作者是很有名的專家, 在microsoft有很崇高的地位, 不過在2009時加入了Google, 看起來Google應該也是很重視測試才會把他挖角過去吧!!

Exploratory Software Testing: Tips, Tricks, Tours, and Techniques to Guide Test Design
by James A. Whittaker 
http://www.amazon.com/exec/obidos/tg/detail/-/0321636414/ref=ord_cart_shr?%5Fencoding=UTF8&m=ATVPDKIKX0DER&v=glance

kojenchieh 發表在 痞客邦 PIXNET 留言(0) 人氣()

從管理階層的觀點來看Scripted和Exploratory testing
 
Scripted vs Exploratory testing from a managerial perspective
July 30, 2009
http://thetesteye.com/blog/2009/07/scripted-vs-exploratory-testing-from-a-managerial-perspective/
Posted by Martin Jansson

kojenchieh 發表在 痞客邦 PIXNET 留言(2) 人氣()

Exploratory Testing
http://insidesqa.blogspot.com/2009/02/exploratory-testing.html

2009 Feb 17
Pubished in Inside QA

ET(Exploratory Testing) 在agile中常常被人拿出來使用, 但是即使在非agile的狀況下, 作者也認為ET也是非常適用的. 它能幫助你找到一些你沒想到的問題.

在Agile的project中, 正常來說要執行, unit testing, 用FIT來做integration testing, 要和CI(continuous integration) 整合, 要做acceptance testing等等, 如果有正確被執行, 相信品質會有一定的水準.

但是事實上, 事情通常沒有這麼順利, 可能是那些事情沒有做, 或是即使有做, 但是那些測試項目範圍或是深度不夠.  因此tester通常需要適時執行一些ET, 來確保軟體的品質.

以下是作者列出的checking list, 來確認ET要使用的時機
1. Always use in conjunction with planned tests on high impact stories. Cover as much as you can!
2. Use when trying to reproduce system failure.
3. Use when defect clusters have been identified. This will flush out even more defects.
4. Always use when you have a good technical understanding of the system architecture. You will already be aware of what usually breaks certain systems.

當你在執行ET時, 作者建議必須要小心以下事情
1. Demonstrate a plan of action. Even a quick outline of what you aim to achieve by carrying out certain actions will give confidence in your actions.
2. Write down all tests that are performed. I use a test short hand that describes navigation/action/result in just one sentence. This enables you to create more tests further down the line.
3. Let the system risk analysis guide you to critical areas of the application. This is where exploratory testing pays off.
4. Sit near to, or with the development team to enable quick solutions to problems and questions.
5. Never rely on just doing exploratory testing.

記住: 完全沒有計畫的隨機測試, 不是exploratory Testing, 而是糟糕的測試.

kojenchieh 發表在 痞客邦 PIXNET 留言(0) 人氣()

為什麼需要做Exploratory Testing

explaining exploratory testing
http://blogs.msdn.com/james_whittaker/archive/2009/01/08/explaining-exploratory-testing.aspx

January 08, 2009
Posted
by James Whittaker
Published in JW on Test

公司最近有介紹Exploratory Testing 的sharing, 在會議中發現大家幾乎都不知道這是什麼, 因此希望這篇文章可以多幫忙了解一點為何要做Exploratory Testing


軟體測試之所以複雜, 是在於它有過多可能的組合. (在我認為是接近無限多):
- inputs
- code paths to state
- operational environment.
這三種的情況, 前兩種的數量接近無限多, 第三種是很多, 所以combine起來後, 狀況幾乎是無法想像的多.

所以不管你採取什麼方法, Scripted Testing (也就是事先規劃) 或是 Exploratory Testing (一邊學, 一邊規劃, 和一邊測試), 要完整測試是不可能的事

然而Exploratory Testing的好處是, 讓QA能在測試過程中, 根據他所學到, 所收集到的, 然後來決定接下來要整麼調整, 怎麼進行接下來的測試. 而不是只是follow up之前規劃好的test cases 或是test plan.

這是最主要比Scripted Testing(泛指事先規劃的方法)佔優勢的地方.

你可以想像, 如果你若是要在季賽前, 要預測 Super Bowl (美國橄欖球超級杯大賽) or Premier League(英超聯賽)的贏家, 你會不會認為是一件非常難的事情.

因為你需要觀察team如何來運作, 進攻; 或是主力球員是否整季都健康, 沒有受傷; 或是教練和球員整合程度. 這些都是變因, 這些資訊會隨著賽季的展開, 你才能逐步調整你的預測, 然後會對的機會, 才會越來越高.

同理, 再軟體測試上也是一樣, Exploratory Testing也是在過程中, 一方面根據之前的歷史資料, 所學會的domain knowledge, 再加上目前狀況的需求, 不斷再調整所要進攻的方向, 已確定你所要測試的重點

所以有效的使用Exploratory Testing, 可以幫助你在面對軟體測試複雜的狀況, 能夠更有信心去deliver quality software.

kojenchieh 發表在 痞客邦 PIXNET 留言(0) 人氣()

Exploratory Testing常見的Q&A

Exploratory Testing challenged - Part I
http://shrinik.blogspot.com/2007/12/exploratory-testing-challenged-part-i.html

2007 Dec 29
Published in Thinking Tester

1. Exploratory Testing (ET)的好處是什麼?
Ans:
ET will extend the test coverage over traditional scripted testing, you will be able discover those bugs that are not “catchable” by scripted tests.

2. 為什麼Scripted Testing無法找到所有bugs?
Ans:
- Our scripts are based on specifications which are one narrow, fallible source of information about “intended software behavior”.
- Since specifications are written in English – there could interpretations/misinterpretations. Since our  specifications are fallible, so are our scripts.
- There is a human limitation to understand and interpret specifications (objectively) and design the test cases that cover the entire test space.
- So, there is good possibility that scripts will not find all bugs that potentially be discovered.

3. 為什麼test cases為什麼被執行多次後, 越來越沒有用處呢?
Ans:
- The scripted approach to testing attempts to mechanize the test process by taking test ideas out of a test designer's head and putting them on paper.
- Test cases wear out as they do not have any in built mechanism in them to alter themselves to changing product environment.
- Test scripts can not think, infer, improvise, get frustrated as intelligent human testers do. Hence test scripts can not find bugs that repeatedly than a human tester.
- But exploratory testers take the view that writing down test scripts and following them tends to disrupt the intellectual processes that make testers able to find important problems quickly.

4. 那Ad Hoc Testing和Exploratory Testing是相同的東西嗎?
Ans:
- Yes as Adhoc testing is well known predecessor to ET. Cem Kaner coined this term ET around early 80’s to distinguish ET and Adhoc Testing.
- Cem thought that there were lots of confusions regarding some kind of “impromptu” testing that does not rely on predefined scripts.
- Ad hoc testing normally refers to a process of improvised, impromptu bug searching. By definition, anyone can do ad hoc testing.
- The term "exploratory testing"--coined by Cem Kaner, in “Testing Computer Software” -- refers to ET as a sophisticated, thoughtful approach to ad hoc testing.

5. 聽說 ET要求要有經驗, 並且Domain knowledge要高. 那average tester可以做好ET嗎?
Ans:
- I believe, ET requires skills like “questioning”, “modeling”, “critical thinking” among others.
- Domain knowledge certainly helps in ET but I do not consider it as mandatory.

6. 那ET可以找到什麼種類的bug?
Ans:
- It depends upon what kind of bugs you want to discover.
- ET can be performed in controlled, small time boxed sessions with specific charters to explore a specific feature of the application.
- ET can be configured to cater to specific investigative missions.
- You could use few ET sessions to develop a software product documentation or to analyse and isolate performance test results.

7.
當你有5000 test scripts, 你執行1到2的cycles, 你就從目前的test script再也找不出錯誤了. 你會不會覺得這系統已經沒有問題了?
或者你找到的bug都不是從這些test scripts中發現的, 你會不會覺得這test script有問題?
如果你還要在執行5 個test cycles, 你會全跑這些test script或是怎麼做呢?
Ans:
Shrini沒有提到答案, 但是已經暗示我們scripted testing是不夠的. 或許你的Testing Manager或是其他的Manager 無法被說服去採去ET, 但是你自己內心深處, 一定知道scripted testing無法解決所有問題

這時候大師出馬了, Cem Kaner 說
I rarely try to persuade a test manager to switch to exploratory testing.
Instead, I recommend to them that they include some exploratory testing in their regular testing process.
Over time, a thoughtful manager will find a context- appropriate balance between scripted and exploratory tests.

這裡他提出一個經驗法則, 告訴你要如何混搭ET 和ST
In every build, even the last build:
- spend 25% of your time trying new things. Some of these will find bugs. Some will give you good ideas for reusable tests.
- spend 25% of your time implementing new tests as reusable tests.
- spend 50% of your time running old tests (including bug regressions).

這個經驗法則有個問題: 花太多時間在做test execution, learning的時間不夠多
 -Learning is a very big part of exploration, and that doesn't come just from running tests and seeing results.
- It comes from finding information about the market, the platform, the customer expectations, the risks for similar products, etc.
- This learning guides initial test design (the 25% for new things) and the design of the reusable tests.
- It also provides important guidance for writing bug reports (helping you decide, and explain) whether a bug is serious and what customer reactions might be.

不過對於一般的manager來說, 這個經驗法則的好處是: it is very easy for many managers to understand and try.

壞處則是:
- It hides the non-test-execution research that competent explorers must do -- is a serious problem.
- If you are the consultant, then as your client gains experience with exploratory work, you can raise higher level cognitive issues and the value of preparatory research gradually, as examples present themselves in the situation of your client.

kojenchieh 發表在 痞客邦 PIXNET 留言(0) 人氣()

如何讓 Exploratory Testing是可以管理的


雖然ET並不等於unstructed或是unplanned, 但是對於test manager來說, 它還是不好管理和控制. 因此有人就發展出一套對ET的管理方法, 讓你可以掌握ET的狀況, 讓相關的stakeholder可以清楚了解我們在做什麼.

這個流程是由以下幾各單元所組成的

kojenchieh 發表在 痞客邦 PIXNET 留言(0) 人氣()

Exploratory Testing是否只適用於senior testers的身上?

Is exploratory testing only for senior testers?
September 16th, 2008 in SearchSoftwareQuality
http://www.michaeldkelly.com/blog/archives/190

上一篇"Exploratory Testing的優缺點及適用時機"
http://www.wretch.cc/blog/kojenchieh/13053436#comment249421680

kojenchieh 發表在 痞客邦 PIXNET 留言(0) 人氣()

Exploratory Testing的優缺點及適用時機

from Exploratory Testing: Evolution or Revolution?, SVEN SAMBAER
http://www.ctg.com/pdf/exploratory-testing.pdf

Exploratory Testing的優點
(1) 它鼓勵創造性
(2) 它增加機會去找到新的, 未知的bugs

kojenchieh 發表在 痞客邦 PIXNET 留言(4) 人氣()

Where Does Exploratory Testing Fit?
by James Bach
http://www.satisfice.com/articles/where_fits.shtml

1. Exploratory Testing(ET)的定義
(1)concurrent test design and test execution.
(2) Any testing is exploratory to the extent that

kojenchieh 發表在 痞客邦 PIXNET 留言(0) 人氣()

What is Exploratory Testing?
by James Bach
http://www.satisfice.com/articles/what_is_et.shtml

Exploratory testing (ET) 是agile testing中最有名的方法, 可是也是大家最陌生的方法. 因為很多人很容易把它和Ad Hoc testing想在一起, 而對它有錯誤的誤解. 所以這篇文章企圖給大家一些基本介紹, 讓你了解ET到底是什麼

1. ET和scripted testing的定義

Exploratory testing

kojenchieh 發表在 痞客邦 PIXNET 留言(0) 人氣()

找更多相關文章與討論

您尚未登入,將以訪客身份留言。亦可以上方服務帳號登入留言

請輸入暱稱 ( 最多顯示 6 個中文字元 )

請輸入標題 ( 最多顯示 9 個中文字元 )

請輸入內容 ( 最多 140 個中文字元 )

請輸入左方認證碼:

看不懂,換張圖

請輸入驗證碼