要將Test Automation外包的十誡
10 Commandments for Test Automation Outsourcing
http://shrinik.blogspot.com/2008/09/10-commandments-for-automation.html
當你要將test automation外包時, 要記得考慮下面十件事情:
1. Test Automation不能解決你測試上遇到的難題
- 像是測試時間不夠, 受測程式的品質不好等等這些問題, 是無法靠它解決的. 唯一能做的還是要改進你的manual testing, 而不是去想要加強test automation
- 若是你的manual testing做的不好, 花時間在test automation只是讓你把這些poor的testing, 比較快做完而已.
2. Test Automation需要精心設計, 但是有可能結果是無用的
只有在精心設計和規劃下, test automation才會發會它的功效. 所以test automation不是萬能的系統, 當你的受測系統有改變, 你的test automation系統通常也要更新. 另外還需要加上tool的費用, training 的費用, 以及test case 和project管理的費用. 你的供應商有充分地告訴你這方面的事情嗎? 還是他低估這方面的事情? 要小心哦!!
3. 任何可以快速產生的東西, 也通常容易快速腐敗
- Do not believe claims of “script creation in minutes”, “Automated script generation” etc.
- Any enterprise level serious automation is software development and requires clear attention and methodical approach to design/create and maintain.
4. 你的Your business users是無法建立和維護Automation solutions
- Let your business users/SME’s and domain experts do what they are best at – business support.
- Do not believe claims of business users creating and maintaining automation solutions.
- It requires good amount of testing and automation knowledge.
5. 藉由Vendor詢問有關automation的問題來判斷它的成熟度
建議在提出automation的solution時, 必須要參考專案的範圍, release cycle,專案目前狀態, business expectations, 和現有automation practice的狀態
如果Vendor都沒有詢問這些事情, 就只想提出所謂的"standard", "gold plated"的automation solution賣給你, 請你可能要好好在考慮一下
6. 較高的外包代表需要較高的投資
- While moving work to offshore will give cost benefits, when it comes to automation there are number of factors to be considered while deciding how much of automation can happen at offshore.
- Key thing is about creation of application environment at offshore.
- If a local application environment can be created at offshore with tool licenses – higher degree of automation work can happen at offshore.
- It is important to note that for GUI centric/COTS automation tool based automation, both automation tool and application should reside on same machine.
- Hence pay attention to your application environment and feasibility of creating local environment at vendors offshore location before deciding the amount of work that can move to offshore.
- Even when such local environment is available at offshore, certain activities like acceptance testing, demo to users, and certain types of test cases will need to be done only at onsite.
- Be sensitive to these factors.
- Make sure your vendor asks about this and suggests suitable alternatives.
7. Pay attention to dependencies and Quality of current test Artifacts
- A successful outsourced automation requires that project dependencies are well understood by all stakeholders.
- Access of application from offshore, access to testers/developers/SMEs on test cases, Access to reviewers and code acceptance people are few dependencies that one needs to track as part of project.
- If automation is on the basis of existing manual test cases- make sure that these are detailed enough and available in a form that can be sent across to vendor offshore team.
8. 決定驗收的標準
- Formulation of Acceptance criteria is not item that is often given least importance while planning outsourced automation projects.
- Identify an internal owner in your organization who will accept the code/solution delivered by the vendor.
- Make sure that this person gets engaged in the project from the beginning and formulates the acceptance criteria along with vendor technical team.
- Failure to identify automation acceptance person from your end and getting a formal agreement on acceptance criteria can leave you “High and Dry” and leaves an open space for the vendor to deliver any “working” automation code but not necessarily the one that stays for long time.
9. 避免automation project的成敗和受測軟體的release date綁在一起
Automation tool一出來並不能馬上保證可以讓manual testing的時間減短, 並且受測軟體就可以快速release.
若是Vendor號稱可以將90% 的manual test cases 給自動化, 所以你只需要原先manual testing時間的10%就好, 你的專案就可以縮短很多時間.
你會相信你vendor所說的, 然後就縮短測試時間和project的schedule嗎? 若是automated program 延遲怎麼辦? 如果受測軟體改變很多, automated program也能有一致的測試結果嗎? 如果受測軟體開發延遲時, automated program也能快速反應及準時交貨嗎?
10. 僅提供Record and Playback (RP)機制的Automatio tool, 只是玩具而已
- This one point can never be overemphasized.
很多IT的manager認為RP的功能非常好用和簡單, 可以讓automation省下大讓的時間. 並且之後maintain這些scripts也是沒有太多問題
但是根據經驗顯示, PR在sustainable上需要花很大的effort, 如果vendor把這個工具列為他們solution重要的一環, 你可能要開始擔心
留言列表
