小心BVT的陷阱
Escape the BVT automation trap
http://blogs.msdn.com/dustin_andrews/archive/2007/11/16/escape-the-bvt-automation-trap.aspx
這是另一個微軟的員工提出對BVT的建議. 不外乎就是要確定BVT要快, 要真的有用, 能真的檢查出這各build是否ready for testing.
1. BVT automation is a tool.
Just as a hammer is a great tool for nail, BVT automation is a great tool for a specific purpose. It’s not a catch all place to dump all the automation you have ever created for your product.
A. A good chance that testing can proceed is enough what the tool is for.
(1) The purpose of good BVTs is a quick way to tell if a particular build is worth testing further
(2) Ask yourself if this test will catch future breaks. If the answer isn’t “probably” then don’t add it to the BVT suite.
B. Fast results are money in the bank.
(1) When you build a new version of your product you want the green light as soon as possible.
(2) BVT systems that take hours to run are wasting valuable cycles
C. An automated test in the hand is worth two in the bush.
(1) You want to deliver BVT automation as early in the process as possible.
(2) When you are delivering BVTs time is the first problem you need to solve for
(3) Go for a few cheap (and yes, possibly dirty) tests you can deliver today.
2. BVT automation needs to trusted
If your team has no faith in the results they get from the BVTs you have wasted your time writing them
A. Create fast diagnostic BVTs
(1) Create BVTs with diagnostic results.
(2) A BVT that points clearly to a method in code is superior to one that can’t isolate failures.
B. Create adaptable BVTs
(1) Adapt the BVT test suite to these changes quickly
(2) If the product changes you can easily decide what tests are obsolete and need to go and what new tests need to be created.
C. Create maintainable BVTs
(1) Make your tests as maintainable as possible
3. BVT automation needs to be manageable
An automated system of tests has to have a lot of moving parts. This can make the systems difficult to manage.
Help to keep the complexity down by leveraging things you will be doing anyway.
A. Leverage unit tests
(1) If you work with developers up front and support them they will often be happy to write the tests to in whatever harness the test team is using.
(2) Developers often only create a functional test and maybe a negative test. You can improve these by adding boundary tests and tests that randomize valid inputs.
B. Leverage simple component tests
(1) My favorite is a test that checks to see if the component installed correctly
(2) Don’t worry about all the core scenarios for the component. Save that for other automation. Just look for quick high level tests that are simple.
C. everage core API functions
(1) If your product has simple to use APIs they make good BVTs.
(2) Say the product has a web services XML interface listening for HTTPS connections. A good test would be to check if it’s actually listening on the port.
留言列表
