The Role of Testers in XP, Cem Kaner, August 13, 2003
這篇文章介紹tester在XP中扮演怎樣的角色, 心態上要做怎樣的修正
首先, 一開始提出了傳統testing 的觀念是什麼
接著, 再提到在XP 中的testing 是在強調什麼
最後才提到QA的心態要做什麼調整.
若是你要pilot run XP, 這是一偏值得參考的文章
1. Testing: The Traditional View
(1) The purpose of testing is to find bugs.
(2) The test group should work independently of the programming group.
(3) There is a conflict of interest between the testing group and the programmers or project manager
(4) testing group’s task is to get the software "right“
(5) the programmers’ task is to get the software to market
(6) The test group should have veto power over the release of a product to the customer.
(7) Tests are designed without knowledge of the underlying code.
(8) Automated tests are developed at the user interface level, by non-programmers.
(9) Tests are designed early in development.
(10) Tests are designed to be reused time and time again, as regression tests
(11) To the maximum extent possible, tests should be automated.
(12) Programmers should do extensive unit testing, but they probably won't,
- so testers should assume that the programmers did a light job of testing and
- so should extensively cover the basics
(13) The pool of tests should cover every line and branch in the program, or perhaps every basis path
(14) Manual tests should be documented in great procedural detail so that they can be handed down to less experienced or less skilled testers, who will
- repeat the tests consistently, in the way they were intended,
- learn about test design from executing these tests, and
- learn the program from testing it, using these tests.
(15) A test design must include the expected result for the test. A test without an expected result is not a test.
(16) There should be at least one thoroughly documented test for every requirement item or specification item
Test cases should be based on documented characteristics of the program,
Test cases should be individually documented and, ideally, stored in a test case management system that describes the pro-conditions, procedural details, post-conditions, and basis (such as trace to requirements) of each individual test case
Failures should be reported into a bug tracking system
A count of the number of defects missed, or a ratio of defects missed to defects found, is a good measure of the effectiveness of the test group.
2. The Practice about Glass Box Testing in XP
(1) Unit-level change detectors.
a. These are unit tests, preferably written test-first, to help the programmer envision the code more clearly.
b. The objective of these tests is not to hunt bugs. It is to facilitate programming
- Improve reliability, through clarifying detail-level requirements the implementation must meet
- Support refactoring
- Help subsequent code readers understand the intent and detail of the code
(2) Integration-level change detectors (e.g. using FIT)
a. Tests of communication between two (or more) routines.
b. Tests in which we set values of multiple variables or drive the program through multiple tasks or events
- As with unit-level change detectors, the goal is not bug hunting, nor quality assessment.
- A tool like FIT might be used for those (bug-hunting / assessment, etc) tasks but that is a different use from change detection
The practice:
(1) the programmer creates change detectors
- to get insight into the problem and
- rapid feedback as she writes or modifies code.
(2) Run a test set (< 2 minutes) every compile, move additional tests into overflow and run them every few hours, or every day
3. The Adv. about Glass Box Testing in XP
(1) No roundtrip cost, compared to GUI automation and bug reporting.
- The "roundtrip cost" is the processing cost of a bug, counting the labor of all involved persons.
- For example, the tester discovers a bug, rechecks her work, retests, simplifies, then files a bug report which is read and prioritized by a project manager, assigned to a programmer who debugs and fixes the bug, returns it to the tester, who replicates and explores the bug fix before closing the bug.
- This can easily take 4 labor-hours and in more bureaucratic companies, might take 16 or more per bug.
- In contrast, to detect, debug, fix and retest a bug just after she has made the mistake, a programmer might only spend a few minutes.
- This is an enormous cost difference. If we eliminate roundtrip cost for hundreds of bugs, we can save the project thousands of labor-hours.
(2) No (or brief) delay in feedback loop compared to external tester loop
- Human factors research consistently concludes that short feedback loops are important to development of skills.
- The programmer who gets fast feedback about her errors is more likely to learn from them and
- connect her successful solutions back to the original problems she was trying to solve.
留言列表
