Agile Testing 的十個神話
Ten myths of agile testing
http://www.aphids.com/agiletesting/2008/09/ten_myths_of_agile_testing.html
如果你的公司也採用agile development, 你可能也會面臨到和George Wilson (他是AIG Computer Services的Business Group Manager)相同的挑戰.
Wilson提到 :
"Agile projects are an excellent opportunity for QA to take leadership of the [testing] processes,"
與其在developers主導時坐在後面, Wilson建議testers是可以主導一些事情:
"Who else is better placed to bridge the gap between users and developers, understand … what is required, how it can be achieved and how it can be assured prior to deployment?"
要做到這些 QA team需要更了解agile 是在做什麼, 所以Wilson列出了下面事實, 讓你可以看清agile testing的實質是什麼, 要如何做才是正確的
1. 你只需要Unit Test - TDD 測試已經足夠
- Test Driven Development is a good start, but for those who think it's all there is, "for the vast majority of commercial developments this simply isn’t true.
- Even strong proponents of agile development recognize the need for their armory to include a range of testing techniques...including white box, black box, regression testing, stress testing and UAT," said Wilson.
- The most effective agile projects will include investigative (black box) testing efforts that support (rather than compete) with white box testing.
- "Good investigative testing will reveal problems that the developer missed before they get too far downstream," said Wilson.
2. 你可以再利用 Unit Tests去建立Regression Test Suite
- Has anyone ever told you that conventional downstream testing is unnecessary because every line of application code has a corresponding test case?
- "Some TDD proponents ... suggest that by reassembling unit tests, everything from User Acceptance Tests to Regression Tests can be performed," said Wilson.
- While this may sound feasible, Wilson says it isn't realistic because the granularity and objectives of white box unit tests developed in TDD serve a different purpose than downstream black box testing.
- "While the overall objective of a unit test is to prove that the code will do what is expected, the aim of regression testing is to ensure that no unexpected effects result from the application code being changed. These two objectives are not synonymous.
3. 我們不再需要 Tesers或是 Automation Tools
- Not true.
- Testers perform a "different and equally valid role from their development colleagues," says Wilson, and should therefore be have an equal place at the project table.
- "It is [also] widely recognized that traditional test automation tools have failed to live up to the hype of the vendors." Perhaps vendors (no offense, Original) should play down the hype.
4. Unit Tests 可以排除要Manual Testing的需求
- Few would disagree that manual testing is repetitive, expensive, boring and error-prone.
- While TDD can reduce the amount of manual effort needed for functional testing, it does not remove the need for further manual or automated black box testing.
- "By automatically capturing the tester’s process and documenting their keystrokes and mouse clicks, the tester will have more time for the interesting, value-add activities, such as testing complex scenarios that would be difficult or impossible to justify automating.
- Though manual testing is a time-consuming (and therefore expensive) way to find errors, the costs of not finding them are often much higher," said Wilson.
5. User Acceptance Testing 不再是需要的
- Acceptance testing is often positioned as working with the customer to resolve incorrect requirements rather than correcting functionality that does not map to what the user needs.
- And maybe it's actually both. "When the user initially defines their requirements, they do it based on their initial expectations. When they see the system 'in the flesh,' they will invariably come up with different or additional requirements," he says.
- While the agile process might reduce the frequency of this happening, Wilson believes it is unreasonable to expect the approach to resolve them entirely, so there should be no expectation that user acceptance testing will be obviated.
- "This is especially true when it comes to the business user signing off approval of the user interface, since they may have envisaged something different from the developer."
6. Automation 是不可能的
- "Automation in the early stages of an agile project is usually very tough, but as the system grows and evolves, some aspects settle," says Wilson, simplifying the deployment of automation that can cope with changes.
- "To begin with, almost all testing will be manual, but this testing effort and design can be beneficial later if captured and reused. Knowing the right time to automate is tough, so using technology that proactively supports early manual testing but provides a path to evolve this into automation is key."
7. Developers 有足夠和適當的Ttesting Skills
- "If testing was easy, everybody would do it and we’d deliver perfect code every time," says Wilson. Just as writers benefit from proofreading by eyes other than their own, a testing team independent of developers can better see the big picture and can validate not just the functionality, but also the quality of the deliverable.
- "While developers tend towards proving the system functions as required, a good tester will be detached enough to ask, 'What happens if...?' ”
8. Unit Test可以涵蓋100% Design Spec
- Regardless of the development method, requirements must be known before code is developed.
- “While TDD ‘done well’ may identify a fair percentage of the design specification, there are still concerns about gaps between the unit tests,” says Wilson.
- He asserts that there are other approaches that are equally viable, and that the argument that you need TDD to prove the requirements are captured accurately isn’t substantiated by history.
- “Defining test cases to check the accuracy and succinctness of the requirements is nothing new. The V-model, for example, is a valid approach to understanding testing requirements, and by implication, functional requirements. Like TDD, the V-model and most other models fall down when the practitioner’s approach is rigid, while development processes are fluid. Software engineering is not like mechanical engineering, and trying to force conformity is wasted effort,” Wilson says.
- Whichever approach a team chooses, each user requirement should still be challenged by asking how it should be tested.
- “The important factor here is that test cases are challenged before they are committed to code, otherwise you’ll be doing a lot of recoding and calling it refactoring.”
- When the requirements are refined through collaboration, he adds, “the developer receives a [better] specification that is less likely to change because it has been openly appraised from several people’s perspectives.”
9. TDD 可適用於每個專案
- As the size of the project increases, so too does the time required to run the tests.
- Wilson says this can be addressed by partitioning the project, the tests or both. “Either route results in tests that run at different frequencies depending on their relevance to the current coding effort.”
- This approach, he says, introduces the need for test planning and execution management.
- “To achieve this efficiently, in addition to white box testing, you need to be thinking about…
10. Developers 和 Testers 就像油和水一樣格格不入
- There has long been an “us and them” tension between developers and testers, but it doesn’t have to be contentious.
- “This is usually a healthy symbiotic relationship which, when working correctly, provides a mutually beneficial relationship between the two groups, resulting in a higher quality deliverable for the customer," says Wilson.
- To help keep a handle on tensions, Wilson suggests focusing discussions on the following:
* On meeting business objectives and NOT on who owns which parts of the process.
* Involving testers in the requirements gathering phase and the TDD process.
* Maximizing reuse of test assets created during the development phase.
* The role of the traditional tester in TDD, and how to acquire new skills and enable them to adapt.
* How developers and testers can find mutual benefit in exploiting new advances in software development and testing tools.
Ten myths of agile testing
http://www.aphids.com/agiletesting/2008/09/ten_myths_of_agile_testing.html
如果你的公司也採用agile development, 你可能也會面臨到和George Wilson (他是AIG Computer Services的Business Group Manager)相同的挑戰.
Wilson提到 :
"Agile projects are an excellent opportunity for QA to take leadership of the [testing] processes,"
與其在developers主導時坐在後面, Wilson建議testers是可以主導一些事情:
"Who else is better placed to bridge the gap between users and developers, understand … what is required, how it can be achieved and how it can be assured prior to deployment?"
要做到這些 QA team需要更了解agile 是在做什麼, 所以Wilson列出了下面事實, 讓你可以看清agile testing的實質是什麼, 要如何做才是正確的
1. 你只需要Unit Test - TDD 測試已經足夠
- Test Driven Development is a good start, but for those who think it's all there is, "for the vast majority of commercial developments this simply isn’t true.
- Even strong proponents of agile development recognize the need for their armory to include a range of testing techniques...including white box, black box, regression testing, stress testing and UAT," said Wilson.
- The most effective agile projects will include investigative (black box) testing efforts that support (rather than compete) with white box testing.
- "Good investigative testing will reveal problems that the developer missed before they get too far downstream," said Wilson.
2. 你可以再利用 Unit Tests去建立Regression Test Suite
- Has anyone ever told you that conventional downstream testing is unnecessary because every line of application code has a corresponding test case?
- "Some TDD proponents ... suggest that by reassembling unit tests, everything from User Acceptance Tests to Regression Tests can be performed," said Wilson.
- While this may sound feasible, Wilson says it isn't realistic because the granularity and objectives of white box unit tests developed in TDD serve a different purpose than downstream black box testing.
- "While the overall objective of a unit test is to prove that the code will do what is expected, the aim of regression testing is to ensure that no unexpected effects result from the application code being changed. These two objectives are not synonymous.
3. 我們不再需要 Tesers或是 Automation Tools
- Not true.
- Testers perform a "different and equally valid role from their development colleagues," says Wilson, and should therefore be have an equal place at the project table.
- "It is [also] widely recognized that traditional test automation tools have failed to live up to the hype of the vendors." Perhaps vendors (no offense, Original) should play down the hype.
4. Unit Tests 可以排除要Manual Testing的需求
- Few would disagree that manual testing is repetitive, expensive, boring and error-prone.
- While TDD can reduce the amount of manual effort needed for functional testing, it does not remove the need for further manual or automated black box testing.
- "By automatically capturing the tester’s process and documenting their keystrokes and mouse clicks, the tester will have more time for the interesting, value-add activities, such as testing complex scenarios that would be difficult or impossible to justify automating.
- Though manual testing is a time-consuming (and therefore expensive) way to find errors, the costs of not finding them are often much higher," said Wilson.
5. User Acceptance Testing 不再是需要的
- Acceptance testing is often positioned as working with the customer to resolve incorrect requirements rather than correcting functionality that does not map to what the user needs.
- And maybe it's actually both. "When the user initially defines their requirements, they do it based on their initial expectations. When they see the system 'in the flesh,' they will invariably come up with different or additional requirements," he says.
- While the agile process might reduce the frequency of this happening, Wilson believes it is unreasonable to expect the approach to resolve them entirely, so there should be no expectation that user acceptance testing will be obviated.
- "This is especially true when it comes to the business user signing off approval of the user interface, since they may have envisaged something different from the developer."
6. Automation 是不可能的
- "Automation in the early stages of an agile project is usually very tough, but as the system grows and evolves, some aspects settle," says Wilson, simplifying the deployment of automation that can cope with changes.
- "To begin with, almost all testing will be manual, but this testing effort and design can be beneficial later if captured and reused. Knowing the right time to automate is tough, so using technology that proactively supports early manual testing but provides a path to evolve this into automation is key."
7. Developers 有足夠和適當的Ttesting Skills
- "If testing was easy, everybody would do it and we’d deliver perfect code every time," says Wilson. Just as writers benefit from proofreading by eyes other than their own, a testing team independent of developers can better see the big picture and can validate not just the functionality, but also the quality of the deliverable.
- "While developers tend towards proving the system functions as required, a good tester will be detached enough to ask, 'What happens if...?' ”
8. Unit Test可以涵蓋100% Design Spec
- Regardless of the development method, requirements must be known before code is developed.
- “While TDD ‘done well’ may identify a fair percentage of the design specification, there are still concerns about gaps between the unit tests,” says Wilson.
- He asserts that there are other approaches that are equally viable, and that the argument that you need TDD to prove the requirements are captured accurately isn’t substantiated by history.
- “Defining test cases to check the accuracy and succinctness of the requirements is nothing new. The V-model, for example, is a valid approach to understanding testing requirements, and by implication, functional requirements. Like TDD, the V-model and most other models fall down when the practitioner’s approach is rigid, while development processes are fluid. Software engineering is not like mechanical engineering, and trying to force conformity is wasted effort,” Wilson says.
- Whichever approach a team chooses, each user requirement should still be challenged by asking how it should be tested.
- “The important factor here is that test cases are challenged before they are committed to code, otherwise you’ll be doing a lot of recoding and calling it refactoring.”
- When the requirements are refined through collaboration, he adds, “the developer receives a [better] specification that is less likely to change because it has been openly appraised from several people’s perspectives.”
9. TDD 可適用於每個專案
- As the size of the project increases, so too does the time required to run the tests.
- Wilson says this can be addressed by partitioning the project, the tests or both. “Either route results in tests that run at different frequencies depending on their relevance to the current coding effort.”
- This approach, he says, introduces the need for test planning and execution management.
- “To achieve this efficiently, in addition to white box testing, you need to be thinking about…
10. Developers 和 Testers 就像油和水一樣格格不入
- There has long been an “us and them” tension between developers and testers, but it doesn’t have to be contentious.
- “This is usually a healthy symbiotic relationship which, when working correctly, provides a mutually beneficial relationship between the two groups, resulting in a higher quality deliverable for the customer," says Wilson.
- To help keep a handle on tensions, Wilson suggests focusing discussions on the following:
* On meeting business objectives and NOT on who owns which parts of the process.
* Involving testers in the requirements gathering phase and the TDD process.
* Maximizing reuse of test assets created during the development phase.
* The role of the traditional tester in TDD, and how to acquire new skills and enable them to adapt.
* How developers and testers can find mutual benefit in exploiting new advances in software development and testing tools.
