Scrum + Manual Testing methodology
Scrum + Manual Testing methodology (for WEB apps.)
http://www.testingreflections.com/node/view/7378
作者認為在run Scrum時, 對於testing經常會有以下兩種錯誤的作法
- to give up manual testing skills and learn automation or
- give up agile principle of quick feedback and test one iteration behind.
他認為應該在Scrum中加入一個role叫做Tester (而不是只有team, scrum master和project owner). 這樣才會讓Scrum更完整. 可是目前他並沒有看到任何研究, 是有關這方面的. 因此他會致力於這方面的研究, 如何有興趣的人可以和他一起討論.(作者的mail: ojnjars@inbox.lv)
以下是他方法的一些前提假設(Context – Assumptions)
1. Development Scrum consists of 2 week sprints
2. The software is a web based with certain data in database.
- The role of tester will include testing software in “QA environment”, while developers are still doing unit tests in their environment which is not as close to production one.
3. It is decided that software deployment in “QA environment” should follow the manual installation process the same as it will take place in production environment.
- It optionally needs smart decision if to redeploy data into database.
- The process may take up to 30 minutes and is never done this way by developers in their environment
4. The process of creating installation package (build process) may also take up to 30 minutes, so will be executed once per night (nightly build)
接下來, 他描述在一個2 weeks 的sprint中, QA的活動在做些什麼:
Day 1. Sprint Planning
During the first day of a spring – sprint planning meeting – tester participates in order to help to define acceptance criteria (by asking what if questions to product owner) and estimating testing effort for stories.
- It may appear that there are stories requiring a little development effort but a lot of testing effort.
- Story selection may be changed to disallow too much stories of such type into one sprint.
- Alternatively part of testing activities may be postponed, creating a separate story (adding what I would call “quality debt”) and adding to backlog.
Day 2. Test Case Drafting
- A tester works: one day behind” the team.
- The second day of a sprint is dedicated by tester to draft initial set of test cases for each features included in the sprint.
Day 3-8. Normal days
(1) 基本作法
- Each day starting day 3 of sprint in the morning tester installs latest nightly build (manually using installation procedure/instructions supplied by development team) to QA environment, do fast smoke tests if they fail and emergency fixes are applied by the team.
- Features developed yesterday shall be tested today by tester.
- As required tester communicates with each developer to transition the knowledge of the feature functionality.
- Tester must also update (complete) Test Cases in the process of testing as more test ideas may come out during Exploratory Testing.
(2) 如果測試無法進行
- If a feature testing requires too much effort or tester discovers too much ideas and can’t accomplish testing within the day he create new story ticket for further feature investigation.
- If the ticket is not closed by the end of a sprint it appears to be “Quality Debt” and must be announced to product owner for prioritization.
- Because developers do unit testing most of the features should pass acceptance criteria.
- However if for any reason they do not pass them in QA environment, QA may return ticket to the team.
- If acceptance criteria are met but there are more bugs around, tester reports them separately, sending the initial story to End-To-End testing.
(3) 如果bug無法處理時
- Tester also has to do wider investigation (based on project quality goals) and report any usability suggestions, performance, security or other potential treats.
- Such bugs must be assigned to Product Owner if they don’t directly break the acceptance criteria of a story.
- Bugs that (developer) team disagrees with or appears to require too much effort also get assigned to Product Owner.
- They have to be addressed during sprint retrospective meeting.
- So it appears that during sprint retrospective no feature could be demonstrated if it has an open bug against it assigned to anyone but product owner.
Day 9 – The QA day
- Because tester works one day behind, the day 8 of the sprint shall be the last day of a new feature development.
- That is the reason why day 9 can be dedicated by team to bug fixing, re-factoring and analysis of new features which might be included into next sprint i.e. next sprint preparation.
Day 10 - retrospective meeting
- In order to demonstrate a feature in retrospective meeting it should be implemented; tested and all bugs either fixed or assigned to Product Owner for clarification (otherwise feature is postponed to next sprint).
- Tester should demonstrate that features meet acceptance criteria and demonstrate the issues assigned to product owner if any.
- Product Owner should decide and prioritize those features/defects.
- The team participates in the meeting to qualify their performance and probably counter tester’s negative opinion.
- Tester also describes “Quality Debt” changes – what was not tested due to lack of time.
- Product owner may prioritize the postponed tests or even allow skipping them.
在Sprint結束後, QA還有很多事要做
- Test Cases created during sprint shall be peer-reviewed by another tester at any time later. For example by End-To-End tester if there are any.
- Ideally review should be done during or by end of the day 10 so that tester has the feedback by next planning meeting, but it may be impossible.
- In any case tester is not forced to respond to review feedback (fix test cases and run additional test if identified) until the end of sprint (so that tester commitment during sprint planning is not compromised).
- If more tests are identified during review they must be executed later and any defects discovered should be assigned to product owner (added to backlog).
不包含在Sprint中的測試
(1) Non-function的測試
- Still there are types of tests that are not directly related to functionality implemented by sprints.
- For example non-functional tests: to validate that given architecture satisfies high availability requirements.
- Actually the tester may not have enough skills to do performance tests and a separate team or person may be assigned to do performance testing based on agreed schedule.
(2) Regression Testing
- More over it is highly welcome to develop system-level automated regression tests (based on, but not necessarily repeating manual test cases) along with developer created unit tests.
- Those tests will aim for different type of regression bugs.
- This activity may also take place with a significant delay to feature implementation.
- The tester should make a smart decision to nominate the build that was especially successful (without critical bugs) to be installed to the special environment such as for performance test or test automation environment.
Scrum + Manual Testing methodology (for WEB apps.)
http://www.testingreflections.com/node/view/7378
作者認為在run Scrum時, 對於testing經常會有以下兩種錯誤的作法
- to give up manual testing skills and learn automation or
- give up agile principle of quick feedback and test one iteration behind.
他認為應該在Scrum中加入一個role叫做Tester (而不是只有team, scrum master和project owner). 這樣才會讓Scrum更完整. 可是目前他並沒有看到任何研究, 是有關這方面的. 因此他會致力於這方面的研究, 如何有興趣的人可以和他一起討論.(作者的mail: ojnjars@inbox.lv)
以下是他方法的一些前提假設(Context – Assumptions)
1. Development Scrum consists of 2 week sprints
2. The software is a web based with certain data in database.
- The role of tester will include testing software in “QA environment”, while developers are still doing unit tests in their environment which is not as close to production one.
3. It is decided that software deployment in “QA environment” should follow the manual installation process the same as it will take place in production environment.
- It optionally needs smart decision if to redeploy data into database.
- The process may take up to 30 minutes and is never done this way by developers in their environment
4. The process of creating installation package (build process) may also take up to 30 minutes, so will be executed once per night (nightly build)
接下來, 他描述在一個2 weeks 的sprint中, QA的活動在做些什麼:
Day 1. Sprint Planning
During the first day of a spring – sprint planning meeting – tester participates in order to help to define acceptance criteria (by asking what if questions to product owner) and estimating testing effort for stories.
- It may appear that there are stories requiring a little development effort but a lot of testing effort.
- Story selection may be changed to disallow too much stories of such type into one sprint.
- Alternatively part of testing activities may be postponed, creating a separate story (adding what I would call “quality debt”) and adding to backlog.
Day 2. Test Case Drafting
- A tester works: one day behind” the team.
- The second day of a sprint is dedicated by tester to draft initial set of test cases for each features included in the sprint.
Day 3-8. Normal days
(1) 基本作法
- Each day starting day 3 of sprint in the morning tester installs latest nightly build (manually using installation procedure/instructions supplied by development team) to QA environment, do fast smoke tests if they fail and emergency fixes are applied by the team.
- Features developed yesterday shall be tested today by tester.
- As required tester communicates with each developer to transition the knowledge of the feature functionality.
- Tester must also update (complete) Test Cases in the process of testing as more test ideas may come out during Exploratory Testing.
(2) 如果測試無法進行
- If a feature testing requires too much effort or tester discovers too much ideas and can’t accomplish testing within the day he create new story ticket for further feature investigation.
- If the ticket is not closed by the end of a sprint it appears to be “Quality Debt” and must be announced to product owner for prioritization.
- Because developers do unit testing most of the features should pass acceptance criteria.
- However if for any reason they do not pass them in QA environment, QA may return ticket to the team.
- If acceptance criteria are met but there are more bugs around, tester reports them separately, sending the initial story to End-To-End testing.
(3) 如果bug無法處理時
- Tester also has to do wider investigation (based on project quality goals) and report any usability suggestions, performance, security or other potential treats.
- Such bugs must be assigned to Product Owner if they don’t directly break the acceptance criteria of a story.
- Bugs that (developer) team disagrees with or appears to require too much effort also get assigned to Product Owner.
- They have to be addressed during sprint retrospective meeting.
- So it appears that during sprint retrospective no feature could be demonstrated if it has an open bug against it assigned to anyone but product owner.
Day 9 – The QA day
- Because tester works one day behind, the day 8 of the sprint shall be the last day of a new feature development.
- That is the reason why day 9 can be dedicated by team to bug fixing, re-factoring and analysis of new features which might be included into next sprint i.e. next sprint preparation.
Day 10 - retrospective meeting
- In order to demonstrate a feature in retrospective meeting it should be implemented; tested and all bugs either fixed or assigned to Product Owner for clarification (otherwise feature is postponed to next sprint).
- Tester should demonstrate that features meet acceptance criteria and demonstrate the issues assigned to product owner if any.
- Product Owner should decide and prioritize those features/defects.
- The team participates in the meeting to qualify their performance and probably counter tester’s negative opinion.
- Tester also describes “Quality Debt” changes – what was not tested due to lack of time.
- Product owner may prioritize the postponed tests or even allow skipping them.
在Sprint結束後, QA還有很多事要做
- Test Cases created during sprint shall be peer-reviewed by another tester at any time later. For example by End-To-End tester if there are any.
- Ideally review should be done during or by end of the day 10 so that tester has the feedback by next planning meeting, but it may be impossible.
- In any case tester is not forced to respond to review feedback (fix test cases and run additional test if identified) until the end of sprint (so that tester commitment during sprint planning is not compromised).
- If more tests are identified during review they must be executed later and any defects discovered should be assigned to product owner (added to backlog).
不包含在Sprint中的測試
(1) Non-function的測試
- Still there are types of tests that are not directly related to functionality implemented by sprints.
- For example non-functional tests: to validate that given architecture satisfies high availability requirements.
- Actually the tester may not have enough skills to do performance tests and a separate team or person may be assigned to do performance testing based on agreed schedule.
(2) Regression Testing
- More over it is highly welcome to develop system-level automated regression tests (based on, but not necessarily repeating manual test cases) along with developer created unit tests.
- Those tests will aim for different type of regression bugs.
- This activity may also take place with a significant delay to feature implementation.
- The tester should make a smart decision to nominate the build that was especially successful (without critical bugs) to be installed to the special environment such as for performance test or test automation environment.
全站熱搜
留言列表
