David Ko的學習之旅

Part IV QA Paradigm

1. QA Paradigm #1: Quality Assurance is Testing
(1) Most QA people are actually employed as testers
(2) “Did you QA this?”
(3) “Independent testing is better testing”
(4) Are used to testing untested code and struggle when working with agile developers
    - E.g., overuse of boundary testing is common

Part III Agile Activities and Reality Check
(這裡談到在agile activities的agile testing的關係, 在這些activities中testing 要如何運作和apply)
1. Test-First Programming
(1) Developers write unit tests before coding.
   - Motivates coding
   - Improves design (reducing coupling and improving cohesion)
   - Supports refactoring
(2) Many open-source test tools have been developed to support this
xUnit

Agile Testing Introduction
我survey了Bret Pettichord幾篇介紹agile testing的簡報. 我想應該可以幫大家快速入門
Content
Agile/Agile methodologies
Agile Testing
Agile Activities and Reality Check
QA Paradigm
4 Practices for Agile Testing

4. Functional Acceptance Testing
(1) Story-based (aka use-case based)
   - In the main books on XP, including Crispin's book on testing in XP, story-based tests are described as the primary vehicle for functional acceptance testing.
(2) Typical tests are fairly simple
   a. Happy path:
      - The main sequence(s) that lead to the result that the actor is trying to achieve
   b. Sad paths
      - Error cases or other paths that don't achieve the desired result
   c. Alternate paths
      - Non-error sequences that lead to results other than the typical, main result
(3) Might concatenate stories to develop more complex scenario tests
   - www.testingeducation.org/articles
(4) It's common for industrial test groups to use only one or two dominant test techniques. 

(5) We recommend that they balance their efforts by using several techniques
   - http://www.testingeducation.org/articles/blackbox_paradigms_tutorial.pdf
(6) because different tests are more effective for different types of bugs, or under different contexts.
   - http://www.satisfice.com/tools/satisfice-tsm-4p.pdf
(7) Scenario testing is just one style (or main technique) for functional acceptance testing. 

(8) One point of disagreement with XP literature
   a. It's often said that all acceptance testing should be automated. I think this is a serious error.
      - There are significant cost/benefit tradeoffs associated with UI-level automated testing.
      - The choice of all-versus-some automation should be pragmatic, based on the project's context.
   b. I disagree that all non-automated testing should be fully scripted, so the person behaves as if s/he were an automaton.
      - There are significant benefits to exploratory manual testing
      - The documentation cost of scripting is high and cost to future change is high because of the cost of maintenance of the documentation
      - It's an ineffective way to find bugs
     
5. Parafunctional Testing
(1) Functional testing is only part of the story. Consider these other attributes, which we call para-functional (or non-functional):
Security, Accessibility, Supportability, Localizability, Compatibility ,(configurations), Interoperability, Installability and , uninstallability, Usability, Performance, Scalability
(2) These are probably not well handled by customer stories.
   a. These aren't well defined as a set of features.
   b. They are (or aren't) built into every feature.
   c. These also are often very technical
      - the customer is not likely to be an authority on scalability or interoperability in the way that she is on her own business processes.

The Role of Testers in XP, Cem Kaner,  August 13, 2003
這篇文章介紹tester在XP中扮演怎樣的角色, 心態上要做怎樣的修正
首先, 一開始提出了傳統testing 的觀念是什麼
接著, 再提到在XP 中的testing 是在強調什麼
最後才提到QA的心態要做什麼調整.
若是你要pilot run XP, 這是一偏值得參考的文章
1. Testing: The Traditional View
(1) The purpose of testing is to find bugs.
(2) The test group should work independently of the programming group.
(3) There is a conflict of interest between the testing group and the programmers or project manager
(4) testing group’s task is to get the software "right“
(5) the programmers’ task is to get the software to market
(6) The test group should have veto power over the release of a product to the customer.
(7) Tests are designed without knowledge of the underlying code.
(8) Automated tests are developed at the user interface level, by non-programmers.
(9) Tests are designed early in development.
(10) Tests are designed to be reused time and time again, as regression tests
(11) To the maximum extent possible, tests should be automated.
(12) Programmers should do extensive unit testing, but they probably won't,
   - so testers should assume that the programmers did a light job of testing and
   - so should extensively cover the basics
(13) The pool of tests should cover every line and branch in the program, or perhaps every basis path
(14) Manual tests should be documented in great procedural detail so that they can be handed down to less experienced or less skilled testers, who will
   - repeat the tests consistently, in the way they were intended,
   - learn about test design from executing these tests, and
   - learn the program from testing it, using these tests.
(15) A test design must include the expected result for the test. A test without an expected result is not a test.
(16) There should be at least one thoroughly documented test for every requirement item or specification item
Test cases should be based on documented characteristics of the program,
Test cases should be individually documented and, ideally, stored in a test case management system that describes the pro-conditions, procedural details, post-conditions, and basis (such as trace to requirements) of each individual test case
Failures should be reported into a bug tracking system
A count of the number of defects missed, or a ratio of defects missed to defects found, is a good measure of the effectiveness of the test group.