David Ko的學習之旅

Regression Testing Strategy (1)

Regression Testing是測試人員心中最大的痛, 因為這時間通常要花很多(因為test case實在太多了), 而且次數也不會只有一次. 因此好的regression testing strategy是非常重要的 

我從"Testing Object-Oriented System" 中survey一些方法, 希望能對大家有幫助

A. Retest Risky Use Cases
(1) Intent
Use risk-based heuristics to select regression test suite. 

(2) Context
The available time, personnel or equipment is not sufficient for a full regression run
This pattern is applicable at any scope. It may be used to support iterative velopment, the instantiation of reusable components and maintenance 

(3) Strategy
a. Test Model: 
The goal is to skip enough non-critical, low priority, or highly stable use cases to meet or budget constraint. 

     a-1 Risk criteria
            - Suspicious use cases 
                      Are individually unstable or unproven 
                      Have not been shown to work together before or are unstable 
                      Implement complex business rules 
                      Have a complex implementation 
                      Were subject to high churn during development
           - Critical use cases 
                      Select tests for use cases that are necessary for safe, effective operation. 
                      For each use cases, try to identify the worst-case effects of failure

b. Test procedure
      Identify, develop, and run the reduced test suite 

c. Oracle
      The results of the baseline test run provide the oracle. That is, if the SUT produces the same results in the regression run as it did in the baseline run, the test passes

(4) Entry Criteria
     The delta components pass component scope testing
     A suitable baseline test suite exists
     The test environment has been restored to the same configuration used to run the original baseline test suite 

(5) Exit Criteria
     All no pass test cases reveal bugs whose presence and severity are deemed acceptable
     All remaining test cases pass 

(6) Consequences
a. Inclusiveness
     Retest risky use cases is unsafe, because test cases are not selected by analysis of dependencies
b. Precision
     Some tests that could be skipped may be repeated
c. Efficiency
     The time and cost associated with testing are constrained
d. Generality
     Retest risky use cases can be applied in nearly any circumstance and at any scope