Regression Testing Strategy (1)
Regression Testing是測試人員心中最大的痛, 因為這時間通常要花很多(因為test case實在太多了), 而且次數也不會只有一次. 因此好的regression testing strategy是非常重要的
我從"Testing Object-Oriented System" 中survey一些方法, 希望能對大家有幫助
A. Retest Risky Use Cases
(1) Intent
Use risk-based heuristics to select regression test suite.
(2) Context
The available time, personnel or equipment is not sufficient for a full regression run
This pattern is applicable at any scope. It may be used to support iterative velopment, the instantiation of reusable components and maintenance
(3) Strategy
a. Test Model:
The goal is to skip enough non-critical, low priority, or highly stable use cases to meet or budget constraint.
a-1 Risk criteria
- Suspicious use cases
Are individually unstable or unproven
Have not been shown to work together before or are unstable
Implement complex business rules
Have a complex implementation
Were subject to high churn during development
- Critical use cases
Select tests for use cases that are necessary for safe, effective operation.
For each use cases, try to identify the worst-case effects of failure
b. Test procedure
Identify, develop, and run the reduced test suite
c. Oracle
The results of the baseline test run provide the oracle. That is, if the SUT produces the same results in the regression run as it did in the baseline run, the test passes
(4) Entry Criteria
The delta components pass component scope testing
A suitable baseline test suite exists
The test environment has been restored to the same configuration used to run the original baseline test suite
(5) Exit Criteria
All no pass test cases reveal bugs whose presence and severity are deemed acceptable
All remaining test cases pass
(6) Consequences
a. Inclusiveness
Retest risky use cases is unsafe, because test cases are not selected by analysis of dependencies
b. Precision
Some tests that could be skipped may be repeated
c. Efficiency
The time and cost associated with testing are constrained
d. Generality
Retest risky use cases can be applied in nearly any circumstance and at any scope