Baselines Creation
在做Performacne Testing時, 有一件很重要的事就是找出baseline. 有了baseline, 才會有比較的基礎. 否則你將無法判斷, 目前turning或是修改的結果是好是壞, 必須要有相對比較的東西
那找出baseline的流程是在做什麼呢, 定義如下:
- the process of running a set of tests
- to capture performance metric data
- for the purpose of evaluating the effectiveness of subsequent performance-improving changes to the system or application.
在找baseline 的過程, 你必須知道你所也相關的設定, 環境, scenarios為何, 因為之後這些東西都要維持不便, 否則比較就沒有意義了
所以日後若是這些條件變了, 可能是需求變了, 或是業界標準變了, 你都因此要重新建立新的baseline
在建立baseline時, 以下事情必須要注意
(1) 什麼東西我們可以對他建立baseline
- a system, component, or application, or
- different layers of the application, including a database, Web services, and so on.
(2) Basline可以用來設定比較的標準, 以供未來回歸測試或是最佳化(optimizations)的調整來使用
- It is important to validate that the baseline results are repeatable, because considerable fluctuations may occur across test results due to environment and workload characteristics.
(3) Basline可以用來追蹤和找出performance上面的改變
- Baselines can help product teams identify changes in performance that reflect degradation or optimization over the course of the development life cycle.
- Identifying these changes in comparison to a well-known state or configuration often makes resolving performance issues simpler.
(4) Baselines的資料必須可以在被使用的
- Baselines are most valuable if they are created by using a set of reusable test assets.
- It is important that such tests accurately simulate repeatable and actionable workload characteristics.
(5) Baselines是數據導向的
- Baseline results can be articulated by using a broad set of key performance indicators, including response time, processor capacity, memory usage, disk capacity, and network bandwidth.
(6) Baselines 可被作為一個共同的參照基礎結構
- Sharing baseline results allows your team to build a common store of acquired knowledge about the performance characteristics of an application or component.
(7) 避免過度一般化您的baselnes
- If your project entails a major reengineering of the application, you need to reestablish the baseline for testing that application.
- A baseline is application-specific and is most useful for comparing performance across different versions.
- Sometimes, subsequent versions of an application are so different that previous baselines are no longer valid for comparisons.
(8) 要知道你系統的運作行為
- It is a good idea to ensure that you completely understand the behavior of the application at the time a baseline is created.
- Failure to do so before making changes to the system with a focus on optimization objectives is frequently counterproductive.
(9) Baselines 會演變
- At times you will have to redefine your baseline because of changes that have been made to the system since the time the baseline was initially captured.
留言列表
