David Ko的學習之旅

在進行Benchmarking時常發生的錯誤

Source: High Performance MySQL, 
Baron Schwartz, Peter Zaitsev, Vadim Tkachenko, Jeremy D. Zawodny, Arjen Lentz, and Derek J. Balling, 
Oreilly

當我在讀"High Performance MySQL"一書時, 作者提到在執行Benchmarking時, 有些錯誤要小心不要去犯. 可是我看完後發現, 幾乎每一條我們都遇到了, 還覺得真得是很可恥. 因此特地整理出來給大家看一下, 希望大家不會在遇到同樣的問題.

1. Using a subset of the real data size

2. Using incorrectly distributed data, such as uniformly distributed data when the real system’s data will have “hot spots.”

3. Using unrealistically distributed parameters, such as pretending that all user pro-files are equally likely to be viewed.

4. Using a single-user scenario for a multiuser application.

5. Benchmarking a distributed application on a single server.

6. Failing to match real user behavior, such as “think time” on a web page.

7. Running identical queries in a loop. Real queries aren’t identical, so they cause cache misses.

8. Failing to check for errors. Always check error logs after benchmarks, as a matter of principle.

9. Ignoring how the system performs when it’s not warmed up, such as right after a restart.

10. Using default server settings.