数据库测量(Database Benchmarking)
在采购新设备或者数据库软件前，公司都会考虑使用什么样的硬件和软件才能够满足公司当前和未来的需求，如何去判断设备和数据库的承载能力是非常关键的。事务处理委员会（Transaction Processing Council）提供了量化的方法和标准。
以下是在oracle wiki中看到的相关文章，其中介绍了测量的原因、标准，同时介绍了几款商业和开源的测量工具软件，还为大家推荐了测量的相关书籍。
以下是原文内容：
代码
Database Benchmarking
The following is my intro information for database benchmarking...
Reasons for Benchmarking
Benchmarks are performed for various reasons. However, benchmarks are primarily used:
* To compare different hardware configurations
Benchmarks can be used to compare the relative performance of different hardware running the same application. This is generally used to directly compare hardware configurations between two hardware vendors.
* To compare different database vendor software
By running the same benchmark using different database software on the same machine, one can easily compare between different database vendors. This is generally used to make a price/performance decision between vendors such as Oracle, Microsoft, IBM, etc.
* To compare different database software releases
Similar to the above, one can use different versions of the same vendor's database software to compare the one they want to use or check for performance regressions due to upgrades (i.e. 10g vs. 11g).
Real World Benchmarks
While there are several industry standard benchmarks, most of the time, they don't reflect a company's actual workload. As such, you will often find custom benchmarks being performed. The goals of a real-world benchmark are the same reasons mentioned above.
Industry Standard Database Benchmarks
Industry standard benchmarks are generally used by businesses to compare different hardware and software system performance for purchase-related reasons. While industry standard benchmarks are modelled after real-world workloads, they rarely reflect a company's real workload. However, it is a quick way to compare the performance of various hardware and software combinations operating in a well-defined scenario. The major industry standard benchmarks are listed below.
Transaction Processing Council (TPC)
The TPC is a non-profit corporation which supports a consortium of hardware and database software vendors devoted to defining transaction processing and database-related benchmarks. The primary goal behind TPC benchmarks is the definition of functional requirements which can be run on any database, regardless of the hardware or operating system. This allows vendors to implement their own benchmark kits in order to satisfy the functional requirements. Similarly, after having publicly submitted proof that a benchmark was conducted according to the specification, end-users have more of an assurance that what they are presented with is a valid, apples-to-apples comparison.
* TPC Benchmark C (TPC-C)
The TPC-C simulates an order-entryenvironment where a population of terminal operators executes transactions against a database. The benchmark is comprised of transactions which include entering and delivering orders, recording payments, checking the status of orders, and monitoring the level of stock at the warehouses. The most frequent transaction consists of entering a new order which, on average, is comprised of ten different items. Each warehouse tries to maintain stock for the 100,000 items in the Company's catalog and fill orders from that stock. The performance metric reported by TPC-C measures the number of orders that can be fully processed per minute and is expressed in tpm-C. The TPC-C will soon be deprecated in favor of the TPC-E.
* TPC Benchmark E (TPC-E)
The TPC-E benchmark simulates the OLTP workload of a brokerage firm. The focus of the benchmark is the central database that executes transactions related to the firm’s customer accounts. The TPC-E metric is given in transactions per second (tps). It specifically refers to the number of Trade-Result transactions the server can sustain over a period of time.
* TPC Benchmark H (TPC-H)
TheTPC-H is a decision support benchmark which consists of several business-oriented ad-hoc queries and concurrent data modifications. This benchmark illustrates decision support systems that examine large volumes of data, execute queries with a high degree of complexity, and give answers to critical business questions.
The performance metric reported by TPC-H is called the TPC-H Composite Query-per-Hour Performance Metric (QphH@Size), and reflects multiple aspects of the capability of the system to process queries.
Standard Performance Evaluation Corporation (SPEC)
The SPEC is a
* SPEC jAppServer
The TPC-C is
Free and Open Source Benchmark Kits
As the benchmark kits used by various database vendors are proprietary, several open source database benchmark kits have been written to implement not only the industry standard workloads, but others as well.
* Open Source Development Labs Database Test Suite
The OSDL DBT suite is the most comprehensive of all open source benchmark kits. While its benchmark implementations are based on TPC standards, they differ in some areas and are not certified; making a comparison between a certified TPC-C result and OSDL's implementation of TPC-C (DBT-2) impossible. As the original OSDL DBT suite lacks Oracle support, Oracle has added and released it as part of the Oracle Linux Test kit.
* PolePosition
PolePosition is an open source benchmark test suite to compare database engines and object-relational mapping technology.
* jTPCC
jTPCC is an open source Java implementation of the TPC-C benchmark. While it claims compliance with TPC-C, it is not.
* BenchmarkSQL
BenchmarkSQL is an open source, easy to use JDBC benchmark which closely resembles the TPC-C standard for OLTP; it was originally based on jTPCC.
* Bristlecone
Bristlecone is a simple SELECT+INSERT/UPDATE/DELETE throughput and response-time benchmark written in Java.
* FinTime
FinTime is an open source set of data and queries which reflects the needs of financial analysts who are studying patterns in stock market data, but it should appeal to the designers of any system that has pretensions of handling ordered data well.
* Java TPC-W Implementation
This open-source implementation is based on the TPC-W specification version 1.0.1. It strays from the official benchmark specification in a few minor areas, which are specified in documentation included with the distribution. This version itself lacks Oracle support, but it can be found on the web.
* Swingbench
Swingbench is a free load generator (and benchmarks) designed to stress test an Oracle database (9i,10g,11g).
* OpenLink ODBC Bench
OpenLink ODBC Bench is an open-source ODBC Benchmarking tool providing real-time comparative benchmarking for ODBC Drivers, Database Engines, and Operating Systems combinations. The Benchmarks in this application are loosely based on the TPC-A and TPC-C standard benchmarks, with modifications to specifically test the performance of an ODBC Driver and/or Database Engine in a client/server environment.
* OpenLink JDBC Bench
Similar to OpenLink's ODBC Bench, this is a JDBC version.
* TPCC-UVa
A free, open-source implementation of the TPC-C Benchmark.
* Hammerora
An open source load generation tool.
Commercial Benchmark Kits
There are several industry standard benchmarks.
* Official Oracle Benchmark Kits
Similar to Microsoft and IBM, the official benchmark kits used by Oracle can only be obtained under certain conditions. For more information, contact your Oracle Consulting/Partner/Support representative.
* Real Application Testing
While many wouldn't think of Real Application Testing as a traditional benchmark kit, it does perform workload execution primarily for the real world benchmarking case.
* Quest Software's Benchmark Factory for Databases
Benchmark Factory for Databases is a workload simulation and scalability testing tool that stress tests your environment by simulating users and transactions on the database. You can either replay production workload or use synthetic workload to test the limits of your database.
* SPEC jAppServer Benchmark Kit
This can be acquired directly from SPEC.
Benchmarking-related Books
There are several good books regarding benchmarking. The best I've found include the following:
* The Benchmark Handbook: For Database and Transaction Processing Systems (Jim Gray)
A great, albeit outdated book. While several of the benchmarks it discusses have been deprecated, it presents good concepts and background information regarding benchmarking.
* Performance Analysis of Transaction Processing Systems (Wilbur H. Highleyman)
While out of print, this book presents both plain-text and math-heavy concepts of queueing and performance analysis techniques oriented for OLTP.
* Database Benchmarking: Practical Methods for Oracle & SQL Server
This book explores all aspects of database benchmarking and shows you a real-world approach that ensures that you are prepared for whatever the future brings to your mission-critical database.
