pgbench 安装试用
pgbench 是一个方便的pg 性能测试工具，以下是简单的测试试用
安装
安装pg
yum install https://download.postgresql.org/pub/repos/yum/10/redhat/rhel-7-x86_64/pgdg-centos10-10-2.noarch.rpm
yum install postgresql10
yum install postgresql10-server
yum install -y postgresql10-contrib
配置环境变量
默认安装是/usr/pgsql-10/bin，所以需要配置path 路径
/etc/profile
export PATH=$PATH:/usr/pgsql-10/bin简单试用
启动参数
以下为支持的参数
pgbench is a benchmarking tool for PostgreSQL.
Usage:
pgbench [OPTION]... [DBNAME]
Initialization options:
-i, --initialize invokes initialization mode
-F, --fillfactor=NUM set fill factor
-n, --no-vacuum do not run VACUUM after initialization
-q, --quiet quiet logging (one message each 5 seconds)
-s, --scale=NUM scaling factor
--foreign-keys create foreign key constraints between tables
--index-tablespace=TABLESPACE
create indexes in the specified tablespace
--tablespace=TABLESPACE create tables in the specified tablespace
--unlogged-tables create tables as unlogged tables
Options to select what to run:
-b, --builtin=NAME[@W] add builtin script NAME weighted at W (default: 1)
(use "-b list" to list available scripts)
-f, --file=FILENAME[@W] add script FILENAME weighted at W (default: 1)
-N, --skip-some-updates skip updates of pgbench_tellers and pgbench_branches
(same as "-b simple-update")
-S, --select-only perform SELECT-only transactions
(same as "-b select-only")
Benchmarking options:
-c, --client=NUM number of concurrent database clients (default: 1)
-C, --connect establish new connection for each transaction
-D, --define=VARNAME=VALUE
define variable for use by custom script
-j, --jobs=NUM number of threads (default: 1)
-l, --log write transaction times to log file
-L, --latency-limit=NUM count transactions lasting more than NUM ms as late
-M, --protocol=simple|extended|prepared
protocol for submitting queries (default: simple)
-n, --no-vacuum do not run VACUUM before tests
-P, --progress=NUM show thread progress report every NUM seconds
-r, --report-latencies report average latency per command
-R, --rate=NUM target rate in transactions per second
-s, --scale=NUM report this scale factor in output
-t, --transactions=NUM number of transactions each client runs (default: 10)
-T, --time=NUM duration of benchmark test in seconds
-v, --vacuum-all vacuum all four standard tables before tests
--aggregate-interval=NUM aggregate data over NUM seconds
--log-prefix=PREFIX prefix for transaction time log file
(default: "pgbench_log")
--progress-timestamp use Unix epoch timestamps for progress
--sampling-rate=NUM fraction of transactions to log (e.g., 0.01 for 1%)
Common options:
-d, --debug print debugging output
-h, --host=HOSTNAME database server host or socket directory
-p, --port=PORT database server port number
-U, --username=USERNAME connect as specified database user
-V, --version output version information, then exit
-?, --help show this help, then exit
Report bugs to <pgsql-bugs@postgresql.org>.准备一个数据库
为了方便使用docker-compose 运行,内容有点多，包含了mysql pg 以及一个pgbouncer（连接池工具）
version: "3"
services:
mysql:
image: mysql:5.7.16
volumes:
- ./gogs/mysql:/var/lib/mysql
ports:
- 3306:3306
command: --character-set-server=utf8mb4 --collation-server=utf8mb4_unicode_ci
environment:
MYSQL_ROOT_PASSWORD: dalongrong
MYSQL_DATABASE: gogs
MYSQL_USER: gogs
MYSQL_PASSWORD: dalongrong
TZ: Asia/Shanghai
postgres:
image: postgres:9.6
ports:
- "5432:5432"
environment:
- "POSTGRES_PASSWORD:dalong"
volumes:
- ./db_data2:/var/lib/postgresql/data
pgbouncer:
image: brainsam/pgbouncer
environment:
- DB_HOST=postgres
- DB_USER=postgres
- DB_PASSWORD=dalong
ports:
- "6432:6432"启动数据库
docker-compose up -d简单测试
- 初始化
pgbench -i --unlogged-tables -s 2 -U postgres -p 5432 -d postgres -h localhost效果
pgbench -i --unlogged-tables -s 2 -U postgres -p 5432 -d postgres -h localhost
NOTICE: table "pgbench_history" does not exist, skipping
NOTICE: table "pgbench_tellers" does not exist, skipping
NOTICE: table "pgbench_accounts" does not exist, skipping
NOTICE: table "pgbench_branches" does not exist, skipping
creating tables...
100000 of 200000 tuples (50%) done (elapsed 0.03 s, remaining 0.03 s)
200000 of 200000 tuples (100%) done (elapsed 0.06 s, remaining 0.00 s)
vacuum...
set primary keys...
done.
pgbench -i --unlogged-tables -s 16 -U postgres -p 5432 -h localhost -d postgres
creating tables...
100000 of 1600000 tuples (6%) done (elapsed 0.03 s, remaining 0.42 s)
200000 of 1600000 tuples (12%) done (elapsed 0.06 s, remaining 0.40 s)
300000 of 1600000 tuples (18%) done (elapsed 0.09 s, remaining 0.37 s)
400000 of 1600000 tuples (25%) done (elapsed 0.14 s, remaining 0.41 s)
500000 of 1600000 tuples (31%) done (elapsed 0.17 s, remaining 0.37 s)
600000 of 1600000 tuples (37%) done (elapsed 0.20 s, remaining 0.33 s)
700000 of 1600000 tuples (43%) done (elapsed 0.23 s, remaining 0.29 s)
800000 of 1600000 tuples (50%) done (elapsed 0.25 s, remaining 0.25 s)
900000 of 1600000 tuples (56%) done (elapsed 0.28 s, remaining 0.22 s)
1000000 of 1600000 tuples (62%) done (elapsed 0.31 s, remaining 0.19 s)
1100000 of 1600000 tuples (68%) done (elapsed 0.42 s, remaining 0.19 s)
1200000 of 1600000 tuples (75%) done (elapsed 0.52 s, remaining 0.17 s)
1300000 of 1600000 tuples (81%) done (elapsed 0.75 s, remaining 0.17 s)
1400000 of 1600000 tuples (87%) done (elapsed 0.96 s, remaining 0.14 s)
1500000 of 1600000 tuples (93%) done (elapsed 1.15 s, remaining 0.08 s)
1600000 of 1600000 tuples (100%) done (elapsed 1.55 s, remaining 0.00 s)
vacuum...
set primary keys...
done.- 执行压力测试
pgbench -M prepared -r -c 8 -j 2 -T 10 -U postgres -h localhost -p 5432 -d postgres -l参数说明
以上参数中，-M prepared表示绑定变量形式的调用SQL，-r表示报告测试文件中每条SQL的平均执行延迟，
-c 8表示模拟8个客户端，-j 2表示pgbench的工作线程是2个，
-T 10表示压力测试的时间是10秒，-l表示把事务统计写入log，其余的是postgres连接相关的参数
以上参数，也可以结合命令帮助看出来
效果
scaling factor: 16
query mode: prepared
number of clients: 8
number of threads: 2
duration: 10 s
number of transactions actually processed: 4115
latency average = 19.924 ms
tps = 401.527925 (including connections establishing)
tps = 401.690314 (excluding connections establishing)
script statistics:
- statement latencies in milliseconds:
0.370 \set aid random(1, 100000 * :scale)
0.005 \set bid random(1, 1 * :scale)
0.005 \set tid random(1, 10 * :scale)
0.005 \set delta random(-5000, 5000)
3.051 BEGIN;
2.684 UPDATE pgbench_accounts SET abalance = abalance + :delta WHERE aid = :aid;
1.693 SELECT abalance FROM pgbench_accounts WHERE aid = :aid;
2.799 UPDATE pgbench_tellers SET tbalance = tbalance + :delta WHERE tid = :tid;
3.317 UPDATE pgbench_branches SET bbalance = bbalance + :delta WHERE bid = :bid;
3.203 INSERT INTO pgbench_history (tid, bid, aid, delta, mtime) VALUES (:tid, :bid, :aid, :delta, CURRENT_TIMESTAMP);
2.740 END;说明
我们主要比较关注的是tps 参数，结合一些系统调优的技巧进行tps 参数的调整，类似的sysbench 也是可以进行pg 压测的,以下为
sysbench 的简单测试命令
sysbench \
--db-driver=pgsql \
--pgsql-host=127.0.0.1 \
--pgsql-port=5432 \
--pgsql-user=postgres \
--pgsql-password=dalong \
--pgsql-db=postgres \
--oltp-table-size=10000 \
--rand-init=on \
--threads=10 \
--time=120 \
--events=0 \
--report-interval=10 \
--percentile=99 \
/usr/share/sysbench/tests/include/oltp_legacy/oltp.lua \
prepare
sysbench \
--db-driver=pgsql \
--pgsql-host=127.0.0.1 \
--pgsql-port=5432 \
--pgsql-user=postgres \
--pgsql-password=dalong \
--pgsql-db=postgres \
--oltp-table-size=10000 \
--threads=10 \
--time=120 \
--events=0 \
--report-interval=10 \
--percentile=99 \
/usr/share/sysbench/tests/include/oltp_legacy/oltp.lua \
run
sysbench \
--db-driver=pgsql \
--pgsql-host=127.0.0.1 \
--pgsql-port=5432 \
--pgsql-user=postgres \
--pgsql-password=dalong \
--pgsql-db=postgres \
--oltp-table-size=10000 \
--threads=10 \
--time=120 \
--events=0 \
--report-interval=10 \
--percentile=99 \
/usr/share/sysbench/tests/include/oltp_legacy/oltp.lua \
cleanup参考资料
https://www.postgresql.org/docs/10/pgbench.html
https://github.com/rongfengliang/mysql-postgres-pgbouncer-docker-compose
