KingbaseES 表级设置autovacuum有关参数和触发机制
前言
在表级别设置autovacuum有关参数清理表的死亡元祖可以有效规避autovacuum触发全局默认阈值的高峰。例如，如果想让表的autovacuum任务更频繁，可以在表级设置更小的触发autovacuum有关参数autovacuum_vacuum_scale_factor。
如果需要在很多表上单独设置autovacuum有关参数，别忘了增加autovacuum_max_workers进程，默认是3，意味着同一时间autovacuum进程只能清理3张触发autovacuum阈值的表。
如果数据库中有多个大量数据的表，需要设置足够大的maintenance_work_mem内存空间，避免出现多次索引scan。本例中将测试如何在表上设置autovacuum有关参数，以及何时触发。
设置表级autovacuum阈值的步骤:
1.开启自动清理守护进程 alter table t2 set(autovacuum_enabled);
2.设置触发VACUUM/ANALYZE的被更新或被删除元组的最小数量
alter table t2 set(autovacuum_vacuum_threshold=1);
3.或者设置表尺寸 alter table t2 set(autovacuum_vacuum_scale_factor =0.1/autovacuum_analyze_scale_factor =0.01);
autovacuum_vacuum_scale_factor 默认值为0.2. autovacuum_analyze_scale_factor 默认值为0.1
取消表级参数:
alter table t2 reset(autovacuum_enabled);
alter table t2 reset(autovacuum_vacuum_threshold);
测试
1.测试t表，仅设置autovacuum_vacuum_threshold参数
alter table t2 set(autovacuum_enabled);
alter table t2 set(autovacuum_vacuum_threshold=1);
TEST=# \d+ t2
Table "public.t2"
Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
------------+--------------------------------+-----------+----------+---------+---------+--------------+-------------
id | integer | | | | plain | |
start_time | timestamp(3) without time zone | | | | plain | |
Access method: heap
Options: autovacuum_enabled=true, autovacuum_vacuum_threshold=1
update t2 set id=11 where id=1;
update t2 set id=12 where id=2;
t2表总共10行数据
autovacuum阈值=autovacuum_vacuum_threshold+autovacuum_vacuum_scale_factor*总行数=1+0.2*10=3条
已经更新两条数据后没有触发autovacuum，可以看到autovacuum_count字段为0
TEST=# select * from sys_stat_user_tables where relname='t2';
relid | schemaname | relname | seq_scan | seq_tup_read | idx_scan | idx_tup_fetch | n_tup_ins | n_tup_upd | n_tup_del | n_tup_hot_upd | n_live_tup | n_dead_tup | n_mod_since_analyze | last_va
cuum | last_autovacuum | last_analyze | last_autoanalyze | vacuum_count | autovacuum_count | analyze_count | autoanalyze_count
-------+------------+---------+----------+--------------+----------+---------------+-----------+-----------+-----------+---------------+------------+------------+---------------------+-----------------
--------------+-----------------+--------------+------------------+--------------+------------------+---------------+-------------------
16425 | public | t2 | 14 | 159 | | | 2 | 3 | 5 | 3 | 10 | 3 | 10 | 2023-04-26 15:10
:32.712381+08 | | | | 1 | 0 | 0 | 0
(1 row)
update t2 set id=13 where id=3;
此时触发autovacuum，就是说仅仅在表上设置了autovacuum_vacuum_threshold=1并不是更新一条记录后触发autovacuum，而是更新3条记录，因为此时触发了autovacuum阈值。
我们没有在表级设置autovacuum_vacuum_scale_factor，但是使用了默认的全局参数autovacuum_vacuum_scale_factor，值为0.2。
继续更新数据
update t2 set id=14 where id=4;
等待3min，超过参数autovacuum_naptime=1min，仍然没有触发autovacuum。
update t2 set id=15 where id=5;
update t2 set id=16 where id=6;
update t2 set id=17 where id=7;
这里看到更新到第4条后才触发的autovacuum
TEST=# select * from sys_stat_user_tables where relname='t2';
relid | schemaname | relname | seq_scan | seq_tup_read | idx_scan | idx_tup_fetch | n_tup_ins | n_tup_upd | n_tup_del | n_tup_hot_upd | n_live_tup | n_dead_tup | n_mod_since_analyze | last_vacuum
| last_autovacuum | last_analyze | last_autoanalyze | vacuum_count | autovacuum_count | analyze_count | autoanalyze_count
-------+------------+---------+----------+--------------+----------+---------------+-----------+-----------+-----------+---------------+------------+------------+---------------------+---------------------
----------+-------------------------------+--------------+------------------+--------------+------------------+---------------+-------------------
16428 | public | t2 | 15 | 164 | | | 20 | 12 | 0 | 12 | 10 | 4 | 32 | 2023-05-18 16:03:25.
276072+08 | 2023-05-18 16:16:08.123656+08 | | | 1 | 1 | 0 | 0
(1 row)
TEST=# select * from sys_stat_user_tables where relname='t2';
relid | schemaname | relname | seq_scan | seq_tup_read | idx_scan | idx_tup_fetch | n_tup_ins | n_tup_upd | n_tup_del | n_tup_hot_upd | n_live_tup | n_dead_tup | n_mod_since_analyze | last_vacuum
| last_autovacuum | last_analyze | last_autoanalyze | vacuum_count | autovacuum_count | analyze_count | autoanalyze_count
-------+------------+---------+----------+--------------+----------+---------------+-----------+-----------+-----------+---------------+------------+------------+---------------------+---------------------
----------+-------------------------------+--------------+------------------+--------------+------------------+---------------+-------------------
16428 | public | t2 | 15 | 164 | | | 20 | 12 | 0 | 12 | 10 | 0 | 32 | 2023-05-18 16:03:25.
276072+08 | 2023-05-18 16:27:08.561672+08 | | | 1 | 2 | 0 | 0
(1 row)
2.在表级设置只受一个参数autovacuum_vacuum_threshold影响，需要这样设置
添加条件
alter table t2 set(autovacuum_vacuum_scale_factor=0);
表级设置参数结果是这样的
TEST=# \d+ t2
Table "public.t2"
Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
------------+--------------------------------+-----------+----------+---------+---------+--------------+-------------
id | integer | | | | plain | |
start_time | timestamp(3) without time zone | | | | plain | |
Access method: heap
Options: autovacuum_enabled=true, autovacuum_vacuum_threshold=1, autovacuum_vacuum_scale_factor=0
这时只更新一条记录
不触发autovacuum。
TEST=# update t2 set id=18 where id=8;
UPDATE 1
TEST=#
TEST=# select * from sys_stat_user_tables where relname='t2';
relid | schemaname | relname | seq_scan | seq_tup_read | idx_scan | idx_tup_fetch | n_tup_ins | n_tup_upd | n_tup_del | n_tup_hot_upd | n_live_tup | n_dead_tup | n_mod_since_analyze | last_vacuum
| last_autovacuum | last_analyze | last_autoanalyze | vacuum_count | autovacuum_count | analyze_count | autoanalyze_count
-------+------------+---------+----------+--------------+----------+---------------+-----------+-----------+-----------+---------------+------------+------------+---------------------+---------------------
----------+-------------------------------+--------------+------------------+--------------+------------------+---------------+-------------------
16428 | public | t2 | 17 | 184 | | | 20 | 13 | 0 | 13 | 10 | 1 | 33 | 2023-05-18 16:03:25.
276072+08 | 2023-05-18 16:27:08.561672+08 | | | 1 | 2 | 0 | 0
(1 row)
又更新一条记录，n_dead_tup=2>autovacuum_vacuum_threshold ,这时候触发了autovacuum。
update t2 set id=19 where id=9;
TEST=# select * from sys_stat_user_tables where relname='t2';
relid | schemaname | relname | seq_scan | seq_tup_read | idx_scan | idx_tup_fetch | n_tup_ins | n_tup_upd | n_tup_del | n_tup_hot_upd | n_live_tup | n_dead_tup | n_mod_since_analyze | last_vacuum
| last_autovacuum | last_analyze | last_autoanalyze | vacuum_count | autovacuum_count | analyze_count | autoanalyze_count
-------+------------+---------+----------+--------------+----------+---------------+-----------+-----------+-----------+---------------+------------+------------+---------------------+---------------------
----------+-------------------------------+--------------+------------------+--------------+------------------+---------------+-------------------
16428 | public | t2 | 19 | 204 | | | 20 | 14 | 0 | 14 | 10 | 0 | 34 | 2023-05-18 16:03:25.
276072+08 | 2023-05-18 16:34:08.762470+08 | | | 1 | 3 | 0 | 0
(1 row)
3.另一个和analyze有关的两个参数autovacuum_analyze_threshold，autovacuum_analyze_scale_factor设置同理
需要表级设置两个参数，如果仅仅设置其中一个，另一个参数会使用默认的全局参数设置值
alter table t set(autovacuum_analyze_threshold=1);
alter table t set(autovacuum_analyze_scale_factor=0);
TEST=# \d+ t
Table "public.t"
Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
------------+--------------------------------+-----------+----------+---------+---------+--------------+-------------
id | integer | | | | plain | |
start_time | timestamp(3) without time zone | | | | plain | |
Access method: heap
Options: autovacuum_vacuum_threshold=1, autovacuum_enabled=true, autovacuum_vacuum_scale_factor=0, autovacuum_analyze_threshold=1, autovacuum_analyze_scale_factor=0
只更新一条记录
update t set id=21 where id=11;
查看结果last_autoanalyze是上次发生autoanalyze时间，autoanalyze_count=1，说明触发了autoanalyze。
TEST=# select * from sys_stat_user_tables where relname='t';
relid | schemaname | relname | seq_scan | seq_tup_read | idx_scan | idx_tup_fetch | n_tup_ins | n_tup_upd | n_tup_del | n_tup_hot_upd | n_live_tup | n_dead_tup | n_mod_since_analyze | last_v
acuum | last_autovacuum | last_analyze | last_autoanalyze | vacuum_count | autovacuum_count | analyze_count | autoanalyze_count
-------+------------+---------+----------+--------------+----------+---------------+-----------+-----------+-----------+---------------+------------+------------+---------------------+----------------
---------------+-------------------------------+--------------+-------------------------------+--------------+------------------+---------------+-------------------
16425 | public | t | 30 | 319 | | | 2 | 11 | 5 | 11 | 10 | 1 | 1 | 2023-04-26 15:1
0:32.712381+08 | 2023-04-26 16:05:50.740441+08 | | 2023-04-26 16:16:51.070583+08 | 1 | 3 | 0 | 1
(1 row)
4.怎样在表级别设置参数而不发生autoanalyze功能
只需要设置autovacuum_analyze_threshold>表的总行数(这里需要评估随着时间的推移，表数据增长的情况)。
alter table t set(autovacuum_analyze_threshold=25);
TEST=# \d+ t
Table "public.t"
Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
------------+--------------------------------+-----------+----------+---------+---------+--------------+-------------
id | integer | | | | plain | |
start_time | timestamp(3) without time zone | | | | plain | |
Access method: heap
Options: autovacuum_vacuum_threshold=1, autovacuum_enabled=true, autovacuum_vacuum_scale_factor=0, autovacuum_analyze_scale_factor=0, autovacuum_analyze_threshold=25
update t set id =30 where id<=21;
没有触发 autoanalyze
update t set id =40 where id=30;
总共更新了20行记录，仍然没更新autoanalyze
再次更新10行记录，总共更新30行记录，大于设置的阈值25，触发了autoanalyze。
update t set id =50 where id=40;
TEST=# select * from sys_stat_user_tables where relname='t';
relid | schemaname | relname | seq_scan | seq_tup_read | idx_scan | idx_tup_fetch | n_tup_ins | n_tup_upd | n_tup_del | n_tup_hot_upd | n_live_tup | n_dead_tup | n_mod_since_analyze | last_v
acuum | last_autovacuum | last_analyze | last_autoanalyze | vacuum_count | autovacuum_count | analyze_count | autoanalyze_count
-------+------------+---------+----------+--------------+----------+---------------+-----------+-----------+-----------+---------------+------------+------------+---------------------+----------------
---------------+-------------------------------+--------------+-------------------------------+--------------+------------------+---------------+-------------------
16425 | public | t | 34 | 359 | | | 2 | 41 | 5 | 41 | 10 | 0 | 0 | 2023-04-26 15:1
0:32.712381+08 | 2023-04-26 16:32:51.480662+08 | | 2023-04-26 16:32:51.484595+08 | 1 | 6 | 0 | 2
(1 row)
因此不触发autoanalyze只需要设置autovacuum_analyze_threshold成一个天文数字（以后插入，更新或者删除的元组永远不会达到这个数），同时设置 autovacuum_analyze_scale_factor=1。
总结
1.设置了表级的autovacuum_vacuum_threshold=1，还需要设置autovacuum_vacuum_scale_factor=0，这样大于一条死亡元祖触发autovacuum。
如果表级只设置autovacuum_vacuum_threshold=1，没有设置autovacuum_vacuum_scale_factor，则按照默认的全局参数autovacuum_vacuum_scale_factor=0.2，也就是死亡元祖大于0.2*总记录+autovacuum_vacuum_threshold触发autovacuum。
2.如果一个服务器有60个数据库并且autovacuum_naptime设置为60秒，那么自动清理工作器每秒启动一次[自动清理_naptime/DB数]。这里需要考虑数据库数量。
大量数据库场景
如果实例中有更多数据库，最好增大autovacuum_naptime。同时，可以通过增大autovacuum_cost_limit和减少autovacuum_cost_delay参数以及将autovacuum_max_workers的默认值从3增加到4或5来使自动清理过程更加激进。
KINGBASE研究院
