本文主要是介绍PostgreSQL中vacuum 物理文件truncate发生的条件，希望对大家解决编程问题提供一定的参考价值，需要的开发者们随着小编来一起学习吧！
前言
前段时间，有些同学说到vacuum截断的行为时，认为，只要末尾是空页，无论多少，都会被截断，真是这样的吗？
PostgreSQL当中，由于vacuum的操作并不总能将死元组的空间进行”物理截断”，虽然说是回收了（表示空间可重用），但是真正的物理文件的大小依然不会收缩。那么什么时候这个空间会被真正截断呢？
我们不妨看看源代码：
摘自： src/backend/access/heap/vacuumlazy.c/** Space/time tradeoff parameters: do these need to be user-tunable?** To consider truncating the relation, we want there to be at least* REL_TRUNCATE_MINIMUM or (relsize / REL_TRUNCATE_FRACTION) (whichever* is less) potentially-freeable pages.*/
#define REL_TRUNCATE_MINIMUM 1000
#define REL_TRUNCATE_FRACTION 16/** should_attempt_truncation - should we attempt to truncate the heap?** Don't even think about it unless we have a shot at releasing a goodly* number of pages. Otherwise, the time taken isn't worth it, mainly because* an AccessExclusive lock must be replayed on any hot standby, where it can* be particularly disruptive.** Also don't attempt it if wraparound failsafe is in effect. The entire* system might be refusing to allocate new XIDs at this point. The system* definitely won't return to normal unless and until VACUUM actually advances* the oldest relfrozenxid -- which hasn't happened for target rel just yet.* If lazy_truncate_heap attempted to acquire an AccessExclusiveLock to* truncate the table under these circumstances, an XID exhaustion error might* make it impossible for VACUUM to fix the underlying XID exhaustion problem.* There is very little chance of truncation working out when the failsafe is* in effect in any case. lazy_scan_prune makes the optimistic assumption* that any LP_DEAD items it encounters will always be LP_UNUSED by the time* we're called.*/
static bool
should_attempt_truncation(LVRelState *vacrel)
{BlockNumber possibly_freeable;if (!vacrel->do_rel_truncate || VacuumFailsafeActive)return false;possibly_freeable = vacrel->rel_pages - vacrel->nonempty_pages;if (possibly_freeable > 0 &&(possibly_freeable >= REL_TRUNCATE_MINIMUM ||possibly_freeable >= vacrel->rel_pages / REL_TRUNCATE_FRACTION))return true;return false;
}
从代码里头可以看出，只有不少于1000个空页或者空页比例不低于总页数/16的情况下，才会真正发生truncate。两者必须满足其中之一。
这方面的原理分析，在cc老师的文章里都有谈及 （https://mp.weixin.qq.com/s/ymFYOAGin2kqo96gfNYDnQ），为加深印象，我们可以通过实验来进一步验证。
实验验证
-- 安装几个内置的插件：
create extension pg_buffercache;
create extension pg_freespacemap;
create extension pageinspect;
create extension pg_visibility;
create extension pgstattuple
准备表及数据
CREATE OR REPLACE FUNCTION random_string( int ) RETURNS TEXT as $$
SELECT string_agg(substring('abcdefghijklmnopqrstuvwxyz', round(random() * 25 + 0.5)::integer, 1), '') FROM generate_series(1, $1);
$$ language sql;postgres=# create table t(id int primary key, col2 varchar(4000));
CREATE TABLE
postgres=# insert into t select n, random_string(2000) from generate_series(1, 4000) as n;
INSERT 0 4000
相当于是建一个表t, 往里边插入4000条数据。里边使用随机串，主要是为了避免字符串压缩。我们看看相关表大小。
select pg_total_relation_size('t') total, pg_table_size('t') table, pg_indexes_size('t') indexes, pg_table_size('t')+pg_indexes_size('t') as sum, pg_relation_size('t') relation, pg_table_size('t')-pg_relation_size('t') as toast;total | table | indexes | sum | relation | toast
---------+---------+---------+---------+----------+-------8331264 | 8224768 | 106496 | 8331264 | 8192000 | 32768
(1 row)
我们看到，上边的空间大小主要集中在表本身。因为实际插入长度2000左右，还不会进入到Toast空间。
postgres=# select min(blkno), max(blkno) from pg_freespace('t');min | max
-----+-----0 | 999
(1 row)postgres=# select pg_relpages('t');pg_relpages
-------------1000
(1 row)postgres=# select * from pgstattuple('t') \gx
-[ RECORD 1 ]------+--------
table_len | 8192000
tuple_count | 4000
tuple_len | 8128000
tuple_percent | 99.22
dead_tuple_count | 0
dead_tuple_len | 0
dead_tuple_percent | 0
free_space | 20000
free_percent | 0.24
可以看出，这个表实际占用了1000个页面（每4条记录占用一页，符合预期）。
删除末尾的4条记录
我们就只删除末尾的4条记录，看下情况，相当于是最后一页。
postgres=# delete from t where id >= 3997;
DELETE 4
postgres=# select * from pgstattuple('t') \gx
-[ RECORD 1 ]------+--------
table_len | 8192000
tuple_count | 3996
tuple_len | 8119872
tuple_percent | 99.12
dead_tuple_count | 0
dead_tuple_len | 0
dead_tuple_percent | 0
free_space | 28128
free_percent | 0.34postgres=# vacuum verbose t;
INFO: vacuuming "public.t"
INFO: table "t": index scan bypassed: 1 pages from table (0.10% of total) have 4 dead item identifiers
INFO: table "t": found 0 removable, 0 nonremovable row versions in 1 out of 1000 pages
DETAIL: 0 dead row versions cannot be removed yet, oldest xmin: 223185
Skipped 0 pages due to buffer pins, 0 frozen pages.
CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s.
INFO: vacuuming "pg_toast.pg_toast_29708"
INFO: table "pg_toast_29708": found 0 removable, 0 nonremovable row versions in 0 out of 0 pages
DETAIL: 0 dead row versions cannot be removed yet, oldest xmin: 223186
Skipped 0 pages due to buffer pins, 0 frozen pages.
CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s.
VACUUM
postgres=# select * from pgstattuple('t') \gx
-[ RECORD 1 ]------+--------
table_len | 8192000
tuple_count | 3996
tuple_len | 8119872
tuple_percent | 99.12
dead_tuple_count | 0
dead_tuple_len | 0
dead_tuple_percent | 0
free_space | 28128
free_percent | 0.34
上边的信息也可以看到，并没有什么截断发生。统计一下物理空间：
select pg_total_relation_size('t') total, pg_table_size('t') table, pg_indexes_size('t') indexes, pg_table_size('t')+pg_indexes_size('t') as sum, pg_relation_size('t') relation, pg_table_size('t')-pg_relation_size('t') as toast;total | table | indexes | sum | relation | toast
---------+---------+---------+---------+----------+-------8339456 | 8232960 | 106496 | 8339456 | 8192000 | 40960
(1 row)
表的物理大小，仍然为：8192000
postgres=# select pg_relation_filepath('t');pg_relation_filepath
----------------------base/13236/29708\! stat postgres/data/base/13236/29708
16777234 104752459 -rw------- 1 ***** ***** 0 8192000 "Feb 4 06:40:12 2024" "Feb 4 06:44:03 2024" "Feb 4 06:44:03 2024" "Feb 4 06:40:12 2024" 4096 16512 0 postgres/data/base/13236/29708
继续删除12条记录：
postgres=# delete from t where id >= 4000 - 16 + 1;
DELETE 12
INFO: vacuuming "public.t"
INFO: table "t": index scan bypassed: 4 pages from table (0.40% of total) have 16 dead item identifiers
INFO: table "t": found 16 removable, 3984 nonremovable row versions in 1000 out of 1000 pages
DETAIL: 0 dead row versions cannot be removed yet, oldest xmin: 223205
Skipped 0 pages due to buffer pins, 0 frozen pages.
CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s.
INFO: vacuuming "pg_toast.pg_toast_29729"
INFO: table "pg_toast_29729": found 0 removable, 0 nonremovable row versions in 0 out of 0 pages
DETAIL: 0 dead row versions cannot be removed yet, oldest xmin: 223205
Skipped 0 pages due to buffer pins, 0 frozen pages.
CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s.
VACUUM
postgres=# select pg_total_relation_size('t') total, pg_table_size('t') table, pg_indexes_size('t') indexes, pg_table_size('t')+pg_indexes_size('t') as sum, pg_relation_size('t') relation, pg_table_size('t')-pg_relation_size('t') as toast;total | table | indexes | sum | relation | toast
---------+---------+---------+---------+----------+-------8339456 | 8232960 | 106496 | 8339456 | 8192000 | 40960
(1 row)
累计16个页面:
postgres=# delete from t where id >= 4000 - 64 + 1;
DELETE 32
postgres=# vacuum verbose t;
INFO: vacuuming "public.t"
INFO: table "t": index scan bypassed: 16 pages from table (1.60% of total) have 64 dead item identifiers
INFO: table "t": found 32 removable, 0 nonremovable row versions in 16 out of 1000 pages
DETAIL: 0 dead row versions cannot be removed yet, oldest xmin: 223209
Skipped 0 pages due to buffer pins, 0 frozen pages.
CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s.
INFO: vacuuming "pg_toast.pg_toast_29729"
INFO: table "pg_toast_29729": found 0 removable, 0 nonremovable row versions in 0 out of 0 pages
DETAIL: 0 dead row versions cannot be removed yet, oldest xmin: 223210
Skipped 0 pages due to buffer pins, 0 frozen pages.
CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s.
VACUUM
还是一点动静都没有。
那么要多少个空页，才有效：1000/16 = 62页？
我们不妨逐步推进这个实验，累计16一直往上，看到底多少次以后，开始有截断？
。。。。。。
发现，删除80条记录（20个页面），结果不变
删除84条记录时，结果仍不变。
删除88条记录的时候（22个页面），这个时候会截断20个页面。（INFO: table "t": truncated 1000 to 980 pages）
postgres=# delete from t where id >= 4000 - 88 + 1;
DELETE 8
postgres=# vacuum verbose t;
INFO: vacuuming "public.t"
INFO: table "t": index scan bypassed: 2 pages from table (0.20% of total) have 8 dead item identifiers
INFO: table "t": found 8 removable, 0 nonremovable row versions in 22 out of 1000 pages
DETAIL: 0 dead row versions cannot be removed yet, oldest xmin: 223218
Skipped 0 pages due to buffer pins, 0 frozen pages.
CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s.
INFO: table "t": truncated 1000 to 980 pages
DETAIL: CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s
INFO: vacuuming "pg_toast.pg_toast_29736"
INFO: table "pg_toast_29736": found 0 removable, 0 nonremovable row versions in 0 out of 0 pages
DETAIL: 0 dead row versions cannot be removed yet, oldest xmin: 223219
Skipped 0 pages due to buffer pins, 0 frozen pages.
CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s.
VACUUMpostgres=# select pg_relpages('t');pg_relpages
-------------980
(1 row)
事实上，我们发现，在删除87条记录的时候，仍然不会发生截断：
postgres=# delete from t where id >= 4000 - 87+1; vacuum verbose t;
DELETE 1
INFO: vacuuming "public.t"
INFO: table "t": index scan bypassed: 1 pages from table (0.10% of total) have 3 dead item identifiers
INFO: table "t": found 1 removable, 1 nonremovable row versions in 22 out of 1000 pages
DETAIL: 0 dead row versions cannot be removed yet, oldest xmin: 223228
Skipped 0 pages due to buffer pins, 0 frozen pages.
CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s.
INFO: vacuuming "pg_toast.pg_toast_29743"
INFO: table "pg_toast_29743": found 0 removable, 0 nonremovable row versions in 0 out of 0 pages
DETAIL: 0 dead row versions cannot be removed yet, oldest xmin: 223228
Skipped 0 pages due to buffer pins, 0 frozen pages.
CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s.
VACUUM
autovacuum禁掉，去除影响
我们禁掉autovacuum, 并进行连续操作
drop table t;create table t(id int, col2 varchar(4000)); alter table t SET (autovacuum_enabled = off); insert into t select n, random_string(2000) from generate_series(1, 4000) as n;postgres=# insert into t select n, random_string(2000) from generate_series(1, 4000) as n;
INSERT 0 4000
postgres=# delete from t where id >= 4000 - 88+1; vacuum verbose t;
DELETE 88
INFO: vacuuming "public.t"
INFO: table "t": removed 88 dead item identifiers in 22 pages
INFO: table "t": found 88 removable, 3912 nonremovable row versions in 1000 out of 1000 pages
DETAIL: 0 dead row versions cannot be removed yet, oldest xmin: 223240
Skipped 0 pages due to buffer pins, 0 frozen pages.
CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s.
INFO: vacuuming "pg_toast.pg_toast_29756"
INFO: table "pg_toast_29756": found 0 removable, 0 nonremovable row versions in 0 out of 0 pages
DETAIL: 0 dead row versions cannot be removed yet, oldest xmin: 223240
Skipped 0 pages due to buffer pins, 0 frozen pages.
CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s.
VACUUM
postgres=# select pg_relpages('t');pg_relpages
-------------1000
(1 row)
如果我们快速进行处理：
postgres=# drop table t;create table t(id int, col2 varchar(4000)); alter table t SET (autovacuum_enabled = off); insert into t select n, random_string(2000) from generate_series(1, 4000) as n;
DROP TABLE
CREATE TABLE
ALTER TABLE
INSERT 0 4000
postgres=# delete from t where id >= 4000 - 247+1; vacuum verbose t; select pg_relpages('t');
DELETE 247
INFO: vacuuming "public.t"
INFO: table "t": removed 247 dead item identifiers in 62 pages
INFO: table "t": found 247 removable, 3753 nonremovable row versions in 1000 out of 1000 pages
DETAIL: 0 dead row versions cannot be removed yet, oldest xmin: 223267
Skipped 0 pages due to buffer pins, 0 frozen pages.
CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s.
INFO: vacuuming "pg_toast.pg_toast_29776"
INFO: table "pg_toast_29776": found 0 removable, 0 nonremovable row versions in 0 out of 0 pages
DETAIL: 0 dead row versions cannot be removed yet, oldest xmin: 223267
Skipped 0 pages due to buffer pins, 0 frozen pages.
CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s.
VACUUMpg_relpages
-------------1000
(1 row)-- 再执行一次
postgres=# delete from t where id >= 4000 - 247+1; vacuum verbose t; select pg_relpages('t');
DELETE 0
INFO: vacuuming "public.t"
INFO: table "t": found 0 removable, 0 nonremovable row versions in 1 out of 1000 pages
DETAIL: 0 dead row versions cannot be removed yet, oldest xmin: 223267
Skipped 0 pages due to buffer pins, 60 frozen pages.
CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s.
INFO: table "t": truncated 1000 to 939 pages
DETAIL: CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s
INFO: vacuuming "pg_toast.pg_toast_29776"
INFO: table "pg_toast_29776": found 0 removable, 0 nonremovable row versions in 0 out of 0 pages
DETAIL: 0 dead row versions cannot be removed yet, oldest xmin: 223268
Skipped 0 pages due to buffer pins, 0 frozen pages.
CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s.
VACUUMpg_relpages
-------------939
(1 row)
这里会因为: Skipped 0 pages due to buffer pins, 60 frozen pages., 最后还是触发truncate. 但是从实验结果来看，在没有到达62个页面之前，第一次插，确实没有被trunate.
删除62个页面对应的记录：
postgres=# drop table t;create table t(id int primary key, col2 varchar(4000)); alter table t SET (autovacuum_enabled = off); insert into t select n, random_string(2000) from generate_series(1, 4000) as n;
DROP TABLE
CREATE TABLE
ALTER TABLE
INSERT 0 4000
postgres=# delete from t where id >= 4000 - 248+1; vacuum verbose t; select pg_relpages('t');
DELETE 248
INFO: vacuuming "public.t"
INFO: scanned index "t_pkey" to remove 248 row versions
DETAIL: CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s
INFO: table "t": removed 248 dead item identifiers in 62 pages
DETAIL: CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s
INFO: index "t_pkey" now contains 3752 row versions in 13 pages
DETAIL: 248 index row versions were removed.
0 index pages were newly deleted.
0 index pages are currently deleted, of which 0 are currently reusable.
CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s.
INFO: table "t": found 248 removable, 3752 nonremovable row versions in 1000 out of 1000 pages
DETAIL: 0 dead row versions cannot be removed yet, oldest xmin: 223284
Skipped 0 pages due to buffer pins, 0 frozen pages.
CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s.
INFO: table "t": truncated 1000 to 938 pages
DETAIL: CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s
INFO: vacuuming "pg_toast.pg_toast_29795"
INFO: table "pg_toast_29795": found 0 removable, 0 nonremovable row versions in 0 out of 0 pages
DETAIL: 0 dead row versions cannot be removed yet, oldest xmin: 223285
Skipped 0 pages due to buffer pins, 0 frozen pages.
CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s.
VACUUMpg_relpages
-------------938postgres=# select pg_total_relation_size('t') total, pg_table_size('t') table, pg_indexes_size('t') indexes, pg_table_size('t')+pg_indexes_size('t') as sum, pg_relation_size('t') relation, pg_table_size('t')-pg_relation_size('t') as toast;total | table | indexes | sum | relation | toast
---------+---------+---------+---------+----------+-------7831552 | 7725056 | 106496 | 7831552 | 7684096 | 40960
(1 row)
小结
vacuum一个表，产生物理截断，默认情况下，需要空页达到一定条件。上边的实验表明，在空页没达到1000个页同时也没达到总页数/16的情况下，第一次尝试截断，并不会真正发生。（autovacuum关闭，是为了屏蔽自动vacuum的影响）因为autovacuum还会触发freeze等其它动作，会间接产生影响。
上边的实验，禁掉auto vacuum, 在删除最后62个页面的情况下，会发生截断。
如果不禁掉auto vacuum， 你会发现在20个页面被删除的情况下，会发生截断，那刚好是autovacuum默认触发的条件（20%）。而它本身又会触发与freeze action相关的操作，最终会引发截断。
一个小实验，结合代码，可以引发很多思考。希望这个实验对vacuum原理感兴趣的人有所帮助。
与我联系：
这篇关于PostgreSQL中vacuum 物理文件truncate发生的条件的文章就介绍到这儿，希望我们推荐的文章对编程师们有所帮助！
