To: pgsql-hackers@postgresql.org
From: Fujii Masao <masao.fujii@oss.nttdata.com>
Subject: FailedAssertion at ReorderBufferCheckMemoryLimit()
Message-ID: <2b7ba291-22e0-a187-d167-9e5309a3458d@oss.nttdata.com>
Date: Tue, 9 Jun 2020 17:26:24 +0900
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:68.0)
 Gecko/20100101 Thunderbird/68.9.0
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Precedence: bulk

Hi,

I encountered the following assertion failure when I changed
logical_decoding_work_mem to lower value while logical replication
is running. This happend in the master branch.

TRAP: FailedAssertion("rb->size < logical_decoding_work_mem * 1024L", File: "reorderbuffer.c", Line: 2403)
0   postgres                            0x000000010755bf80 ExceptionalCondition + 160
1   postgres                            0x00000001072d9f81 ReorderBufferCheckMemoryLimit + 257
2   postgres                            0x00000001072d9b74 ReorderBufferQueueChange + 228
3   postgres                            0x00000001072cd107 DecodeInsert + 391
4   postgres                            0x00000001072cc4ef DecodeHeapOp + 335
5   postgres                            0x00000001072cb9e4 LogicalDecodingProcessRecord + 196
6   postgres                            0x000000010730bf06 XLogSendLogical + 166
7   postgres                            0x000000010730b409 WalSndLoop + 217
8   postgres                            0x00000001073075fc StartLogicalReplication + 716
9   postgres                            0x0000000107305d38 exec_replication_command + 1192
10  postgres                            0x000000010737ea7f PostgresMain + 2463
11  postgres                            0x00000001072a9c4a BackendRun + 570
12  postgres                            0x00000001072a907b BackendStartup + 475
13  postgres                            0x00000001072a7fe1 ServerLoop + 593
14  postgres                            0x00000001072a5a5a PostmasterMain + 5898
15  postgres                            0x0000000107187b59 main + 761
16  libdyld.dylib                       0x00007fff6c00c3d5 start + 1


ReorderBufferCheckMemoryLimit() explains that it relies on
the following (commented) assumption. But this seems incorrect
when logical_decoding_work_mem is decreased. I wonder if we may
need to keep evicting the transactions until we don't exceed
memory limit.

	/*
	 * And furthermore, evicting the transaction should get us below the
	 * memory limit again - it is not possible that we're still exceeding the
	 * memory limit after evicting the transaction.
	 *
	 * This follows from the simple fact that the selected transaction is at
	 * least as large as the most recent change (which caused us to go over
	 * the memory limit). So by evicting it we're definitely back below the
	 * memory limit.
	 */
	Assert(rb->size < logical_decoding_work_mem * 1024L);

Regards,

-- 
Fujii Masao
Advanced Computing Technology Center
Research and Development Headquarters
NTT DATA CORPORATION