Date: Wed, 30 Mar 2022 09:35:36 -0500
From: Justin Pryzby <pryzby@telsasoft.com>
To: Robert Haas <robertmhaas@gmail.com>
Cc: Jeevan Ladhe <jeevanladhe.os@gmail.com>,
	Dipesh Pandit <dipesh.pandit@gmail.com>,
	Andres Freund <andres@anarazel.de>,
	Alvaro Herrera <alvherre@alvh.no-ip.org>,
	"Shinoda, Noriyoshi (PN Japan FSIP)" <noriyoshi.shinoda@hpe.com>,
	Abhijit Menon-Sen <ams@toroid.org>,
	Dmitry Dolgov <9erthalion6@gmail.com>,
	Jeevan Ladhe <jeevan.ladhe@enterprisedb.com>,
	Mark Dilger <mark.dilger@enterprisedb.com>,
	pgsql-hackers@postgresql.org,
	tushar <tushar.ahuja@enterprisedb.com>
Subject: basebackup/lz4 crash
Message-ID: <20220330143536.GG28503@telsasoft.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.9.4 (2018-02-28)
Archived-At: 
 <https://www.postgresql.org/message-id/20220330143536.GG28503%40telsasoft.com>
Precedence: bulk

Forking: <20220316151253.GB28503@telsasoft.com>

On Wed, Mar 16, 2022 at 10:12:54AM -0500, Justin Pryzby wrote:
> Also, with a partial regression DB, this crashes when writing to stdout.
> 
> $ src/bin/pg_basebackup/pg_basebackup --wal-method fetch -Ft -D - -h /tmp --no-sync --compress=lz4 |wc -c
> pg_basebackup: bbstreamer_lz4.c:172: bbstreamer_lz4_compressor_content: Assertion `mystreamer->base.bbs_buffer.maxlen >= out_bound' failed.
> 24117248
> 
> #4  0x000055555555e8b4 in bbstreamer_lz4_compressor_content (streamer=0x5555555a5260, member=0x7fffffffc760, 
>     data=0x7ffff3068010 "{ \"PostgreSQL-Backup-Manifest-Version\": 1,\n\"Files\": [\n{ \"Path\": \"backup_label\", \"Size\": 227, \"Last-Modified\": \"2022-03-16 02:29:11 GMT\", \"Checksum-Algorithm\": \"CRC32C\", \"Checksum\": \"46f69d99\" },\n{ \"Pa"..., len=401072, context=BBSTREAMER_MEMBER_CONTENTS) at bbstreamer_lz4.c:172
>         mystreamer = 0x5555555a5260
>         next_in = 0x7ffff3068010 "{ \"PostgreSQL-Backup-Manifest-Version\": 1,\n\"Files\": [\n{ \"Path\": \"backup_label\", \"Size\": 227, \"Last-Modified\": \"2022-03-16 02:29:11 GMT\", \"Checksum-Algorithm\": \"CRC32C\", \"Checksum\": \"46f69d99\" },\n{ \"Pa"...
>         ...
> 
> (gdb) p mystreamer->base.bbs_buffer.maxlen
> $1 = 524288
> (gdb) p (int) LZ4F_compressBound(len, &mystreamer->prefs)
> $4 = 524300
> 
> This is with: liblz4-1:amd64 1.9.2-2ubuntu0.20.04.1

It looks like maybe this code was copied from
bbstreamer_lz4_compressor_finalize() which has an Assert rather than expanding
the buffer like in bbstreamer_lz4_compressor_new()

commit e70c12214b5ba0bc93c083fdb046304a633018ef
Author: Justin Pryzby <pryzbyj@telsasoft.com>
Date:   Mon Mar 28 23:24:15 2022 -0500

    basebackup: fix crash with lz4 + stdout + manifests
    
    That's just one known way to trigger this issue.

diff --git a/src/bin/pg_basebackup/bbstreamer_lz4.c b/src/bin/pg_basebackup/bbstreamer_lz4.c
index 67f841d96a9..8e8352a450c 100644
--- a/src/bin/pg_basebackup/bbstreamer_lz4.c
+++ b/src/bin/pg_basebackup/bbstreamer_lz4.c
@@ -170,7 +170,11 @@ bbstreamer_lz4_compressor_content(bbstreamer *streamer,
 	 * forward the content to next streamer and empty the buffer.
 	 */
 	out_bound = LZ4F_compressBound(len, &mystreamer->prefs);
-	Assert(mystreamer->base.bbs_buffer.maxlen >= out_bound);
+
+	/* Enlarge buffer if it falls short of compression bound. */
+	if (mystreamer->base.bbs_buffer.maxlen < out_bound)
+		enlargeStringInfo(&mystreamer->base.bbs_buffer, out_bound);
+
 	if (avail_out < out_bound)
 	{
 			bbstreamer_content(mystreamer->base.bbs_next, member,