MIME-Version: 1.0
References: <CA+hUKG+2hZ0sBztPW4mkLfng0qfkNtAHFUfxOMLizJ0BPmi5+g@mail.gmail.com>
 <202403061129.gnbmxwwn5lem@alvherre.pgsql>
In-Reply-To: <202403061129.gnbmxwwn5lem@alvherre.pgsql>
From: Robert Haas <robertmhaas@gmail.com>
Date: Thu, 7 Mar 2024 12:54:07 -0500
Message-ID: <CA+TgmoZoHXOCWBUJuPP2g7U2_xSjRJAwum8kSRCRUTMesVfgoA@mail.gmail.com>
Subject: Re: Potential stack overflow in incremental base backup
To: Alvaro Herrera <alvherre@alvh.no-ip.org>
Cc: Thomas Munro <thomas.munro@gmail.com>, 
	PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Archived-At: <https://www.postgresql.org/message-id/CA%2BTgmoZoHXOCWBUJuPP2g7U2_xSjRJAwum8kSRCRUTMesVfgoA%40mail.gmail.com>
Precedence: bulk

On Wed, Mar 6, 2024 at 6:29=E2=80=AFAM Alvaro Herrera <alvherre@alvh.no-ip.=
org> wrote:
> On 2024-Mar-06, Thomas Munro wrote:
> > Even on the heap, 16GB is too much to assume we can allocate during a
> > base backup.  I don't claim that's a real-world problem for
> > incremental backup right now in master, because I don't have any
> > evidence that anyone ever really uses --with-segsize (do they?), but
> > if we make it an initdb option it will be more popular and this will
> > become a problem.  Hmm.
>
> Would it work to use a radix tree from the patchset at
> https://postgr.es/m/CANWCAZb43ZNRK03bzftnVRAfHzNGzH26sjc0Ep-sj8+w20VzSg@m=
ail.gmail.com
> ?

Probably not that much, because we actually send the array to the
client very soon after we construct it:

        push_to_sink(sink, &checksum_ctx, &header_bytes_done,
                     incremental_blocks,
                     sizeof(BlockNumber) * num_incremental_blocks);

This is hard to do without materializing the array somewhere, so I
don't think an alternate representation is the way to go in this
instance.

--=20
Robert Haas
EDB: http://www.enterprisedb.com