From: Tom Lane <tgl@sss.pgh.pa.us>
To: =?UTF-8?Q?Juan_Jos=C3=A9_Santamar=C3=ADa_Flecha?=
 <juanjo.santamaria@gmail.com>
cc: egashira.yusuke@fujitsu.com,
        PostgreSQL mailing lists <pgsql-bugs@lists.postgresql.org>
Subject: Re: BUG #17254: Crash with 0xC0000409 in pg_stat_statements when
 pg_stat_tmp\pgss_query_texts.stat exceeded 2GB.
In-reply-to: 
 <CAC+AXB2tWxoZgj6BnoQaxiWw=vKBxUPtyBn7d8jhHRqxv7xRRw@mail.gmail.com>
References: <17254-a926c89dc03375c2@postgresql.org>
 <CAC+AXB21yF41p4fWgUOkG1PBA-Ezw-8U0Q=Bint9OQ56qyMPYw@mail.gmail.com>
 <CAC+AXB0fozNPMgRTP+PgFMrGcSn2zER0zgcrEbRV0NvpZqmE=A@mail.gmail.com>
 <CAC+AXB2tWxoZgj6BnoQaxiWw=vKBxUPtyBn7d8jhHRqxv7xRRw@mail.gmail.com>
Comments: In-reply-to =?UTF-8?Q?Juan_Jos=C3=A9_Santamar=C3=ADa_Flecha?=
 <juanjo.santamaria@gmail.com>
	message dated "Sat, 30 Oct 2021 10:27:29 +0200"
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-ID: <840435.1635611203.1@sss.pgh.pa.us>
Content-Transfer-Encoding: quoted-printable
Date: Sat, 30 Oct 2021 12:26:43 -0400
Message-ID: <840436.1635611203@sss.pgh.pa.us>
Archived-At: 
 <https://www.postgresql.org/message-id/840436.1635611203%40sss.pgh.pa.us>
Precedence: bulk

=3D?UTF-8?Q?Juan_Jos=3DC3=3DA9_Santamar=3DC3=3DADa_Flecha?=3D <juanjo.sant=
amaria@gmail.com> writes:
> Now, with 100% more patch attached.

That seems like a pretty poor solution.  It will cause pg_stat_statements
to fail altogether as soon as the stats file exceeds 1GB.  (Admittedly,
failing is better than crashing, but not by that much.)  Worse, it causes
that to happen on EVERY platform, not only Windows where the problem is.

I think instead, we need to turn the subsequent one-off read() call into a
loop that reads no more than INT_MAX bytes at a time.  It'd be possible
to restrict that to Windows, but probably no harm in doing it the same
way everywhere.

A different line of thought is that maybe we shouldn't be letting the
file get so big in the first place.  Letting every backend have its
own copy of a multi-gigabyte stats file is going to be problematic,
and not only on Windows.  It looks like the existing logic just considers
the number of hash table entries, not their size ... should we rearrange
things to keep a running count of the space used?

			regards, tom lane