From: Tom Lane <tgl@sss.pgh.pa.us>
To: Andres Freund <andres@anarazel.de>
cc: Tomas Vondra <tv@fuzzy.cz>, Simon Riggs <simon@2ndquadrant.com>,
        pgsql-committers <pgsql-committers@postgresql.org>
Subject: Re: pgsql: Generational memory allocator
In-reply-to: <21466.1511644760@sss.pgh.pa.us>
References: <E1eHa4J-0006hI-Q8@gemulon.postgresql.org>
 <20171123001649.y2nf7422iay6cw3v@alap3.anarazel.de>
 <CANP8+jKoT=Pg5W_3FcKiDf+FMrJhFYacxt_g5E0=tkmb5s8M6g@mail.gmail.com>
 <c903f275-2150-fa52-64bf-dca7b53ebf8d@fuzzy.cz>
 <16449.1511548313@sss.pgh.pa.us> <4618.1511573138@sss.pgh.pa.us>
 <bf84d940-90d4-de91-19dd-612e011007f4@fuzzy.cz>
 <6189.1511587105@sss.pgh.pa.us> <8991.1511639441@sss.pgh.pa.us>
 <20171125200014.qbewtip5oydqsklt@alap3.anarazel.de>
 <21466.1511644760@sss.pgh.pa.us>
Comments: In-reply-to Tom Lane <tgl@sss.pgh.pa.us>
	message dated "Sat, 25 Nov 2017 16:19:20 -0500"
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="----- =_aaaaaaaaaa0"
Content-ID: <2833.1511646307.0@sss.pgh.pa.us>
Date: Sat, 25 Nov 2017 16:46:02 -0500
Message-ID: <2875.1511646362@sss.pgh.pa.us>
Precedence: bulk

------- =_aaaaaaaaaa0
Content-Type: text/plain; charset="us-ascii"
Content-ID: <2833.1511646307.1@sss.pgh.pa.us>

I wrote:
> Instead I propose that we should make sure that the palloc request size
> for XLogReaderState->main_data is always maxalign'd.  The existing
> behavior in DecodeXLogRecord of palloc'ing it only just barely big
> enough for the current record seems pretty brain-dead performance-wise
> even without this consideration.  Generally, if we need to enlarge
> that buffer, we should enlarge it significantly, IMO.

I've confirmed that the attached is sufficient to stop the valgrind crash
on my machine.  But as I said, I think we should be more aggressive at
resizing the buffer, to reduce resize cycles.  I'm inclined to start out
with a buffer size of 128 or 256 or so bytes and double it when needed.
Anybody have a feeling for a typical size for the "main data" part
of a WAL record?

			regards, tom lane


------- =_aaaaaaaaaa0
Content-Type: text/x-diff; name="pad-xlog-buffer-size.patch";
	charset="us-ascii"
Content-ID: <2833.1511646307.2@sss.pgh.pa.us>
Content-Description: pad-xlog-buffer-size.patch
Content-Transfer-Encoding: quoted-printable

diff --git a/src/backend/access/transam/xlogreader.c b/src/backend/access/=
transam/xlogreader.c
index aeaafed..4840b0e 100644
*** a/src/backend/access/transam/xlogreader.c
--- b/src/backend/access/transam/xlogreader.c
*************** DecodeXLogRecord(XLogReaderState *state,
*** 1279,1285 ****
  		{
  			if (state->main_data)
  				pfree(state->main_data);
! 			state->main_data_bufsz =3D state->main_data_len;
  			state->main_data =3D palloc(state->main_data_bufsz);
  		}
  		memcpy(state->main_data, ptr, state->main_data_len);
--- 1279,1285 ----
  		{
  			if (state->main_data)
  				pfree(state->main_data);
! 			state->main_data_bufsz =3D MAXALIGN(state->main_data_len);
  			state->main_data =3D palloc(state->main_data_bufsz);
  		}
  		memcpy(state->main_data, ptr, state->main_data_len);

------- =_aaaaaaaaaa0--