From: Tom Lane <tgl@sss.pgh.pa.us>
To: Andres Freund <andres@anarazel.de>
Cc: Tomas Vondra <tv@fuzzy.cz>
Cc: Simon Riggs <simon@2ndquadrant.com>
Cc: pgsql-committers <pgsql-committers@postgresql.org>
Subject: Re: pgsql: Generational memory allocator
Date: Sat, 25 Nov 2017 16:46:02 -0500
Message-ID: <2875.1511646362@sss.pgh.pa.us> (raw)
In-Reply-To: <21466.1511644760@sss.pgh.pa.us>
References: <E1eHa4J-0006hI-Q8@gemulon.postgresql.org>
<20171123001649.y2nf7422iay6cw3v@alap3.anarazel.de>
<CANP8+jKoT=Pg5W_3FcKiDf+FMrJhFYacxt_g5E0=tkmb5s8M6g@mail.gmail.com>
<c903f275-2150-fa52-64bf-dca7b53ebf8d@fuzzy.cz>
<16449.1511548313@sss.pgh.pa.us>
<4618.1511573138@sss.pgh.pa.us>
<bf84d940-90d4-de91-19dd-612e011007f4@fuzzy.cz>
<6189.1511587105@sss.pgh.pa.us>
<8991.1511639441@sss.pgh.pa.us>
<20171125200014.qbewtip5oydqsklt@alap3.anarazel.de>
<21466.1511644760@sss.pgh.pa.us>
I wrote:
> Instead I propose that we should make sure that the palloc request size
> for XLogReaderState->main_data is always maxalign'd. The existing
> behavior in DecodeXLogRecord of palloc'ing it only just barely big
> enough for the current record seems pretty brain-dead performance-wise
> even without this consideration. Generally, if we need to enlarge
> that buffer, we should enlarge it significantly, IMO.
I've confirmed that the attached is sufficient to stop the valgrind crash
on my machine. But as I said, I think we should be more aggressive at
resizing the buffer, to reduce resize cycles. I'm inclined to start out
with a buffer size of 128 or 256 or so bytes and double it when needed.
Anybody have a feeling for a typical size for the "main data" part
of a WAL record?
regards, tom lane
Attachments:
[text/x-diff] pad-xlog-buffer-size.patch (791B, ../2875.1511646362@sss.pgh.pa.us/2-pad-xlog-buffer-size.patch)
download | inline diff:diff --git a/src/backend/access/transam/xlogreader.c b/src/backend/access/transam/xlogreader.cindex aeaafed..4840b0e 100644
*** a/src/backend/access/transam/xlogreader.c
--- b/src/backend/access/transam/xlogreader.c
*************** DecodeXLogRecord(XLogReaderState *state,
*** 1279,1285 ****
{
if (state->main_data)
pfree(state->main_data);
! state->main_data_bufsz = state->main_data_len;
state->main_data = palloc(state->main_data_bufsz);
}
memcpy(state->main_data, ptr, state->main_data_len);
--- 1279,1285 ----
{
if (state->main_data)
pfree(state->main_data);
! state->main_data_bufsz = MAXALIGN(state->main_data_len);
state->main_data = palloc(state->main_data_bufsz);
}
memcpy(state->main_data, ptr, state->main_data_len);
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-committers@postgresql.org
Cc: tgl@sss.pgh.pa.us, andres@anarazel.de, tv@fuzzy.cz, simon@2ndquadrant.com
Subject: Re: pgsql: Generational memory allocator
In-Reply-To: <2875.1511646362@sss.pgh.pa.us>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox