pg.ddx.io  pgsql-committers@postgresql.org mailing list archive  

help / color / mirror / Atom feed
From: Tom Lane <tgl@sss.pgh.pa.us>
To: Simon Riggs <simon@2ndquadrant.com>
Cc: Andres Freund <andres@anarazel.de>
Cc: Tomas Vondra <tv@fuzzy.cz>
Cc: pgsql-committers <pgsql-committers@postgresql.org>
Subject: Re: pgsql: Generational memory allocator
Date: Sun, 26 Nov 2017 12:46:46 -0500
Message-ID: <16797.1511718406@sss.pgh.pa.us> (raw)
In-Reply-To: <CANP8+jKRt0Yenw8Ln4J4u=UsDVkCBgPmjDE2buYou-H9VTMZjw@mail.gmail.com>
References: <E1eHa4J-0006hI-Q8@gemulon.postgresql.org>
	<20171123001649.y2nf7422iay6cw3v@alap3.anarazel.de>
	<CANP8+jKoT=Pg5W_3FcKiDf+FMrJhFYacxt_g5E0=tkmb5s8M6g@mail.gmail.com>
	<c903f275-2150-fa52-64bf-dca7b53ebf8d@fuzzy.cz>
	<16449.1511548313@sss.pgh.pa.us>
	<4618.1511573138@sss.pgh.pa.us>
	<bf84d940-90d4-de91-19dd-612e011007f4@fuzzy.cz>
	<6189.1511587105@sss.pgh.pa.us>
	<8991.1511639441@sss.pgh.pa.us>
	<20171125200014.qbewtip5oydqsklt@alap3.anarazel.de>
	<21466.1511644760@sss.pgh.pa.us>
	<2875.1511646362@sss.pgh.pa.us>
	<CANP8+jKRt0Yenw8Ln4J4u=UsDVkCBgPmjDE2buYou-H9VTMZjw@mail.gmail.com>

Simon Riggs <simon@2ndquadrant.com> writes:
> On 26 November 2017 at 08:46, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> I've confirmed that the attached is sufficient to stop the valgrind crash
>> on my machine.  But as I said, I think we should be more aggressive at
>> resizing the buffer, to reduce resize cycles.  I'm inclined to start out
>> with a buffer size of 128 or 256 or so bytes and double it when needed.
>> Anybody have a feeling for a typical size for the "main data" part
>> of a WAL record?

> We reuse the buffer and only pfree/palloc when we need to enlarge the
> buffer, so not sure we need to do the doubling thing and it probably
> doesn't matter what the typical size is.

Well, I'm concerned about the possibility of a lot of palloc thrashing
if the first bunch of records it reads happen to have steadily increasing
sizes.  However, rather than doubling, it might be sufficient to set a
robust minimum on the first allocation, ie use something along the lines
of Max(1024, MAXALIGN(state->main_data_len)).

			regards, tom lane




view thread (37+ messages)  latest in thread

Message-ID: <16797.1511718406@sss.pgh.pa.us>
Permalink:  ../16797.1511718406@sss.pgh.pa.us/
Also on:    postgresql.org/message-id/16797.1511718406@sss.pgh.pa.us

 · 

reply
Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-committers@postgresql.org
  Cc: tgl@sss.pgh.pa.us, simon@2ndquadrant.com, andres@anarazel.de, tv@fuzzy.cz
  Subject: Re: pgsql: Generational memory allocator
  In-Reply-To: <16797.1511718406@sss.pgh.pa.us>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox