From: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
To: Tomas Vondra <tomas.vondra@2ndquadrant.com>
To: Greg Stark <stark@mit.edu>
Cc: PostgreSQL Hackers <pgsql-hackers@postgresql.org>
Subject: Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions
Date: Thu, 18 Jan 2018 08:24:06 -0500
Message-ID: <4d7ed7b8-df43-9b79-b04e-ad4c43ef519c@2ndquadrant.com> (raw)
In-Reply-To: <d72caaf1-a2da-081c-57ea-d30e5f47fd1e@2ndquadrant.com>
References: <688b0b7f-2f6c-d827-c27b-216a8e3ea700@2ndquadrant.com>
<ec3fbad0-06f6-08cb-7f0e-edd3fb0c2785@2ndquadrant.com>
<CAM-w4HNWQMnE8j9=Fa1QdwfKYs=XQanx03PL23yULbh1hPYSGQ@mail.gmail.com>
<5d00346a-b565-7fde-c6da-35f39c72936a@2ndquadrant.com>
<d72caaf1-a2da-081c-57ea-d30e5f47fd1e@2ndquadrant.com>
On 1/12/18 23:19, Tomas Vondra wrote:
> Wouldn't the 'toplevel_by_lsn' be suitable for this? Subtransactions
> don't really commit independently, but as part of the toplevel xact. And
> that list is ordered by LSN, which is pretty much exactly the order in
> which we see the transactions.
Yes indeed. There is even ReorderBufferGetOldestTXN().
> Another somewhat non-intuitive detail is that because ReorderBuffer
> switched to Generation allocator for changes (which usually represent
> 99% of the memory used during decoding), it does not reuse memory the
> way AllocSet does. Actually, it does not reuse memory at all, aiming to
> eventually give the memory back to libc (which AllocSet can't do).
>
> Because of this evicting the youngest transactions seems like a quite
> bad idea, because those chunks will not be reused and there may be other
> chunks on the blocks, preventing their release.
Right. But this raises the question whether we are doing the memory
accounting on the right level. If we are doing all this tracking based
on ReorderBufferChanges, but then serializing changes possibly doesn't
actually free any memory in the operating system, that's no good. Can
we get some usage statistics out of the memory context? It seems like
we need to keep serializing transactions until we actually see the
memory context size drop.
--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: peter.eisentraut@2ndquadrant.com, tomas.vondra@2ndquadrant.com, stark@mit.edu
Subject: Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions
In-Reply-To: <4d7ed7b8-df43-9b79-b04e-ad4c43ef519c@2ndquadrant.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox