From: alvherre@alvh.no-ip.org <alvherre@alvh.no-ip.org>
To: Bossart, Nathan <bossartn@amazon.com>
Cc: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Cc: x4mmm@yandex-team.ru <x4mmm@yandex-team.ru>
Cc: a.lubennikova@postgrespro.ru <a.lubennikova@postgrespro.ru>
Cc: hlinnaka@iki.fi <hlinnaka@iki.fi>
Cc: matsumura.ryo@fujitsu.com <matsumura.ryo@fujitsu.com>
Cc: masao.fujii@gmail.com <masao.fujii@gmail.com>
Cc: pgsql-hackers@postgresql.org <pgsql-hackers@postgresql.org>
Subject: Re: archive status ".ready" files may be created too early
Date: Fri, 20 Aug 2021 10:50:18 -0400
Message-ID: <202108201450.6aao43wd3ju3@alvherre.pgsql> (raw)
In-Reply-To: <202108201433.dkqbqi7c6uw5@alvherre.pgsql>
Two things.
1. We use a hash table in shared memory. That's great. The part that's
not so great is that in both places where we read items from it, we
have to iterate in some way. This seems a bit silly. An array would
serve us better, if only we could expand it as needed. However, in
shared memory we can't do that. (I think the list of elements we
need to memoize is arbitrary long, if enough processes can be writing
WAL at the same time.)
Now that I think about this again, maybe it's limited by
NUM_XLOGINSERT_LOCKS, since there can only be that many records being
written down at a time ...
2. There is a new LWLock acquisition that may be a new contention point.
We acquire the lock in these cases:
- WAL record insert, when a segment boundary is crosses (rare
enough).
- XLogWrite, when a segment needs to be notified.
Looking again, I think the last point might be a problem actually,
because XLogWrite is called with WALWriteLock held. Maybe we should
take the NotifySegmentsReadyForArchive() call outside the section
locked by WALWriteLock (so put it in XLogWrite callers instead of
XLogWrite itself).
--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/
"Cómo ponemos nuestros dedos en la arcilla del otro. Eso es la amistad; jugar
al alfarero y ver qué formas se pueden sacar del otro" (C. Halloway en
La Feria de las Tinieblas, R. Bradbury)
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: alvherre@alvh.no-ip.org, bossartn@amazon.com, horikyota.ntt@gmail.com, x4mmm@yandex-team.ru, a.lubennikova@postgrespro.ru, hlinnaka@iki.fi, matsumura.ryo@fujitsu.com, masao.fujii@gmail.com
Subject: Re: archive status ".ready" files may be created too early
In-Reply-To: <202108201450.6aao43wd3ju3@alvherre.pgsql>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox