pg.ddx.io pgsql-committers@postgresql.org mailing list archive
help / color / mirror / Atom feedpgsql: xlog.c: Remove global variables ReadRecPtr and EndRecPtr.
11+ messages / 5 participants
[nested] [flat]
Commits from this thread:
2021-11-25 3030903dfe Replace straggling uses of ReadRecPtr/EndRecPtr.
* pgsql: xlog.c: Remove global variables ReadRecPtr and EndRecPtr.
@ 2021-11-24 16:27 Robert Haas <rhaas@postgresql.org>
2021-11-24 18:16 ` Re: pgsql: xlog.c: Remove global variables ReadRecPtr and EndRecPtr. Robert Haas <robertmhaas@gmail.com>
2021-11-24 23:12 ` Re: pgsql: xlog.c: Remove global variables ReadRecPtr and EndRecPtr. Andres Freund <andres@anarazel.de>
0 siblings, 2 replies; 11+ messages in thread
From: Robert Haas @ 2021-11-24 16:27 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
xlog.c: Remove global variables ReadRecPtr and EndRecPtr.
In most places, the variables necessarily store the same value as the
eponymous members of the XLogReaderState that we use during WAL
replay, because ReadRecord() assigns the values from the structure
members to the global variables just after XLogReadRecord() returns.
However, XLogBeginRead() adjusts the structure members but not the
global variables, so after XLogBeginRead() and before the completion
of XLogReadRecord() the values can differ. Otherwise, they must be
identical. According to my analysis, the only place where either
variable is referenced at a point where it might not have the same
value as the structure member is the refrence to EndRecPtr within
XLogPageRead.
Therefore, at every other place where we are using the global
variable, we can just switch to using the structure member instead,
and remove the global variable. However, we can, and in fact should,
do this in XLogPageRead() as well, because at that point in the code,
the global variable will actually store the start of the record we
want to read - either because it's where the last WAL record ended, or
because the read position has been changed using XLogBeginRead since
the last record was read. The structure member, on the other hand,
will already have been updated to point to the end of the record we
just read. Elsewhere, the latter is what we use as an argument to
emode_for_corrupt_record(), so we should do the same here.
This part of the patch is perhaps a bug fix, but I don't think it has
any important consequences, so no back-patch. The point here is just
to continue to whittle down the entirely excessive use of global
variables in xlog.c.
Discussion: http://postgr.es/m/CA+Tgmoao96EuNeSPd+hspRKcsCddu=b1h-QNRuKfY8VmfNQdfg@mail.gmail.com
Branch
------
master
Details
-------
https://git.postgresql.org/pg/commitdiff/d2ddfa681db27a138acb63c8defa8cc6fa588922
Modified Files
--------------
src/backend/access/transam/xlog.c | 53 ++++++++++++++++++---------------------
1 file changed, 24 insertions(+), 29 deletions(-)
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: pgsql: xlog.c: Remove global variables ReadRecPtr and EndRecPtr.
2021-11-24 16:27 pgsql: xlog.c: Remove global variables ReadRecPtr and EndRecPtr. Robert Haas <rhaas@postgresql.org>
@ 2021-11-24 18:16 ` Robert Haas <robertmhaas@gmail.com>
2021-11-24 18:46 ` Re: pgsql: xlog.c: Remove global variables ReadRecPtr and EndRecPtr. Tom Lane <tgl@sss.pgh.pa.us>
2021-11-24 18:51 ` Re: pgsql: xlog.c: Remove global variables ReadRecPtr and EndRecPtr. Alvaro Herrera <alvherre@alvh.no-ip.org>
1 sibling, 2 replies; 11+ messages in thread
From: Robert Haas @ 2021-11-24 18:16 UTC (permalink / raw)
To: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On Wed, Nov 24, 2021 at 11:28 AM Robert Haas <rhaas@postgresql.org> wrote:
> xlog.c: Remove global variables ReadRecPtr and EndRecPtr.
> https://git.postgresql.org/pg/commitdiff/d2ddfa681db27a138acb63c8defa8cc6fa588922
There is a buildfarm failure on lapwing which looks likely to be
attributable to this commit, but I don't understand what has gone
wrong exactly. The error message that is reported is:
[17:07:19] t/025_stuck_on_old_timeline.pl ....... ok 6048 ms
# poll_query_until timed out executing this query:
# SELECT '0/201E8E0'::pg_lsn <= pg_last_wal_replay_lsn()
# expecting this output:
# t
# last actual query output:
#
# with stderr:
# psql: error: connection to server on socket
"/tmp/SrZekgRh7K/.s.PGSQL.57959" failed: No such file or directory
# Is the server running locally and accepting connections on that socket?
This output confused me for a while because I thought the failure was
in test 025, but I should have realized that the problem must be in
026, since 025 finished with "ok". Anyway, in
026_overwrite_contrecord_standby.log there's this:
2021-11-24 17:07:41.388 UTC [1473:1] LOG: started streaming WAL from
primary at 0/2000000 on timeline 1
2021-11-24 17:07:41.428 UTC [1449:6] FATAL: mismatching overwritten
LSN 0/1FFE014 -> 0/1FFE000
2021-11-24 17:07:41.428 UTC [1449:7] CONTEXT: WAL redo at 0/2000024
for XLOG/OVERWRITE_CONTRECORD: lsn 0/1FFE014; time 2021-11-24
17:07:41.127049+00
Beyond the fact that something bad has happened, I'm not sure what
that is trying to tell me.
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: pgsql: xlog.c: Remove global variables ReadRecPtr and EndRecPtr.
2021-11-24 16:27 pgsql: xlog.c: Remove global variables ReadRecPtr and EndRecPtr. Robert Haas <rhaas@postgresql.org>
2021-11-24 18:16 ` Re: pgsql: xlog.c: Remove global variables ReadRecPtr and EndRecPtr. Robert Haas <robertmhaas@gmail.com>
@ 2021-11-24 18:46 ` Tom Lane <tgl@sss.pgh.pa.us>
1 sibling, 0 replies; 11+ messages in thread
From: Tom Lane @ 2021-11-24 18:46 UTC (permalink / raw)
To: Robert Haas <robertmhaas@gmail.com>; +Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Robert Haas <robertmhaas@gmail.com> writes:
> 2021-11-24 17:07:41.428 UTC [1449:6] FATAL: mismatching overwritten
> LSN 0/1FFE014 -> 0/1FFE000
> 2021-11-24 17:07:41.428 UTC [1449:7] CONTEXT: WAL redo at 0/2000024
> for XLOG/OVERWRITE_CONTRECORD: lsn 0/1FFE014; time 2021-11-24
> 17:07:41.127049+00
> Beyond the fact that something bad has happened, I'm not sure what
> that is trying to tell me.
Pre-existing issue:
https://www.postgresql.org/message-id/45597.1637694259%40sss.pgh.pa.us
regards, tom lane
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: pgsql: xlog.c: Remove global variables ReadRecPtr and EndRecPtr.
2021-11-24 16:27 pgsql: xlog.c: Remove global variables ReadRecPtr and EndRecPtr. Robert Haas <rhaas@postgresql.org>
2021-11-24 18:16 ` Re: pgsql: xlog.c: Remove global variables ReadRecPtr and EndRecPtr. Robert Haas <robertmhaas@gmail.com>
@ 2021-11-24 18:51 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2021-11-24 18:58 ` Re: pgsql: xlog.c: Remove global variables ReadRecPtr and EndRecPtr. Robert Haas <robertmhaas@gmail.com>
1 sibling, 1 reply; 11+ messages in thread
From: Alvaro Herrera @ 2021-11-24 18:51 UTC (permalink / raw)
To: Robert Haas <robertmhaas@gmail.com>; +Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On 2021-Nov-24, Robert Haas wrote:
> On Wed, Nov 24, 2021 at 11:28 AM Robert Haas <rhaas@postgresql.org> wrote:
> > xlog.c: Remove global variables ReadRecPtr and EndRecPtr.
> > https://git.postgresql.org/pg/commitdiff/d2ddfa681db27a138acb63c8defa8cc6fa588922
>
> There is a buildfarm failure on lapwing which looks likely to be
> attributable to this commit, but I don't understand what has gone
> wrong exactly. The error message that is reported is:
No, this is an unrelated problem; Tom reported this yesterday also:
https://postgr.es/m/45597.1637694259@sss.pgh.pa.us
--
Álvaro Herrera Valdivia, Chile — https://www.EnterpriseDB.com/
"La espina, desde que nace, ya pincha" (Proverbio africano)
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: pgsql: xlog.c: Remove global variables ReadRecPtr and EndRecPtr.
2021-11-24 16:27 pgsql: xlog.c: Remove global variables ReadRecPtr and EndRecPtr. Robert Haas <rhaas@postgresql.org>
2021-11-24 18:16 ` Re: pgsql: xlog.c: Remove global variables ReadRecPtr and EndRecPtr. Robert Haas <robertmhaas@gmail.com>
2021-11-24 18:51 ` Re: pgsql: xlog.c: Remove global variables ReadRecPtr and EndRecPtr. Alvaro Herrera <alvherre@alvh.no-ip.org>
@ 2021-11-24 18:58 ` Robert Haas <robertmhaas@gmail.com>
0 siblings, 0 replies; 11+ messages in thread
From: Robert Haas @ 2021-11-24 18:58 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On Wed, Nov 24, 2021 at 1:51 PM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> > There is a buildfarm failure on lapwing which looks likely to be
> > attributable to this commit, but I don't understand what has gone
> > wrong exactly. The error message that is reported is:
>
> No, this is an unrelated problem; Tom reported this yesterday also:
> https://postgr.es/m/45597.1637694259@sss.pgh.pa.us
Oh, OK. I couldn't see exactly what that could have to do with my
commit, but it happened right afterwards so I was hesitant to call it
a coincidence.
Thanks,
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: pgsql: xlog.c: Remove global variables ReadRecPtr and EndRecPtr.
2021-11-24 16:27 pgsql: xlog.c: Remove global variables ReadRecPtr and EndRecPtr. Robert Haas <rhaas@postgresql.org>
@ 2021-11-24 23:12 ` Andres Freund <andres@anarazel.de>
2021-11-25 01:01 ` Re: pgsql: xlog.c: Remove global variables ReadRecPtr and EndRecPtr. Andres Freund <andres@anarazel.de>
1 sibling, 1 reply; 11+ messages in thread
From: Andres Freund @ 2021-11-24 23:12 UTC (permalink / raw)
To: Robert Haas <rhaas@postgresql.org>; +Cc: pgsql-committers@lists.postgresql.org
Hi,
On 2021-11-24 16:27:58 +0000, Robert Haas wrote:
> xlog.c: Remove global variables ReadRecPtr and EndRecPtr.
This fails when building with WAL_DEBUG. There's remaining Read/EndRecPtr
references in #ifdef WAL_DEBUG sections...
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: pgsql: xlog.c: Remove global variables ReadRecPtr and EndRecPtr.
2021-11-24 16:27 pgsql: xlog.c: Remove global variables ReadRecPtr and EndRecPtr. Robert Haas <rhaas@postgresql.org>
2021-11-24 23:12 ` Re: pgsql: xlog.c: Remove global variables ReadRecPtr and EndRecPtr. Andres Freund <andres@anarazel.de>
@ 2021-11-25 01:01 ` Andres Freund <andres@anarazel.de>
2021-11-26 01:30 ` Re: pgsql: xlog.c: Remove global variables ReadRecPtr and EndRecPtr. Robert Haas <robertmhaas@gmail.com>
0 siblings, 1 reply; 11+ messages in thread
From: Andres Freund @ 2021-11-25 01:01 UTC (permalink / raw)
To: Robert Haas <rhaas@postgresql.org>; +Cc: pgsql-committers@lists.postgresql.org
Hi,
On 2021-11-24 15:12:06 -0800, Andres Freund wrote:
> This fails when building with WAL_DEBUG. There's remaining Read/EndRecPtr
> references in #ifdef WAL_DEBUG sections...
Pushed the obvious fix for that. Somehow thought I'd seen more compile failure
than the one WAL_DEBUG...
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: pgsql: xlog.c: Remove global variables ReadRecPtr and EndRecPtr.
2021-11-24 16:27 pgsql: xlog.c: Remove global variables ReadRecPtr and EndRecPtr. Robert Haas <rhaas@postgresql.org>
2021-11-24 23:12 ` Re: pgsql: xlog.c: Remove global variables ReadRecPtr and EndRecPtr. Andres Freund <andres@anarazel.de>
2021-11-25 01:01 ` Re: pgsql: xlog.c: Remove global variables ReadRecPtr and EndRecPtr. Andres Freund <andres@anarazel.de>
@ 2021-11-26 01:30 ` Robert Haas <robertmhaas@gmail.com>
2021-11-26 04:02 ` Re: pgsql: xlog.c: Remove global variables ReadRecPtr and EndRecPtr. Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 11+ messages in thread
From: Robert Haas @ 2021-11-26 01:30 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: pgsql-committers <pgsql-committers@lists.postgresql.org>
On Wed, Nov 24, 2021 at 8:01 PM Andres Freund <andres@anarazel.de> wrote:
> Pushed the obvious fix for that. Somehow thought I'd seen more compile failure
> than the one WAL_DEBUG...
Hmm, thanks. I guess i put too much trust in the compiler.
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: pgsql: xlog.c: Remove global variables ReadRecPtr and EndRecPtr.
2021-11-24 16:27 pgsql: xlog.c: Remove global variables ReadRecPtr and EndRecPtr. Robert Haas <rhaas@postgresql.org>
2021-11-24 23:12 ` Re: pgsql: xlog.c: Remove global variables ReadRecPtr and EndRecPtr. Andres Freund <andres@anarazel.de>
2021-11-25 01:01 ` Re: pgsql: xlog.c: Remove global variables ReadRecPtr and EndRecPtr. Andres Freund <andres@anarazel.de>
2021-11-26 01:30 ` Re: pgsql: xlog.c: Remove global variables ReadRecPtr and EndRecPtr. Robert Haas <robertmhaas@gmail.com>
@ 2021-11-26 04:02 ` Tom Lane <tgl@sss.pgh.pa.us>
2021-11-26 15:24 ` Re: pgsql: xlog.c: Remove global variables ReadRecPtr and EndRecPtr. Robert Haas <robertmhaas@gmail.com>
0 siblings, 1 reply; 11+ messages in thread
From: Tom Lane @ 2021-11-26 04:02 UTC (permalink / raw)
To: Robert Haas <robertmhaas@gmail.com>; +Cc: Andres Freund <andres@anarazel.de>; pgsql-committers <pgsql-committers@lists.postgresql.org>
Robert Haas <robertmhaas@gmail.com> writes:
> On Wed, Nov 24, 2021 at 8:01 PM Andres Freund <andres@anarazel.de> wrote:
>> Pushed the obvious fix for that. Somehow thought I'd seen more compile failure
>> than the one WAL_DEBUG...
> Hmm, thanks. I guess i put too much trust in the compiler.
My approach to such patches is always "in grep we trust, all others
pay cash". Even without #ifdef issues, you are highly likely to
miss comments that need to be updated.
regards, tom lane
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: pgsql: xlog.c: Remove global variables ReadRecPtr and EndRecPtr.
2021-11-24 16:27 pgsql: xlog.c: Remove global variables ReadRecPtr and EndRecPtr. Robert Haas <rhaas@postgresql.org>
2021-11-24 23:12 ` Re: pgsql: xlog.c: Remove global variables ReadRecPtr and EndRecPtr. Andres Freund <andres@anarazel.de>
2021-11-25 01:01 ` Re: pgsql: xlog.c: Remove global variables ReadRecPtr and EndRecPtr. Andres Freund <andres@anarazel.de>
2021-11-26 01:30 ` Re: pgsql: xlog.c: Remove global variables ReadRecPtr and EndRecPtr. Robert Haas <robertmhaas@gmail.com>
2021-11-26 04:02 ` Re: pgsql: xlog.c: Remove global variables ReadRecPtr and EndRecPtr. Tom Lane <tgl@sss.pgh.pa.us>
@ 2021-11-26 15:24 ` Robert Haas <robertmhaas@gmail.com>
2021-11-27 01:55 ` Re: pgsql: xlog.c: Remove global variables ReadRecPtr and EndRecPtr. Andres Freund <andres@anarazel.de>
0 siblings, 1 reply; 11+ messages in thread
From: Robert Haas @ 2021-11-26 15:24 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Andres Freund <andres@anarazel.de>; pgsql-committers <pgsql-committers@lists.postgresql.org>
On Thu, Nov 25, 2021 at 11:02 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Robert Haas <robertmhaas@gmail.com> writes:
> > On Wed, Nov 24, 2021 at 8:01 PM Andres Freund <andres@anarazel.de> wrote:
> >> Pushed the obvious fix for that. Somehow thought I'd seen more compile failure
> >> than the one WAL_DEBUG...
>
> > Hmm, thanks. I guess i put too much trust in the compiler.
>
> My approach to such patches is always "in grep we trust, all others
> pay cash". Even without #ifdef issues, you are highly likely to
> miss comments that need to be updated.
Right. I didn't rely completely on grep and did spend a lot of time
looking through the code. But sometimes my eyes glaze over and I get
sloppy after too much time working on one thing, and this seems to
have been one of those times.
Fortunately, it seems to have been only a minor oversight.
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: pgsql: xlog.c: Remove global variables ReadRecPtr and EndRecPtr.
2021-11-24 16:27 pgsql: xlog.c: Remove global variables ReadRecPtr and EndRecPtr. Robert Haas <rhaas@postgresql.org>
2021-11-24 23:12 ` Re: pgsql: xlog.c: Remove global variables ReadRecPtr and EndRecPtr. Andres Freund <andres@anarazel.de>
2021-11-25 01:01 ` Re: pgsql: xlog.c: Remove global variables ReadRecPtr and EndRecPtr. Andres Freund <andres@anarazel.de>
2021-11-26 01:30 ` Re: pgsql: xlog.c: Remove global variables ReadRecPtr and EndRecPtr. Robert Haas <robertmhaas@gmail.com>
2021-11-26 04:02 ` Re: pgsql: xlog.c: Remove global variables ReadRecPtr and EndRecPtr. Tom Lane <tgl@sss.pgh.pa.us>
2021-11-26 15:24 ` Re: pgsql: xlog.c: Remove global variables ReadRecPtr and EndRecPtr. Robert Haas <robertmhaas@gmail.com>
@ 2021-11-27 01:55 ` Andres Freund <andres@anarazel.de>
0 siblings, 0 replies; 11+ messages in thread
From: Andres Freund @ 2021-11-27 01:55 UTC (permalink / raw)
To: Robert Haas <robertmhaas@gmail.com>; Tom Lane <tgl@sss.pgh.pa.us>; +Cc: pgsql-committers <pgsql-committers@lists.postgresql.org>
Hi,
On November 26, 2021 5:53:31 PM PST, Andres Freund <andres@anarazel.de> wrote:
>Hi,
>
>On November 26, 2021 7:24:15 AM PST, Robert Haas <robertmhaas@gmail.com> wrote:
>>Fortunately, it seems to have been only a minor oversight.
>
Agreed.
I wonder if we should turn some of these ifdefs into something boiling down to if (0) { ifdef body}... No need to make it impossible for the compiler to help us in most of these cases...
Andres
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.
^ permalink raw reply [nested|flat] 11+ messages in thread
end of thread, other threads:[~2021-11-27 01:55 UTC | newest]
Thread overview: 11+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2021-11-24 16:27 pgsql: xlog.c: Remove global variables ReadRecPtr and EndRecPtr. Robert Haas <rhaas@postgresql.org>
2021-11-24 18:16 ` Robert Haas <robertmhaas@gmail.com>
2021-11-24 18:46 ` Tom Lane <tgl@sss.pgh.pa.us>
2021-11-24 18:51 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2021-11-24 18:58 ` Robert Haas <robertmhaas@gmail.com>
2021-11-24 23:12 ` Andres Freund <andres@anarazel.de>
2021-11-25 01:01 ` Andres Freund <andres@anarazel.de>
2021-11-26 01:30 ` Robert Haas <robertmhaas@gmail.com>
2021-11-26 04:02 ` Tom Lane <tgl@sss.pgh.pa.us>
2021-11-26 15:24 ` Robert Haas <robertmhaas@gmail.com>
2021-11-27 01:55 ` Andres Freund <andres@anarazel.de>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox