pg.ddx.io pgsql-committers@postgresql.org mailing list archive
help / color / mirror / Atom feedRe: API stability
14+ messages / 4 participants
[nested] [flat]
Commits from this thread:
2022-04-14 10520f4346 d18c913b78 68e605b9ef 6270ee4450 48b398943b Rethink the delay-checkpoint-end mechanism in the back-branches.
* Re: API stability
@ 2022-04-06 01:36 Kyotaro Horiguchi <horikyota.ntt@gmail.com>
0 siblings, 1 reply; 14+ messages in thread
From: Kyotaro Horiguchi @ 2022-04-06 01:36 UTC (permalink / raw)
To: robertmhaas@gmail.com; +Cc: markus.wanner@enterprisedb.com; pgsql-hackers@postgresql.org
At Tue, 5 Apr 2022 10:01:56 -0400, Robert Haas <robertmhaas@gmail.com> wrote in
> I think as the person who committed that patch I'm on the hook to fix
> this if nobody else would like to do it, but let me ask whether
> Kyotaro Horiguchi would like to propose a patch, since the original
> patch did, and/or whether you would like to propose a patch, as the
> person reporting the issue.
I'd like to do that. Let me see.
regards.
--
Kyotaro Horiguchi
NTT Open Source Software Center
^ permalink raw reply [nested|flat] 14+ messages in thread
* Re: API stability
@ 2022-04-06 05:30 Kyotaro Horiguchi <horikyota.ntt@gmail.com>
parent: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
0 siblings, 1 reply; 14+ messages in thread
From: Kyotaro Horiguchi @ 2022-04-06 05:30 UTC (permalink / raw)
To: robertmhaas@gmail.com; +Cc: markus.wanner@enterprisedb.com; pgsql-hackers@postgresql.org
me> I'd like to do that. Let me see.
At Tue, 5 Apr 2022 10:29:03 -0400, Robert Haas <robertmhaas@gmail.com> wrote in
> On Tue, Apr 5, 2022 at 10:17 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> > What I think you need to do is:
> >
> > 1. In the back branches, revert delayChkpt to its previous type and
> > semantics. Squeeze a separate delayChkptEnd bool in somewhere
> > (you can't change the struct size either ...).
> >
> > 2. In HEAD, rename the field to something like delayChkptFlags,
> > to ensure that any code touching it has to be inspected and updated.
>
> Well, we can do it that way, I suppose.
The change is easy on head, but is it better use uint8 instead of int
for delayChkptFlags?
In the back branches, we have, on gcc/Linux/x86-64,
14's PGPROC is 880 bytes and has gaps:
- 6 bytes after statusFlag
- 4 bytes after syncRepState
- 2 bytes after subxidStatus
- 3 bytes after procArrayGroupMember
- 3 bytes after clogGroupMember
- 3 bytes after fpVXIDLock
It seems that we can place the new variable in the first place above,
since the two are not bonded together, or at least in less tightly
bonded than other candidates.
13's PGPROC is 856 bytes and has a 7 bytes gap after delayChkpt.
Versions Ealier than 13 have delayChkpt in PGXACT (12 bytes). It is
tightly packed and dones't have a room for a new member. Can we add
the new flag to PGPROC instead of PGXACT?
12 and 11's PGPROC is 848 bytes and has gaps:
- 4 bytes after syncRepState
- 3 bytes after procArrayGroupMember
- 3 bytes after clogGroupMember
- 4 bytes after clogGroupMemberPage
- 3 bytes after fpVXIDLock
10's PGPROC is 816 bytes and has gaps:
- 4 bytes after cvWaitLink
- 4 bytes after syncRepState
- 3 bytes after procArrayGroupMember
- 3 bytes after fpVXIDLock
So if we don't want to move any member in PGPROC, we do:
14: after statusFlags.
13: after delayChkpt.
12-10: after syncRepState (and before syncRepLinks).
If we allow to shift some members, the new flag can be placed more
saner place.
14: after delayChkpt ((uint8)statusFlags moves forward by 1 byte)
13: after delayChkpt (no member moves)
12-10: after subxids ((bool)procArrayGroupMember moves forward by 1 byte)
I continue working on the last direction above.
regards.
--
Kyotaro Horiguchi
NTT Open Source Software Center
^ permalink raw reply [nested|flat] 14+ messages in thread
* Re: API stability
@ 2022-04-06 06:31 Kyotaro Horiguchi <horikyota.ntt@gmail.com>
parent: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
0 siblings, 1 reply; 14+ messages in thread
From: Kyotaro Horiguchi @ 2022-04-06 06:31 UTC (permalink / raw)
To: robertmhaas@gmail.com; +Cc: markus.wanner@enterprisedb.com; pgsql-hackers@postgresql.org
At Wed, 06 Apr 2022 14:30:37 +0900 (JST), Kyotaro Horiguchi <horikyota.ntt@gmail.com> wrote in
> So if we don't want to move any member in PGPROC, we do:
>
> 14: after statusFlags.
> 13: after delayChkpt.
> 12-10: after syncRepState (and before syncRepLinks).
>
> If we allow to shift some members, the new flag can be placed more
> saner place.
>
> 14: after delayChkpt ((uint8)statusFlags moves forward by 1 byte)
> 13: after delayChkpt (no member moves)
> 12-10: after subxids ((bool)procArrayGroupMember moves forward by 1 byte)
>
> I continue working on the last direction above.
Hmm. That is ABI break. I go with the first way.
regards.
--
Kyotaro Horiguchi
NTT Open Source Software Center
^ permalink raw reply [nested|flat] 14+ messages in thread
* Re: API stability
@ 2022-04-06 06:53 Kyotaro Horiguchi <horikyota.ntt@gmail.com>
parent: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
0 siblings, 1 reply; 14+ messages in thread
From: Kyotaro Horiguchi @ 2022-04-06 06:53 UTC (permalink / raw)
To: robertmhaas@gmail.com; +Cc: markus.wanner@enterprisedb.com; pgsql-hackers@postgresql.org
At Wed, 06 Apr 2022 15:31:53 +0900 (JST), Kyotaro Horiguchi <horikyota.ntt@gmail.com> wrote in
> At Wed, 06 Apr 2022 14:30:37 +0900 (JST), Kyotaro Horiguchi <horikyota.ntt@gmail.com> wrote in
> > So if we don't want to move any member in PGPROC, we do:
> >
> > 14: after statusFlags.
> > 13: after delayChkpt.
> > 12-10: after syncRepState (and before syncRepLinks).
> >
> > If we allow to shift some members, the new flag can be placed more
> > saner place.
> >
> > 14: after delayChkpt ((uint8)statusFlags moves forward by 1 byte)
> > 13: after delayChkpt (no member moves)
> > 12-10: after subxids ((bool)procArrayGroupMember moves forward by 1 byte)
> >
> > I continue working on the last direction above.
>
> Hmm. That is ABI break. I go with the first way.
By the way, the patch for -14 changed the sigunature of two public
functions.
-GetVirtualXIDsDelayingChkpt(int *nvxids)
+GetVirtualXIDsDelayingChkpt(int *nvxids, int type)
-HaveVirtualXIDsDelayingChkpt(VirtualTransactionId *vxids, int nvxids)
+HaveVirtualXIDsDelayingChkpt(VirtualTransactionId *vxids, int nvxids, int type)
Do I need to restore the signature?
regards.
--
Kyotaro Horiguchi
NTT Open Source Software Center
^ permalink raw reply [nested|flat] 14+ messages in thread
* Re: API stability
@ 2022-04-06 07:45 Kyotaro Horiguchi <horikyota.ntt@gmail.com>
parent: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
0 siblings, 1 reply; 14+ messages in thread
From: Kyotaro Horiguchi @ 2022-04-06 07:45 UTC (permalink / raw)
To: robertmhaas@gmail.com; +Cc: markus.wanner@enterprisedb.com; pgsql-hackers@postgresql.org
At Wed, 06 Apr 2022 15:53:32 +0900 (JST), Kyotaro Horiguchi <horikyota.ntt@gmail.com> wrote in
> At Wed, 06 Apr 2022 15:31:53 +0900 (JST), Kyotaro Horiguchi <horikyota.ntt@gmail.com> wrote in
> > At Wed, 06 Apr 2022 14:30:37 +0900 (JST), Kyotaro Horiguchi <horikyota.ntt@gmail.com> wrote in
> > > So if we don't want to move any member in PGPROC, we do:
> > >
> > > 14: after statusFlags.
> > > 13: after delayChkpt.
> > > 12-10: after syncRepState (and before syncRepLinks).
> > >
> > > If we allow to shift some members, the new flag can be placed more
> > > saner place.
> > >
> > > 14: after delayChkpt ((uint8)statusFlags moves forward by 1 byte)
> > > 13: after delayChkpt (no member moves)
> > > 12-10: after subxids ((bool)procArrayGroupMember moves forward by 1 byte)
> > >
> > > I continue working on the last direction above.
> >
> > Hmm. That is ABI break. I go with the first way.
>
> By the way, the patch for -14 changed the sigunature of two public
> functions.
>
> -GetVirtualXIDsDelayingChkpt(int *nvxids)
> +GetVirtualXIDsDelayingChkpt(int *nvxids, int type)
>
> -HaveVirtualXIDsDelayingChkpt(VirtualTransactionId *vxids, int nvxids)
> +HaveVirtualXIDsDelayingChkpt(VirtualTransactionId *vxids, int nvxids, int type)
>
> Do I need to restore the signature?
For master, renamed delayChkpt to delayChkptFlags and changed it to uint8.
For 14, restored delayChkpt to bool and added delayChkptEnd into a gap in PGPROC, then restored the signature of the two functions above to before the patch. Then added a new functions ..DelayingChkptEnd().
regards.
--
Kyotaro Horiguchi
NTT Open Source Software Center
Attachments:
v1-0001-Rename-and-change-type-of-delayChkpt.patch (text/x-patch, 0B) download
^ permalink raw reply [nested|flat] 14+ messages in thread
* Re: API stability
@ 2022-04-06 08:30 Alvaro Herrera <alvherre@alvh.no-ip.org>
parent: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
0 siblings, 1 reply; 14+ messages in thread
From: Alvaro Herrera @ 2022-04-06 08:30 UTC (permalink / raw)
To: Kyotaro Horiguchi <horikyota.ntt@gmail.com>; +Cc: robertmhaas@gmail.com; markus.wanner@enterprisedb.com; pgsql-hackers@postgresql.org
On 2022-Apr-06, Kyotaro Horiguchi wrote:
> For master, renamed delayChkpt to delayChkptFlags and changed it to
> uint8.
For code documentation purposes, I think it is slightly better to use
bits8 than uint8 for variables where you're storing independent bit flags.
--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/
"Nunca se desea ardientemente lo que solo se desea por razón" (F. Alexandre)
^ permalink raw reply [nested|flat] 14+ messages in thread
* Re: API stability
@ 2022-04-06 09:13 Kyotaro Horiguchi <horikyota.ntt@gmail.com>
parent: Alvaro Herrera <alvherre@alvh.no-ip.org>
0 siblings, 1 reply; 14+ messages in thread
From: Kyotaro Horiguchi @ 2022-04-06 09:13 UTC (permalink / raw)
To: alvherre@alvh.no-ip.org; +Cc: robertmhaas@gmail.com; markus.wanner@enterprisedb.com; pgsql-hackers@postgresql.org
At Wed, 6 Apr 2022 10:30:32 +0200, Alvaro Herrera <alvherre@alvh.no-ip.org> wrote in
> On 2022-Apr-06, Kyotaro Horiguchi wrote:
>
> > For master, renamed delayChkpt to delayChkptFlags and changed it to
> > uint8.
>
> For code documentation purposes, I think it is slightly better to use
> bits8 than uint8 for variables where you're storing independent bit flags.
Oh, agreed. Will fix in the next version along with other fixes.
Thanks!
--
Kyotaro Horiguchi
NTT Open Source Software Center
^ permalink raw reply [nested|flat] 14+ messages in thread
* Re: API stability
@ 2022-04-06 09:21 Kyotaro Horiguchi <horikyota.ntt@gmail.com>
parent: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
0 siblings, 0 replies; 14+ messages in thread
From: Kyotaro Horiguchi @ 2022-04-06 09:21 UTC (permalink / raw)
To: alvherre@alvh.no-ip.org; +Cc: robertmhaas@gmail.com; markus.wanner@enterprisedb.com; pgsql-hackers@postgresql.org
At Wed, 06 Apr 2022 18:13:17 +0900 (JST), Kyotaro Horiguchi <horikyota.ntt@gmail.com> wrote in
> At Wed, 6 Apr 2022 10:30:32 +0200, Alvaro Herrera <alvherre@alvh.no-ip.org> wrote in
> > For code documentation purposes, I think it is slightly better to use
> > bits8 than uint8 for variables where you're storing independent bit flags.
>
> Oh, agreed. Will fix in the next version along with other fixes.
The immediately folloing member statusFlags is in uint8. So using
bits8 here results in the following look.
> bits8 delayChkptFlags;/* for DELAY_CHKPT_* flags */
>
> uint8 statusFlags; /* this backend's status flags, see PROC_*
> * above. mirrored in
PGPROC has another member that fits bits*.
> uint64 fpLockBits; /* lock modes held for each fast-path slot */
Do I change this in this patch? Or leave them for another chance?
regards.
--
Kyotaro Horiguchi
NTT Open Source Software Center
^ permalink raw reply [nested|flat] 14+ messages in thread
* Re: API stability
@ 2022-04-08 01:32 Kyotaro Horiguchi <horikyota.ntt@gmail.com>
0 siblings, 1 reply; 14+ messages in thread
From: Kyotaro Horiguchi @ 2022-04-08 01:32 UTC (permalink / raw)
To: michael@paquier.xyz; +Cc: robertmhaas@gmail.com; tgl@sss.pgh.pa.us; markus.wanner@enterprisedb.com; pgsql-hackers@postgresql.org
At Fri, 8 Apr 2022 08:47:42 +0900, Michael Paquier <michael@paquier.xyz> wrote in
> On Thu, Apr 07, 2022 at 11:19:15AM -0400, Robert Haas wrote:
> > Here are patches for master and v14 to do things this way. Comments?
>
> Thanks for the patches. They look correct. For ~14, I'd rather avoid
> the code duplication done by GetVirtualXIDsDelayingChkptEnd() and
> HaveVirtualXIDsDelayingChkpt() that could be avoided with an extra
> bool argument to the existing routine. The same kind of duplication
> happens with GetVirtualXIDsDelayingChkpt() and
> GetVirtualXIDsDelayingChkptEnd().
FWIW, it is done in [1].
[1] https://www.postgresql.org/message-id/20220406.164521.17171257901083417.horikyota.ntt%40gmail.com
regards.
--
Kyotaro Horiguchi
NTT Open Source Software Center
^ permalink raw reply [nested|flat] 14+ messages in thread
* Re: API stability
@ 2022-04-08 01:34 Kyotaro Horiguchi <horikyota.ntt@gmail.com>
parent: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
0 siblings, 0 replies; 14+ messages in thread
From: Kyotaro Horiguchi @ 2022-04-08 01:34 UTC (permalink / raw)
To: michael@paquier.xyz; +Cc: robertmhaas@gmail.com; tgl@sss.pgh.pa.us; markus.wanner@enterprisedb.com; pgsql-hackers@postgresql.org
(Mmm. My mailer automatically teared off the [was: ..] part from the
subject..)
--
Kyotaro Horiguchi
NTT Open Source Software Center
^ permalink raw reply [nested|flat] 14+ messages in thread
* Re: API stability
@ 2022-04-11 04:29 Kyotaro Horiguchi <horikyota.ntt@gmail.com>
0 siblings, 1 reply; 14+ messages in thread
From: Kyotaro Horiguchi @ 2022-04-11 04:29 UTC (permalink / raw)
To: robertmhaas@gmail.com; +Cc: michael@paquier.xyz; tgl@sss.pgh.pa.us; markus.wanner@enterprisedb.com; pgsql-hackers@postgresql.org
(a bit off-topic)
I'm not sure where I am..
At Wed, 06 Apr 2022 10:36:30 +0900 (JST), Kyotaro Horiguchi <horikyota.ntt@gmail.com> wrote in
me> > this if nobody else would like to do it, but let me ask whether
me> > Kyotaro Horiguchi would like to propose a patch, since the original
me> > patch did, and/or whether you would like to propose a patch, as the
me> > person reporting the issue.
me>
me> I'd like to do that. Let me see.
At Thu, 7 Apr 2022 10:04:20 -0400, Robert Haas <robertmhaas@gmail.com> wrote in
> struct, which is what we now need to fix. Since I don't hear anyone
> else volunteering to take care of that, I'll go work on it.
Just confirmation. Is my message above didn't look like declaring that
I'd like to volunteering? If so, please teach me the correct way to
say that, since I don't want to repeat the same mistake. Or are there
some other reasons? (Sorry if this looks like a blame, but I asking
plainly (really:).)
regards.
--
Kyotaro Horiguchi
NTT Open Source Software Center
^ permalink raw reply [nested|flat] 14+ messages in thread
* Re: API stability
@ 2022-04-11 10:48 Matthias van de Meent <boekewurm+postgres@gmail.com>
parent: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
0 siblings, 2 replies; 14+ messages in thread
From: Matthias van de Meent @ 2022-04-11 10:48 UTC (permalink / raw)
To: Kyotaro Horiguchi <horikyota.ntt@gmail.com>; +Cc: Robert Haas <robertmhaas@gmail.com>; Michael Paquier <michael@paquier.xyz>; Tom Lane <tgl@sss.pgh.pa.us>; markus.wanner@enterprisedb.com, PostgreSQL Hackers <pgsql-hackers@postgresql.org>
On Mon, 11 Apr 2022 at 06:30, Kyotaro Horiguchi <horikyota.ntt@gmail.com> wrote:
>
> (a bit off-topic)
>
> I'm not sure where I am..
>
> At Wed, 06 Apr 2022 10:36:30 +0900 (JST), Kyotaro Horiguchi <horikyota.ntt@gmail.com> wrote in
> me> > this if nobody else would like to do it, but let me ask whether
> me> > Kyotaro Horiguchi would like to propose a patch, since the original
> me> > patch did, and/or whether you would like to propose a patch, as the
> me> > person reporting the issue.
> me>
> me> I'd like to do that. Let me see.
>
> At Thu, 7 Apr 2022 10:04:20 -0400, Robert Haas <robertmhaas@gmail.com> wrote in
> > struct, which is what we now need to fix. Since I don't hear anyone
> > else volunteering to take care of that, I'll go work on it.
>
> Just confirmation. Is my message above didn't look like declaring that
> I'd like to volunteering? If so, please teach me the correct way to
> say that, since I don't want to repeat the same mistake. Or are there
> some other reasons? (Sorry if this looks like a blame, but I asking
> plainly (really:).)
I won't speak for Robert H., but this might be because of gmail not
putting this mail in the right thread: Your mail client dropped the
"[was: pgsql: ...]" tag, which Gmail subsequently displays as a
different thread (that is, in my Gmail UI there's three "Re: API
stability" threads, one of which has the [was: pgsql: ...]-tag, and
two of which seem to be started by you as a reply on the original
thread, but with the [was: pgsql: ...]-tag dropped and thus considered
a new thread).
So, this might be the reason Robert overlooked your declaration to
volunteer: he was looking for volunteers in the thread "Re: API
Stability [was: pgsql: ...]" in the Gmail UI, which didn't show your
messages there because of the different subject line.
Kind regards,
Matthias van de Meent
^ permalink raw reply [nested|flat] 14+ messages in thread
* Re: API stability
@ 2022-04-11 18:00 Robert Haas <robertmhaas@gmail.com>
parent: Matthias van de Meent <boekewurm+postgres@gmail.com>
1 sibling, 0 replies; 14+ messages in thread
From: Robert Haas @ 2022-04-11 18:00 UTC (permalink / raw)
To: Matthias van de Meent <boekewurm+postgres@gmail.com>; +Cc: Kyotaro Horiguchi <horikyota.ntt@gmail.com>; Michael Paquier <michael@paquier.xyz>; Tom Lane <tgl@sss.pgh.pa.us>; Markus Wanner <markus.wanner@enterprisedb.com>; PostgreSQL Hackers <pgsql-hackers@postgresql.org>
On Mon, Apr 11, 2022 at 6:48 AM Matthias van de Meent
<boekewurm+postgres@gmail.com> wrote:
> So, this might be the reason Robert overlooked your declaration to
> volunteer: he was looking for volunteers in the thread "Re: API
> Stability [was: pgsql: ...]" in the Gmail UI, which didn't show your
> messages there because of the different subject line.
Yes, that's what happened.
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 14+ messages in thread
* Re: API stability
@ 2022-04-12 02:57 Kyotaro Horiguchi <horikyota.ntt@gmail.com>
parent: Matthias van de Meent <boekewurm+postgres@gmail.com>
1 sibling, 0 replies; 14+ messages in thread
From: Kyotaro Horiguchi @ 2022-04-12 02:57 UTC (permalink / raw)
To: boekewurm+postgres@gmail.com; +Cc: robertmhaas@gmail.com; michael@paquier.xyz; tgl@sss.pgh.pa.us; markus.wanner@enterprisedb.com; pgsql-hackers@postgresql.org
At Mon, 11 Apr 2022 12:48:25 +0200, Matthias van de Meent <boekewurm+postgres@gmail.com> wrote in
> On Mon, 11 Apr 2022 at 06:30, Kyotaro Horiguchi <horikyota.ntt@gmail.com> wrote:
> I won't speak for Robert H., but this might be because of gmail not
> putting this mail in the right thread: Your mail client dropped the
> "[was: pgsql: ...]" tag, which Gmail subsequently displays as a
> different thread (that is, in my Gmail UI there's three "Re: API
> stability" threads, one of which has the [was: pgsql: ...]-tag, and
> two of which seem to be started by you as a reply on the original
> thread, but with the [was: pgsql: ...]-tag dropped and thus considered
> a new thread).
Mmm. d*** gmail.. My main mailer does that defaltly but I think I can
*fix* that behavior.
Thanks!
--
Kyotaro Horiguchi
NTT Open Source Software Center
^ permalink raw reply [nested|flat] 14+ messages in thread
end of thread, other threads:[~2022-04-12 02:57 UTC | newest]
Thread overview: 14+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2022-04-06 01:36 Re: API stability Kyotaro Horiguchi <horikyota.ntt@gmail.com>
2022-04-06 05:30 ` Kyotaro Horiguchi <horikyota.ntt@gmail.com>
2022-04-06 06:31 ` Kyotaro Horiguchi <horikyota.ntt@gmail.com>
2022-04-06 06:53 ` Kyotaro Horiguchi <horikyota.ntt@gmail.com>
2022-04-06 07:45 ` Kyotaro Horiguchi <horikyota.ntt@gmail.com>
2022-04-06 08:30 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2022-04-06 09:13 ` Kyotaro Horiguchi <horikyota.ntt@gmail.com>
2022-04-06 09:21 ` Kyotaro Horiguchi <horikyota.ntt@gmail.com>
2022-04-08 01:32 Re: API stability Kyotaro Horiguchi <horikyota.ntt@gmail.com>
2022-04-08 01:34 ` Kyotaro Horiguchi <horikyota.ntt@gmail.com>
2022-04-11 04:29 Re: API stability Kyotaro Horiguchi <horikyota.ntt@gmail.com>
2022-04-11 10:48 ` Matthias van de Meent <boekewurm+postgres@gmail.com>
2022-04-11 18:00 ` Robert Haas <robertmhaas@gmail.com>
2022-04-12 02:57 ` Kyotaro Horiguchi <horikyota.ntt@gmail.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox