Subject: Re: First draft of PG 17 release notes

From: Alvaro Herrera <[email-masked]>
Date: Sat, 7 Sep 2024 11:55:09 +0200

On 2024-Sep-05, Bruce Momjian wrote:

> That seems more infrastructure/extension author stuff which isn't
> normally mentioned in the release notes.  I think such people really
> need to look at all the commit messages.

Are you saying all extension authors should be reading the complete git
log for every single major release?  That's a strange position to take.
Isn't this a good fit for "E.1.3.10. Source Code"?

-- 
Álvaro Herrera               48°01'N 7°57'E  —  https://www.EnterpriseDB.com/
<Schwern> It does it in a really, really complicated way
<crab> why does it need to be complicated?
<Schwern> Because it's MakeMaker.

---

From: Bruce Momjian <[email-masked]>
Date: Mon, 9 Sep 2024 22:46:50 -0400

> Isn't this a good fit for "E.1.3.10. Source Code"?

Yes.  There are so many changes at the source code level it is unwise to
try and get them into the main release notes.  If someone wants to
create an addendum, like was suggested for pure performance
improvements, that would make sense.

-- 
  Bruce Momjian  <[email-masked]>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

---

From: Jelte Fennema-Nio <[email-masked]>
Date: Tue, 10 Sep 2024 08:28:42 +0200

> improvements, that would make sense.

I agree that the release notes cannot fit every change. But I also
don't think any extension author reads the complete git commit log
every release, so taking the stance that they should be seems
unhelpful. And the "Source Code" section does exist so at some level
you seem to disagree with that too. So what is the way to decide that
something makes the cut for the "Source Code" section?

I think as an extension author there are usually three types of
changes that are relevant:
1. New APIs/hooks that are meant for extension authors
2. Stuff that causes my existing code to not compile anymore
3. Stuff that changes behaviour of existing APIs code in a
incompatible but silent way

For 1, I think adding them to the release notes makes total sense,
especially if the new APIs are documented not only in source code, but
also on the website. Nathan his change is of this type, so I agree
with him it should be in the release notes.

For 2, I'll be able to easily find the PG commit that caused the
compilation failure by grepping git history for the old API. So having
these changes in the release notes seems unnecessary.

For 3, it would be very useful if it would be in the release notes,
but I think in many cases it's hard to know what commits do this. So
unless it's obviously going to break a bunch of extensions silently, I
think we don't have to add such changes to the release notes.

---

From: Alvaro Herrera <[email-masked]>
Date: Tue, 10 Sep 2024 10:51:39 +0200

> with him it should be in the release notes.

I agree.  The volume of such items should be pretty small.

> think we don't have to add such changes to the release notes.

While we cannot be 100% vigilant (and it doesn't seem likely for
automated tools to detect this), we try to avoid API changes that would
still compile but behave incompatibly.  In many review discussions you
can see suggestions to change some function signature so that
third-party authors would be aware that they need to adapt their code to
new behavior, turning cases of (3) into (2).  I agree that these don't
need release notes items.

-- 
Álvaro Herrera         PostgreSQL Developer  —  https://www.EnterpriseDB.com/
"XML!" Exclaimed C++.  "What are you doing here? You're not a programming
language."
"Tell that to the people who use me," said XML.
https://burningbird.net/the-parable-of-the-languages/

---

From: Masahiko Sawada <[email-masked]>
Date: Tue, 10 Sep 2024 09:52:42 -0700

> with him it should be in the release notes.

+1. I think that the increment JSON parser that is already mentioned
in the release note would fall in this type too; it's not a feature
aimed just for extension authors, but it's kind of source and internal
changes IMO. Since the DSM registry feature is described in the doc, I
think it would make sense to have it in the release notes and probably
has a link to the "Requesting Shared Memory After Startup" section.

Regards,

-- 
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com

---

From: Bruce Momjian <[email-masked]>
Date: Wed, 11 Sep 2024 10:10:26 -0400

> think we don't have to add such changes to the release notes.

So, we are looking at this commit:

    commit b5a9b18cd0b
    Author: Thomas Munro <[email-masked]>
    Date:   Wed Apr 3 00:17:06 2024 +1300

    Provide API for streaming relation data.

    Introduce an abstraction allowing relation data to be accessed as a
    stream of buffers, with an implementation that is more efficient than
    the equivalent sequence of ReadBuffer() calls.

    Client code supplies a callback that can say which block number it wants
    next, and then consumes individual buffers one at a time from the
    stream.  This division puts read_stream.c in control of how far ahead it
    can see and allows it to read clusters of neighboring blocks with
    StartReadBuffers().  It also issues POSIX_FADV_WILLNEED advice ahead of
    time when random access is detected.

    Other variants of I/O stream will be proposed in future work (for
    example to support recovery, whose LsnReadQueue device in
    xlogprefetcher.c is a distant cousin of this code and should eventually
    be replaced by this), but this basic API is sufficient for many common
    executor usage patterns involving predictable access to a single fork of
    a single relation.

    Several patches using this API are proposed separately.

    This stream concept is loosely based on ideas from Andres Freund on how
    we should pave the way for later work on asynchronous I/O.

You are right that I do mention changes specifically designed for the
use of extensions, but there is no mention in the commit message of its
use for extensions.  In fact, I thought this was too low-level to be of
use for extensions.  However, if people feel it should be added, we have
enough time to add it.

I also mention changes that are _likely_ to affect extensions, but not
all changes that could affect extensions.  An interesting idea would be
to report all function signature changes in each major release in some
way.

  When a patient asks the doctor, "Am I going to die?", he means 
  "Am I going to die soon?"

---

From: Bruce Momjian <[email-masked]>
Date: Wed, 11 Sep 2024 10:12:58 -0400

> has a link to the "Requesting Shared Memory After Startup" section.

This commit?

	commit 8b2bcf3f287
	Author: Nathan Bossart <[email-masked]>
	Date:   Fri Jan 19 14:24:36 2024 -0600
	
	    Introduce the dynamic shared memory registry.

Yes, we have time to add it.

---

From: Nathan Bossart <[email-masked]>
Date: Wed, 11 Sep 2024 09:36:35 -0500

> Yes, we have time to add it.

Yes, that's the one.

-- 
nathan

---

From: Bruce Momjian <[email-masked]>
Date: Fri, 13 Sep 2024 16:17:31 -0400

> > Yes, we have time to add it.

Attached patch applied, with commit URL link.

  When a patient asks the doctor, "Am I going to die?", he means 
  "Am I going to die soon?"
diff --git a/doc/src/sgml/release-17.sgml b/doc/src/sgml/release-17.sgml
index 8aed30428bd..36e19bb8329 100644
--- a/doc/src/sgml/release-17.sgml
+++ b/doc/src/sgml/release-17.sgml
@@ -3011,6 +3011,23 @@ Author: Michael Paquier <[email-masked]>
       </para>
      </listitem>
 
+<!--
+Author: Nathan Bossart <[email-masked]>
+2024-01-19 [8b2bcf3f2] Introduce the dynamic shared memory registry.
+-->
+
+     <listitem>
+      <para>
+      Add dynamic shared memory registry (Nathan Bossart)
+      <ulink url="&commit_baseurl;8b2bcf3f2">&sect;</ulink>
+      </para>
+
+      <para>
+       This allows shared libraries which are not initialized at startup
+       to coordinate dynamic shared memory access.
+      </para>
+     </listitem>
+
 <!--
 Author: Michael Paquier <[email-masked]>
 2024-04-04 [2a217c371] Coordinate emit_log_hook and all log destinations to sha

---

From: Nathan Bossart <[email-masked]>
Date: Fri, 13 Sep 2024 16:00:07 -0500

> Attached patch applied, with commit URL link.

Looks good, thanks.

-- 
nathan

---

From: Jelte Fennema-Nio <[email-masked]>
Date: Tue, 17 Sep 2024 10:01:28 +0200

> use for extensions.  However, if people feel it should be added, we have

Another new API that is useful for extension authors is the following
one (I'm obviously biased since I'm the author, and I don't know if
there's still time):

commit 14dd0f27d7cd56ffae9ecdbe324965073d01a9ff
Author: Nathan Bossart <[email-masked]>
Date:   Thu Jan 4 16:09:34 2024 -0600

    Add macros for looping through a List without a ListCell.

    Many foreach loops only use the ListCell pointer to retrieve the
    content of the cell, like so:

        ListCell   *lc;

        foreach(lc, mylist)
        {
            int         myint = lfirst_int(lc);

            ...
        }

    This commit adds a few convenience macros that automatically
    declare the loop variable and retrieve the current cell's contents.
    This allows us to rewrite the previous loop like this:

        foreach_int(myint, mylist)
        {
            ...
        }

> An interesting idea would be

I think that might be useful, but it very much depends how long that
list gets. If it gets too long I think authors will just try to
compile and only look at the ones that break for them.

---

From: Bruce Momjian <[email-masked]>
Date: Wed, 18 Sep 2024 17:33:18 -0400

>         foreach_int(myint, mylist)

Can someone else comment on the idea of adding this release note item? 
I don't feel confident in my ability to evaluate this.  I obviously did
not see it as significant the first time.

---

From: Nathan Bossart <[email-masked]>
Date: Wed, 18 Sep 2024 20:09:27 -0500

> not see it as significant the first time.

I'm not sure precisely what criteria you use to choose what goes in the
release notes, but this one seems like a judgement call to me.  My initial
reaction is that it shouldn't be included, but I do see some items with a
similar scope, such as "Remove some SPI macros."

-- 
nathan

---

From: Robert Haas <[email-masked]>
Date: Thu, 19 Sep 2024 12:23:21 -0400

> similar scope, such as "Remove some SPI macros."

I wouldn't mention either this or "Remove some unused SPI macros".

-- 
Robert Haas
EDB: http://www.enterprisedb.com

---

From: Bruce Momjian <[email-masked]>
Date: Thu, 19 Sep 2024 13:04:40 -0400

> I wouldn't mention either this or "Remove some unused SPI macros".

I mentioned the SPI macros because that could lead to breakage, and
there might be applications, as well as extensions, that use it.

---

From: Robert Haas <[email-masked]>
Date: Thu, 19 Sep 2024 13:23:40 -0400

> there might be applications, as well as extensions, that use it.

Sure, this is all a judgement call. I don't think it's particularly
likely that many people are relying on those macros, though, and if
they are, they will mostly likely find out that they're gone when they
try to compile, rather than from reading the release notes. Likewise,
I feel that the new list iteration macros are both optional and minor,
so there's not really a reason to tell people about them. But opinions
will vary, and that's fine. I just mentioned my opinion since you
seemed to be asking. :-)