Subject: PG 16 draft release notes ready

From: Bruce Momjian <[email-masked]>
Date: Thu, 18 May 2023 16:49:47 -0400

I have completed the first draft of the PG 16 release notes.  You can
see the output here:

	https://momjian.us/pgsql_docs/release-16.html

I will adjust it to the feedback I receive;  that URL will quickly show
all updates.

I learned a few things creating it this time:

*  I can get confused over C function names and SQL function names in
   commit messages.

*  The sections and ordering of the entries can greatly clarify the
   items.

*  The feature count is slightly higher than recent releases:

	release-10:  189
	release-11:  170
	release-12:  180
	release-13:  178
	release-14:  220
	release-15:  184
-->	release-16:  200

-- 
  Bruce Momjian  <[email-masked]>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

---

From: "Jonathan S. Katz" <[email-masked]>
Date: Thu, 18 May 2023 17:26:17 -0400

> I will adjust it to the feedback I receive;  that URL will quickly show

Thanks for going through this. The release announcement draft will 
follow shortly after in a different thread.

> *  The feature count is slightly higher than recent releases:

This definitely feels like a very full release. Personally I'm very excited.

Thanks,

Jonathan

---

From: "Jonathan S. Katz" <[email-masked]>
Date: Thu, 18 May 2023 17:39:08 -0400

> I will adjust it to the feedback I receive;  that URL will quickly show

Still reading, but saw this:

   Allow incremental sorts in more cases, including DISTINCT (David 
Rowley)window

I didn't realize we had a DISTINCT (David Rowley) window, but it sounds 
like an awesome feature ;)

Thanks,

Jonathan

---

From: Peter Geoghegan <[email-masked]>
Date: Thu, 18 May 2023 14:53:25 -0700

> *  I can get confused over C function names and SQL function names in

The commit history covering pg_walinspect was complicated. Some of the
newly added stuff was revised multiple times, by multiple authors due
to changing ideas about the best UI. Here is some concrete feedback
about that:

* Two functions that were in 15 that each end in *_till_end_of_wal()
were removed for 16, since the same functionality is now provided
through a more intuitive UI: we now tolerate invalid end_lsn values
"from the future", per a new "Tip" in the pg_walinspect documentation
for 16.

In my opinion this should (at most) be covered as a compatibility
item. It's not really new functionality.

* There is one truly new pg_walinspect function added to 16:
pg_get_wal_block_info(). Its main purpose is to see how each
individual block changed -- it's far easier to track how blocks
changed over time using the new function. The original "record
orientated" functions made that very difficult (regex hacks were
required).

pg_get_wal_block_info first appeared under another name, and had
somewhat narrower functionality to the final version, all of which
shouldn't matter to users -- since they never saw a stable release
with any of that. There is no point in telling users about the commits
that changed the name/functionality of pg_get_wal_block_info that came
only a couple of months after the earliest version was commited -- to
users, it is simply a new function with new functionality.

I also suggest merging the pg_waldump items with the section you've
added covering pg_walinspect. A "pg_walinspect and pg_waldump" section
seems natural to me. Some of the enhancements in this area benefit
pg_walinspect and pg_waldump in exactly the same way, since
pg_walinspect is essentially a backend/SQL interface equivalent of
pg_waldump's frontend/CLI interface. Melanie's work on improving the
descriptions output for WAL records like Heap's PRUNE and VACUUM
records is a great example of this -- it does exactly the same thing
for pg_walinspect and pg_waldump, without directly targeting either
(it also affects the wal_debug developer option).

It might also make sense to say that the enhanced WAL record
descriptions from Melanie generally apply to records used by VACUUM
only.

Note also that the item "Add pg_waldump option --save-fullpage to dump
full page images (David Christensen)" is tangentially related to
pg_get_wal_block_info(), since you can also get FPIs using
pg_get_wal_block_info() (in fact, that was originally its main
purpose). I'm not saying that you necessarily need to connect them
together in any way, but you might consider it.

-- 
Peter Geoghegan

---

From: Bruce Momjian <[email-masked]>
Date: Thu, 18 May 2023 18:51:59 -0400

> item. It's not really new functionality.

So, I looked at this and the problem is that this is best as a single
release note entry because we are removing and adding, and if I moved it
to compatibility, I am concerned the new feature will be missed.  Since
WAL inspection is a utility operation, inn general, I think having it in
the pg_walinspect section makes the most sense.
 
> users, it is simply a new function with new functionality.

Right.

> (it also affects the wal_debug developer option).

Well, pg_waldump is an installed binary while pg_walinspect is an
extension, so I am not sure where I would put a merged section.

> descriptions from Melanie generally apply to records used by VACUUM

Okay, I went with:

	Improve descriptions of pg_walinspect WAL record descriptions
	(Melanie Plageman, Peter Geoghegan)

> together in any way, but you might consider it.

Well, there is so much _new_ in that tool that listing everything new
seems confusing.

FYI, I have just added an item about more aggressing freezing:

	<!--
	Author: Peter Geoghegan <[email-masked]>
	2022-09-08 [d977ffd92] Instrument freezing in autovacuum log reports.
	Author: Peter Geoghegan <[email-masked]>
	2022-11-15 [9e5405993] Deduplicate freeze plans in freeze WAL records.
	Author: Peter Geoghegan <[email-masked]>
	2022-12-28 [1de58df4f] Add page-level freezing to VACUUM.
	-->
	
	<listitem>
	<para>
	During non-freeze operations, perform page freezing where appropriate
	Peter Geoghegan)
	</para>
	
	<para>
	This makes full-table freeze vacuums less necessary.
	</para>
	</listitem>

All changes committed.

---

From: Peter Geoghegan <[email-masked]>
Date: Thu, 18 May 2023 16:12:26 -0700

> the pg_walinspect section makes the most sense.

I don't understand what you mean by that. The changes to
*_till_end_of_wal() (the way that those duplicative functions were
removed, and more permissive end_lsn behavior was added) is unrelated
to all of the other changes. Plus it's just not very important.

> Well, there is so much _new_ in that tool that listing everything new

There is pretty much one truly new piece of functionality added to
pg_walinspect (the function called pg_get_wal_block_info was added) --
since the enhancement to rmgr description output applies equally to
pg_waldump, no matter where you place it in the release notes. So not
sure what you mean.

> All changes committed.

Even after these changes, the release notes still refer to a function
called "pg_get_wal_block". There is no such function, though -- not in
Postgres 16, and not in any other major version.

As I said, there is a new function called "pg_get_wal_block_info". It
should simply be presented as a whole new function that offers novel
new functionality compared to what was available in Postgres 15 --
without any further elaboration. (It happens to be true that
pg_get_wal_block_info only reached its final form following multiple
rounds of work in multiple commits, but that is of no consequence to
users -- even the earliest form of the function appeared in a commit
in the Postgres 16 cycle.)

-- 
Peter Geoghegan

---

From: Matthias van de Meent <[email-masked]>
Date: Fri, 19 May 2023 01:33:17 +0200

> I will adjust it to the feedback I receive;  that URL will quickly show

I'm not sure if bugfixes like these are considered for release notes,
but I'm putting it up here just in case:

As of 8fcb32db (new, only in PG16) we now enforce limits on the size
of WAL records during construction, where previously we hoped that the
WAL records didn't exceed those limits.
This change is immediately user-visible through a change in behaviour
of `pg_logical_emit_message(true, repeat('_', 2^30 - 10), repeat('_',
2^30 - 10))`, and extensions that implement their own rmgrs could also
see a change in behavior from this change.

Kind regards,

Matthias van de Meent
Neon, Inc.

---

From: Bruce Momjian <[email-masked]>
Date: Thu, 18 May 2023 21:45:33 -0400

> see a change in behavior from this change.

I saw that commit but I considered it sufficiently rare and sufficiently
internal that I did not include it.

---

From: "Drouvot, Bertrand" <[email-masked]>
Date: Fri, 19 May 2023 09:49:18 +0200

Hi,

> I will adjust it to the feedback I receive;  that URL will quickly show

Thanks!

"
This adds the function pg_log_standby_snapshot(). TEXT?:
"

My proposal:

This adds the function pg_log_standby_snapshot() to log details of the current snapshot
to WAL. If the primary is idle, the slot creation on a standby can take a while.
This function can be used on the primary to speed up the logical slot creation on
the standby.

Regards,

-- 
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

---

From: Bruce Momjian <[email-masked]>
Date: Fri, 19 May 2023 08:29:06 -0400

> This function can be used on the primary to speed up the logical slot creation on

Yes, I got this concept from the commit message, but I am unclear on
what is actually happening so I can clearly explain it.  Slot creation
on the standby needs a snapshot, and that is only created when there is
activity, or happens periodically, and this forces it to happen, or
something?  And what snapshot is this?  The current session's?

---

From: Nathan Bossart <[email-masked]>
Date: Fri, 19 May 2023 07:31:40 -0700

Thanks!

> Allow GRANT to give vacuum and analyze permission to users beyond the
> table owner or superusers (Nathan Bossart)

This one was effectively reverted in favor of the MAINTAIN privilege.

> Create a predefined role with permission to perform maintenance
> operations (Nathan Bossart)

IMO this should also mention the grantable MAINTAIN privilege.
Alternatively, the item above about granting vacuum/analyze privileges
could be adjusted.

-- 
Nathan Bossart

---

From: Sehrope Sarkuni <[email-masked]>
Date: Fri, 19 May 2023 11:07:29 -0400

The intro in "E.1.3. Changes" says "... between PostgreSQL 15 and the
previous major release".

That should be "... between PostgreSQL __16__ ..." right?

Regards,
-- Sehrope Sarkuni
Founder & CEO | JackDB, Inc. | https://www.jackdb.com/

---

From: Bruce Momjian <[email-masked]>
Date: Fri, 19 May 2023 12:27:35 -0400

> This one was effectively reverted in favor of the MAINTAIN privilege.

Okay, removed.

> Alternatively, the item above about granting vacuum/analyze privileges

Very good point --- here is the new text:

	<!--
	Author: Jeff Davis <[email-masked]>
	2022-12-13 [60684dd83] Add grantable MAINTAIN privilege and pg_maintain role.
	Author: Andrew Dunstan <[email-masked]>
	2022-11-28 [4441fc704] Provide non-superuser predefined roles for vacuum and an
	Author: Jeff Davis <[email-masked]>
	2023-01-14 [ff9618e82] Fix MAINTAIN privileges for toast tables and partitions.
	-->
	
	<listitem>
	<para>
	Create a predefined role and grantable privilege with permission to perform maintenance operations (Nathan Bossart)
	</para>
	
	<para>
	The predefined role is is called pg_maintain.
	</para>
	</listitem>

I will commit this change now.

---

From: jian he <[email-masked]>
Date: Sat, 20 May 2023 00:59:35 +0800

> *  The feature count is slightly higher than recent releases:

Add function pg_dissect_walfile_name() to report the segment and timeline
> values of WAL file names (Bharath Rupireddy)

https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=13e0d7a603852b8b05c03b45228daabffa0cced2
the function rename to pg_split_walfile_name.

seems didn't mention pg_input_is_valid,pg_input_error_info?
https://www.postgresql.org/docs/devel/functions-info.html#FUNCTIONS-INFO-VALIDITY-TABLE

---

From: Bruce Momjian <[email-masked]>
Date: Sat, 20 May 2023 19:22:24 -0400

> the function rename to pg_split_walfile_name.

Fixed.  I copied the commit that did the rename, but forgot to actually
update the release note text to match.

> FUNCTIONS-INFO-VALIDITY-TABLE

Good point.  I incorrectly interpreted the commit text as part of our
test infrastuture and not the addition of two SQL functions:

    Add test scaffolding for soft error reporting from input functions.

    pg_input_is_valid() returns boolean, while pg_input_error_message()
    returns the primary error message if the input is bad, or NULL
    if the input is OK.  The main reason for having two functions is
    so that we can test both the details-wanted and the no-details-wanted
    code paths.

I have added this release note item:

	<!--
	Author: Tom Lane <[email-masked]>
	2022-12-09 [1939d2628] Add test scaffolding for soft error reporting from input
	-->
	
	<listitem>
	<para>
	Add functions pg_input_is_valid() and pg_input_error_message() to check for type conversion errors (Tom Lane)
	</para>
	</listitem>

---

From: Ian Lawrence Barwick <[email-masked]>
Date: Sun, 21 May 2023 21:30:01 +0900

> I will adjust it to the feedback I receive;  that URL will quickly show

Hi

Below are a few commits which are not referenced in the current iteration
of the release notes, but which seem worthy of inclusion.
Apologies if they have been previously discussed, or I'm overlooking something
obvious.

d09dbeb9b Speedup hash index builds by skipping needless binary searches
  "Testing has shown that this can improve hash index build speeds by 5-15%
   with a unique set of integer values."

e09d7a126 Improve speed of hash index build.
  "This seems to be good for overall
  speedup of 5%-9%, depending on the incoming data."

594f8d377 Allow batching of inserts during cross-partition updates.
  seems reasonable to mention this as it's related to 97da48246, which
  is mentioned in the notes

1349d2790 Improve performance of ORDER BY / DISTINCT aggregates
  This is the basis for da5800d5, which is mentioned in the notes, but AFAICS
   the latter is an implementation fix for the former (haven't looked
into either
   in detail though).

The following are probably not headline features, but are the kind of
behavioural
changes I'd expect to find in the release notes (when, at some point
in the far and
distant future, trying to work out when they were introduced when considering
application compatibility etc.):

13a185f54 Allow publications with schema and table of the same schema.
2ceea5adb Accept "+infinity" in date and timestamp[tz] input.
d540a02a7 Display the leader apply worker's PID for parallel apply workers.

Regards

Ian Barwick

---

From: Bruce Momjian <[email-masked]>
Date: Sun, 21 May 2023 11:52:34 -0400

>   speedup of 5%-9%, depending on the incoming data."

For the above two items, I mention items that would change user behavior
like new features or changes that are significant enough that they would
change user behavior.  For example, if a new join method increases
performance by 5x, that could change user behavior.  Based on the quoted
numbers above, I didn't think "hash now faster" would be appropriate to
mention.  Right?

>   is mentioned in the notes

I wasn't sure if that was significant, based on the above logic, but
97da48246 has a user API to control it so I mentioned that one.

>    the latter is an implementation fix for the former (haven't looked

I have added this commit to the existing entry, thanks.

> 13a185f54 Allow publications with schema and table of the same schema.

This seemed like a rare enough case that I did not add it.

> 2ceea5adb Accept "+infinity" in date and timestamp[tz] input.

I have this but didn't add that commit, added.

> d540a02a7 Display the leader apply worker's PID for parallel apply workers.

Parallelism of apply is a new feature and I don't normally mention
output _additions_ that are related to new features.

---

From: Tom Lane <[email-masked]>
Date: Sun, 21 May 2023 13:11:05 -0400

> I have this but didn't add that commit, added.

That's really not related to the commit you added it to...

I don't have time today to read through all the relnotes, but I went
through those that have my name on them.  Suggested wording modifications
attached.

			regards, tom lane

diff --git a/doc/src/sgml/release-16.sgml b/doc/src/sgml/release-16.sgml
index e156284b71..b31e31fccd 100644
--- a/doc/src/sgml/release-16.sgml
+++ b/doc/src/sgml/release-16.sgml
@@ -60,6 +60,8 @@ Change assignment rules for PL/pgSQL bound cursor variables (Tom Lane)
 
 <para>
 Previously, the string value of such variables was set to match the variable name during cursor assignment;  now it will be assigned during OPEN, and will not match the variable name.
+To restore the previous behavior, assign the desired portal name to the cursor
+variable before OPEN.
 </para>
 </listitem>
 
@@ -257,7 +259,7 @@ Author: Tom Lane <[email-masked]>
 
 <listitem>
 <para>
-Allow memoize atop of UNION ALL and partitions (Richard Guo)
+Allow memoize atop a UNION ALL (Richard Guo)
 </para>
 </listitem>
 
@@ -268,7 +270,8 @@ Author: Tom Lane <[email-masked]>
 
 <listitem>
 <para>
-Allow anti-joins to be constructed on the right/outer side (Richard Guo)
+Allow anti-joins to be performed with the non-nullable input as the
+inner relation (Richard Guo)
 </para>
 </listitem>
 
@@ -924,7 +927,7 @@ Allow makeaclitem() to accept multiple privilege names (Robins Tharakan)
 </para>
 
 <para>
-Previously only a single privilege names, like SELECT, were supported.
+Previously only a single privilege name, like SELECT, was accepted.
 </para>
 </listitem>
 
@@ -972,11 +975,7 @@ Author: Tom Lane <[email-masked]>
 
 <listitem>
 <para>
-Store server variables in a hash table (Tom Lane)
-</para>
-
-<para>
-This allows the faster addition of server variables.
+Improve performance of server variable management (Tom Lane)
 </para>
 </listitem>
 
@@ -1081,7 +1080,9 @@ Allow the postmaster to terminate children with an abort signal (Tom Lane)
 </para>
 
 <para>
-Abort normally creates a core dump.  This is controlled by send_abort_for_crash and send_abort_for_kill.  postmaster -T is now the same as setting send_abort_for_crash.
+This allows collection of a core dump for a stuck child process.
+This is controlled by send_abort_for_crash and send_abort_for_kill.
+The postmaster's -T switch is now the same as setting send_abort_for_crash.
 </para>
 </listitem>
 
@@ -1092,7 +1093,7 @@ Author: Tom Lane <[email-masked]>
 
 <listitem>
 <para>
-Remove the unnecessary postmaster -n option (Tom Lane)
+Remove the non-functional postmaster -n option (Tom Lane)
 </para>
 </listitem>
 
@@ -1462,7 +1463,8 @@ Author: Tom Lane <[email-masked]>
 
 <listitem>
 <para>
-Add EXPLAIN option GENERIC_PLAN to display the query's generic plan (Laurenz Albe)
+Add EXPLAIN option GENERIC_PLAN to display the generic plan
+for a parameterized query (Laurenz Albe)
 </para>
 </listitem>
 
@@ -1542,7 +1544,7 @@ Author: Tom Lane <[email-masked]>
 
 <listitem>
 <para>
-Add VACUUM option to skip or update all frozen statistics (Tom Lane, Nathan Bossart)
+Add VACUUM options to skip or update all frozen statistics (Tom Lane, Nathan Bossart)
 </para>
 
 <para>
@@ -1635,13 +1637,23 @@ This can improve readability for long strings of digits.
 <!--
 Author: Tom Lane <[email-masked]>
 2023-01-01 [2ceea5adb] Accept "+infinity" in date and timestamp[tz] input.
+-->
+
+<listitem>
+<para>
+Accept the spelling "+infinity" in datetime input (Vik Fearing)
+</para>
+</listitem>
+
+<!--
 Author: Tom Lane <[email-masked]>
 2023-03-09 [bcc704b52] Reject combining "epoch" and "infinity" with other datet
 -->
 
 <listitem>
 <para>
-Prevent the specification of "epoch" and "infinity" with other units in datetime strings (Joseph Koshakow)
+Prevent the specification of "epoch" and "infinity" together with other fields
+in datetime strings (Joseph Koshakow)
 </para>
 </listitem>
 
@@ -1652,7 +1664,9 @@ Author: Tom Lane <[email-masked]>
 
 <listitem>
 <para>
-Remove support for datetime input that prefixes year-month-day by Y/M/D (Joseph Koshakow)
+Remove undocumented support for date input in the form
+"<literal>Y<replaceable>year</replaceable>M<replaceable>month</replaceable>D<replaceable>day</replaceable></literal>"
+(Joseph Koshakow)
 </para>
 </listitem>
 
@@ -1909,7 +1923,7 @@ Author: Tom Lane <[email-masked]>
 
 <listitem>
 <para>
-Allow to_reg* functions to accept OIDs parameters (Tom Lane)
+Allow to_reg* functions to accept numeric OIDs as input (Tom Lane)
 </para>
 </listitem>
 
@@ -2024,7 +2038,8 @@ Author: Tom Lane <[email-masked]>
 
 <listitem>
 <para>
-Allow ECPG variable declarations to use type names which match SQL keywords (Tom Lane)
+Allow ECPG variable declarations to use typedef names that match unreserved
+SQL keywords (Tom Lane)
 </para>
 
 <para>
@@ -2130,7 +2145,8 @@ Author: Tom Lane <[email-masked]>
 
 <listitem>
 <para>
-Allow psql to detect the exit status of shell commands and queries (Corey Huinker, Tom Lane)
+Allow psql scripts to obtain the exit status of shell commands and queries
+(Corey Huinker, Tom Lane)
 </para>
 
 <para>
@@ -2558,7 +2574,12 @@ Author: Andres Freund <[email-masked]>
 
 <listitem>
 <para>
-Prevent extension libraries from export their symbols by default (Andres Freund, Tom Lane)
+Prevent extension libraries from exporting their symbols by default (Andres Freund, Tom Lane)
+</para>
+
+<para>
+Functions that need to be called from the core backend or other extensions
+must now be explicitly marked PGDLLEXPORT.
 </para>
 </listitem>
 
@@ -2980,7 +3001,8 @@ Author: Tom Lane <[email-masked]>
 
 <listitem>
 <para>
-Allow the schemas of dependent extensions to be referenced using the new syntax @extschema:dependent_extension_name@ (Regina Obe)
+Allow the schemas of required extensions to be referenced in extension scripts
+using the new syntax @extschema:referenced_extension_name@ (Regina Obe)
 </para>
 </listitem>
 
@@ -2991,11 +3013,11 @@ Author: Tom Lane <[email-masked]>
 
 <listitem>
 <para>
-Allow dependent extensions to marked as non-relocatable using "no_relocate" (Regina Obe)
+Allow required extensions to marked as non-relocatable using "no_relocate" (Regina Obe)
 </para>
 
 <para>
-This allows @extschema:dependent_extension_name@ to be treated as a constant for the lifetime of the extension.
+This allows @extschema:referenced_extension_name@ to be treated as a constant for the lifetime of the extension.
 </para>
 </listitem>

---

From: Andres Freund <[email-masked]>
Date: Sun, 21 May 2023 10:13:41 -0700

Hi,

Thanks for the release notes!

> <!--
> Author: Andres Freund <[email-masked]>
> 2023-04-06 [00d1e02be] hio: Use ExtendBufferedRelBy() to extend tables more eff
> Author: Andres Freund <[email-masked]>
> 2023-04-06 [26158b852] Use ExtendBufferedRelTo() in XLogReadBufferExtended()
> -->
> 
> <listitem>
> <para>
> Allow more efficient addition of multiple heap and index pages (Andres Freund)
> </para>
> </listitem>

While the case of extending by multiple pages improved the most, even
extending by a single page at a time got a good bit more scalable. Maybe just
"Improve efficiency of extending relations"?

I think:

> <!--
> Author: Andres Freund <[email-masked]>
> 2023-04-08 [0fdab27ad] Allow logical decoding on standbys
> -->
> 
> <listitem>
> <para>
> Allow logical decoding on standbys (Bertrand Drouvot, Andres Freund, Amit Khandekar)
> </para>
> </listitem>

pretty much includes:

> <!--
> Author: Andres Freund <[email-masked]>
> 2023-04-07 [be87200ef] Support invalidating replication slots due to horizon an
> Author: Andres Freund <[email-masked]>
> 2023-04-08 [26669757b] Handle logical slot conflicts on standby
> -->
> 
> <listitem>
> <para>
> Allow invalidation of replication slots due to row removal, wal_level, and conflicts (Bertrand Drouvot, Andres Freund, Amit Khandekar)
> </para>

as it is a prerequisite.

I'd probably also merge

> <!--
> Author: Andres Freund <[email-masked]>
> 2023-04-08 [0fdab27ad] Allow logical decoding on standbys
> -->
> 
> <listitem>
> <para>
> Add function pg_log_standby_snapshot() to force creation of a WAL snapshot (Bertrand Drouvot)
> </para>
> 
> <para>
> WAL snapshots are required for logical slot creation so this function speeds their creation on standbys.
> </para>
> </listitem>

As there really isn't a use case outside of logical decoding on a standby.

> <!--
> Author: Andres Freund <[email-masked]>
> 2022-07-17 [089480c07] Default to hidden visibility for extension libraries whe
> Author: Andres Freund <[email-masked]>
> 2022-07-17 [8cf64d35e] Mark all symbols exported from extension libraries PGDLL
> -->
> 
> <listitem>
> <para>
> Prevent extension libraries from export their symbols by default (Andres Freund, Tom Lane)
> </para>
> </listitem>

s/export/exporting/?

Looking through the release notes, I didn't see an entry for

commit c6e0fe1f2a08505544c410f613839664eea9eb21
Author: David Rowley <[email-masked]>
Date:   2022-08-29 17:15:00 +1200
 
    Improve performance of and reduce overheads of memory management

even though I think that's one of the more impactful improvements. What was
the reason for leaving that out?

Greetings,

Andres Freund

---

From: "Jonathan S. Katz" <[email-masked]>
Date: Sun, 21 May 2023 14:46:56 -0400

> the reason for leaving that out?

IIUC in[1], would this "just speed up" read-heavy workloads?

Thanks,

Jonathan

[1] 
https://www.postgresql.org/message-id/CAApHDvpjauCRXcgcaL6%2Be3eqecEHoeRm9D-kcbuvBitgPnW%3Dvw%40mail.gmail.com

---

From: Andres Freund <[email-masked]>
Date: Sun, 21 May 2023 12:24:31 -0700

Hi,

>IIUC in[1], would this "just speed up" read-heavy workloads?

I don't think so. It can speed up write workloads as well. But more importantly it can noticeably reduce memory usage, including for things like the relcache.

Andres
-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

---

From: Bruce Momjian <[email-masked]>
Date: Sun, 21 May 2023 22:46:58 -0400

> "Improve efficiency of extending relations"?

Okay, I made this change:

	-Allow more efficient addition of multiple heap and index pages (Andres Freund)
	+Allow more efficient addition of heap and index pages (Andres Freund)

> > Allow invalidation of replication slots due to row removal, wal_level, and conflicts (Bertrand Drouvot, Andres Freund, Amit Khandekar)

Okay, I merged the commit entries and the authors, and removed the item.

> As there really isn't a use case outside of logical decoding on a standby.

Okay new merged item is:

	<!--
	
	<listitem>
	<para>
	Allow logical decoding on standbys (Bertrand Drouvot, Andres Freund, Amit Khandekar, Bertrand Drouvot)
	</para>
	</listitem>
	
	<listitem>
	<para>
	New function pg_log_standby_snapshot() forces creation of WAL snapshots.
	Snapshots are required for logical slot creation so this function speeds their creation on standbys.
	</para>
	</listitem>

> > Prevent extension libraries from export their symbols by default (Andres Freund, Tom Lane)

Seems Tom's patch already fixed that.

If you read my previous email:

> For the above two items, I mention items that would change user 

I can see this item as a big win, but I don't know how to describe it in a way
that is helpful for the user to know.

---

From: Bruce Momjian <[email-masked]>
Date: Sun, 21 May 2023 22:52:09 -0400

> "Improve efficiency of extending relations"?

Do average users know heap and index files are both relations?  That
seems too abstract so I spelled out heap and index pages.

---

From: John Naylor <[email-masked]>
Date: Tue, 23 May 2023 09:58:30 +0700

Hi Bruce,

> Add support for SSE2 (Streaming SIMD Extensions 2) vector operations on
x86-64 architectures (John Naylor)

> Add support for Advanced SIMD (Single Instruction Multiple Data) (NEON)
instructions on ARM architectures (Nathan Bossart)

Nit: It's a bit odd that SIMD is spelled out in only the Arm entry, and
perhaps expanding the abbreviations can be left out.

> Allow arrays searches to use vector operations on x86-64 architectures
(John Naylor)

We can leave out the architecture here (see below). Typo: "array searches"

All the above seem appropriate for the "source code" section, but the
following entries might be better in the "performance" section:

> Allow ASCII string detection to use vector operations on x86-64
architectures (John Naylor)
> Allow JSON string processing to use vector operations on x86-64
architectures (John Naylor)
> ARM?

Arm as well. For anything using 16-byte vectors the two architectures are
equivalently supported. For all the applications, I would just say "vector"
or "SIMD".

And here maybe /processing/parsing/.

> Allow xid/subxid searches to use vector operations on x86-64
architectures (Nathan Bossart)

When moved to the performance section, it would be something like "improve
scalability when a large number of write transactions are in progress".

-- 
John Naylor
EDB: http://www.enterprisedb.com

---

From: Bruce Momjian <[email-masked]>
Date: Tue, 23 May 2023 00:26:23 -0400

On Tue, May 23, 2023 at 09:58:30AM +0700, John Naylor wrote:
> Nit: It's a bit odd that SIMD is spelled out in only the Arm entry, and perhaps
> expanding the abbreviations can be left out.

The issue is that x86-64's SSE2 uses an embedded acronym:

	SSE2 (Streaming SIMD Extensions 2)

so technically it is:

	SSE2 (Streaming (Single Instruction Multiple Data) Extensions 2

but embedded acronyms is something I wanted to avoid.  ;-)

> > Allow arrays searches to use vector operations on x86-64 architectures (John
> Naylor)
> 

Both fixed.

> All the above seem appropriate for the "source code" section, but the following
> entries might be better in the "performance" section:
> 
> > Allow ASCII string detection to use vector operations on x86-64 architectures
> (John Naylor)
> > Allow JSON string processing to use vector operations on x86-64 architectures
> (John Naylor)
> >
> > ARM?
> 
> equivalently supported. For all the applications, I would just say "vector" or
> "SIMD".

Okay, I kept "vector".  I don't think moving them into performance makes
sense because there I don't think this would impact user behavior or
choice, and it can't be controlled.

> And here maybe /processing/parsing/.

Done.

> > Allow xid/subxid searches to use vector operations on x86-64 architectures
> (Nathan Bossart)
> 

Uh, again, see above, this does not impact user behavior or choices.  I
assume this is x86-64-only.

---

From: John Naylor <[email-masked]>
Date: Tue, 23 May 2023 12:14:04 +0700

> > > Allow ASCII string detection to use vector operations on x86-64
architectures
> > > Allow JSON string processing to use vector operations on x86-64
architectures
> choice, and it can't be controlled.

Well, these two items were only committed because of measurable speed
increases, and have zero effect on how developers work with "source code",
so that's a category error.

Whether they rise to the significance of warranting inclusion in release
notes is debatable.

> > > Allow xid/subxid searches to use vector operations on x86-64
architectures
> Uh, again, see above, this does not impact user behavior or choices.

So that turns a scalability improvement into "source code"?

> I assume this is x86-64-only.

Au contraire, I said "For anything using 16-byte vectors the two
architectures are equivalently supported". It's not clear from looking at
individual commit messages, that's why I piped in to help.

---

From: Bruce Momjian <[email-masked]>
Date: Wed, 24 May 2023 00:07:24 -0400

On Tue, May 23, 2023 at 12:14:04PM +0700, John Naylor wrote:
> Au contraire, I said "For anything using 16-byte vectors the two architectures
> are equivalently supported". It's not clear from looking at individual commit
> messages, that's why I piped in to help.

Okay, updated text:

	Allow xid/subxid searches to use vector operations (Nathan Bossart)

---

From: Bruce Momjian <[email-masked]>
Date: Wed, 24 May 2023 00:19:40 -0400

On Tue, May 23, 2023 at 12:14:04PM +0700, John Naylor wrote:
> or
> increases, and have zero effect on how developers work with "source code", so
> that's a category error.
> 
> Whether they rise to the significance of warranting inclusion in release notes
> is debatable.

Okay, let's dissect this.  First, I am excited about these features
because I think they show innovation, particularly for high scaling, so
I want to highlight this.

Second, you might be correct that the section is wrong.  I thought of
CPU instructions as something tied to the compiler, so part of the build
process or source code, but the point we should be make is that we have
these acceleration, not how it is implemented.  We can move the entire
group to the "General Performance" section, or we can split it out:

Keep in source code:

move to General Performance:

	Allow ASCII string detection to use vector operations (John Naylor)

and add these to data types:

	Allow JSON string parsing to use vector operations (John Naylor)

	Allow array searches to use vector operations (John Naylor)

---

From: John Naylor <[email-masked]>
Date: Wed, 24 May 2023 12:23:02 +0700

> group to the "General Performance" section, or we can split it out:

Splitting out like that seems like a good idea to me.

>         Allow ASCII string detection to use vector operations (John
Naylor)

(The ASCII part is most relevant for COPY FROM, just in case that matters.)

>         Allow array searches to use vector operations (John Naylor)

The last one refers to new internal functions, so it could stay in source
code. (Either way, we don't want to imply that arrays of SQL types are
accelerated this way, it's so far only for internal arrays.)

---

From: Bruce Momjian <[email-masked]>
Date: Wed, 24 May 2023 09:58:00 -0400

> Splitting out like that seems like a good idea to me. 

Okay, items split into sections and several merged.  I left the
CPU-specific parts in Source Code, and moved the rest into a merged item
in General Performance, but moved the JSON item to Data Types.

Patch attached, and you can see the results at:

> The last one refers to new internal functions, so it could stay in source code.
> (Either way, we don't want to imply that arrays of SQL types are accelerated
> this way, it's so far only for internal arrays.)

Good point.  I called them "C arrays" but it it into the General
Performance item.

  Only you can decide what is important to you.
commit ad5406246b
Author: Bruce Momjian <[email-masked]>
Date:   Wed May 24 09:54:34 2023 -0400

    doc: PG 16 relnotes, merge and move vector items
    
    Reported-by: John Naylor
    
    Discussion: https://postgr.es/m/CAFBsxsEPg8L2MmGqavc8JByC=[email-masked]

diff --git a/doc/src/sgml/release-16.sgml b/doc/src/sgml/release-16.sgml
index c30c530065..bb92fe5cf9 100644
--- a/doc/src/sgml/release-16.sgml
+++ b/doc/src/sgml/release-16.sgml
@@ -472,6 +472,28 @@ Author: David Rowley <[email-masked]>
 <para>
 Improve the speed of updating the process title (David Rowley)
 </para>
+</listitem>
+
+<!--
+Author: John Naylor <[email-masked]>
+2022-08-11 [37a6e5df3] Optimize xid/subxid searches in XidInMVCCSnapshot().
+Author: John Naylor <[email-masked]>
+2022-08-26 [121d2d3d7] Use SSE2 in is_valid_ascii() where available.
+Author: John Naylor <[email-masked]>
+2022-08-10 [b6ef16756] Introduce optimized routine for linear searches of array
+Author: John Naylor <[email-masked]>
+2022-08-26 [e813e0e16] Add optimized functions for linear search within byte ar
+-->
+
+<listitem>
+<para>
+Allow xid/subxid searches and ASCII string detection to use vector operations (Nathan Bossart)
+</para>
+
+<para>
+ASCII detection is particularly useful for COPY FROM.  Vector operations are also used for some C array searches.
+</para>
+
 </listitem>
 
      </itemizedlist>
@@ -1781,6 +1803,17 @@ The IS JSON checks include checks for values, arrays, objects, scalars, and uniq
 </para>
 </listitem>
 
+<!--
+Author: John Naylor <[email-masked]>
+2022-09-02 [0a8de93a4] Speed up lexing of long JSON strings
+-->
+
+<listitem>
+<para>
+Allow JSON string parsing to use vector operations (John Naylor)
+</para>
+</listitem>
+
 <!--
 Author: Tom Lane <[email-masked]>
 2023-01-19 [5a617d75d] Fix ts_headline() to handle ORs and phrase queries more 
@@ -2523,56 +2556,6 @@ Add support for Advanced SIMD (Single Instruction Multiple Data) (NEON) instruct
 </para>
 </listitem>
 
-<!--
-Author: John Naylor <[email-masked]>
-2022-08-26 [121d2d3d7] Use SSE2 in is_valid_ascii() where available.
--->
-
-<listitem>
-<para>
-Allow ASCII string detection to use vector operations (John Naylor)
-</para>
-</listitem>
-
-<!--
-Author: John Naylor <[email-masked]>
-2022-09-02 [0a8de93a4] Speed up lexing of long JSON strings
--->
-
-<listitem>
-<para>
-Allow JSON string parsing to use vector operations (John Naylor)
-</para>
-
-<para>
-ARM?
-</para>
-</listitem>
-
-<!--
-Author: John Naylor <[email-masked]>
-2022-08-10 [b6ef16756] Introduce optimized routine for linear searches of array
-Author: John Naylor <[email-masked]>
-2022-08-26 [e813e0e16] Add optimized functions for linear search within byte ar
--->
-
-<listitem>
-<para>
-Allow array searches to use vector operations (John Naylor)
-</para>
-</listitem>
-
-<!--
-Author: John Naylor <[email-masked]>
-2022-08-11 [37a6e5df3] Optimize xid/subxid searches in XidInMVCCSnapshot().
--->
-
-<listitem>
-<para>
-Allow xid/subxid searches to use vector operations (Nathan Bossart)
-</para>
-</listitem>
-
 <!--
 Author: Michael Paquier <[email-masked]>
 2022-08-28 [36389a060] Enable RandomizedBaseAddress (ASLR) on Windows with MSVC

---

From: Erik Rijkers <[email-masked]>
Date: Wed, 24 May 2023 16:57:59 +0200

>> On Wed, May 24, 2023 at 11:19 AM Bruce Momjian <[email-masked]> wrote:

Typos:

'from standbys servers'  should be
'from standby servers'

'reindexedb'  should be
'reindexdb'
   (2x: the next line mentions, erroneously,  'reindexedb --system')

'created only created'  should be
'only created'
   (I think)

'could could'  should be
'could'

'are now require the role'  should be
'now require the role'

'values is'  should be
'value is'

'to marked'  should be
'to be marked'

thanks,
Erik

---

From: Bruce Momjian <[email-masked]>
Date: Wed, 24 May 2023 12:17:51 -0400

> 'are now require the role'  should be

All good, patch attached and applied.  Updated docs are at:

index bb92fe5cf9..88d6514ad7 100644
@@ -27,7 +27,7 @@
 
     <listitem>
      <para>
-      Allow logical replication from standbys servers
+      Allow logical replication from standby servers
      </para>
     </listitem>
 
@@ -126,11 +126,11 @@ Author: Michael Paquier <[email-masked]>
 
 <listitem>
 <para>
-Change REINDEX DATABASE and reindexedb to not process indexes on system catalogs (Simon Riggs)
+Change REINDEX DATABASE and reindexdb to not process indexes on system catalogs (Simon Riggs)
 </para>
 
 <para>
-Processing such indexes is still possible using REINDEX SYSTEM and reindexedb --system.
+Processing such indexes is still possible using REINDEX SYSTEM and reindexdb --system.
 </para>
 </listitem>
 
@@ -593,7 +593,7 @@ Create subscription statistics entries at subscription creation time so stats_re
 </para>
 
 <para>
-Previously entries were created only created when the first statistics were reported.
+Previously entries were created only when the first statistics were reported.
 </para>
 </listitem>
 
@@ -777,7 +777,7 @@ Simplify permissions for LOCK TABLE (Jeff Davis)
 </para>
 
 <para>
-Previously the ability to perform LOCK TABLE at various lock levels was bound to specific query-type permissions.  For example, UPDATE could could perform all lock levels except ACCESS SHARE, which
+Previously the ability to perform LOCK TABLE at various lock levels was bound to specific query-type permissions.  For example, UPDATE could perform all lock levels except ACCESS SHARE, which
 required SELECT permissions.  Now UPDATE can issue all lock levels.  MORE?
 </para>
 </listitem>
@@ -808,7 +808,7 @@ Restrict the privileges of CREATEROLE roles (Robert Haas)
 </para>
 
 <para>
-Previously roles with CREATEROLE privileges could change many aspects of any non-superuser role.  Such changes, including adding members, are now require the role requesting the change to have ADMIN OPTION
+Previously roles with CREATEROLE privileges could change many aspects of any non-superuser role.  Such changes, including adding members, now require the role requesting the change to have ADMIN OPTION
 permission.
 </para>
 </listitem>
@@ -946,7 +946,7 @@ Add dependency tracking of grantors for GRANT records (Robert Haas)
 </para>
 
 <para>
-This will guarantee that pg_auth_members.grantor values is always valid.
+This guarantees that pg_auth_members.grantor values are always valid.
 </para>
 </listitem>
 
@@ -3017,7 +3017,7 @@ Author: Tom Lane <[email-masked]>
 
 <listitem>
 <para>
-Allow required extensions to marked as non-relocatable using "no_relocate" (Regina Obe)
+Allow required extensions to be marked as non-relocatable using "no_relocate" (Regina Obe)
 </para>
 
 <para>

---

From: Laurenz Albe <[email-masked]>
Date: Thu, 25 May 2023 23:51:24 +0200

On Thu, 2023-05-18 at 16:49 -0400, Bruce Momjian wrote:
> I have completed the first draft of the PG 16 release notes.

I found two typos.

Yours,
Laurenz Albe
diff --git a/doc/src/sgml/release-16.sgml b/doc/src/sgml/release-16.sgml
index faecae7c42..7dad0b8550 100644
--- a/doc/src/sgml/release-16.sgml
+++ b/doc/src/sgml/release-16.sgml
@@ -1294,7 +1294,7 @@ Determine the ICU default locale from the environment (Jeff Davis)
 </para>
 
 <para>
-However, ICU doesn't support the C local so UTF-8 is used in such cases.  Previously the default was always UTF-8.
+However, ICU doesn't support the C locale so UTF-8 is used in such cases.  Previously the default was always UTF-8.
 </para>
 </listitem>
 
@@ -1335,7 +1335,7 @@ Author: Peter Eisentraut <[email-masked]>
 
 <listitem>
 <para>
-Add Windows process the system collations (Jose Santamaria Flecha)
+Add Windows to process the system collations (Jose Santamaria Flecha)
 ADD THIS?
 </para>
 </listitem>

---

From: Alvaro Herrera <[email-masked]>
Date: Fri, 26 May 2023 12:21:23 +0200

> +Add Windows to process the system collations (Jose Santamaria Flecha)

Hmm, not sure this describes the change properly.  Maybe something like
"On Windows, system locales are now imported automatically.  Previously,
only ICU locales were imported automatically on Windows."

Maybe the Windows improvements should be listed together in a separate
section.

Also, "Juan José Santamaría Flecha" is the spelling Juan José uses for
his name.

-- 
Álvaro Herrera         PostgreSQL Developer  —  https://www.EnterpriseDB.com/

---

From: Bruce Momjian <[email-masked]>
Date: Sat, 27 May 2023 22:21:29 -0400

> +However, ICU doesn't support the C locale so UTF-8 is used in such cases.  Previously the default was always UTF-8.

I have made this change.

> +Add Windows to process the system collations (Jose Santamaria Flecha)

I will deal with this item in the email from Álvaro Herrera.

---

From: Bruce Momjian <[email-masked]>
Date: Sat, 27 May 2023 23:03:16 -0400

> Also, "Juan José Santamaría Flecha" is the spelling Juan José uses for

Okay, I reword this and fixed Juan's name, attached, and applied.

index faecae7c42..1db72eeef3 100644
 
 <para>
 
@@ -1335,8 +1335,11 @@ Author: Peter Eisentraut <[email-masked]>
 
 <listitem>
 <para>
-Add Windows process the system collations (Jose Santamaria Flecha)
-ADD THIS?
+Allow Windows to import system locales automatically (Juan José Santamaría Flecha)
+</para>
+
+<para>
+Previously, only ICU locales could be imported on Windows.
 </para>
 </listitem>

---

From: Masahiko Sawada <[email-masked]>
Date: Tue, 30 May 2023 06:33:09 -0400

Hi,

> I have completed the first draft of the PG 16 release notes.  You can

I have one suggestion on this item:

<!--
Author: Amit Kapila <[email-masked]>
2022-07-21 [366283961] Allow users to skip logical replication of data having o
Author: Amit Kapila <[email-masked]>
2022-09-08 [875693019] Raise a warning if there is a possibility of data from m
-->

<listitem>
<para>
Allow logical replication subscribers to process only changes that
have no origin (Vignesh C, Amit Kapila)
</para>

<para>
This can be used to avoid replication loops.
</para>
</listitem>

I think it's better to mention the new 'origin' option as other new
subscription options are mentioned. For example,

<para>
This can be used to avoid replication loops. This can be controlled by
the subscription "origin" option.
</para>

Regards,

--
Masahiko Sawada

---

From: Bruce Momjian <[email-masked]>
Date: Tue, 30 May 2023 19:07:37 -0400

> the subscription "origin" option.

Great, new text is:

	This can be used to avoid replication loops.  This is controlled
	by the new CREATE SUBSCRIPTION "origin" option.

---

From: Masahiko Sawada <[email-masked]>
Date: Thu, 8 Jun 2023 14:23:33 +0900

Hi,

<!--
Author: Michael Paquier <[email-masked]>
2023-03-14 [5c1b66280] Rework design of functions in pg_walinspect
-->

<listitem>
<para>
Remove pg_walinspect functions
pg_get_wal_records_info_till_end_of_wal() and
pg_get_wal_stats_till_end_of_wal().
</para>
</listitem>

I found that this item misses the author, Bharath Rupireddy. Please
add his name.

Regards,

---

From: Bruce Momjian <[email-masked]>
Date: Fri, 9 Jun 2023 21:04:36 -0400

> I found that this item misses the author, Bharath Rupireddy. Please

Thanks, fixed.

---

From: Roberto Mello <[email-masked]>
Date: Tue, 27 Jun 2023 15:49:44 -0600

Adding to this thread as suggested by jkatz for consideration of
adding to release notes...

In [1] I mention the omission of ldap_password_hook and a suggested paragraph.

Roberto

[1] https://www.postgresql.org/message-id/CAKz%3D%3DbLzGb-9O294AoZHqEWpAi2Ki58yCr4gaqg1HnZyh3L1uA%40mail.gmail.com

---

From: Bruce Momjian <[email-masked]>
Date: Tue, 27 Jun 2023 22:56:07 -0400

> [1] https://www.postgresql.org/message-id/CAKz%3D%3DbLzGb-9O294AoZHqEWpAi2Ki58yCr4gaqg1HnZyh3L1uA%40mail.gmail.com

I did see that commit:

	commit 419a8dd814
	Author: Andrew Dunstan <[email-masked]>
	Date:   Wed Mar 15 16:37:28 2023 -0400
	
	    Add a hook for modifying the ldapbind password
	
	    The hook can be installed by a shared_preload library.
	
	    A similar mechanism could be used for radius paswords, for example, and
	    the type name auth_password_hook_typ has been shosen with that in mind.
	
	    John Naylor and Andrew Dunstan
	
	    Discussion: https://postgr.es/m/[email-masked]

However, there is no user documentation of this hook, so it didn't seem
like something to add to the release notes.

---

From: Masahiro Ikeda <[email-masked]>
Date: Fri, 30 Jun 2023 17:29:17 +0900

Hi,

Thanks for making the release notes. I found the release note of
PG16 beta2 mentions a reverted following feature.

```
<!--
Author: Jeff Davis <[email-masked]>
2023-03-09 [27b62377b] Use ICU by default at initdb time.
-->

<listitem>
<para>
Have initdb use ICU by default if ICU is enabled in the binary (Jeff 
Davis)
</para>

<para>
Option --locale-provider=libc can be used to disable ICU.
</para>
</listitem>
```

Unfortunately, the feature is reverted with the commit.
* 
https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=2535c74b1a6190cc42e13f6b6b55d94bff4b7dd6

Regards,
-- 
Masahiro Ikeda
NTT DATA CORPORATION

---

From: Bruce Momjian <[email-masked]>
Date: Fri, 30 Jun 2023 17:36:08 -0400

On Fri, Jun 30, 2023 at 05:29:17PM +0900, Masahiro Ikeda wrote:
> Have initdb use ICU by default if ICU is enabled in the binary (Jeff Davis)
> </para>
> 
> <para>
> * https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=2535c74b1a6190cc42e13f6b6b55d94bff4b7dd6

Oh, I didn't notice the revert --- item removed.

---

From: Michael Paquier <[email-masked]>
Date: Tue, 4 Jul 2023 15:31:05 +0900

> I will adjust it to the feedback I receive;  that URL will quickly show

Sawada-san has mentioned on twitter that fdd8937 is not mentioned in
the release notes, and it seems to me that he is right.  This is
described as a bug in the commit log, but it did not get backpatched
because of the lack of complaints.  Also, because we've removed
support for anything older than Windows 10 in PG16, this change very
easy to do.
--
Michael

---

From: Laurenz Albe <[email-masked]>
Date: Fri, 14 Jul 2023 20:16:38 +0200

> I have completed the first draft of the PG 16 release notes.

The release notes say:

- Prevent \df+ from showing function source code (Isaac Morland)

  Function bodies are more easily viewed with \ev and \ef.

That should be \sf, not \ev or \ef, right?

Yours,
Laurenz Albe

---

From: Laurenz Albe <[email-masked]>
Date: Fri, 14 Jul 2023 20:20:59 +0200

> I have completed the first draft of the PG 16 release notes.

The release notes still have:

- Have initdb use ICU by default if ICU is enabled in the binary (Jeff Davis)

But this was reverted in 2535c74b1a6190cc42e13f6b6b55d94bff4b7dd6.

Yours,
Laurenz Albe

---

From: Noah Misch <[email-masked]>
Date: Sat, 5 Aug 2023 16:08:47 -0700

> <!--
> Author: Robert Haas <[email-masked]>
> 2023-01-10 [cf5eb37c5] Restrict the privileges of CREATEROLE users.
> -->
> 
> <listitem>
> <para>
> Restrict the privileges of CREATEROLE roles (Robert Haas)
> </para>
> 
> <para>
> Previously roles with CREATEROLE privileges could change many aspects of any non-superuser role.  Such changes, including adding members, now require the role requesting the change to have ADMIN OPTION
> permission.
> </para>
> </listitem>
> 
> <!--
> Author: Robert Haas <[email-masked]>
> 2023-01-24 [f1358ca52] Adjust interaction of CREATEROLE with role properties.
> -->
> 
> <listitem>
> <para>
> Improve logic of CREATEROLE roles ability to control other roles (Robert Haas)
> </para>
> 
> <para>
> For example, they can change the CREATEDB, REPLICATION, and BYPASSRLS properties only if they also have those permissions.
> </para>
> </listitem>

CREATEROLE is a radically different feature in v16.  In v15-, it was an
almost-superuser.  In v16, informally speaking, it can create and administer
its own collection of roles, but it can't administer roles outside its
collection or grant memberships or permissions not offered to itself.  Hence,
let's move these two into the incompatibilities section.  Let's also merge
them, since f1358ca52 is just doing to clauses like CREATEDB what cf5eb37c5
did to role memberships.

> <!--
> Author: Robert Haas <[email-masked]>
> 2022-08-25 [e3ce2de09] Allow grant-level control of role inheritance behavior.
> -->
> 
> <listitem>
> <para>
> Allow GRANT to control role inheritance behavior (Robert Haas)
> </para>
> 
> <para>
> By default, role inheritance is controlled by the inheritance status of the member role.  The new GRANT clauses WITH INHERIT and WITH ADMIN can now override this.
> </para>
> </listitem>
> 
> <!--
> Author: Robert Haas <[email-masked]>
> 2023-01-10 [e5b8a4c09] Add new GUC createrole_self_grant.
> Author: Daniel Gustafsson <[email-masked]>
> 2023-02-22 [e00bc6c92] doc: Add default value of createrole_self_grant
> -->
> 
> <listitem>
> <para>
> Allow roles that create other roles to automatically inherit the new role's rights or SET ROLE to the new role (Robert Haas, Shi Yu)
> </para>
> 
> <para>
> This is controlled by server variable createrole_self_grant.
> </para>
> </listitem>

Similarly, v16 radically changes the CREATE ROLE ... WITH INHERIT clause.  The
clause used to "change the behavior of already-existing grants."  Let's merge
these two and move the combination to the incompatibilities section.

> Remove libpq support for SCM credential authentication (Michael Paquier)

Since the point of removing it is the deep unlikelihood of anyone using it, I
wouldn't list this in "incompatibilities".

> Deprecate createuser option --role (Nathan Bossart)

This is indeed a deprecation, not a removal.  By the definition of
"deprecate", it's not an incompatibility.

---

From: Bruce Momjian <[email-masked]>
Date: Wed, 9 Aug 2023 13:24:12 -0400

> That should be \sf, not \ev or \ef, right?

Agreed, fixed.  I am not sure why I put \ev and \ef there.

---

From: Bruce Momjian <[email-masked]>
Date: Wed, 9 Aug 2023 13:29:18 -0400

> But this was reverted in 2535c74b1a6190cc42e13f6b6b55d94bff4b7dd6.

FYI, this was corrected in this commit:

	commit c729642bd7
	Author: Bruce Momjian <[email-masked]>
	Date:   Fri Jun 30 17:35:47 2023 -0400
	
	    doc: PG 16 relnotes, remove "Have initdb use ICU by default"
	
	    Item reverted.
	
	    Backpatch-through: 16 only

---

From: Bruce Momjian <[email-masked]>
Date: Wed, 9 Aug 2023 18:03:15 -0400

> them, since f1358ca52 is just doing to clauses like CREATEDB what cf5eb37c5

Good point. I have adjusted this item with the attached patch.

  Only you can decide what is important to you.
diff --git a/doc/src/sgml/release-16.sgml b/doc/src/sgml/release-16.sgml
index 1213f876f4..cccdc01d11 100644
--- a/doc/src/sgml/release-16.sgml
+++ b/doc/src/sgml/release-16.sgml
@@ -244,6 +244,24 @@ Collations and locales can vary between databases so having them as read-only se
 </para>
 </listitem>
 
+<!--
+Author: Robert Haas <[email-masked]>
+2023-01-10 [cf5eb37c5] Restrict the privileges of CREATEROLE users.
+Author: Robert Haas <[email-masked]>
+2023-01-24 [f1358ca52] Adjust interaction of CREATEROLE with role properties.
+-->
+
+<listitem>
+<para>
+Restrict the privileges of CREATEROLE and its ability to modify other roles (Robert Haas)
+</para>
+
+<para>
+Previously roles with CREATEROLE privileges could change many aspects of any non-superuser role.  Such changes, including adding members, now require the role requesting the change to have ADMIN OPTION
+permission.  For example, they can now change the CREATEDB, REPLICATION, and BYPASSRLS properties only if they also have those permissions.
+</para>
+</listitem>
+
 <!--
 Author: Nathan Bossart <[email-masked]>
 2023-05-21 [2dcd1578c] Rename some createuser options.
@@ -822,37 +840,6 @@ Previously CREATEROLE permission was required.
 </para>
 </listitem>
 
-<!--
-Author: Robert Haas <[email-masked]>
-2023-01-10 [cf5eb37c5] Restrict the privileges of CREATEROLE users.
--->
-
-<listitem>
-<para>
-Restrict the privileges of CREATEROLE roles (Robert Haas)
-</para>
-
-<para>
-Previously roles with CREATEROLE privileges could change many aspects of any non-superuser role.  Such changes, including adding members, now require the role requesting the change to have ADMIN OPTION
-permission.
-</para>
-</listitem>
-
-<!--
-Author: Robert Haas <[email-masked]>
-2023-01-24 [f1358ca52] Adjust interaction of CREATEROLE with role properties.
--->
-
-<listitem>
-<para>
-Improve logic of CREATEROLE roles ability to control other roles (Robert Haas)
-</para>
-
-<para>
-For example, they can change the CREATEDB, REPLICATION, and BYPASSRLS properties only if they also have those permissions.
-</para>
-</listitem>
-
 <!--
 Author: Robert Haas <[email-masked]>
 2022-08-25 [e3ce2de09] Allow grant-level control of role inheritance behavior.

---

From: Bruce Momjian <[email-masked]>
Date: Wed, 9 Aug 2023 20:35:21 -0400

> these two and move the combination to the incompatibilities section.

I need help with this.  I don't understand how they can be combined, and
I don't understand the incompatibility text in commit e3ce2de09d:

    If a GRANT does not specify WITH INHERIT, the behavior based on
    whether the member role is marked INHERIT or NOINHERIT. This means
    that if all roles are marked INHERIT or NOINHERIT before any role
    grants are performed, the behavior is identical to what we had before;
    otherwise, it's different, because ALTER ROLE [NO]INHERIT now only
    changes the default behavior of future grants, and has no effect on
    existing ones.

---

From: Bruce Momjian <[email-masked]>
Date: Wed, 9 Aug 2023 20:48:16 -0400

> wouldn't list this in "incompatibilities".

I moved this to the Source Code section.

> "deprecate", it's not an incompatibility.

I moved this to the Server Applications section.

Thanks.

---

From: Bruce Momjian <[email-masked]>
Date: Wed, 9 Aug 2023 22:11:04 -0400

FYI, the current PG 16 release notes are available at:

I plan to add markup next week.  I am sorry I was away most of July so
could not update this until now.

---

From: Bruce Momjian <[email-masked]>
Date: Wed, 16 Aug 2023 22:36:05 -0400

You can view the Postgres 16 release notes, with markup and links to our
docs, here:

---

From: Bruce Momjian <[email-masked]>
Date: Wed, 16 Aug 2023 22:36:45 -0400

>     changes the default behavior of future grants, and has no effect on

I am waiting for an answer to this question, or can I assume the release
notes are acceptable?

---

From: Pavel Luzanov <[email-masked]>
Date: Thu, 17 Aug 2023 08:37:28 +0300

> I am waiting for an answer to this question, or can I assume the release

I can try to explain how I understand it myself.

In v15 and early, inheritance of granted to role privileges depends on 
INHERIT attribute of a role:

create user alice;
grant pg_read_all_settings to alice;

By default privileges inherited:
\c - alice
show data_directory;
        data_directory
-----------------------------
  /var/lib/postgresql/15/main
(1 row)

After disabling the INHERIT attribute, privileges are not inherited:

\c - postgres
alter role alice noinherit;

\c - alice
show data_directory;
ERROR:  must be superuser or have privileges of pg_read_all_settings to 
examine "data_directory"

In v16 changing INHERIT attribute on alice role doesn't change 
inheritance behavior of already granted roles.
If we repeat the example, Alice still inherits pg_read_all_settings 
privileges after disabling the INHERIT attribute for the role.

Information for making decisions about role inheritance has been moved 
from the role attribute to GRANT role TO role [WITH INHERIT|NOINHERIT] 
command and can be viewed by the new \drg command:

\drg
                     List of role grants
  Role name |      Member of       |   Options    | Grantor
-----------+----------------------+--------------+----------
  alice     | pg_read_all_settings | INHERIT, SET | postgres
(1 row)

Changing the INHERIT attribute for a role now will affect (as the 
default value) only future GRANT commands without an INHERIT clause.

-- 
Pavel Luzanov
Postgres Professional: https://postgrespro.com

---

From: Bruce Momjian <[email-masked]>
Date: Sat, 19 Aug 2023 12:59:47 -0400

On Thu, Aug 17, 2023 at 08:37:28AM +0300, Pavel Luzanov wrote:
> In v16 changing INHERIT attribute on alice role doesn't change inheritance
> behavior of already granted roles.
> Information for making decisions about role inheritance has been moved from
> the role attribute to GRANT role TO role [WITH INHERIT|NOINHERIT] command
> and can be viewed by the new \drg command:
> 
> \drg
>                     List of role grants
> Changing the INHERIT attribute for a role now will affect (as the default
> value) only future GRANT commands without an INHERIT clause.

I was able to create this simple example to illustrate it:

	CREATE ROLE a1;
	CREATE ROLE a2;
	CREATE ROLE a3;
	CREATE ROLE a4;
	CREATE ROLE b INHERIT;

	GRANT a1 TO b WITH INHERIT TRUE;
	GRANT a2 TO b WITH INHERIT FALSE;

	GRANT a3 TO b;
	ALTER USER b NOINHERIT;
	GRANT a4 TO b;

	\drg
	               List of role grants
	 Role name | Member of |   Options    | Grantor
	-----------+-----------+--------------+----------
	 b         | a1        | INHERIT, SET | postgres
	 b         | a2        | SET          | postgres
	 b         | a3        | INHERIT, SET | postgres
	 b         | a4        | SET          | postgres

I will work on the relase notes adjustments for this and reply in a few
days.

---

From: Bruce Momjian <[email-masked]>
Date: Mon, 21 Aug 2023 17:58:36 -0400

> 	 Role name | Member of |   Options    | Grantor

Attached is an applied patch that moves the inherit item into
incompatibilities. clarifies it, and splits out the ADMIN syntax item.

Please let me know if I need any other changes.  Thanks.

  Only you can decide what is important to you.
diff --git a/doc/src/sgml/release-16.sgml b/doc/src/sgml/release-16.sgml
index c9c4fc07ca..c4ae566900 100644
--- a/doc/src/sgml/release-16.sgml
+++ b/doc/src/sgml/release-16.sgml
@@ -229,6 +229,24 @@ Collations and locales can vary between databases so having them as read-only se
 </para>
 </listitem>
 
+<!--
+Author: Robert Haas <[email-masked]>
+2022-08-25 [e3ce2de09] Allow grant-level control of role inheritance behavior.
+-->
+
+<listitem>
+<para>
+Role inheritance now controls the default inheritance status of member roles added during <link linkend="sql-grant"><command>GRANT</command></link> (Robert Haas)
+</para>
+
+<para>
+The role's default inheritance behavior can be overridden with the new <command>GRANT ... WITH INHERIT</command> clause.
+This allows inheritance of some roles and not others because the members' inheritance status is set at <command>GRANT</command> time.
+Previously the inheritance status of member roles was controlled only by the role's inheritance status, and
+changes to a role's inheritance status affected all previous and future member roles.
+</para>
+</listitem>
+
 <!--
 Author: Robert Haas <[email-masked]>
 2023-01-10 [cf5eb37c5] Restrict the privileges of CREATEROLE users.
@@ -814,11 +832,11 @@ Author: Robert Haas <[email-masked]>
 
 <listitem>
 <para>
-Allow <link linkend="sql-grant"><command>GRANT</command></link> to control role inheritance behavior (Robert Haas)
+Allow <link linkend="sql-grant"><command>GRANT</command></link> to use <literal>WITH ADMIN TRUE</literal>/<literal>FALSE</literal> syntax (Robert Haas)
 </para>
 
 <para>
-By default, role inheritance is controlled by the inheritance status of the member role.  The new <command>GRANT</command> clauses <literal>WITH INHERIT</literal> and <literal>WITH ADMIN</literal> can now override this.
+Previously only the <literal>WITH ADMIN OPTION</literal> syntax was supported.
 </para>
 </listitem>

---

From: Pavel Luzanov <[email-masked]>
Date: Tue, 22 Aug 2023 10:02:16 +0300

> The role's default inheritance behavior can be overridden with the 
new <command>GRANT ... WITH INHERIT</command> clause.

The only question about "can be". Why not "will be"? The inheritance 
behavior will be changed anyway.

> Please let me know if I need any other changes.  Thanks.

* Allow psql's access privilege commands to show system objects (Nathan 
Bossart, Pavel Luzanov)
     The options are \dpS, \zS, and \drg.

I think that this description correct only for the \dpS and \zS commands.
(By the way, unfortunately after reverting MAINTAIN privilege this 
commands are not much useful in v16.)

But the \drg command is a different thing. This is a full featured 
replacement for "Member of" column of the \du, \dg commands.
It shows not only members, but granted options (admin, inherit, set) and 
grantor.
This is important information for membership usage and administration.
IMO, removing the "Member of" column from the \du & \dg commands also 
requires attention in release notes.

So, I suggest new item in the psql section for \drg. Something like this:

* Add psql \drg command to display role grants and remove the "Member 
of" column from \du & \dg altogether.

---

From: Bruce Momjian <[email-masked]>
Date: Tue, 22 Aug 2023 15:14:31 -0400

On Tue, Aug 22, 2023 at 10:02:16AM +0300, Pavel Luzanov wrote:
> > The role's default inheritance behavior can be overridden with the new
> <command>GRANT ... WITH INHERIT</command> clause.
> 

I used "can be" to highlight you "can" override it, but don't need to.

> (By the way, unfortunately after reverting MAINTAIN privilege this commands
> are not much useful in v16.)
> 
> * Add psql \drg command to display role grants and remove the "Member of"
> column from \du & \dg altogether.

I see your point.  Attached is an applied patch which fixes this by
splitting \drg into a separate item and adding text.

  Only you can decide what is important to you.
diff --git a/doc/src/sgml/release-16.sgml b/doc/src/sgml/release-16.sgml
index 1f98ccdc62..c464be5ee1 100644
--- a/doc/src/sgml/release-16.sgml
+++ b/doc/src/sgml/release-16.sgml
@@ -2176,11 +2176,28 @@ Author: Tom Lane <[email-masked]>
 
 <listitem>
 <para>
-Allow <application>psql</application>'s access privilege commands to show system objects (Nathan Bossart, Pavel Luzanov)
+Add <application>psql</application> command <link linkend="app-psql-meta-command-drg"><command>\drg</command></link> to show role membership details (Pavel Luzanov)
 </para>
 
 <para>
-The options are <link linkend="app-psql-meta-command-dp-lc"><command>\dpS</command></link>, <link linkend="app-psql-meta-command-z"><command>\zS</command></link>, and <link linkend="app-psql-meta-command-drg"><command>\drg</command></link>.
+The <literal>Member of</literal> output column has been removed from <command>\du</command> and <command>\dg</command> because this new command displays this informaion in more detail.
+</para>
+</listitem>
+
+<!--
+Author: Dean Rasheed <[email-masked]>
+2023-01-07 [d913928c9] psql: Add support for \dpS and \zS.
+Author: Tom Lane <[email-masked]>
+2023-07-19 [d65ddaca9] Add psql \drg command to display role
+-->
+
+<listitem>
+<para>
+Allow <application>psql</application>'s access privilege commands to show system objects (Nathan Bossart)
+</para>
+
+<para>
+The options are <link linkend="app-psql-meta-command-dp-lc"><command>\dpS</command></link> and <link linkend="app-psql-meta-command-z"><command>\zS</command></link>.
 </para>
 </listitem>

---

From: Jeff Davis <[email-masked]>
Date: Tue, 22 Aug 2023 13:42:41 -0700

> I have completed the first draft of the PG 16 release notes.  You can

https://www.postgresql.org/docs/16/release-16.html#RELEASE-16-LOCALIZATION

I notice that this item is still listed:

 * Determine the ICU default locale from the environment (Jeff Davis)

But that was reverted as part of 2535c74b1a.

Regards,
	Jeff Davis