Subject: Re: Improving connection scalability: GetSnapshotData()
To: Alexander Korotkov <a.korotkov@postgrespro.ru>,
 Andres Freund <andres@anarazel.de>
Cc: PostgreSQL Hackers <pgsql-hackers@postgresql.org>,
 Peter Geoghegan <pg@bowt.ie>, Alvaro Herrera <alvherre@2ndquadrant.com>,
 Bruce Momjian <bruce@momjian.us>, Thomas Munro <thomas.munro@gmail.com>,
 David Rowley <dgrowleyml@gmail.com>
References: <20200301083601.ews6hz5dduc3w2se@alap3.anarazel.de>
 <CAH2-WzmNpBm0vqOaxzJhd-31SZVU904xrZvWPp1RyGOEFZQTug@mail.gmail.com>
 <CAPpHfdsJUWeZ=kCUZuA5oADyustFQfUrWMimpD4YL-n8tqShhQ@mail.gmail.com>
 <F0E7D264-FEA3-47D3-815E-C834569EBCDA@anarazel.de>
 <CAPpHfdthCaCt09Nnj6oCYq93_PUfJSw5QOLGgnid3qJsEWXuGA@mail.gmail.com>
 <20200406133959.viql5fqecog6mppj@alap3.anarazel.de>
 <20200406235228.lnxcau2wgocnnn22@alap3.anarazel.de>
 <20200407121503.zltbpqmdesurflnm@alap3.anarazel.de>
 <20200408124318.amzvnpsqhy747wqp@alap3.anarazel.de>
 <CAPpHfdt78x-aOhbTdY01HJ-ydM9ZSo-sRJNi0pwk7WmWd0xxAA@mail.gmail.com>
From: "Jonathan S. Katz" <jkatz@postgresql.org>
Autocrypt: addr=jkatz@postgresql.org; keydata=
 mQINBFtShwABEAC7PNHDUOTYuifpUCk23KqfxdQQkn4nkoxOXRK0+rAj36FiwqGB4TJFuOVZ
 sDFAEiSlC8Jt4y5Cs7B5tetT8JNd2cs6zp/udMJJDz9d65O9PDpdlMgAmIiTzpLlSdx8FG56
 DTksaDv1d8j3cTJPSE4/fWSxqzA7o3Y9UuL7atZPrzfImgpRKs0of7elIHwOa8GucjyhYqcR
 h60wFBJc2KXqQdDYRTZy43DSnY/0VNc0omiH355fustvpm+m5HjD3w7qZyfN3fpKJpnX1LCF
 f3MnPHaDGITIYGRCBXvf0UqUtD6OEVWPv2C2gyqWMIpWmZTOgDufltKyIByKBoS9x0PlFkij
 04X3KODCngt+N8Ssc9OICc6QSxhjoP48PYPdmiTmkrGuf0LX084wj1xeo1NX7XxZK39F6dTJ
 DhsIiW0sNS0xMxQHLHG9VLbPjx3SANQBh6BuryPz5ZupW9/TIDmkvprtU/oXfKgtfYm3fxmk
 EctxbWrEPsFTFPyuMqQu6l+xyQv0s1VLZfjNWaua6H1/gGoIt6kRnn5qMXDVVpijuWkHbv7G
 ngaQMd258UrrOEHnnjzhQ7jxMWV9D+emxbAtlIxnYvCWlV4IwAQhEHfvudqYaIY3hNWrvQ6H
 GB2KXoTZYN9g5djm14/5nj1IU5zOcovkjJnKhoo9iStnpFF2cwARAQABtCdKb25hdGhhbiBT
 LiBLYXR6IDxqa2F0ekBwb3N0Z3Jlc3FsLm9yZz6JAlQEEwEIAD4CGwMFCQeGH4AFCwkIBwIG
 FQoJCAsCBBYCAwECHgECF4AWIQT6hLaVryv3miBkP/HxBJxynxxlJwUCW1KJJQAKCRDxBJxy
 nxxlJwjrD/kBgqsW4QpNpTFw7ifRokZV08CCX4huPBJQ91rrv+UEWlEcotFBHVkYyHnpzARl
 tcZxhJ9CbFxjniH9cOTty5T/O1yolbOHtZSW8Z8aWV6BVEbjMb+BFxSSLm7RnvJdzQbGCZq2
 ZZvfVpB6z3EHYph4KDdVKvMFjoLskxmdS1DE0tE3zTxvoQsi24Q+HOS07kUjs6fsu/WICMfz
 mgO++AWG9Y0CvN0mm4TkujESzyKM9E5irD+leEMIcddl51Aa2c/VMfBXQbRmpHIgUFTmuHQD
 CnQih+9i3OJAksDg66SP8a7yiXv5mwvyDi1EfTGVKYR2j+pwyjwnC3oIbvDMmB3uTn2JIjnT
 iZKPVtAcAylXjubFltihQgNyuShdP4W+kBwZizhUFqUVL8Anx+KoytYmJPfMRFLGuK4obXKq
 a2ZS3k9KB+H+isOx2nFJOsc7V360Zp1DVaNmuiK10TT6QndShSPaqkJqFtCb6r92rZ9sZM/L
 3vtCI4Rrl3Pt1MgtENXupS8gZpJnAYS0j5A1PAZ09r6ANoaeMHspF+5J5fOHeEvqphXr36mm
 a83Vl1t4orPb0+QmmijmlpseDU63M88Aw5p3c4qj7t8Qr2EZ5zrn7/sFn5wOfbs8Nymxafif
 QCnlV2vg9p0m7vSk/yLJ4PFZvs52FgqAGRCdRn0s2EC99bkCDQRbUocAARAAv8ho/toQ9DG3
 j4f9h9n1aRHr2FlviN2Utpy6L8+dfDggO0geilmkGQOolZ2E60gGfye/kUtF9W3NByO4hxDR
 9u6qbOXcdqnuA+cc68EfqlWFJrVtYFxt0h4ElWYOYnIezKthriWch/FY70FGrxs3z8UHOHq5
 0wBW433eTvZm90WixBiXEt2v1DgW4Vr3ymfO7Aap/IYyPuE4JzgudAuAl0HKPyNEHWHG1dAb
 jX1RiCw9gknIDWQOF0B4UAaJctWGVcnZ3A2ULwNGMa1P9ZJlBWf1vcj01aiHMU0yQ7JjJiSp
 vfm9eM0uSLwRdDrJjyl5ZZqVumjdv2SMNQ8GvYRbEMys3GGDSt9zXgfCSUnPnJfYxjzBHRI6
 x44Wfsx8S6hWxepOogCJJ/g67Bk9mY8YV4klWIXDJVOL5jnBC09DbsZG81JaE2QxB8Y7W36Z
 Mroi9XMxg3s805hQAQUvdG/poU8hN8BWdrnTm/+4eQQp7gDY1ePDmGM6bJC+OHOSnFtR/f+7
 0zpKJ10cc7cBygGnl1yR3KjhFyAWUFvP4ZGziKCcpMwXZfe9PGuyA/YOubMphxIn3YsK2wrd
 faKZYX2GMZCZhMMcvx9IpQrxIJgU+VlwXu/O+Lk10VIPcxPJJwmpdI6HzcS8ZgG6IMcC444X
 XTuLaP8j2mgcMvYak3ScCykAEQEAAYkCPAQYAQgAJhYhBPqEtpWvK/eaIGQ/8fEEnHKfHGUn
 BQJbUocAAhsMBQkHhh+AAAoJEPEEnHKfHGUnReMP/RA2UhGQj+G4uBshkRLjpRysabdPqgQB
 dEBk6wYbio88Wg/2/hgY7UzmDDEwX3sZfQDcrI6+vIobI8uqstZID+WgAAa1JLfChMyVQnSy
 0zfWMOABXscc5tGuvFRZvJklTissMFjXUwaREEKp4ZikTvJ/62MCjSdtrUhnPLvoTsHTKRKD
 ichE+b5A54alwsubTijw12O6N22r5IjZiiSZV0u7dsShyKw+7wCSax9fuBoE43NMYf+dnjMK
 nerAQYUcZWYMnk+EC8RaqYAxv6XZ2tKx1AkGGktwkQIBwrz9IlDSvJ3LWJ2UIIuLRTdngNgg
 GIL4zzuUa2F56FqskQIuYMaNETk6LYfalBDQ6TVLAcgCPQxp4k4i/PRsZ3lZ2ZhRHRYciOvm
 kp+I6EfHwllQpYrWs1thluBGqlJSVJgKl0IOFvKLsQ6KKfqzAwh6FxrO5qajp4viNIgtWoFw
 O5Bp0jgFTbH2OrMWIRfUdUCH1Djbuo65svhNj7FNsQVYzHDI+Nd1I/LOBoPc3UorRMF2M4JV
 kUR7skOHWUbPTSNUr7qc14NSMY3PKjGeVGAsBVHBPvmRx/Ss2tW/TpJWpxg4pmquFQSXuaYh
 Yf4FN8Sxy320pcr+FqN8AhZkYJcNY82OTtR5VEKRC/mYcyq61qXIKqngydn79bJjGxHipxyF FmEC
Message-ID: <07c6af6c-c30d-52e4-8716-e3688c3ef265@postgresql.org>
Date: Wed, 8 Apr 2020 09:26:42 -0400
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:68.0)
 Gecko/20100101 Thunderbird/68.6.0
MIME-Version: 1.0
In-Reply-To: 
 <CAPpHfdt78x-aOhbTdY01HJ-ydM9ZSo-sRJNi0pwk7WmWd0xxAA@mail.gmail.com>
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature";
 boundary="EAMNNmBPI7bXqajv4VjCNpRth1QX3ahnw"
Precedence: bulk

This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--EAMNNmBPI7bXqajv4VjCNpRth1QX3ahnw
Content-Type: multipart/mixed; boundary="SeINzkILhir9WZjEtyJad8NjCpm0JwgIs"

--SeINzkILhir9WZjEtyJad8NjCpm0JwgIs
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable

On 4/8/20 8:59 AM, Alexander Korotkov wrote:
> On Wed, Apr 8, 2020 at 3:43 PM Andres Freund <andres@anarazel.de> wrote=
:
>> Realistically it still 2-3 hours of proof-reading.
>>
>> This makes me sad :(
>=20
> Can we ask RMT to extend feature freeze for this particular patchset?
> I think it's reasonable assuming extreme importance of this patchset.

One of the features of RMT responsibilities[1] is to be "hands off" as
much as possible, so perhaps a reverse ask: how would people feel about
this patch going into PG13, knowing that the commit would come after the
feature freeze date?

My 2=C2=A2, with RMT hat off:

As mentioned earlier[2], we know that connection scalability is a major
pain point with PostgreSQL and any effort that can help alleviate that
is a huge win, even in incremental gains. Andres et al experimentation
show that this is more than incremental gains, and will certainly make a
huge difference in people's PostgreSQL experience. It is one of those
features where you can "plug in and win" -- you get a performance
benefit just by upgrading. That is not insignificant.

However, I also want to ensure that we are fair: in the past there have
also been other patches that have been "oh-so-close" to commit before
feature freeze but have not made it in (an example escapes me at the
moment). Therefore, we really need to have consensus among ourselves
that the right decision is to allow this to go in after feature freeze.

Did this come in (very) late into the development cycle? Yes, and I
think normally that's enough to give cause for pause. But I could also
argue that Andres is fixing a "bug" with PostgreSQL (probably several
bugs ;) with PostgreSQL -- and perhaps the fixes can't be backpatched
per se, but they do improve the overall stability and usability of
PostgreSQL and it'd be a shame if we have to wait on them.

Lastly, with the ongoing world events, perhaps time that could have been
dedicated to this and other patches likely affected their completion. I
know most things in my life take way longer than they used to (e.g.
taking out the trash/recycles has gone from a 15s to 240s routine). The
same could be said about other patches as well, but this one has a far
greater impact (a double-edged sword, of course) given it's a feature
that everyone uses in PostgreSQL ;)

So with my RMT hat off, I say +1 to allowing this post feature freeze,
though within a reasonable window.

My 2=C2=A2, with RMT hat on:

I believe in[2] I outlined a way a path for including the patch even at
this stage in the game. If it is indeed committed, I think we
immediately put it on the "Recheck a mid-Beta" list. I know it's not as
trivial to change as something like "Determine if jit=3D"on" by default"
(not picking on Andres, I just remember that example from RMT 11), but
it at least provides a notable reminder that we need to ensure we test
this thoroughly, and point people to really hammer it during beta.

So with my RMT hat on, I say +0 but with a ;)

Thanks,

Jonathan

[1] https://wiki.postgresql.org/wiki/Release_Management_Team#History
[2]
https://www.postgresql.org/message-id/6be8c321-68ea-a865-d8d0-50a3af61646=
3%40postgresql.org


--SeINzkILhir9WZjEtyJad8NjCpm0JwgIs--

--EAMNNmBPI7bXqajv4VjCNpRth1QX3ahnw
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQIzBAEBCAAdFiEE+oS2la8r95ogZD/x8QSccp8cZScFAl6N0RIACgkQ8QSccp8c
ZScjOA//Z+VPTThyexg5TTCckdpn7QqTWiLMPHbnLOGRgF6vnTgOZrEN2hRZtJIQ
u7/zLqLtgZ6wtI7uVjsWvXPa0m4qTYqmmv0XER6Az2+TcFCyiPQecxDDwk/fNfwP
qfbHhpYB7NFSdE5w9JONnycLUoXPkSRQOAswJKwwbMlaNmFAgi1b0Q0V0NDg8nC1
kXzc68VB+a3/Rzb9JMYqe9Bmr2B2Ca2rP3WO+RROnTn9U/yHVo1yPziqwcWIhqVZ
6azuVN8BqKLMGVtWSo1sgqFd/VekD2tmwGNQ4Unr9PZwgo/C7LzRI6vDJD2zwm5b
QiseX7qrENZbxjhvRC2BVSOqxVYeO7QYIkDWYragX5P/z0gL33aBw19vzCSyCQGm
1dY8ucoXb5J8PWPOhk58Cuobov08DcCkOmcBJRWx2ckWrBS0r6y9trHz+UbWTFI4
09zxlKbhLdfKz4FGK6X3iuRhUpSZP6gkuJT0vT1GvPCkHEc3Gdwi7mr34DgmHxsI
EpU6drAPmqYIt/TKm0V6WDA9Z6MlIyo1VXQlIb9QnPrqpQ25+ouIe/5/fMbYQ+cN
x84coZ6AM8DYGTzYQg2B+f/O5e1pT6UkQt0xBxnx4tf2k9cGbavbH4wQK98gv8Hl
9BbSFjz9IBeeQBnrWrzbK60vqIWrB7eHd2TAk78ySuPKFfJot6Y=
=0gub
-----END PGP SIGNATURE-----

--EAMNNmBPI7bXqajv4VjCNpRth1QX3ahnw--