From: "Amonson, Paul D" <paul.d.amonson@intel.com>
To: Andres Freund <andres@anarazel.de>
CC: "pgsql-hackers@lists.postgresql.org" <pgsql-hackers@lists.postgresql.org>,
	Nathan Bossart <nathandbossart@gmail.com>, "Shankaran, Akash"
	<akash.shankaran@intel.com>
Subject: RE: Proposal for Updating CRC32C with AVX-512 Algorithm.
Thread-Topic: Proposal for Updating CRC32C with AVX-512 Algorithm.
Thread-Index: Adqb27jiuXmC5sH1QzmzETmUIqng2AhKQCWAAQB9aiA=
Date: Mon, 17 Jun 2024 22:42:54 +0000
Message-ID: <BL1PR11MB5304F180D87B1222C7601C96DCCD2@BL1PR11MB5304.namprd11.prod.outlook.com>
References: <BL1PR11MB530401FA7E9B1CA432CF9DC3DC192@BL1PR11MB5304.namprd11.prod.outlook.com>
 <20240612201135.kk77tiqcux77lgev@awork3.anarazel.de>
In-Reply-To: <20240612201135.kk77tiqcux77lgev@awork3.anarazel.de>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?BumR+RFveQVPSTGgAqSpE5iV4sUg7eqjWpi6750e9dwmXmejuWrEpNliJp7T?=
 =?us-ascii?Q?6lZVVBggByhq9WnWaKi05d4BapMkjPVsz5DQ1qvdfmEV3qlEqaZvhsv4cdOF?=
 =?us-ascii?Q?YYLr8LGn0WgBD4l2yCesQnTTal8Rilv+v6yyBNMk0gs6ikfUJCZaTCqOYZv5?=
 =?us-ascii?Q?E/DbEHyQRzM3mh2YSkBGtZZ4o50RsUzSUreVEtPYN8r+KbJZBQIuUu6d2Agp?=
 =?us-ascii?Q?jlw7/X6axW9zxnzNatx1lBC+TOd53/R9LwlBFosKSUPt/3ERll+tKVDGx6Wb?=
 =?us-ascii?Q?CVdjH3Y1uexZC2wuJCo1EP+nRBFjzi2oGwu1dKPi/VPHGZCqjaRzaW+BaHy6?=
 =?us-ascii?Q?wGReFBpsHVhX024QeTVWTFoWjUqL6bb3IVmUna3m34gqsiZ/xFwCdfc2FWK6?=
 =?us-ascii?Q?sDjLRtJZ49heTnmr+RQ+5FR76gg5rVUx5C0ttOYZJlHDIFjH9c55yqaB89dM?=
 =?us-ascii?Q?Nr2WUoqKIzTq184ckDOQiYwdTSvJSPFi2ugJUcRqS+1Ghuk+tnA31v1Zxsrf?=
 =?us-ascii?Q?bL5Vy5MkbLktdH8ek5wY8w68Ce3JmehysiqZFgPmTfPht5WjrirjDmHjkqyl?=
 =?us-ascii?Q?aoBvuIfwW3atRagPkNiQEGefGYEzigl9iASGQDIimzeNeEPXTZRGRdYmrkM3?=
 =?us-ascii?Q?ZMUZiXFy7puFKRShU5mhpeGiIOj8q/unjYtQgN0kyhhgth/wgIw0XKawRWWM?=
 =?us-ascii?Q?99pfIzobSZQWciIHeUR5+q5BjjKuUTxCIFLSzf3QJ8qWvoCQO7HjYvMYOJcZ?=
 =?us-ascii?Q?WkdaL+wXtxBaO9OahJcsA8X6oj1CIqbCWnvqNUVcfAaQncY+Rjl8wo53gvxf?=
 =?us-ascii?Q?qw/GzzfdzfQSE/RzTXEERTFZ0iVO3SKNAFGCXpcaz6rhIVjU4EFGr4sWGXKy?=
 =?us-ascii?Q?5hEMTWg9D9RSVgcltvJnFIZh59z2aNYiCmt4T9h3SwlnzcahTmGR0Eq8qZPJ?=
 =?us-ascii?Q?tqwXvdiFQGeQ2YzU/sSD2+Roxv0XvcBQoiSMDp6OxHNHQz7Os6YSeG/MGhCO?=
 =?us-ascii?Q?uT7frsMrnZkYFvxEDMMEE2k2eCg4So+mn+lBSOUV91P4zbRSOmykijBQm9Qo?=
 =?us-ascii?Q?L7qi0fdO6ut2r3KQmooPPHigtpNeHEZ6HIBoDATgmkhIZELV4P49AR+EkJ+A?=
 =?us-ascii?Q?IsjmDuvqGt31R5uCCEilCzJyVIPPPnrqBDPEJe62XdFYB++g980nH0Txc6Vd?=
 =?us-ascii?Q?mW2H6THKtovumiE/uWhCBk0y+7Np1vjTdhjT5gliwwCqDdVaew1q+SkKfhda?=
 =?us-ascii?Q?+Ozasj9XVMrLahZbzjiw7x7GOrqwG+Rt0xcbso/4wW91mAnVZ6aZypIncEbk?=
 =?us-ascii?Q?sUlNVOi17k1Su8y+Vc0V1BOIebkReiZD1pjNlYkCk+SOC/83P4uljWMkvkvo?=
 =?us-ascii?Q?Q5v7Lft4xG5D6ox6UjAUeyJFZ8ZnY59gQMu289pA0XzqLGJ0RZBbaYbSD7g8?=
 =?us-ascii?Q?TW/AY0f+faEMl8xDUTPbK++PuVx/taRjfhp5Ay998Ow3dOwE73HvIfYRPY9D?=
 =?us-ascii?Q?RZyPqbmf9aRJLzQ3k+hef+B6HxKv9YpNIwsMeuX/6OyCiNLnKpzOZRVilYTr?=
 =?us-ascii?Q?qGNJYTGPGrHfQX8iKwPhYLuaFqAWnU9ts9337XN0?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BL1PR11MB5304.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 27089cbf-dc2e-4c0a-0ed9-08dc8f1ed3b0
X-MS-Exchange-CrossTenant-originalarrivaltime: 17 Jun 2024 22:42:54.2756
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: Dw2EjpzOUUQs+liNRTwrZU9Jcg5MKC4kq/zAxWpIrQtBUZNB+2jH4tgZxodB2NWcCrg6dd0TFMOLdrYAbJyRNExI8scv9IyKVj5ytX7/Ze4=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR11MB6037
X-OriginatorOrg: intel.com
List-Id: <pgsql-hackers.lists.postgresql.org>
List-Help: <https://lists.postgresql.org/manage/>
List-Subscribe: <https://lists.postgresql.org/manage/>
List-Post: <mailto:pgsql-hackers@lists.postgresql.org>
List-Owner: <mailto:pgsql-hackers-owner@lists.postgresql.org>
List-Archive: <https://www.postgresql.org/list/pgsql-hackers>
Archived-At: <https://www.postgresql.org/message-id/BL1PR11MB5304F180D87B1222C7601C96DCCD2%40BL1PR11MB5304.namprd11.prod.outlook.com>
Precedence: bulk

> This is extremely workload dependent, it's not hard to find workloads wit=
h
> lots of very small record and very few big ones...  What you observed mig=
ht
> have "just" been the warmup behaviour where more full page writes have to
> be written.

Can you tell me how to avoid capturing this "warm-up" so that the numbers a=
re more accurate?
=20
> There a very frequent call computing COMP_CRC32C over just 20 bytes, whil=
e
> holding a crucial lock.  If we were to do introduce something like this
> AVX-512 algorithm, it'd probably be worth to dispatch differently in case=
 of
> compile-time known small lengths.

So are you suggesting that we be able to directly call into the 64/32 bit b=
ased algorithm directly from these known small byte cases in the code? I th=
ink that we can do that with a separate API being exposed.

> How does the latency of the AVX-512 algorithm compare to just using the
> CRC32C instruction?

I think I need more information on this one as I am not sure I understand t=
he use case? The same function pointer indirect methods are used with or wi=
thout the AVX-512 algorithm?

Paul