Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1t4BAR-00GoXu-U6 for pgsql-hackers@arkaria.postgresql.org; Fri, 25 Oct 2024 03:39:16 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1t4BAP-00BPmo-Db for pgsql-hackers@arkaria.postgresql.org; Fri, 25 Oct 2024 03:39:13 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1t4BAP-00BPmg-2H for pgsql-hackers@lists.postgresql.org; Fri, 25 Oct 2024 03:39:13 +0000 Received: from mail-yb1-xb2a.google.com ([2607:f8b0:4864:20::b2a]) by magus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.94.2) (envelope-from ) id 1t4BAM-002qQf-6C for pgsql-hackers@lists.postgresql.org; Fri, 25 Oct 2024 03:39:12 +0000 Received: by mail-yb1-xb2a.google.com with SMTP id 3f1490d57ef6-e2e32bc454fso1742227276.2 for ; Thu, 24 Oct 2024 20:39:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1729827548; x=1730432348; darn=lists.postgresql.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=cAZy5GzSLwV3J3xPNlKOtXXdYXaOgbbLjLVU2in3+Xk=; b=iXxxBfe5mGezWdrZy9dYtnJv8ROY/5qoJF+ElerONXrjZAJfcW7hbKuQQgC5tZdJWf 6KQ/BG8+pHFNZdD4q1sSn4eYyC83xgxBkNWj3kAo3XO/YOel1CqWICrdvFbDSupbPrZV L+G9zJK7n7ewR5W2BRc0PowVBem/gQ7MZslqn0Ip7XK4iBYuOrWx5JSh392yLlh0bM1j 6oZy+ccbLxyeG6L6aZizQrQFICOKvRzxqkMneBL+GXs8bz5SosFAKAi3awuA2eC4xLTj /MOzgrtKRoib7g+ibN2Khu1cNQqxVrFVHvJ6zOWgU1nPoWtTmHE/84gTmGj/XFhjjnFc ekPw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1729827548; x=1730432348; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=cAZy5GzSLwV3J3xPNlKOtXXdYXaOgbbLjLVU2in3+Xk=; b=ML9v4ArEeRj+rOiR6h0xfJ07xQmJrxcJdlt0MHQn+QHJRg4FofqY0v9XAdfz+q504r xvZIvU4rMtSDENoIx25OIzCGv4JlAbyGqWtWr2n9FOeah6NcS07zuh0xF56lG2adFoKe Yob38iIDBEGQNZFy8iYqVqoT4o8TW4LOv+O76eAqxFx0O36miDR2HigqmUmABDPT+3ph Dr+9Y8bAgb4mV6iFWPC2azYxuyYOwaIwFNvYkczEA+fCd9WQnC8RYYLynedPzkCoX6hy qt+GVzAOn8ovqdo+I4cWnR+FWrEX9S523Ir8ywmM6NJbxzpAE21uWMQdyL/n6ezydXpK ncqQ== X-Forwarded-Encrypted: i=1; AJvYcCXwJVbIOEVBJq6ivQTD7f3x7NaKlTcg7CMd5NrFfBKfrWMDffLIMQWC3Ky9S6DYdNbyeKDR+rWgX96QWAY4@lists.postgresql.org X-Gm-Message-State: AOJu0Yy4GmIDpGSDy/6/pXvpAup7Zi/1L9hRU8c+P3GnrWz9Yy880fOY jqL3B6olUoyWPdFihO3iS8AG4MMmNreYuvkLOGAn0WfkhkNnjZB2BsxEV3F7xgcOrXvD6qFpUel yvCj0+TSY7VQLNw8kPgOOPFAEZ2c= X-Google-Smtp-Source: AGHT+IELrX6v6KYgLNnAQKjndnm7EzSjvSqIdwNvw2CKDmb1Aud4CaWOx7C4yu5kjbg/LMsMIsn7nYHuu1191Wj7kcg= X-Received: by 2002:a05:690c:2907:b0:6be:3601:7189 with SMTP id 00721157ae682-6e858163fefmr36676337b3.10.1729827548309; Thu, 24 Oct 2024 20:39:08 -0700 (PDT) MIME-Version: 1.0 References: <5ecc35f5-1111-47fc-8a02-36d89490a50d@iki.fi> In-Reply-To: From: wenhui qiu Date: Fri, 25 Oct 2024 11:38:57 +0800 Message-ID: Subject: Re: POC: make mxidoff 64 bits To: Maxim Orlov Cc: Heikki Linnakangas , Alexander Korotkov , Postgres hackers Content-Type: multipart/alternative; boundary="0000000000008a326e062544dccc" List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --0000000000008a326e062544dccc Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable HI Maxim Orlov > After a bit of thought, I've realized that to be conservative here is the way to go. >We can reuse a maximum of existing logic. I mean, we can remove offset wraparound "error logic" and reuse "warning logic". But set the threshold for "warning >logic" to a much higher value. For now, I choose 2^32-1. In other world, legit logic, in my view, here would be to trigger autovacuum if the number of offsets (i.e. >difference nextOffset - oldestOffset) exceeds 2^32-1. PFA patch set. good point ,Couldn't agree with you more. xid64 is the solution to the wraparound problem,The previous error log is no longer meaningful ,But we might want to refine the output waring log a little(For example, checking the underlying reasons why age has been increasing),Though we don't have to worry about xid wraparound +/* + * Multixact members warning threshold. + * + * If difference bettween nextOffset and oldestOffset exceed this value, w= e + * trigger autovacuum in order to release the disk space if possible. + */ +#define MULTIXACT_MEMBER_AUTOVAC_THRESHOLD UINT64CONST(0xFFFFFFFF) Can we refine this annotation a bit? for example +/* + * Multixact members warning threshold. + * + * If difference bettween nextOffset and oldestOffset exceed this value, w= e + * trigger autovacuum in order to release the disk space ,reduce table bloat if possible. + */ +#define MULTIXACT_MEMBER_AUTOVAC_THRESHOLD UINT64CONST(0xFFFFFFFF) Thanks Maxim Orlov =E4=BA=8E2024=E5=B9=B410=E6=9C=8823=E6=97= =A5=E5=91=A8=E4=B8=89 23:55=E5=86=99=E9=81=93=EF=BC=9A > After a bit of thought, I've realized that to be conservative here is the > way to go. > > We can reuse a maximum of existing logic. I mean, we can remove offset > wraparound "error logic" and reuse "warning logic". But set the threshold > for "warning logic" to a much higher value. For now, I choose 2^32-1. In > other world, legit logic, in my view, here would be to trigger autovacuum > if the number of offsets (i.e. difference nextOffset - oldestOffset) > exceeds 2^32-1. PFA patch set. > > -- > Best regards, > Maxim Orlov. > --0000000000008a326e062544dccc Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
HI Maxim Orlov
> After a bit of thought, I've r= ealized that to be conservative here is the way to go.
>We can reus= e a maximum of existing logic. I mean, we can remove offset wraparound &quo= t;error logic" and reuse "warning logic". But set the thresh= old for "warning >logic" to a much higher value. For now, I ch= oose 2^32-1. In other world, legit logic, in my view, here would be to trig= ger autovacuum if the number of offsets (i.e. >difference nextOffset - o= ldestOffset) exceeds 2^32-1. PFA patch set.
good point ,Couldn't ag= ree with you more.=C2=A0xid64 is the solution to the wraparound=C2=A0proble= m,The previous error log is no longer meaningful ,But we might want to refi= ne the output waring log a little(For example, checking the underlying reas= ons why age has been increasing),Though we don't have to worry about xi= d wraparound

+/*
+ * Multixact members warn= ing threshold.
+ *
+ * If difference bettween nextOffset and oldestOf= fset exceed this value, we
+ * trigger autovacuum in order to release th= e disk space if possible.
+ */
+#define MULTIXACT_MEMBER_AUTOVAC_THRE= SHOLD UINT64CONST(0xFFFFFFFF)
Can we refine this annotation = a bit? for example
+/*
+ * Multixact members warning thres= hold.
+ *
+ * If difference bettween nextOffset and oldestOffset exce= ed this value, we
+ * trigger autovacuum in order to release the disk sp= ace ,reduce table bloat=C2=A0if possible.
+ */
+#define MULTIXACT_MEM= BER_AUTOVAC_THRESHOLD UINT64CONST(0xFFFFFFFF)

= Thanks

Maxim Orlov <orlovmg@gm= ail.com> =E4=BA=8E2024=E5=B9=B410=E6=9C=8823=E6=97=A5=E5=91=A8=E4=B8= =89 23:55=E5=86=99=E9=81=93=EF=BC=9A
After a bit of thought, I've realized that to be conservative= here is the way to go.

We can reuse a maximum of existing logic. I = mean, we can remove offset wraparound "error logic" and reuse &qu= ot;warning logic". But set the threshold for "warning logic"= to a much higher value. For now, I choose 2^32-1. In other world, legit lo= gic, in my view, here would be to trigger autovacuum if the number of offse= ts (i.e. difference nextOffset - oldestOffset) exceeds 2^32-1. PFA patch se= t.

--
Best regards,
<= /div>
Maxim Orlov.
--0000000000008a326e062544dccc--