Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1r1UeP-00BUmw-Oz for pgsql-hackers@arkaria.postgresql.org; Fri, 10 Nov 2023 16:46:33 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1r1UeO-004Ob6-6R for pgsql-hackers@arkaria.postgresql.org; Fri, 10 Nov 2023 16:46:32 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1r1UeN-004Oay-Re for pgsql-hackers@lists.postgresql.org; Fri, 10 Nov 2023 16:46:31 +0000 Received: from mail-pl1-x642.google.com ([2607:f8b0:4864:20::642]) by magus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.94.2) (envelope-from ) id 1r1UeL-005lSJ-LS for pgsql-hackers@lists.postgresql.org; Fri, 10 Nov 2023 16:46:31 +0000 Received: by mail-pl1-x642.google.com with SMTP id d9443c01a7336-1cc13149621so1445315ad.1 for ; Fri, 10 Nov 2023 08:46:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1699634787; x=1700239587; darn=lists.postgresql.org; h=references:to:cc:in-reply-to:date:subject:mime-version:message-id :from:from:to:cc:subject:date:message-id:reply-to; bh=CjSlXmmhsk1pX8rGcJXbJm6VS+VSziOLoi41riJ2kCI=; b=LXrzcwNfkJ8Fnqj8x635VC9uVH/qs/kC3BtNvyVcNnm3Tzi48y02u0neWH3ixWv2qz k6xVeznhnjRcALHaKiStakyOaNcJtUlrUdG/j0sZFKTX/PK5YE9wlyVVyonRpyo0lY09 tJ4GzETb5WXZTM6/ANRfH0Bie1ybFdFKYOsBn+ZqQoQS2rhT9x/ZxW/w0d5772hfzxnA 33HE/hOpUL2A3ZJE6u7CSo5OJBqOg6MpFFlb0Z8poRg89oQda0bsXJEuurxhPGmeg84V 8Kg0xFqlUySbELK0u04yUKR6hU1KIyyspOrOLLZkz7+ux75LxgcQ/4yzZTVf38JLhOJG 1VfA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1699634787; x=1700239587; h=references:to:cc:in-reply-to:date:subject:mime-version:message-id :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=CjSlXmmhsk1pX8rGcJXbJm6VS+VSziOLoi41riJ2kCI=; b=EoMUi66aXhycU1RSTJSJaR+iSS7xTFqtVPmEcrgTaavX6fek1vjxBLfLHsaFT4vc7V ugG2bCvEm8DSJRNxr2mBVL3hPWHQf52Ug9AmKgLF6qtvaiHgkRy1V2qPQ7+cxctO+6c6 nBCFWrNxChTdaAqMLzSQpZiEBL+Q62ziiuUXLO1fQmdhZi3/IEXtyT0hsntXbhydWdID efq1RAnxg8E2Ps2eMKh/h0q/0dxG2OnZTYPU0dn8hCk5DZT/KCu/XT7wekGhP/osAgHQ N16BKMAE+f3QUROPHOrRW2B2T5EKiVvf5SCmKaKswdusecjIn6TkIx+b03FQh/VXfBvb tuOg== X-Gm-Message-State: AOJu0Ywh3lXbxA+pESDmnjgnwMmgenEY5SpUM7JeQZ9PDR3bZWsXd+s2 wLXdaeU1cq03tFiE4o1n/p/UIG8RpU5dk8tqxPI= X-Google-Smtp-Source: AGHT+IGJshs8YCcTd/5jHa4qTvCr0Nfn1Gc079spWgMBJNr36CUM1iJkUu6hcr+C5pWqEzWIyBKNzw== X-Received: by 2002:a17:902:e887:b0:1bb:83ec:832 with SMTP id w7-20020a170902e88700b001bb83ec0832mr9274105plg.2.1699634786865; Fri, 10 Nov 2023 08:46:26 -0800 (PST) Received: from smtpclient.apple ([2406:da18:d69:3200:453e:4974:5cb8:8a68]) by smtp.gmail.com with ESMTPSA id j24-20020a170902759800b001bdd7579b5dsm5580890pll.240.2023.11.10.08.46.25 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 10 Nov 2023 08:46:26 -0800 (PST) From: jacktby jacktby Message-Id: <124274C8-E6E6-42C3-980C-4B25D19C2CD1@gmail.com> Content-Type: multipart/alternative; boundary="Apple-Mail=_5FA4A149-3F16-414C-BEBC-657EA8DFC053" Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.600.7\)) Subject: Re: Buffer Cache Problem Date: Sat, 11 Nov 2023 00:46:13 +0800 In-Reply-To: <8ADBC847-2D66-4EFB-89AF-76338B8B46F4@gmail.com> Cc: boekewurm+postgres@gmail.com To: pgsql-hackers@lists.postgresql.org References: <059DE243-F4B0-41CC-84BE-9362FC5FB99A@gmail.com> <8ADBC847-2D66-4EFB-89AF-76338B8B46F4@gmail.com> X-Mailer: Apple Mail (2.3731.600.7) List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --Apple-Mail=_5FA4A149-3F16-414C-BEBC-657EA8DFC053 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Hi, I have 3 questions here: 1. I see comments in but_internals.h below: =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D * Also, in places we do one-time reads of the flags without bothering = to * lock the buffer header; this is generally for situations where we = don't * expect the flag bit being tested to be changing. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D In fact, the flag is in state filed which is an atomic_u32, so we = don=E2=80=99t need to acquire buffer header lock in any case, but for = this comment, seems it=E2=80=99s saying we need to hold a buffer header = lock when read flag in general. 2. Another question: =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D * We can't physically remove items from a disk page if another backend = has * the buffer pinned. Hence, a backend may need to wait for all other = pins * to go away. This is signaled by storing its own pgprocno into * wait_backend_pgprocno and setting flag bit BM_PIN_COUNT_WAITER. At = present, * there can be only one such waiter per buffer. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D The comments above, in fact for now, if a backend plan to remove items = from a disk page, this is a mutation operation, so this backend must = hold a exclusive lock for this buffer page, then in this case, there are = no other backends pinning this buffer, so the pin refcount must be 1 = (it=E2=80=99s by this backend), then this backend can remove the items = safely and no need to wait other backends (because there are no other = backends pinning this buffer). So my question is below: The operation =E2=80=9Cstoring its own pgprocno into * wait_backend_pgprocno and setting flag bit BM_PIN_COUNT_WAITER=E2=80=9D= is whether too expensive, we should not do like this, right? 3. Where is the array? =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D * Per-buffer I/O condition variables are currently kept outside this = struct in * a separate array. They could be moved in here and still fit within = that * limit on common systems, but for now that is not done. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= --Apple-Mail=_5FA4A149-3F16-414C-BEBC-657EA8DFC053 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8
Hi, I have 3 questions here:
1. I see comments in = but_internals.h = below:
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
&= nbsp;* Also, in places we do one-time reads of the flags without = bothering to
 * lock the buffer header; this is generally = for situations where we don't
 * expect the flag bit = being tested to be changing.
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
In fact, the flag is in state filed which is an = atomic_u32, so we don=E2=80=99t need to acquire buffer header lock in = any case, but for this comment, seems it=E2=80=99s saying we need to = hold a buffer header lock when read flag in = general.

2. = Another question:
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
 * We can't physically remove items from a disk page if = another backend has
 * the buffer pinned.  Hence, a = backend may need to wait for all other pins
 * to go = away.  This is signaled by storing its own pgprocno = into
 * wait_backend_pgprocno and setting flag bit = BM_PIN_COUNT_WAITER.  At present,
 * there can be = only one such waiter per = buffer.
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
The comments above,  in fact for now, if a backend plan to = remove items from a disk page, this is a mutation operation, so this = backend must hold a exclusive lock for this buffer page, then in this = case, there are no other backends pinning this buffer, so the pin = refcount must be 1 (it=E2=80=99s by this backend), then this backend can = remove the items safely and no need to wait other backends (because = there are no other backends pinning this buffer). So my question is = below:
 The operation =E2=80=9Cstoring its own pgprocno = into
 * wait_backend_pgprocno and setting flag bit = BM_PIN_COUNT_WAITER=E2=80=9D is whether too expensive, we should not do = like this, right?

3. Where is the array?
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
 * = Per-buffer I/O condition variables are currently kept outside this = struct in
 * a separate array.  They could be moved = in here and still fit within that
 * limit on common = systems, but for now that is not = done.
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
= --Apple-Mail=_5FA4A149-3F16-414C-BEBC-657EA8DFC053--