Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1qGebU-0003e9-8o for pgsql-hackers@arkaria.postgresql.org; Tue, 04 Jul 2023 11:53:56 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1qGebT-0006gJ-4s for pgsql-hackers@arkaria.postgresql.org; Tue, 04 Jul 2023 11:53:55 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1qGebS-0006fr-Qz for pgsql-hackers@lists.postgresql.org; Tue, 04 Jul 2023 11:53:54 +0000 Received: from mail-vs1-xe32.google.com ([2607:f8b0:4864:20::e32]) by magus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.94.2) (envelope-from ) id 1qGebQ-0028oy-BA for pgsql-hackers@lists.postgresql.org; Tue, 04 Jul 2023 11:53:54 +0000 Received: by mail-vs1-xe32.google.com with SMTP id ada2fe7eead31-440ac4b44a8so1832587137.3 for ; Tue, 04 Jul 2023 04:53:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1688471631; x=1691063631; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=jcWZ49SCf/VLl42AuAvVVPjlPW6G9nwAOEfYlhlu4JU=; b=Hw428A8Sl8sYshvjoAlmzN2GUp83WvKiIT6mdpCuQ5SvAnOV3BmpobIeikFXTVEtuv p0Br/xk2mVZddRcrurs/fTWsKexEg74P9d1x8GD1ybS4+RYqdr6blFroFCH+l967u/KL qsdcbmdUoxN1IPhoYVbWfz8t/iYyub2Xe3c4DwndmkyCZ7gK3ngyh9QrWsMV7u7lH/iQ gFN7ddmscyVccg4YMHQxShgKbyItKE3GSvetE4AY8b5upENuwp1nikDyAYTWgRBB7mtc KBOpxjJcpgPdpa7WA8TrWrlfqkajMPL/7HnY+0NLmoA4CPK4YSaPeDzKL5szFemckPVK 7WEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688471631; x=1691063631; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=jcWZ49SCf/VLl42AuAvVVPjlPW6G9nwAOEfYlhlu4JU=; b=KUhq/GitwNqClQflBj5VotW4Lxv8hc0j6vukf2AROYfjB65f7x8k0g07u/RbdHHLXv +xLbPBNd1B8UuvHw2Cd+DsS0uYMgDbwQBMFOO8q8J/OJOvelMU2VZxU/6tAUXtBqVcKM 8Voc9tLtKsX45F4j2CvRwadjNyKfos5p07dQQnni9c+ZMatpbk3AqW8z5PYcHLU220BO 7D/O6Y+8h/CCw9Phnvo+DSZ+LPppS/xAiEUw7XKBSiWsWmkHYGevl8Q+VGbHCqAy3WvO miBNZ5g64Grzcvh4ikh5NCYsLNPVJnAp9XC2MQmr37PqQI9SlTTIMj+FireRdGJauhIr hskQ== X-Gm-Message-State: ABy/qLZf31KCFrybqinyAbPrhOgGNDHq8HTEGLg8tNUA7Jd1vOvpgH8o irJe1vEdoDe1GAxDjUsu3LDRjBpZwoTN1Ssu8YU= X-Google-Smtp-Source: APBJJlHXwN6Zuvisnk9Dy0CsWEoQqERvZuEXEFjdAp3t3q2RjH/TvH7Dw1ffVdxYHeWpFXJ0Q1eHUHWgOyHSPgtlh5M= X-Received: by 2002:a67:f553:0:b0:443:6180:778f with SMTP id z19-20020a67f553000000b004436180778fmr6161614vsn.33.1688471630856; Tue, 04 Jul 2023 04:53:50 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: jian he Date: Tue, 4 Jul 2023 19:53:39 +0800 Message-ID: Subject: Re: Extension Enhancement: Buffer Invalidation in pg_buffercache To: Japin Li Cc: Palak Chaturvedi , Thomas Munro , pgsql-hackers@postgresql.org, pgsql-hackers@lists.postgresql.org Content-Type: multipart/alternative; boundary="000000000000c5466505ffa7ef8c" List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --000000000000c5466505ffa7ef8c Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Tue, Jul 4, 2023 at 5:45=E2=80=AFPM Japin Li wrote= : > > On Tue, 04 Jul 2023 at 17:00, jian he wrote= : > > the following will also crash. no idea why. > > begin; > > select count(*) from onek; > > select relpages from pg_class where relname =3D 'onek'; --queryA > > > > SELECT count(*) FROM pg_buffercache WHERE relfilenode =3D > > pg_relation_filenode('onek'::regclass); --queryB > > > > insert into onek values(default); > > > > select count(pg_buffercache_invalidate(bufferid)) from > > pg_buffercache where relfilenode =3D > > pg_relation_filenode('onek'::regclass); > > > > --------------------------------- > > queryA returns 35, queryB returns 37. > > ---------------------------------- > > crash info: > > test_dev=3D*# insert into onek values(default); > > INSERT 0 1 > > test_dev=3D*# select count(pg_buffercache_invalidate(bufferid)) fro= m > > pg_buffercache where relfilenode =3D > > pg_relation_filenode('onek'::regclass); > > TRAP: failed Assert("resarr->nitems < resarr->maxitems"), File: > > > "../../Desktop/pg_sources/main/postgres/src/backend/utils/resowner/resown= er.c", > > Line: 275, PID: 1533312 > > According to the comments of ResourceArrayAdd(), the caller must have > previously > done ResourceArrayEnlarge(). I tried to call ResourceOwnerEnlargeBuffers(= ) > before > PinBuffer_Locked(), so it can avoid this crash. > > if ((buf_state & BM_DIRTY) =3D=3D BM_DIRTY) > { > + /* make sure we can handle the pin */ > + ResourceOwnerEnlargeBuffers(CurrentResourceOwner)= ; > + > /* > * Try once to flush the dirty buffer. > */ > PinBuffer_Locked(bufHdr); > > -- > Regrads, > Japin Li. > thanks. tested flush pg_catalog, public schema, now, both works as pitched. --000000000000c5466505ffa7ef8c Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


On Tue, Jul 4, 2023 at 5:45=E2=80=AF= PM Japin Li <japinli@hotmail.com<= /a>> wrote:
<= br> On Tue, 04 Jul 2023 at 17:00, jian he <jian.universality@gmail.com> wrote:<= br> > the following will also crash. no idea why.
> begin;
>=C2=A0 =C2=A0 =C2=A0select count(*) from onek;
>=C2=A0 =C2=A0 =C2=A0select relpages from pg_class where relname =3D = 9;onek'; --queryA
>
>=C2=A0 =C2=A0 =C2=A0SELECT count(*) FROM pg_buffercache WHERE relfileno= de =3D
> pg_relation_filenode('onek'::regclass); --queryB
>
>=C2=A0 =C2=A0 =C2=A0insert into onek values(default);
>
>=C2=A0 =C2=A0 =C2=A0select count(pg_buffercache_invalidate(bufferid)) f= rom
>=C2=A0 =C2=A0 =C2=A0pg_buffercache where relfilenode =3D
> pg_relation_filenode('onek'::regclass);
>
> ---------------------------------
> queryA returns 35, queryB returns 37.
> ----------------------------------
> crash info:
> test_dev=3D*#=C2=A0 =C2=A0 =C2=A0insert into onek values(default);
> INSERT 0 1
> test_dev=3D*#=C2=A0 =C2=A0 =C2=A0select count(pg_buffercache_invalidat= e(bufferid)) from
>=C2=A0 =C2=A0 =C2=A0pg_buffercache where relfilenode =3D
> pg_relation_filenode('onek'::regclass);
> TRAP: failed Assert("resarr->nitems < resarr->maxitems&q= uot;), File:
> "../../Desktop/pg_sources/main/postgres/src/backend/utils/resowne= r/resowner.c",
> Line: 275, PID: 1533312

According to the comments of ResourceArrayAdd(), the caller must have previ= ously
done ResourceArrayEnlarge(). I tried to call ResourceOwnerEnlargeBuffers() = before
PinBuffer_Locked(), so it can avoid this crash.

=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 if ((buf_state &= ; BM_DIRTY) =3D=3D BM_DIRTY)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 {
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0/* make sure we can handle the pin */
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0ResourceOwnerEnlargeBuffers(CurrentResourceOwner);
+
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 /*
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0* Try once to flush the dirty buffer.
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0*/
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 PinBuffer_Locked(bufHdr);

--
Regrads,
Japin Li.


thanks. tested flush pg_ca= talog, public schema, now, both works as pitched.


--000000000000c5466505ffa7ef8c--