pg.ddx.io pgsql-committers@postgresql.org mailing list archive
help / color / mirror / Atom feedpgsql: Increase width of RelFileNumbers from 32 bits to 56 bits.
4+ messages / 3 participants
[nested] [flat]
Commits from this thread:
2022-09-27 4667d97ca6 Fix typos in commit 05d4cbf9b6ba708858984b01ca0fc56d59d4ec7c.
* pgsql: Increase width of RelFileNumbers from 32 bits to 56 bits.
@ 2022-09-27 17:32 Robert Haas <rhaas@postgresql.org>
0 siblings, 1 reply; 4+ messages in thread
From: Robert Haas @ 2022-09-27 17:32 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
Increase width of RelFileNumbers from 32 bits to 56 bits.
RelFileNumbers are now assigned using a separate counter, instead of
being assigned from the OID counter. This counter never wraps around:
if all 2^56 possible RelFileNumbers are used, an internal error
occurs. As the cluster is limited to 2^64 total bytes of WAL, this
limitation should not cause a problem in practice.
If the counter were 64 bits wide rather than 56 bits wide, we would
need to increase the width of the BufferTag, which might adversely
impact buffer lookup performance. Also, this lets us use bigint for
pg_class.relfilenode and other places where these values are exposed
at the SQL level without worrying about overflow.
This should remove the need to keep "tombstone" files around until
the next checkpoint when relations are removed. We do that to keep
RelFileNumbers from being recycled, but now that won't happen
anyway. However, this patch doesn't actually change anything in
this area; it just makes it possible for a future patch to do so.
Dilip Kumar, based on an idea from Andres Freund, who also reviewed
some earlier versions of the patch. Further review and some
wordsmithing by me. Also reviewed at various points by Ashutosh
Sharma, Vignesh C, Amul Sul, Álvaro Herrera, and Tom Lane.
Discussion: http://postgr.es/m/CA+Tgmobp7+7kmi4gkq7Y+4AM9fTvL+O1oQ4-5gFTT+6Ng-dQ=g@mail.gmail.com
Branch
------
master
Details
-------
https://git.postgresql.org/pg/commitdiff/05d4cbf9b6ba708858984b01ca0fc56d59d4ec7c
Modified Files
--------------
contrib/pg_buffercache/Makefile | 4 +-
.../pg_buffercache/pg_buffercache--1.3--1.4.sql | 30 +++
contrib/pg_buffercache/pg_buffercache.control | 2 +-
contrib/pg_buffercache/pg_buffercache_pages.c | 39 +++-
contrib/pg_prewarm/autoprewarm.c | 4 +-
contrib/pg_walinspect/expected/pg_walinspect.out | 4 +-
contrib/pg_walinspect/sql/pg_walinspect.sql | 4 +-
doc/src/sgml/catalogs.sgml | 2 +-
doc/src/sgml/func.sgml | 5 +
doc/src/sgml/pgbuffercache.sgml | 2 +-
doc/src/sgml/storage.sgml | 11 +-
src/backend/access/gin/ginxlog.c | 2 +-
src/backend/access/rmgrdesc/gistdesc.c | 2 +-
src/backend/access/rmgrdesc/heapdesc.c | 2 +-
src/backend/access/rmgrdesc/nbtdesc.c | 2 +-
src/backend/access/rmgrdesc/seqdesc.c | 2 +-
src/backend/access/rmgrdesc/xlogdesc.c | 21 ++-
src/backend/access/transam/README | 5 +-
src/backend/access/transam/varsup.c | 209 ++++++++++++++++++++-
src/backend/access/transam/xlog.c | 60 ++++++
src/backend/access/transam/xlogprefetcher.c | 14 +-
src/backend/access/transam/xlogrecovery.c | 6 +-
src/backend/access/transam/xlogutils.c | 6 +-
src/backend/backup/basebackup.c | 2 +-
src/backend/catalog/catalog.c | 95 ----------
src/backend/catalog/heap.c | 27 +--
src/backend/catalog/index.c | 11 +-
src/backend/catalog/storage.c | 8 +
src/backend/commands/tablecmds.c | 12 +-
src/backend/commands/tablespace.c | 2 +-
src/backend/nodes/gen_node_support.pl | 4 +-
src/backend/replication/logical/decode.c | 1 +
src/backend/replication/logical/reorderbuffer.c | 2 +-
src/backend/storage/file/reinit.c | 28 +--
src/backend/storage/freespace/fsmpage.c | 2 +-
src/backend/storage/lmgr/lwlocknames.txt | 1 +
src/backend/storage/smgr/md.c | 7 +
src/backend/storage/smgr/smgr.c | 2 +-
src/backend/utils/adt/dbsize.c | 7 +-
src/backend/utils/adt/pg_upgrade_support.c | 13 +-
src/backend/utils/cache/relcache.c | 2 +-
src/backend/utils/cache/relfilenumbermap.c | 4 +-
src/backend/utils/misc/pg_controldata.c | 9 +-
src/bin/pg_checksums/pg_checksums.c | 4 +-
src/bin/pg_controldata/pg_controldata.c | 2 +
src/bin/pg_dump/pg_dump.c | 26 +--
src/bin/pg_rewind/filemap.c | 6 +-
src/bin/pg_upgrade/info.c | 3 +-
src/bin/pg_upgrade/pg_upgrade.c | 6 +-
src/bin/pg_upgrade/relfilenumber.c | 4 +-
src/bin/pg_waldump/pg_waldump.c | 2 +-
src/bin/scripts/t/090_reindexdb.pl | 2 +-
src/common/relpath.c | 20 +-
src/fe_utils/option_utils.c | 40 ++++
src/include/access/transam.h | 40 ++++
src/include/access/xlog.h | 1 +
src/include/catalog/catalog.h | 3 -
src/include/catalog/catversion.h | 2 +-
src/include/catalog/pg_class.h | 16 +-
src/include/catalog/pg_control.h | 2 +
src/include/catalog/pg_proc.dat | 10 +-
src/include/common/relpath.h | 7 +-
src/include/fe_utils/option_utils.h | 2 +
src/include/storage/buf_internals.h | 55 +++++-
src/include/storage/relfilelocator.h | 12 +-
src/test/regress/expected/alter_table.out | 24 ++-
src/test/regress/expected/fast_default.out | 4 +-
src/test/regress/expected/oidjoins.out | 2 +-
src/test/regress/sql/alter_table.sql | 8 +-
src/test/regress/sql/fast_default.sql | 4 +-
70 files changed, 694 insertions(+), 290 deletions(-)
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: pgsql: Increase width of RelFileNumbers from 32 bits to 56 bits.
@ 2022-09-27 18:51 Justin Pryzby <pryzby@telsasoft.com>
parent: Robert Haas <rhaas@postgresql.org>
0 siblings, 1 reply; 4+ messages in thread
From: Justin Pryzby @ 2022-09-27 18:51 UTC (permalink / raw)
To: Robert Haas <rhaas@postgresql.org>; +Cc: pgsql-committers@lists.postgresql.org
This seems to be breaking cfbot:
https://cirrus-ci.com/github/postgresql-cfbot/postgresql
For example:
https://cirrus-ci.com/task/6720256776339456
Some other minor issues:
thais is only used during
=> this
+ elog(ERROR, "unexpected relnumber " UINT64_FORMAT "that is bigger than nextRelFileNumber " UINT64_FORMAT,
=> there should be a space before "that".
+ "tli %u; prev tli %u; fpw %s; xid %u:%u; relfilenumber " UINT64_FORMAT ";oid %u; "
=> and a space before "oid"
+ * Parse relfilenumber value for an option. If the parsing is successful,
+ * returns; if parsing fails, returns false.
returns *true;
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: pgsql: Increase width of RelFileNumbers from 32 bits to 56 bits.
@ 2022-09-27 19:12 Robert Haas <robertmhaas@gmail.com>
parent: Justin Pryzby <pryzby@telsasoft.com>
0 siblings, 1 reply; 4+ messages in thread
From: Robert Haas @ 2022-09-27 19:12 UTC (permalink / raw)
To: Justin Pryzby <pryzby@telsasoft.com>; +Cc: pgsql-hackers@postgresql.org <pgsql-hackers@postgresql.org>
On Tue, Sep 27, 2022 at 2:51 PM Justin Pryzby <pryzby@telsasoft.com> wrote:
> This seems to be breaking cfbot:
> https://cirrus-ci.com/github/postgresql-cfbot/postgresql
>
> For example:
> https://cirrus-ci.com/task/6720256776339456
OK, so it looks like the pg_buffercache test is failing there. But it
doesn't fail for me, and I don't see a regression.diffs file in the
output that would enable me to see what is failing. If it's there, can
you tell me how to find it?
> Some other minor issues:
Will push fixes.
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: pgsql: Increase width of RelFileNumbers from 32 bits to 56 bits.
@ 2022-09-27 19:17 Justin Pryzby <pryzby@telsasoft.com>
parent: Robert Haas <robertmhaas@gmail.com>
0 siblings, 0 replies; 4+ messages in thread
From: Justin Pryzby @ 2022-09-27 19:17 UTC (permalink / raw)
To: Robert Haas <robertmhaas@gmail.com>; +Cc: pgsql-hackers@postgresql.org
On Tue, Sep 27, 2022 at 03:12:56PM -0400, Robert Haas wrote:
> On Tue, Sep 27, 2022 at 2:51 PM Justin Pryzby <pryzby@telsasoft.com> wrote:
> > This seems to be breaking cfbot:
> > https://cirrus-ci.com/github/postgresql-cfbot/postgresql
> >
> > For example:
> > https://cirrus-ci.com/task/6720256776339456
>
> OK, so it looks like the pg_buffercache test is failing there. But it
> doesn't fail for me, and I don't see a regression.diffs file in the
> output that would enable me to see what is failing. If it's there, can
> you tell me how to find it?
It's here in the artifacts.
https://api.cirrus-ci.com/v1/artifact/task/5647133427630080/testrun/build/testrun/pg_buffercache/reg...
Actually, this worked under autoconf but failed under meson.
I think you just need to make the corresponding change in
contrib/pg_buffercache/meson.build that's in ./Makefile.
^ permalink raw reply [nested|flat] 4+ messages in thread
end of thread, other threads:[~2022-09-27 19:17 UTC | newest]
Thread overview: 4+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2022-09-27 17:32 pgsql: Increase width of RelFileNumbers from 32 bits to 56 bits. Robert Haas <rhaas@postgresql.org>
2022-09-27 18:51 ` Justin Pryzby <pryzby@telsasoft.com>
2022-09-27 19:12 ` Robert Haas <robertmhaas@gmail.com>
2022-09-27 19:17 ` Justin Pryzby <pryzby@telsasoft.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox