From: wangsh.fnst@fujitsu.com <wangsh.fnst@fujitsu.com>
To: pgsql-bugs@lists.postgresql.org <pgsql-bugs@lists.postgresql.org>
Subject: "unexpected duplicate for tablespace" problem in logical replication
Date: Wed, 6 Apr 2022 02:14:26 +0000
Message-ID: <bbaaf9f9-ebb2-645f-54bb-34d6efc7ac42@fujitsu.com> (raw)
Hi,
I met a problem while using logical replication in PG11 and I think all
the PG version have this problem.
The log looks like:
> ERROR: unexpected duplicate for tablespace 0, relfilenode xxxxxxx
Someone also reported this problem in [1], but no one has responded to it.
I did some investigation, and found a way to reproduce this problem. The
steps are:
1. create a table (call it tableX) and truncate it.
2. cycle through 2^32 OIDs.
3. restart the database to clear all the cache.
4. create a temp table which make the temp table's OID equals to the
tableX's relfilenode and insert any data into tableX.
The attachment(run.sh) can reproduce this problem in PG10 and PG11with
the help of option 'WITH OIDS'. I don't find any way to cycle the OIDs
quickly in branch master, but I use the gdb to reproduce this problem too.
Now, function GetNewRelFileNode() only checks:
1. duplicated OIDs in pg_class.
2. relpath(rnode) is exists in disk.
However, the result of relpath(temp table) and relpath(non-temp table)
are different, temp table's relpath() has a prefix "t%d". That means, if
there is a table that value of relfilenode is 20000(but the value of oid
isn't 20000), it's possible to create a temp table that value of
relfilenode is also 20000.
I think function GetNewRelFileNode() should always check the duplicated
relfilenode, see the patch(a simple to way to fix this problem is master
branch).
Any comment?
Regards,
Shenhao Wang
[1]
https://www.postgresql.org/message-id/flat/CAM5YvKTPxmMT%3DS7iPcu5SgmaOv4S4nhE1HZRO_sdFX9cXeXXOQ%40m...Attachments:
[application/x-shellscript] run.sh (1.4K, ../bbaaf9f9-ebb2-645f-54bb-34d6efc7ac42@fujitsu.com/2-run.sh)
download
[text/x-patch] 0001-Check-the-duplicated-relfilenode-when-create-table-o.patch (4.4K, ../bbaaf9f9-ebb2-645f-54bb-34d6efc7ac42@fujitsu.com/3-0001-Check-the-duplicated-relfilenode-when-create-table-o.patch)
download | inline diff:
From bc3a756e71bef744f1c9cf652c0eb0e4933c965b Mon Sep 17 00:00:00 2001
From: Shenhao Wang <wangsh.fnst@fujitsu.com>
Date: Sat, 2 Apr 2022 11:03:44 +0800
Subject: [PATCH] Check the duplicated relfilenode when create table or
truncate table.
---
src/backend/catalog/catalog.c | 102 ++++++++++++++++++++++++++--------
1 file changed, 78 insertions(+), 24 deletions(-)
diff --git a/src/backend/catalog/catalog.c b/src/backend/catalog/catalog.cindex dfd5fb669e..7f51445573 100644--- a/src/backend/catalog/catalog.c+++ b/src/backend/catalog/catalog.c@@ -55,6 +55,8 @@
#define GETNEWOID_LOG_THRESHOLD 1000000
#define GETNEWOID_LOG_MAX_INTERVAL 128000000
+static bool isRelNodeCollision(RelFileNodeBackend rnode, Relation relation);+
/*
* IsSystemRelation
* True iff the relation is either a system catalog or a toast table.
@@ -490,8 +492,7 @@ Oid
GetNewRelFileNode(Oid reltablespace, Relation pg_class, char relpersistence)
{
RelFileNodeBackend rnode;
- char *rpath;- bool collides;+ bool usedAsOID;
BackendId backend;
/*
@@ -515,6 +516,16 @@ GetNewRelFileNode(Oid reltablespace, Relation pg_class, char relpersistence)
return InvalidOid; /* placate compiler */
}
+ if (pg_class != NULL)+ {+ usedAsOID = true;+ }+ else+ {+ pg_class = table_open(RelationRelationId, AccessShareLock);+ usedAsOID = false;+ }+
/* This logic should match RelationInitPhysicalAddr */
rnode.node.spcNode = reltablespace ? reltablespace : MyDatabaseTableSpace;
rnode.node.dbNode = (rnode.node.spcNode == GLOBALTABLESPACE_OID) ? InvalidOid : MyDatabaseId;
@@ -531,34 +542,16 @@ GetNewRelFileNode(Oid reltablespace, Relation pg_class, char relpersistence)
CHECK_FOR_INTERRUPTS();
/* Generate the OID */
- if (pg_class)+ if (usedAsOID)
rnode.node.relNode = GetNewOidWithIndex(pg_class, ClassOidIndexId,
Anum_pg_class_oid);
else
rnode.node.relNode = GetNewObjectId();
- /* Check for existing file of same name */- rpath = relpath(rnode, MAIN_FORKNUM);+ } while (isRelNodeCollision(rnode, pg_class));- if (access(rpath, F_OK) == 0)- {- /* definite collision */- collides = true;- }- else- {- /*- * Here we have a little bit of a dilemma: if errno is something- * other than ENOENT, should we declare a collision and loop? In- * practice it seems best to go ahead regardless of the errno. If- * there is a colliding file we will get an smgr failure when we- * attempt to create the new relation file.- */- collides = false;- }-- pfree(rpath);- } while (collides);+ if (!usedAsOID)+ table_close(pg_class, AccessShareLock);
return rnode.node.relNode;
}
@@ -666,3 +659,64 @@ pg_stop_making_pinned_objects(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
++/*+ * isRelNodeCollision+ * Checking the rnode collides with existing one.+ */+static bool+isRelNodeCollision(RelFileNodeBackend rnode, Relation relation)+{+ char *rpath;+ bool collides;+ SysScanDesc scandesc;+ static ScanKeyData skey[2];++ /* Check for existing file of same name */+ rpath = relpath(rnode, MAIN_FORKNUM);++ if (access(rpath, F_OK) == 0)+ {+ /* definite collision */+ collides = true;+ }+ else+ {+ ScanKeyInit(&skey[0],+ Anum_pg_class_reltablespace,+ BTEqualStrategyNumber, F_OIDEQ,+ ObjectIdGetDatum(+ rnode.node.spcNode == MyDatabaseTableSpace ?+ InvalidOid : MyDatabaseTableSpace));+ ScanKeyInit(&skey[1],+ Anum_pg_class_relfilenode,+ BTEqualStrategyNumber, F_OIDEQ,+ ObjectIdGetDatum(rnode.node.relNode));++ scandesc = systable_beginscan(relation,+ ClassTblspcRelfilenodeIndexId,+ true, NULL, 2, skey);++ if (HeapTupleIsValid(systable_getnext(scandesc)))+ {+ /* duplicate tablespace and relfilenode in pg_class */+ collides = true;+ }+ else+ {+ /*+ * Here we have a little bit of a dilemma: if errno is something+ * other than ENOENT, should we declare a collision and loop? In+ * practice it seems best to go ahead regardless of the errno. If+ * there is a colliding file we will get an smgr failure when we+ * attempt to create the new relation file.+ */+ collides = false;+ }++ systable_endscan(scandesc);+ }++ pfree(rpath);+ return collides;+}--
2.32.0
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-bugs@postgresql.org
Cc: wangsh.fnst@fujitsu.com, pgsql-bugs@lists.postgresql.org
Subject: Re: "unexpected duplicate for tablespace" problem in logical replication
In-Reply-To: <bbaaf9f9-ebb2-645f-54bb-34d6efc7ac42@fujitsu.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox