Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1sL47s-00DJKH-CG for pgsql-general@arkaria.postgresql.org; Sat, 22 Jun 2024 17:02:08 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1sL47p-00C8sx-UQ for pgsql-general@arkaria.postgresql.org; Sat, 22 Jun 2024 17:02:06 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1sL47p-00C8oi-6V for pgsql-general@lists.postgresql.org; Sat, 22 Jun 2024 17:02:05 +0000 Received: from mail-vk1-xa2e.google.com ([2607:f8b0:4864:20::a2e]) by makus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.94.2) (envelope-from ) id 1sL47h-002b05-BJ for pgsql-general@lists.postgresql.org; Sat, 22 Jun 2024 17:02:04 +0000 Received: by mail-vk1-xa2e.google.com with SMTP id 71dfb90a1353d-4ef6e79b47dso59418e0c.0 for ; Sat, 22 Jun 2024 10:01:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1719075716; x=1719680516; darn=lists.postgresql.org; h=to:subject:message-id:date:from:mime-version:from:to:cc:subject :date:message-id:reply-to; bh=hOaQRhfKiH+tjZVCPyDaqANGuG650QmR6YM6iTsVJ3E=; b=fLVe2cHmyhBwECF92IiGP2k6gVgme0SK2YIDHXTkMLxNprp1M6Z4DjVIFsiaXcGE76 cLPXBjHTvszQGiiS5PI6q7xth76eGE3lQvjibFSmmBcWsJy+Ktcb4P2QHgZETlmoJkXc 8elcBjGT9BSU7bOxFrSHy7X+r1G/iY7YPTYHexqo3eukss7qm1Q4kssjauDSY75HTzxW wBXizh6rqf3G9OsT/4fdJTUshErV2oeq3zIqJqy5fqwhljLqvvjr9kypPbDBFcN/tYAV amXozPUl5/C8m52j1oKaUWfoHFkL/HPXo9/QzSqbrGF2XQlAQ2clxFiti2RqiaQBS/BR Kj/w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1719075716; x=1719680516; h=to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=hOaQRhfKiH+tjZVCPyDaqANGuG650QmR6YM6iTsVJ3E=; b=GrlgJ4Vc20mHUY/bHgg8iMX1vTkS97URgy5iplUA9ujU9aw+R7B07ZwxeLEAFfpaKK lPn1d5dxUe5kstGLo/WCXhBPwXoPaGa6m65HbJFXkIXRuC435EbIhkhXjhFf7B0lwJYs MVtV9Uu0I+UvVhcCFHjvxyP2k9YzRjYgvcW5QCZllob0FBS5gpCCjU3tLQTvp8q4nU+/ 46W2Uqroi1jI5fJwG5SNs8AZ6veYII3yhHQotcH193+5tWygbUQ+IpF2pXWzo+QI0COP u9n3dSwBKpQNH2oBMf6/3beGtdKRzkx5Y3Sfmb//j/2H0AUcnUTc1J0X7vdoDiAYRGGo RRvA== X-Gm-Message-State: AOJu0YyMGVvUaKA93FETk6pY/oB5Apz6TD3PEdAdLXPy9+tP1yM3Y8AO Ya5vXKDSJMdCbtAke1RnhZytvDohlP5ohbhbxRfw22jcrYOb68F54b+g+m/UVbgtvYgYa1XB03e Y+mGBjajjFlPRKPA2riJiZfC5JZSgQtWW X-Google-Smtp-Source: AGHT+IHIs6g7HvShfu0feoRcNMzK2EJJI0QacGTOLnhCNPs2LriAc+UkeBbwqu45lfP8Q85mrdfoNi3IILgxtrr2BhM= X-Received: by 2002:a05:6122:3696:b0:4e9:7f5c:88ae with SMTP id 71dfb90a1353d-4ef6534a8a2mr924367e0c.1.1719075715497; Sat, 22 Jun 2024 10:01:55 -0700 (PDT) MIME-Version: 1.0 From: Shaheed Haque Date: Sat, 22 Jun 2024 18:01:44 +0100 Message-ID: Subject: pg_dump restores as expected on some machines and reports duplicate keys on others To: pgsql-general list Content-Type: multipart/alternative; boundary="0000000000005d45f8061b7d817d" List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --0000000000005d45f8061b7d817d Content-Type: text/plain; charset="UTF-8" Hi, I am using Postgres 14 on AWS RDS and am seeing the output of pg_dump be restored as expected by pg_restore on some database instances, and fail with reports of duplicate keys on other database instances: - My deployments are always a pair, one "logic VM" for Django etc and one "RDS instance". The psql client runs on the logic VM. The Postgres version is the same in all cases; psql reports: - psql (14.12 (Ubuntu 14.12-0ubuntu0.22.04.1), server 14.9) - The pg_restore is done using the same script in both cases. - In the failing cases, there are always the same 26 errors (listed in detail below), but in summary, 3 distinct "child" tables complain of a duplicate id=1, id=2 and id=3 respectively. - These "child" tables are FK-related via some intermediate table to a top level table. They form a polymorphic set. There are other similar child tables which do not appear to be affected: - polymorphicmodel - companybankdetail - companybankdetailde - companybankdetailgb <<< 1 duplicate, id=2 - companybankdetailus - companypostaldetail - companypostaldetailde - companypostaldetailgb <<< 1 duplicate, id=1 - companypostaldetailus - companytaxdetail - companytaxdetailde - companytaxdetailgb <<< 1 duplicate, id=3 - companytaxdetailus - ... - several other hierarchies, all error free - ... - I've looked at the dumped NNNN.dat files but they contain no duplicates. - The one difference I can think of between deployment pairs which work ok, and those which fail is that the logic VM (i.e. where the psql client script runs) is the use of a standard AWS ubuntu image for the OK case, versus a custom AWS image for the failing case. - The custom image is a saved snapshot of one created using the standard image. Why should the use of one type of VM image versus another cause pg_restore to hallucinate the duplicate records? Encls: 26 errors as mentioned... ======== pg_restore: while PROCESSING TOC: pg_restore: from TOC entry 5635; 1262 53894 DATABASE foo postgres pg_restore: error: could not execute query: ERROR: database "foo" already exists Command was: CREATE DATABASE foo WITH TEMPLATE = template0 ENCODING = 'UTF8' LOCALE = 'en_US.UTF-8'; pg_restore: from TOC entry 4808; 2606 80439 CONSTRAINT paiyroll_companybankdetail paiyroll_companybankdetail_pkey dbcoreuser pg_restore: error: could not execute query: ERROR: could not create unique index "paiyroll_companybankdetail_pkey" DETAIL: Key (polymorphicmodel_ptr_id)=(2) is duplicated. Command was: ALTER TABLE ONLY public.paiyroll_companybankdetail ADD CONSTRAINT paiyroll_companybankdetail_pkey PRIMARY KEY (polymorphicmodel_ptr_id); pg_restore: from TOC entry 4812; 2606 80443 CONSTRAINT paiyroll_companybankdetailgb paiyroll_companybankdetailgb_pkey dbcoreuser pg_restore: error: could not execute query: ERROR: could not create unique index "paiyroll_companybankdetailgb_pkey" DETAIL: Key (companybankdetail_ptr_id)=(2) is duplicated. Command was: ALTER TABLE ONLY public.paiyroll_companybankdetailgb ADD CONSTRAINT paiyroll_companybankdetailgb_pkey PRIMARY KEY (companybankdetail_ptr_id); pg_restore: from TOC entry 4817; 2606 80447 CONSTRAINT paiyroll_companypostaldetail paiyroll_companypostaldetail_pkey dbcoreuser pg_restore: error: could not execute query: ERROR: could not create unique index "paiyroll_companypostaldetail_pkey" DETAIL: Key (polymorphicmodel_ptr_id)=(1) is duplicated. Command was: ALTER TABLE ONLY public.paiyroll_companypostaldetail ADD CONSTRAINT paiyroll_companypostaldetail_pkey PRIMARY KEY (polymorphicmodel_ptr_id); pg_restore: from TOC entry 4821; 2606 80451 CONSTRAINT paiyroll_companypostaldetailgb paiyroll_companypostaldetailgb_pkey dbcoreuser pg_restore: error: could not execute query: ERROR: could not create unique index "paiyroll_companypostaldetailgb_pkey" DETAIL: Key (companypostaldetail_ptr_id)=(1) is duplicated. Command was: ALTER TABLE ONLY public.paiyroll_companypostaldetailgb ADD CONSTRAINT paiyroll_companypostaldetailgb_pkey PRIMARY KEY (companypostaldetail_ptr_id); pg_restore: from TOC entry 4826; 2606 80455 CONSTRAINT paiyroll_companytaxdetail paiyroll_companytaxdetail_pkey dbcoreuser pg_restore: error: could not execute query: ERROR: could not create unique index "paiyroll_companytaxdetail_pkey" DETAIL: Key (polymorphicmodel_ptr_id)=(3) is duplicated. Command was: ALTER TABLE ONLY public.paiyroll_companytaxdetail ADD CONSTRAINT paiyroll_companytaxdetail_pkey PRIMARY KEY (polymorphicmodel_ptr_id); pg_restore: from TOC entry 4830; 2606 80459 CONSTRAINT paiyroll_companytaxdetailgb paiyroll_companytaxdetailgb_pkey dbcoreuser pg_restore: error: could not execute query: ERROR: could not create unique index "paiyroll_companytaxdetailgb_pkey" DETAIL: Key (companytaxdetail_ptr_id)=(3) is duplicated. Command was: ALTER TABLE ONLY public.paiyroll_companytaxdetailgb ADD CONSTRAINT paiyroll_companytaxdetailgb_pkey PRIMARY KEY (companytaxdetail_ptr_id); pg_restore: from TOC entry 5018; 2606 80614 CONSTRAINT paiyroll_polymorphicmodel paiyroll_polymorphicmodel_pkey dbcoreuser pg_restore: error: could not execute query: ERROR: could not create unique index "paiyroll_polymorphicmodel_pkey" DETAIL: Key (id)=(3) is duplicated. Command was: ALTER TABLE ONLY public.paiyroll_polymorphicmodel ADD CONSTRAINT paiyroll_polymorphicmodel_pkey PRIMARY KEY (id); pg_restore: from TOC entry 5207; 2606 81004 FK CONSTRAINT paiyroll_companybankdetailde paiyroll_companybank_companybankdetail_pt_282859e0_fk_paiyroll_ dbcoreuser pg_restore: error: could not execute query: ERROR: there is no unique constraint matching given keys for referenced table "paiyroll_companybankdetail" Command was: ALTER TABLE ONLY public.paiyroll_companybankdetailde ADD CONSTRAINT paiyroll_companybank_companybankdetail_pt_282859e0_fk_paiyroll_ FOREIGN KEY (companybankdetail_ptr_id) REFERENCES public.paiyroll_companybankdetail(polymorphicmodel_ptr_id) DEF ERRABLE INITIALLY DEFERRED; pg_restore: from TOC entry 5209; 2606 81009 FK CONSTRAINT paiyroll_companybankdetailus paiyroll_companybank_companybankdetail_pt_555d62e4_fk_paiyroll_ dbcoreuser pg_restore: error: could not execute query: ERROR: there is no unique constraint matching given keys for referenced table "paiyroll_companybankdetail" Command was: ALTER TABLE ONLY public.paiyroll_companybankdetailus ADD CONSTRAINT paiyroll_companybank_companybankdetail_pt_555d62e4_fk_paiyroll_ FOREIGN KEY (companybankdetail_ptr_id) REFERENCES public.paiyroll_companybankdetail(polymorphicmodel_ptr_id) DEF ERRABLE INITIALLY DEFERRED; pg_restore: from TOC entry 5208; 2606 81014 FK CONSTRAINT paiyroll_companybankdetailgb paiyroll_companybank_companybankdetail_pt_5cf33abb_fk_paiyroll_ dbcoreuser pg_restore: error: could not execute query: ERROR: there is no unique constraint matching given keys for referenced table "paiyroll_companybankdetail" Command was: ALTER TABLE ONLY public.paiyroll_companybankdetailgb ADD CONSTRAINT paiyroll_companybank_companybankdetail_pt_5cf33abb_fk_paiyroll_ FOREIGN KEY (companybankdetail_ptr_id) REFERENCES public.paiyroll_companybankdetail(polymorphicmodel_ptr_id) DEF ERRABLE INITIALLY DEFERRED; pg_restore: from TOC entry 5206; 2606 81019 FK CONSTRAINT paiyroll_companybankdetail paiyroll_companybank_polymorphicmodel_ptr_d05e0f06_fk_paiyroll_ dbcoreuser pg_restore: error: could not execute query: ERROR: there is no unique constraint matching given keys for referenced table "paiyroll_polymorphicmodel" Command was: ALTER TABLE ONLY public.paiyroll_companybankdetail ADD CONSTRAINT paiyroll_companybank_polymorphicmodel_ptr_d05e0f06_fk_paiyroll_ FOREIGN KEY (polymorphicmodel_ptr_id) REFERENCES public.paiyroll_polymorphicmodel(id) DEFERRABLE INITIALLY DEFER RED; pg_restore: from TOC entry 5213; 2606 81029 FK CONSTRAINT paiyroll_companypostaldetailgb paiyroll_companypost_companypostaldetail__86015d44_fk_paiyroll_ dbcoreuser pg_restore: error: could not execute query: ERROR: there is no unique constraint matching given keys for referenced table "paiyroll_companypostaldetail" Command was: ALTER TABLE ONLY public.paiyroll_companypostaldetailgb ADD CONSTRAINT paiyroll_companypost_companypostaldetail__86015d44_fk_paiyroll_ FOREIGN KEY (companypostaldetail_ptr_id) REFERENCES public.paiyroll_companypostaldetail(polymorphicmodel_ptr_id) DEFERRABLE INITIALLY DEFERRED; pg_restore: from TOC entry 5214; 2606 81034 FK CONSTRAINT paiyroll_companypostaldetailus paiyroll_companypost_companypostaldetail__963c29cd_fk_paiyroll_ dbcoreuser pg_restore: error: could not execute query: ERROR: there is no unique constraint matching given keys for referenced table "paiyroll_companypostaldetail" Command was: ALTER TABLE ONLY public.paiyroll_companypostaldetailus ADD CONSTRAINT paiyroll_companypost_companypostaldetail__963c29cd_fk_paiyroll_ FOREIGN KEY (companypostaldetail_ptr_id) REFERENCES public.paiyroll_companypostaldetail(polymorphicmodel_ptr_id) DEFERRABLE INITIALLY DEFERRED; pg_restore: from TOC entry 5212; 2606 81039 FK CONSTRAINT paiyroll_companypostaldetailde paiyroll_companypost_companypostaldetail__f51f0ce0_fk_paiyroll_ dbcoreuser pg_restore: error: could not execute query: ERROR: there is no unique constraint matching given keys for referenced table "paiyroll_companypostaldetail" Command was: ALTER TABLE ONLY public.paiyroll_companypostaldetailde ADD CONSTRAINT paiyroll_companypost_companypostaldetail__f51f0ce0_fk_paiyroll_ FOREIGN KEY (companypostaldetail_ptr_id) REFERENCES public.paiyroll_companypostaldetail(polymorphicmodel_ptr_id) DEFERRABLE INITIALLY DEFERRED; pg_restore: from TOC entry 5211; 2606 81044 FK CONSTRAINT paiyroll_companypostaldetail paiyroll_companypost_polymorphicmodel_ptr_d9fc1568_fk_paiyroll_ dbcoreuser pg_restore: error: could not execute query: ERROR: there is no unique constraint matching given keys for referenced table "paiyroll_polymorphicmodel" Command was: ALTER TABLE ONLY public.paiyroll_companypostaldetail ADD CONSTRAINT paiyroll_companypost_polymorphicmodel_ptr_d9fc1568_fk_paiyroll_ FOREIGN KEY (polymorphicmodel_ptr_id) REFERENCES public.paiyroll_polymorphicmodel(id) DEFERRABLE INITIALLY DEFER RED; pg_restore: from TOC entry 5217; 2606 81054 FK CONSTRAINT paiyroll_companytaxdetailde paiyroll_companytaxd_companytaxdetail_ptr_6509fa4d_fk_paiyroll_ dbcoreuser pg_restore: error: could not execute query: ERROR: there is no unique constraint matching given keys for referenced table "paiyroll_companytaxdetail" Command was: ALTER TABLE ONLY public.paiyroll_companytaxdetailde ADD CONSTRAINT paiyroll_companytaxd_companytaxdetail_ptr_6509fa4d_fk_paiyroll_ FOREIGN KEY (companytaxdetail_ptr_id) REFERENCES public.paiyroll_companytaxdetail(polymorphicmodel_ptr_id) DEFER RABLE INITIALLY DEFERRED; pg_restore: from TOC entry 5219; 2606 81059 FK CONSTRAINT paiyroll_companytaxdetailus paiyroll_companytaxd_companytaxdetail_ptr_b8b74e53_fk_paiyroll_ dbcoreuser pg_restore: error: could not execute query: ERROR: there is no unique constraint matching given keys for referenced table "paiyroll_companytaxdetail" Command was: ALTER TABLE ONLY public.paiyroll_companytaxdetailus ADD CONSTRAINT paiyroll_companytaxd_companytaxdetail_ptr_b8b74e53_fk_paiyroll_ FOREIGN KEY (companytaxdetail_ptr_id) REFERENCES public.paiyroll_companytaxdetail(polymorphicmodel_ptr_id) DEFER RABLE INITIALLY DEFERRED; pg_restore: from TOC entry 5218; 2606 81064 FK CONSTRAINT paiyroll_companytaxdetailgb paiyroll_companytaxd_companytaxdetail_ptr_d9437c9a_fk_paiyroll_ dbcoreuser pg_restore: error: could not execute query: ERROR: there is no unique constraint matching given keys for referenced table "paiyroll_companytaxdetail" Command was: ALTER TABLE ONLY public.paiyroll_companytaxdetailgb ADD CONSTRAINT paiyroll_companytaxd_companytaxdetail_ptr_d9437c9a_fk_paiyroll_ FOREIGN KEY (companytaxdetail_ptr_id) REFERENCES public.paiyroll_companytaxdetail(polymorphicmodel_ptr_id) DEFER RABLE INITIALLY DEFERRED; pg_restore: from TOC entry 5216; 2606 81069 FK CONSTRAINT paiyroll_companytaxdetail paiyroll_companytaxd_polymorphicmodel_ptr_41f5cde7_fk_paiyroll_ dbcoreuser pg_restore: error: could not execute query: ERROR: there is no unique constraint matching given keys for referenced table "paiyroll_polymorphicmodel" Command was: ALTER TABLE ONLY public.paiyroll_companytaxdetail ADD CONSTRAINT paiyroll_companytaxd_polymorphicmodel_ptr_41f5cde7_fk_paiyroll_ FOREIGN KEY (polymorphicmodel_ptr_id) REFERENCES public.paiyroll_polymorphicmodel(id) DEFERRABLE INITIALLY DEFER RED; pg_restore: from TOC entry 5221; 2606 81079 FK CONSTRAINT paiyroll_debbie paiyroll_debbie_polymorphicmodel_ptr_8b0fe25a_fk_paiyroll_ dbcoreuser pg_restore: error: could not execute query: ERROR: there is no unique constraint matching given keys for referenced table "paiyroll_polymorphicmodel" Command was: ALTER TABLE ONLY public.paiyroll_debbie ADD CONSTRAINT paiyroll_debbie_polymorphicmodel_ptr_8b0fe25a_fk_paiyroll_ FOREIGN KEY (polymorphicmodel_ptr_id) REFERENCES public.paiyroll_polymorphicmodel(id) DEFERRABLE INITIALLY DEFERRED; pg_restore: from TOC entry 5234; 2606 81159 FK CONSTRAINT paiyroll_employeebankdetail paiyroll_employeeban_polymorphicmodel_ptr_6b4ae3ff_fk_paiyroll_ dbcoreuser pg_restore: error: could not execute query: ERROR: there is no unique constraint matching given keys for referenced table "paiyroll_polymorphicmodel" Command was: ALTER TABLE ONLY public.paiyroll_employeebankdetail ADD CONSTRAINT paiyroll_employeeban_polymorphicmodel_ptr_6b4ae3ff_fk_paiyroll_ FOREIGN KEY (polymorphicmodel_ptr_id) REFERENCES public.paiyroll_polymorphicmodel(id) DEFERRABLE INITIALLY DEFER RED; pg_restore: from TOC entry 5239; 2606 81184 FK CONSTRAINT paiyroll_employeepostaldetail paiyroll_employeepos_polymorphicmodel_ptr_4b75f2aa_fk_paiyroll_ dbcoreuser pg_restore: error: could not execute query: ERROR: there is no unique constraint matching given keys for referenced table "paiyroll_polymorphicmodel" Command was: ALTER TABLE ONLY public.paiyroll_employeepostaldetail ADD CONSTRAINT paiyroll_employeepos_polymorphicmodel_ptr_4b75f2aa_fk_paiyroll_ FOREIGN KEY (polymorphicmodel_ptr_id) REFERENCES public.paiyroll_polymorphicmodel(id) DEFERRABLE INITIALLY DEFER RED; pg_restore: from TOC entry 5249; 2606 81229 FK CONSTRAINT paiyroll_employeetaxeedetail paiyroll_employeetax_polymorphicmodel_ptr_2554ee34_fk_paiyroll_ dbcoreuser pg_restore: error: could not execute query: ERROR: there is no unique constraint matching given keys for referenced table "paiyroll_polymorphicmodel" Command was: ALTER TABLE ONLY public.paiyroll_employeetaxeedetail ADD CONSTRAINT paiyroll_employeetax_polymorphicmodel_ptr_2554ee34_fk_paiyroll_ FOREIGN KEY (polymorphicmodel_ptr_id) REFERENCES public.paiyroll_polymorphicmodel(id) DEFERRABLE INITIALLY DEFER RED; pg_restore: from TOC entry 5244; 2606 81234 FK CONSTRAINT paiyroll_employeetaxdetail paiyroll_employeetax_polymorphicmodel_ptr_8c064f71_fk_paiyroll_ dbcoreuser pg_restore: error: could not execute query: ERROR: there is no unique constraint matching given keys for referenced table "paiyroll_polymorphicmodel" Command was: ALTER TABLE ONLY public.paiyroll_employeetaxdetail ADD CONSTRAINT paiyroll_employeetax_polymorphicmodel_ptr_8c064f71_fk_paiyroll_ FOREIGN KEY (polymorphicmodel_ptr_id) REFERENCES public.paiyroll_polymorphicmodel(id) DEFERRABLE INITIALLY DEFER RED; pg_restore: from TOC entry 5258; 2606 81264 FK CONSTRAINT paiyroll_missing paiyroll_missing_polymorphicmodel_ptr_a1958c79_fk_paiyroll_ dbcoreuser pg_restore: error: could not execute query: ERROR: there is no unique constraint matching given keys for referenced table "paiyroll_polymorphicmodel" Command was: ALTER TABLE ONLY public.paiyroll_missing ADD CONSTRAINT paiyroll_missing_polymorphicmodel_ptr_a1958c79_fk_paiyroll_ FOREIGN KEY (polymorphicmodel_ptr_id) REFERENCES public.paiyroll_polymorphicmodel(id) DEFERRABLE INITIALLY DEFERRED; pg_restore: warning: errors ignored on restore: 26 ======== --0000000000005d45f8061b7d817d Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi,

I am using Postgres 14 o= n AWS RDS and am seeing the output of pg_dump be restored as expected by pg= _restore on some database instances, and fail with reports of duplicate key= s on other database instances:
  • My deployments are always = a pair, one "logic VM" for Django etc and one "RDS instance&= quot;. The psql client runs on the logic VM. The Postgres version is the sa= me in all cases; psql reports:
    • psql (14.12 (Ubuntu 14.12-0ubuntu0.22.04.1), server 14.9)
  • The pg_restore is done using the same script in both cases= .
  • In the failing cases, there are always the same 26 errors (listed= in detail below), but in summary, 3 distinct "child" tables comp= lain of a duplicate id=3D1, id=3D2 and id=3D3 respectively.=C2=A0
  • T= hese "child" tables are FK-related via some intermediate table to= a top level table. They form a polymorphic set. There are other similar ch= ild tables which do not appear to be affected:
    • polymorphicmodel=
      • companybankdetail
        • companybankdetailde
        • comp= anybankdetailgb=C2=A0 <<< 1 duplicate, id=3D2
        • companyb= ankdetailus
      • companypostaldetail
        • companypostald= etailde
        • companypostaldetailgb=C2=A0 <<< 1 duplicate, id=3D= 1
        • companypostaldetailus
      • companytaxdetail
        • <= li>companytaxdetailde
        • companytaxdetailgb=C2=A0 <<< 1 dupli= cate, id=3D3
        • companytaxdetailus
      • ...
      • severa= l other hierarchies, all error free
      • ...
  • I've = looked at the dumped NNNN.dat files but they contain no duplicates.
  • The one difference I can think of between deployment pairs which work ok, = and those which fail is that the logic VM (i.e. where the psql client scrip= t runs) is the use of a standard AWS ubuntu image for the OK case, versus a= custom AWS image for the failing case.
    • The custom image is a s= aved snapshot of one created using the standard image.
Why should the use of one type of VM image versus another cause pg_resto= re to hallucinate the duplicate records?

Encls= : 26 errors as mentioned...

=3D=3D=3D=3D=3D=3D=3D= =3D
pg_restore: while PROCESSING TO= C:
pg_restore: from TOC entry 5635; 1262 53894 DATABASE foo postgre= s
pg_restore: error: could not execute query: ERROR: =C2=A0database "= ;foo" already exists
Command was: CREATE DATABASE foo WITH TEMPLATE =3D template0 ENCODING = =3D 'UTF8' LOCALE =3D 'en_US.UTF-8';


pg_restore: from TOC entry 4808; 2606 80439 CONSTRAINT paiyroll_company= bankdetail paiyroll_companybankdetail_pkey dbcoreuser
pg_restore: error: could not execute query: ERROR: =C2=A0could not crea= te unique index "paiyroll_companybankdetail_pkey"
DETAIL: =C2=A0Key (polymorphicmodel_ptr_id)=3D(2) is duplicated.
Command was: ALTER TABLE ONLY public.paiyroll_companybankdetail
=C2=A0=C2=A0=C2=A0ADD CONSTRAINT paiyroll_companybankdetail_pkey PRIMA= RY KEY (polymorphicmodel_ptr_id);


pg_restore: from TOC entry 4812; 2606 80443 CONSTRAINT paiyroll_company= bankdetailgb paiyroll_companybankdetailgb_pkey dbcoreuser
pg_restore: error: could not execute query: ERROR: =C2=A0could not crea= te unique index "paiyroll_companybankdetailgb_pkey"
DETAIL: =C2=A0Key (companybankdetail_ptr_id)=3D(2) is duplicated.
Command was: ALTER TABLE ONLY public.paiyroll_companybankdetailgb
=C2=A0=C2=A0=C2=A0ADD CONSTRAINT paiyroll_companybankdetailgb_pkey PRI= MARY KEY (companybankdetail_ptr_id);


pg_restore: from TOC entry 4817; 2606 80447 CONSTRAINT paiyroll_company= postaldetail paiyroll_companypostaldetail_pkey dbcoreuser
pg_restore: error: could not execute query: ERROR: =C2=A0could not crea= te unique index "paiyroll_companypostaldetail_pkey"
DETAIL: =C2=A0Key (polymorphicmodel_ptr_id)=3D(1) is duplicated.
Command was: ALTER TABLE ONLY public.paiyroll_companypostaldetail
=C2=A0=C2=A0=C2=A0ADD CONSTRAINT paiyroll_companypostaldetail_pkey PRI= MARY KEY (polymorphicmodel_ptr_id);


pg_restore: from TOC entry 4821; 2606 80451 CONSTRAINT paiyroll_company= postaldetailgb paiyroll_companypostaldetailgb_pkey dbcoreuser
pg_restore: error: could not execute query: ERROR: =C2=A0could not crea= te unique index "paiyroll_companypostaldetailgb_pkey"
DETAIL: =C2=A0Key (companypostaldetail_ptr_id)=3D(1) is duplicated.
Command was: ALTER TABLE ONLY public.paiyroll_companypostaldetailgb
=C2=A0=C2=A0=C2=A0ADD CONSTRAINT paiyroll_companypostaldetailgb_pkey P= RIMARY KEY (companypostaldetail_ptr_id);


pg_restore: from TOC entry 4826; 2606 80455 CONSTRAINT paiyroll_company= taxdetail paiyroll_companytaxdetail_pkey dbcoreuser
pg_restore: error: could not execute query: ERROR: =C2=A0could not crea= te unique index "paiyroll_companytaxdetail_pkey"
DETAIL: =C2=A0Key (polymorphicmodel_ptr_id)=3D(3) is duplicated.
Command was: ALTER TABLE ONLY public.paiyroll_companytaxdetail
=C2=A0=C2=A0=C2=A0ADD CONSTRAINT paiyroll_companytaxdetail_pkey PRIMAR= Y KEY (polymorphicmodel_ptr_id);


pg_restore: from TOC entry 4830; 2606 80459 CONSTRAINT paiyroll_company= taxdetailgb paiyroll_companytaxdetailgb_pkey dbcoreuser
pg_restore: error: could not execute query: ERROR: =C2=A0could not crea= te unique index "paiyroll_companytaxdetailgb_pkey"
DETAIL: =C2=A0Key (companytaxdetail_ptr_id)=3D(3) is duplicated.
Command was: ALTER TABLE ONLY public.paiyroll_companytaxdetailgb
=C2=A0=C2=A0=C2=A0ADD CONSTRAINT paiyroll_companytaxdetailgb_pkey PRIM= ARY KEY (companytaxdetail_ptr_id);


pg_restore: from TOC entry 5018; 2606 80614 CONSTRAINT paiyroll_polymor= phicmodel paiyroll_polymorphicmodel_pkey dbcoreuser
pg_restore: error: could not execute query: ERROR: =C2=A0could not crea= te unique index "paiyroll_polymorphicmodel_pkey"
DETAIL: =C2=A0Key (id)=3D(3) is duplicated.
Command was: ALTER TABLE ONLY public.paiyroll_polymorphicmodel
=C2=A0=C2=A0=C2=A0ADD CONSTRAINT paiyroll_polymorphicmodel_pkey PRIMAR= Y KEY (id);


pg_restore: from TOC entry 5207; 2606 81004 FK CONSTRAINT paiyroll_comp= anybankdetailde paiyroll_companybank_companybankdetail_pt_282859e0_fk_paiyr= oll_ dbcoreuser
pg_restore: error: could not execute query: ERROR: =C2=A0there is no un= ique constraint matching given keys for referenced table "paiyroll_com= panybankdetail"
Command was: ALTER TABLE ONLY public.paiyroll_companybankdetailde
=C2=A0=C2=A0=C2=A0ADD CONSTRAINT paiyroll_companybank_companybankdetai= l_pt_282859e0_fk_paiyroll_ FOREIGN KEY (companybankdetail_ptr_id) REFERENCE= S public.paiyroll_companybankdetail(polymorphicmodel_ptr_id) DEF
ERRABLE= INITIALLY DEFERRED;


pg_restore: from TOC entry 5209; 2606 81009 FK CONSTRAINT paiyroll_comp= anybankdetailus paiyroll_companybank_companybankdetail_pt_555d62e4_fk_paiyr= oll_ dbcoreuser
pg_restore: error: could not execute query: ERROR: =C2=A0there is no un= ique constraint matching given keys for referenced table "paiyroll_com= panybankdetail"
Command was: ALTER TABLE ONLY public.paiyroll_companybankdetailus
=C2=A0=C2=A0=C2=A0ADD CONSTRAINT paiyroll_companybank_companybankdetai= l_pt_555d62e4_fk_paiyroll_ FOREIGN KEY (companybankdetail_ptr_id) REFERENCE= S public.paiyroll_companybankdetail(polymorphicmodel_ptr_id) DEF
ERRABLE= INITIALLY DEFERRED;


pg_restore: from TOC entry 5208; 2606 81014 FK CONSTRAINT paiyroll_comp= anybankdetailgb paiyroll_companybank_companybankdetail_pt_5cf33abb_fk_paiyr= oll_ dbcoreuser
pg_restore: error: could not execute query: ERROR: =C2=A0there is no un= ique constraint matching given keys for referenced table "paiyroll_com= panybankdetail"
Command was: ALTER TABLE ONLY public.paiyroll_companybankdetailgb
=C2=A0=C2=A0=C2=A0ADD CONSTRAINT paiyroll_companybank_companybankdetai= l_pt_5cf33abb_fk_paiyroll_ FOREIGN KEY (companybankdetail_ptr_id) REFERENCE= S public.paiyroll_companybankdetail(polymorphicmodel_ptr_id) DEF
ERRABLE= INITIALLY DEFERRED;


pg_restore: from TOC entry 5206; 2606 81019 FK CONSTRAINT paiyroll_comp= anybankdetail paiyroll_companybank_polymorphicmodel_ptr_d05e0f06_fk_paiyrol= l_ dbcoreuser
pg_restore: error: could not execute query: ERROR: =C2=A0there is no un= ique constraint matching given keys for referenced table "paiyroll_pol= ymorphicmodel"
Command was: ALTER TABLE ONLY public.paiyroll_companybankdetail
=C2=A0=C2=A0=C2=A0ADD CONSTRAINT paiyroll_companybank_polymorphicmodel= _ptr_d05e0f06_fk_paiyroll_ FOREIGN KEY (polymorphicmodel_ptr_id) REFERENCES= public.paiyroll_polymorphicmodel(id) DEFERRABLE INITIALLY DEFER
RED;


pg_restore: from TOC entry 5213; 2606 81029 FK CONSTRAINT paiyroll_comp= anypostaldetailgb paiyroll_companypost_companypostaldetail__86015d44_fk_pai= yroll_ dbcoreuser
pg_restore: error: could not execute query: ERROR: =C2=A0there is no un= ique constraint matching given keys for referenced table "paiyroll_com= panypostaldetail"
Command was: ALTER TABLE ONLY public.paiyroll_companypostaldetailgb
=C2=A0=C2=A0=C2=A0ADD CONSTRAINT paiyroll_companypost_companypostaldet= ail__86015d44_fk_paiyroll_ FOREIGN KEY (companypostaldetail_ptr_id) REFEREN= CES public.paiyroll_companypostaldetail(polymorphicmodel_ptr_id)
DEFERR= ABLE INITIALLY DEFERRED;


pg_restore: from TOC entry 5214; 2606 81034 FK CONSTRAINT paiyroll_comp= anypostaldetailus paiyroll_companypost_companypostaldetail__963c29cd_fk_pai= yroll_ dbcoreuser
pg_restore: error: could not execute query: ERROR: =C2=A0there is no un= ique constraint matching given keys for referenced table "paiyroll_com= panypostaldetail"
Command was: ALTER TABLE ONLY public.paiyroll_companypostaldetailus
=C2=A0=C2=A0=C2=A0ADD CONSTRAINT paiyroll_companypost_companypostaldet= ail__963c29cd_fk_paiyroll_ FOREIGN KEY (companypostaldetail_ptr_id) REFEREN= CES public.paiyroll_companypostaldetail(polymorphicmodel_ptr_id)
DEFERR= ABLE INITIALLY DEFERRED;


pg_restore: from TOC entry 5212; 2606 81039 FK CONSTRAINT paiyroll_comp= anypostaldetailde paiyroll_companypost_companypostaldetail__f51f0ce0_fk_pai= yroll_ dbcoreuser
pg_restore: error: could not execute query: ERROR: =C2=A0there is no un= ique constraint matching given keys for referenced table "paiyroll_com= panypostaldetail"
Command was: ALTER TABLE ONLY public.paiyroll_companypostaldetailde
=C2=A0=C2=A0=C2=A0ADD CONSTRAINT paiyroll_companypost_companypostaldet= ail__f51f0ce0_fk_paiyroll_ FOREIGN KEY (companypostaldetail_ptr_id) REFEREN= CES public.paiyroll_companypostaldetail(polymorphicmodel_ptr_id)
DEFERR= ABLE INITIALLY DEFERRED;


pg_restore: from TOC entry 5211; 2606 81044 FK CONSTRAINT paiyroll_comp= anypostaldetail paiyroll_companypost_polymorphicmodel_ptr_d9fc1568_fk_paiyr= oll_ dbcoreuser
pg_restore: error: could not execute query: ERROR: =C2=A0there is no un= ique constraint matching given keys for referenced table "paiyroll_pol= ymorphicmodel"
Command was: ALTER TABLE ONLY public.paiyroll_companypostaldetail
=C2=A0=C2=A0=C2=A0ADD CONSTRAINT paiyroll_companypost_polymorphicmodel= _ptr_d9fc1568_fk_paiyroll_ FOREIGN KEY (polymorphicmodel_ptr_id) REFERENCES= public.paiyroll_polymorphicmodel(id) DEFERRABLE INITIALLY DEFER
RED;


pg_restore: from TOC entry 5217; 2606 81054 FK CONSTRAINT paiyroll_comp= anytaxdetailde paiyroll_companytaxd_companytaxdetail_ptr_6509fa4d_fk_paiyro= ll_ dbcoreuser
pg_restore: error: could not execute query: ERROR: =C2=A0there is no un= ique constraint matching given keys for referenced table "paiyroll_com= panytaxdetail"
Command was: ALTER TABLE ONLY public.paiyroll_companytaxdetailde
=C2=A0=C2=A0=C2=A0ADD CONSTRAINT paiyroll_companytaxd_companytaxdetail= _ptr_6509fa4d_fk_paiyroll_ FOREIGN KEY (companytaxdetail_ptr_id) REFERENCES= public.paiyroll_companytaxdetail(polymorphicmodel_ptr_id) DEFER
RABLE I= NITIALLY DEFERRED;


pg_restore: from TOC entry 5219; 2606 81059 FK CONSTRAINT paiyroll_comp= anytaxdetailus paiyroll_companytaxd_companytaxdetail_ptr_b8b74e53_fk_paiyro= ll_ dbcoreuser
pg_restore: error: could not execute query: ERROR: =C2=A0there is no un= ique constraint matching given keys for referenced table "paiyroll_com= panytaxdetail"
Command was: ALTER TABLE ONLY public.paiyroll_companytaxdetailus
=C2=A0=C2=A0=C2=A0ADD CONSTRAINT paiyroll_companytaxd_companytaxdetail= _ptr_b8b74e53_fk_paiyroll_ FOREIGN KEY (companytaxdetail_ptr_id) REFERENCES= public.paiyroll_companytaxdetail(polymorphicmodel_ptr_id) DEFER
RABLE I= NITIALLY DEFERRED;


pg_restore: from TOC entry 5218; 2606 81064 FK CONSTRAINT paiyroll_comp= anytaxdetailgb paiyroll_companytaxd_companytaxdetail_ptr_d9437c9a_fk_paiyro= ll_ dbcoreuser
pg_restore: error: could not execute query: ERROR: =C2=A0there is no un= ique constraint matching given keys for referenced table "paiyroll_com= panytaxdetail"
Command was: ALTER TABLE ONLY public.paiyroll_companytaxdetailgb
=C2=A0=C2=A0=C2=A0ADD CONSTRAINT paiyroll_companytaxd_companytaxdetail= _ptr_d9437c9a_fk_paiyroll_ FOREIGN KEY (companytaxdetail_ptr_id) REFERENCES= public.paiyroll_companytaxdetail(polymorphicmodel_ptr_id) DEFER
RABLE I= NITIALLY DEFERRED;


pg_restore: from TOC entry 5216; 2606 81069 FK CONSTRAINT paiyroll_comp= anytaxdetail paiyroll_companytaxd_polymorphicmodel_ptr_41f5cde7_fk_paiyroll= _ dbcoreuser
pg_restore: error: could not execute query: ERROR: =C2=A0there is no un= ique constraint matching given keys for referenced table "paiyroll_pol= ymorphicmodel"
Command was: ALTER TABLE ONLY public.paiyroll_companytaxdetail
=C2=A0=C2=A0=C2=A0ADD CONSTRAINT paiyroll_companytaxd_polymorphicmodel= _ptr_41f5cde7_fk_paiyroll_ FOREIGN KEY (polymorphicmodel_ptr_id) REFERENCES= public.paiyroll_polymorphicmodel(id) DEFERRABLE INITIALLY DEFER
RED;


pg_restore: from TOC entry 5221; 2606 81079 FK CONSTRAINT paiyroll_debb= ie paiyroll_debbie_polymorphicmodel_ptr_8b0fe25a_fk_paiyroll_ dbcoreuser
pg_restore: error: could not execute query: ERROR: =C2=A0there is no un= ique constraint matching given keys for referenced table "paiyroll_pol= ymorphicmodel"
Command was: ALTER TABLE ONLY public.paiyroll_debbie
=C2=A0=C2=A0=C2=A0ADD CONSTRAINT paiyroll_debbie_polymorphicmodel_ptr_= 8b0fe25a_fk_paiyroll_ FOREIGN KEY (polymorphicmodel_ptr_id) REFERENCES publ= ic.paiyroll_polymorphicmodel(id) DEFERRABLE INITIALLY DEFERRED;


pg_restore: from TOC entry 5234; 2606 81159 FK CONSTRAINT paiyroll_empl= oyeebankdetail paiyroll_employeeban_polymorphicmodel_ptr_6b4ae3ff_fk_paiyro= ll_ dbcoreuser
pg_restore: error: could not execute query: ERROR: =C2=A0there is no un= ique constraint matching given keys for referenced table "paiyroll_pol= ymorphicmodel"
Command was: ALTER TABLE ONLY public.paiyroll_employeebankdetail
=C2=A0=C2=A0=C2=A0ADD CONSTRAINT paiyroll_employeeban_polymorphicmodel= _ptr_6b4ae3ff_fk_paiyroll_ FOREIGN KEY (polymorphicmodel_ptr_id) REFERENCES= public.paiyroll_polymorphicmodel(id) DEFERRABLE INITIALLY DEFER
RED;


pg_restore: from TOC entry 5239; 2606 81184 FK CONSTRAINT paiyroll_empl= oyeepostaldetail paiyroll_employeepos_polymorphicmodel_ptr_4b75f2aa_fk_paiy= roll_ dbcoreuser
pg_restore: error: could not execute query: ERROR: =C2=A0there is no un= ique constraint matching given keys for referenced table "paiyroll_pol= ymorphicmodel"
Command was: ALTER TABLE ONLY public.paiyroll_employeepostaldetail
=C2=A0=C2=A0=C2=A0ADD CONSTRAINT paiyroll_employeepos_polymorphicmodel= _ptr_4b75f2aa_fk_paiyroll_ FOREIGN KEY (polymorphicmodel_ptr_id) REFERENCES= public.paiyroll_polymorphicmodel(id) DEFERRABLE INITIALLY DEFER
RED;


pg_restore: from TOC entry 5249; 2606 81229 FK CONSTRAINT paiyroll_empl= oyeetaxeedetail paiyroll_employeetax_polymorphicmodel_ptr_2554ee34_fk_paiyr= oll_ dbcoreuser
pg_restore: error: could not execute query: ERROR: =C2=A0there is no un= ique constraint matching given keys for referenced table "paiyroll_pol= ymorphicmodel"
Command was: ALTER TABLE ONLY public.paiyroll_employeetaxeedetail
=C2=A0=C2=A0=C2=A0ADD CONSTRAINT paiyroll_employeetax_polymorphicmodel= _ptr_2554ee34_fk_paiyroll_ FOREIGN KEY (polymorphicmodel_ptr_id) REFERENCES= public.paiyroll_polymorphicmodel(id) DEFERRABLE INITIALLY DEFER
RED;


pg_restore: from TOC entry 5244; 2606 81234 FK CONSTRAINT paiyroll_empl= oyeetaxdetail paiyroll_employeetax_polymorphicmodel_ptr_8c064f71_fk_paiyrol= l_ dbcoreuser
pg_restore: error: could not execute query: ERROR: =C2=A0there is no un= ique constraint matching given keys for referenced table "paiyroll_pol= ymorphicmodel"
Command was: ALTER TABLE ONLY public.paiyroll_employeetaxdetail
=C2=A0=C2=A0=C2=A0ADD CONSTRAINT paiyroll_employeetax_polymorphicmodel= _ptr_8c064f71_fk_paiyroll_ FOREIGN KEY (polymorphicmodel_ptr_id) REFERENCES= public.paiyroll_polymorphicmodel(id) DEFERRABLE INITIALLY DEFER
RED;


pg_restore: from TOC entry 5258; 2606 81264 FK CONSTRAINT paiyroll_miss= ing paiyroll_missing_polymorphicmodel_ptr_a1958c79_fk_paiyroll_ dbcoreuser
pg_restore: error: could not execute query: ERROR: =C2=A0there is no un= ique constraint matching given keys for referenced table "paiyroll_pol= ymorphicmodel"
Command was: ALTER TABLE ONLY public.paiyroll_missing
=C2=A0=C2=A0=C2=A0ADD CONSTRAINT paiyroll_missing_polymorphicmodel_ptr= _a1958c79_fk_paiyroll_ FOREIGN KEY (polymorphicmodel_ptr_id) REFERENCES pub= lic.paiyroll_polymorphicmodel(id) DEFERRABLE INITIALLY DEFERRED;


pg_restore: warning: errors ignored on restore: 26
=3D=3D=3D=3D=3D=3D=3D=3D
--0000000000005d45f8061b7d817d--