Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kubWo-0002ui-Vy for pgsql-admin@arkaria.postgresql.org; Wed, 30 Dec 2020 13:28:38 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1kubWm-0008PS-W3 for pgsql-admin@arkaria.postgresql.org; Wed, 30 Dec 2020 13:28:36 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kubWm-0008PL-OY for pgsql-admin@lists.postgresql.org; Wed, 30 Dec 2020 13:28:36 +0000 Received: from mailout.easymail.ca ([64.68.200.34]) by magus.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kubWk-0003yH-F1 for pgsql-admin@postgresql.org; Wed, 30 Dec 2020 13:28:36 +0000 Received: from localhost (localhost [127.0.0.1]) by mailout.easymail.ca (Postfix) with ESMTP id F323DC157E; Wed, 30 Dec 2020 13:28:32 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at emo04-pco.easydns.vpn Received: from mailout.easymail.ca ([127.0.0.1]) by localhost (emo04-pco.easydns.vpn [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id nJzF4SfhxaDd; Wed, 30 Dec 2020 13:28:32 +0000 (UTC) Received: from [192.168.0.5] (75-166-125-160.hlrn.qwest.net [75.166.125.160]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mailout.easymail.ca (Postfix) with ESMTPSA id 16B8CC156E; Wed, 30 Dec 2020 13:28:27 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.120.23.2.4\)) Subject: Re: how to make duplicate finding query faster? From: Scott Ribe In-Reply-To: Date: Wed, 30 Dec 2020 06:28:26 -0700 Cc: Pgsql-admin , krishna@thewebconz.com, pgsql-admin@lists.postgresql.org Content-Transfer-Encoding: quoted-printable Message-Id: <84F2C1B0-3510-410B-8924-0EB7377684A9@elevated-dev.com> References: To: Sachin Kumar X-Mailer: Apple Mail (2.3608.120.23.2.4) List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk > On Dec 30, 2020, at 6:24 AM, Sachin Kumar = wrote: >=20 > Yes, I am checking one by one because my goal is to fail the whole = upload if there is any duplicate entry and to inform the user that they = have a duplicate entry in the file. That's not what I said, though. If you want to fail the whole copy, then = you don't have to check one by one, just try the copy--assuming you have = the correct constraints in place. Unless you want to tell the user *which* rows are duplicates, in which = case you can try a variant on my prior suggestion, copy into a temp = table, use a join to find duplicates...=