Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1ueafD-00CA1i-8m for pgsql-general@arkaria.postgresql.org; Wed, 23 Jul 2025 14:41:48 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1ueafB-002bFl-8k for pgsql-general@arkaria.postgresql.org; Wed, 23 Jul 2025 14:41:45 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1ueafA-002bFc-5g for pgsql-general@lists.postgresql.org; Wed, 23 Jul 2025 14:41:45 +0000 Received: from fhigh-b3-smtp.messagingengine.com ([202.12.124.154]) by makus.postgresql.org with smtp (Exim 4.96) (envelope-from ) id 1ueaf8-000Oru-1V for pgsql-general@postgresql.org; Wed, 23 Jul 2025 14:41:43 +0000 Received: from phl-compute-06.internal (phl-compute-06.phl.internal [10.202.2.46]) by mailfhigh.stl.internal (Postfix) with ESMTP id 330B67A02A6; Wed, 23 Jul 2025 10:41:41 -0400 (EDT) Received: from phl-mailfrontend-02 ([10.202.2.163]) by phl-compute-06.internal (MEProxy); Wed, 23 Jul 2025 10:41:41 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=aklaver.com; h= cc:content-transfer-encoding:content-type:content-type:date:date :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to; s=fm1; t=1753281701; x=1753368101; bh=2cV0ydWwyxB3quxfoZG5ihugaZJ4B/fEgso2hIdcOkA=; b= VbvZ+K79wBTGarCUoHqVrNMsCw/XMd9s+DUfbbWQdfd31bCJwLsmzO7p9sjgpc7Y +Rv0YdCxsDJw94NEmyN+RVZQkhkt2i52dZ3uR7nkdR3IFHL4vGo3Q8uC8/ObuO3H rP9hiI/utrhPAGVwUafbhD226BGCLjgx29+18+aOuLp9P0rFVvdLsz9IlloOfs9h k67rSYWNM2ZrvlBpr7trKCE+Nb69xLsglPdL8e/x1gwpJqIZJhw+eu9nu4bdOvij H0kckbk9wlnfpAudcfKoH+3+y8aCypBGC9TdrjVgY7T0tE4rSySiiPBAXimSp8HH 0lY5noUyIc4jBiOTyr+wJw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:subject:subject:to:to:x-me-proxy:x-me-sender :x-me-sender:x-sasl-enc; s=fm2; t=1753281701; x=1753368101; bh=2 cV0ydWwyxB3quxfoZG5ihugaZJ4B/fEgso2hIdcOkA=; b=B5S1QbQrQ/nQSsRYD nGikJeTdVfVhs+XubjmQRYIBvNIgakyiCbUCh26y2Jw3Pqma5bqCxltvbMv/Q0jX a25U26dE3Y3+6TmcnwBa6tTYsMLVqHh1HqxfhamR29COHi0CGG/UJ77O/o6SJQbl m8goSb3a9g8YJtRtnhyV3X50ZF3/fkc1/gIdXNKyjWl50Cll0yOd17hT+Hqo1Wu0 65Jy8rknDrYaoJPga1CDbx11R6ljvXRRmZibCCSww2wJC7sk6L+eKZeJGm44hw8X goQqyF9xa153KXB2JMww0kK2EwDRtuNcvmyTnGxe28TkvetaAeeyydfTFZbpS97k 6ew8A== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeeffedrtdefgdejkedthecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpuffrtefokffrpgfnqfghnecuuegr ihhlohhuthemuceftddtnecunecujfgurhepkfffgggfuffvfhfhjggtgfesthekredttd dvjeenucfhrhhomheptegurhhirghnucfmlhgrvhgvrhcuoegrughrihgrnhdrkhhlrghv vghrsegrkhhlrghvvghrrdgtohhmqeenucggtffrrghtthgvrhhnpeelgeevkeekkeeuie fgtdevieeluefhfedufeetkeejffekjeeujeehgeehgeektdenucffohhmrghinhepphho shhtghhrvghsqhhlrdhorhhgnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpe hmrghilhhfrhhomheprggurhhirghnrdhklhgrvhgvrhesrghklhgrvhgvrhdrtghomhdp nhgspghrtghpthhtohepvddpmhhouggvpehsmhhtphhouhhtpdhrtghpthhtohepshhivh grphhoshhtghhrvghsseihrghhohhordgtohhmpdhrtghpthhtohepphhgshhqlhdqghgv nhgvrhgrlhesphhoshhtghhrvghsqhhlrdhorhhg X-ME-Proxy: Feedback-ID: i76984098:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed, 23 Jul 2025 10:41:40 -0400 (EDT) Message-ID: <0ff2e34f-a73c-490b-aee6-79ad7bbd4f0c@aklaver.com> Date: Wed, 23 Jul 2025 07:41:39 -0700 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: Is there any limit on the number of rows to import using copy command To: "sivapostgres@yahoo.com" , Pgsql-general References: <1453510076.1900935.1753260637232.ref@mail.yahoo.com> <1453510076.1900935.1753260637232@mail.yahoo.com> Content-Language: en-US From: Adrian Klaver In-Reply-To: <1453510076.1900935.1753260637232@mail.yahoo.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk On 7/23/25 01:50, sivapostgres@yahoo.com wrote: > Tried in PostgreSQL 11.11 , PostgreSQL 15.2 in Windows 10 > > The above command succeeds, when > 1.  The trigger in Table1 is disabled with all other constraints on. > 2.  The no. of rows is within 16000 or less, with Trigger enabled.  We > haven't tried with higher no of rows. Why not? > > The above command goes on infinite loop, when > 1.  We try to transfer all 85000 rows at once, with Trigger and other > constraints in table1 enabled.  We waited for 1.5 hrs first time and 2.5 > hrs second time before cancelling the operation. Try with the triggers disabled. > > I read in the documentation that the fastest way to transfer data is to > use Copy command.  And I couldn't find any limit in transferring data > using that command.  One could easily transfer millions of rows using > this command. It is, I have used it for much larger datasets then 85000 rows and it completed in less time. As example using Duckdb it took the NYC taxi data set yellow_tripdata_2023-09.parquet, transformed it and loaded using COPY in 5.4 secs for ~2.8 million rows. FYI, BEGIN in plpgsql is not the same as in SQL. In plpgsql it represents a block. I don't think you need the BEGIN/END around the UPDATE and INSERT queries. See https://www.postgresql.org/docs/current/plpgsql-structure.html for more information. > Any (other) suggestion to transfer successfully is really appreciated. > > Happiness Always > BKR Sivaprakash > -- Adrian Klaver adrian.klaver@aklaver.com