Subject: Loading data from one table to another

From: Sabyasachi Mukherjee <[email-masked]>
Date: Sun, 13 Oct 2024 06:24:04 +0000

Dear All,

We have a Postgres DB with 99 GB of data. The query performances are slow as expected. I am trying to introduce monthly partitions for this table.
To copy the data - the command that I am trying to execute is (Insert into ...(Select * from )) using PGAdmin
The DB is hosted in Windows server having 8 GB of RAM.

Note - we do not have any surrogate key in this table. And the event time recorded is also duplicated.

The command fails every time.
Can you please advise some strategy to get this done?

Once the pgadmin lost connection.
Another error that I have got is,

ERROR:  Failed on request of size 1048576 in memory context "AfterTriggerEvents".out of memory
ERROR:  out of memory
SQL state: 53200
Detail: Failed on request of size 1048576 in memory context "AfterTriggerEvents".

Thanks & regards

Sabyasachi Mukherjee

---

From: Jean-Paul POZZI <[email-masked]>
Date: Sun, 13 Oct 2024 11:36:48 +0200 (CEST)

Hello,

It seems that the command is too lonbg to execute from pgadmin, try to execute the same Sql from a terminal ... or split your work with "from date1 to date2", you can split the work in 2,3 or more tasks and it will succed.

Regards

JP P

De: "Sabyasachi Mukherjee" <[email-masked]>
À: "undefined" <[email-masked]>
Envoyé: dimanche 13 octobre 2024 08:26
Objet: Loading data from one table to another

Dear All,

Thanks & regards

Sabyasachi Mukherjee

---

From: Holger Jakobs <[email-masked]>
Date: Sun, 13 Oct 2024 12:31:00 +0200

Don't forget to look up the SQL state. There are some entries regarding it on stackoverflow

-- 
Holger Jakobs, Bergisch Gladbach 
Tel. +49 178 9759012

---

From: Scott Ribe <[email-masked]>
Date: Sun, 13 Oct 2024 08:34:51 -0600

Failing to allocate a memory segment of only 1MB seems suspicious. What are your hardware specs?

You might want to look at your memory & parallel query settings in postgresql.conf--perhaps some of those are set too high relative to your server's memory.

I wouldn't really expect queries to be slow on 99GB, by the way. That is not a large database. Partitioning is reasonable if you want to prune old data periodically, as then you can just drop old partitions. But unless your queries are regularly have to perform table scans, I wouldn't expect them to be slow. And if they are performing table scans, you'd need to investigate how to fix that--what indexes to add. Partitioning will get you incremental performance gains in such a case, while the right index will get you exponential gains.

---

From: Laurenz Albe <[email-masked]>
Date: Mon, 14 Oct 2024 08:36:08 +0200

> Detail: Failed on request of size 1048576 in memory context "AfterTriggerEvents".

You have a row level AFTER INSERT trigger on the destination table.
Try without the trigger.

Yours,
Laurenz Albe

---

From: vignesh kumar <[email-masked]>
Date: Tue, 15 Oct 2024 16:01:38 +0000

Check if you have foreign table references and they should hold on delete cascade that should resolve data being dupliated

Sent from Outlook for Android<https://aka.ms/AAb9ysg>
________________________________
From: Laurenz Albe <[email-masked]>
Sent: Monday, October 14, 2024 12:06:08 PM
To: Sabyasachi Mukherjee <[email-masked]>; [email-masked] <[email-masked]>
Subject: Re: Loading data from one table to another

Yours,
Laurenz Albe