MIME-Version: 1.0
From: me nefcanto <sn.1361@gmail.com>
Date: Sat, 1 Mar 2025 18:48:41 +0330
Message-ID: <CAEHBEOCqqkSitU5gx=8aHD3ZeLtRy-1eC=Xs=xB-thFNR9JpYA@mail.gmail.com>
Subject: Please implement a catch-all error handler per row, for COPY
To: pgsql-general@lists.postgresql.org
Content-Type: multipart/alternative; boundary="000000000000d4be81062f4970f9"
Archived-At: <https://www.postgresql.org/message-id/CAEHBEOCqqkSitU5gx%3D8aHD3ZeLtRy-1eC%3DXs%3DxB-thFNR9JpYA%40mail.gmail.com>
Precedence: bulk

--000000000000d4be81062f4970f9
Content-Type: text/plain; charset="UTF-8"

Hello

Please consider these scenarios:

- I want to create a million fake products, sometimes even 100 million
(we're on MariaDB now and we plan to migrate to Postgres). My team uses
fake data for performance tests and other use cases.
- Another scenario is translations. Even in production, we have translation
files for more than 20 languages, and for more than 2 thousand keys. That
means we need to insert 40 thousand translation records in the production.
- Another scenario is updating nested model values for a large hierarchical
table. For example, the categories table. Anytime the user changes a record
in that table we need to recalculate the nested model for the entire
categories and bulk update the results.

All of these scenarios are such that data sanitation is difficult if not
possible before doing the bulk operation (copy).

I realized that when we specify `on_error ignore` it just handles a handful
of errors. I thought this was a bug and sent an email to the pgsql-bugs
maling list. But they said it's the intended behavior.

Can you please provide a row-level catch-all handler for the copy command?

Regards
Saeed

--000000000000d4be81062f4970f9
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div class=3D"gmail_default" style=3D"font-family:tahoma,s=
ans-serif"><div class=3D"gmail_default">Hello</div><div class=3D"gmail_defa=
ult"><br></div><div class=3D"gmail_default">Please consider these scenarios=
:</div><div class=3D"gmail_default"><br></div><div class=3D"gmail_default">=
- I want to create a million fake products, sometimes even 100 million (we&=
#39;re on MariaDB now and we plan to migrate to Postgres). My team uses fak=
e data for performance tests and other use cases.</div><div class=3D"gmail_=
default">- Another scenario is translations. Even in production, we have tr=
anslation files for more than 20 languages, and for more than 2 thousand ke=
ys. That means we need to insert 40 thousand translation records in the pro=
duction.</div><div class=3D"gmail_default">- Another scenario is updating n=
ested model values for a large hierarchical table. For example, the categor=
ies table. Anytime the user changes a record in that table we need to recal=
culate the nested model for the entire categories and bulk update the resul=
ts.</div><div class=3D"gmail_default"><br></div><div class=3D"gmail_default=
">All of these scenarios are such that data sanitation is difficult if not =
possible before doing the bulk operation (copy).</div><div class=3D"gmail_d=
efault"><br></div><div class=3D"gmail_default">I realized that when we spec=
ify `on_error ignore` it just handles a handful of errors. I thought this w=
as a bug and sent an email to the pgsql-bugs maling list. But they said it&=
#39;s the intended behavior.</div><div class=3D"gmail_default"><br></div><d=
iv class=3D"gmail_default">Can you please provide a row-level catch-all han=
dler for the copy command?<br><br>Regards<br>Saeed</div></div></div>

--000000000000d4be81062f4970f9--