Subject: Please implement a catch-all error handler per row, for COPY

From: me nefcanto <[email-masked]>
Date: Sat, 1 Mar 2025 18:48:41 +0330

Hello

Please consider these scenarios:

- I want to create a million fake products, sometimes even 100 million
(we're on MariaDB now and we plan to migrate to Postgres). My team uses
fake data for performance tests and other use cases.
- Another scenario is translations. Even in production, we have translation
files for more than 20 languages, and for more than 2 thousand keys. That
means we need to insert 40 thousand translation records in the production.
- Another scenario is updating nested model values for a large hierarchical
table. For example, the categories table. Anytime the user changes a record
in that table we need to recalculate the nested model for the entire
categories and bulk update the results.

All of these scenarios are such that data sanitation is difficult if not
possible before doing the bulk operation (copy).

I realized that when we specify `on_error ignore` it just handles a handful
of errors. I thought this was a bug and sent an email to the pgsql-bugs
maling list. But they said it's the intended behavior.

Can you please provide a row-level catch-all handler for the copy command?

Regards
Saeed

---

From: Adrian Klaver <[email-masked]>
Date: Sat, 1 Mar 2025 08:31:08 -0800

On 3/1/25 07:18, me nefcanto wrote:
> I realized that when we specify `on_error ignore` it just handles a 
> handful of errors. I thought this was a bug and sent an email to the 
> pgsql-bugs maling list. But they said it's the intended behavior.
> 

Take a look at:

http://ossc-db.github.io/pg_bulkload/index.html

and

https://pgloader.io/

They may be able to meet your needs.

-- 
Adrian Klaver
[email-masked]