pg.ddx.io pgsql-performance@postgresql.org mailing list archivehelp / color / mirror / Atom feed
copy csv into partitioned table with unique index 2+ messages / 2 participants [nested] [flat]
* copy csv into partitioned table with unique index @ 2018-01-28 09:11 Mariel Cherkassky <mariel.cherkassky@gmail.com> 0 siblings, 1 reply; 2+ messages in thread From: Mariel Cherkassky @ 2018-01-28 09:11 UTC (permalink / raw) To: pgsql-performance Hi, I configured a master table that is called "year_2018" : create table year_2018(a int,b int c date); The master table has a unique constraint on those 3 columns so that I wont have any duplicated rows. Moreover, I configured a before insert trigger on that table that creates a child table for each day in the year. The child should include all the data related to that specific day. Now, every day I got a csv file that I need to load to the year table. I must load the data as fast as possible but I have 2 problems : 1)I must load the data as a bulk - via the copy command. However, the copy command fails because sometimes I have duplicated rows. 2)I tried to use the pgloader extension but it fails because I have a trigger before each insert. -I cant load all the data into a temp table and then run insert into year_2018 select * from temp because it takes too much time. Any idea ? ^ permalink raw reply [nested|flat] 2+ messages in thread
* Re: copy csv into partitioned table with unique index @ 2018-01-28 10:55 legrand legrand <legrand_legrand@hotmail.com> parent: Mariel Cherkassky <mariel.cherkassky@gmail.com> 0 siblings, 0 replies; 2+ messages in thread From: legrand legrand @ 2018-01-28 10:55 UTC (permalink / raw) To: pgsql-performance Did you try to transform your temp table into a table partition using the ALTER TABLE ATTACH syntax https://www.postgresql.org/docs/10/static/ddl-partitioning.html Regards PAscal -- Sent from: http://www.postgresql-archive.org/PostgreSQL-performance-f2050081.html ^ permalink raw reply [nested|flat] 2+ messages in thread
end of thread, other threads:[~2018-01-28 10:55 UTC | newest] Thread overview: 2+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2018-01-28 09:11 copy csv into partitioned table with unique index Mariel Cherkassky <mariel.cherkassky@gmail.com> 2018-01-28 10:55 ` legrand legrand <legrand_legrand@hotmail.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions for how to clone and mirror all data and code used for this inbox