public inbox for [email protected]
help / color / mirror / Atom feedFrom: Lok P <[email protected]>
To: Muhammad Usman Khan <[email protected]>
Cc: pgsql-general <[email protected]>
Subject: Re: Faster data load
Date: Fri, 6 Sep 2024 10:12:49 +0530
Message-ID: <CAKna9VYFp4Bqc8-9vyb_AbZH4xpXnVUsM_2papyw10WMpbjiOA@mail.gmail.com> (raw)
In-Reply-To: <CAPnRvGswoifDeosThKM5zkFZzph5u86ozYkrETpGD=oig14pOA@mail.gmail.com>
References: <CAKna9VaVsDzfOfOGu1+grStp9BBHFMKrH5DCEbbtGcQUWJ74KQ@mail.gmail.com>
<CAPnRvGswoifDeosThKM5zkFZzph5u86ozYkrETpGD=oig14pOA@mail.gmail.com>
On Fri, 6 Sept, 2024, 9:20 am Muhammad Usman Khan, <[email protected]>
wrote:
> Hi,
>
> You can use pg_partman. If your table is partitioned, you can manage
> partitions in parallel by distributing the load across partitions
> concurrently. Or you can use citus. It can be an excellent solution,
> especially for handling large data volumes and parallelizing data operations
>
Thank you.
The tables are partitioned. Also during index creation we are trying to do
it multiple partitions at same time from multiple sessions.But seeing out
of memory error in 5th or 6th session. And even each sessions taking 30mins
per partitions for index creation. Attach index partitions happening in
seconds though.
>
>
view thread (9+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected]
Subject: Re: Faster data load
In-Reply-To: <CAKna9VYFp4Bqc8-9vyb_AbZH4xpXnVUsM_2papyw10WMpbjiOA@mail.gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox