public inbox for [email protected]  

help / color / mirror / Atom feed

From: yudhi s <[email protected]>
To: pgsql-general <[email protected]>
Subject: Moving delta data faster
Date: Thu, 4 Apr 2024 02:08:36 +0530
Message-ID: <CAEzWdqcYGi0U5_cK1FVykx2-OZHmEUD8EZ_VE=kpoVaZKYWJeg@mail.gmail.com> (raw)

 Hi All,
 It's postgresql database version 15.4. We have a requirement in which we
will be initially moving full table data for 3-4 tables, from source
database to target(i.e. postgres) . Maximum number of rows will be
~10million rows in those tables. Then subsequently these rows will be
inserted/updated based on the delta number of rows that got
inserted/updated in the source database. In some cases these changed data
can flow multiple times per day to the downstream i.e. postgres database
and in other cases once daily.

 Want to understand , if we should use upsert(insert on conflict) or merge
statements or anything else in such a scenario so as to persist those delta
records faster in the target database, while making the system online to
the users?

Regards
Yudhi

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: [email protected]
  Cc: [email protected], [email protected]
  Subject: Re: Moving delta data faster
  In-Reply-To: <CAEzWdqcYGi0U5_cK1FVykx2-OZHmEUD8EZ_VE=kpoVaZKYWJeg@mail.gmail.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox