public inbox for [email protected]
help / color / mirror / Atom feedFrom: mahamood hussain <[email protected]>
To: Brian Crockard <[email protected]>
Cc: Pgsql-admin <[email protected]>
Subject: Re: Guidance Requested: Migrating Large-Scale DB2 Databases to PostgreSQL
Date: Sat, 18 Oct 2025 01:19:38 +0530
Message-ID: <CAGc_7HmsiEpekifed=xWpx0Cn-c8=GhOfjxai=aEFA-GSK+WDg@mail.gmail.com> (raw)
In-Reply-To: <[email protected]>
References: <CAGc_7HnBQZva+cnnm_=_pWSmrS6v217FPW7b8DC43RKXi9jqvQ@mail.gmail.com>
<[email protected]>
Hi Brian, Ron, Kurtoglu,
Thank you all for the thoughtful responses. It took a bit of effort to pull
this information together, but I'm glad to see the insights coming in —
it's given me more confidence that this migration is indeed achievable with
the right approach.
To clarify our setup:
We're currently running DB2 LUW v11.5.9 (Advanced Enterprise Server
Edition). One of our key concerns is licensing cost, which is driving our
move to PostgreSQL.
A few follow-up questions based on your responses:
1.
InfoSphere Data Replication (CDC):
We understand this is a strong option, but it likely incurs additional
licensing costs. We’re exploring Fivetran for data replication between DB2
and PostgreSQL, with the flexibility to fall back to DB2 if performance or
stability issues arise in production.
-
Can CDC be used for such a failback scenario (i.e., from PostgreSQL →
DB2) similar to how Fivetran supports bidirectional sync?
2.
Open-source Tools:
Do we have any robust open-source tools that support both forward and
backward replication between DB2 and PostgreSQL? This would help us plan
for both migration and rollback scenarios without heavy vendor lock-in.
3.
Pitfalls to Avoid During Migration:
Since our primary goal is cost reduction, we would ideally avoid needing
to fall back to DB2. Based on your experience, what are some common
pitfalls or gotchas we should look out for when moving from DB2 to
PostgreSQL?
4.
PostgreSQL Performance Tuning & Config:
If there’s a list of standard PostgreSQL parameters that you'd
recommend tweaking right after installation — especially for large datasets
— that would be very helpful. We're anticipating around 100+ concurrent
connections during peak hours and migrating some very large tables (up to
80B rows).
5.
Community vs Enterprise Support:
In production, having support with a guaranteed response time (e.g.,
under an hour) is crucial.
-
Does the PostgreSQL community offer any premium or paid support
options with SLAs, or would we need to go with providers like
EDB for this
level of assurance?
------------------------------
On Thu, Oct 16, 2025 at 6:15 PM Brian Crockard <[email protected]> wrote:
> You may want to look into an IBM product called InfoSphere Data
> Replciation (CDC)
>
>
> https://www.ibm.com/docs/nl/idr/11.4.0?topic=requirements-supported-source-targets
>
> It will replicate data from DB2 to PostgreSQL. You can set it up to
> actively replciate the data and keep it consistent. Then perform a cutover
> at some point to the new system running against the new replicated
> database. You will then have all of your historical data in the new system.
> We used this on an extremely active system and it hard very few issues.
>
> On Wednesday, October 15, 2025 at 05:15:00 PM EDT, mahamood hussain <
> [email protected]> wrote:
>
>
> Hi Team,
>
> We are in the process of migrating several DB2 databases to PostgreSQL,
> primarily to reduce the high licensing costs associated with DB2. These
> databases support retail applications (e.g., supermarkets and stores), and
> during peak hours, we anticipate over 100 concurrent connections.
> ------------------------------
> Current Database Profile:
>
> -
>
> Approximately 3,000 tables in total
> -
>
> Around 100 tables contain active data
> -
>
> Most tables have low data volume
> -
>
> A few large tables range from 10 GB to 2 TB
> -
>
> The largest table contains approximately 80 billion rows
>
> ------------------------------
> Migration Approach:
>
> -
>
> We are using Ispirer for code conversion (DB2 to PostgreSQL).
> -
>
> For data migration, we are evaluating Fivetran, but noted that it
> relies on the COPY method for data loading.
>
> ------------------------------
> Questions & Areas Where We Need Guidance:
>
> 1.
>
> Is Fivetran a suitable option for migrating very large datasets (e.g.,
> tables with 80+ billion rows)?
> 2.
>
> Are there any reliable open-source tools for DB2 to PostgreSQL data
> migration that we can use internally, without needing to invest in a tool
> like Fivetran?
> 3.
>
> Are there more scalable or efficient alternatives for both the initial
> load and ongoing/incremental sync to PostgreSQL?
>
> ------------------------------
> Additional Input Requested:
>
> -
>
> What are the key best practices (Do’s and Don’ts) to keep in mind
> during a large-scale DB2 → PostgreSQL migration?
> -
>
> Are there specific PostgreSQL settings or configurations we should pay
> attention to for optimizing performance, especially for large datasets and
> DB2-style workloads?
>
> ------------------------------
>
> We are keen to ensure performance, data integrity, and scalability
> throughout this migration. Any insights—particularly from those with
> experience in similar large-scale PostgreSQL implementations—would be
> highly appreciated.
>
> If this is not the right forum for these questions, please do let me know
> if there is a better place to seek this guidance.
>
> Thanks in advance for your support!
>
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected]
Subject: Re: Guidance Requested: Migrating Large-Scale DB2 Databases to PostgreSQL
In-Reply-To: <CAGc_7HmsiEpekifed=xWpx0Cn-c8=GhOfjxai=aEFA-GSK+WDg@mail.gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox