MIME-Version: 1.0
References: <CAHER7LrDGLmmB6SGiQh7prAJ4pQy7QczoPa2EcJ4zkFTTTZ61A@mail.gmail.com>
 <CANzqJaCb=aEycqm7hQEUc-NAeNVqDdGAeetTOmG4qWOTx7GgkQ@mail.gmail.com> <CAGeimVqZ0=n+yPc6O67+08QBS1Gjpz_U7enJAK5ADPQ7pWxaAw@mail.gmail.com>
In-Reply-To: <CAGeimVqZ0=n+yPc6O67+08QBS1Gjpz_U7enJAK5ADPQ7pWxaAw@mail.gmail.com>
From: Avinash Vallarapu <avinash.vallarapu@gmail.com>
Date: Fri, 3 May 2024 05:38:31 -0400
Message-ID: <CAN0TujeJgfuMByPcMy+H_w1ZuwUpi7djnnwyYtMA2Cvp_gpgQg@mail.gmail.com>
Subject: Re: Ora2pg Delta Migration: Oracle to PostgreSQL
To: Muhammad Ikram <mmikram@gmail.com>, Amit Sharma <amitpgsql@gmail.com>
Cc: Ron Johnson <ronljohnsonjr@gmail.com>, 
	PostgreSQL General <pgsql-general@lists.postgresql.org>
Content-Type: multipart/alternative; boundary="0000000000005546950617897c74"
Archived-At: <https://www.postgresql.org/message-id/CAN0TujeJgfuMByPcMy%2BH_w1ZuwUpi7djnnwyYtMA2Cvp_gpgQg%40mail.gmail.com>
Precedence: bulk

--0000000000005546950617897c74
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi,

Has anyone tried delta/incremental data migration for Oracle to PostgreSQL
> using Ora2pg? Or what are the best options to run delta migration for
> Oracle to PostgreSQL?


There are a few ways of dealing with it when using Ora2Pg.

*Option 1 : *Ora2Pg configuration file allows us to pass the
conditions(predicates) to be applied to a table while migrating the data
from a table. So, if there is a way to filter the data since the last full
load, you can pass the condition to the configuration file, for each table.
This is some work for sure but it works great if you are able to identify
those conditions such as creation_date or last_updated_date, let's say.

*Option2 :* Ora2Pg is currently CDC ready. What this means is that, Ora2Pg
can provide the SCN at which the table copy has been initiated.
Technically, we provide the SCN to the CDC tools or custom CDC techniques
to continue replication since that SCN.

*Option 2.1 :* Leverage Debezium, an Open Source solution to perform
continuous replication from Oracle to PostgreSQL from the SCN produced by
Ora2Pg for that table.

So, it is all about Ora2Pg + Debezium

It might be initially challenging, but you will get there.

Regards,
Avi Vallarapu,
CEO,
HexaCluster Corp.


On Fri, May 3, 2024 at 3:25=E2=80=AFAM Muhammad Ikram <mmikram@gmail.com> w=
rote:

> Hi,
>
> Not related to Ora2PG but in the past I have used the EDB Migration
> toolkit for such scenarios. Filterprops option can help in specifying
> filter (where clause).
>
> Regards,
> Ikram
>
>
> On Fri, May 3, 2024 at 8:12=E2=80=AFAM Ron Johnson <ronljohnsonjr@gmail.c=
om>
> wrote:
>
>> On Thu, May 2, 2024 at 8:28=E2=80=AFPM Amit Sharma <amitpgsql@gmail.com>=
 wrote:
>>
>>> Hello,
>>>
>>> Has anyone tried delta/incremental data migration for Oracle to
>>> PostgreSQL using Ora2pg? Or what are the best options to run delta
>>> migration for Oracle to PostgreSQL?
>>>
>>
>> What do the ora2pg docs say about whether or not that feature is
>> implemented?  (It wasn't when I last used it in 2022.)
>>
>>
>
>
> --
> Muhammad Ikram
>
>

--0000000000005546950617897c74
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div dir=3D"ltr"><div>Hi,</div><div><br></div><blockquote =
class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px sol=
id rgb(204,204,204);padding-left:1ex">Has anyone tried delta/incremental da=
ta migration for Oracle to PostgreSQL using Ora2pg? Or what are the best op=
tions to run delta migration for Oracle to PostgreSQL?</blockquote><div><br=
></div><div>There are a few ways of dealing with it when using Ora2Pg.</div=
><div><br></div><div><b>Option 1 : </b>Ora2Pg configuration file allows us =
to pass the conditions(predicates) to be=C2=A0applied to a table while migr=
ating the data from a table. So, if there is a way to filter the data since=
 the last full load, you can pass the condition to the configuration file, =
for each table. This is some work for sure but it works great if you are ab=
le to identify those conditions such=C2=A0as creation_date or last_updated_=
date, let&#39;s say.</div><div><br></div><div><b>Option2 :</b> Ora2Pg is cu=
rrently CDC ready. What this means is that, Ora2Pg can provide the SCN at w=
hich the table copy has been initiated. Technically, we provide the SCN to =
the CDC tools or custom CDC techniques to continue replication since that S=
CN.</div><div><b><br></b></div><div><b>Option 2.1 :</b> Leverage Debezium, =
an Open Source solution to perform continuous replication from Oracle to Po=
stgreSQL from the SCN produced by Ora2Pg for that table.</div><div><br></di=
v><div>So, it is all about Ora2Pg=C2=A0+ Debezium</div><div><br></div><div>=
It might be initially challenging, but you will get there.=C2=A0</div><div>=
<br></div><div>Regards,</div><div>Avi Vallarapu,</div><div>CEO,=C2=A0</div>=
<div>HexaCluster Corp.=C2=A0</div><div><br></div><div><br></div><div>=C2=A0=
</div><div><br></div><div><br></div></div><br><div class=3D"gmail_quote"><d=
iv dir=3D"ltr" class=3D"gmail_attr">On Fri, May 3, 2024 at 3:25=E2=80=AFAM =
Muhammad Ikram &lt;<a href=3D"mailto:mmikram@gmail.com">mmikram@gmail.com</=
a>&gt; wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0p=
x 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><d=
iv dir=3D"ltr">Hi,<div><br></div><div>Not related to Ora2PG but in the past=
 I have used the EDB Migration toolkit for such scenarios. Filterprops opti=
on can help in specifying filter (where clause).</div><div><br></div><div>R=
egards,</div><div>Ikram</div><div><br></div></div><br><div class=3D"gmail_q=
uote"><div dir=3D"ltr" class=3D"gmail_attr">On Fri, May 3, 2024 at 8:12=E2=
=80=AFAM Ron Johnson &lt;<a href=3D"mailto:ronljohnsonjr@gmail.com" target=
=3D"_blank">ronljohnsonjr@gmail.com</a>&gt; wrote:<br></div><blockquote cla=
ss=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid =
rgb(204,204,204);padding-left:1ex"><div dir=3D"ltr"><div dir=3D"ltr">On Thu=
, May 2, 2024 at 8:28=E2=80=AFPM Amit Sharma &lt;<a href=3D"mailto:amitpgsq=
l@gmail.com" target=3D"_blank">amitpgsql@gmail.com</a>&gt; wrote:<br></div>=
<div class=3D"gmail_quote"><blockquote class=3D"gmail_quote" style=3D"margi=
n:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex=
"><div dir=3D"ltr">Hello,<div><br></div><div>Has anyone tried delta/increme=
ntal data migration for Oracle to PostgreSQL using Ora2pg? Or what are the =
best options to run delta migration for Oracle to PostgreSQL?</div></div></=
blockquote><div><br></div><div>What do the ora2pg docs say about whether or=
 not that feature is implemented?=C2=A0 (It wasn&#39;t when I last used it =
in 2022.)</div><div>=C2=A0<br></div></div></div>
</blockquote></div><br clear=3D"all"><div><br></div><span class=3D"gmail_si=
gnature_prefix">-- </span><br><div dir=3D"ltr" class=3D"gmail_signature"><d=
iv dir=3D"ltr"><div>Muhammad Ikram<br><br></div></div></div>
</blockquote></div><div><br></div></div>

--0000000000005546950617897c74--