Subject: Incremental load from SQL server DB to postgressql

From: Sabyasachi Mukherjee <[email-masked]>
Date: Thu, 5 Sep 2024 13:54:31 +0000

Hello All,

I have to implement an incremental load from a SQL server DB to a postgresql DB. I would prefer it to be pull from postgres.
SSIS is not the preferred option. Can you please advise about the possibilities?

Sabyasachi Mukherjee

---

From: Holger Jakobs <[email-masked]>
Date: Thu, 05 Sep 2024 16:01:36 +0200

You could try to use BCP.exe for exporting from MS SQL to a flat file per table. Load into PG SQL Server with COPY or \copy (psql).

Using the TSQL foreign data wrapper to pull data over is another option. This FDW isn't very well maintained, though.

Regards 
Holger

-- 
Holger Jakobs, Bergisch Gladbach 
Tel. +49 178 9759012

---

From: Wasim Devale <[email-masked]>
Date: Thu, 5 Sep 2024 19:41:28 +0530

ADF Pipeline would be great for your need.

Thanks,
Wasim

On Thu, 5 Sept, 2024, 7:24 pm Sabyasachi Mukherjee, <
[email-masked]> wrote:

> Hello All,
>
> I have to implement an incremental load from a SQL server DB to a
> postgresql DB. I would prefer it to be pull from postgres.

---

From: Bo Victor Thomsen <[email-masked]>
Date: Thu, 5 Sep 2024 16:56:05 +0200

You could use the the /*ogc_fdw*/ foreign data wrapper.

It's a data wrapper for accessing spatial data using the GDAL library. 
But it works just fine with non-spatial data too, including 
MS-SqlServer. And it's well maintained.

Med venlig hilsen / Best regards

Bo Victor Thomsen

On 05-09-2024 16:01, Holger Jakobs wrote:
> You could try to use BCP.exe for exporting from MS SQL to a flat file 
> per table. Load into PG SQL Server with COPY or \copy (psql).
>
> Using the TSQL foreign data wrapper to pull data over is another 
> option. This FDW isn't very well maintained, though.
>
> Regards
> Holger
>
> --

---

From: Muhammad Usman Khan <[email-masked]>
Date: Fri, 6 Sep 2024 08:44:20 +0500

Hi,

You can use  pgloader. It is a powerful tool that can migrate data from SQL
Server to PostgreSQL efficiently, and it supports incremental loads or the
other option is to use file_fdw, it allows PostgreSQL to directly query
flat files  as if they were tables, without importing the data into
PostgreSQL. It can be useful if you can periodically export data from SQL
Server into flat files and then query them from PostgreSQL.

---

From: <[email-masked]>
Date: Sun, 8 Sep 2024 18:01:33 -0400

Option  use ELT tool , if PostgreSQL in Azure use Azure Data Factory 

Option 2 , write SQL statements to extract data in PostgreSQL format , load
into PostgreSQL using \copy  command

If AWS use Schema Conversion Tool (SCT)  along with Database Migration
Services (DMS)

From: Sabyasachi Mukherjee <[email-masked]> 
Sent: Thursday, September 5, 2024 9:55 AM
To: [email-masked]
Subject: Incremental load from SQL server DB to postgressql

Hello All,

I have to implement an incremental load from a SQL server DB to a postgresql
DB. I would prefer it to be pull from postgres.

SSIS is not the preferred option. Can you please advise about the
possibilities?

Sabyasachi Mukherjee