public inbox for [email protected]
help / color / mirror / Atom feedFrom: David Parenteau <[email protected]>
To: Paul van Rixel <[email protected]>
To: [email protected] <[email protected]>
Subject: Re: Data transfer from PG to SQL Server
Date: Wed, 9 Jun 2021 13:53:55 +0000
Message-ID: <YQXPR0101MB1301EA345FE85FA8929EE81CC2369@YQXPR0101MB1301.CANPRD01.PROD.OUTLOOK.COM> (raw)
In-Reply-To: <CALXfQ0=ER9SzpPFGE8H5Gm5DgmMT0c+WMmk9cWkHnrfO44ULCA@mail.gmail.com>
References: <CALXfQ0=ER9SzpPFGE8H5Gm5DgmMT0c+WMmk9cWkHnrfO44ULCA@mail.gmail.com>
Maybe you could use a Scheduled Powershell script to get your data using PostGres ODBC and save them in SQL Server using the Bulk Copy object? Otherwise, getting data to json/csv files and then BULK INSERT these file sinto SQL Server might also be wuick based on my experience.
Ex:
$bc = new-object ("System.Data.SqlClient.SqlBulkCopy") $cn
NOTE: On my side, to get my ODBC long calls to work I had to set the tcp timeout managed by pgbouncer to 120 instead of default.
HTH.
David
________________________________
From: Paul van Rixel <[email protected]>
Sent: Wednesday, June 9, 2021 8:48 AM
To: [email protected] <[email protected]>
Subject: Data transfer from PG to SQL Server
Hi all,
I'm looking for the best way to transfer data on a weekly basis to a SQL Server Instance which will be the staging area for BI reporting. A SQL Server Linked Server configuration is because of security issues not allowed so now I need some help!
What I did previously was installing the Postgres ODBC Driver 12_02 version, configuration is ok because a test is successful. So next step was to configure a linked server and in that way I can see the tables. To get the data from straigtforwarded tables without JSON or XML columns I used openquery from SQL Server. That works for 80% of the tables. I also see in the Postgresql logfiles that I'm connected and read that table.
Large tables (> 3GB) however did not transfer!
Also tried with Export/Import wizard from SQL Server Managememnt Studio but also from SSIS. Both failed.
So my simple question is how to transfer Postgresql data/tables to SQL Server without Linked Server or other paid tools like pgOledb? I could not find answer so that's why I'm here!
Any help would be appreciated!
Best regards,
Paul van Rixel
DBA
view thread (2+ messages)
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected], [email protected]
Subject: Re: Data transfer from PG to SQL Server
In-Reply-To: <YQXPR0101MB1301EA345FE85FA8929EE81CC2369@YQXPR0101MB1301.CANPRD01.PROD.OUTLOOK.COM>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox