Subject: Table copy

From: Andy Hartman <[email-masked]>
Date: Tue, 4 Feb 2025 14:51:47 -0500

I'm copying a large table from mssql to PG using SImplysql and in the Log I
see this message

2025-02-04 14:42:17.975 EST [4796] ERROR:  unexpected EOF on client
connection with an open transaction
2025-02-04 14:42:17.975 EST [4796] CONTEXT:  COPY sqlt_data_1_2022_03, line
208274199, column tagid
2025-02-04 14:42:17.975 EST [4796] STATEMENT:  COPY sqlt_data_1_2022_03
(tagid, intvalue, floatvalue, stringvalue, datevalue, dataintegrity,
t_stamp) FROM STDIN (FORMAT BINARY)
2025-02-04 14:42:18.158 EST [4796] FATAL:  terminating connection because
protocol synchronization was lost
2025-02-04 14:42:18.265 EST [4796] LOG:  could not send data to client: An
established connection was aborted by the software in your host machine.

it then dies with a data stream error.I have copied other tables on this
size with no problems.

 THanks.

---

From: Adrian Klaver <[email-masked]>
Date: Tue, 4 Feb 2025 12:02:19 -0800

On 2/4/25 11:51 AM, Andy Hartman wrote:
> I'm copying a large table from mssql to PG using SImplysql and in the 
> Log I see this message
> 

The above pretty much spells it out.

Something messed with the connection from the client.

You need to look at Postgres and system logs to see if you can track 
down what?

FYI, more information would be helpful:

1) MS SQL version.

2) Postgres version.

3) SimplySQL version.

4) The complete copy command.

5) The amount of data being transferred.

6) The OS'es and their versions on both ends.

> 2025-02-04 14:42:17.975 EST [4796] CONTEXT:  COPY sqlt_data_1_2022_03, 
> line 208274199, column tagid
> 2025-02-04 14:42:18.158 EST [4796] FATAL:  terminating connection 
> because protocol synchronization was lost
> 2025-02-04 14:42:18.265 EST [4796] LOG:  could not send data to client: 
> An established connection was aborted by the software in your host machine.

-- 
Adrian Klaver
[email-masked]

---

From: Andy Hartman <[email-masked]>
Date: Wed, 5 Feb 2025 08:40:00 -0500

mssql - 2016
PG - 16.1
latest release for Simplysql
Invoke-SqlBulkCopy -SourceConnectionName "src" -SourceTable
"sqlt_data_1_2022_03" -DestinationConnectionName "dst" -DestinationTable
"sqlt_data_1_2022_03" -BatchSize 10000 -Notify

2.4 billion records  -- I have down other tables of same size no problems

mssql OS Windows Server 2019
PG OS Windows Server 2022

Table:

---

From: Adrian Klaver <[email-masked]>
Date: Wed, 5 Feb 2025 08:15:14 -0800

> latest release for Simplysql
Latest is a relative term and requires anyone in the future coming 
across this thread to work out what you are talking about. For that 
person the current version is 2.1.0.

> 2.4 billion records  -- I have down other tables of same size no problems

Same version of SimplySql?

Same source and destination databases?

Did you look at the Postgres and system logs to see if there was 
relevant information?

> image.png

---

From: Andy Hartman <[email-masked]>
Date: Wed, 5 Feb 2025 12:46:35 -0500

[6992] ERROR:  unexpected EOF on client connection with an open transaction
2025-02-05 12:19:44.919 EST [6992] CONTEXT:  COPY sqlt_data_1_2022_03, line
24431524, column dataintegrity
2025-02-05 12:19:44.919 EST [6992] STATEMENT:  COPY sqlt_data_1_2022_03
(tagid, intvalue, floatvalue, stringvalue, datevalue, dataintegrity,
t_stamp) FROM STDIN (FORMAT BINARY)
2025-02-05 12:19:44.919 EST [6992] FATAL:  terminating connection because
protocol synchronization was lost