pg.ddx.io pgsql-bugs@postgresql.org mailing list archive
help / color / mirror / Atom feedBUG #17193: Issue with pg_output login
7+ messages / 5 participants
[nested] [flat]
* BUG #17193: Issue with pg_output login
@ 2021-09-14 13:44 PG Bug reporting form <noreply@postgresql.org>
2021-09-15 18:46 ` Re: BUG #17193: Issue with pg_output login Euler Taveira <euler@eulerto.com>
0 siblings, 1 reply; 7+ messages in thread
From: PG Bug reporting form @ 2021-09-14 13:44 UTC (permalink / raw)
To: pgsql-bugs@lists.postgresql.org; +Cc: sushant.kamboj@getfareye.com
The following bug has been logged on the website:
Bug reference: 17193
Logged by: DBA TEAM
Email address: sushant.kamboj@getfareye.com
PostgreSQL version: 9.6.23
Operating system: Ubuntu 20.04.2 LTS
Description:
We identified that there was an issue with the encoding with UTF8 characters
with pg_output plugin. The data was visible correctly in the database but
while pushing the data through the CDC pipeline there was some distortion
with the Greek letters.
Note:- We are pushing data through debezium.
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: BUG #17193: Issue with pg_output login
2021-09-14 13:44 BUG #17193: Issue with pg_output login PG Bug reporting form <noreply@postgresql.org>
@ 2021-09-15 18:46 ` Euler Taveira <euler@eulerto.com>
2021-09-15 18:53 ` Re: BUG #17193: Issue with pg_output login Sushant Kamboj <sushant.kamboj@getfareye.com>
2021-09-16 12:36 ` Re: BUG #17193: Issue with pg_output login Sushant Kamboj <sushant.kamboj@getfareye.com>
0 siblings, 2 replies; 7+ messages in thread
From: Euler Taveira @ 2021-09-15 18:46 UTC (permalink / raw)
To: sushant.kamboj@getfareye.com; pgsql-bugs@lists.postgresql.org
On Tue, Sep 14, 2021, at 10:44 AM, PG Bug reporting form wrote:
> We identified that there was an issue with the encoding with UTF8 characters
> with pg_output plugin. The data was visible correctly in the database but
> while pushing the data through the CDC pipeline there was some distortion
> with the Greek letters.
You don't provide enough information. What's your database encoding?
IIRC Debezium only supports database whose encoding is UTF-8. Since pgoutput
doesn't do encoding conversion and it uses binary output, it seems the issue is
in the Debezium side.
--
Euler Taveira
EDB https://www.enterprisedb.com/
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: BUG #17193: Issue with pg_output login
2021-09-14 13:44 BUG #17193: Issue with pg_output login PG Bug reporting form <noreply@postgresql.org>
2021-09-15 18:46 ` Re: BUG #17193: Issue with pg_output login Euler Taveira <euler@eulerto.com>
@ 2021-09-15 18:53 ` Sushant Kamboj <sushant.kamboj@getfareye.com>
1 sibling, 0 replies; 7+ messages in thread
From: Sushant Kamboj @ 2021-09-15 18:53 UTC (permalink / raw)
To: Euler Taveira <euler@eulerto.com>; +Cc: pgsql-bugs@lists.postgresql.org
Hi Euler,
Thanks for your reply.
I will check from the debezium side and will try to provide some more logs
over this case.
On Thu, Sep 16, 2021 at 12:17 AM Euler Taveira <euler@eulerto.com> wrote:
> On Tue, Sep 14, 2021, at 10:44 AM, PG Bug reporting form wrote:
>
> We identified that there was an issue with the encoding with UTF8
> characters
> with pg_output plugin. The data was visible correctly in the database but
> while pushing the data through the CDC pipeline there was some distortion
> with the Greek letters.
>
> You don't provide enough information. What's your database encoding?
>
> IIRC Debezium only supports database whose encoding is UTF-8. Since
> pgoutput
> doesn't do encoding conversion and it uses binary output, it seems the
> issue is
> in the Debezium side.
>
>
> --
> Euler Taveira
> EDB https://www.enterprisedb.com/
>
>
--
Thanks and Regards
Sushant Kamboj
Database Administrator
+91-9999518201
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: BUG #17193: Issue with pg_output login
2021-09-14 13:44 BUG #17193: Issue with pg_output login PG Bug reporting form <noreply@postgresql.org>
2021-09-15 18:46 ` Re: BUG #17193: Issue with pg_output login Euler Taveira <euler@eulerto.com>
@ 2021-09-16 12:36 ` Sushant Kamboj <sushant.kamboj@getfareye.com>
2021-09-21 20:46 ` Re: BUG #17193: Issue with pg_output login Euler Taveira <euler@eulerto.com>
1 sibling, 1 reply; 7+ messages in thread
From: Sushant Kamboj @ 2021-09-16 12:36 UTC (permalink / raw)
To: Euler Taveira <euler@eulerto.com>; pgsql-bugs@lists.postgresql.org; +Cc: Ashish Kumar <ashish.kumar2@getfareye.com>; Satyam Chaudhary <satyam.chaudhary@getfareye.com>
Hi Euler,
As we have verified in our environment ,your above statement looks like a
contradiction since this is working fine when we are using wal2json plugin
instead of pgoutput .
Since pgoutput is a native plugin, kindly let us know any settings or
parameter change that we can do to fix this on postgresql or debezium side.
FYI:We are using postgresql 12.5 RDS
On Thu, Sep 16, 2021 at 12:17 AM Euler Taveira <euler@eulerto.com> wrote:
> On Tue, Sep 14, 2021, at 10:44 AM, PG Bug reporting form wrote:
>
> We identified that there was an issue with the encoding with UTF8
> characters
> with pg_output plugin. The data was visible correctly in the database but
> while pushing the data through the CDC pipeline there was some distortion
> with the Greek letters.
>
> You don't provide enough information. What's your database encoding?
>
> IIRC Debezium only supports database whose encoding is UTF-8. Since
> pgoutput
> doesn't do encoding conversion and it uses binary output, it seems the
> issue is
> in the Debezium side.
>
>
> --
> Euler Taveira
> EDB https://www.enterprisedb.com/
>
>
--
Thanks and Regards
Sushant Kamboj
Database Administrator
+91-9999518201
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: BUG #17193: Issue with pg_output login
2021-09-14 13:44 BUG #17193: Issue with pg_output login PG Bug reporting form <noreply@postgresql.org>
2021-09-15 18:46 ` Re: BUG #17193: Issue with pg_output login Euler Taveira <euler@eulerto.com>
2021-09-16 12:36 ` Re: BUG #17193: Issue with pg_output login Sushant Kamboj <sushant.kamboj@getfareye.com>
@ 2021-09-21 20:46 ` Euler Taveira <euler@eulerto.com>
2021-09-21 21:09 ` Re: BUG #17193: Issue with pg_output login Gerdan Rezende dos Santos <gerdan@gmail.com>
0 siblings, 1 reply; 7+ messages in thread
From: Euler Taveira @ 2021-09-21 20:46 UTC (permalink / raw)
To: Sushant Kamboj <sushant.kamboj@getfareye.com>; pgsql-bugs@lists.postgresql.org; +Cc: Ashish Kumar <ashish.kumar2@getfareye.com>; Satyam Chaudhary <satyam.chaudhary@getfareye.com>
On Thu, Sep 16, 2021, at 9:36 AM, Sushant Kamboj wrote:
> As we have verified in our environment ,your above statement looks like a contradiction since this is working fine when we are using wal2json plugin instead of pgoutput .
The main difference is: wal2json uses text output and pgoutput uses binary
output. I'm not sure if it matters for Debezium.
> Since pgoutput is a native plugin, kindly let us know any settings or parameter change that we can do to fix this on postgresql or debezium side.
> FYI:We are using postgresql 12.5 RDS
You don't provide enough information for an investigation. You can start with a
reproducible test case.
--
Euler Taveira
EDB https://www.enterprisedb.com/
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: BUG #17193: Issue with pg_output login
2021-09-14 13:44 BUG #17193: Issue with pg_output login PG Bug reporting form <noreply@postgresql.org>
2021-09-15 18:46 ` Re: BUG #17193: Issue with pg_output login Euler Taveira <euler@eulerto.com>
2021-09-16 12:36 ` Re: BUG #17193: Issue with pg_output login Sushant Kamboj <sushant.kamboj@getfareye.com>
2021-09-21 20:46 ` Re: BUG #17193: Issue with pg_output login Euler Taveira <euler@eulerto.com>
@ 2021-09-21 21:09 ` Gerdan Rezende dos Santos <gerdan@gmail.com>
2021-09-22 06:59 ` Re: BUG #17193: Issue with pg_output login Ashish Kumar <ashish.kumar2@getfareye.com>
0 siblings, 1 reply; 7+ messages in thread
From: Gerdan Rezende dos Santos @ 2021-09-21 21:09 UTC (permalink / raw)
To: Euler Taveira <euler@eulerto.com>; +Cc: Sushant Kamboj <sushant.kamboj@getfareye.com>; pgsql-bugs@lists.postgresql.org, Ashish Kumar <ashish.kumar2@getfareye.com>; Satyam Chaudhary <satyam.chaudhary@getfareye.com>
On Tue, Sep 21, 2021 at 5:47 PM Euler Taveira <euler@eulerto.com> wrote:
> On Thu, Sep 16, 2021, at 9:36 AM, Sushant Kamboj wrote:
>
> As we have verified in our environment ,your above statement looks like a
> contradiction since this is working fine when we are using wal2json plugin
> instead of pgoutput .
>
> The main difference is: wal2json uses text output and pgoutput uses binary
> output. I'm not sure if it matters for Debezium.
>
> Since pgoutput is a native plugin, kindly let us know any settings or
> parameter change that we can do to fix this on postgresql or debezium side.
> FYI:We are using postgresql 12.5 RDS
>
> You don't provide enough information for an investigation. You can start
> with a
> reproducible test case.
>
>
> --
> Euler Taveira
> EDB https://www.enterprisedb.com/
>
>
At Debezium documentation at [Debezium connector for PostgreSQL ::
Debezium Documentation
<https://debezium.io/documentation/reference/connectors/postgresql.html;],
we can see:
*Debezium currently supports databases with UTF-8 character encoding only.
With a single byte character encoding, it is not possible to correctly
process strings that contain extended ASCII code characters.*
I believe this is not PostgreSQL BUG, but a *Debezium Limitation.*
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: BUG #17193: Issue with pg_output login
2021-09-14 13:44 BUG #17193: Issue with pg_output login PG Bug reporting form <noreply@postgresql.org>
2021-09-15 18:46 ` Re: BUG #17193: Issue with pg_output login Euler Taveira <euler@eulerto.com>
2021-09-16 12:36 ` Re: BUG #17193: Issue with pg_output login Sushant Kamboj <sushant.kamboj@getfareye.com>
2021-09-21 20:46 ` Re: BUG #17193: Issue with pg_output login Euler Taveira <euler@eulerto.com>
2021-09-21 21:09 ` Re: BUG #17193: Issue with pg_output login Gerdan Rezende dos Santos <gerdan@gmail.com>
@ 2021-09-22 06:59 ` Ashish Kumar <ashish.kumar2@getfareye.com>
0 siblings, 0 replies; 7+ messages in thread
From: Ashish Kumar @ 2021-09-22 06:59 UTC (permalink / raw)
To: Gerdan Rezende dos Santos <gerdan@gmail.com>; +Cc: Euler Taveira <euler@eulerto.com>; Sushant Kamboj <sushant.kamboj@getfareye.com>; pgsql-bugs@lists.postgresql.org, Satyam Chaudhary <satyam.chaudhary@getfareye.com>
Hi Gerdan,
The same scenario is working when we are using wal2json plugin instead of
pg output plugin. Kindly help us understanding if this is related to
pgoutput plugin ? If yes , then its a postgresql limitation or not and how
to solve this issue?
On Wed, Sep 22, 2021 at 2:39 AM Gerdan Rezende dos Santos <gerdan@gmail.com>
wrote:
>
>
>
>
>
> On Tue, Sep 21, 2021 at 5:47 PM Euler Taveira <euler@eulerto.com> wrote:
>
>> On Thu, Sep 16, 2021, at 9:36 AM, Sushant Kamboj wrote:
>>
>> As we have verified in our environment ,your above statement looks like a
>> contradiction since this is working fine when we are using wal2json plugin
>> instead of pgoutput .
>>
>> The main difference is: wal2json uses text output and pgoutput uses binary
>> output. I'm not sure if it matters for Debezium.
>>
>> Since pgoutput is a native plugin, kindly let us know any settings or
>> parameter change that we can do to fix this on postgresql or debezium side.
>> FYI:We are using postgresql 12.5 RDS
>>
>> You don't provide enough information for an investigation. You can start
>> with a
>> reproducible test case.
>>
>>
>> --
>> Euler Taveira
>> EDB https://www.enterprisedb.com/
>>
>>
> At Debezium documentation at [Debezium connector for PostgreSQL ::
> Debezium Documentation
> <https://debezium.io/documentation/reference/connectors/postgresql.html;],
> we can see:
>
> *Debezium currently supports databases with UTF-8 character encoding only.
> With a single byte character encoding, it is not possible to correctly
> process strings that contain extended ASCII code characters.*
>
> I believe this is not PostgreSQL BUG, but a *Debezium Limitation.*
>
--
*Regards,*
*Ashish *
*Sr. Database Administrator*
*+91-8699382848*
^ permalink raw reply [nested|flat] 7+ messages in thread
end of thread, other threads:[~2021-09-22 06:59 UTC | newest]
Thread overview: 7+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2021-09-14 13:44 BUG #17193: Issue with pg_output login PG Bug reporting form <noreply@postgresql.org>
2021-09-15 18:46 ` Euler Taveira <euler@eulerto.com>
2021-09-15 18:53 ` Sushant Kamboj <sushant.kamboj@getfareye.com>
2021-09-16 12:36 ` Sushant Kamboj <sushant.kamboj@getfareye.com>
2021-09-21 20:46 ` Euler Taveira <euler@eulerto.com>
2021-09-21 21:09 ` Gerdan Rezende dos Santos <gerdan@gmail.com>
2021-09-22 06:59 ` Ashish Kumar <ashish.kumar2@getfareye.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox