Subject: SQLGetConnectAttrW + SQL_ATTR_CURRENT_CATALOG => wrong byte count?

From: Marsupilami79 <[email-masked]>
Date: Tue, 22 Nov 2022 16:56:54 +0100

Hello,

I am a co author of a data access library and we recently added an ODBC
bridge. This bridge has the capability to detemine the current Catalog /
Database. This is done by calling SQLGetConnectAttrW.

We try to determine the size of the buffer that is needed for the
catalog name in the following manner:
SQLGetConnectAttrW(fHDBC, SQL_ATTR_CURRENT_CATALOG, null, 0, &aLen)

The ODBC driver for Microsoft SQL server correctly returns the number of
bytes required (10 bytes for the Database name "Stork") in the aLen
parameter. The ODBC driver for PostgreSQL returns the number of
characters (8 characters for a database named "topsales"), where it
should return 16 for the number of bytes required.

I tested this with the psqlodbc_13_02_0000-x86 download for Windows 10
and installed the Unicode ODBC driver.

I assume this is a bug and needs to be fixed. I just don't know if this
is the right place to report the bug to?

With best regards,

Jan

---

From: "Wal, Jan Tjalling van der" <[email-masked]>
Date: Tue, 22 Nov 2022 16:21:56 +0000

Hello Marsupilami79, Jan,

It could be that the answers you receive are in fact correct for postgres.
In a database set to charset=UTF-8 I get the following answers.

select  'topsales' as string, char_length('topsales'), length('topsales'), octet_length('topsales');
"string"		"char_length"	"length"	"octet_length"
"topsales"	8		8		8

However for a variation that requires more bytes to store the answer start to differ.
select  'töpsålés'as string, char_length('töpsålés'), length('töpsålés'), octet_length('töpsålés');
"string"		"char_length"	"length"	"octet_length"
"töpsålés"	8		8		11

In the above octet_length is a postgres-function that yields results in bytes. 
And the three characters with a diacritical added, each requires 2 bytes, yielding a resulting lengt of 11 instead of 8. 

Kind regards, Jan Tjalling van der Wal

-----Original Message-----
From: Marsupilami79 <[email-masked]> 
Sent: 22 November 2022 16:57
To: [email-masked]
Subject: SQLGetConnectAttrW + SQL_ATTR_CURRENT_CATALOG => wrong byte count?

Hello,

I am a co author of a data access library and we recently added an ODBC bridge. This bridge has the capability to detemine the current Catalog / Database. This is done by calling SQLGetConnectAttrW.

We try to determine the size of the buffer that is needed for the catalog name in the following manner:
SQLGetConnectAttrW(fHDBC, SQL_ATTR_CURRENT_CATALOG, null, 0, &aLen)

The ODBC driver for Microsoft SQL server correctly returns the number of bytes required (10 bytes for the Database name "Stork") in the aLen parameter. The ODBC driver for PostgreSQL returns the number of characters (8 characters for a database named "topsales"), where it should return 16 for the number of bytes required.

I tested this with the psqlodbc_13_02_0000-x86 download for Windows 10 and installed the Unicode ODBC driver.

I assume this is a bug and needs to be fixed. I just don't know if this is the right place to report the bug to?

With best regards,

Jan

---

From: Jon Raiford <[email-masked]>
Date: Fri, 25 Nov 2022 02:58:25 +0000

I believe the point is that the function is a "W" (wide char 16-bit) function so the strings should be UTF-16.

Jon
________________________________
From: Wal, Jan Tjalling van der <[email-masked]>
Sent: Tuesday, November 22, 2022 11:21:56 AM
To: Marsupilami79 <[email-masked]>; [email-masked] <[email-masked]>
Subject: RE: SQLGetConnectAttrW + SQL_ATTR_CURRENT_CATALOG => wrong byte count?

-----Original Message-----
From: Marsupilami79 <[email-masked]>
Sent: 22 November 2022 16:57
To: [email-masked]
Subject: SQLGetConnectAttrW + SQL_ATTR_CURRENT_CATALOG => wrong byte count?

Hello,

With best regards,

Jan

---

From: "Wal, Jan Tjalling van der" <[email-masked]>
Date: Fri, 25 Nov 2022 19:22:48 +0000

Okay, getting out of my comfort zone here.
I have found encoding options for PostgreSQL (v15) here:
PostgreSQL: Documentation: 15: 24.3. Character Set Support<https://www.postgresql.org/docs/current/multibyte.html>; they do not include UTF16, just UTF8.
There is no mention of UTF16 anywhere on that page.

I also found this: PostgreSQL: Re: DataDirect Driver, ExecDirect and UTF-8<https://www.postgresql.org/message-id/[email-masked]> that does mention WCHAR being different form CHAR and how that could work.

I hope there is something on these pages that helps you further.

Kind regards, Jan Tjalling

From: Jon Raiford <[email-masked]>
Sent: 25 November 2022 03:58
To: Wal, Jan Tjalling van der <[email-masked]>; Marsupilami79 <[email-masked]>; [email-masked]
Subject: Re: SQLGetConnectAttrW + SQL_ATTR_CURRENT_CATALOG => wrong byte count?

Jon
________________________________
From: Wal, Jan Tjalling van der <[email-masked]<mailto:[email-masked]>>
Sent: Tuesday, November 22, 2022 11:21:56 AM
To: Marsupilami79 <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]> <[email-masked]<mailto:[email-masked]>>
Subject: RE: SQLGetConnectAttrW + SQL_ATTR_CURRENT_CATALOG => wrong byte count?

-----Original Message-----
From: Marsupilami79 <[email-masked]<mailto:[email-masked]>>
Sent: 22 November 2022 16:57
To: [email-masked]<mailto:[email-masked]>
Subject: SQLGetConnectAttrW + SQL_ATTR_CURRENT_CATALOG => wrong byte count?

Hello,

With best regards,

Jan

---

From: Marsupilami79 <[email-masked]>
Date: Mon, 28 Nov 2022 12:16:59 +0100

Hello Jan Tjalling, hello Jon,

Jon is right - SQLGetConnectAttrW is a WideChar function (2 Bytes per
Character). So the encoding in the PG database should not matter. It
should return the length for the SQL_ATTR_CURRENT_CATALOG attribute in
bytes. So for a database named "topsales" it should return 16 because
each character uses two bytes.

I assume this is a bug in the PostgreSQL ODBC driver. The question is
where to file a bug report and how to get this fixed? Is there a chance
to get this fixed?

With best regards,

Jan Baumgarten

Am 25.11.2022 um 20:22 schrieb Wal, Jan Tjalling van der:
> PostgreSQL: Documentation: 15: 24.3. Character Set Support
> <https://www.postgresql.org/docs/current/multibyte.html>; they do not
> include UTF16, just UTF8.
>
> I also found this: PostgreSQL: Re: DataDirect Driver, ExecDirect and
> UTF-8
> <https://www.postgresql.org/message-id/[email-masked]>
> that does mention WCHAR being different form CHAR and how that could work.
>
> *From:*Jon Raiford <[email-masked]>
> *Sent:* 25 November 2022 03:58
> *To:* Wal, Jan Tjalling van der <[email-masked]>;
> Marsupilami79 <[email-masked]>; [email-masked]
> *Subject:* Re: SQLGetConnectAttrW + SQL_ATTR_CURRENT_CATALOG => wrong
> byte count?
>
> I believe the point is that the function is a "W" (wide char 16-bit)
> function so the strings should be UTF-16.
>
> Jon
>
> ------------------------------------------------------------------------
>
> *From:*Wal, Jan Tjalling van der <[email-masked]>
> *Sent:* Tuesday, November 22, 2022 11:21:56 AM
> *To:* Marsupilami79 <[email-masked]>;
> [email-masked] <[email-masked]>
> *Subject:* RE: SQLGetConnectAttrW + SQL_ATTR_CURRENT_CATALOG => wrong
> byte count?
>
> select  'topsales' as string, char_length('topsales'),
> length('topsales'), octet_length('topsales');
> select  'töpsålés'as string, char_length('töpsålés'),
> length('töpsålés'), octet_length('töpsålés');
> And the three characters with a diacritical added, each requires 2
> bytes, yielding a resulting lengt of 11 instead of 8.
>
> I am a co author of a data access library and we recently added an
> ODBC bridge. This bridge has the capability to detemine the current
> Catalog / Database. This is done by calling SQLGetConnectAttrW.
>
> The ODBC driver for Microsoft SQL server correctly returns the number
> of bytes required (10 bytes for the Database name "Stork") in the aLen
> I assume this is a bug and needs to be fixed. I just don't know if
> this is the right place to report the bug to?
>
> With best regards,
>
> Jan
>

---

From: Jon Raiford <[email-masked]>
Date: Mon, 28 Nov 2022 14:01:54 +0000

I don’t know if there is an official place to file a bug report for this issue.  I have seen several issues reported in this email list which were later fixed, so presumably this is an appropriate place to document the issue.

I did some research on this and found something interesting that may help.  If you look at the documentation for SQLGetConnectAttr() from Microsoft, they specifically mention the wide char format:

https://learn.microsoft.com/en-us/sql/odbc/reference/syntax/sqlgetconnectattr-function

Under the description for the BufferLength argument (describing the length of the buffer used to answer the attribute value):
“If the value in *ValuePtr is a Unicode string (when calling SQLGetConnectAttrW), the BufferLength argument must be an even number.”

The reason this is expected to be an even number is because all Unicode strings use 16-bit characters in Win32 wide char functions.

I hope this helps,

Jon

From: Marsupilami79 <[email-masked]>
Date: Monday, November 28, 2022 at 6:22 AM
To: Wal, Jan Tjalling van der <[email-masked]>, Jon Raiford <[email-masked]>, [email-masked] <[email-masked]>
Subject: Re: SQLGetConnectAttrW + SQL_ATTR_CURRENT_CATALOG => wrong byte count?
Hello Jan Tjalling, hello Jon,

Jon is right - SQLGetConnectAttrW is a WideChar function (2 Bytes per Character). So the encoding in the PG database should not matter. It should return the length for the SQL_ATTR_CURRENT_CATALOG attribute in bytes. So for a database named "topsales" it should return 16 because each character uses two bytes.

I assume this is a bug in the PostgreSQL ODBC driver. The question is where to file a bug report and how to get this fixed? Is there a chance to get this fixed?

With best regards,

Jan Baumgarten
PostgreSQL: Documentation: 15: 24.3. Character Set Support<https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fprotect-us.mimecast.com%2Fs%2Fa-zKC1wv1rHlZXDtLkZBF%3Fdomain%3Dpostgresql.org&data=05%7C01%7Craiford%40labware.com%7C050b022beed84021061f08dad132d1e5%7Cb5db03221aa04c0a859cad0f96966f4c%7C0%7C0%7C638052313459015632%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=uhFT6mwmKUuIL3J2Su%2BDh0B%2FiNnzAyXp13SmaQo89eg%3D&reserved=0>; they do not include UTF16, just UTF8.

I also found this: PostgreSQL: Re: DataDirect Driver, ExecDirect and UTF-8<https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fprotect-us.mimecast.com%2Fs%2FAAWRC2kw4vUgmQJc1gEOy%3Fdomain%3Dpostgresql.org&data=05%7C01%7Craiford%40labware.com%7C050b022beed84021061f08dad132d1e5%7Cb5db03221aa04c0a859cad0f96966f4c%7C0%7C0%7C638052313459015632%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=0SfOO6NczbbCACRcxjoKxeHLmq%2BVLtJJjedWxGyKr7w%3D&reserved=0> that does mention WCHAR being different form CHAR and how that could work.

From: Jon Raiford <[email-masked]><mailto:[email-masked]>
Sent: 25 November 2022 03:58
To: Wal, Jan Tjalling van der <[email-masked]><mailto:[email-masked]>; Marsupilami79 <[email-masked]><mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>
Subject: Re: SQLGetConnectAttrW + SQL_ATTR_CURRENT_CATALOG => wrong byte count?

Jon

-----Original Message-----
From: Marsupilami79 <[email-masked]<mailto:[email-masked]>>
Sent: 22 November 2022 16:57
To: [email-masked]<mailto:[email-masked]>
Subject: SQLGetConnectAttrW + SQL_ATTR_CURRENT_CATALOG => wrong byte count?

Hello,

With best regards,

Jan

---

From: "Inoue,Hiroshi" <[email-masked]>
Date: Wed, 30 Nov 2022 20:19:52 +0900

Hi Jan,

The psqlodbc unicode driver correctly returns the number of bytes(10 bytes
for a database name "visco")  here.

regards,
Hiroshi Inoue