pg.ddx.io  pgsql-bugs@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Matthias Apitz <guru@unixarea.de>
To: Michael Meskes <meskes@postgresql.org>
Cc: guru@unixarea.de
Cc: pgsql-bugs@lists.postgresql.org
Subject: Re: BUG #16200: returned data from ESQL/C FETCH is trampling outside assigned memory for CHAR column
Date: Fri, 10 Jan 2020 16:01:14 +0100
Message-ID: <20200110150114.GA3186@c720-r342378> (raw)
In-Reply-To: <c3b5c8c3375e5e4da53aae70b5152930dc3460e0.camel@postgresql.org>
References: <16200-358624daecff6820@postgresql.org>
	<c3b5c8c3375e5e4da53aae70b5152930dc3460e0.camel@postgresql.org>

El día viernes, enero 10, 2020 a las 01:43:38p. m. +0100, Michael Meskes escribió:

> Hi,
> 
> > We encounter the following problem with ESQL/C: Imagine a table with 
> 
> I assume you mean ECPG, right? ESQL/C would be the Informix compiler.

Hi.

I mean with ESQL/C the general term "Embedded SQL in C". Of course I do
know that for PostgreSQL the precompiler is named "ecpg".

> > In the database the CHAR column can contain not only 16 bytes, but 16
> > Unicode chars,
> > which are longer than 16 bytes if one or more of the chars is an UTF-
> > 8
> > multibyte
> > encoded char.
> > ...
> > the DATE. Now the function ECPGdo() places the DATE as "MM.DD.YYYY"
> > into the area pointed to for the 2nd argument and with this
> > overwrites
> > the NULL terminator of the string[17] element. Result is later a
> > SIGSEGV because the expected string in string[17] is not NULL
> > terminated anymore :-)
> > 
> > I would call it a bug, that ECPGdo() puts more than 17 bytes (16
> > bytes +
> > NULL) as return into the place pointed to by the host var pointer
> > when
> > the column in the database has more (UTF-8) chars as will fit into
> > 16+1 byte.
> 
> Actually I am not sure if this is a bug. I do not remember the standard
> asking for a null termination at the end of a partial string copy.

Neither do I know if there is some written standard, but char strings IMHO
should be NULL terminated. Our application expects in this example no
more than 16 bytes and that's why we provide a host variable as the
struct member of 17 bytes so the NULL fits into.

I digged into the sources of the ecpglib and this small fix solved our problem:

postgresql-11.4/src/interfaces/ecpg> diff ecpglib/data.c ecpglib/data.c.orig          
527,531c527
<                                                       /* guru@unixarea.de: strncpy() only varcharsize-1 */
<                                                       strncpy(str, pval, varcharsize-1);
<                                                       /* guru@unixarea.de: and terminate the string */
<                                                       str[varcharsize - 1] = '\0';
<                                                       ecpg_log("DEBUG ESQL/C: result [%s] len %d\n", str, strlen(str));
---
>                                                       strncpy(str, pval, varcharsize);

If I now store 16 UTF-8 chars German Umlaut 'ä' into the column, the ECPG log is this:

[30481] [10.01.2020 12:29:28:936]: ecpg_get_data on line 3181: RESULT: ääääääääääääääää offset: 1600; array: no
[30481] [10.01.2020 12:29:28:936]: DEBUG ESQL/C: varcharsize 17 size 32
[30481] [10.01.2020 12:29:28:936]: DEBUG ESQL/C: result [ääääääää] len 16

(The two lines with DEBUG are added by me in execute.c to understand
better the problem).

ECPG would return as RESULT as it says 'ääääääääääääääää' which is 32
bytes (16 UTF-8 chars), our varcharsize is only 17 and we now truncate the RESULT 
to 16 bytes (8 UTF-8 'ä') and a trailing \0 with my proposed change and all is fine.

Btw: truncating UTF-8 strings in any case brings the risk, that the resulting
string is not longer clean UTF-8 if you hit into the middle of a multibyte
char. But such issue should be handled by the application and not by ECPG.


> Please correct me if I am wrong. What it does ask for is setting the
> indicator accordingly. However, you do not mention any indicator, so I
> wonder if you checked that one at all. If the string is truncated and
> the appropriate error action is not taken, that would definitely
> qualify as a bug.
> 
> Could you please verify if the indicator is set accordingly? 

If I read the docs here https://www.postgresql.org/docs/11/ecpg-variables.html#ECPG-INDICATORS
indicator vars are meant to show null values in the table and not the
string truncation. Am I wrong? We do not use indicators at all, but
catch the error condition -213.

In any case, thanks for your attention.

	matthias

-- 
Matthias Apitz, ✉ guru@unixarea.de, http://www.unixarea.de/ +49-176-38902045
Public GnuPG key: http://www.unixarea.de/key.pub

"Glaube wenig, hinterfrage alles, denke selbst: Wie man Manipulationen durchschaut"
"Believe little, scrutinise all, think by your own: How see through manipulations"
ISBN-10: 386489218X

Attachments:

  signature.asc (application/pgp-signature, 832B) download

view thread (4+ messages)  latest in thread

Message-ID: <20200110150114.GA3186@c720-r342378>
Permalink:  ../20200110150114.GA3186@c720-r342378/
Also on:    postgresql.org/message-id/20200110150114.GA3186@c720-r342378

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-bugs@postgresql.org
  Cc: guru@unixarea.de, meskes@postgresql.org
  Subject: Re: BUG #16200: returned data from ESQL/C FETCH is trampling outside assigned memory for CHAR column
  In-Reply-To: <20200110150114.GA3186@c720-r342378>
  References: <16200-358624daecff6820@postgresql.org> <c3b5c8c3375e5e4da53aae70b5152930dc3460e0.camel@postgresql.org> <20200110150114.GA3186@c720-r342378>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox