Subject: Re: More thoughts about FE/BE protocol
From: Hannu Krosing <hannu@tm.ee>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Peter Galbavy <peter.galbavy@knowtion.net>,
	pgsql-hackers@postgreSQL.org, pgsql-interfaces@postgreSQL.org
In-Reply-To: <3625.1049983061@sss.pgh.pa.us>
References: <10478.1049927451@sss.pgh.pa.us>
	<003f01c2ff40$9f1a0e70$7c28a8c0@cblan.mblox.com>
	<3625.1049983061@sss.pgh.pa.us>
Content-Type: text/plain
Content-Transfer-Encoding: 7bit
Organization: 
Message-Id: <1049993435.1886.24.camel@fuji.krosing.net>
Mime-Version: 1.0
Date: 10 Apr 2003 19:50:35 +0300

Tom Lane kirjutas N, 10.04.2003 kell 16:57:
> "Peter Galbavy" <peter.galbavy@knowtion.net> writes:
> > Is there any message - speaking from a standpoint of a normal user and not a
> > source hacker WRT postgresql - where knowing the length of the response is
> > either unknown or is expensive (in buffering) to find out ?
> 
> See my response to ljb --- I think that in practice people assemble each
> message before sending anyway. 

I just tested it by running "select *" on 68M records (6.5 GB data)
table and you seem to be wrong - while psql shows nothing, its size
starts rapidly growing (I ^C it at ~500M) , while backend stays at
stable 32M, which indicates that postgres starts to push data out as
fast as it can get it.

> If you don't do it that way, you've got
> a problem with recovering if you hit an error condition after sending a
> partial message.

If you hit an error condition after sending a partial message then I'm
in trouble anyway. Assembling the message beforehand just makes hitting
error less likely.

I would propose something like X11 protocol (from memory)

each request (query) has a serial number.

eash response is sent in resonable sized chunks, and each chunk has a
serial number of request, chunk number of response and an indicator for
EndOfResponse (or perhaps to for Completed or Aborted).

This would make resyncing easier as you can safely ignore responsed you
dont wan't to see anymore even if for some reasons some of them are
still in pipeline.

What could be useful (and is often requested) is sending the number of
records in response *if it is known without extra work*.

Also there should be a way to tell the backend not to send some types of
notices/warnings.

-----------------
Hannu