pg.ddx.io  pgsql-hackers@postgresql.org mailing list archive  

help / color / mirror / Atom feed
From: Heikki Linnakangas <heikki.linnakangas@enterprisedb.com>
To: Joachim Wieland <joe@mcknight.de>
Cc: Tom Lane <tgl@sss.pgh.pa.us>
Cc: José Arthur Benetasso Villanova <jose.arthur@gmail.com>
Cc: pgsql-hackers@postgresql.org
Subject: Re: directory archive format for pg_dump
Date: Wed, 01 Dec 2010 16:03:15 +0200
Message-ID: <4CF655A3.4090308@enterprisedb.com> (raw)
In-Reply-To: <4CF40B5F.1010407@enterprisedb.com>
References: <AANLkTi=AEi3F6jC0Zy3dG9njiwJJC8Y6V090-hMonyyn@mail.gmail.com>
	<AANLkTik+PR2HCsqS7PVAdETPyKSGn2hb-eTeg9QZpRew@mail.gmail.com>
	<4CEA9F29.5060106@enterprisedb.com>
	<28890.1290445623@sss.pgh.pa.us>
	<4CEAD61A.6060207@enterprisedb.com>
	<AANLkTi=neaBqvLa2ZnDpuAMSen3P4ZHE8__Q7L2pmjD0@mail.gmail.com>
	<4CF40B5F.1010407@enterprisedb.com>

On 29.11.2010 22:21, Heikki Linnakangas wrote:
> On 29.11.2010 07:11, Joachim Wieland wrote:
>> On Mon, Nov 22, 2010 at 3:44 PM, Heikki Linnakangas
>> <heikki.linnakangas@enterprisedb.com> wrote:
>>> * wrap long lines
>>> * use extern in function prototypes in header files
>>> * "inline" some functions like _StartDataCompressor, _EndDataCompressor,
>>> _DoInflate/_DoDeflate that aren't doing anything but call some other
>>> function.
>>
>> So here is a new round of patches. It turned out that the feature to
>> allow to also restore files from a different dump and with a different
>> compression required some changes in the compressor API. And in the
>> end I didn't like all the #ifdefs either and made a less #ifdef-rich
>> version using function pointers.
>
> Ok. The separate InitCompressorState() and AllocateCompressorState()
> functions seem unnecessary. As the code stands, there's little
> performance gain from re-using the same CompressorState, just
> re-initializing it, and I can't see any other justification for them
> either.
>
> I combined those, and the Free/Flush steps, and did a bunch of other
> editorializations and cleanups. Here's an updated patch, also available
> in my git repository at
> git://git.postgresql.org/git/users/heikki/postgres.git, branch
> "pg_dump-dir". I'm going to continue reviewing this later, tomorrow
> hopefully.

Here's another update. I changed things quite heavily. I didn't see the 
point of having the Alloc+Free functions for uncompressing, because the 
ReadDataFromArchive processed the whole input stream in one go anyway. 
So the new API consists of four functions, AllocateCompressor, 
WriteDataToArchive and EndCompressor for writing, and 
ReadDataFromArchive for reading.

Also, I reverted the zlib buffer size from 64k to 4k. If you want to 
raise that, let's discuss that separately.

Please let me know what you think of this version, or if you spot any 
bugs. I'll keep working on this, I'm hoping to get this into committable 
shape by the end of the week.

The pg_backup_directory patch naturally won't apply over this anymore. 
Once we have the compress_io part in shape, that will need to be fixed.

-- 
   Heikki Linnakangas
   EnterpriseDB   http://www.enterprisedb.com



view thread (40+ messages)  latest in thread

Message-ID: <4CF655A3.4090308@enterprisedb.com>
Permalink:  ../4CF655A3.4090308@enterprisedb.com/
Also on:    postgresql.org/message-id/4CF655A3.4090308@enterprisedb.com

 · 

reply
Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-hackers@postgresql.org
  Cc: heikki.linnakangas@enterprisedb.com, joe@mcknight.de, tgl@sss.pgh.pa.us, jose.arthur@gmail.com
  Subject: Re: directory archive format for pg_dump
  In-Reply-To: <4CF655A3.4090308@enterprisedb.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox