Received: from localhost (unknown [200.46.204.183]) by mail.postgresql.org (Postfix) with ESMTP id A426D634AD6 for ; Thu, 2 Apr 2009 17:18:23 -0300 (ADT) Received: from mail.postgresql.org ([200.46.204.86]) by localhost (mx1.hub.org [200.46.204.183]) (amavisd-maia, port 10024) with ESMTP id 21945-02 for ; Thu, 2 Apr 2009 17:18:21 -0300 (ADT) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from col0-omc2-s3.col0.hotmail.com (col0-omc2-s3.col0.hotmail.com [65.55.34.77]) by mail.postgresql.org (Postfix) with ESMTP id AA3736343C8 for ; Thu, 2 Apr 2009 17:18:21 -0300 (ADT) Received: from COL104-W10 ([65.55.34.72]) by col0-omc2-s3.col0.hotmail.com with Microsoft SMTPSVC(6.0.3790.3959); Thu, 2 Apr 2009 13:18:20 -0700 Message-ID: Content-Type: multipart/alternative; boundary="_8d93f1dc-f58e-4969-b851-dfb6823d5bd7_" X-Originating-IP: [86.81.58.36] From: henk de wit To: Tom Lane CC: Subject: Re: How to get parallel restore in PG 8.4 to work? Date: Thu, 2 Apr 2009 22:18:20 +0200 Importance: Normal In-Reply-To: <4107.1238593597@sss.pgh.pa.us> References: <20300.1238540913@sss.pgh.pa.us> <4107.1238593597@sss.pgh.pa.us> MIME-Version: 1.0 X-OriginalArrivalTime: 02 Apr 2009 20:18:20.0080 (UTC) FILETIME=[2A498300:01C9B3D0] X-Virus-Scanned: Maia Mailguard 1.0.1 X-Spam-Status: No, hits=0.001 tagged_above=0 required=5 tests=HTML_MESSAGE=0.001 X-Spam-Level: X-Archive-Number: 200904/45 X-Sequence-Number: 33412 --_8d93f1dc-f58e-4969-b851-dfb6823d5bd7_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable >> I still have some work to do to find out why dumping in the custom >> format is so much slower. >=20 > Offhand the only reason I can see for it to be much different from > plain-text output is that -Fc compresses by default. If you don't > care about that=2C try -Fc -Z0. Ok=2C I did some performance testing today and I appeared to be wrong after= all. My apologies for the noise. Here are some test results: Scenarioxfsjfs patchedjfscat backup | gunzip | psql45 min--pg_dump> hdd (un= compressed) (=3D=3Dpg_dump -Fp)--10 min 15 secpg_dump -Fc> hdd (uncompresse= d)10 min 20 sec10 min 21 sec10 min 28 secpg_dump -Fc | gzip> hdd11 min 20 s= ec11 min 25 sec12 min 04 secpg_restore 8 threads14 min 23 sec11 min 40 sec1= 1 min 20 secpg_restore 16 threads11 min 46 sec12 min 40 sec12 min 33 secpg_= restore 32 threads11 min 42 sec12 min 30 sec12 min 30 sec As can be seen in the table (hope this renders correctly on the mailing lis= t)=2C there is barely a difference between a plain dump and a custom format= dump. For who it concerns=2C xfs performance a little better than jfs here= =2C but the difference is marginal. More on topic=2C beyond 16 processes th= ere isn't any notable speed improvement for the parallel restore (as expect= ed). Kind regards=2CHenk _________________________________________________________________ See all the ways you can stay connected to friends and family http://www.microsoft.com/windows/windowslive/default.aspx= --_8d93f1dc-f58e-4969-b851-dfb6823d5bd7_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable >> I still have some work to do to find out why dumping in the custom
>>= format is so much slower.
>
> Offhand the only reason I can see for= it to be much different from
> plain-text output is that -Fc compresses= by default. If you don't
> care about that=2C try -Fc -Z0.

Ok=2C I did some performance testing today and I appeared to be wrong afte= r all. My apologies for the noise.

Here are some t= est results:

cat backup | gunzip | psql
Scenarioxfsjfs patchedjfs
45 min--=
pg_dump> hdd (uncompressed) (= =3D=3Dpg_dump -Fp)--10 min 15 sec
pg_dump -Fc> hdd (uncompressed)10 = min 20 sec10 min 21 sec10 min 28 sec
pg_dump -Fc = | gzip> hdd11 min 20 sec11 min 25 sec12 min 04 sec
pg_restore 8 threads14 min 23 sec11 min 40 sec11 min 20 sec
pg_rest= ore 16 threads11 min 46 sec12 min 40 sec12 min 33 sec
pg_restore 32 threads11 min 42 sec12 min 30 sec12 min 30 sec

As = can be seen in the table (hope this renders correctly on the mailing list)= =2C there is barely a difference between a plain dump and a custom format d= ump. For who it concerns=2C xfs performance a little better than jfs here= =2C but the difference is marginal. More on topic=2C beyond 16 processes th= ere isn't any notable speed improvement for the parallel restore (as expect= ed).

Kind regards=2C
Henk

See all the ways you can stay connected to frien= ds and family = --_8d93f1dc-f58e-4969-b851-dfb6823d5bd7_--