Received: from malur.postgresql.org ([217.196.149.56])
 by arkaria.postgresql.org with esmtp (Exim 4.80) (envelope-from
 <pgsql-docs-owner+M8049=pgsql+2Ddocs=arkaria.postgresql.org@postgresql.org>)
 id 1XHmf9-0008Lv-OJ
 for pgsql-docs@arkaria.postgresql.org; Thu, 14 Aug 2014 04:29:19 +0000
Received: from localhost ([127.0.0.1] helo=postgresql.org)
 by malur.postgresql.org with smtp (Exim 4.80) (envelope-from
 <pgsql-docs-owner+M8049=pgsql+2Ddocs=arkaria.postgresql.org@postgresql.org>)
 id 1XHmf8-0006FL-KG
 for pgsql-docs@arkaria.postgresql.org; Thu, 14 Aug 2014 04:29:18 +0000
Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29])
 by malur.postgresql.org with esmtps (TLS1.2:DHE_RSA_AES_256_CBC_SHA256:256)
 (Exim 4.80) (envelope-from <maumau307@gmail.com>) id 1XHmf7-0006FE-LU
 for pgsql-docs@postgresql.org; Thu, 14 Aug 2014 04:29:17 +0000
Received: from mail-pa0-x229.google.com ([2607:f8b0:400e:c03::229])
 by magus.postgresql.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:256)
 (Exim 4.80) (envelope-from <maumau307@gmail.com>) id 1XHmf3-0001eu-78
 for pgsql-docs@postgresql.org; Thu, 14 Aug 2014 04:29:16 +0000
Received: by mail-pa0-f41.google.com with SMTP id rd3so908293pab.0
 for <pgsql-docs@postgresql.org>; Wed, 13 Aug 2014 21:29:10 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
 h=message-id:from:to:references:in-reply-to:subject:date:mime-version
 :content-type:content-transfer-encoding;
 bh=Vw9r3fDey0ZusjR/PKF7nBb41OUVCTQXMBKSi/SiQ8A=;
 b=UDxZPrHJ+Hd/iJb2FIpiGGzc2sj0gtwuse6v3/S+A3++CD0nlA51UrKGBDwuidL/uc
 75+w1j5KfwheUPt/WSW3NGdnEDBhEtzhHgUs/6uj+S0pCrz3weqp9lSzqZgpKSZ+0/SB
 XOE+d4m18RzoxK6md2vU5wfjQAxGlrGFngiI3q0lpsWzYESJKQYF2U92wwEgpsVJkHLa
 IBk6jwSXt1SN7Z/h1yl+bYA6v7CzweqpfTgQegMTcZYXhGSpGyST1ksKEaPjLailLn7L
 iNcntXTAsDW7HQ6WvTEQS8HAkt/O8MrENyZKB6m6su83yjMoEUgyxdUy+1x2pdzYQdD/
 PeSw==
X-Received: by 10.66.139.106 with SMTP id qx10mr1859705pab.126.1407990550321;
 Wed, 13 Aug 2014 21:29:10 -0700 (PDT)
Received: from maumau (p23196-ipngn100307sizuokaden.shizuoka.ocn.ne.jp.
 [153.131.144.196])
 by mx.google.com with ESMTPSA id pv2sm3756230pbb.13.2014.08.13.21.29.07
 for <multiple recipients>
 (version=TLSv1 cipher=ECDHE-RSA-AES128-SHA bits=128/128);
 Wed, 13 Aug 2014 21:29:09 -0700 (PDT)
Message-ID: <35326E59461948B394A861F69C272795@maumau>
From: "MauMau" <maumau307@gmail.com>
To: "Peter Eisentraut" <peter_e@gmx.net>,
 "Kevin Grittner" <kgrittn@ymail.com>,
 "Josh Berkus" <josh@agliodbs.com>, <pgsql-docs@postgresql.org>
References: <53E5603B.5040102@agliodbs.com>
 <1407777686.41146.YahooMailNeo@web122306.mail.ne1.yahoo.com>
 <0F73426A2EA544878BCAC33BB989D671@maumau> <53EBCCEA.3090604@gmx.net>
In-Reply-To: <53EBCCEA.3090604@gmx.net>
Subject: Re: Sample archive_command is still problematic
Date: Thu, 14 Aug 2014 13:31:55 +0900
MIME-Version: 1.0
Content-Type: text/plain; format=flowed; charset="windows-1252";
 reply-type=original
Content-Transfer-Encoding: 7bit
X-Priority: 3
X-MSMail-Priority: Normal
X-Mailer: Microsoft Windows Mail 6.0.6002.18197
X-MimeOLE: Produced By Microsoft MimeOLE V6.0.6002.18463
X-Pg-Spam-Score: -0.1 (/)
List-Archive: <http://archives.postgresql.org/pgsql-docs>
List-Help: <mailto:majordomo@postgresql.org?body=help>
List-ID: <pgsql-docs.postgresql.org>
List-Owner: <mailto:pgsql-docs-owner@postgresql.org>
List-Post: <mailto:pgsql-docs@postgresql.org>
List-Subscribe: <mailto:majordomo@postgresql.org?body=sub%20pgsql-docs>
List-Unsubscribe: <mailto:majordomo@postgresql.org?body=unsub%20pgsql-docs>
X-Mailing-List: pgsql-docs
Precedence: bulk
Sender: pgsql-docs-owner@postgresql.org

From: "Peter Eisentraut" <peter_e@gmx.net>
> I realize that there are about 128 different ways people set this up
> (which is itself a problem), but it appears to me that a solution like
> pg_copy only provides local copying, which implies the use of something
> like NFS.  Which may be OK, but then we'd need to get into the details
> of how to set up NFS properly for this.

Yes, I think the flexibility of archive_command is nice.  The problem I want 
to address is that users don't have a simple way to realiably archive files 
in very simple use cases -- local copying to local or network storage. 
pg_copy is a low-level command to fill the gap.


> Also, I think you can get local copy+fsync with dd.

Yes, dd on Linux has "sync" option.  But dd on Solaris doesn't.  I can't 
find a command on Windows which is installed by default.

> The alternatives of doing remote copying inside archive_command are also
> questionable if you have multiple standbys.

Yes, we may need another interface than archive_command for archiving files 
to multiple locations.  That's another issue.


> Basically, this whole interface is terrible.  Maybe it's time to phase
> it out and start looking into pg_receivexlog.

pg_receivexlog seems difficult to me.  Users have to start, stop, and 
monitor pg_receivexlog.  That's burdonsome.  For example, how do we start 
pg_receivexlog easily on Windows when the PostgreSQL is configured to 
start/stop automatically on OS startup/shutdown with Windows service?  In 
addition, users have to be aware of connection slots (max_connections and 
max_wal_senders) and replication slots.

pg_receivexlog impose extra overhead even on simple use cases.  I want 
backup-related facilities to use as less resources as possible.  e.g., with 
archive_command, the data flows like this:

disk -> OS cache -> copy command's buffer -> OS cache -> disk

OTOH, with pg_receivexlog:

disk -> OS cache -> walsender's buffer -> socket send buffer -> kernel 
buffer? -> socket receive buffer -> pg_receivexlog's buffer -> OS cache -> 
disk

For reference, \copy of psql is described like this:

Tip: This operation is not as efficient as the SQL COPY command because all 
data must pass through the client/server connection. For large amounts of 
data the SQL command might be preferable.

Regards
MauMau



-- 
Sent via pgsql-docs mailing list (pgsql-docs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-docs