pg.ddx.io  pgsql-admin@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Jim C. Nasby <jnasby@pervasive.com>
To: Pallav Kalva <pkalva@livedatagroup.com>
Cc: Steve Crawford <scrawford@pinpointresearch.com>
Cc: pgsql-admin@postgresql.org
Subject: Re: Archive Command Configuration
Date: Fri, 24 Mar 2006 13:03:27 -0600
Message-ID: <20060324190327.GX90527@pervasive.com> (raw)
In-Reply-To: <44242896.1020701@livedatagroup.com>
References: <4422FD80.5080009@livedatagroup.com>
	<442304F9.4090708@pinpointresearch.com>
	<20060324114222.GV90527@pervasive.com>
	<442412BD.6090706@livedatagroup.com>
	<20060324164530.GT90527@pervasive.com>
	<44242896.1020701@livedatagroup.com>

On Fri, Mar 24, 2006 at 12:12:54PM -0500, Pallav Kalva wrote:
> Jim C. Nasby wrote:
> >On Fri, Mar 24, 2006 at 10:39:41AM -0500, Pallav Kalva wrote:
> > 
> >> What do you think about this option ? If I just do scp from the 
> >>/archives folder, I will be ok and wont copy any partial files at any 
> >>time right ?
> >>archive_command = 'cp %p /archives-temp/%f && mv /archives-temp/%f 
> >>/archives/%f' 
> >>   
> >
> >A simple scp leaves you vulnerable to a WAL file only being partially
> >copied if a backhoe hits at just the right moment. I believe that
> >wouldn't be an issue with rsync, since it won't put a file in place
> >until it's completely transfered. So if you want to use scp, I think
> >you'd need to scp to a temporary directory, and then do a final move.
> > 
> 
> Are you talking about scp to a temporary directory and move on a remote 
> machine ?
> 
> The idea with this archive command is to make postgres copy first to 
> "archive-temp" directory  and once the copy of the file is done 
> completely then move it to "archives" directory on the source machine.
> My cronjob then does scp from the archives folder on source machine 
> regularly to a remote location, this way it never does a scp on a 
> partial file since files are movied  only when it is completely copied.
> 
> This way I will never scp a partial file and be sure that copy on the 
> remote machine is a complete clean copy.
> 
> (archive_command = 'cp %p /archives-temp/%f && mv /archives-temp/%f  
> /archives/%f')

scp archives/%f remote-machine:temp
ssh remote-machine mv temp/%f archive/%f

That's the only way I can think of to ensure you don't get a partial
logfile on the remote machine.
-- 
Jim C. Nasby, Sr. Engineering Consultant      jnasby@pervasive.com
Pervasive Software      http://pervasive.com    work: 512-231-6117
vcard: http://jim.nasby.net/pervasive.vcf       cell: 512-569-9461



view thread (9+ messages)

Message-ID: <20060324190327.GX90527@pervasive.com>
Permalink:  ../20060324190327.GX90527@pervasive.com/
Also on:    postgresql.org/message-id/20060324190327.GX90527@pervasive.com

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-admin@postgresql.org
  Cc: jnasby@pervasive.com, pkalva@livedatagroup.com, scrawford@pinpointresearch.com
  Subject: Re: Archive Command Configuration
  In-Reply-To: <20060324190327.GX90527@pervasive.com>
  References: <4422FD80.5080009@livedatagroup.com> <442304F9.4090708@pinpointresearch.com> <20060324114222.GV90527@pervasive.com> <442412BD.6090706@livedatagroup.com> <20060324164530.GT90527@pervasive.com> <44242896.1020701@livedatagroup.com> <20060324190327.GX90527@pervasive.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox