pg.ddx.io  pgsql-admin@postgresql.org mailing list archive  
help / color / mirror / Atom feed
files to ignore during pitr base backup
4+ messages / 3 participants
[nested] [flat]

* files to ignore during pitr base backup
@ 2005-10-25 15:44  Jeff Frost <jeff@frostconsultingllc.com>
  0 siblings, 1 reply; 4+ messages in thread

From: Jeff Frost @ 2005-10-25 15:44 UTC (permalink / raw)
  To: pgsql-admin

I noticed recently that my PITR base backups were occassionally returning 
non-zero, so I turned up the verbosity on rsync and it appears that 
occassionally a file vanishes out from under rsync while it's doing the base 
backup:

          700   4%   52.58kB/s    0:00:00
        16384 100%    1.12MB/s    0:00:00  (125, 71.4% of 1024)
file has vanished: "/usr/local/pgsql/data/base/1219609879/pg_internal.init"
base/17229/

sent 258563798 bytes  received 1348515 bytes  2809862.84 bytes/sec
total size is 8273133704  speedup is 31.83
rsync warning: some files vanished before they could be transferred (code
24) at main.c(791)
PITR base backup rsync command failed!

Would it be ok to just exclude the pg_internal.init file since it disappeared 
during the base backup?

-- 
Jeff Frost, Owner 	<jeff@frostconsultingllc.com>
Frost Consulting, LLC 	http://www.frostconsultingllc.com/
Phone: 650-780-7908	FAX: 650-649-1954



^ permalink  raw  reply  [nested|flat] 4+ messages in thread

* Re: files to ignore during pitr base backup
@ 2005-10-25 18:09  Tom Lane <tgl@sss.pgh.pa.us>
  parent: Jeff Frost <jeff@frostconsultingllc.com>
  0 siblings, 1 reply; 4+ messages in thread

From: Tom Lane @ 2005-10-25 18:09 UTC (permalink / raw)
  To: Jeff Frost <jeff@frostconsultingllc.com>; +Cc: pgsql-admin

Jeff Frost <jeff@frostconsultingllc.com> writes:
> I noticed recently that my PITR base backups were occassionally returning 
> non-zero, so I turned up the verbosity on rsync and it appears that 
> occassionally a file vanishes out from under rsync while it's doing the base 
> backup:
> ...
> Would it be ok to just exclude the pg_internal.init file since it disappeared
> during the base backup?

It's safe to exclude those, but those are hardly the only files that
might "disappear" in a live database.  It'd be better to use a dump
tool that's less picky about the source data changing under it.  We've
had similar reports about GNU tar being too smart for this purpose :-(

			regards, tom lane



^ permalink  raw  reply  [nested|flat] 4+ messages in thread

* Re: files to ignore during pitr base backup
@ 2005-10-25 19:29  Jeff Frost <jeff@frostconsultingllc.com>
  parent: Tom Lane <tgl@sss.pgh.pa.us>
  0 siblings, 1 reply; 4+ messages in thread

From: Jeff Frost @ 2005-10-25 19:29 UTC (permalink / raw)
  To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: pgsql-admin

On Tue, 25 Oct 2005, Tom Lane wrote:

> It's safe to exclude those, but those are hardly the only files that
> might "disappear" in a live database.  It'd be better to use a dump
> tool that's less picky about the source data changing under it.  We've
> had similar reports about GNU tar being too smart for this purpose :-(

I would have thought that the pg_start_backup() function would tell the 
postmasters not to take files out from under the base backup process?

In any event, if it's ok to ignore those, rsync conveniently has a specific 
error code for that:

        24     Partial transfer due to vanished source files

So I can just consider an exit code of 0 or 24 as both being success.

-- 
Jeff Frost, Owner 	<jeff@frostconsultingllc.com>
Frost Consulting, LLC 	http://www.frostconsultingllc.com/
Phone: 650-780-7908	FAX: 650-649-1954



^ permalink  raw  reply  [nested|flat] 4+ messages in thread

* Re: files to ignore during pitr base backup
@ 2005-10-25 20:35  Alvaro Herrera <alvherre@alvh.no-ip.org>
  parent: Jeff Frost <jeff@frostconsultingllc.com>
  0 siblings, 0 replies; 4+ messages in thread

From: Alvaro Herrera @ 2005-10-25 20:35 UTC (permalink / raw)
  To: Jeff Frost <jeff@frostconsultingllc.com>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; pgsql-admin

Jeff Frost wrote:
> On Tue, 25 Oct 2005, Tom Lane wrote:
> 
> >It's safe to exclude those, but those are hardly the only files that
> >might "disappear" in a live database.  It'd be better to use a dump
> >tool that's less picky about the source data changing under it.  We've
> >had similar reports about GNU tar being too smart for this purpose :-(
> 
> I would have thought that the pg_start_backup() function would tell the 
> postmasters not to take files out from under the base backup process?

That would be impossible unless you wanted backends to freeze from time
to time while you take the base backup.

-- 
Alvaro Herrera                  http://www.amazon.com/gp/registry/5ZYLFMCVHXC
"La rebeldía es la virtud original del hombre" (Arthur Schopenhauer)



^ permalink  raw  reply  [nested|flat] 4+ messages in thread


end of thread, other threads:[~2005-10-25 20:35 UTC | newest]

Thread overview: 4+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2005-10-25 15:44 files to ignore during pitr base backup Jeff Frost <jeff@frostconsultingllc.com>
2005-10-25 18:09 ` Tom Lane <tgl@sss.pgh.pa.us>
2005-10-25 19:29   ` Jeff Frost <jeff@frostconsultingllc.com>
2005-10-25 20:35     ` Alvaro Herrera <alvherre@alvh.no-ip.org>

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox