pg.ddx.io  pgsql-performance@postgresql.org mailing list archive  
help / color / mirror / Atom feed
WEIRD CRASH?!?!
15+ messages / 7 participants
[nested] [flat]

* WEIRD CRASH?!?!
@ 2003-01-24 23:59 Noah Silverman <noah@allresearch.com>
  2003-01-25 00:03 ` Re: WEIRD CRASH?!?! Josh Berkus <josh@agliodbs.com>
  0 siblings, 1 reply; 15+ messages in thread

From: Noah Silverman @ 2003-01-24 23:59 UTC (permalink / raw)
  To: josh@agliodbs.com; +Cc: pgsql-performance

I think my server crashed and then restarted itself.  Does anybody know 
what all this means:

2003-01-24 18:28:06 PANIC:  link from 
/RAID/pgsql/pg_xlog/00000009000000BC to 
/RAID/pgsql/pg_xlog/00000009000000C4 (initialization of log file 9, 
segment 196) failed: File exists
2003-01-24 18:28:06 LOG:  server process (pid 1574) was terminated by 
signal 6
2003-01-24 18:28:06 LOG:  terminating any other active server processes
2003-01-24 18:28:06 WARNING:  Message from PostgreSQL backend:
         The Postmaster has informed me that some other backend
         died abnormally and possibly corrupted shared memory.
         I have rolled back the current transaction and am
         going to terminate your database system connection and exit.
         Please reconnect to the database system and repeat your query.
2003-01-24 18:28:06 LOG:  all server processes terminated; 
reinitializing shared memory and semaphores
2003-01-24 18:28:06 LOG:  database system was interrupted at 2003-01-24 
18:28:06 EST
2003-01-24 18:28:06 LOG:  checkpoint record is at 9/C4574974
2003-01-24 18:28:06 LOG:  redo record is at 9/C200D144; undo record is 
at 0/0; shutdown FALSE
2003-01-24 18:28:06 LOG:  next transaction id: 5159292; next oid: 
50856954
2003-01-24 18:28:06 LOG:  database system was not properly shut down; 
automatic recovery in progress
2003-01-24 18:28:06 LOG:  redo starts at 9/C200D144
2003-01-24 18:28:13 LOG:  ReadRecord: record with zero length at 
9/C4578CC0
2003-01-24 18:28:13 LOG:  redo done at 9/C4578C9C
2003-01-24 18:29:02 LOG:  recycled transaction log file 00000009000000C0
2003-01-24 18:29:02 LOG:  recycled transaction log file 00000009000000C1
2003-01-24 18:29:02 LOG:  recycled transaction log file 00000009000000BC
2003-01-24 18:29:02 LOG:  recycled transaction log file 00000009000000BD
2003-01-24 18:29:02 LOG:  recycled transaction log file 00000009000000BE
2003-01-24 18:29:02 LOG:  recycled transaction log file 00000009000000BF
2003-01-24 18:29:02 LOG:  database system is ready







^ permalink  raw  reply  [nested|flat] 15+ messages in thread

* Re: WEIRD CRASH?!?!
  2003-01-24 23:59 WEIRD CRASH?!?! Noah Silverman <noah@allresearch.com>
@ 2003-01-25 00:03 ` Josh Berkus <josh@agliodbs.com>
  2003-01-25 00:08   ` Re: WEIRD CRASH?!?! Noah Silverman <noah@allresearch.com>
  2003-01-25 00:27   ` Re: WEIRD CRASH?!?! Stephan Szabo <sszabo@megazone23.bigpanda.com>
  0 siblings, 2 replies; 15+ messages in thread

From: Josh Berkus @ 2003-01-25 00:03 UTC (permalink / raw)
  To: Noah Silverman <noah@allresearch.com>; +Cc: pgsql-performance

Noah,

> I think my server crashed and then restarted itself.  Does anybody know 
> what all this means:
> 
> 2003-01-24 18:28:06 PANIC:  link from 
> /RAID/pgsql/pg_xlog/00000009000000BC to 
> /RAID/pgsql/pg_xlog/00000009000000C4 (initialization of log file 9, 
> segment 196) failed: File exists
> 2003-01-24 18:28:06 LOG:  server process (pid 1574) was terminated by 
> signal 6
> 2003-01-24 18:28:06 LOG:  terminating any other active server processes
> 2003-01-24 18:28:06 WARNING:  Message from PostgreSQL backend:
>          The Postmaster has informed me that some other backend
>          died abnormally and possibly corrupted shared memory.
>          I have rolled back the current transaction and am
>          going to terminate your database system connection and exit.
>          Please reconnect to the database system and repeat your query.

This means that somebody KILL -9'd a postgres process or the postmaster, and 
Postgres restarted in order to clear the shared buffers.   If the database 
started up again, you are fine.

-- 
-Josh Berkus
 Aglio Database Solutions
 San Francisco




^ permalink  raw  reply  [nested|flat] 15+ messages in thread

* Re: WEIRD CRASH?!?!
  2003-01-24 23:59 WEIRD CRASH?!?! Noah Silverman <noah@allresearch.com>
  2003-01-25 00:03 ` Re: WEIRD CRASH?!?! Josh Berkus <josh@agliodbs.com>
@ 2003-01-25 00:08   ` Noah Silverman <noah@allresearch.com>
  2003-01-25 00:13     ` Re: WEIRD CRASH?!?! scott.marlowe <scott.marlowe@ihs.com>
  2003-01-25 00:15     ` Re: WEIRD CRASH?!?! Josh Berkus <josh@agliodbs.com>
  2003-01-25 18:44     ` Re: WEIRD CRASH?!?! Andrew Sullivan <andrew@libertyrms.info>
  1 sibling, 3 replies; 15+ messages in thread

From: Noah Silverman @ 2003-01-25 00:08 UTC (permalink / raw)
  To: josh@agliodbs.com; +Cc: pgsql-performance

Yes,
but I'm the only one logged into this box, and I didn't kill anything.  
It appears to have died all by itself.

Thanks,

-N


On Friday, January 24, 2003, at 07:03 PM, Josh Berkus wrote:

> Noah,
>
>> I think my server crashed and then restarted itself.  Does anybody 
>> know
>> what all this means:
>>
>> 2003-01-24 18:28:06 PANIC:  link from
>> /RAID/pgsql/pg_xlog/00000009000000BC to
>> /RAID/pgsql/pg_xlog/00000009000000C4 (initialization of log file 9,
>> segment 196) failed: File exists
>> 2003-01-24 18:28:06 LOG:  server process (pid 1574) was terminated by
>> signal 6
>> 2003-01-24 18:28:06 LOG:  terminating any other active server 
>> processes
>> 2003-01-24 18:28:06 WARNING:  Message from PostgreSQL backend:
>>          The Postmaster has informed me that some other backend
>>          died abnormally and possibly corrupted shared memory.
>>          I have rolled back the current transaction and am
>>          going to terminate your database system connection and exit.
>>          Please reconnect to the database system and repeat your 
>> query.
>
> This means that somebody KILL -9'd a postgres process or the 
> postmaster, and
> Postgres restarted in order to clear the shared buffers.   If the 
> database
> started up again, you are fine.
>
> -- 
> -Josh Berkus
>  Aglio Database Solutions
>  San Francisco
>
>




^ permalink  raw  reply  [nested|flat] 15+ messages in thread

* Re: WEIRD CRASH?!?!
  2003-01-24 23:59 WEIRD CRASH?!?! Noah Silverman <noah@allresearch.com>
  2003-01-25 00:03 ` Re: WEIRD CRASH?!?! Josh Berkus <josh@agliodbs.com>
  2003-01-25 00:08   ` Re: WEIRD CRASH?!?! Noah Silverman <noah@allresearch.com>
@ 2003-01-25 00:13     ` scott.marlowe <scott.marlowe@ihs.com>
  2 siblings, 0 replies; 15+ messages in thread

From: scott.marlowe @ 2003-01-25 00:13 UTC (permalink / raw)
  To: Noah Silverman <noah@allresearch.com>; +Cc: josh@agliodbs.com; pgsql-performance

On Fri, 24 Jan 2003, Noah Silverman wrote:

> Yes,
> but I'm the only one logged into this box, and I didn't kill anything.  
> It appears to have died all by itself.
> 

It certainly sounds that way.  Can you recreate the circumstances and make 
it happen reliably?  If not, the likely it's just an isolated occurance 
and nothing to get too worried about.  Your data is still coherent, that's 
why all the backends were forced to reset, to cleanse the buffers from 
possible corruption.





^ permalink  raw  reply  [nested|flat] 15+ messages in thread

* Re: WEIRD CRASH?!?!
  2003-01-24 23:59 WEIRD CRASH?!?! Noah Silverman <noah@allresearch.com>
  2003-01-25 00:03 ` Re: WEIRD CRASH?!?! Josh Berkus <josh@agliodbs.com>
  2003-01-25 00:08   ` Re: WEIRD CRASH?!?! Noah Silverman <noah@allresearch.com>
@ 2003-01-25 00:15     ` Josh Berkus <josh@agliodbs.com>
  2003-01-25 00:17       ` Re: WEIRD CRASH?!?! Noah Silverman <noah@allresearch.com>
  2 siblings, 1 reply; 15+ messages in thread

From: Josh Berkus @ 2003-01-25 00:15 UTC (permalink / raw)
  To: Noah Silverman <noah@allresearch.com>; +Cc: pgsql-performance


Noah,

> but I'm the only one logged into this box, and I didn't kill anything.  
> It appears to have died all by itself.

I'd check your disk array, then.  It doesn't happen to be a Mylex, does it?

-- 
-Josh Berkus
 Aglio Database Solutions
 San Francisco




^ permalink  raw  reply  [nested|flat] 15+ messages in thread

* Re: WEIRD CRASH?!?!
  2003-01-24 23:59 WEIRD CRASH?!?! Noah Silverman <noah@allresearch.com>
  2003-01-25 00:03 ` Re: WEIRD CRASH?!?! Josh Berkus <josh@agliodbs.com>
  2003-01-25 00:08   ` Re: WEIRD CRASH?!?! Noah Silverman <noah@allresearch.com>
  2003-01-25 00:15     ` Re: WEIRD CRASH?!?! Josh Berkus <josh@agliodbs.com>
@ 2003-01-25 00:17       ` Noah Silverman <noah@allresearch.com>
  2003-01-25 01:04         ` Re: WEIRD CRASH?!?! Tom Lane <tgl@sss.pgh.pa.us>
  2003-01-25 01:08         ` Re: WEIRD CRASH?!?! Rod Taylor <rbt@rbt.ca>
  0 siblings, 2 replies; 15+ messages in thread

From: Noah Silverman @ 2003-01-25 00:17 UTC (permalink / raw)
  To: josh@agliodbs.com; +Cc: pgsql-performance

We are using a 3ware escalade on this box.

One clue.

I actually moved the pg_xlog directory to another drive and then 
symbolically linked it back to the data directory.

Another idea is that Linux killed one of the processes because postgres 
was using up too much memory.  I belive the part of the kernel  is 
called "oomkiller".  We're not sure if this happened, just a guess.

Thanks,

-N



On Friday, January 24, 2003, at 07:15 PM, Josh Berkus wrote:

>
> Noah,
>
>> but I'm the only one logged into this box, and I didn't kill anything.
>> It appears to have died all by itself.
>
> I'd check your disk array, then.  It doesn't happen to be a Mylex, 
> does it?
>
> -- 
> -Josh Berkus
>  Aglio Database Solutions
>  San Francisco
>
>




^ permalink  raw  reply  [nested|flat] 15+ messages in thread

* Re: WEIRD CRASH?!?!
  2003-01-24 23:59 WEIRD CRASH?!?! Noah Silverman <noah@allresearch.com>
  2003-01-25 00:03 ` Re: WEIRD CRASH?!?! Josh Berkus <josh@agliodbs.com>
  2003-01-25 00:08   ` Re: WEIRD CRASH?!?! Noah Silverman <noah@allresearch.com>
  2003-01-25 00:15     ` Re: WEIRD CRASH?!?! Josh Berkus <josh@agliodbs.com>
  2003-01-25 00:17       ` Re: WEIRD CRASH?!?! Noah Silverman <noah@allresearch.com>
@ 2003-01-25 01:04         ` Tom Lane <tgl@sss.pgh.pa.us>
  1 sibling, 0 replies; 15+ messages in thread

From: Tom Lane @ 2003-01-25 01:04 UTC (permalink / raw)
  To: Noah Silverman <noah@allresearch.com>; +Cc: josh@agliodbs.com; pgsql-performance

Noah Silverman <noah@allresearch.com> writes:
> One clue.
> I actually moved the pg_xlog directory to another drive and then 
> symbolically linked it back to the data directory.

Uh, did you have the postmaster shut down while you did that?

This looks like a collision between two processes both trying to create
the next segment of the xlog at about the same time.  But there are
interlocks that are supposed to prevent that.

I don't think you need to worry about the integrity of your data; the
panic reset should put everything right.  But I'd sure be interested
if you can reproduce this problem.

			regards, tom lane



^ permalink  raw  reply  [nested|flat] 15+ messages in thread

* Re: WEIRD CRASH?!?!
  2003-01-24 23:59 WEIRD CRASH?!?! Noah Silverman <noah@allresearch.com>
  2003-01-25 00:03 ` Re: WEIRD CRASH?!?! Josh Berkus <josh@agliodbs.com>
  2003-01-25 00:08   ` Re: WEIRD CRASH?!?! Noah Silverman <noah@allresearch.com>
  2003-01-25 00:15     ` Re: WEIRD CRASH?!?! Josh Berkus <josh@agliodbs.com>
  2003-01-25 00:17       ` Re: WEIRD CRASH?!?! Noah Silverman <noah@allresearch.com>
@ 2003-01-25 01:08         ` Rod Taylor <rbt@rbt.ca>
  2003-01-25 01:14           ` Re: WEIRD CRASH?!?! Noah Silverman <noah@allresearch.com>
  1 sibling, 1 reply; 15+ messages in thread

From: Rod Taylor @ 2003-01-25 01:08 UTC (permalink / raw)
  To: Noah Silverman <noah@allresearch.com>; +Cc: pgsql-performance

On Fri, 2003-01-24 at 19:17, Noah Silverman wrote:
> We are using a 3ware escalade on this box.
> 
> One clue.
> 
> I actually moved the pg_xlog directory to another drive and then 
> symbolically linked it back to the data directory.

You shut it down first right?

-- 
Rod Taylor <rbt@rbt.ca>

PGP Key: http://www.rbt.ca/rbtpub.asc

Attachments:

  signature.asc (application/pgp-signature, 186B) download

^ permalink  raw  reply  [nested|flat] 15+ messages in thread

* Re: WEIRD CRASH?!?!
  2003-01-24 23:59 WEIRD CRASH?!?! Noah Silverman <noah@allresearch.com>
  2003-01-25 00:03 ` Re: WEIRD CRASH?!?! Josh Berkus <josh@agliodbs.com>
  2003-01-25 00:08   ` Re: WEIRD CRASH?!?! Noah Silverman <noah@allresearch.com>
  2003-01-25 00:15     ` Re: WEIRD CRASH?!?! Josh Berkus <josh@agliodbs.com>
  2003-01-25 00:17       ` Re: WEIRD CRASH?!?! Noah Silverman <noah@allresearch.com>
  2003-01-25 01:08         ` Re: WEIRD CRASH?!?! Rod Taylor <rbt@rbt.ca>
@ 2003-01-25 01:14           ` Noah Silverman <noah@allresearch.com>
  2003-01-25 01:20             ` Re: WEIRD CRASH?!?! Rod Taylor <rbt@rbt.ca>
  2003-01-25 01:38             ` Re: WEIRD CRASH?!?! Tom Lane <tgl@sss.pgh.pa.us>
  0 siblings, 2 replies; 15+ messages in thread

From: Noah Silverman @ 2003-01-25 01:14 UTC (permalink / raw)
  To: Rod Taylor <rbt@rbt.ca>; +Cc: pgsql-performance; josh@agliodbs.com

OF COURSE!

It actually looks like the problem was with file descriptors.  Our 
shell only had 1024 set, and we also have mysql running and using up a 
bunch of those.  We just upped to limit to 8000 to see it that would 
give postgres more room to breathe.

-N

On Friday, January 24, 2003, at 08:08 PM, Rod Taylor wrote:

> On Fri, 2003-01-24 at 19:17, Noah Silverman wrote:
>> We are using a 3ware escalade on this box.
>>
>> One clue.
>>
>> I actually moved the pg_xlog directory to another drive and then
>> symbolically linked it back to the data directory.
>
> You shut it down first right?
>
> -- 
> Rod Taylor <rbt@rbt.ca>
>
> PGP Key: http://www.rbt.ca/rbtpub.asc
> <signature.asc>




^ permalink  raw  reply  [nested|flat] 15+ messages in thread

* Re: WEIRD CRASH?!?!
  2003-01-24 23:59 WEIRD CRASH?!?! Noah Silverman <noah@allresearch.com>
  2003-01-25 00:03 ` Re: WEIRD CRASH?!?! Josh Berkus <josh@agliodbs.com>
  2003-01-25 00:08   ` Re: WEIRD CRASH?!?! Noah Silverman <noah@allresearch.com>
  2003-01-25 00:15     ` Re: WEIRD CRASH?!?! Josh Berkus <josh@agliodbs.com>
  2003-01-25 00:17       ` Re: WEIRD CRASH?!?! Noah Silverman <noah@allresearch.com>
  2003-01-25 01:08         ` Re: WEIRD CRASH?!?! Rod Taylor <rbt@rbt.ca>
  2003-01-25 01:14           ` Re: WEIRD CRASH?!?! Noah Silverman <noah@allresearch.com>
@ 2003-01-25 01:20             ` Rod Taylor <rbt@rbt.ca>
  1 sibling, 0 replies; 15+ messages in thread

From: Rod Taylor @ 2003-01-25 01:20 UTC (permalink / raw)
  To: Noah Silverman <noah@allresearch.com>; +Cc: pgsql-performance; josh@agliodbs.com

On Fri, 2003-01-24 at 20:14, Noah Silverman wrote:
> OF COURSE!

Sorry, but I've seen people try to do that stuff before.


> On Friday, January 24, 2003, at 08:08 PM, Rod Taylor wrote:
> 
> > On Fri, 2003-01-24 at 19:17, Noah Silverman wrote:
> >> We are using a 3ware escalade on this box.
> >>
> >> One clue.
> >>
> >> I actually moved the pg_xlog directory to another drive and then
> >> symbolically linked it back to the data directory.
> >
> > You shut it down first right?
> >
> > -- 
> > Rod Taylor <rbt@rbt.ca>
> >
> > PGP Key: http://www.rbt.ca/rbtpub.asc
> > <signature.asc>
-- 
Rod Taylor <rbt@rbt.ca>

PGP Key: http://www.rbt.ca/rbtpub.asc

Attachments:

  signature.asc (application/pgp-signature, 186B) download

^ permalink  raw  reply  [nested|flat] 15+ messages in thread

* Re: WEIRD CRASH?!?!
  2003-01-24 23:59 WEIRD CRASH?!?! Noah Silverman <noah@allresearch.com>
  2003-01-25 00:03 ` Re: WEIRD CRASH?!?! Josh Berkus <josh@agliodbs.com>
  2003-01-25 00:08   ` Re: WEIRD CRASH?!?! Noah Silverman <noah@allresearch.com>
  2003-01-25 00:15     ` Re: WEIRD CRASH?!?! Josh Berkus <josh@agliodbs.com>
  2003-01-25 00:17       ` Re: WEIRD CRASH?!?! Noah Silverman <noah@allresearch.com>
  2003-01-25 01:08         ` Re: WEIRD CRASH?!?! Rod Taylor <rbt@rbt.ca>
  2003-01-25 01:14           ` Re: WEIRD CRASH?!?! Noah Silverman <noah@allresearch.com>
@ 2003-01-25 01:38             ` Tom Lane <tgl@sss.pgh.pa.us>
  1 sibling, 0 replies; 15+ messages in thread

From: Tom Lane @ 2003-01-25 01:38 UTC (permalink / raw)
  To: Noah Silverman <noah@allresearch.com>; +Cc: Rod Taylor <rbt@rbt.ca>; pgsql-performance; josh@agliodbs.com

Noah Silverman <noah@allresearch.com> writes:
> It actually looks like the problem was with file descriptors.  Our 
> shell only had 1024 set, and we also have mysql running and using up a 
> bunch of those.  We just upped to limit to 8000 to see it that would 
> give postgres more room to breathe.

Ah-hah.  You might also want to set max_files_per_process (in
postgresql.conf) to something small enough to ensure Postgres can't run
you out of descriptors.  Linux has a bad habit of promising more than
it can deliver when Postgres asks how many FDs are okay to use.  The
max_files_per_process setting is useful to prevent Postgres from
believing whatever fairy-tale sysconf(3) tells it.

			regards, tom lane



^ permalink  raw  reply  [nested|flat] 15+ messages in thread

* Re: WEIRD CRASH?!?!
  2003-01-24 23:59 WEIRD CRASH?!?! Noah Silverman <noah@allresearch.com>
  2003-01-25 00:03 ` Re: WEIRD CRASH?!?! Josh Berkus <josh@agliodbs.com>
  2003-01-25 00:08   ` Re: WEIRD CRASH?!?! Noah Silverman <noah@allresearch.com>
@ 2003-01-25 18:44     ` Andrew Sullivan <andrew@libertyrms.info>
  2 siblings, 0 replies; 15+ messages in thread

From: Andrew Sullivan @ 2003-01-25 18:44 UTC (permalink / raw)
  To: pgsql-performance

On Fri, Jan 24, 2003 at 07:08:29PM -0500, Noah Silverman wrote:
> Yes,
> but I'm the only one logged into this box, and I didn't kill anything.  
> It appears to have died all by itself.

Is this on Linux, and were you short on memory?  Linux, in a
completely brain-dead design, runs around 'kill -9'-ing random
processes when it starts to think the machine is going to exhaust its
memory (or at least it used to.  I dunno if it still does).

A

-- 
----
Andrew Sullivan                         204-4141 Yonge Street
Liberty RMS                           Toronto, Ontario Canada
<andrew@libertyrms.info>                              M2P 2A8
                                         +1 416 646 3304 x110




^ permalink  raw  reply  [nested|flat] 15+ messages in thread

* Re: WEIRD CRASH?!?!
  2003-01-24 23:59 WEIRD CRASH?!?! Noah Silverman <noah@allresearch.com>
  2003-01-25 00:03 ` Re: WEIRD CRASH?!?! Josh Berkus <josh@agliodbs.com>
@ 2003-01-25 00:27   ` Stephan Szabo <sszabo@megazone23.bigpanda.com>
  2003-01-25 01:12     ` Re: WEIRD CRASH?!?! Tom Lane <tgl@sss.pgh.pa.us>
  1 sibling, 1 reply; 15+ messages in thread

From: Stephan Szabo @ 2003-01-25 00:27 UTC (permalink / raw)
  To: Josh Berkus <josh@agliodbs.com>; +Cc: Noah Silverman <noah@allresearch.com>; pgsql-performance

On Fri, 24 Jan 2003, Josh Berkus wrote:

> Noah,
>
> > I think my server crashed and then restarted itself.  Does anybody know
> > what all this means:
> >
> > 2003-01-24 18:28:06 PANIC:  link from
> > /RAID/pgsql/pg_xlog/00000009000000BC to
> > /RAID/pgsql/pg_xlog/00000009000000C4 (initialization of log file 9,
> > segment 196) failed: File exists
> > 2003-01-24 18:28:06 LOG:  server process (pid 1574) was terminated by
> > signal 6
> > 2003-01-24 18:28:06 LOG:  terminating any other active server processes
> > 2003-01-24 18:28:06 WARNING:  Message from PostgreSQL backend:
> >          The Postmaster has informed me that some other backend
> >          died abnormally and possibly corrupted shared memory.
> >          I have rolled back the current transaction and am
> >          going to terminate your database system connection and exit.
> >          Please reconnect to the database system and repeat your query.
>
> This means that somebody KILL -9'd a postgres process or the postmaster, and
> Postgres restarted in order to clear the shared buffers.   If the database
> started up again, you are fine.

Actually, it looks like an abort() (signal 6) to me.  Probably from the
PANIC listed.

The question is why did it get confused and end up linking to a filename
that already existed?






^ permalink  raw  reply  [nested|flat] 15+ messages in thread

* Re: WEIRD CRASH?!?!
  2003-01-24 23:59 WEIRD CRASH?!?! Noah Silverman <noah@allresearch.com>
  2003-01-25 00:03 ` Re: WEIRD CRASH?!?! Josh Berkus <josh@agliodbs.com>
  2003-01-25 00:27   ` Re: WEIRD CRASH?!?! Stephan Szabo <sszabo@megazone23.bigpanda.com>
@ 2003-01-25 01:12     ` Tom Lane <tgl@sss.pgh.pa.us>
  2003-01-25 03:10       ` Re: WEIRD CRASH?!?! Tom Lane <tgl@sss.pgh.pa.us>
  0 siblings, 1 reply; 15+ messages in thread

From: Tom Lane @ 2003-01-25 01:12 UTC (permalink / raw)
  To: Stephan Szabo <sszabo@megazone23.bigpanda.com>; +Cc: Josh Berkus <josh@agliodbs.com>; Noah Silverman <noah@allresearch.com>; pgsql-performance

Stephan Szabo <sszabo@megazone23.bigpanda.com> writes:
> The question is why did it get confused and end up linking to a filename
> that already existed?

The message comes from InstallXLogFileSegment(), which is careful to
ensure that the link() cannot fail, either by unlinking the previous
file, or searching for an unused name.  But it failed anyway.

It seems to me that there are only two possible explanations: a race
condition (but holding ControlFileLock should prevent that) or 
BasicOpenFile() failed for a reason other than nonexistence of the file.

Hmm ... I wonder if Noah's machine could have been running out of kernel
file table slots, or something like that?  It does seem that it'd be
more robust to use something like stat(2) to probe for an existing file.

			regards, tom lane



^ permalink  raw  reply  [nested|flat] 15+ messages in thread

* Re: WEIRD CRASH?!?!
  2003-01-24 23:59 WEIRD CRASH?!?! Noah Silverman <noah@allresearch.com>
  2003-01-25 00:03 ` Re: WEIRD CRASH?!?! Josh Berkus <josh@agliodbs.com>
  2003-01-25 00:27   ` Re: WEIRD CRASH?!?! Stephan Szabo <sszabo@megazone23.bigpanda.com>
  2003-01-25 01:12     ` Re: WEIRD CRASH?!?! Tom Lane <tgl@sss.pgh.pa.us>
@ 2003-01-25 03:10       ` Tom Lane <tgl@sss.pgh.pa.us>
  0 siblings, 0 replies; 15+ messages in thread

From: Tom Lane @ 2003-01-25 03:10 UTC (permalink / raw)
  To: Stephan Szabo <sszabo@megazone23.bigpanda.com>; Josh Berkus <josh@agliodbs.com>; Noah Silverman <noah@allresearch.com>; pgsql-performance

I wrote:
> Hmm ... I wonder if Noah's machine could have been running out of kernel
> file table slots, or something like that?  It does seem that it'd be
> more robust to use something like stat(2) to probe for an existing file.

I've applied a patch to do it that way in CVS HEAD.  After examining the
code further I'm inclined not to risk back-patching it into 7.3, though.
xlog.c is full of open() calls that will elog(PANIC) if they fail, so
I think there was only a very small window of opportunity for Noah to
see this failure and not another one.  The patch thus probably
contributes little real gain in reliability.

			regards, tom lane



^ permalink  raw  reply  [nested|flat] 15+ messages in thread


end of thread, other threads:[~2003-01-25 18:44 UTC | newest]

Thread overview: 15+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2003-01-24 23:59 WEIRD CRASH?!?! Noah Silverman <noah@allresearch.com>
2003-01-25 00:03 ` Josh Berkus <josh@agliodbs.com>
2003-01-25 00:08   ` Noah Silverman <noah@allresearch.com>
2003-01-25 00:13     ` scott.marlowe <scott.marlowe@ihs.com>
2003-01-25 00:15     ` Josh Berkus <josh@agliodbs.com>
2003-01-25 00:17       ` Noah Silverman <noah@allresearch.com>
2003-01-25 01:04         ` Tom Lane <tgl@sss.pgh.pa.us>
2003-01-25 01:08         ` Rod Taylor <rbt@rbt.ca>
2003-01-25 01:14           ` Noah Silverman <noah@allresearch.com>
2003-01-25 01:20             ` Rod Taylor <rbt@rbt.ca>
2003-01-25 01:38             ` Tom Lane <tgl@sss.pgh.pa.us>
2003-01-25 18:44     ` Andrew Sullivan <andrew@libertyrms.info>
2003-01-25 00:27   ` Stephan Szabo <sszabo@megazone23.bigpanda.com>
2003-01-25 01:12     ` Tom Lane <tgl@sss.pgh.pa.us>
2003-01-25 03:10       ` Tom Lane <tgl@sss.pgh.pa.us>

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox