pg.ddx.io  pgsql-admin@postgresql.org mailing list archive  
help / color / mirror / Atom feed
Runtime error: could not open segment 1 of relation ...
7+ messages / 3 participants
[nested] [flat]

* Runtime error: could not open segment 1 of relation ...
@ 2006-09-26 21:54 Gregory S. Williamson <gsw@globexplorer.com>
  2006-09-26 22:21 ` Re: Runtime error: could not open segment 1 of relation ... Talha Khan <talha.amjad@gmail.com>
  0 siblings, 1 reply; 7+ messages in thread

From: Gregory S. Williamson @ 2006-09-26 21:54 UTC (permalink / raw)
  To: pgsql-admin

This morning one of our production servers (8.1.1, Linux) threw a number of  errors like these (different final number; first two [1663/43801] are all the same]:

2006-09-26 10:02:15.124 PDT 1387333718 ERROR:  could not open segment 1 of relation 1663/43801/7274801 (target block 478216192): No such file or directory
2006-09-26 10:02:52.991 PDT 1387340456 ERROR:  could not open segment 1 of relation 1663/43801/7274801 (target block 478216192): No such file or directory
2006-09-26 10:02:53.002 PDT 1387340458 ERROR:  could not open segment 1 of relation 1663/43801/7274801 (target block 478216192): No such file or directory

The server never really recovered -- it kept running but was horribly slow. Rebooting seems to have cured the problem, but I am wondering what I made look into, etc.

The server in question is really a read-only beast with several hundred connections, typically, 4 gigs of RAM, and has not misbehaved before.

Any suggestions would be welcome! (Although I fear I know the answer -- disk issues.)

Thanks,

Greg Williamson
DBA
GlobeXplorer LLC



^ permalink  raw  reply  [nested|flat] 7+ messages in thread

* Re: Runtime error: could not open segment 1 of relation ...
  2006-09-26 21:54 Runtime error: could not open segment 1 of relation ... Gregory S. Williamson <gsw@globexplorer.com>
@ 2006-09-26 22:21 ` Talha Khan <talha.amjad@gmail.com>
  2006-09-26 22:28   ` Re: Runtime error: could not open segment 1 of relation ... Gregory S. Williamson <gsw@globexplorer.com>
  0 siblings, 1 reply; 7+ messages in thread

From: Talha Khan @ 2006-09-26 22:21 UTC (permalink / raw)
  To: Gregory S. Williamson <gsw@globexplorer.com>; +Cc: pgsql-admin

Hi Greg,

Is it possible that the server was shutdown using -mi switch?

cuz immediate shutdown ends causing these problems.

Regards
Talha Khan


On 9/26/06, Gregory S. Williamson <gsw@globexplorer.com> wrote:
>
> This morning one of our production servers (8.1.1, Linux) threw a number
> of  errors like these (different final number; first two [1663/43801] are
> all the same]:
>
> 2006-09-26 10:02:15.124 PDT 1387333718 ERROR:  could not open segment 1 of
> relation 1663/43801/7274801 (target block 478216192): No such file or
> directory
> 2006-09-26 10:02:52.991 PDT 1387340456 ERROR:  could not open segment 1 of
> relation 1663/43801/7274801 (target block 478216192): No such file or
> directory
> 2006-09-26 10:02:53.002 PDT 1387340458 ERROR:  could not open segment 1 of
> relation 1663/43801/7274801 (target block 478216192): No such file or
> directory
>
> The server never really recovered -- it kept running but was horribly
> slow. Rebooting seems to have cured the problem, but I am wondering what I
> made look into, etc.
>
> The server in question is really a read-only beast with several hundred
> connections, typically, 4 gigs of RAM, and has not misbehaved before.
>
> Any suggestions would be welcome! (Although I fear I know the answer --
> disk issues.)
>
> Thanks,
>
> Greg Williamson
> DBA
> GlobeXplorer LLC
>
> ---------------------------(end of broadcast)---------------------------
> TIP 2: Don't 'kill -9' the postmaster
>

^ permalink  raw  reply  [nested|flat] 7+ messages in thread

* Re: Runtime error: could not open segment 1 of relation ...
  2006-09-26 21:54 Runtime error: could not open segment 1 of relation ... Gregory S. Williamson <gsw@globexplorer.com>
  2006-09-26 22:21 ` Re: Runtime error: could not open segment 1 of relation ... Talha Khan <talha.amjad@gmail.com>
@ 2006-09-26 22:28   ` Gregory S. Williamson <gsw@globexplorer.com>
  2006-09-26 22:33     ` Re: Runtime error: could not open segment 1 of relation ... Talha Khan <talha.amjad@gmail.com>
  2006-09-26 22:36     ` Re: Runtime error: could not open segment 1 of relation ... Tom Lane <tgl@sss.pgh.pa.us>
  0 siblings, 2 replies; 7+ messages in thread

From: Gregory S. Williamson @ 2006-09-26 22:28 UTC (permalink / raw)
  To: Talha Khan <talha.amjad@gmail.com>; +Cc: pgsql-admin

Thanks for the suggestion ...no evidence of such a shutdown. The first of these errors (since 2005-10-28) was at 2006-09-26 05:37:19.157 PDT today; machine was last shut serveral weeks ago. There's been several large updates since then (sort of batch process for updates, then the server is read-only for 99.999 of the time).

Greg

-----Original Message-----
From:	Talha Khan [mailto:talha.amjad@gmail.com]
Sent:	Tue 9/26/2006 3:21 PM
To:	Gregory S. Williamson
Cc:	pgsql-admin@postgresql.org
Subject:	Re: [ADMIN] Runtime error: could not open segment 1 of relation ...

Hi Greg,

Is it possible that the server was shutdown using -mi switch?

cuz immediate shutdown ends causing these problems.

Regards
Talha Khan


On 9/26/06, Gregory S. Williamson <gsw@globexplorer.com> wrote:
>
> This morning one of our production servers (8.1.1, Linux) threw a number
> of  errors like these (different final number; first two [1663/43801] are
> all the same]:
>
> 2006-09-26 10:02:15.124 PDT 1387333718 ERROR:  could not open segment 1 of
> relation 1663/43801/7274801 (target block 478216192): No such file or
> directory
> 2006-09-26 10:02:52.991 PDT 1387340456 ERROR:  could not open segment 1 of
> relation 1663/43801/7274801 (target block 478216192): No such file or
> directory
> 2006-09-26 10:02:53.002 PDT 1387340458 ERROR:  could not open segment 1 of
> relation 1663/43801/7274801 (target block 478216192): No such file or
> directory
>
> The server never really recovered -- it kept running but was horribly
> slow. Rebooting seems to have cured the problem, but I am wondering what I
> made look into, etc.
>
> The server in question is really a read-only beast with several hundred
> connections, typically, 4 gigs of RAM, and has not misbehaved before.
>
> Any suggestions would be welcome! (Although I fear I know the answer --
> disk issues.)
>
> Thanks,
>
> Greg Williamson
> DBA
> GlobeXplorer LLC
>
> ---------------------------(end of broadcast)---------------------------
> TIP 2: Don't 'kill -9' the postmaster
>



-------------------------------------------------------
Click link below if it is SPAM gsw@globexplorer.com
"https://mailscanner.globexplorer.com/dspam/dspam.cgi?signatureID=4519a6f898846491211187&user=gsw...;
!DSPAM:4519a6f898846491211187!
-------------------------------------------------------






^ permalink  raw  reply  [nested|flat] 7+ messages in thread

* Re: Runtime error: could not open segment 1 of relation ...
  2006-09-26 21:54 Runtime error: could not open segment 1 of relation ... Gregory S. Williamson <gsw@globexplorer.com>
  2006-09-26 22:21 ` Re: Runtime error: could not open segment 1 of relation ... Talha Khan <talha.amjad@gmail.com>
  2006-09-26 22:28   ` Re: Runtime error: could not open segment 1 of relation ... Gregory S. Williamson <gsw@globexplorer.com>
@ 2006-09-26 22:33     ` Talha Khan <talha.amjad@gmail.com>
  2006-09-26 23:23       ` Re: Runtime error: could not open segment 1 of relation ... Gregory S. Williamson <gsw@globexplorer.com>
  1 sibling, 1 reply; 7+ messages in thread

From: Talha Khan @ 2006-09-26 22:33 UTC (permalink / raw)
  To: Gregory S. Williamson <gsw@globexplorer.com>; +Cc: pgsql-admin

Try reindexing your database and see if that solves your problem.

Regards
Talha Khan


On 9/26/06, Gregory S. Williamson <gsw@globexplorer.com> wrote:
>
> Thanks for the suggestion ...no evidence of such a shutdown. The first of
> these errors (since 2005-10-28) was at 2006-09-26 05:37:19.157 PDT today;
> machine was last shut serveral weeks ago. There's been several large updates
> since then (sort of batch process for updates, then the server is read-only
> for 99.999 of the time).
>
> Greg
>
> -----Original Message-----
> From:   Talha Khan [mailto:talha.amjad@gmail.com]
> Sent:   Tue 9/26/2006 3:21 PM
> To:     Gregory S. Williamson
> Cc:     pgsql-admin@postgresql.org
> Subject:        Re: [ADMIN] Runtime error: could not open segment 1 of
> relation ...
>
> Hi Greg,
>
> Is it possible that the server was shutdown using -mi switch?
>
> cuz immediate shutdown ends causing these problems.
>
> Regards
> Talha Khan
>
>
> On 9/26/06, Gregory S. Williamson <gsw@globexplorer.com> wrote:
> >
> > This morning one of our production servers (8.1.1, Linux) threw a number
> > of  errors like these (different final number; first two [1663/43801]
> are
> > all the same]:
> >
> > 2006-09-26 10:02:15.124 PDT 1387333718 ERROR:  could not open segment 1
> of
> > relation 1663/43801/7274801 (target block 478216192): No such file or
> > directory
> > 2006-09-26 10:02:52.991 PDT 1387340456 ERROR:  could not open segment 1
> of
> > relation 1663/43801/7274801 (target block 478216192): No such file or
> > directory
> > 2006-09-26 10:02:53.002 PDT 1387340458 ERROR:  could not open segment 1
> of
> > relation 1663/43801/7274801 (target block 478216192): No such file or
> > directory
> >
> > The server never really recovered -- it kept running but was horribly
> > slow. Rebooting seems to have cured the problem, but I am wondering what
> I
> > made look into, etc.
> >
> > The server in question is really a read-only beast with several hundred
> > connections, typically, 4 gigs of RAM, and has not misbehaved before.
> >
> > Any suggestions would be welcome! (Although I fear I know the answer --
> > disk issues.)
> >
> > Thanks,
> >
> > Greg Williamson
> > DBA
> > GlobeXplorer LLC
> >
> > ---------------------------(end of broadcast)---------------------------
> > TIP 2: Don't 'kill -9' the postmaster
> >
>
>
>
> -------------------------------------------------------
> Click link below if it is SPAM gsw@globexplorer.com
> "
> https://mailscanner.globexplorer.com/dspam/dspam.cgi?signatureID=4519a6f898846491211187&user=gsw...
> "
> !DSPAM:4519a6f898846491211187!
> -------------------------------------------------------
>
>
>
>

^ permalink  raw  reply  [nested|flat] 7+ messages in thread

* Re: Runtime error: could not open segment 1 of relation ...
  2006-09-26 21:54 Runtime error: could not open segment 1 of relation ... Gregory S. Williamson <gsw@globexplorer.com>
  2006-09-26 22:21 ` Re: Runtime error: could not open segment 1 of relation ... Talha Khan <talha.amjad@gmail.com>
  2006-09-26 22:28   ` Re: Runtime error: could not open segment 1 of relation ... Gregory S. Williamson <gsw@globexplorer.com>
  2006-09-26 22:33     ` Re: Runtime error: could not open segment 1 of relation ... Talha Khan <talha.amjad@gmail.com>
@ 2006-09-26 23:23       ` Gregory S. Williamson <gsw@globexplorer.com>
  0 siblings, 0 replies; 7+ messages in thread

From: Gregory S. Williamson @ 2006-09-26 23:23 UTC (permalink / raw)
  To: Talha Khan <talha.amjad@gmail.com>; +Cc: pgsql-admin

Talha,

Seems like a plan ... once we have it out of runtime I'll be able to redo indexes, whatever.

Thanks,

Greg

-----Original Message-----
From:	Talha Khan [mailto:talha.amjad@gmail.com]
Sent:	Tue 9/26/2006 3:33 PM
To:	Gregory S. Williamson
Cc:	pgsql-admin@postgresql.org
Subject:	Re: [ADMIN] Runtime error: could not open segment 1 of relation ...

Try reindexing your database and see if that solves your problem.

Regards
Talha Khan


On 9/26/06, Gregory S. Williamson <gsw@globexplorer.com> wrote:
>
> Thanks for the suggestion ...no evidence of such a shutdown. The first of
> these errors (since 2005-10-28) was at 2006-09-26 05:37:19.157 PDT today;
> machine was last shut serveral weeks ago. There's been several large updates
> since then (sort of batch process for updates, then the server is read-only
> for 99.999 of the time).
>
> Greg
>
> -----Original Message-----
> From:   Talha Khan [mailto:talha.amjad@gmail.com]
> Sent:   Tue 9/26/2006 3:21 PM
> To:     Gregory S. Williamson
> Cc:     pgsql-admin@postgresql.org
> Subject:        Re: [ADMIN] Runtime error: could not open segment 1 of
> relation ...
>
> Hi Greg,
>
> Is it possible that the server was shutdown using -mi switch?
>
> cuz immediate shutdown ends causing these problems.
>
> Regards
> Talha Khan
>
>
> On 9/26/06, Gregory S. Williamson <gsw@globexplorer.com> wrote:
> >
> > This morning one of our production servers (8.1.1, Linux) threw a number
> > of  errors like these (different final number; first two [1663/43801]
> are
> > all the same]:
> >
> > 2006-09-26 10:02:15.124 PDT 1387333718 ERROR:  could not open segment 1
> of
> > relation 1663/43801/7274801 (target block 478216192): No such file or
> > directory
> > 2006-09-26 10:02:52.991 PDT 1387340456 ERROR:  could not open segment 1
> of
> > relation 1663/43801/7274801 (target block 478216192): No such file or
> > directory
> > 2006-09-26 10:02:53.002 PDT 1387340458 ERROR:  could not open segment 1
> of
> > relation 1663/43801/7274801 (target block 478216192): No such file or
> > directory
> >
> > The server never really recovered -- it kept running but was horribly
> > slow. Rebooting seems to have cured the problem, but I am wondering what
> I
> > made look into, etc.
> >
> > The server in question is really a read-only beast with several hundred
> > connections, typically, 4 gigs of RAM, and has not misbehaved before.
> >
> > Any suggestions would be welcome! (Although I fear I know the answer --
> > disk issues.)
> >
> > Thanks,
> >
> > Greg Williamson
> > DBA
> > GlobeXplorer LLC
> >
> > ---------------------------(end of broadcast)---------------------------
> > TIP 2: Don't 'kill -9' the postmaster
> >
>
>
>
> -------------------------------------------------------
> Click link below if it is SPAM gsw@globexplorer.com
> "
> https://mailscanner.globexplorer.com/dspam/dspam.cgi?signatureID=4519a6f898846491211187&user=gsw...
> "
> 
> -------------------------------------------------------
>
>
>
>



-------------------------------------------------------
Click link below if it is SPAM gsw@globexplorer.com
"https://mailscanner.globexplorer.com/dspam/dspam.cgi?signatureID=4519a9a399831804284693&user=gsw...;
!DSPAM:4519a9a399831804284693!
-------------------------------------------------------






^ permalink  raw  reply  [nested|flat] 7+ messages in thread

* Re: Runtime error: could not open segment 1 of relation ...
  2006-09-26 21:54 Runtime error: could not open segment 1 of relation ... Gregory S. Williamson <gsw@globexplorer.com>
  2006-09-26 22:21 ` Re: Runtime error: could not open segment 1 of relation ... Talha Khan <talha.amjad@gmail.com>
  2006-09-26 22:28   ` Re: Runtime error: could not open segment 1 of relation ... Gregory S. Williamson <gsw@globexplorer.com>
@ 2006-09-26 22:36     ` Tom Lane <tgl@sss.pgh.pa.us>
  2006-09-26 23:22       ` Re: Runtime error: could not open segment 1 of relation ... Gregory S. Williamson <gsw@globexplorer.com>
  1 sibling, 1 reply; 7+ messages in thread

From: Tom Lane @ 2006-09-26 22:36 UTC (permalink / raw)
  To: Gregory S. Williamson <gsw@globexplorer.com>; +Cc: Talha Khan <talha.amjad@gmail.com>; pgsql-admin

"Gregory S. Williamson" <gsw@globexplorer.com> writes:
>> 2006-09-26 10:02:15.124 PDT 1387333718 ERROR:  could not open segment 1 of
>> relation 1663/43801/7274801 (target block 478216192): No such file or
>> directory

This is most likely caused by trying to follow a bogus item pointer in
an index.  If the problem went away after a restart, the bogus value
could not have been on disk but must have been in a bad in-memory copy
of an index page.  Which suggests a hardware glitch ... although if you
are using contrib/ltree, I recall seeing several bug fixes go by
recently that were described as fixing potential ltree-index corruption
issues.

			regards, tom lane



^ permalink  raw  reply  [nested|flat] 7+ messages in thread

* Re: Runtime error: could not open segment 1 of relation ...
  2006-09-26 21:54 Runtime error: could not open segment 1 of relation ... Gregory S. Williamson <gsw@globexplorer.com>
  2006-09-26 22:21 ` Re: Runtime error: could not open segment 1 of relation ... Talha Khan <talha.amjad@gmail.com>
  2006-09-26 22:28   ` Re: Runtime error: could not open segment 1 of relation ... Gregory S. Williamson <gsw@globexplorer.com>
  2006-09-26 22:36     ` Re: Runtime error: could not open segment 1 of relation ... Tom Lane <tgl@sss.pgh.pa.us>
@ 2006-09-26 23:22       ` Gregory S. Williamson <gsw@globexplorer.com>
  0 siblings, 0 replies; 7+ messages in thread

From: Gregory S. Williamson @ 2006-09-26 23:22 UTC (permalink / raw)
  To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Talha Khan <talha.amjad@gmail.com>; pgsql-admin

Tom,

I haven't used the ltree module ... I'll take a look at our DB and see. It's possible that this was an index problem as there was a bulk update with indexes in tact, so there would have been a lot of index shuffling, and the db was not shut down since that update.

Thanks,

Greg


-----Original Message-----
From:	Tom Lane [mailto:tgl@sss.pgh.pa.us]
Sent:	Tue 9/26/2006 3:36 PM
To:	Gregory S. Williamson
Cc:	Talha Khan; pgsql-admin@postgresql.org
Subject:	Re: [ADMIN] Runtime error: could not open segment 1 of relation ... 

"Gregory S. Williamson" <gsw@globexplorer.com> writes:
>> 2006-09-26 10:02:15.124 PDT 1387333718 ERROR:  could not open segment 1 of
>> relation 1663/43801/7274801 (target block 478216192): No such file or
>> directory

This is most likely caused by trying to follow a bogus item pointer in
an index.  If the problem went away after a restart, the bogus value
could not have been on disk but must have been in a bad in-memory copy
of an index page.  Which suggests a hardware glitch ... although if you
are using contrib/ltree, I recall seeing several bug fixes go by
recently that were described as fixing potential ltree-index corruption
issues.

			regards, tom lane


-------------------------------------------------------
Click link below if it is SPAM gsw@globexplorer.com
"https://mailscanner.globexplorer.com/dspam/dspam.cgi?signatureID=4519aa77100151804284693&user=gs...;
!DSPAM:4519aa77100151804284693!
-------------------------------------------------------








^ permalink  raw  reply  [nested|flat] 7+ messages in thread


end of thread, other threads:[~2006-09-26 23:23 UTC | newest]

Thread overview: 7+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2006-09-26 21:54 Runtime error: could not open segment 1 of relation ... Gregory S. Williamson <gsw@globexplorer.com>
2006-09-26 22:21 ` Talha Khan <talha.amjad@gmail.com>
2006-09-26 22:28   ` Gregory S. Williamson <gsw@globexplorer.com>
2006-09-26 22:33     ` Talha Khan <talha.amjad@gmail.com>
2006-09-26 23:23       ` Gregory S. Williamson <gsw@globexplorer.com>
2006-09-26 22:36     ` Tom Lane <tgl@sss.pgh.pa.us>
2006-09-26 23:22       ` Gregory S. Williamson <gsw@globexplorer.com>

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox