From: Stephane Bailliez <sbailliez@gmail.com>
Cc: pgsql-performance@postgresql.org
Subject: Re: ERROR: "invalid memory alloc request size" or "unexpected end of data" on large table
Date: Tue, 13 Nov 2007 16:06:50 +0100
Message-ID: <4739BD8A.2080203@gmail.com> (raw)
In-Reply-To: <13448.1194908062@sss.pgh.pa.us>
References: <4738B03C.8090901@gmail.com>
<13448.1194908062@sss.pgh.pa.us>
Tom Lane wrote:
> This sounds like corrupt data --- specifically, 1664639562 showing
> up where a variable-width field's length word ought to be. It
> may or may not be relevant that the ASCII equivalent of that bit
> pattern is Jb8c ... do you work with data that contains such
> substrings?
>
Not specifically but I cannot rule it out entirely, it 'could' be in one
of the column which may have a combination of uppercase/lowercase/number
otherwise all other text entries would be lowercase.
> If it's not 100% repeatable I'd start to wonder about flaky hardware.
> Have you run memory and disk diagnostics on this machine recently?
I did extensive tests a month or two ago (long crunching queries running
non stop for 24h) which were ok but honestly cannot say I'm not very
trusty in this particular hardware. Would need to put it offline and
memtest it for good obviously.
I moved some data (and a bit more, same table has 35M rows) to another
machine (also 8.1.10 on ubuntu with 2.6.17-10 smp, 2 Xeon 2GHZ instead
of P4 3GHz) and it passes with flying colors (though it's much much
slower with the same settings so I need to check a few things in there,
I had tmp dir topping 3GB so not sure if I could have more in memory)
Thanks for the insight, Tom.
-- stephane
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: sbailliez@gmail.com
Subject: Re: ERROR: "invalid memory alloc request size" or "unexpected end of data" on large table
In-Reply-To: <4739BD8A.2080203@gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox