From: Dzmitry <dzmitry.nikitsin@gmail.com>
To: Frank Lanitz <frank@frank.uvena.de>
To: pgsql-admin@postgresql.org
Subject: Re: postgres crashes on insert in 40 different threads
Date: Mon, 19 Aug 2013 18:07:48 +0300
Message-ID: <CE380E09.1199F%dzmitry.nikitsin@gmail.com> (raw)
In-Reply-To: <521232C2.1060801@frank.uvena.de>
List-Unsubscribe: <mailto:majordomo@postgresql.org?body=unsub%20pgsql-admin>
Yes- I need so many threads, every night I need load jobs from xml to DB.
I need do it as fast as I can, currently it take 4h to load all of
them(around 1000000 jobs).
CPU/IO wait/percent is about 25%. Do you know how can I check other params
related to IO wait ?
Thanks,
Dzmitry
On 8/19/13 5:59 PM, "Frank Lanitz" <frank@frank.uvena.de> wrote:
>Am 19.08.2013 10:07, schrieb Dzmitry:
>> I am doing a lot of writes to DB in 40 different threads so every
>> thread check if record exists if not => insert record, if exists =>
>> update record.
>> During this update, my disk IO almost always 100% and sometimes it
>> crash my DB with following message:
>
>Is this really needed to have so many threads and chances for race
>conditions?
>
>Beside the mention RAM thing you box might have to low IO capacities for
>this kind of trnasactions. What is the load and in detail the iowait
>saying?
>
>Cheers,
>Frank
>
>
>--
>Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
>To make changes to your subscription:
>http://www.postgresql.org/mailpref/pgsql-admin
--
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-admin@postgresql.org
Cc: dzmitry.nikitsin@gmail.com, frank@frank.uvena.de
Subject: Re: postgres crashes on insert in 40 different threads
In-Reply-To: <CE380E09.1199F%dzmitry.nikitsin@gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox