From: Dilek Küçük <dilekkucuk@gmail.com>
To: Achilleas Mantzios <achill@matrix.gatewaynet.com>
Cc: pgsql-admin@postgresql.org
Subject: Re: max_files_per_process limit
Date: Tue, 11 Nov 2008 14:10:06 +0200
Message-ID: <5d5d98b30811110410o35620cdbref05a72f6974f9b8@mail.gmail.com> (raw)
In-Reply-To: <200811101651.16538.achill@matrix.gatewaynet.com>
References: <5d5d98b30811100618v2c0f935fka3f7970b622d6760@mail.gmail.com>
<200811101651.16538.achill@matrix.gatewaynet.com>
On Mon, Nov 10, 2008 at 4:51 PM, Achilleas Mantzios <
achill@matrix.gatewaynet.com> wrote:
> Στις Monday 10 November 2008 16:18:37 ο/η Dilek Küçük έγραψε:
> > Hi,
> >
> > We have a database of about 62000 tables (about 2000 tablespaces) with an
> > index on each table. Postgresql version is 8.1.
> >
>
> So you have about 62000 distinct schemata in your db?
> Imagine that the average enterprise has about 200 tables max,
> and an average sized country has about 300 such companies,
> including public sector, with 62000 tables you could blindly model
> .... the whole activity of a whole country.
>
> Is this some kind of replicated data?
> Whats the story?
> Just curious.
Actually we had 31 distinct tables but this amounted to tens of billions of
records (streaming data from 2000 sites) per table a year, so we
horizontally partition each table into 2000 tables. This allowed us to
discard one of the indexes that we have created and freed us from periodical
cluster operations which turned out to be infeasible for a system with tight
querying constraints in terms of time.
>
>
> > Although after the initial inserts to about 32000 tables the subsequent
> > inserts are considerable fast, subsequent inserts to more than 32000
> tables
> > are very slow.
> >
> > This seems to be due to the datatype (integer) of max_files_per_process
> > option in the postgres.conf file which is used to set the maximum number
> of
> > open file descriptors.
> > Is there anything we could do about this max_files_per_process limit or
> any
> > other way to speed up inserts to all these tables?
> >
> > Any suggestions are wellcome.
> >
> > Kind regards,
> > Dilek Küçük
> >
>
>
>
> --
> Achilleas Mantzios
>
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-admin@postgresql.org
Cc: dilekkucuk@gmail.com, achill@matrix.gatewaynet.com
Subject: Re: max_files_per_process limit
In-Reply-To: <5d5d98b30811110410o35620cdbref05a72f6974f9b8@mail.gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox