pg.ddx.io pgsql-admin@postgresql.org mailing list archive
help / color / mirror / Atom feedPre-Allocate tablespace on disk
6+ messages / 6 participants
[nested] [flat]
* Pre-Allocate tablespace on disk
@ 2010-11-15 19:42 Chris Ruprecht <chris@ruprecht.org>
2010-11-15 20:58 ` Re: Pre-Allocate tablespace on disk Guillaume Lelarge <guillaume@lelarge.info>
2010-11-15 21:49 ` Re: Pre-Allocate tablespace on disk Craig James <craig_james@emolecules.com>
2010-11-19 13:01 ` Re: Pre-Allocate tablespace on disk Florian Weimer <fweimer@bfk.de>
2010-11-26 20:39 ` Re: Pre-Allocate tablespace on disk Dimitri Fontaine <dimitri@2ndQuadrant.fr>
0 siblings, 4 replies; 6+ messages in thread
From: Chris Ruprecht @ 2010-11-15 19:42 UTC (permalink / raw)
To: pgsql-admin
Hello,
I was wondering if there is a way to pre-allocate tablespace on disk before adding data and indexes.
My understanding is:
PG writes data into files sequentially. If more space is needed, disk space is requested from the OS and if there is space, the OS will give PG a file system block and PG will write data/indexes until that block is full and so on.
Other database that I have worked with before and that I'm still working with, allow you to pre-allocate disk space so you get huge chunks of contiguous space at one, which has major impacts on database performance.
Can this be done with PG at all? If not, is this planned for any time in the future?
best regards,
chris
--
chris ruprecht
database grunt and bit pusher extraordinaíre
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: Pre-Allocate tablespace on disk
2010-11-15 19:42 Pre-Allocate tablespace on disk Chris Ruprecht <chris@ruprecht.org>
@ 2010-11-15 20:58 ` Guillaume Lelarge <guillaume@lelarge.info>
3 siblings, 0 replies; 6+ messages in thread
From: Guillaume Lelarge @ 2010-11-15 20:58 UTC (permalink / raw)
To: Chris Ruprecht <chris@ruprecht.org>; +Cc: pgsql-admin
Le 15/11/2010 20:42, Chris Ruprecht a écrit :
> Hello,
>
> I was wondering if there is a way to pre-allocate tablespace on disk before adding data and indexes.
> My understanding is:
> PG writes data into files sequentially. If more space is needed, disk space is requested from the OS and if there is space, the OS will give PG a file system block and PG will write data/indexes until that block is full and so on.
> Other database that I have worked with before and that I'm still working with, allow you to pre-allocate disk space so you get huge chunks of contiguous space at one, which has major impacts on database performance.
>
> Can this be done with PG at all?
No. A tablespace is a directory on PostgreSQL. And you can't allocate
space for a directory.
> If not, is this planned for any time in the future?
AFAICT, nope.
--
Guillaume
http://www.postgresql.fr
http://dalibo.com
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: Pre-Allocate tablespace on disk
2010-11-15 19:42 Pre-Allocate tablespace on disk Chris Ruprecht <chris@ruprecht.org>
@ 2010-11-15 21:49 ` Craig James <craig_james@emolecules.com>
3 siblings, 0 replies; 6+ messages in thread
From: Craig James @ 2010-11-15 21:49 UTC (permalink / raw)
To: pgsql-admin
On 11/15/10 11:42 AM, Chris Ruprecht wrote:
> I was wondering if there is a way to pre-allocate tablespace on disk before
> adding data and indexes.
> My understanding is:
> PG writes data into files sequentially. If more space is needed, disk space
> is requested from the OS and if there is space, the OS will give PG a file
> system block and PG will write data/indexes until that block is full and so on.
> Other database that I have worked with before and that I'm still working with,
> allow you to pre-allocate disk space so you get huge chunks of contiguous
> space at one, which has major impacts on database performance.
This shouldn't really be a performance issue on modern Unix-type systems. Most file systems have fairly smart block-allocation algorithms that tend to allocate large blocks of contiguous space to files even when lots of processes are asking for little chunks in interleaved requests. It doesn't just allocate a block to the next guy who happens to ask. It's a lot smarter than that. In most cases, you'll barely be able to tell the difference between a file that was allocated all at once and one that was allocated on an as-needed basis. The only time you get bad fragmentation is when your disk gets nearly full, which takes away the file system's ability to be clever.
If you really want to preallocate a file, build a separate partition on your disk and only put one database (or even just one table) on that file system.
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: Pre-Allocate tablespace on disk
2010-11-15 19:42 Pre-Allocate tablespace on disk Chris Ruprecht <chris@ruprecht.org>
@ 2010-11-19 13:01 ` Florian Weimer <fweimer@bfk.de>
2010-11-19 19:42 ` Re: Pre-Allocate tablespace on disk Steve Francis <sfrancis@logicmonitor.com>
3 siblings, 1 reply; 6+ messages in thread
From: Florian Weimer @ 2010-11-19 13:01 UTC (permalink / raw)
To: Chris Ruprecht <chris@ruprecht.org>; +Cc: pgsql-admin
* Chris Ruprecht:
> Other database that I have worked with before and that I'm still
> working with, allow you to pre-allocate disk space so you get huge
> chunks of contiguous space at one, which has major impacts on
> database performance.
PostgreSQL's write patterns do not trigger significant fragmentation
with most file systems, even when other database systems (Oracle
Berkeley DB comes to my mind) would create heavily fragmented files.
--
Florian Weimer <fweimer@bfk.de>
BFK edv-consulting GmbH http://www.bfk.de/
Kriegsstraße 100 tel: +49-721-96201-1
D-76133 Karlsruhe fax: +49-721-96201-99
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: Pre-Allocate tablespace on disk
2010-11-15 19:42 Pre-Allocate tablespace on disk Chris Ruprecht <chris@ruprecht.org>
2010-11-19 13:01 ` Re: Pre-Allocate tablespace on disk Florian Weimer <fweimer@bfk.de>
@ 2010-11-19 19:42 ` Steve Francis <sfrancis@logicmonitor.com>
0 siblings, 0 replies; 6+ messages in thread
From: Steve Francis @ 2010-11-19 19:42 UTC (permalink / raw)
To: pgsql-admin
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Not only that, but if your "disk" is really a bunch of disks behind
a hardware RAID controller, or SAN, or iSCSI or NFS server - what is
contiguous for the OS may not be contiguous on the physical drives
(and vice versa). <br>
Which is why performance can (but may not) improve if you disable
linux kernel IO schedulers that attempt to optimize blocks to be
sequential.<br>
<br>
<br>
<br>
On 11/19/2010 5:01 AM, Florian Weimer wrote:
<blockquote cite="mid:82zkt55ux4.fsf@mid.bfk.de" type="cite">
<pre wrap="">* Chris Ruprecht:
</pre>
<blockquote type="cite">
<pre wrap="">Other database that I have worked with before and that I'm still
working with, allow you to pre-allocate disk space so you get huge
chunks of contiguous space at one, which has major impacts on
database performance.
</pre>
</blockquote>
<pre wrap="">
PostgreSQL's write patterns do not trigger significant fragmentation
with most file systems, even when other database systems (Oracle
Berkeley DB comes to my mind) would create heavily fragmented files.
</pre>
</blockquote>
<br>
<br>
<div class="moz-signature">-- <br>
<style type="text/css"> .sig, .sig a, .sig td { font: 12px Arial; color: DarkBlue; white-space: nowrap; vertical-align: top; text-align: left; } .sig img { border: 0; } .sig a { font: 11px Arial; } .sig .back { color: #ffffff; background-color: DarkBlue; text-decoration: none; } .sig .right { text-align: right; } .sig .border { border: 1px solid DarkBlue; margin:5px; border-collapse: collapse; } .sig .bottom { vertical-align:bottom } .sig p { clear:both; margin:2px; } </style>
<div class="sig">
<table class="border">
<tbody>
<tr>
<td class="back"> <b>Steve Francis</b></td>
<td class="back" style="width: 30px;"><br>
</td>
<td class="back" style="text-align: right;"><b>LogicMonitor
LLC</b></td>
</tr>
<tr>
<td><br>
</td>
<td><br>
</td>
<td class="right"><br>
</td>
</tr>
<tr>
<td class="bottom"><a class="moz-txt-link-abbreviated" href="mailto:sfrancis@logicmonitor.com">sfrancis@logicmonitor.com</a><br>
<a target="_blank" href="http://www.logicmonitor.com">www.logicmonitor.com</a></td;
<td><br>
</td>
<td class="bottom" style="text-align: right;"> Ph: 1 888
41 LOGIC x500<br>
Ph: 1 805 698 0770</td>
</tr>
</tbody>
</table>
<p> <br>
</p>
</div>
</div>
</body>
</html>
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: Pre-Allocate tablespace on disk
2010-11-15 19:42 Pre-Allocate tablespace on disk Chris Ruprecht <chris@ruprecht.org>
@ 2010-11-26 20:39 ` Dimitri Fontaine <dimitri@2ndQuadrant.fr>
3 siblings, 0 replies; 6+ messages in thread
From: Dimitri Fontaine @ 2010-11-26 20:39 UTC (permalink / raw)
To: Chris Ruprecht <chris@ruprecht.org>; +Cc: pgsql-admin
Chris Ruprecht <chris@ruprecht.org> writes:
> I was wondering if there is a way to pre-allocate tablespace on disk
> before adding data and indexes.
As others have already said, it's not clear there's a point in doing
that with PostgreSQL. Now, you can achieve it and see for yourself by
relying on OS tools, such as LVM of ZFS, or some other tricks.
For example, create a big enough file, loop mount it as a device, and
create your tablespace in there. Of course what to do exactly when the
file is full is left as an exercise to the reader. Bonus points for any
solution not involving downtime.
Baseline: it seems that PostgreSQL works differently enough from your
usual software than you may want to reconsider your habits :)
Regards,
--
Dimitri Fontaine
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support
^ permalink raw reply [nested|flat] 6+ messages in thread
end of thread, other threads:[~2010-11-26 20:39 UTC | newest]
Thread overview: 6+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2010-11-15 19:42 Pre-Allocate tablespace on disk Chris Ruprecht <chris@ruprecht.org>
2010-11-15 20:58 ` Guillaume Lelarge <guillaume@lelarge.info>
2010-11-15 21:49 ` Craig James <craig_james@emolecules.com>
2010-11-19 13:01 ` Florian Weimer <fweimer@bfk.de>
2010-11-19 19:42 ` Steve Francis <sfrancis@logicmonitor.com>
2010-11-26 20:39 ` Dimitri Fontaine <dimitri@2ndQuadrant.fr>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox