From: Qingqing Zhou <zhouqq@cs.toronto.edu>
To: pgsql-performance@postgresql.org
Subject: Re: insert performance for win32
Date: Thu, 3 Nov 2005 18:30:12 -0500
Message-ID: <dke6hk$11ed$1@news.hub.org> (raw)
References: <6EE64EF3AB31D5448D0007DD34EEB3417DD79B@Herge.rcsinc.local>
<dke0ft$i6$1@news.hub.org>
"Qingqing Zhou" <zhouqq@cs.toronto.edu> wrote
>
> Not to 100%, so this means the server is always starving. It is waiting on
> something -- of couse not lock. That's why I think there is some problem
> on network communication. Another suspect will be the write - I knwo NTFS
> system will issue an internal log when extending a file. To remove the
> second suspect, I will try to hack the source to do a "fake" write ...
>
To patch:
-------------------------
Here is a quite straight hack to implement "fake" write for both relation
and xlog. Now the server becomes pure CPU play.
1. RelationGetBufferForTuple()/hio.c: remove line (if you do not enable
cassert, then doesn't matter):
- Assert(PageIsNew((PageHeader) pageHeader));
2. ReadBuffer()/bufmgr.c: remove line
- smgrextend(reln->rd_smgr, blockNum, (char *) bufBlock,
- reln->rd_istemp);
3. XLogWrite()/xlog.c
errno = 0;
+ goto fake;
if (write(openLogFile, from, nbytes) != nbytes)
{
/* if write didn't set errno, assume no disk space */
...
}
+
+ fake:
/* Update state for write */
To use it:
-------------------------
1. have several copies of a correct data;
2. patch the server;
3. when you startup postmaster, use the following parameters:
postmaster -c"checkpoint_timeout=3600" -c"bgwriter_all_percent=0" -Ddata
Note now the database server is one-shoot usable -- after you shutdown, it
won't startup again. Just run
begin;
many inserts;
end;
To observe:
-------------------------
(1) In this case, what's the remote server CPU usage -- 100%? I don't have
several machines to test it. In my single machine, I run 35000 insert
commands from psql by cut and paste into it and could observe that:
---
25% kernel time
75% user time
20% postgresql (--enable-debug --enable-cassert)
65% psql (as same above)
10% csrss (system process, manage graphics commands (not sure, just googled
it), etc)
5% system (system process)
---
(2) In this case, Linux still keeps almost 10 times faster?
After this, we may need more observations like comparison of simple "select
1;" to reduce the code space we may want to explore ...
Regards,
Qingqing
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: zhouqq@cs.toronto.edu
Subject: Re: insert performance for win32
In-Reply-To: <dke6hk$11ed$1@news.hub.org>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox