From: "Qingqing Zhou" <zhouqq@cs.toronto.edu>
Subject: Re: insert performance for win32
Date: Thu, 3 Nov 2005 18:30:12 -0500
Organization: Hub.Org Networking Services
Lines: 76
Message-ID: <dke6hk$11ed$1@news.hub.org>
References: <6EE64EF3AB31D5448D0007DD34EEB3417DD79B@Herge.rcsinc.local>
	<dke0ft$i6$1@news.hub.org>
To: pgsql-performance@postgresql.org


"Qingqing Zhou" <zhouqq@cs.toronto.edu> wrote
>
> Not to 100%, so this means the server is always starving. It is waiting on 
> something -- of couse not lock. That's why I think there is some problem 
> on network communication. Another suspect will be the write - I knwo NTFS 
> system will issue an internal log when extending a file. To remove the 
> second suspect, I will try to hack the source to do a "fake" write ...
>

To patch:
-------------------------
Here is a quite straight hack to implement "fake" write for both relation 
and xlog. Now the server becomes pure CPU play.

1. RelationGetBufferForTuple()/hio.c: remove line (if you do not enable 
cassert, then doesn't matter):
- Assert(PageIsNew((PageHeader) pageHeader));

2. ReadBuffer()/bufmgr.c: remove line
- smgrextend(reln->rd_smgr, blockNum, (char *) bufBlock,
-     reln->rd_istemp);

3. XLogWrite()/xlog.c
   errno = 0;
+  goto fake;
   if (write(openLogFile, from, nbytes) != nbytes)
   {
    /* if write didn't set errno, assume no disk space */
    ...
   }
+
+ fake:
   /* Update state for write */


To use it:
-------------------------
1. have several copies of a correct data;

2. patch the server;

3. when you startup postmaster, use the following parameters:
postmaster -c"checkpoint_timeout=3600" -c"bgwriter_all_percent=0" -Ddata

Note now the database server is one-shoot usable -- after you shutdown, it 
won't startup again. Just run
begin;
    many inserts;
end;

To observe:
-------------------------
(1) In this case, what's the remote server CPU usage -- 100%? I don't have 
several machines to test it. In my single machine, I run 35000 insert 
commands from psql by cut and paste into it and could observe that:
---
25% kernel time
75% user time

20% postgresql (--enable-debug --enable-cassert)
65% psql (as same above)
10% csrss (system process, manage graphics commands (not sure, just googled 
it), etc)
5%  system (system process)
---

(2) In this case, Linux still keeps almost 10 times faster?

After this, we may need more observations like comparison of simple "select 
1;" to reduce the code space we may want to explore ...

Regards,
Qingqing