X-Original-To: pgsql-hackers-postgresql.org@localhost.postgresql.org Received: from localhost (av.hub.org [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id 7DBAEDA261 for ; Wed, 5 Oct 2005 16:43:58 -0300 (ADT) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 61823-07 for ; Wed, 5 Oct 2005 19:43:54 +0000 (GMT) Received: from zproxy.gmail.com (zproxy.gmail.com [64.233.162.207]) by svr1.postgresql.org (Postfix) with ESMTP id 18BCFDA257 for ; Wed, 5 Oct 2005 16:43:52 -0300 (ADT) Received: by zproxy.gmail.com with SMTP id z31so85685nzd for ; Wed, 05 Oct 2005 12:43:54 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=l3fCysXH+V87Eod3R3y+m9upm3u9hi2AAChqwy2totmH47ZSjUMJCjplXMh+yoSwkDa+pS+YJXWgCGkKqXbom1GKEgWf5KteAqMoZfJs9Pp0Sga3Tth1knD9IT25T55IID0M0+3YY59ZMuX4elI+opYNsaaN7Kl2Ktjq+3kwh7A= Received: by 10.36.60.9 with SMTP id i9mr821566nza; Wed, 05 Oct 2005 12:43:54 -0700 (PDT) Received: by 10.36.23.13 with HTTP; Wed, 5 Oct 2005 12:43:54 -0700 (PDT) Message-ID: Date: Thu, 6 Oct 2005 08:43:54 +1300 From: Andrej Ricnik-Bay Reply-To: Andrej Ricnik-Bay To: pgsql-hackers@postgresql.org Subject: Re: [PERFORM] A Better External Sort? In-Reply-To: <20051005153349.GZ2241@mathom.us> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline References: <3E37B936B592014B978C4415F90D662DE11B55@MI8NYCMAIL06.Mi8.com> <20051005153349.GZ2241@mathom.us> X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, hits=0.061 required=5 tests=[AWL=0.037, RCVD_BY_IP=0.024] X-Spam-Level: X-Archive-Number: 200510/238 X-Sequence-Number: 74097 On 10/6/05, Michael Stone wrote: > On Wed, Oct 05, 2005 at 11:24:07AM -0400, Luke Lonergan wrote: > >Nope - it would be disk wait. > > I said I/O overhead; i.e., it could be the overhead of calling the > kernel for I/O's. E.g., the following process is having I/O problems: > > time dd if=3D/dev/sdc of=3D/dev/null bs=3D1 count=3D10000000 > 10000000+0 records in > 10000000+0 records out > 10000000 bytes transferred in 8.887845 seconds (1125132 bytes/sec) > > real 0m8.889s > user 0m0.877s > sys 0m8.010s > > it's not in disk wait state (in fact the whole read was cached) but it's > only getting 1MB/s. > > Mike Stone > > ---------------------------(end of broadcast)--------------------------- > TIP 5: don't forget to increase your free space map settings > I think you only proved that dd isn't the smartest tool out there... or that using it with a blocksize of 1 byte doesn't make too much sense. [andrej@diggn:~]$ time dd if=3D/dev/sr0 of=3D/dev/null bs=3D2048 count=3D48= 83 4883+0 records in 4883+0 records out real 0m6.824s user 0m0.010s sys 0m0.060s [andrej@diggn:~]$ time dd if=3D/dev/sr0 of=3D/dev/null bs=3D1 count=3D10000= 000 10000000+0 records in 10000000+0 records out real 0m18.523s user 0m7.410s sys 0m10.310s [andrej@diggn:~]$ time dd if=3D/dev/sr0 of=3D/dev/null bs=3D8192 count=3D12= 20 1220+0 records in 1220+0 records out real 0m6.796s user 0m0.000s sys 0m0.070s That's with caching, and all. Or did I miss the point of your post completely? Interestingly, the CPU usage with the bs=3D1 goes up to 97%, it stays at a mellow 3% with the 8192 and 2048. Cheers, Andrej