X-Original-To: pgsql-performance-postgresql.org@localhost.postgresql.org Received: from localhost (av.hub.org [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id 0BCCED78D0 for ; Thu, 25 Aug 2005 20:46:57 -0300 (ADT) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 37751-09 for ; Thu, 25 Aug 2005 23:46:54 +0000 (GMT) Received: from smtpauth05.mail.atl.earthlink.net (smtpauth05.mail.atl.earthlink.net [209.86.89.65]) by svr1.postgresql.org (Postfix) with ESMTP id AB486D789C for ; Thu, 25 Aug 2005 20:46:52 -0300 (ADT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=dk20050327; d=earthlink.net; b=tGouZISx5acGbjd110zxz8F1ASDWQ2nto6247G5c+DVO9G9Q8xetSVyLymjV1k4x; h=Received:Message-Id:X-Mailer:Date:To:From:Subject:In-Reply-To:References:Mime-Version:Content-Type:Content-Transfer-Encoding:X-ELNK-Trace:X-Originating-IP; Received: from [24.41.3.216] (helo=RJP-01.earthlink.net) by smtpauth05.mail.atl.earthlink.net with asmtp (Exim 4.34) id 1E8RQu-0002ky-6U; Thu, 25 Aug 2005 19:46:56 -0400 Message-Id: <6.2.3.4.0.20050825185441.05e63ef0@pop.earthlink.net> X-Mailer: QUALCOMM Windows Eudora Version 6.2.3.4 Date: Thu, 25 Aug 2005 19:46:51 -0400 To: Chris Browne , pgsql-performance@postgresql.org From: Ron Subject: Re: Read/Write block sizes In-Reply-To: <604q9doi7d.fsf@dba2.int.libertyrms.com> References: <21700.1124863004@sss.pgh.pa.us> <1124864572.11270.16.camel@noodles> <200508251245.03473.josh@agliodbs.com> <6.2.3.4.0.20050825162308.05dbf298@pop.earthlink.net> <604q9doi7d.fsf@dba2.int.libertyrms.com> Mime-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1"; format=flowed Content-Transfer-Encoding: quoted-printable X-ELNK-Trace: acd68a6551193be5d780f4a490ca69563f9fea00a6dd62bc2469d0c2931e1f8d42fb3ee955c9969c350badd9bab72f9c350badd9bab72f9c350badd9bab72f9c X-Originating-IP: 24.41.3.216 X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, hits=0.433 required=5 tests=[AWL=0.059, DNS_FROM_RFC_ABUSE=0.374] X-Spam-Level: X-Archive-Number: 200508/426 X-Sequence-Number: 14173 At 04:49 PM 8/25/2005, Chris Browne wrote: >rjpeace@earthlink.net (Ron) writes: > > At 03:45 PM 8/25/2005, Josh Berkus wrote: > >> > Ask me sometime about my replacement for GNU sort. =C2 It uses the > >> > same sorting algorithm, but it's an order of magnitude faster due > >> > to better I/O strategy. =C2 Someday, in my infinite spare time, I > >> > hope to demonstrate that kind of improvement with a patch to pg. > >> > >>Since we desperately need some improvements in sort performance, I > >>do hope you follow up on this. > > > > I'll generalize that. IMO we desperately need any and all > > improvements in IO performance. Even more so than we need > > improvements in sorting or sorting IO performance. > >That's frankly a step backwards. Feel free to "specialise" that instead. We can agree to disagree, I'm cool with that. I'm well aware that a Systems Approach to SW=20 Architecture is not always popular in the Open=20 Source world. Nonetheless, my POV is that if we=20 want to be taken seriously and beat "the big=20 boys", we have to do everything smarter and=20 faster, as well as cheaper, than they do. You=20 are not likely to be able to do that consistently=20 without using some of the "icky" stuff one is=20 required to study as part of formal training in=20 the Comp Sci and SW Engineering fields. >A patch that improves some specific aspect of=20 >performance is a thousand times better than any=20 >sort of "desperate desire for any and >all improvements in I/O performance." minor twisting of my words: substituting "desire"=20 for "need". The need is provable. Just put "the=20 big 5" (SQL Server, Oracle, DB2, mySQL, and=20 PostgreSQL) into some realistic benches to see that. Major twisting of my words: the apparent=20 implication by you that I don't appreciate=20 improvements in the IO behavior of specific=20 things like sorting as much as I'd appreciate=20 more "general" IO performance=20 improvements. Performance optimization is best=20 done as an iterative improvement process that=20 starts with measuring where the need is greatest,=20 then improving that greatest need by the most you=20 can, then repeating the whole cycle. _Every_=20 improvement in such a process is a specific=20 improvement, even if the improvement is a=20 decision to re-architect the entire product to=20 solve the current biggest issue. Improving=20 sorting IO is cool. OTOH, if pg's biggest IO=20 problems are elsewhere, then the amount of=20 overall benefit we will get from improving=20 sorting IO is going to be minimized until we=20 improve the bigger problem(s). Amdahl's Law. >The "specialized patch" is also pointedly better=20 >in that a *confidently submitted* patch is=20 >likely to be way better than any sort of=20 >"desperate clutching at whatever may come to hand." Another distortion of my statement and POV. I=20 never suggested nor implied any sort of=20 "desperate clutching...". We have _measurable_=20 IO issues that need to be addressed in order for=20 pg to be a better competitor in the=20 marketplace. Just as we do with sorting performance. >Far too often, I see people trying to address=20 >performance problems via the "desperate=20 >clutching at whatever seems near to hand," and that >generally turns out very badly as a particular=20 >result of the whole "desperate clutching" part. > >If you can get a sort improvement submitted, that's a concrete= improvement... As I said, I'm all in favor of concrete,=20 measurable improvement. I do not think I ever=20 stated I was in favor of anything else. You evidently are mildly ranting because you've=20 seen some examples of poor SW Engineering=20 Discipline/Practice by people with perhaps=20 inadequate skills for the issues they were trying=20 to address. We all have. "90% of everything is=20 Jreck (eg of too low a quality)." OTOH, I do not think I've given you any reason to=20 think I lack such Clue, nor do I think my post was advocating such= thrashing. My post was intended to say that we need an=20 Overall Systems Approach to pg optimization=20 rather than just applying what compiler writer's=20 call "peephole optimizations" to pg. No more, no less. I apologize if I somehow misled you, Ron Peacetree