X-Original-To: pgsql-performance-postgresql.org@localhost.postgresql.org Received: from localhost (av.hub.org [200.46.204.144]) by postgresql.org (Postfix) with ESMTP id 408FD9DC842 for ; Fri, 24 Mar 2006 06:14:41 -0400 (AST) Received: from postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 70890-02 for ; Fri, 24 Mar 2006 06:14:42 -0400 (AST) X-Greylist: from auto-whitelisted by SQLgrey- Received: from noel.decibel.org (noel.decibel.org [67.100.216.10]) by postgresql.org (Postfix) with ESMTP id D09769DC82F for ; Fri, 24 Mar 2006 06:14:38 -0400 (AST) Received: by noel.decibel.org (Postfix, from userid 1001) id 1E01C56427; Fri, 24 Mar 2006 04:14:41 -0600 (CST) Received: (hashcash-sendmail, from uid 1001); Fri, 24 Mar 2006 04:14:40 -0600 Date: Fri, 24 Mar 2006 04:14:40 -0600 From: "Jim C. Nasby" To: Christopher Browne Cc: pgsql-performance@postgresql.org Subject: Re: Scaling up PostgreSQL in Multiple CPU / Dual Core Message-ID: <20060324101440.GP90527@pervasive.com> References: <65f9ddfb0603222219r1742c2aevda37acd55ca845be@mail.gmail.com> <1143131901.17999.67.camel@state.g2switchworks.com> <4422D033.2030407@commandprompt.com> <1143133364.17999.77.camel@state.g2switchworks.com> <87hd5olgyd.fsf@wolfe.cbbrowne.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <87hd5olgyd.fsf@wolfe.cbbrowne.com> X-Operating-System: FreeBSD 6.0-RELEASE-p4 amd64 X-Distributed: Join the Effort! http://www.distributed.net User-Agent: Mutt/1.5.11 X-Hashcash: 1:20:060324:cbbrowne@acm.org::VqXyjH08aU3/khSi:01701 X-Hashcash: 1:20:060324:pgsql-performance@postgresql.org::I3fflTCe2p1o1EzC:00000 0000000000000000000000005eDB X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, score=0.108 required=5 tests=[AWL=0.108] X-Spam-Score: 0.108 X-Spam-Level: X-Archive-Number: 200603/482 X-Sequence-Number: 17919 On Thu, Mar 23, 2006 at 09:22:34PM -0500, Christopher Browne wrote: > Martha Stewart called it a Good Thing when smarlowe@g2switchworks.com (Scott Marlowe) wrote: > > On Thu, 2006-03-23 at 10:43, Joshua D. Drake wrote: > >> > Has someone been working on the problem of splitting a query into pieces > >> > and running it on multiple CPUs / multiple machines? Yes. Bizgress has > >> > done that. > >> > >> I believe that is limited to Bizgress MPP yes? > > > > Yep. I hope that someday it will be released to the postgresql global > > dev group for inclusion. Or at least parts of it. > > Question: Does the Bizgress/MPP use threading for this concurrency? > Or forking? > > If it does so via forking, that's more portable, and less dependent on > specific complexities of threading implementations (which amounts to > non-portability ;-)). > > Most times Jan comes to town, we spend a few minutes musing about the > "splitting queries across threads" problem, and dismiss it again; if > there's the beginning of a "split across processes," that's decidedly > neat :-). Correct me if I'm wrong, but there's no way to (reasonably) accomplish that without having some dedicated extra processes laying around that you can use to execute the queries, no? In other words, the cost of a fork() during query execution would be too prohibitive... FWIW, DB2 executes all queries in a dedicated set of processes. The process handling the connection from the client will pass a query request off to one of the executor processes. I can't remember which process actually plans the query, but I know that the executor runs it. -- Jim C. Nasby, Sr. Engineering Consultant jnasby@pervasive.com Pervasive Software http://pervasive.com work: 512-231-6117 vcard: http://jim.nasby.net/pervasive.vcf cell: 512-569-9461