pg.ddx.io  pgsql-performance@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: PFC <lists@boutiquenumerique.com>
To: Tom Lane <tgl@sss.pgh.pa.us>
To: pgsql-hackers@postgresql.org
Cc: Miroslav Ć ulc <miroslav.sulc@startnet.cz>
Cc: pgsql-performance@postgresql.org
Subject: Re: [PERFORM] Avoiding tuple construction/deconstruction during joining
Date: Tue, 15 Mar 2005 20:53:13 +0100
Message-ID: <opsno2uzlzth1vuj@localhost> (raw)
In-Reply-To: <14577.1110821062@sss.pgh.pa.us>
References: <42345D24.8070104@startnet.cz>
	<14577.1110821062@sss.pgh.pa.us>


	On my machine (Laptop with Pentium-M 1.6 GHz and 512MB DDR333) I get the  
following timings :

	Big Joins Query will all the fields and no order by (I just put a SELECT  
* in the first table) yielding about 6k rows :
	=> 12136.338 ms

	Replacing the SELECT * from the table with many fields by just a SELECT  
of the foreign key columns :
	=> 1874.612 ms

	I felt like playing a bit so I implemented a hash join in python  
(download the file, it works on Miroslav's data) :
	All timings do not include time to fetch the data from the database.  
Fetching all the tables takes about 1.1 secs.

	* With something that looks like the current implementation (copying  
tuples around) and fetching all the fields from the big table :
	=> Fetching all the tables : 1.1 secs.
	=> Joining : 4.3 secs

	* Fetching only the integer fields
	=> Fetching all the tables : 0.4 secs.
	=> Joining : 1.7 secs

	* A smarter join which copies nothing and updates the rows as they are  
processed, adding fields :
	=> Fetching all the tables :  1.1 secs.
	=> Joining : 0.4 secs
	With the just-in-time compiler activated, it goes down to about 0.25  
seconds.

	First thing, this confirms what Tom said.
	It also means that doing this query in the application can be a lot  
faster than doing it in postgres including fetching all of the tables.  
There's a problem somewhere ! It should be the other way around ! The  
python mappings (dictionaries : { key : value } ) are optimized like crazy  
but they store column names for each row. And it's a dynamic script  
language ! Argh.

	Note : run the program like this :

python test.py |less -S

	So that the time spent scrolling your terminal does not spoil the  
measurements.

Download test program :
http://boutiquenumerique.com/pf/miroslav/test.py



view thread (6+ messages)

Message-ID: <opsno2uzlzth1vuj@localhost>
Permalink:  ../opsno2uzlzth1vuj@localhost/
Also on:    postgresql.org/message-id/opsno2uzlzth1vuj@localhost

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-performance@postgresql.org
  Cc: lists@boutiquenumerique.com, tgl@sss.pgh.pa.us, pgsql-hackers@postgresql.org, miroslav.sulc@startnet.cz
  Subject: Re: [PERFORM] Avoiding tuple construction/deconstruction during joining
  In-Reply-To: <opsno2uzlzth1vuj@localhost>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox