Date: Sun, 23 Oct 2011 19:25:08 -0400
From: Stephen Frost <sfrost@snowman.net>
To: Stefan Keller <sfkeller@gmail.com>
Cc: pgsql-performance@postgresql.org
Subject: Re: hstore query: Any better idea than adding more
 memory?
Message-ID: <20111023232507.GV12765@tamriel.snowman.net>
References: 
 <CAFcOn293DCQd-OsoPaTu8Upo2PJw0X2XmCJoe+BWjBGSm0eqbg@mail.gmail.com>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha1;
	protocol="application/pgp-signature"; boundary="DyGujRxquesowawl"
Content-Disposition: inline
In-Reply-To: 
 <CAFcOn293DCQd-OsoPaTu8Upo2PJw0X2XmCJoe+BWjBGSm0eqbg@mail.gmail.com>
User-Agent: Mutt/1.5.20 (2009-06-14)


--DyGujRxquesowawl
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline

* Stefan Keller (sfkeller@gmail.com) wrote:
> Adding more memory (say to total of 32 GB) would only postpone the problem.

Erm, seems like you're jumping to conclusions here...

> First time the query lasts about 10 time longer (~ 1010 ms) - but I'd
> like to get better results already in the first query.

Do you mean first time after a database restart?

> => 1. When I add the "actual time" from EXPLAIN above, I get 11 + 10 +
> 10ms which is three times greater than the 11ms reported.  Why?

Because they include the times from the nodes under them.

> => 2. Why does the planner choose to sort first instead of sorting the
> (smaller)  result query at the end the?

You're reading the explain 'backwards' regarding time..  It *does* do
the sort last.  Nodes which are indented feed the nodes above them, so
the bitmap index scan and recheck feed into the sort, hence the sort is
actually done after.  Can't really work any other way anyway, PG has to
get the data before it can sort it..

> => 3. What could I do to speed up such queries (first time, i.e.
> without caching) besides simply adding more memory?

There didn't look like anything there that could really be done much
faster, at the plan level.  It's not uncommon for people to
intentionally get a box with more memory than the size of their
database, so everything is in memory.

At the end of the day, if the blocks aren't in memory then PG has to get
them from disk.  If disk is slow, the query is going to be slow.  Now,
hopefully, you're hitting this table often enough with similar queries
that important, common, parts of the table and index are already in
memory, but there's no magic PG can perform to ensure that.

If there's a lot of updates/changes to this table, you might check if
there's a lot of bloat (check_postgres works great for this..).
Eliminating excessive bloat, if there is any, could help with all
accesses to that table, of course, since it would reduce the amount of
data which would need to be.

	Thanks,

		Stephen

--DyGujRxquesowawl
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Digital signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)

iEUEARECAAYFAk6kolMACgkQrzgMPqB3kihg4QCXexrQGTz2Z/lKOfyhuHUQrAS8
5ACcDSA9Kw0wZsWgkyzJOk+9ygmMIB4=
=bLR9
-----END PGP SIGNATURE-----

--DyGujRxquesowawl--