pg.ddx.io pgsql-admin@postgresql.org mailing list archivehelp / color / mirror / Atom feed
Schema design question as it pertains to performance 5+ messages / 2 participants [nested] [flat]
* Schema design question as it pertains to performance @ 2013-01-22 21:33 Benjamin Krajmalnik <kraj@servoyant.com> 0 siblings, 0 replies; 5+ messages in thread From: Benjamin Krajmalnik @ 2013-01-22 21:33 UTC (permalink / raw) To: pgsql-admin This question pertains to PG 9.0.4 running on FreeBSD amd64. Was not sure if it should go into the general list or the performance list, so my apologies if I opted for the wrong list. We currently have a db schema which contains many wide indices which usually contain one column which is constantly changing (such as a last update time). It was designed that way because at the time we were unaware that PG could utilize multiple indices in a query. Our database is very heavy un update queries, since the same records get updated many times per minute. As a result, that very wide index is constantly being updated, which I am sure is one of the sources of some heavy disk i/o I am seeing. Until recently, this design was not affecting performance, but as we have more devices sending data, we are seeing a significant hit on the i/o, and data is getting queued by the daemons which insert it creating a backlog of "real time" data. From a performance standpoint, is there a big hit on select performance if a query ends up utilizing more than one index, taking into account that an index has been used already to reduce the data set of potential records, and the secondary index would mostly be used in the ordering of the result set (such as a last updated time)? I also assume that if no data has changed in an index, nothing is done when the record is updated as pertains to the particular index - am I correct in this assumption? Our data is already partitioned and triggers send it to the correct partition. Thanks in advance, Benjamin -- Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-admin ^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: Schema design question as it pertains to performance @ 2013-01-22 21:56 Kevin Grittner <kgrittn@mail.com> 0 siblings, 1 reply; 5+ messages in thread From: Kevin Grittner @ 2013-01-22 21:56 UTC (permalink / raw) To: Benjamin Krajmalnik <kraj@servoyant.com>; pgsql-admin Benjamin Krajmalnik wrote: > From a performance standpoint, is there a big hit on select performance > if a query ends up utilizing more than one index, taking into account > that an index has been used already to reduce the data set of potential > records, and the secondary index would mostly be used in the ordering of > the result set (such as a last updated time)? That depends on so many variables it is hard to give a simple answer. > I also assume that if no data has changed in an index, nothing is done > when the record is updated as pertains to the particular index - am I > correct in this assumption? No. If the update doesn't affect *any* indexed column, and there is room in the page, it will do a HOT update and can skip all index updates. If any indexed column is updated, it must expire the old tuple and create a new tuple to represent the updated version of the row, and this new tuple is not likely to land in the same page as the old tuple; so it needs new entries in all the indexes. The old index entries must remain until they can no longer be the visible version of the row for any database transaction, so both versions of the row will be on the index for a while. -Kevin -- Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-admin ^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: Schema design question as it pertains to performance @ 2013-01-22 22:27 Benjamin Krajmalnik <kraj@servoyant.com> parent: Kevin Grittner <kgrittn@mail.com> 0 siblings, 0 replies; 5+ messages in thread From: Benjamin Krajmalnik @ 2013-01-22 22:27 UTC (permalink / raw) To: Kevin Grittner <kgrittn@mail.com>; pgsql-admin Thanks, Kevin. That was my intent - if no column of an index changes in an update then no changes are done on the index. That helps quite a bit - I will try moving the columns which change continuously to their own index - hopefully this will tame the overwhelming IO, since the index will now have a single column. I am seeing too many semwaits or ufs states when running top, which means we are not quite keeping up. A lot of the load has been offloaded to the read only hot standby - thanks for the pointers on the delays. I only had one query cancelled today (as opposed to many previously), so it is just a matter of fine tuning right now. -----Original Message----- From: Kevin Grittner [mailto:kgrittn@mail.com] Sent: Tuesday, January 22, 2013 2:56 PM To: Benjamin Krajmalnik; pgsql-admin@postgresql.org Subject: Re: [ADMIN] Schema design question as it pertains to performance Benjamin Krajmalnik wrote: > From a performance standpoint, is there a big hit on select > performance if a query ends up utilizing more than one index, taking > into account that an index has been used already to reduce the data > set of potential records, and the secondary index would mostly be used > in the ordering of the result set (such as a last updated time)? That depends on so many variables it is hard to give a simple answer. > I also assume that if no data has changed in an index, nothing is done > when the record is updated as pertains to the particular index - am I > correct in this assumption? No. If the update doesn't affect *any* indexed column, and there is room in the page, it will do a HOT update and can skip all index updates. If any indexed column is updated, it must expire the old tuple and create a new tuple to represent the updated version of the row, and this new tuple is not likely to land in the same page as the old tuple; so it needs new entries in all the indexes. The old index entries must remain until they can no longer be the visible version of the row for any database transaction, so both versions of the row will be on the index for a while. -Kevin -- Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-admin ^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: Schema design question as it pertains to performance @ 2013-01-23 00:23 Kevin Grittner <kgrittn@mail.com> 0 siblings, 1 reply; 5+ messages in thread From: Kevin Grittner @ 2013-01-23 00:23 UTC (permalink / raw) To: Benjamin Krajmalnik <kraj@servoyant.com>; pgsql-admin Benjamin Krajmalnik wrote: > Kevin Grittner wrote: >> Benjamin Krajmalnik wrote: >>> I also assume that if no data has changed in an index, nothing >>> is done when the record is updated as pertains to the >>> particular index - am I correct in this assumption? >> >> No. [...] If any indexed column is updated, [...] it needs new >> entries in all the indexes. > That was my intent - if no column of an index changes in an > update then no changes are done on the index. I don't think you understood -- if any column referenced by any index is updated, then all indexes must get a new entry for that row whether or not they include that column. The new version of the row will be at a new location, so new entries are needed in all indexes to point to the new location. I did *not* say that only indexes referencing the updated column need a new entry. -Kevin -- Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-admin ^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: Schema design question as it pertains to performance @ 2013-01-23 00:48 Benjamin Krajmalnik <kraj@servoyant.com> parent: Kevin Grittner <kgrittn@mail.com> 0 siblings, 0 replies; 5+ messages in thread From: Benjamin Krajmalnik @ 2013-01-23 00:48 UTC (permalink / raw) To: Kevin Grittner <kgrittn@mail.com>; pgsql-admin Thanks for the clarification. Now I understand - it is an all or nothing due to required entries for each row version on the main table. I will have to see if removing the constantly changing value from the indices will affect the performance of our selects. -----Original Message----- From: Kevin Grittner [mailto:kgrittn@mail.com] Sent: Tuesday, January 22, 2013 5:23 PM To: Benjamin Krajmalnik; pgsql-admin@postgresql.org Subject: RE: [ADMIN] Schema design question as it pertains to performance Benjamin Krajmalnik wrote: > Kevin Grittner wrote: >> Benjamin Krajmalnik wrote: >>> I also assume that if no data has changed in an index, nothing is >>> done when the record is updated as pertains to the particular index >>> - am I correct in this assumption? >> >> No. [...] If any indexed column is updated, [...] it needs new >> entries in all the indexes. > That was my intent - if no column of an index changes in an update > then no changes are done on the index. I don't think you understood -- if any column referenced by any index is updated, then all indexes must get a new entry for that row whether or not they include that column. The new version of the row will be at a new location, so new entries are needed in all indexes to point to the new location. I did *not* say that only indexes referencing the updated column need a new entry. -Kevin -- Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-admin ^ permalink raw reply [nested|flat] 5+ messages in thread
end of thread, other threads:[~2013-01-23 00:48 UTC | newest] Thread overview: 5+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2013-01-22 21:33 Schema design question as it pertains to performance Benjamin Krajmalnik <kraj@servoyant.com> 2013-01-22 21:56 Re: Schema design question as it pertains to performance Kevin Grittner <kgrittn@mail.com> 2013-01-22 22:27 ` Benjamin Krajmalnik <kraj@servoyant.com> 2013-01-23 00:23 Re: Schema design question as it pertains to performance Kevin Grittner <kgrittn@mail.com> 2013-01-23 00:48 ` Benjamin Krajmalnik <kraj@servoyant.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions for how to clone and mirror all data and code used for this inbox